Depending on the result of ssl_client_verify, I want to use a different upstream. So if $ssl_client_verify = SUCCESS route to yes-mtls-backend:80 K8s service, otherwise route to no-mtls-backend:80 K8s service. The following did not work:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: dynamic-upstream
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/auth-tls-secret: default/ca-cert
nginx.ingress.kubernetes.io/auth-tls-verify-client: optional
nginx.ingress.kubernetes.io/auth-tls-verify-depth: "3"
nginx.ingress.kubernetes.io/configuration-snippet: |
if ($ssl_client_verify = SUCCESS) {
set $proxy_host "default-yes-mtls-backend-80";
}
spec:
rules:
- host: xyz.com
http:
paths:
- backend:
service:
name: no-mtls-backend
port:
number: 80
path: /
pathType: Prefix
Note that auth-tls-verify-client is optional so that Nginx will not ask for a client certificate.
How to get this working? Is $proxy_host the wrong thing to set or this whole approach is flawed?
This worked:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: dynamic-upstream
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/auth-tls-secret: default/ca-cert
nginx.ingress.kubernetes.io/auth-tls-verify-client: optional
nginx.ingress.kubernetes.io/auth-tls-verify-depth: "3"
nginx.ingress.kubernetes.io/configuration-snippet: |
if ($ssl_client_verify = SUCCESS) {
set $service_name "yes-mtls-backend";
set $service_port "80";
set $proxy_upstream_name "default-yes-mtls-backend-80";
}
spec:
rules:
- host: xyz.com
http:
paths:
- backend:
service:
name: no-mtls-backend
port:
number: 80
path: /
pathType: Prefix
- host: xyz.com
http:
paths:
- backend:
service:
name: yes-mtls-backend
port:
number: 80
path: /
pathType: Prefix
In my version of Nginx, upstreams are dynamically loaded using Lua, so duplicating the rule will tell Lua to load yes-mtls-backend in the upstream block. That way, when overwriting $service_name, $service_port and $proxy_upstream_name there will be a backend to route traffic to.
I'm facing a very weird issue with Ingress Nginx. The Nginx was provisioned by RKE2 deployment as a daemonset (rke2-ingress-nginx-controller).
It works fine for any Ingress definition for which path points to / .
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: sealed-secrets-sealed-secrets-web
namespace: sealed-secrets
spec:
ingressClassName: nginx
rules:
- host: host.domain.com
http:
paths:
- backend:
service:
name: sealed-secrets-sealed-secrets-web
port:
number: 80
path: /
pathType: Prefix
But if instead of "/", the path points to something different (e.g. /sealed-secrets), I get an error 404 (not found), even if rewrite-target set.
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: sealed-secrets-sealed-secrets-web
namespace: sealed-secrets
spec:
ingressClassName: nginx
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /
rules:
- host: host.domain.com
http:
paths:
- backend:
service:
name: sealed-secrets-sealed-secrets-web
port:
number: 80
path: /sealed-secrets
pathType: Prefix
I'm using sealed-secrets to illustrate, but the same happens for whatever ingress resource defined.
Any help will be appreciated.
It seems that I've missed the need for adding a regex to rewrite paths.
annotations:
nginx.ingress.kubernetes.io/ssl-redirect: 'false'
nginx.ingress.kubernetes.io/use-regex: "true"
nginx.ingress.kubernetes.io/rewrite-target: /$2
# -- Ingress hosts
hosts:
- host: host.domain.com
paths:
- path: /sealed-secrets(/|$)(.*)
pathType: Prefix
Now the connection is reaching the app and displaying the expected page.
I have deployed an ingress in Kubernetes for graylog and when I try to access it, it returns a blank page. but it is getting into that service. I will add my code below for your reference. and also it is the same result when i add path for another service.
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: demo-ingress
namespace: logging
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /$1
spec:
ingressClassName: demoingress
rules:
- host: demo.graylog.com
http:
paths:
- path: /graylog(/|$)(.*)
pathType: Prefix
backend:
service:
name: graylog-web
port:
number: 9000
I use countor as ingress nginx class for my ingress and try to add basic http authentication but it doesn't work.I wonder why it doesn't work?
What i'm missed?
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: my-ingress
annotations:
cert-manager.io/cluster-issuer: letsencrypt-prod
ingress.kubernetes.io/force-ssl-redirect: "true"
kubernetes.io/ingress.class: contour
kubernetes.io/tls-acme: "true"
# It doesn't work
nginx.ingress.kubernetes.io/auth-type: basic
nginx.ingress.kubernetes.io/auth-secret: basic-auth
nginx.ingress.kubernetes.io/auth-realm: 'Authentication Required - foo'
spec:
tls:
- hosts:
- my_domen.com
secretName: mysecretname
rules:
- host: my_domen.com
http:
paths:
- pathType: Prefix
path: /
backend:
service:
name: some-service
port:
number:
8083
Contour is not nginx based so the nginx annotations don't have an effect. As far as I'm aware, you need to run this process: https://projectcontour.io/guides/external-authorization/
I am bit confused the way path resolves the endpoint, does it show in any logs the final endpoint it creates. I am stuck with this now. Below is the endpoint which I wanted to call:-
https://hostname/api/orders/employees. And to call this endpoint through Ingress application gateway, this is how I configured but it always return 502 bad gateway error.
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ordersapi
namespace: orders
annotations:
kubernetes.io/ingress.class: azure/application-gateway
appgw.ingress.kubernetes.io/appgw-ssl-certificate: "wildcard.apps.com"
appgw.ingress.kubernetes.io/ssl-redirect: "true"
spec:
rules:
- host: orders.apps.com
http:
paths:
- path: /api/orders/employees
backend:
serviceName: orderservice
servicePort: 80
Finally, there are two solution to this problem:-
First
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ordersapi
namespace: orders
annotations:
kubernetes.io/ingress.class: azure/application-gateway
appgw.ingress.kubernetes.io/appgw-ssl-certificate: "wildcard.apps.com"
appgw.ingress.kubernetes.io/ssl-redirect: "true"
spec:
rules:
- host: orders.apps.com
http:
paths:
- path: /api/orders/employees
backend:
serviceName: orderservice
servicePort: 80
- path: /api/product/products
backend:
serviceName: orderservice
servicePort: 80h
Second
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ordersapi
namespace: orders
annotations:
kubernetes.io/ingress.class: azure/application-gateway
appgw.ingress.kubernetes.io/backend-path-prefix: "/api/"
appgw.ingress.kubernetes.io/appgw-ssl-certificate: "wildcard.apps.com"
appgw.ingress.kubernetes.io/ssl-redirect: "true"
spec:
rules:
- host: orders.apps.com
http:
paths:
- path: /api/*
backend:
serviceName: orderservice
servicePort: 80
You seem to have enabled SSL redirect but your service is serving on a non ssl port.
this could explain the bad gateway.
Often, the Azure AppGW will return 502 Bad Gateway when there are bad Certs involved, the health check for the backend service is wrong, and other reason
You should look at this:
https://support.microsoft.com/en-ca/help/4504111/azure-application-gateway-with-bad-gateway-502-errors
and this
https://learn.microsoft.com/en-us/azure/application-gateway/application-gateway-troubleshooting-502
I got same issue. My ingress setup was like:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ingress-name
annotations:
kubernetes.io/ingress.class: azure/application-gateway
spec:
rules:
- host: <myhostname>
http:
paths:
- path: /api*
backend:
serviceName: backend-app
servicePort: 80
- path: /
backend:
serviceName: frontend-app
servicePort: 80
If I defined path like /api/todolist, the endpoint worked fine. On the other hand if I went with /api* my requests were redirected to frontend app.
The problem was that endpoint /api/todolist existed on my backend and returned status was 200, for the /api endpoint I did not setup anything so I got 404 status.
In my case I needed to add healthcheck under /api edpoint, that returned proper http status :) For me returning string "Healthy" was enough.
#djsly #Jean-Philippe Bond - Thanks for your response and pointing the URL that helped me to investigate further. Having the backend application deployed on port 80 had a reason as SSL terminates at application gateway and listener redirects the request to backend application running on port 80, which works fine.
After further investigation, I added backend path prefix in ingress file (appgw.ingress.kubernetes.io/backend-path-prefix: "/api/orders/employees") which resolved the problem for one endpoint but not for all.
To describe the problem in details, application contains some of the restful services mentioned below and their endpoints are such as -
http://hostname/api/orders/employees
http://hostname/api/Lookup/officeHierarchy
http://hostname/api/Department/codes
http://hostname/api/position/members
Now if you see, these different endpoints starts with prefix "/api/" and then the controller name and actions.
Here Expected Result is
If any of these endpoints are called (via HTTP Get), data should be returned but it fails.
Investigation done so far
I added the prefix and did some changes to it. Thus If I configure my ingress like below, it returns the result successfully for only one specific endpoint - >
curl -v http://orders.apps.com/api/orders/employees returns 200 but fails for others.
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ordersapi
namespace: orders
annotations:
kubernetes.io/ingress.class: azure/application-gateway
appgw.ingress.kubernetes.io/backend-path-prefix: "/api/orders/employees"
appgw.ingress.kubernetes.io/appgw-ssl-certificate: "wildcard.apps.com"
appgw.ingress.kubernetes.io/ssl-redirect: "true"
spec:
rules:
- host: orders.apps.com
http:
paths:
- path: /api/orders/employees
backend:
serviceName: orderservice
servicePort: 80
Thus , to make all endpoints works, I did the below changes in the above mentioned ingress file but calling
curl -v http://orders.apps.com/api/orders/employees returns 404. And the same goes with other endpoints like
curl -v http://orders.apps.com/api/department/codes returns 404.
As per my understanding, by doing the below changes - "path - /api/*" should be overwritten to the path - /api/orders/employees being called but it does not.
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ordersapi
namespace: orders
annotations:
kubernetes.io/ingress.class: azure/application-gateway
appgw.ingress.kubernetes.io/backend-path-prefix: "/api/"
appgw.ingress.kubernetes.io/appgw-ssl-certificate: "wildcard.apps.com"
appgw.ingress.kubernetes.io/ssl-redirect: "true"
spec:
rules:
- host: orders.apps.com
http:
paths:
- path: /api*
backend:
serviceName: orderservice
servicePort: 80
Your suggestions are appreciated.
Thanks
Defining multiple paths without backend-path-prefix worked, e.g.
http:
paths:
path: /api/order/employees
backend:
serviceName: orderservice
servicePort: 80
path: /api/position/members
backend:
serviceName: positionservice
servicePort: 80
This seems to be a complex way but so far I found this the only way which worked.
For me the solution was to just add "*" to paths, e.g.
rules:
- http:
paths:
- pathType: Prefix
path: /api/*
backend:
service:
name: api
port:"
number: 80
- pathType: Prefix
path: /graphql*
backend:
service:
name: api
port:
number: 80
without any extra annotations. For whatever reason it does not work without "*" at the end.