I assume more than two domains to be used for my multi-tenant code, domain1.com, mydomain2.com each will have a unique identifier and an ID would be maintained for each
I want my common app myApp hosted at myappdomain.com to serve both domains using CNAME and the service container should get a request prefix in the path based on the domain like:
domain1.com/home should translate the myApp request like myApp/111/home where 111 is the id
domain2.com/home should translate the myApp request like myApp/222/home where 222 is the id
This is working when the request is formed like domain1.com/myApp/111/home with the config below, but I want to use a generic rewrite rule to mask domain1.com/home to myApp/111/home & domain2.com/home to myApp/222/home
kind: Ingress
apiVersion: extensions/v1beta1
metadata:
name: myapp-custom-ingress
namespace: myapp-dev
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /myapp/$1
spec:
rules:
- host: myappdomain.com
http:
paths:
- path: /myapp/(.*)
pathType: ImplementationSpecific
backend:
serviceName: ui-myapp-proxy-mgmt-webapp
servicePort: 80
- host: domain1.com
http:
paths:
- path: /myapp/(.*)
pathType: ImplementationSpecific
backend:
serviceName: ui-myapp-proxy-mgmt-webapp
servicePort: 80
Note: currently myApp is deployed in html\myApp directory of apache in the service image
Related
Depending on the result of ssl_client_verify, I want to use a different upstream. So if $ssl_client_verify = SUCCESS route to yes-mtls-backend:80 K8s service, otherwise route to no-mtls-backend:80 K8s service. The following did not work:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: dynamic-upstream
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/auth-tls-secret: default/ca-cert
nginx.ingress.kubernetes.io/auth-tls-verify-client: optional
nginx.ingress.kubernetes.io/auth-tls-verify-depth: "3"
nginx.ingress.kubernetes.io/configuration-snippet: |
if ($ssl_client_verify = SUCCESS) {
set $proxy_host "default-yes-mtls-backend-80";
}
spec:
rules:
- host: xyz.com
http:
paths:
- backend:
service:
name: no-mtls-backend
port:
number: 80
path: /
pathType: Prefix
Note that auth-tls-verify-client is optional so that Nginx will not ask for a client certificate.
How to get this working? Is $proxy_host the wrong thing to set or this whole approach is flawed?
This worked:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: dynamic-upstream
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/auth-tls-secret: default/ca-cert
nginx.ingress.kubernetes.io/auth-tls-verify-client: optional
nginx.ingress.kubernetes.io/auth-tls-verify-depth: "3"
nginx.ingress.kubernetes.io/configuration-snippet: |
if ($ssl_client_verify = SUCCESS) {
set $service_name "yes-mtls-backend";
set $service_port "80";
set $proxy_upstream_name "default-yes-mtls-backend-80";
}
spec:
rules:
- host: xyz.com
http:
paths:
- backend:
service:
name: no-mtls-backend
port:
number: 80
path: /
pathType: Prefix
- host: xyz.com
http:
paths:
- backend:
service:
name: yes-mtls-backend
port:
number: 80
path: /
pathType: Prefix
In my version of Nginx, upstreams are dynamically loaded using Lua, so duplicating the rule will tell Lua to load yes-mtls-backend in the upstream block. That way, when overwriting $service_name, $service_port and $proxy_upstream_name there will be a backend to route traffic to.
I have configured the nginx-ingress controller with path-based routing. Below is my ingress.yaml file.
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: voting-app-ingress
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /$2
kubernetes.io/ingress.class: nginx
spec:
rules:
- host: voteapp.com
http:
paths:
- backend:
serviceName: vote
servicePort: vote-http
path: /vote(/|$)(.*)
- backend:
serviceName: result
servicePort: result-http
path: /result(/|$)(.*)
I can access the application successfully, but when I click anything within the application it routes to 404 page not found.
http://voteapp.com/vote/
http://voteapp.com/result/
kubectl get ing
NAME CLASS HOSTS ADDRESS PORTS AGE
voting-app-ingress <none> voteapp.com 192.168.99.100 80 40s
But the same thing works fine when I use the host-based routing.
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: voting-app-ingress
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /$2
kubernetes.io/ingress.class: nginx
spec:
rules:
- host: voteapp.com
http:
paths:
- backend:
serviceName: vote
servicePort: vote-http
- host: resultapp.com
http:
paths:
- backend:
serviceName: result
servicePort: result-http
.
kubectl get ing
NAME CLASS HOSTS ADDRESS PORTS AGE
voting-app-ingress <none> voteapp.com,resultapp.com 192.168.99.100 80 40s
Kubernetes version - v1.18.2
Any suggestions where I am possibly going wrong?
I am using alb ingress controller and the ingress yaml file is pasted below.
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
namespace: default
name: alb-ingress
annotations:
kubernetes.io/ingress.class: alb
alb.ingress.kubernetes.io/certificate-arn: arn:aws:acm:us-east-1:123456789123:certificate/xxxxxxx-yyyy-zzzz-abce-ffffffffffff
alb.ingress.kubernetes.io/scheme: internet-facing
alb.ingress.kubernetes.io/security-groups: sg-xxxxxxxxxxxxxx
alb.ingress.kubernetes.io/listen-ports: '[{"HTTP": 80}, {"HTTPS":443}]'
alb.ingress.kubernetes.io/actions.ssl-redirect: '{"Type": "redirect", "RedirectConfig": { "Protocol": "HTTPS", "Port": "443", "StatusCode": "HTTP_301"}}'
spec:
rules:
- http:
paths:
- path: /*
backend:
serviceName: ssl-redirect
servicePort: use-annotation
- host: domain.com
http:
paths:
- backend:
serviceName: greensvc
servicePort: 80
path: /green
- backend:
serviceName: redsvc
servicePort: 80
path: /red
- host: mongo.domain.com
http:
paths:
- backend:
serviceName: mongo-express-service
servicePort: 8081
corresponding rules in ALB is
ALB was deployed successfully but the logs from ingress "alb-ingress" reported two error
/* ssl-redirect:use-annotation (<error: endpoints "ssl-redirect" not found>)
Default backend: default-http-backend:80 (<error: endpoints "default-http-backend" not found>)
what I am trying to achieve is to create a single default action (no need of any other rules) on port 80 which is forcing the https traffic. A default action in https port that is forwarded to a service, Along with other host or path based routing as per the scrren shot. Currently both default action is set to return a fixed responce 404.
How can I modify current yaml file such that the ALB rules will be updated as per my interest. Also why the ingress is logging the annotation error for ssl-redirect.
Using apiVersion: networking.k8s.io/v1 worked for me.
You have to be aware of some changes needed to be done as you are using a different apiVersion as listed here https://kubernetes.io/docs/reference/using-api/deprecation-guide/#ingress-v122
My final code was like these:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
namespace: ...
name: ...
annotations:
kubernetes.io/ingress.class: alb
alb.ingress.kubernetes.io/scheme: internet-facing
alb.ingress.kubernetes.io/target-type: ip
alb.ingress.kubernetes.io/actions.response-420: >
{"type":"fixed-response","fixedResponseConfig":{"contentType":"text/plain","statusCode":"420","messageBody":"..."}}
spec:
rules:
- http:
paths:
- path: /myPath
pathType: Prefix
backend:
service:
name: response-420
port:
name: use-annotation
I've not tested this myself, so please evaluate this closely before applying directly.
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
namespace: default
name: alb-ingress
annotations:
kubernetes.io/ingress.class: alb
alb.ingress.kubernetes.io/certificate-arn: arn:aws:acm:us-east-1:123456789123:certificate/xxxxxxx-yyyy-zzzz-abce-ffffffffffff
alb.ingress.kubernetes.io/scheme: internet-facing
alb.ingress.kubernetes.io/security-groups: sg-xxxxxxxxxxxxxx
alb.ingress.kubernetes.io/listen-ports: '[{"HTTP": 80}, {"HTTPS":443}]'
alb.ingress.kubernetes.io/actions.ssl-redirect: '{"Type": "redirect", "RedirectConfig": { "Protocol": "HTTPS", "Port": "443", "StatusCode": "HTTP_301"}}'
alb.ingress.kubernetes.io/actions.response-404: >
{"type":"fixed-response","fixedResponseConfig":{"contentType":"text/plain","statusCode":"404","messageBody":"Not Found"}}
alb.ingress.kubernetes.io/actions.green-svc: >
{"type":"forward","forwardConfig":{"targetGroups":[{"serviceName":"green-k8s-service","servicePort":80,"weight":100}]}}
alb.ingress.kubernetes.io/conditions.green-svc: >
[{"field":"host-header","hostHeaderConfig":{"values":["domain.com"]}},{"field":"path-pattern","pathPatternConfig":{"values":["/green"]}}]
alb.ingress.kubernetes.io/actions.red-svc: >
{"type":"forward","forwardConfig":{"targetGroups":[{"serviceName":"red-k8s-service","servicePort":80,"weight":100}]}}
alb.ingress.kubernetes.io/conditions.red-svc: >
[{"field":"host-header","hostHeaderConfig":{"values":["domain.com"]}},{"field":"path-pattern","pathPatternConfig":{"values":["/red"]}}]
alb.ingress.kubernetes.io/actions.mongo-svc: >
{"type":"forward","forwardConfig":{"targetGroups":[{"serviceName":"mongo-k8s-service","servicePort":8081,"weight":100}]}}
alb.ingress.kubernetes.io/conditions.mongo-svc: >
[{"field":"host-header","hostHeaderConfig":{"values":["mongo.domain.com"]}}]
spec:
backend:
serviceName: response-404
servicePort: use-annotation
rules:
- http:
paths:
- backend:
serviceName: ssl-redirect
servicePort: use-annotation
- backend:
serviceName: green-svc
servicePort: use-annotation
- backend:
serviceName: red-svc
servicePort: use-annotation
- backend:
serviceName: mongo-svc
servicePort: use-annotation
The default ALB rule is specified directly under the spec object but before the rules object.
For the annotations, actions and conditions are interpreted together when they share the same name.
The Ingress Annotations doc page for AWS LoadBalancer Controller has a lot of great examples that should be able to help you work out any other tweaks or changes you might want to make. Specifically, check out Traffic Routing
I am bit confused the way path resolves the endpoint, does it show in any logs the final endpoint it creates. I am stuck with this now. Below is the endpoint which I wanted to call:-
https://hostname/api/orders/employees. And to call this endpoint through Ingress application gateway, this is how I configured but it always return 502 bad gateway error.
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ordersapi
namespace: orders
annotations:
kubernetes.io/ingress.class: azure/application-gateway
appgw.ingress.kubernetes.io/appgw-ssl-certificate: "wildcard.apps.com"
appgw.ingress.kubernetes.io/ssl-redirect: "true"
spec:
rules:
- host: orders.apps.com
http:
paths:
- path: /api/orders/employees
backend:
serviceName: orderservice
servicePort: 80
Finally, there are two solution to this problem:-
First
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ordersapi
namespace: orders
annotations:
kubernetes.io/ingress.class: azure/application-gateway
appgw.ingress.kubernetes.io/appgw-ssl-certificate: "wildcard.apps.com"
appgw.ingress.kubernetes.io/ssl-redirect: "true"
spec:
rules:
- host: orders.apps.com
http:
paths:
- path: /api/orders/employees
backend:
serviceName: orderservice
servicePort: 80
- path: /api/product/products
backend:
serviceName: orderservice
servicePort: 80h
Second
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ordersapi
namespace: orders
annotations:
kubernetes.io/ingress.class: azure/application-gateway
appgw.ingress.kubernetes.io/backend-path-prefix: "/api/"
appgw.ingress.kubernetes.io/appgw-ssl-certificate: "wildcard.apps.com"
appgw.ingress.kubernetes.io/ssl-redirect: "true"
spec:
rules:
- host: orders.apps.com
http:
paths:
- path: /api/*
backend:
serviceName: orderservice
servicePort: 80
You seem to have enabled SSL redirect but your service is serving on a non ssl port.
this could explain the bad gateway.
Often, the Azure AppGW will return 502 Bad Gateway when there are bad Certs involved, the health check for the backend service is wrong, and other reason
You should look at this:
https://support.microsoft.com/en-ca/help/4504111/azure-application-gateway-with-bad-gateway-502-errors
and this
https://learn.microsoft.com/en-us/azure/application-gateway/application-gateway-troubleshooting-502
I got same issue. My ingress setup was like:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ingress-name
annotations:
kubernetes.io/ingress.class: azure/application-gateway
spec:
rules:
- host: <myhostname>
http:
paths:
- path: /api*
backend:
serviceName: backend-app
servicePort: 80
- path: /
backend:
serviceName: frontend-app
servicePort: 80
If I defined path like /api/todolist, the endpoint worked fine. On the other hand if I went with /api* my requests were redirected to frontend app.
The problem was that endpoint /api/todolist existed on my backend and returned status was 200, for the /api endpoint I did not setup anything so I got 404 status.
In my case I needed to add healthcheck under /api edpoint, that returned proper http status :) For me returning string "Healthy" was enough.
#djsly #Jean-Philippe Bond - Thanks for your response and pointing the URL that helped me to investigate further. Having the backend application deployed on port 80 had a reason as SSL terminates at application gateway and listener redirects the request to backend application running on port 80, which works fine.
After further investigation, I added backend path prefix in ingress file (appgw.ingress.kubernetes.io/backend-path-prefix: "/api/orders/employees") which resolved the problem for one endpoint but not for all.
To describe the problem in details, application contains some of the restful services mentioned below and their endpoints are such as -
http://hostname/api/orders/employees
http://hostname/api/Lookup/officeHierarchy
http://hostname/api/Department/codes
http://hostname/api/position/members
Now if you see, these different endpoints starts with prefix "/api/" and then the controller name and actions.
Here Expected Result is
If any of these endpoints are called (via HTTP Get), data should be returned but it fails.
Investigation done so far
I added the prefix and did some changes to it. Thus If I configure my ingress like below, it returns the result successfully for only one specific endpoint - >
curl -v http://orders.apps.com/api/orders/employees returns 200 but fails for others.
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ordersapi
namespace: orders
annotations:
kubernetes.io/ingress.class: azure/application-gateway
appgw.ingress.kubernetes.io/backend-path-prefix: "/api/orders/employees"
appgw.ingress.kubernetes.io/appgw-ssl-certificate: "wildcard.apps.com"
appgw.ingress.kubernetes.io/ssl-redirect: "true"
spec:
rules:
- host: orders.apps.com
http:
paths:
- path: /api/orders/employees
backend:
serviceName: orderservice
servicePort: 80
Thus , to make all endpoints works, I did the below changes in the above mentioned ingress file but calling
curl -v http://orders.apps.com/api/orders/employees returns 404. And the same goes with other endpoints like
curl -v http://orders.apps.com/api/department/codes returns 404.
As per my understanding, by doing the below changes - "path - /api/*" should be overwritten to the path - /api/orders/employees being called but it does not.
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ordersapi
namespace: orders
annotations:
kubernetes.io/ingress.class: azure/application-gateway
appgw.ingress.kubernetes.io/backend-path-prefix: "/api/"
appgw.ingress.kubernetes.io/appgw-ssl-certificate: "wildcard.apps.com"
appgw.ingress.kubernetes.io/ssl-redirect: "true"
spec:
rules:
- host: orders.apps.com
http:
paths:
- path: /api*
backend:
serviceName: orderservice
servicePort: 80
Your suggestions are appreciated.
Thanks
Defining multiple paths without backend-path-prefix worked, e.g.
http:
paths:
path: /api/order/employees
backend:
serviceName: orderservice
servicePort: 80
path: /api/position/members
backend:
serviceName: positionservice
servicePort: 80
This seems to be a complex way but so far I found this the only way which worked.
For me the solution was to just add "*" to paths, e.g.
rules:
- http:
paths:
- pathType: Prefix
path: /api/*
backend:
service:
name: api
port:"
number: 80
- pathType: Prefix
path: /graphql*
backend:
service:
name: api
port:
number: 80
without any extra annotations. For whatever reason it does not work without "*" at the end.
I am dockering our current application and deploying on kubernetes cluster.
We have 2 services, namely, service-A and service-B. One of our services(example service-A) uses websocket. we have configured a rule in ingress to route the websocket request directly to service-A on port 8080. Also have a rule to route other requests to service-B on port 443. But ingress controller always route the websocket request to service-B instead of routing to service-A.
So, I removed the service-B rule from ingress, but still its routed as tls request and request never reaches service-A. Not sure why its rerouted as TLS instead of http request upgraded to websocket connection.
Please find my ingress configuration below:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
annotations:
field.cattle.io/publicEndpoints: '[{"addresses":[""],"port":443,"protocol":"HTTPS","serviceName":"cluster42:service-B","ingressName":"cluster42:my-ingress","hostname":"cluster42-phase-0 ","path":"/","allNodes":false},{"addresses":[""],"port":443,"protocol":"HTTPS","serviceName":"cluster42:service-A","ingressName":"cluster42:my-ingress","hostname":"cluster42-phase-0 ","path":"/ws-service","allNodes":false}]'
kubectl.kubernetes.io/last-applied-configuration: |
{"apiVersion":"extensions/v1beta1","kind":"Ingress","metadata":{"annotations":{"nginx.ingress.kubernetes.io/rewrite-target":"/","nginx.ingress.kubernetes.io/ssl-passthrough":"true"},"labels":{"app":"ingress","chart":" myapplication-chart","heritage":"Tiller","release":"installation-cluster42"},"name":"my-ingress","namespace":"cluster42"},"spec":{"rules":[{"host":"cluster42-phase-0 ","http":{"paths":[{"backend":{"serviceName":"service-B","servicePort":443},"path":"/"}]}}],"tls":[{"hosts":["cluster42-phase-0 "]}]}}
nginx.ingress.kubernetes.io/rewrite-target: /
nginx.ingress.kubernetes.io/ssl-passthrough: "true"
nginx.org/websocket-services: "service-A"
creationTimestamp: "2019-05-24T11:46:40Z"
generation: 31
labels:
app: ingress
chart: myapplication-chart
heritage: Tiller
release: installation-cluster42
name: my-ingress
namespace: cluster42
resourceVersion: "57549362"
selfLink: /apis/extensions/v1beta1/namespaces/cluster42/ingresses/my-ingress
uid: 98784b1f-7e19-11e9-b2f1-005056b0b58e
spec:
rules:
- host: cluster42-phase-0
http:
paths:
- backend:
serviceName: service-B
servicePort: 443
path: /
- backend:
serviceName: service-A
servicePort: 8080
path: /ws-service
tls:
- hosts:
- cluster42-phase-0
status:
loadBalancer:
ingress:
- {}
I expect the request to be routed to service-A instead of service-B. Can you please let me know if I am missing something in my configuration or doing anything wrong.
Thanks in Advance.
Check if destination port are opened and not used.
Then check if you have enough right to access by tiller to kube-system namespace.
otherwise you have to create RBAC and special service.
More infromation you can find here: tiller-rbac.
You have some mistakes in your ingress configuration file in spec section.
Take notice that currently the Ingress only supports a single TLS port, 443, and assumes TLS termination. So it's obvious that destination service will be service B with port 443. So you can delete tls section from configuration file.
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
annotations:
field.cattle.io/publicEndpoints: '[{"addresses":[""],"port":443,"protocol":"HTTPS","serviceName":"cluster42:service-B","ingressName":"cluster42:my-ingress","hostname":"cluster42-phase-0 ","path":"/","allNodes":false},{"addresses":[""],"port":443,"protocol":"HTTPS","serviceName":"cluster42:service-A","ingressName":"cluster42:my-ingress","hostname":"cluster42-phase-0 ","path":"/ws-service","allNodes":false}]'
kubectl.kubernetes.io/last-applied-configuration: |
{"apiVersion":"extensions/v1beta1","kind":"Ingress","metadata":{"annotations":{"nginx.ingress.kubernetes.io/rewrite-target":"/","nginx.ingress.kubernetes.io/ssl-passthrough":"true"},"labels":{"app":"ingress","chart":" myapplication-chart","heritage":"Tiller","release":"installation-cluster42"},"name":"my-ingress","namespace":"cluster42"},"spec":{"rules":[{"host":"cluster42-phase-0 ","http":{"paths":[{"backend":{"serviceName":"service-B","servicePort":443},"path":"/"}]}}],"tls":[{"hosts":["cluster42-phase-0 "]}]}}
nginx.ingress.kubernetes.io/rewrite-target: /
nginx.ingress.kubernetes.io/ssl-passthrough: "true"
nginx.org/websocket-services: "service-A"
creationTimestamp: "2019-05-24T11:46:40Z"
generation: 31
labels:
app: ingress
chart: myapplication-chart
heritage: Tiller
release: installation-cluster42
name: my-ingress
namespace: cluster42
resourceVersion: "57549362"
selfLink: /apis/extensions/v1beta1/namespaces/cluster42/ingresses/my-ingress
uid: 98784b1f-7e19-11e9-b2f1-005056b0b58e
spec:
rules:
- host: cluster42-phase-0
http:
paths:
- path: /
backend:
serviceName: service-B
servicePort: 443
- path: /ws-service
backend:
serviceName: service-A
servicePort: 8080
status:
loadBalancer:
ingress:
- {}