How to make websocket working in kubernetes ingress? - kubernetes-ingress

I am dockering our current application and deploying on kubernetes cluster.
We have 2 services, namely, service-A and service-B. One of our services(example service-A) uses websocket. we have configured a rule in ingress to route the websocket request directly to service-A on port 8080. Also have a rule to route other requests to service-B on port 443. But ingress controller always route the websocket request to service-B instead of routing to service-A.
So, I removed the service-B rule from ingress, but still its routed as tls request and request never reaches service-A. Not sure why its rerouted as TLS instead of http request upgraded to websocket connection.
Please find my ingress configuration below:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
annotations:
field.cattle.io/publicEndpoints: '[{"addresses":[""],"port":443,"protocol":"HTTPS","serviceName":"cluster42:service-B","ingressName":"cluster42:my-ingress","hostname":"cluster42-phase-0 ","path":"/","allNodes":false},{"addresses":[""],"port":443,"protocol":"HTTPS","serviceName":"cluster42:service-A","ingressName":"cluster42:my-ingress","hostname":"cluster42-phase-0 ","path":"/ws-service","allNodes":false}]'
kubectl.kubernetes.io/last-applied-configuration: |
{"apiVersion":"extensions/v1beta1","kind":"Ingress","metadata":{"annotations":{"nginx.ingress.kubernetes.io/rewrite-target":"/","nginx.ingress.kubernetes.io/ssl-passthrough":"true"},"labels":{"app":"ingress","chart":" myapplication-chart","heritage":"Tiller","release":"installation-cluster42"},"name":"my-ingress","namespace":"cluster42"},"spec":{"rules":[{"host":"cluster42-phase-0 ","http":{"paths":[{"backend":{"serviceName":"service-B","servicePort":443},"path":"/"}]}}],"tls":[{"hosts":["cluster42-phase-0 "]}]}}
nginx.ingress.kubernetes.io/rewrite-target: /
nginx.ingress.kubernetes.io/ssl-passthrough: "true"
nginx.org/websocket-services: "service-A"
creationTimestamp: "2019-05-24T11:46:40Z"
generation: 31
labels:
app: ingress
chart: myapplication-chart
heritage: Tiller
release: installation-cluster42
name: my-ingress
namespace: cluster42
resourceVersion: "57549362"
selfLink: /apis/extensions/v1beta1/namespaces/cluster42/ingresses/my-ingress
uid: 98784b1f-7e19-11e9-b2f1-005056b0b58e
spec:
rules:
- host: cluster42-phase-0
http:
paths:
- backend:
serviceName: service-B
servicePort: 443
path: /
- backend:
serviceName: service-A
servicePort: 8080
path: /ws-service
tls:
- hosts:
- cluster42-phase-0
status:
loadBalancer:
ingress:
- {}
I expect the request to be routed to service-A instead of service-B. Can you please let me know if I am missing something in my configuration or doing anything wrong.
Thanks in Advance.

Check if destination port are opened and not used.
Then check if you have enough right to access by tiller to kube-system namespace.
otherwise you have to create RBAC and special service.
More infromation you can find here: tiller-rbac.
You have some mistakes in your ingress configuration file in spec section.
Take notice that currently the Ingress only supports a single TLS port, 443, and assumes TLS termination. So it's obvious that destination service will be service B with port 443. So you can delete tls section from configuration file.
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
annotations:
field.cattle.io/publicEndpoints: '[{"addresses":[""],"port":443,"protocol":"HTTPS","serviceName":"cluster42:service-B","ingressName":"cluster42:my-ingress","hostname":"cluster42-phase-0 ","path":"/","allNodes":false},{"addresses":[""],"port":443,"protocol":"HTTPS","serviceName":"cluster42:service-A","ingressName":"cluster42:my-ingress","hostname":"cluster42-phase-0 ","path":"/ws-service","allNodes":false}]'
kubectl.kubernetes.io/last-applied-configuration: |
{"apiVersion":"extensions/v1beta1","kind":"Ingress","metadata":{"annotations":{"nginx.ingress.kubernetes.io/rewrite-target":"/","nginx.ingress.kubernetes.io/ssl-passthrough":"true"},"labels":{"app":"ingress","chart":" myapplication-chart","heritage":"Tiller","release":"installation-cluster42"},"name":"my-ingress","namespace":"cluster42"},"spec":{"rules":[{"host":"cluster42-phase-0 ","http":{"paths":[{"backend":{"serviceName":"service-B","servicePort":443},"path":"/"}]}}],"tls":[{"hosts":["cluster42-phase-0 "]}]}}
nginx.ingress.kubernetes.io/rewrite-target: /
nginx.ingress.kubernetes.io/ssl-passthrough: "true"
nginx.org/websocket-services: "service-A"
creationTimestamp: "2019-05-24T11:46:40Z"
generation: 31
labels:
app: ingress
chart: myapplication-chart
heritage: Tiller
release: installation-cluster42
name: my-ingress
namespace: cluster42
resourceVersion: "57549362"
selfLink: /apis/extensions/v1beta1/namespaces/cluster42/ingresses/my-ingress
uid: 98784b1f-7e19-11e9-b2f1-005056b0b58e
spec:
rules:
- host: cluster42-phase-0
http:
paths:
- path: /
backend:
serviceName: service-B
servicePort: 443
- path: /ws-service
backend:
serviceName: service-A
servicePort: 8080
status:
loadBalancer:
ingress:
- {}

Related

ingress url returns a blank screen

I have deployed an ingress in Kubernetes for graylog and when I try to access it, it returns a blank page. but it is getting into that service. I will add my code below for your reference. and also it is the same result when i add path for another service.
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: demo-ingress
namespace: logging
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /$1
spec:
ingressClassName: demoingress
rules:
- host: demo.graylog.com
http:
paths:
- path: /graylog(/|$)(.*)
pathType: Prefix
backend:
service:
name: graylog-web
port:
number: 9000

ingress contour add basic http authentication

I use countor as ingress nginx class for my ingress and try to add basic http authentication but it doesn't work.I wonder why it doesn't work?
What i'm missed?
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: my-ingress
annotations:
cert-manager.io/cluster-issuer: letsencrypt-prod
ingress.kubernetes.io/force-ssl-redirect: "true"
kubernetes.io/ingress.class: contour
kubernetes.io/tls-acme: "true"
# It doesn't work
nginx.ingress.kubernetes.io/auth-type: basic
nginx.ingress.kubernetes.io/auth-secret: basic-auth
nginx.ingress.kubernetes.io/auth-realm: 'Authentication Required - foo'
spec:
tls:
- hosts:
- my_domen.com
secretName: mysecretname
rules:
- host: my_domen.com
http:
paths:
- pathType: Prefix
path: /
backend:
service:
name: some-service
port:
number:
8083
Contour is not nginx based so the nginx annotations don't have an effect. As far as I'm aware, you need to run this process: https://projectcontour.io/guides/external-authorization/

How to configure ingress controller with multiple paths for the same service?

I have a separate ingress-internal (manifests) for the backend and the frontend.
My backend service has several endpoints: one with GraphqQL and two Rest.
After deploying the project, I find that when I request the Rest endpoint (POST request); I have the error code 404.
How can I configure properly the backend ingress manifest?
I tired too many annotations like:
nginx.ingress.kubernetes.io/use-regex: "true"
# nginx.ingress.kubernetes.io/app-root: /
# nginx.ingress.kubernetes.io/default-backend: mcs-thirdparty-backend
nginx.ingress.kubernetes.io/rewrite-target: /$2
# nginx.ingress.kubernetes.io/rewrite-path: /response
# nginx.ingress.kubernetes.io/preserve-trailing-slash: "true"
This is my current backend's ingress:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: mcs-thirdparty-back-ingress
namespace: namespace
annotations:
nginx.ingress.kubernetes.io/enable-cors: "true"
nginx.ingress.kubernetes.io/cors-allow-methods: "PUT, GET, POST, OPTIONS"
nginx.ingress.kubernetes.io/use-regex: "true"
nginx.ingress.kubernetes.io/rewrite-target: /$2
spec:
ingressClassName: nginx-internal
rules:
- host: backend.exemple.com
http:
paths:
- path: '/(/|$)(.*)'
backend:
service:
name: mcs-thirdparty-backend
port:
number: 8080
pathType: Prefix
This the backend ingress that I arrived to work successfullywith:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: mcs-thirdparty-ingress
namespace: namespace
spec:
ingressClassName: nginx-internal
rules:
- host: bilels.exemple.com
http:
paths:
- path: /
backend:
service:
name: mcs-thirdparty-frontend
port:
number: 80
pathType: Prefix

Unable to access path based routing getting 404 error

I have configured the nginx-ingress controller with path-based routing. Below is my ingress.yaml file.
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: voting-app-ingress
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /$2
kubernetes.io/ingress.class: nginx
spec:
rules:
- host: voteapp.com
http:
paths:
- backend:
serviceName: vote
servicePort: vote-http
path: /vote(/|$)(.*)
- backend:
serviceName: result
servicePort: result-http
path: /result(/|$)(.*)
I can access the application successfully, but when I click anything within the application it routes to 404 page not found.
http://voteapp.com/vote/
http://voteapp.com/result/
kubectl get ing
NAME CLASS HOSTS ADDRESS PORTS AGE
voting-app-ingress <none> voteapp.com 192.168.99.100 80 40s
But the same thing works fine when I use the host-based routing.
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: voting-app-ingress
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /$2
kubernetes.io/ingress.class: nginx
spec:
rules:
- host: voteapp.com
http:
paths:
- backend:
serviceName: vote
servicePort: vote-http
- host: resultapp.com
http:
paths:
- backend:
serviceName: result
servicePort: result-http
.
kubectl get ing
NAME CLASS HOSTS ADDRESS PORTS AGE
voting-app-ingress <none> voteapp.com,resultapp.com 192.168.99.100 80 40s
Kubernetes version - v1.18.2
Any suggestions where I am possibly going wrong?

Azure AKS | Application gateway | Ingress | Backend Prefix

I am bit confused the way path resolves the endpoint, does it show in any logs the final endpoint it creates. I am stuck with this now. Below is the endpoint which I wanted to call:-
https://hostname/api/orders/employees. And to call this endpoint through Ingress application gateway, this is how I configured but it always return 502 bad gateway error.
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ordersapi
namespace: orders
annotations:
kubernetes.io/ingress.class: azure/application-gateway
appgw.ingress.kubernetes.io/appgw-ssl-certificate: "wildcard.apps.com"
appgw.ingress.kubernetes.io/ssl-redirect: "true"
spec:
rules:
- host: orders.apps.com
http:
paths:
- path: /api/orders/employees
backend:
serviceName: orderservice
servicePort: 80
Finally, there are two solution to this problem:-
First
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ordersapi
namespace: orders
annotations:
kubernetes.io/ingress.class: azure/application-gateway
appgw.ingress.kubernetes.io/appgw-ssl-certificate: "wildcard.apps.com"
appgw.ingress.kubernetes.io/ssl-redirect: "true"
spec:
rules:
- host: orders.apps.com
http:
paths:
- path: /api/orders/employees
backend:
serviceName: orderservice
servicePort: 80
- path: /api/product/products
backend:
serviceName: orderservice
servicePort: 80h
Second
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ordersapi
namespace: orders
annotations:
kubernetes.io/ingress.class: azure/application-gateway
appgw.ingress.kubernetes.io/backend-path-prefix: "/api/"
appgw.ingress.kubernetes.io/appgw-ssl-certificate: "wildcard.apps.com"
appgw.ingress.kubernetes.io/ssl-redirect: "true"
spec:
rules:
- host: orders.apps.com
http:
paths:
- path: /api/*
backend:
serviceName: orderservice
servicePort: 80
You seem to have enabled SSL redirect but your service is serving on a non ssl port.
this could explain the bad gateway.
Often, the Azure AppGW will return 502 Bad Gateway when there are bad Certs involved, the health check for the backend service is wrong, and other reason
You should look at this:
https://support.microsoft.com/en-ca/help/4504111/azure-application-gateway-with-bad-gateway-502-errors
and this
https://learn.microsoft.com/en-us/azure/application-gateway/application-gateway-troubleshooting-502
I got same issue. My ingress setup was like:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ingress-name
annotations:
kubernetes.io/ingress.class: azure/application-gateway
spec:
rules:
- host: <myhostname>
http:
paths:
- path: /api*
backend:
serviceName: backend-app
servicePort: 80
- path: /
backend:
serviceName: frontend-app
servicePort: 80
If I defined path like /api/todolist, the endpoint worked fine. On the other hand if I went with /api* my requests were redirected to frontend app.
The problem was that endpoint /api/todolist existed on my backend and returned status was 200, for the /api endpoint I did not setup anything so I got 404 status.
In my case I needed to add healthcheck under /api edpoint, that returned proper http status :) For me returning string "Healthy" was enough.
#djsly #Jean-Philippe Bond - Thanks for your response and pointing the URL that helped me to investigate further. Having the backend application deployed on port 80 had a reason as SSL terminates at application gateway and listener redirects the request to backend application running on port 80, which works fine.
After further investigation, I added backend path prefix in ingress file (appgw.ingress.kubernetes.io/backend-path-prefix: "/api/orders/employees") which resolved the problem for one endpoint but not for all.
To describe the problem in details, application contains some of the restful services mentioned below and their endpoints are such as -
http://hostname/api/orders/employees
http://hostname/api/Lookup/officeHierarchy
http://hostname/api/Department/codes
http://hostname/api/position/members
Now if you see, these different endpoints starts with prefix "/api/" and then the controller name and actions.
Here Expected Result is
If any of these endpoints are called (via HTTP Get), data should be returned but it fails.
Investigation done so far
I added the prefix and did some changes to it. Thus If I configure my ingress like below, it returns the result successfully for only one specific endpoint - >
curl -v http://orders.apps.com/api/orders/employees returns 200 but fails for others.
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ordersapi
namespace: orders
annotations:
kubernetes.io/ingress.class: azure/application-gateway
appgw.ingress.kubernetes.io/backend-path-prefix: "/api/orders/employees"
appgw.ingress.kubernetes.io/appgw-ssl-certificate: "wildcard.apps.com"
appgw.ingress.kubernetes.io/ssl-redirect: "true"
spec:
rules:
- host: orders.apps.com
http:
paths:
- path: /api/orders/employees
backend:
serviceName: orderservice
servicePort: 80
Thus , to make all endpoints works, I did the below changes in the above mentioned ingress file but calling
curl -v http://orders.apps.com/api/orders/employees returns 404. And the same goes with other endpoints like
curl -v http://orders.apps.com/api/department/codes returns 404.
As per my understanding, by doing the below changes - "path - /api/*" should be overwritten to the path - /api/orders/employees being called but it does not.
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ordersapi
namespace: orders
annotations:
kubernetes.io/ingress.class: azure/application-gateway
appgw.ingress.kubernetes.io/backend-path-prefix: "/api/"
appgw.ingress.kubernetes.io/appgw-ssl-certificate: "wildcard.apps.com"
appgw.ingress.kubernetes.io/ssl-redirect: "true"
spec:
rules:
- host: orders.apps.com
http:
paths:
- path: /api*
backend:
serviceName: orderservice
servicePort: 80
Your suggestions are appreciated.
Thanks
Defining multiple paths without backend-path-prefix worked, e.g.
http:
paths:
path: /api/order/employees
backend:
serviceName: orderservice
servicePort: 80
path: /api/position/members
backend:
serviceName: positionservice
servicePort: 80
This seems to be a complex way but so far I found this the only way which worked.
For me the solution was to just add "*" to paths, e.g.
rules:
- http:
paths:
- pathType: Prefix
path: /api/*
backend:
service:
name: api
port:"
number: 80
- pathType: Prefix
path: /graphql*
backend:
service:
name: api
port:
number: 80
without any extra annotations. For whatever reason it does not work without "*" at the end.