How to make websocket working in kubernetes ingress? - kubernetes-ingress

Question

I am dockering our current application and deploying on kubernetes cluster.
We have 2 services, namely, service-A and service-B. One of our services(example service-A) uses websocket. we have configured a rule in ingress to route the websocket request directly to service-A on port 8080. Also have a rule to route other requests to service-B on port 443. But ingress controller always route the websocket request to service-B instead of routing to service-A.
So, I removed the service-B rule from ingress, but still its routed as tls request and request never reaches service-A. Not sure why its rerouted as TLS instead of http request upgraded to websocket connection.
Please find my ingress configuration below:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
annotations:
field.cattle.io/publicEndpoints: '[{"addresses":[""],"port":443,"protocol":"HTTPS","serviceName":"cluster42:service-B","ingressName":"cluster42:my-ingress","hostname":"cluster42-phase-0 ","path":"/","allNodes":false},{"addresses":[""],"port":443,"protocol":"HTTPS","serviceName":"cluster42:service-A","ingressName":"cluster42:my-ingress","hostname":"cluster42-phase-0 ","path":"/ws-service","allNodes":false}]'
kubectl.kubernetes.io/last-applied-configuration: |
{"apiVersion":"extensions/v1beta1","kind":"Ingress","metadata":{"annotations":{"nginx.ingress.kubernetes.io/rewrite-target":"/","nginx.ingress.kubernetes.io/ssl-passthrough":"true"},"labels":{"app":"ingress","chart":" myapplication-chart","heritage":"Tiller","release":"installation-cluster42"},"name":"my-ingress","namespace":"cluster42"},"spec":{"rules":[{"host":"cluster42-phase-0 ","http":{"paths":[{"backend":{"serviceName":"service-B","servicePort":443},"path":"/"}]}}],"tls":[{"hosts":["cluster42-phase-0 "]}]}}
nginx.ingress.kubernetes.io/rewrite-target: /
nginx.ingress.kubernetes.io/ssl-passthrough: "true"
nginx.org/websocket-services: "service-A"
creationTimestamp: "2019-05-24T11:46:40Z"
generation: 31
labels:
app: ingress
chart: myapplication-chart
heritage: Tiller
release: installation-cluster42
name: my-ingress
namespace: cluster42
resourceVersion: "57549362"
selfLink: /apis/extensions/v1beta1/namespaces/cluster42/ingresses/my-ingress
uid: 98784b1f-7e19-11e9-b2f1-005056b0b58e
spec:
rules:
- host: cluster42-phase-0
http:
paths:
- backend:
serviceName: service-B
servicePort: 443
path: /
- backend:
serviceName: service-A
servicePort: 8080
path: /ws-service
tls:
- hosts:
- cluster42-phase-0
status:
loadBalancer:
ingress:
- {}
I expect the request to be routed to service-A instead of service-B. Can you please let me know if I am missing something in my configuration or doing anything wrong.
Thanks in Advance.

Answer 1

Check if destination port are opened and not used.
Then check if you have enough right to access by tiller to kube-system namespace.
otherwise you have to create RBAC and special service.
More infromation you can find here: tiller-rbac.
You have some mistakes in your ingress configuration file in spec section.
Take notice that currently the Ingress only supports a single TLS port, 443, and assumes TLS termination. So it's obvious that destination service will be service B with port 443. So you can delete tls section from configuration file.
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
annotations:
field.cattle.io/publicEndpoints: '[{"addresses":[""],"port":443,"protocol":"HTTPS","serviceName":"cluster42:service-B","ingressName":"cluster42:my-ingress","hostname":"cluster42-phase-0 ","path":"/","allNodes":false},{"addresses":[""],"port":443,"protocol":"HTTPS","serviceName":"cluster42:service-A","ingressName":"cluster42:my-ingress","hostname":"cluster42-phase-0 ","path":"/ws-service","allNodes":false}]'
kubectl.kubernetes.io/last-applied-configuration: |
{"apiVersion":"extensions/v1beta1","kind":"Ingress","metadata":{"annotations":{"nginx.ingress.kubernetes.io/rewrite-target":"/","nginx.ingress.kubernetes.io/ssl-passthrough":"true"},"labels":{"app":"ingress","chart":" myapplication-chart","heritage":"Tiller","release":"installation-cluster42"},"name":"my-ingress","namespace":"cluster42"},"spec":{"rules":[{"host":"cluster42-phase-0 ","http":{"paths":[{"backend":{"serviceName":"service-B","servicePort":443},"path":"/"}]}}],"tls":[{"hosts":["cluster42-phase-0 "]}]}}
nginx.ingress.kubernetes.io/rewrite-target: /
nginx.ingress.kubernetes.io/ssl-passthrough: "true"
nginx.org/websocket-services: "service-A"
creationTimestamp: "2019-05-24T11:46:40Z"
generation: 31
labels:
app: ingress
chart: myapplication-chart
heritage: Tiller
release: installation-cluster42
name: my-ingress
namespace: cluster42
resourceVersion: "57549362"
selfLink: /apis/extensions/v1beta1/namespaces/cluster42/ingresses/my-ingress
uid: 98784b1f-7e19-11e9-b2f1-005056b0b58e
spec:
rules:
- host: cluster42-phase-0
http:
paths:
- path: /
backend:
serviceName: service-B
servicePort: 443
- path: /ws-service
backend:
serviceName: service-A
servicePort: 8080
status:
loadBalancer:
ingress:
- {}