Kubernetes ingress same path multiple ports - kubernetes-ingress

Question

After much googling and searching (even here), I'm not able to find a definitive answer to my question. So I hope someone here might be able to point me in the right direction.
I have a Kube Service definition that's already working for me, but right now I've simply exposed it with just a LoadBalancer. Here's my current Service yaml:
apiVersion: v1
kind: Service
metadata:
name: my-service
namespace: namespace1
labels:
app: my-service
spec:
type: LoadBalancer
selector:
app: my-service
tier: web
ports:
- name: proxy-port
port: 8080
targetPort: 8080
- name: metrics-port
port: 8082
targetPort: 8082
- name: admin-port
port: 8092
targetPort: 8092
- name: grpc-port
port: 50051
targetPort: 50051
This is obviously only TCP load-balanced. What I want to do is secure this with Mutual TLS, so that the server will only accept connections from my client with the authorized certificate.
From all I can tell in Kube land, what I need to do that is an Ingress definition. I've been researching all the docs I can find on kind:Ingress and I can't seem to find anything where it allows me to create a single Ingress with multiple ports on the same path!
Am I missing something here? Is there no way to create a K8s Ingress that simply has the same functionality as the above Service definition?

Answer 1

To my knowledge you cannot use custom ports (e.g 8080) for HTTPS LoadBalancer backed with Ingress Controller (e.g. NGINX HTTP(S) Proxy), as
currently the port of an Ingress is implicitly :80 for http and :443 for https, as official doc reference for IngressRule explains.
I think the workaround would be to use different host per service, like with this example of Ingress resource:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: name-virtual-host-ingress
spec:
rules:
- host: proxy.foo.com
http:
paths:
- backend:
serviceName: proxy-svc
servicePort: 8080
- host: metrics.foo.com
http:
paths:
- backend:
serviceName: metrics-svc
servicePort: 8082
- host: admin.foo.com
http:
paths:
- backend:
serviceName: admin-svc
servicePort: 8092
- host: grpc.foo.com
http:
paths:
- backend:
serviceName: grpc-svc
servicePort: 50051

Answer 2

I faced the same situation where we had to expose port 80,443 and 50051 on the same host. Using traefik v2+ on K3S, this is how I solved it:
Apply this file to modify traefik config; this is with K3S. If you have installed traefik directly with the chart, add a values file with the same config as below.
apiVersion: helm.cattle.io/v1
kind: HelmChartConfig
metadata:
name: traefik
namespace: kube-system
spec:
valuesContent: |-
ports:
grpc:
port: 50051
protocol: TCP
expose: true
exposedPort: 50051
After it is done, watch the traefik service be updated with the new config.
If its not working, make sure the port youve set are free and not used by another service.
When this is done, you can create IngressRoute object. Here I got one for grpc (50051) and one for web (80/443).
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
name: db
spec:
entryPoints:
- web
- websecure
routes:
- kind: Rule
match: Host(`lc1.nebula.global`)
services:
- name: db
port: 80
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
name: db-grpc
spec:
entryPoints:
- grpc
routes:
- kind: Rule
match: Host(`lc1.nebula.global`)
services:
- name: db-grpc
port: 50051
EDIT:::
if running with your own instance of k3s with the community helm chart (not the one provided by k3s). Here is the equivalent config I have:
traefik:
rbac:
enabled: true
ports:
web:
redirectTo: websecure
websecure:
tls:
enabled: true
grpc:
port: 50051
protocol: TCP
expose: true
exposedPort: 50051
podAnnotations:
prometheus.io/port: "8082"
prometheus.io/scrape: "true"
providers:
kubernetesIngress:
publishedService:
enabled: true
priorityClassName: "system-cluster-critical"
tolerations:
- key: "CriticalAddonsOnly"
operator: "Exists"
- key: "node-role.kubernetes.io/control-plane"
operator: "Exists"
effect: "NoSchedule"
- key: "node-role.kubernetes.io/master"
operator: "Exists"
effect: "NoSchedule"
additionalArguments:
- "--entrypoints.grpc.http2.maxconcurrentstreams=10000"
ingress:
enabled: false
host: traefik.local
annotations: {}
In my case, I also increase the max number of concurrent http2 streams for grpc.