Route request based on ssl_client_verify in Nginx ingress controller - kubernetes-ingress

Question

Depending on the result of ssl_client_verify, I want to use a different upstream. So if $ssl_client_verify = SUCCESS route to yes-mtls-backend:80 K8s service, otherwise route to no-mtls-backend:80 K8s service. The following did not work:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: dynamic-upstream
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/auth-tls-secret: default/ca-cert
nginx.ingress.kubernetes.io/auth-tls-verify-client: optional
nginx.ingress.kubernetes.io/auth-tls-verify-depth: "3"
nginx.ingress.kubernetes.io/configuration-snippet: |
if ($ssl_client_verify = SUCCESS) {
set $proxy_host "default-yes-mtls-backend-80";
}
spec:
rules:
- host: xyz.com
http:
paths:
- backend:
service:
name: no-mtls-backend
port:
number: 80
path: /
pathType: Prefix
Note that auth-tls-verify-client is optional so that Nginx will not ask for a client certificate.
How to get this working? Is $proxy_host the wrong thing to set or this whole approach is flawed?

Answer 1

This worked:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: dynamic-upstream
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/auth-tls-secret: default/ca-cert
nginx.ingress.kubernetes.io/auth-tls-verify-client: optional
nginx.ingress.kubernetes.io/auth-tls-verify-depth: "3"
nginx.ingress.kubernetes.io/configuration-snippet: |
if ($ssl_client_verify = SUCCESS) {
set $service_name "yes-mtls-backend";
set $service_port "80";
set $proxy_upstream_name "default-yes-mtls-backend-80";
}
spec:
rules:
- host: xyz.com
http:
paths:
- backend:
service:
name: no-mtls-backend
port:
number: 80
path: /
pathType: Prefix
- host: xyz.com
http:
paths:
- backend:
service:
name: yes-mtls-backend
port:
number: 80
path: /
pathType: Prefix
In my version of Nginx, upstreams are dynamically loaded using Lua, so duplicating the rule will tell Lua to load yes-mtls-backend in the upstream block. That way, when overwriting $service_name, $service_port and $proxy_upstream_name there will be a backend to route traffic to.