I am trying to play with multiple rules on one of my ingress statements:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
annotations:
ingress.kubernetes.io/balance-algorithm: roundrobin
ingress.kubernetes.io/maxconn-server: "10"
#ingress.kubernetes.io/secure-backends: "true"
ingress.kubernetes.io/secure-verify-ca-secret: "tls-secret-portal"
ingress.kubernetes.io/ssl-redirect: "true"
name: ingress-test
namespace: testing
spec:
tls:
- hosts:
- my.domain.name
secretName: tls-secret-portal
rules:
- host: my.domain.name
http:
paths:
- path: /
backend:
serviceName: nginx-service
servicePort: 8080
- path: /link
backend:
serviceName: apache-service
servicePort: 8080
Only the nginx-service at / will work. If i swap the paths then only the apache-service will work. If I give them both a path (/link1 and /link2), neither work and I get a 404 not found, the error on the one that does not work is always 404 not found. The certificate works fine and it is not a certificate issue.
Sean
Rewriting your application is the cleanest way of doing it so I'd suggest you doing that.
You can't just rewrite the response content from server and change urls/paths in flight. It's easier said than done. You could try in theory but generally this is not a good idea.
Related
I had a Kibana that was previously running behind the NGINX ingress controller using this Ingress configuration:
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: es-kibana-ing
annotations:
nginx.ingress.kubernetes.io/backend-protocol: HTTPS
nginx.ingress.kubernetes.io/rewrite-target: /$2
spec:
rules:
- host: "example.com"
http:
paths:
- path: /kibana(/|$)(.*)
pathType: Prefix
backend:
serviceName: es-kibana-svc
servicePort: 443
tls:
- hosts:
- example.com
secretName: example-tls
With this configuration you had to go to www.example.com/kibana to access the kibana.
Since then we migrated to GCP and now I'm trying to achieve the same using the GCE ingress controller. For now I figured how to serve the kibana on path "/*" :
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: es-kibana-ing
annotations:
kubernetes.io/ingress.class: gce
kubernetes.io/ingress.global-static-ip-name: kibana-static-ip
networking.gke.io/managed-certificates: managed-cert
spec:
rules:
- host: "example.com"
http:
paths:
- path: /*
pathType: ImplementationSpecific
backend:
service:
name: es-kibana-svc
port:
number: 443
Instead I would like to serve the Kibana on the /kibana (as in the previous Nginx configuration), but I can't find an equivalent to rewrite-target for the gce controller. Any idea how this can be done?
If I understand what you want to achieve, you cannot do this using GCE Ingress, you would need to enforce Nginx Ingress.
Rewrite behavior of Nginx Ingress cannot be replicated by GCE Ingress. As I mentioned in the comment section, Nginx Ingress contains much more features than GCE Ingress, for example rewrite/capture groups or service type requirement (NodePort in GCE, ClusterIP or NodePort in Nginx).
With GCE Ingress you can achieve some static path rules like in this example. Something like that:
rules:
- http:
paths:
- path: /hello
backend:
serviceName: hello-svc
servicePort: 8080
- path: /hello-v2
backend:
serviceName: hello-v2-svc
servicePort: 8080
- path: /kibana
backend:
serviceName: kibana
servicePort: 443
- path: /randomsvc
backend:
serviceName: randomsvc
servicePort: 8080
However, as I understand by your comment:
I just want to replicate the behavior that I described for Nginx Ingress, that was allowing me to access my application through '/kibana' using the rewrite-target feature.
Rewrite behavior is specific which cannot be replicated on GCE Ingress. There is a request to add a rewrite feature to GCE Ingress since 2018 but it's still open. More details about it you can find here.
You can find some differences between both Ingress in this guide.
It seems you may be using a different NGINX ingress controller and therefore annotations don't work as expected. You may find explanation of differences here.
Plus this closed GitHub issue seems to be very similar to yours so hopefully you can try using the solution mentioned there.
I am following the steps from the Rancher quick start guide and I am useing 2 VMs:
VM #1: I am running the Rancher server (in a Docker container) and a Rancher agent with 3 roles: etcd, control plane and worker
VM #2: a Rancher agent with a worker
I am trying to set up an ingress that will route to a simple Java REST API to a simple nodeJS app - each of these needs to have a path.
This is the ingress definition that we are trying to satisfy:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ingress
annotations:
kubernetes.io/ingress.class: nginx
spec:
rules:
- http:
paths:
- path: /
backend:
serviceName: front-end
servicePort: 3000
- path: /supermarket/
backend:
serviceName: backend
servicePort: 8081
This is ingress definition is working with the GKE.
When I apply it to Rancher, it tells us that we have 2 IP addresses - 1 for VM #1 and another for VM #2.
When we open the IP of VM #1, we get served the Rancher UI, but when we open the IP of VM #2, we get a connection timeout - as if there is no port open there.
So it appears that none of them is serving the ingress.
What is the correct IP that we need to use to hit the ingress?
For example, I want to be able to open http:///supermarket/ and get a response from the backend.
Do you have any other ingress objects in the same namespace? If you do, I would suggest you specify host in your ingress object as below:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ingress
annotations:
kubernetes.io/ingress.class: nginx
spec:
rules:
- host: foo.bar.com
http:
paths:
- path: /
backend:
serviceName: front-end
servicePort: 3000
- path: /supermarket
backend:
serviceName: backend
servicePort: 8081
Once you apply this manifest, you will be able to access your backend on http://foo.bar.com/supermarket and your front-end on http://foo.bar.com/
I am following this web site to develop an API with Nginx Ingress. When I use curl command it works !
curl -v -k -H "Host: myServiceA.foo.org" http:<IP_ADDRESS_INGRESS_NGINX>:80
Now I would like to use a browser like Chrome or Firefox but I don't find any way to do it knowing that http:<IP_ADDRESS_INGRESS_NGINX>:80 doesn't work without header.
Do you know how to do please ?
Regards
It's not working because you've configured the host field in ingress yaml.
Using the same yaml from Nginx docs you've posted:
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: ingress-myServiceA
annotations:
# use the shared ingress-nginx
kubernetes.io/ingress.class: "nginx"
spec:
rules:
- host: myServiceA.foo.org <== HERE
http:
paths:
- path: /
backend:
serviceName: myServiceA
servicePort: 80
The ingress will only accept the connection and forwarding the request to your service if the request contains the host myServiceA.foo.org.
You could test it editing the /etc/hosts of your machine e pointing to the nginx ingress ip:
File /etc/hosts
<INGRESS_IP> myServiceA.foo.org
Or another option is remove the field host in this way the ingress will accept requests coming from the Nginx ingress ip, like this yaml:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: example-ingress
spec:
rules:
- http:
paths:
- path: /
backend:
serviceName: echo-svc
servicePort: 80
After much googling and searching (even here), I'm not able to find a definitive answer to my question. So I hope someone here might be able to point me in the right direction.
I have a Kube Service definition that's already working for me, but right now I've simply exposed it with just a LoadBalancer. Here's my current Service yaml:
apiVersion: v1
kind: Service
metadata:
name: my-service
namespace: namespace1
labels:
app: my-service
spec:
type: LoadBalancer
selector:
app: my-service
tier: web
ports:
- name: proxy-port
port: 8080
targetPort: 8080
- name: metrics-port
port: 8082
targetPort: 8082
- name: admin-port
port: 8092
targetPort: 8092
- name: grpc-port
port: 50051
targetPort: 50051
This is obviously only TCP load-balanced. What I want to do is secure this with Mutual TLS, so that the server will only accept connections from my client with the authorized certificate.
From all I can tell in Kube land, what I need to do that is an Ingress definition. I've been researching all the docs I can find on kind:Ingress and I can't seem to find anything where it allows me to create a single Ingress with multiple ports on the same path!
Am I missing something here? Is there no way to create a K8s Ingress that simply has the same functionality as the above Service definition?
To my knowledge you cannot use custom ports (e.g 8080) for HTTPS LoadBalancer backed with Ingress Controller (e.g. NGINX HTTP(S) Proxy), as
currently the port of an Ingress is implicitly :80 for http and :443 for https, as official doc reference for IngressRule explains.
I think the workaround would be to use different host per service, like with this example of Ingress resource:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: name-virtual-host-ingress
spec:
rules:
- host: proxy.foo.com
http:
paths:
- backend:
serviceName: proxy-svc
servicePort: 8080
- host: metrics.foo.com
http:
paths:
- backend:
serviceName: metrics-svc
servicePort: 8082
- host: admin.foo.com
http:
paths:
- backend:
serviceName: admin-svc
servicePort: 8092
- host: grpc.foo.com
http:
paths:
- backend:
serviceName: grpc-svc
servicePort: 50051
I faced the same situation where we had to expose port 80,443 and 50051 on the same host. Using traefik v2+ on K3S, this is how I solved it:
Apply this file to modify traefik config; this is with K3S. If you have installed traefik directly with the chart, add a values file with the same config as below.
apiVersion: helm.cattle.io/v1
kind: HelmChartConfig
metadata:
name: traefik
namespace: kube-system
spec:
valuesContent: |-
ports:
grpc:
port: 50051
protocol: TCP
expose: true
exposedPort: 50051
After it is done, watch the traefik service be updated with the new config.
If its not working, make sure the port youve set are free and not used by another service.
When this is done, you can create IngressRoute object. Here I got one for grpc (50051) and one for web (80/443).
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
name: db
spec:
entryPoints:
- web
- websecure
routes:
- kind: Rule
match: Host(`lc1.nebula.global`)
services:
- name: db
port: 80
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
name: db-grpc
spec:
entryPoints:
- grpc
routes:
- kind: Rule
match: Host(`lc1.nebula.global`)
services:
- name: db-grpc
port: 50051
EDIT:::
if running with your own instance of k3s with the community helm chart (not the one provided by k3s). Here is the equivalent config I have:
traefik:
rbac:
enabled: true
ports:
web:
redirectTo: websecure
websecure:
tls:
enabled: true
grpc:
port: 50051
protocol: TCP
expose: true
exposedPort: 50051
podAnnotations:
prometheus.io/port: "8082"
prometheus.io/scrape: "true"
providers:
kubernetesIngress:
publishedService:
enabled: true
priorityClassName: "system-cluster-critical"
tolerations:
- key: "CriticalAddonsOnly"
operator: "Exists"
- key: "node-role.kubernetes.io/control-plane"
operator: "Exists"
effect: "NoSchedule"
- key: "node-role.kubernetes.io/master"
operator: "Exists"
effect: "NoSchedule"
additionalArguments:
- "--entrypoints.grpc.http2.maxconcurrentstreams=10000"
ingress:
enabled: false
host: traefik.local
annotations: {}
In my case, I also increase the max number of concurrent http2 streams for grpc.
I have created two different namespaces for different environment. one is devops-qa and another is devops-dev. I created two ingress in different namespaces. So while creating ingress of qa env in devops-qa namespace, the rules written inside ingress of qa is working fine. Means I am able to access the webpage of qa env. The moment I will create the ingress of dev env in devops-dev namespace, I will be able to access the webpage of dev env but wont be able to access the webpage of qa. And when I delete the dev ingress then again I will be able to access the qa env website
Below is the ingree of both dev and qa env.
Dev Ingress
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
annotations:
kubernetes.io/ingress.class: "nginx"
name: cafe-ingress-dev
namespace: devops-dev
spec:
tls:
- hosts:
- cafe-dev.example.com
secretName: default-token-drk6n
rules:
- host: cafe-dev.example.com
http:
paths:
- path: /
backend:
serviceName: miqpdev-svc
servicePort: 80
QA Ingress
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
annotations:
kubernetes.io/ingress.class: "nginx"
name: cafe-ingress-qa
namespace: devops-qa
spec:
tls:
- hosts:
- cafe-qa.example.com
secretName: default-token-jdnqf
rules:
- host: cafe-qa.example.com
http:
paths:
- path: /greentea
backend:
serviceName: greentea-svc
servicePort: 80
- path: /blackcoffee
backend:
serviceName: blackcoffee-svc
servicePort: 80
The token mentioned in the ingress file is of each namespace. And the nginx ingress controller is running in QA namespace
How can i run both the ingress and will be able to get all the websites deployed in both dev and qa env ?
I actually Solved my problem. I did everything correct. But only thing I did not do is to map the hostname with the same ip in Route53. And instead of accessing the website with hostname, I was accessing it from IP. Now after accessing the website from hostname, I was able to access it :)
Seems like you posted here and got your answer. The solution is to deploy a different Ingress for each namespace. However, deploying 2 Ingresses complicates matters because one instance has to run on a non-standard port (eg. 8080, 8443).
I think this is better solved using DNS. Create the CNAME records cafe-qa.example.com and cafe-dev.example.com both pointing to cafe.example.com. Update each Ingress manifest accordingly. Using DNS is somewhat the standard way to separate the Dev/QA/Prod environments.
Had the same issue, found a way to resolve it:
you just need to add the "--watch-namespace" argument to the ingress controller that sits under the ingress service that you've linked to your ingress resource. Then it will be bound only to the services within the same namespace as the ingress service and its pods belong to.
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
namespace: my-namespace
name: nginx-ingress-controller
spec:
replicas: 1
selector:
matchLabels:
name: nginx-ingress-lb
template:
metadata:
labels:
name: nginx-ingress-lb
spec:
serviceAccountName: ingress-account
containers:
- args:
- /nginx-ingress-controller
- "--default-backend-service=$(POD_NAMESPACE)/default-http-backend"
- "--default-ssl-certificate=$(POD_NAMESPACE)/secret-tls"
- "--watch-namespace=$(POD_NAMESPACE)"
env:
- name: POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
name: nginx-ingress-controller
image: "quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.24.1"
livenessProbe:
httpGet:
path: /healthz
port: 10254
scheme: HTTP
ports:
- containerPort: 80
name: http
protocol: TCP
- containerPort: 443
name: https
protocol: TCP
---
apiVersion: v1
kind: Service
metadata:
namespace: my-namespace
name: nginx-ingress
spec:
type: LoadBalancer
ports:
- name: https
port: 443
targetPort: https
selector:
name: nginx-ingress-lb
You can create nginx ingress cotroller in kube-system namespace instead of creating it in QA namespace.