I am new to Kubernetes. I have create simple cluster with 1 master and 1 worker nodes(both running in 2 different VMs). Additionally there is HA proxy setup in a separate VM.
Client Version: v1.24.0
Kustomize Version: v4.5.4
Server Version: v1.26.1
I have setup NGINX ingress controller using manifests(https://docs.nginx.com/nginx-ingress-controller/installation/installation-with-manifests/).
When I try to apply ingress resource with rules I am getting the error:
Configuration for default/i1 was added or updated ; but was not applied: error reloading NGINX for default/i1: nginx reload failed: command /usr/sbin/nginx -s reload -e stderr stdout: "" stderr: "2023/02/06 12:49:28 [emerg] 30#30: zone \"default-i1-sim.daniyar.uk-first-web-app-service-80\" is too small in /etc/nginx/conf.d/default-i1.conf:4\n" finished with error: exit status 1
My ingress resource:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: i1
spec:
rules:
- host: sim.daniyar.uk
http:
paths:
- pathType: Prefix
path: /
backend:
service:
name: first-web-app-service
port:
number: 80
IngressClass yaml:
apiVersion: networking.k8s.io/v1
kind: IngressClass
metadata:
name: nginx
annotations:
ingressclass.kubernetes.io/is-default-class: "true"
spec:
controller: nginx.org/ingress-controller
Let me know if you need more info
Thanks
Found a solution.
In my case I had to disable zones in NGINX config by using annotation:
nginx.org/upstream-zone-size: "0"
in my ingress resource file.
Related
I am trying to expose my application inside the AKS cluster using ingress:
It creates a service and an ingress but somehow does not assign an address to the ingress. What could be a possible reason for this?
apiVersion: apps/v1
kind: Deployment
metadata:
name: dockerdemo
spec:
replicas: 1
selector:
matchLabels:
app: dockerdemo
template:
metadata:
labels:
app: dockerdemo
spec:
nodeSelector:
"kubernetes.io/os": linux
containers:
- name: dockerdemo
image: devsecopsacademy/dockerapp:v3
env:
- name: ALLOW_EMPTY_PASSWORD
value: "yes"
resources:
requests:
cpu: 100m
memory: 128Mi
limits:
cpu: 250m
memory: 256Mi
ports:
- containerPort: 80
name: redis
apiVersion: v1
kind: Service
metadata:
name: dockerdemo-service
spec:
type: ClusterIP
ports:
port: 80
selector:
app: dockerdemo
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: my-ingress15
annotations:
kubernetes.io/ingress.class: addon-http-application-rounting
spec:
rules:
host: curefirsttestapp.cluster15-dns-c42b65ee.hcp.westeurope.azmk8s.io
http:
paths:
path: /
pathType: Prefix
backend:
service:
name: dockerdemo-service
port:
number: 80
Well, first make sure your application is up and functionning inside your K8s Cluster using a port-forword to your localhost
kubectl -n $NAMESPACE port-forward svc/$SERVICE :$PORT
if app is reachable and your call are getting back 200 Status, you can now move to the ingress part:
Make sure ingress controller is well installed under your services
kubectl -n $NAMESPACE get svc
Add a DNS record in your DNS zone which maps your domain.com to ingress controller $EXTERNAL_IP
Take a look at the ingress you created for your $SERVICE
kubectl -n $NAMESPACE get ingress
At this stage, if you application is well running and also the the ingress is well set, the app should be reachable trough domain.com, otherwise we'll need further debugging.
Make sure you have an ingress controller deployed. This is a load balancer service which can have either a public or private ip depending on your situation.
Make sure you have an ingress definition which has a rule to point to your service. This is the metadata which will tell your ingress controller how to route requests to its ip address. These routing rules can contain how to handle paths like strip, exact, etc....
I am trying to create a basic path based routing ingress controller with an AKS managed Load Balancer. Trouble is figuring out how to route from the Load Balancer to the Ingress controller.
Here is my ingress controller yml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: ingress-cpr
annotations:
kubernetes.io/ingress.class: nginx
spec:
rules:
- http:
paths:
- path: /green/
pathType: Prefix
backend:
service:
name: nginx-green-clusterip-service
port:
number: 80
- path: /red/
pathType: Prefix
backend:
service:
name: nginx-red-clusterip-service
port:
number: 80
As you can see, the ingress controller is responsible to send to the appropriate app based on the incoming path.
But how do I get this connected to a managed Load Balancer?
apiVersion: v1
kind: Service
metadata:
name: loadbal-service
spec:
type: LoadBalancer
ports:
- port: 80
selector:
app: ingress-cpr
I know this line isnt correct "-app: ingress-cpr". But what do I have to do so that the LoadBalancer forwards to the ingress controller?
Thanks in advance,
Jake.
In the service manifest the app's selector should be pointing to the backend service name of the ingress. In this particular case instead of ingress-cpr in the service manifest should be either of the two backends (nginx-green-clusterip-service or nginx-red-clusterip-service). Any traffic via external IP of the managed LB on port 80 should be routed to one of the backend defined in the ingress then.
There is also Microsoft example about creation of basic ingress controller in AKS.
I created a new cluster, created an application gateway and then installed AGIC per the tutorial. I then configured the ingress controller with the following config:
# This file contains the essential configs for the ingress controller helm chart
# Verbosity level of the App Gateway Ingress Controller
verbosityLevel: 3
################################################################################
# Specify which application gateway the ingress controller will manage
#
appgw:
subscriptionId: <<subscriptionid>>
resourceGroup: experimental-cluster-rg
name: experimental-cluster-ag
usePrivateIP: false
# Setting appgw.shared to "true" will create an AzureIngressProhibitedTarget CRD.
# This prohibits AGIC from applying config for any host/path.
# Use "kubectl get AzureIngressProhibitedTargets" to view and change this.
shared: false
################################################################################
# Specify which kubernetes namespace the ingress controller will watch
# Default value is "default"
# Leaving this variable out or setting it to blank or empty string would
# result in Ingress Controller observing all acessible namespaces.
#
# kubernetes:
# watchNamespace: <namespace>
################################################################################
# Specify the authentication with Azure Resource Manager
#
# Two authentication methods are available:
# - Option 1: AAD-Pod-Identity (https://github.com/Azure/aad-pod-identity)
# armAuth:
# type: aadPodIdentity
# identityResourceID: <identityResourceId>
## identityClientID: <identityClientId>
## Alternatively you can use Service Principal credentials
armAuth:
type: servicePrincipal
secretJSON: <<hash>>
################################################################################
# Specify if the cluster is RBAC enabled or not
rbac:
enabled: true
When I deploy the application and check the gateway, it appears to be updating the gateway through the ingress controller by creating its own settings. The problem seems to be that the application never gets exposed. I checked the health probe and it stated it was unhealthy due to 404 status. I was unable to access the application directly by IP. I get a 404 or 502 depending on how I try to access the application.
I tried deploying both an nginx and agic ingress and the nginx seems to work fine:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: aks-seed-ingress-main
annotations:
kubernetes.io/ingress.class: azure/application-gateway
# appgw.ingress.kubernetes.io/ssl-redirect: "true"
spec:
tls:
- hosts:
- agic-cluster.company.com
- frontend.<ip0>.nip.io
secretName: zigzypfxtls
rules:
- host: agic-cluster.company.com
http:
paths:
- backend:
serviceName: aks-seed
servicePort: 80
path: /
- host: frontend.<ip0>.nip.io
http:
paths:
- backend:
serviceName: aks-seed
servicePort: 80
path: /
---
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: aks-seed-ingress-nginx
annotations:
kubernetes.io/ingress.class: nginx
spec:
tls:
- hosts:
- frontend.<ip>.nip.io
rules:
- host: frontend.<ip>.nip.io
http:
paths:
- backend:
serviceName: aks-seed # Modify
servicePort: 80
path: /
I am unsure what I am missing. I followed the tutorials as best I could and the agic controller and application gateway appear to be communicating. However the application is inaccessible on the agic controller but accessible on the nginx controller. I only installed the nginx controller afterwards to ensure there was no issue with the application itself.
I am facing the same issue, I followed below article and deployed the resources
https://learn.microsoft.com/en-us/azure/developer/terraform/create-k8s-cluster-with-aks-applicationgateway-ingress
Azure ingress never came up Ready state
NAME READY STATUS RESTARTS AGE
aspnetapp 1/1 Running 0 25h
ingress-azure-1616064464-6694ff48f8-pptnp 0/1 Running 0 72s
$ helm list
NAME NAMESPACE REVISION UPDATED STATUS CHART APP VERSION
ingress-azure-1616064464 default 1 2021-03-18 06:47:45.959459087 -0400 EDT deployed ingress-azure-1.4.0 1.4.0
myrelease default 1 2021-03-18 05:45:12.419235356 -0400 EDT deployed nginx-ingress-controller-7.4.10 0.44.0
From describe pod I see below message
$ kubectl describe pod ingress-azure-1616064464-6694ff48f8-pptnp
Name: ingress-azure-1616064464-6694ff48f8-pptnp
Namespace: default
Warning Unhealthy 4s (x8 over 74s) kubelet Readiness probe failed: Get http://15.0.0.68:8123/health/ready: net/http: request canceled (Client.Timeout exceeded while awaiting headers)
$ kubectl get ingress
NAME CLASS HOSTS ADDRESS PORTS AGE
aspnetapp <none> * 80 10s
cafe-ingress-with-annotations <none> cafe.example.com 20.XX.XX.XX 80 63m
Check the health probes. When the health probes in the ingress controller are not within the accepted default return code range of 200-399, they will prevent you from accessing the app. Within the Ingress controller YAML (this is important), either change the path from '/' to a proper health endpoint within the health probe, or update the accepted range of return codes to 200-500 (for testing purposes).
Example YAML with health probes:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: ingress
annotations:
kubernetes.io/ingress.class: azure/application-gateway
appgw.ingress.kubernetes.io/use-private-ip: "false"
cert-manager.io/cluster-issuer: letsencrypt
appgw.ingress.kubernetes.io/ssl-redirect: "true"
appgw.ingress.kubernetes.io/health-probe-path: "/"
appgw.ingress.kubernetes.io/health-probe-status-codes: "200-500"
spec:
tls:
- hosts:
- dev.mysite.com
secretName: secret
rules:
- host: dev.mysite.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: srv-mysite
port:
number: 80
Please check the permission assigned to the identity Might be you are Missing the Managed Identity Operator assignment please check it
I have AKS configured with Azure Application Gateway as my ingress.
I am trying to deploy a .net core Angular app to a path within the cluster. I would like to access the app on http://<cluster ip>/app1.
My kubernetes deployment (including ingress settings) is as follows:
apiVersion: v1
kind: Pod
metadata:
name: web-app-1
labels:
app: web-app-1
spec:
containers:
- image: "xxx.azurecr.io/web-app-1:latest"
name: web-app-1
imagePullPolicy: Always
ports:
- containerPort: 80
protocol: TCP
---
apiVersion: v1
kind: Service
metadata:
name: web-app-1
spec:
selector:
app: web-app-1
ports:
- protocol: TCP
port: 80
targetPort: 80
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: web-app-1
annotations:
kubernetes.io/ingress.class: azure/application-gateway
spec:
rules:
- http:
paths:
- path: /app1
backend:
serviceName: web-app-1
servicePort: 80
In the Angular app itself, I have left <base href="/" /> in index.html. However, I have amended the build to now be ng build --base-href /app1/"
Issue
When this is deployed and I browse to http://<cluster ip>/app1 then it loads the index.html file. However it returns a 404 for all the additional scripts e.g. 404 on http://<cluster ip>/app1/main-es2015.9ae13a2658e759db61f5.js
The issue could be with how I've configured Angular, but browsing to http://<cluster ip>/app1/index.html returns a 404 when I know it can be accessed just using /app1/.
I believe the issue is that Application Gateway is not routing requests properly for anything after /app1/. How can I get it to allow sub routes through (i.e. the scripts)?
Thanks
Got this working now. If I looked at the 404 response headers it says it was from kestrel, so was hitting the dotnet core api, so it needs configuring there. All the changes I made were:
Client:
Leave the base href as / e.g.
Add the base href to the build argument e.g. ng build --base-href /app1/"
In Configure of Startup.cs, add app.UsePathBase("/app1"); I do this in the else of env.IsDevelopment().
Application Gateway:
Change the path for the rules to - path: /app1*. I didn't have the asterisk so wasn't routing all subsequent routes.
You could also do something like this
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: appgw-ingress
annotations:
kubernetes.io/ingress.class: azure/application-gateway
appgw.ingress.kubernetes.io/backend-path-prefix: "/"
spec:
rules:
- http:
paths:
- path: /api/*
....
Where you are updating the routing prefix form "/" to "/api/*" . Specifically this annotation
appgw.ingress.kubernetes.io/backend-path-prefix: "/"
I am following this web site to develop an API with Nginx Ingress. When I use curl command it works !
curl -v -k -H "Host: myServiceA.foo.org" http:<IP_ADDRESS_INGRESS_NGINX>:80
Now I would like to use a browser like Chrome or Firefox but I don't find any way to do it knowing that http:<IP_ADDRESS_INGRESS_NGINX>:80 doesn't work without header.
Do you know how to do please ?
Regards
It's not working because you've configured the host field in ingress yaml.
Using the same yaml from Nginx docs you've posted:
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: ingress-myServiceA
annotations:
# use the shared ingress-nginx
kubernetes.io/ingress.class: "nginx"
spec:
rules:
- host: myServiceA.foo.org <== HERE
http:
paths:
- path: /
backend:
serviceName: myServiceA
servicePort: 80
The ingress will only accept the connection and forwarding the request to your service if the request contains the host myServiceA.foo.org.
You could test it editing the /etc/hosts of your machine e pointing to the nginx ingress ip:
File /etc/hosts
<INGRESS_IP> myServiceA.foo.org
Or another option is remove the field host in this way the ingress will accept requests coming from the Nginx ingress ip, like this yaml:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: example-ingress
spec:
rules:
- http:
paths:
- path: /
backend:
serviceName: echo-svc
servicePort: 80