My Setup
GKE / EKS - Managed Kubernetes Cluster
As of now for Business requirements, it is k8s cluster with Public Endpoints
What it means is that I have a Public endpoint for API Server as well Nodes have an External Public IP Address
nginx ingress is deployed for route-based traffic and exposed as a Loadbalancer type
And The LoadBalancer is of type Network Load Balancer internet facing(Or External) having a Public IP Address (say 35.200.24.99)
My requirement or I want to understand, is this
If my Pod makes a call to the outside APIs, what will be the source IP that the outside API will receive? Is it my LoadBalencer IP or the Pod Node External IP Address
If it receives the LB IP, is there a way to change this behavior to send the Pod Node IP Address?
Also is there any tool or a way to simulate what is the Source IP, I am getting while Pod makes a request to an outside API
I could not try out anything
I tried hitting curl requests to nginx Pod that wsa running inside, but did not get desired results or I could not figure out
If my Pod makes a call to the outside APIs, what will be the source IP
that the outside API will receive? Is it my LoadBalencer IP or the Pod
Node External IP Address
It your POD sending request and your cluster is public it will be Node's IP on which POD is running/scheduled.
If it receives the LB IP, is there a way to change this behavior to
send the Pod Node IP Address?
it wont get the LB IP, it will be Node's IP only on which POD is running. If you want to manage the Single outgoing IP you can use the NAT gateway so all traffic will go out of the single source IP.
Also is there any tool or a way to simulate what is the Source IP, I
am getting while Pod makes a request to an outside API
Go to the POD using kubectl exec -it <POD name> bash once you are inside the POD run the curl ifconfig.me it will return the IP from which you are hitting the site. Mostly it will be Node's IP.
Consider ifconfig.me as an outside API and you will get your result.
Related
I have 3 node k8s Cluster on my virtual env which is VMware Fusion.
When try to create basic Ingress it takes my one node_ip which is nginx_controller running.
But 80 port is not open on all nodes. I mean it is not working
curl: (7) Failed to connect to 172.16.242.133 port 80: Connection refused
What I missing ?
I installed Nginx Ingress Controller
I installed MetalLB and configured it. It is working if I create service with type: LoadBalancer. It takes ExernalIp Ip and I can access it.
I deploy basic app for test.
I create a service for app. I can access on NodePort or CulesterIP. Both I tried.
I create basic Ingress for manage hosts and routing staff. But this step I stuck.
My Questions ;
1-) Normaly what should Ingress take Ip as Address ? One of my node or External DHCP IP.
2-) When I create service with type: LoadBalancer it takes externalIP. I can record DNS to this IP and clients can access it. What is wrong with that ?
Ingress supports two types of service type: NodePort and LoadBalancer.
While using NodePort service type you should use nodeport number instead of default port 80. Explanation to this behavior is available in nginx ingress documentation:
However, due to the container namespace isolation, a client located
outside the cluster network (e.g. on the public internet) is not able
to access Ingress hosts directly on ports 80 and 443. Instead, the
external client must append the NodePort allocated to the
ingress-nginx Service to HTTP requests.
So your curl should look like this:
curl 172.16.242.133:<node_port_number>
When you use MetalLB with LoadBalancer service type, it takes externalIPs from it's configuration that you specified when installing metallb in cluster.
More information about nginx ingress controller cooperation with metallb is available in nginx documentation.
MetalLB requires a pool of IP addresses in order to be able to take
ownership of the ingress-nginx Service. This pool can be defined in a
ConfigMap named config located in the same namespace as the MetalLB
controller. This pool of IPs must be dedicated to MetalLB's use, you
can't reuse the Kubernetes node IPs or IPs handed out by a DHCP
server.
My Problem was,
I thought Ingress takes the IP and we record DNS to this IP. But It is not. Why Ingress object has Address and Port field I do not know. Just for information I guess but It is confusing for newbies.
Clients access the Ingress Controller not Ingress.
Actually Ingress Controller Service manages the externalIP or NodePort. So we have to configure this.
In my case nginx
kubectl edit service/ingress-nginx-controller -n ingress-nginx
You can change type to LoadBalancer and you will get externalIP after configured the MetalLB. And define your Ingress objects, record DNS Records then you are ready.
I have deployed the Spring boot app on the OCI compute and its comping up nicely. Compute is created with public ip and have the security list updated to allow connections from internet. But, I wasn't able to hit the end point from internet. For that reason, I thought of configuring the load balancer.
Created load balancer in a separate subnet(10.0.1.0/24), routing table and security list. Configured the LB's security list to send all protocol packets to compute's CIDR(10.0.0.0/24) and configured compute's security list to accept the packets from LB. I was expecting LB to make connection with back end. But, its not.
I am able to hit the LB from internet :-
Lb's routing table with all ips routed through internet gateway. There is no routing defined for compute's CIDR as its in the VCN.
LB has its own security list, which has allowed out going packets to compute and incoming from internet as below:
Compute's security list accepting packet's from LB:
Let me know, if I am missing something here.
My internet gateway :-
My backend set connection configuration from LB:
LB fails to make connection with backend, there seems to be no logging info available :
App is working fine , if I access from the compute node :
The LB has a health check that tests the connection to your service. If it fails, the LB will keep your backend out of rotation and give you the critical health like you're seeing.
You can get to it by looking at the backend set and clicking the Update Health Check button.
Edit:
Ultimately I figured it out, you should run the following commands on your backend:
sudo firewall-cmd --permanent --add-port=8080/tcp
sudo firewall-cmd --reload
Use the port that you configured your app to listen on.
I used httpd instead of spring, but I also did the following
sudo semanage fcontext -a -t httpd_sys_content_t "/var/www/html(/.*)?"
sudo restorecon -F -R -v /var/www/html
I'm not really too familiar with selinux but you may need to do something similar for your application.
Additionally, setting up a second host in the same subnet to login to and test connecting to the other host will help troubleshooting, since it will verify if your app is accessible at all outside the host that it's on. Once it is, the LB should come up fine.
TL;DR In my case it helped to switch the Security List rules from stateful to stateless on the 2 relevant subnets (where the loadbalancer was hosted and where the backends were located).
In our deployment I had a loadbalancer with public IP located on one subnet, while the backend to this loadbalancer was on another subnet. Both subnets had one ingress and one egress rule - to allow everything (i.e. 0.0.0.0/0 and all ports allowed). The backends were still not reachable from the loadbalancer and the healthchecks were failing.
Even despite the fact that in my case as per the documentation switching between stateful and stateless should not have an effect, it solved my issue.
I have a Python application which has been deployed to openshift.
I am using an external REST service in my application. In order to use this service, the developers of the REST service have to whitelist my IP because a Firewall blocks unauthorized IP addresses.
How can I find the external IP of my application? How can I find it in openshift? I tried a few OC commands, but I am not sure if I have to get the IP of the pod or the service.
Out of the box the traffic from internal cluster components will appear to external infrastructure like they are coming from whichever OpenShift compute host their pods are currently scheduled on.
Information on internal cluster networking and how traffic traverses from a process running inside a pod to the external network can be found at SDN: Packet Flow.
In your case you could have the external application whitelist all of the ip addresses of the compute hosts that are expected to run your application pods.
Alternately you could set up an EgressIP. This will cause all traffic originating from a specific OpenShift project to appear as if it is originating from a single ip address. You could then have your external application whitelist the EgressIP address.
Documentation for configuring EgressIP can be found in the official documentation under Enabling Static IPs for External Project Traffic
What you are searching for is the external IP of the Service. A Service acts as a load balancer for your pods but by default it only has a cluster-wide IP address. If you need a URL to access it from the outside, you can create a Route. For your purpose where you need an actual external IP address, you can assign the Service an external IP manually. Information on how to do this can be found in the official OpenShift Docs.
We are in the process of move all our services over to Docker hosted on Google Container Engine. In the mean time we have have some services in docker and some not.
Within Kubernetes services discovery is easy via DNS, but how do I resolve services from outside my container cluster? ie, How do I connect from a Google Compute Engine instance to a service running in Kubernetes?
The solution I have for now is to use the service clusterIP address.
You can see this IP address by executing kubectl get svc. This ip address is by default not static, but you can assign it when defining you service.
From the documentation:
You can specify your own cluster IP address as part of a Service creation request. To do this, set the spec.clusterIP
The services are accessed outside the cluster via IP address instead of DNS name.
Update
After deploying another cluster the above solution did not work. It turns out that the new IP range could not be reached and that you do need to add a network route.
You can get the cluster IP range by running
$ gcloud container clusters describe CLUSTER NAME --zone ZONE
In the output the ip range is shown with the key clusterIpv4Cidr, in my case it was 10.32.0.0/14.
Then create a route for that ip range that points to one of the nodes in your cluster. $ gcloud compute routes create --destination-range 10.32.0.0/14 --next-hop-instance NODE0 INSTANCE NAME
I have deployed a hadoop cluster on google compute engine. I then run a machine learning algorithm (Cloudera's Oryx) on the master node of the hadoop cluster. The output of this algorithm is accessed via an HTTP REST API. Thus I need to access the output either by a web browser, or via REST commands. However, I cannot resolve the address for the output of the master node which takes the form http://CLUSTER_NAME-m.c.PROJECT_NAME.internal:8091.
I have allowed http traffic and allowed access to ports 80 and 8091 on the network. But I cannot resolve the address given. Note this http address is NOT the IP address of the master node instance.
I have followed along with examples for accessing IP addresses of compute instances. However, I cannot find examples of accessing a single node of a hadoop cluster on GCE, that follows this form http://CLUSTER_NAME-m.c.PROJECT_NAME.internal:8091. Any help would be appreciated. Thank you.
The reason you're seeing this is that the "HOSTNAME.c.PROJECT.internal" name is only resolvable from within the GCE network of that same instance itself; these domain names are not globally visible. So, if you were to SSH into your master node first, and then try to curl http://CLUSTER_NAME-m.c.PROJECT_NAME.internal:8091 then you should successfully retrieve the contents, whereas trying to access from your personal browser will just fail to resolve that hostname into any IP address.
So unfortunately, the quickest way for you to retrieve those contents is indeed to use the external IP address of your GCE instance. If you've already opened port 8091 on the network, simply use gcutil getinstance CLUSTER_NAME-m and look for the entry specifying external IP address; then plug that in as your URL: http://[external ip address]:8091.
If you turned up the cluster using bdutil, a more involved but nicer way to access your cluster is by running the bdutil socksproxy command. This opens a dynamic-port-forwarding SSH tunnel to your master node as a SOCKS5 proxy, so that you can then configure your browser to use localhost:1080 as your proxy server, make sure to enable remote DNS resolution, and then visit your browser using the normal http://CLUSTER_NAME-m.c.PROJECT_NAME.internal:8091 URL.