How can be a service that does not use HTTP/s be exposed in Openshift 3.11 or 4.x?
I think routes only support HTTP/s traffic.
I have read about using ExternalIP configuration for services but that makes the operation of the cluster complicated and static compared to routes/ingress.
For example Nginx-ingress-controller allows it with special configurations: https://kubernetes.github.io/ingress-nginx/user-guide/exposing-tcp-udp-services/
What are the options in Openshift 3.11 or 4.x?
Thank you.
There is a section in the official OpenShift documentation for this called Getting Traffic Into the Cluster.
The recommendation, in order or preference, is:
- If you have HTTP/HTTPS, use a router.
- If you have a TLS-encrypted protocol other than HTTPS (for example, TLS with the SNI header), use a router.
- Otherwise, use a Load Balancer, an External IP, or a NodePort.
NodePort exposes the Service on each Node’s IP at a static port (30000~32767)[0].
You’ll be able to contact the NodePort Service, from outside the cluster, by requesting : format.
apiVersion: v1
kind: Service
metadata:
name: nodeport
spec:
type: NodePort
ports:
- name: "8080"
protocol: "TCP"
port: 8080
targetPort: 80
nodePort: 30000
selector:
labelName: targetname
Related
I am trying to expose certain Services Port Directly via Traefik Ingress, running on K3s. I have a usecase where I want to expose certain ports directly on Loadbalancer, like:
8080 - Tomcat - TCP
1700 - UDP
1795 - TCP
I tried creating following config based on this thread from Nginx Repo (I know it's for Nginx, but, I couldn't find a suitable doc in Traefik) :
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: multiportsvc
annotations:
kubernetes.io/ingress.class: traefik
spec:
rules:
- host: node2.givmecloud.in
http:
paths:
#- path: /
- backend:
serviceName: webui
servicePort: 80
- backend:
serviceName: the-udp-service
servicePort: 1700
- backend:
serviceName: tomcat-tcp-service
servicePort: 8080
However, the PORTS are not directly exposed on Ingress, just port 80/443 seems to be exposed on loadbalancer. I think this SO thread mentions that it's not directly possible:
Open other ports more than HTTP & HTTPS in Traefik Kubernetes Ingress
Nginx seems to support this, I check this Nginx documentation: https://kubernetes.github.io/ingress-nginx/user-guide/exposing-tcp-udp-services/
Is it something that's possible with traefik either via v1 or v2 or I need to use Nginx or MetalLB for this type of usecase if not, what's recommended for this kind of use cases?
Thanks,
Could someone help to connect to rethinkdb in openshift using rethinkdbdash
I have deployed rethinkdb in openshift & create 3 clusterIP services
1.8080 - admin
2.29015 - intracluster communicated
3.28015 - client connection
I have created a route which targets client connection clusterIP service(port 28015)
I tried to use that from client side as below
const r = require('rethinkdbdash')({
cursor: true,
silent: true,
host: rethink-client.test.exchange.com,
port: 80
)}
I am getting below error
data: Timeout during operation
(node:5739) UnhandledPromiseRejectionWarning: Error: Cannot wrap non-Error object
You should use NodePort or LoadBalancer type Services to expose your DB connection to external instead of Route. Because Route does not support TCP protocol. Refer here for supported protocols.
For instance of mysql db, further details are provided in Using a NodePort to Get Traffic into the Cluster.
apiVersion: v1
kind: Service
metadata:
name: mysql
labels:
name: mysql
spec:
type: NodePort
ports:
- port: 3306
nodePort: 30036
name: http
selector:
name: mysql
I have a scenario where I have multiple mysql servers running in different namespaces in single kubernetes cluster. All mysql servers belong to different departments.
My requirement is I should be able to connect to different mysql servers using hostname, i.e.,
mysqlServerA running in ServerA namespace should be reachable from outside the cluster using:
mysql -h mysqlServerA.mydomain.com -A
mysqlServerB running in ServerB namespace should be reachable from outside the cluster using:
mysql -h mysqlServerB.mydomain.com -A
and so on.
I have tried TCP based routing using config maps of Nginx Ingress controller, where I am routing traffic from clients to different mysql servers by assigning different port numbers:
for mysqlServerA:
mysql -h mysqlServer.mydomain.com -A -P 3301
for mysqlServerB:
mysql -h mysqlServer.mydomain.com -A -P 3302
this works perfectly. But I want to know if hostname based routing is possible or not, because I don't want separate load balancer for each mysql service.
Thanks
General info
I routing traffic by different port numbers
You are right, the reason for that is that connection to Mysql is done via TCP. That is why it is definitely not possible to have two simultaneous connections to two servers on the same IP:port.
Unlike HTTP, the TCP don't have headers that allows distinguishing the host the traffic shall be routed to. However, still there are at least two ways to achieve the functionality you'd like to achieve :) I'll describe that later.
I want to know if hostname based routing is possible or not
I don't want separate load balancer for each mysql service.
K8s allows a few methods for service to be reachable outside the cluster (namely hostNetwork, hostPort, NodePort , LoadBalancer, Ingress )
The LoadBalancer is the simplest way to serve traffic on LoadBalancerIP:port ; however, due to TCP nature of connection you'll have to use one LoadBalancer per one mysql instance.
kind: Service
apiVersion: v1
metadata:
name: mysql
spec:
type: LoadBalancer
ports:
- port: 3306
selector:
name: my-mysql
The NodePort looks good, but it allows you connecting only when you know port (which can be tedious work for clients)
Proposed solutions
External IPs
If there are external IPs that route to one or more cluster nodes, Kubernetes Services can be exposed on those externalIPs. Traffic that ingresses into the cluster with the external IP (as destination IP), on the Service port, will be routed to one of the Service endpoints. externalIPs are not managed by Kubernetes and are the responsibility of the cluster administrator.
In the Service spec, externalIPs can be specified along with any of the ServiceTypes. In the example below, mysql-1 can be accessed by clients on 1.2.3.4:3306 (externalIP:port) and mysql-2 can be accessed by clients on 4.3.2.1:3306
$ cat stereo-mysql-3306.yaml
apiVersion: v1
kind: Service
metadata:
name: mysql-1234-inst-1
spec:
selector:
app: mysql-prod
ports:
- name: mysql
protocol: TCP
port: 3306
targetPort: 3306
externalIPs:
- 1.2.3.4
---
apiVersion: v1
kind: Service
metadata:
name: mysql-4321-inst-1
spec:
selector:
app: mysql-repl
ports:
- name: mysql
protocol: TCP
port: 3306
targetPort: 3306
externalIPs:
- 4.3.2.1
Note: you need to have 1.2.3.4 and 4.3.2.1 assigned to your Nodes (and resolve mysqlServerA / mysqlserverB at mydomain.comto these IPs as well). I've tested that solution on my GKE cluster and it works :).
With that config all the requests for mysqlServerA.mydomain.com:3306 that resolves to 1.2.3.4 are going to be routed to the Endpoints for service mysql-1234-inst-1 with the app: mysql-prod selector, and mysqlServerA.mydomain.com:3306 will be served by app: mysql-repl.
Of course it is possible to split that config for 2 namespaces (one namespace - one mysql - one service per one namespace).
ClusterIP+OpenVPN
Taking into consideration that your mysql pods have ClusterIPs, it is possible to spawn additional VPN pod in cluster and connect to mysqls through it.
As a result, you can establish VPN connection and have access to all the cluster resources. That is very limited solution which requires establishing the VPN connection for anyone who needs access to mysql.
Good practice is to add a bastion server on top of that solution. That server will be responsible for providing access to cluster services via VPN.
Hope that helps.
I would like to capture the external I.P. address of clients visiting my application. I am using kubernetes on AWS/Kops. The ingress set-up is Voyager configured HAProxy. I am using the LoadBalancer service.
I configured HAProxy through Voyager to add the x-forwarded-for header by using ingress.appscode.com/default-option: '{"forwardfor": "true"}' annotation.
The issue is that when I test the header is coming through with an internal I.P. address of one of my kubernetes nodes, rather than my external I.P. as desired.
I'm not sure what LoadBalancer voyager is using under the covers, there's no associated pod, just one for the ingress-controller.
kubectl describe svc voyager-my-app outputs
Name: <name>
Namespace: <namespace>
Labels: origin=voyager
origin-api-group=voyager.appscode.com
origin-name=<origin-name>
Annotations: ingress.appscode.com/last-applied-annotation-keys:
ingress.appscode.com/origin-api-schema: voyager.appscode.com/v1beta1
ingress.appscode.com/origin-name: <origin-name>
Selector: origin-api-group=voyager.appscode.com,origin-name=<origin-name>,origin=voyager
Type: LoadBalancer
IP: 100.68.184.233
LoadBalancer Ingress: <aws_url>
Port: tcp-443 443/TCP
TargetPort: 443/TCP
NodePort: tcp-443 32639/TCP
Endpoints: 100.96.3.204:443
Port: tcp-80 80/TCP
TargetPort: 80/TCP
NodePort: tcp-80 30263/TCP
Endpoints: 100.96.3.204:80
Session Affinity: None
External Traffic Policy: Cluster
Events: <none>
Typically with Kubernetes ingresses, there are a couple relevant settings:
xff_num_trusted_hops, which specifies the number of hops that are "trusted" i.e., internal. This way you can distinguish between internal and external IP addresses.
You'll want to make sure you set ExternalTrafficPolicy: local in your load balancer (you didn't specify what your LB is)
Note I'm mostly familiar with Ambassador (built on Envoy Proxy) which does this by default.
We have a issue where connecting to AWS RDS in Istio Service Mesh is results in upstream connect error or disconnect/reset before header .
Our Egress rule is as below
apiVersion: config.istio.io/v1alpha2
kind: EgressRule
metadata:
namespace: <our-namespace>
name: rds-egress-rule-with
spec:
destination:
service: <RDS End point>
ports:
- port: 80
protocol: http
- port: 443
protocol: https
- port: 3306
protocol: https
The connection to MySQL works fine in a stand alone MySQL in EC2. The connection to AWS RDS works fine without Istio. The problem only occurs in Istio Service Mesh.
We are using istio in Disabled Mutual TLS Configuration.
The protocol in your EgressRule definition should be tcp. The service should contain the IP address or a range of IP addresses in CIDR notation.
Alternatively, you can use the --includeIPRanges flag of istioctl kube-inject, to specify which IP ranges are handled by Istio. Istio will not interfere with the the not-included IP addresses and will just allow the traffic to pass thru.
References:
https://istio.io/latest/blog/2018/egress-tcp/
https://istio.io/latest/docs/tasks/traffic-management/egress/egress-control/#direct-access-to-external-services