In OpenShift I'm trying to patch a service port number via OpenShift Client executable with this command but I get this error message.
$ oc patch svc fabric-boot-sample -p '{"spec":{"ports":{"port":8090}}}'
Error from server: cannot restore slice from map
The patch targets this part of the yaml service descriptor.
$ oc get svc/fabric-boot-sample -o yaml
...
spec:
clusterIP: 172.30.48.46
ports:
- name: 8090-tcp
port: 8090
protocol: TCP
targetPort: 8090
...
Is there some typo/syntax error in the patch command?
The problem is the ports that you are targeting is a list. So you will need to specify which element of the list you want to patch.
If you are sure there is only a single element in the list you could run:
oc patch svc fabric-boot-sample --type=json -p '[{"op": "replace", "path": "/spec/ports/0/port", "value":9999}]'
Related
I'm following theses instructions (page 181) to create a persistent volume & claim, a mysql replica set & service. I specify mysql v5.6 in the yaml file for the replica set.
After viewing the log for the pod, it looks like it is successful. So then I
kubectl run -it --rm --image=mysql --restart=Never mysql-client -- bash
mysql -h mysql -p 3306 -u root
It prompts me for the password and then I get this error:
ERROR 1130 (HY000): Host '10.1.0.17' is not allowed to connect to this MySQL server
Apparently MySQL has a feature where it does not allow remote connections by default and I have to change the configuration files and I don't know how to do that inside a yaml file. Below is my YAML. How do I change it to allow remote connections?
Thanks
Siegfried
cat <<END-OF-FILE | kubectl apply -f -
apiVersion: apps/v1
kind: ReplicaSet
metadata:
name: mysql
# labels so that we can bind a Service to this Pod
labels:
app: mysql
spec:
replicas: 1
selector:
matchLabels:
app: mysql
template:
metadata:
labels:
app: mysql
spec:
containers:
- name: tododata
image: mysql:5.6
resources:
requests:
cpu: 1
memory: 2Gi
env:
# Environment variables are not a best practice for security,
# but we're using them here for brevity in the example.
# See Chapter 11 for better options.
- name: MYSQL_ROOT_PASSWORD
value: some-password-here
livenessProbe:
tcpSocket:
port: 3306
ports:
- containerPort: 3306
volumeMounts:
- name: tododata
# /var/lib/mysql is where MySQL stores its databases
mountPath: "/var/lib/mysql"
volumes:
- name: tododata
persistentVolumeClaim:
claimName: tododata
END-OF-FILE
Sat Oct 24 2020 3PM EDT Update: Try Bitnami MySQL
I like Ben's idea of using bitnami mysql because then I don't have to create my own custom docker image. However, when using bitnami and trying to connect to they mysql server I get
ERROR 2003 (HY000): Can't connect to MySQL server on 'my-release-mysql.default.svc.cluster.local' (111)
This happens after I successfully get a bash shell with this command:
kubectl run my-release-mysql-client --rm --tty -i --restart='Never' --image docker.io/bitnami/mysql:8.0.22-debian-10-r0 --namespace default --command -- bash
Then, as per the instructions, I do this and get the HY000 error above.
mysql -h my-release-mysql.default.svc.cluster.local -uroot -p
Wed Nov 04 2020 Update:
Thanks Ben.. Yes -- I had already tried that on Oct 24 (approx) and when I do a k describe pod I get mysqladmin: connect to server at 'localhost' failed error: 'Can't connect to local MySQL server through socket '/opt/bitnami/mysql/tmp/mysql.sock' (2)' Check that mysqld is running and that the socket: '/opt/bitnami/mysql/tmp/mysql.sock' exists!.
Of course, when I run the mysql client as described in the nicely generated instructions, the client cannot connect because mysqld has died.
This is after having deleted the pvcs and stss and doing helm delete my-release prior to reinstalling via helm.
Unfortunately, when I tried this the first time (a couple of weeks ago) I did not set the root password and used the default generated password and I think it is still trying to use that.
This did work on azure kubernetes after having created a fresh azure kubernetes cluster. How can I reset my kubernetes cluster in my docker for desktop windows? I tried google searching and no luck so far.
Thanks
Siegfried
After a lot of help from the bitnami folks, I learned that my spinning disks on my 4 year old notebook computer are kinda slow (now why this is a problem with Bitnami MySQL and not Bitnami PostreSQL is a mystery).
This works for me:
helm install my-mysql bitnami/mysql \
--set image.debug=true \
--set primary.persistence.enabled=false,secondary.persistence.enabled=false \
--set primary.readinessProbe.enabled=false,primary.livenessProbe.enabled=false \
--set secondary.readinessProbe.enabled=false,secondary.livenessProbe.enabled=false
This turns off the peristent volumes so the data is lost when the pod dies.
Yes this is useful for me for development purposes and no one should be using Docker For Desktop/Kubernetes for production anyway... I just need to populate a tiny database and test my queries and if I need to repopulate database every time I reboot, well, that is not a big problem.
So maybe I need to get a new notebook computer? The price of notebook computers with 4TB of spinning disk space has gone up in the last couple of years.... And I cannot find any SSD drives of that size so even if I purchased a new replacement with spinning disks I might have the same problem? Hmm....
Thanks everyone for your help!
Siegfried
This appears to work just fine for me on windows. Complete the following steps:
helm repo add bitnami https://charts.bitnami.com/bitnami
helm install my-release --set root.password=awesomePassword bitnami/mysql
This is all you need to run the mysql instance. It does not makes a few services and a statefulset. Then, to connect to it, you
Either have to be in another another kubernetes container. Without this, you will not find the dns record for my-release-mysql.default.svc.cluster.local
run my-release-mysql-client --rm --tty -i --image docker.io/bitnami/mysql:8.0.22-debian-10-r0 --namespace default --command -- bash
mysql -h my-release-mysql.default.svc.cluster.local -uroot -p my_database
For the password, it should be 'awesomePassword'
Port forward the service to your local machine.
kubectl port-forward svc/my-release-mysql 3306:3306
As a note, a bitnami container will have issues if you kill it and restart it with only your helm commands and the password is not set. The persistent volume claim will usually stick around - so you would need to set the password to the old password. If you do not specify the password, you can get the password by running the commands bitnami tells you about.
NAME: my-release
LAST DEPLOYED: Thu Oct 29 20:39:23 2020
NAMESPACE: default
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES: Please be patient while the chart is being deployed
Tip:
Watch the deployment status using the command: kubectl get pods -w
--namespace default
Services:
echo Master: my-release-mysql.default.svc.cluster.local:3306 echo
Slave: my-release-mysql-slave.default.svc.cluster.local:3306
Administrator credentials:
echo Username: root echo Password : $(kubectl get secret
--namespace default my-release-mysql -o jsonpath="{.data.mysql-root-password}" | base64 --decode)
To connect to your database:
Run a pod that you can use as a client:
kubectl run my-release-mysql-client --rm --tty -i --restart='Never' --image docker.io/bitnami/mysql:8.0.22-debian-10-r0 --namespace default --command -- bash
To connect to master service (read/write):
mysql -h my-release-mysql.default.svc.cluster.local -uroot -p my_database
To connect to slave service (read-only):
mysql -h my-release-mysql-slave.default.svc.cluster.local -uroot -p my_database
To upgrade this helm chart:
Obtain the password as described on the 'Administrator credentials' section and set the 'root.password' parameter as shown
below:
ROOT_PASSWORD=$(kubectl get secret --namespace default my-release-mysql -o jsonpath="{.data.mysql-root-password}" | base64
--decode)
helm upgrade my-release bitnami/mysql --set root.password=$ROOT_PASSWORD
I have created pypiserver in kubernetes cluster, I have used https://hub.docker.com/r/pypiserver/pypiserver docker image. I need to create basic auth for the server which I created. I used this method https://kubernetes.github.io/ingress-nginx/examples/auth/basic/
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: pypiserver
labels:
app: pypiserver
annotations:
kubernetes.io/ingress.class: nginx
kubernetes.io/tls-acme: 'true'
ingress.kubernetes.io/auth-type: basic
ingress.kubernetes.io/auth-secret: secret
ingress.kubernetes.io/auth-realm: "Authentication Required - ok"
spec:
rules:
- host: example.com
http:
paths:
- path: /
backend:
serviceName: pypiservice
servicePort: 8080
tls:
- hosts:
- example.com
secretName: secret-tls
But my host name would be "www.example.com/8080" and I don't see ingress has any pod in kubernetes cluster. Ingress is running fine but I don't get auth for this host. (And also I have http://IP adress:8080 which I converted to domain through cloudflare)
Please let me know what am I doing wrong?
I don't know exactly what is your nginx ingress controller version, but I can share what worked for me. I've reproduced it on my GKE cluster.
I installed my nginx ingress controller following this guide. Basically it came down to running the following commands:
If you're using GKE you need to initialize your user as a
cluster-admin with the following command:
kubectl create clusterrolebinding cluster-admin-binding \
--clusterrole cluster-admin \
--user $(gcloud config get-value account)
The following Mandatory Command is required for all deployments.
kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/nginx-0.27.1/deploy/static/mandatory.yaml
I'm using 1.13 version on my GKE so this tip is also applied in my case:
Tip
If you are using a Kubernetes version previous to 1.14, you need to
change kubernetes.io/os to beta.kubernetes.io/os at line 217 of
mandatory.yaml, see Labels details.
But I dealt with it quite differently. Basically you need your Nodes to have kubernetes.io/os=linux label so you can simply label them. Following command will do the job:
kubectl label node --all kubernetes.io/os=linux
Then we're heading to Provider Specific Steps which in case of GKE came down to applying the following yaml:
kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/nginx-0.27.1/deploy/static/provider/cloud-generic.yaml
Then you may want to verify your installation:
To check if the ingress controller pods have started, run the
following command:
kubectl get pods --all-namespaces -l app.kubernetes.io/name=ingress-nginx --watch
or simply run:
kubectl get all -n ingress-nginx
It will also tell you if all the required resorces are properly deployed.
Next we need to write our ingress (ingress object/resource) containing basic-auth related annotations. I was following same tutorial as mentioned in your question.
First we need to create our auth file containing username and hashed password:
$ htpasswd -c auth foo
New password: <bar>
New password:
Re-type new password:
Adding password for user foo
Once we have it, we need to create a Secret object which then we'll use in our ingress:
$ kubectl create secret generic basic-auth --from-file=auth
secret "basic-auth" created
Once it is created we can check if everything went well:
$ kubectl get secret basic-auth -o yaml
apiVersion: v1
data:
auth: Zm9vOiRhcHIxJE9GRzNYeWJwJGNrTDBGSERBa29YWUlsSDkuY3lzVDAK
kind: Secret
metadata:
name: basic-auth
namespace: default
type: Opaque
Alright, so far so good...
Then we need to create our ingress resource/object.
My ingress-with-auth.yaml file looks slightly different than the one in the instruction, namely I just added kubernetes.io/ingress.class: nginx to make sure my nginx ingress controller is used rather than built-in GKE solution:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ingress-with-auth
annotations:
kubernetes.io/ingress.class: nginx
# type of authentication
nginx.ingress.kubernetes.io/auth-type: basic
# name of the secret that contains the user/password definitions
nginx.ingress.kubernetes.io/auth-secret: basic-auth
# message to display with an appropriate context why the authentication is required
nginx.ingress.kubernetes.io/auth-realm: 'Authentication Required - foo'
spec:
rules:
- host: foo.bar.com
http:
paths:
- path: /
backend:
serviceName: pypiserver
servicePort: 80
In your example you may need to add nginx prefix in your basic-auth related annotations:
ingress.kubernetes.io/auth-type: basic
ingress.kubernetes.io/auth-secret: secret
ingress.kubernetes.io/auth-realm: "Authentication Required - ok"
so it looks like this:
nginx.ingress.kubernetes.io/auth-type: basic
nginx.ingress.kubernetes.io/auth-secret: secret
nginx.ingress.kubernetes.io/auth-realm: "Authentication Required - ok"
First I used the address listed in my ingress resource (it doesn't appear there any more once I added kubernetes.io/ingress.class: nginx annotation in my ingress definition:
$ kubectl get ingress
NAME HOSTS ADDRESS PORTS AGE
ingress-with-auth foo.bar.com 80 117m
When I tried to access pypi-server using this IP it brought me directly to the page without a need of any authentication. But it looks like if you didn't define proper ingress class, the default is used instead so in practice your ingress definition with auth-basic details isn't taken into consideration and isn't passed to the nginx ingress controller we installed in one of the previous steps.
So what IP address should be used to access your app ? Run the following command which will show you both CLUSTER-IP (can be accessed within your cluster from any Pod or Node) and EXTERNAL-IP of your nginx ingress controller:
$ kubectl get service --namespace ingress-nginx
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
ingress-nginx LoadBalancer 10.0.3.220 35.111.112.113 80:30452/TCP,443:30006/TCP 18h
You can basically host many different websites in your cluster and all of them will be available through this IP. All of them can be available on default http 80 port (or https 443 in your case). The only difference between them will be the hostname that you pass in http header of your http request.
Since I don't have a domain pointing to this external IP address and can't simply access my website by going to http://foo.bar.com I need to pass somehow the hostname I'm requesting from 35.111.112.113 address. It can be done in a few ways:
I installed in my Google Chrome browser ModHeader extension which allows me to modify my http request headers and set the hostname I'm requestig to any value I want.
You can do it also using curl as follows:
curl -v http://35.111.112.113 -H 'Host: foo.bar.com' -u 'foo:bar'
You should be prompted for authentication.
If you don't provide -u username:password flag you should get 401 Authorization Required.
Basically hat's all.
Let me know if it helped you. Don't hesitate to ask additional questions if something isn't completely clear.
One more thing. If something still doesn't work you may start from attaching to your nginx ingress controller Pod (check your Pod name first by running kubectl get pods -n ingress-nginx):
kubectl exec -ti -n ingress-nginx nginx-ingress-controller-pod /bin/bash
and checking the content of your /etc/nginx/nginx.conf file. Look for foo.bar.com (or in your case example.com). It should contain similar lines:
auth_basic "Authentication Required - foo";
auth_basic_user_file /etc/ingress-controller/auth/default-ingress-with-auth.passwd;
Then check if the file is present in the indicated location /etc/ingress-controller/auth/default-ingress-with-auth.passwd.
One note to your Service definition. The fact that pypiserver container exposes specifically port 8080 doesn't mean that you need to use this port when accessing it via ingress. In Service definition the port exposed by the Container is called targetPort. You need to specify it when defining your Service but Service itself can expose completely different port. I defined my Service using following command:
kubectl expose deployment pypiserver --type=LoadBalancer --port=80 --target-port=8080
Note that the type should be set to NodePort or LoadBalancer. Then in your ingress definition you don't have to use 8080 but 80 which is the port exposed by your pypiserver Service. Note that there is servicePort: 80 in my ingress object/resource definition. Your example.com domain in cloudflare should point with it's A record to your nginx ingress controller LoadBalancer Service IP (kubectl get svc -n ingress-nginx) without specifying any ports.
I want to pass an environment variable that should get evaluated to the hostname of the running container. This is what I am trying to do
oc new-app -e DASHBOARD_PROTOCOL=http -e ADMIN_PASSWORD=abc#123 -e KEYCLOAK_URL=http://keycloak.openidp.svc:8080 -e KEYCLOAK_REALM=master -e DASHBOARD_HOSTNAME=$HOSTNAME -e GF_INSTALL_PLUGINS=grafana-simple-json-datasource,michaeldmoore-annunciator-panel,briangann-gauge-panel,savantly-heatmap-panel,briangann-datatable-panel grafana/grafana:5.2.1
How to ensure that the DASHBOARD_HOSTNAME gets evaluated to the value of the hostname of the running container image
For take the hostname value from a pod you could use the metadata.name.
follow the eg:
env:
- name: HOSTNAME
valueFrom:
fieldRef:
apiVersion: v1
fieldPath: metadata.name
After creating the application, you could edit the deployment config (oc edit dc/<deployment_config>) or patch it to configure the DASHBOARD_HOSTNAME environment variable using the Downward API.
This may be a personal preference but as much as oc new-app is convenient I'd rather work with (declarative) configuration files that are checked in and versioned in a code repo than with imperative commands.
Using kubeadm with a two-node cluster on VirtualBox Centos7 VMs.
I have an app written in R and a mysql database each in their own pods.
I've successfully followed instructions to setup nginx ingress controller so that the app can be reached outside the VMs, by my local machine. check :)
However, now when the app (R) tries to reach to the mysql service, the name doesn't resolve. Same with pinging 'mysql' from bash. This no longer works:
mydb<-dbConnect(MySQL(), user = 'root', password ='password',
dbname = 'prototype', host = 'mysql')
Instead I have to use the pod's IP, which does work.
mydb<-dbConnect(MySQL(), user = 'root', password ='password',
dbname = 'prototype', host = '10.244.1.233')
However, isn't this going to change upon reboots and system changes? I'd like a more static way to refer to the mysql db.
$ kubectl get endpoints
NAME ENDPOINTS AGE
kubernetes 192.168.56.101:6443 5h
mysql 10.244.1.233:3306 41m
r-user-app 10.244.1.232:8787,10.244.1.232:3838 2h
$ kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 5h
mysql ClusterIP 10.96.138.132 <none> 3306/TCP 28m
r-user-app LoadBalancer 10.100.228.80 <pending> 3838:32467/TCP,8787:31754/TCP 2h
$ kubectl get ing
NAME HOSTS ADDRESS PORTS AGE
r-user-app storage.test.com 80, 443 3h
$ kubectl describe service mysql
Name: mysql
Namespace: default
Labels: <none>
Annotations: <none>
Selector: app=neurocore,tier=mysql
Type: ClusterIP
IP: 10.96.138.132
Port: <unset> 3306/TCP
TargetPort: 3306/TCP
Endpoints: 10.244.1.236:3306
Session Affinity: None
Events: <none>
ps auxw | grep kube-proxy
root 1914 0.1 0.3 44848 21668 ? Ssl 11:03 0:20 /usr/local/bin/kube-proxy --config=/var/lib/kube-proxy/config.conf
root 29218 0.0 0.0 112660 980 pts/1 R+ 14:23 0:00 grep --color=auto kube-proxy
$iptables-save | grep mysql
-A KUBE-SEP-7P27CEQL6WJZRBQ5 -s 10.244.1.236/32 -m comment --comment "default/mysql:" -j KUBE-MARK-MASQ
-A KUBE-SEP-7P27CEQL6WJZRBQ5 -p tcp -m comment --comment "default/mysql:" -m tcp -j DNAT --to-destination 10.244.1.236:3306
-A KUBE-SERVICES ! -s 10.244.0.0/16 -d 10.96.138.132/32 -p tcp -m comment --comment "default/mysql: cluster IP" -m tcp --dport 3306 -j KUBE-MARK-MASQ
-A KUBE-SERVICES -d 10.96.138.132/32 -p tcp -m comment --comment "default/mysql: cluster IP" -m tcp --dport 3306 -j KUBE-SVC-M7XME3WTB36R42AM
-A KUBE-SVC-M7XME3WTB36R42AM -m comment --comment "default/mysql:" -j KUBE-SEP-7P27CEQL6WJZRBQ5
Based on your svc, you should be able to reach mysql:3306 from within the cluster.
Have you tried kubectl exec -it r-user-app bash and pinging mysql from within the R app container? host mysql should return something like "mysql.cluster.local" has address 127.21.0.01" (example). Or return any error. If there isn't an error then maybe the dbConnect() doesn't like the host name?
Looks like your service is configured well.
ping 10.96.138.132 no response :(
Each Service has a static address, so the situation when you cannot ping it is normal because that is just a virtual address and the requests to its processing is a bit different than the requests to the real addresses.
I see here only 2 reasons why you can have that problem:
Something wrong with DNS resolution in the container with your application. Try to use 10.96.138.132 as MySQL address instead of mysql. If it fixes your problem - that is a resolving problem. BTW, you can use Service IP instead of DNS, as I already told - it is static.
Something wrong with the forwarding rules. Check kube-proxy logs in kube-system namespace, maybe you will get any additional info for debugging.
This is actually an issue with flannel. When I switched to use Weave as the CNI, the service discovery and DNS kube works fine.
kubectl apply -f "https://cloud.weave.works/k8s/net?k8s-version=$(kubectl version | base64 | tr -d '\n')"
I have Wildfly 10 running on Openshift Origin 3 in AWS with an elastic ip.
I setup a Route in Openshift to map / to the wildfly service. This is working fine. If I go to http://my.ip.address I get the WildFly welcome page.
But if I map a different path, say /wf01, it doesn't work. I get a 404 Not Found error.
My guess is the router is passing along the /wf01 to the service? If that's the case, can I stop it from doing it? Otherwise how can I map http://my.ip.address/wf01 to my wildfly service?
I also want the wildfly console to be accessible from outside (this is a demo server for my own use). I added "-bmanagement","0.0.0.0" to the deploymentconfig but looking at the wildfly logs it is still binding to 127.0.0.1:
02:55:41,483 INFO [org.jboss.as] (Controller Boot Thread) WFLYSRV0051:
Admin console listening on http://127.0.0.1:9990
A router today cannot remap/rewrite the incoming HTTP path to another path value before passing it along. A workaround is to mount another route+service at the root that handles the root and redirects / forwards.
You can also use port-forward :
oc port-forward -h
Forward 1 or more local ports to a pod
Usage:
oc port-forward POD [LOCAL_PORT:]REMOTE_PORT [...[LOCAL_PORT_N:]REMOTE_PORT_N] [options]
Examples:
# Listens on ports 5000 and 6000 locally, forwarding data to/from ports 5000 and 6000 in the pod
$ oc port-forward -p mypod 5000 6000
# Listens on port 8888 locally, forwarding to 5000 in the pod
$ oc port-forward -p mypod 8888:5000
# Listens on a random port locally, forwarding to 5000 in the pod
$ oc port-forward -p mypod :5000
# Listens on a random port locally, forwarding to 5000 in the pod
$ oc port-forward -p mypod 0:5000