I ran into a problem exposing a mysql database running inside a kubernetes cluster publicly. The cluster runs with kops on AWS. Im using a helm chart for nginx-ingress: https://github.com/helm/charts/tree/master/stable/nginx-ingress
controller:
config:
use-proxy-protocol: "true"
metrics:
enabled: true
replicaCount: 2
service:
annotations:
service.beta.kubernetes.io/aws-load-balancer-proxy-protocol: '*'
stats:
enabled: true
rbac:
create: true
tcp:
5000: default/cbioportal-prod-db-mysql:3306
From within the cluster I can telnet to the db through nginx over port 5000 :
# telnet eating-dingo-nginx-ingress-controller 5000
J
5.7.14
ke_|c&tc"ui%]}mysql_native_passwordConnection closed by foreign host
But i can't seem to connect from outside using the hostname of the aws load balancer.
telnet xxx.us-east-1.elb.amazonaws.com 5000
Trying x.x.x.x...
When i look in aws ec2 dashboard i see the load balancer's security group allows connections from everywhere on port 5000.
UPDATE
I can connect when I use port 3306 instead of 5000:
tcp:
3306: default/cbioportal-prod-db-mysql:3306
However now that the port is open:
$ nmap --verbose -Pn x.x.x.x
PORT STATE SERVICE
21/tcp open ftp
80/tcp open http
443/tcp open https
3306/tcp open mysql
I am getting an authorization issue:
$ mysql -h x.x.x.x -uroot -pabcdef
mysql: [Warning] Using a password on the command line interface can be insecure.
ERROR 2013 (HY000): Lost connection to MySQL server at 'reading authorization packet', system error: 2
I can connect directly to the nginx controller without issues from within the cluster:
kubectl run -it --rm --image=mysql:5.7 --restart=Never mysql-client -- mysql -h eating-dingo-nginx-ingress-controller -uroot -pabcdef
I'm using this mysql helm chart:
https://github.com/helm/charts/tree/master/stable/mysql
Related
I'm following theses instructions (page 181) to create a persistent volume & claim, a mysql replica set & service. I specify mysql v5.6 in the yaml file for the replica set.
After viewing the log for the pod, it looks like it is successful. So then I
kubectl run -it --rm --image=mysql --restart=Never mysql-client -- bash
mysql -h mysql -p 3306 -u root
It prompts me for the password and then I get this error:
ERROR 1130 (HY000): Host '10.1.0.17' is not allowed to connect to this MySQL server
Apparently MySQL has a feature where it does not allow remote connections by default and I have to change the configuration files and I don't know how to do that inside a yaml file. Below is my YAML. How do I change it to allow remote connections?
Thanks
Siegfried
cat <<END-OF-FILE | kubectl apply -f -
apiVersion: apps/v1
kind: ReplicaSet
metadata:
name: mysql
# labels so that we can bind a Service to this Pod
labels:
app: mysql
spec:
replicas: 1
selector:
matchLabels:
app: mysql
template:
metadata:
labels:
app: mysql
spec:
containers:
- name: tododata
image: mysql:5.6
resources:
requests:
cpu: 1
memory: 2Gi
env:
# Environment variables are not a best practice for security,
# but we're using them here for brevity in the example.
# See Chapter 11 for better options.
- name: MYSQL_ROOT_PASSWORD
value: some-password-here
livenessProbe:
tcpSocket:
port: 3306
ports:
- containerPort: 3306
volumeMounts:
- name: tododata
# /var/lib/mysql is where MySQL stores its databases
mountPath: "/var/lib/mysql"
volumes:
- name: tododata
persistentVolumeClaim:
claimName: tododata
END-OF-FILE
Sat Oct 24 2020 3PM EDT Update: Try Bitnami MySQL
I like Ben's idea of using bitnami mysql because then I don't have to create my own custom docker image. However, when using bitnami and trying to connect to they mysql server I get
ERROR 2003 (HY000): Can't connect to MySQL server on 'my-release-mysql.default.svc.cluster.local' (111)
This happens after I successfully get a bash shell with this command:
kubectl run my-release-mysql-client --rm --tty -i --restart='Never' --image docker.io/bitnami/mysql:8.0.22-debian-10-r0 --namespace default --command -- bash
Then, as per the instructions, I do this and get the HY000 error above.
mysql -h my-release-mysql.default.svc.cluster.local -uroot -p
Wed Nov 04 2020 Update:
Thanks Ben.. Yes -- I had already tried that on Oct 24 (approx) and when I do a k describe pod I get mysqladmin: connect to server at 'localhost' failed error: 'Can't connect to local MySQL server through socket '/opt/bitnami/mysql/tmp/mysql.sock' (2)' Check that mysqld is running and that the socket: '/opt/bitnami/mysql/tmp/mysql.sock' exists!.
Of course, when I run the mysql client as described in the nicely generated instructions, the client cannot connect because mysqld has died.
This is after having deleted the pvcs and stss and doing helm delete my-release prior to reinstalling via helm.
Unfortunately, when I tried this the first time (a couple of weeks ago) I did not set the root password and used the default generated password and I think it is still trying to use that.
This did work on azure kubernetes after having created a fresh azure kubernetes cluster. How can I reset my kubernetes cluster in my docker for desktop windows? I tried google searching and no luck so far.
Thanks
Siegfried
After a lot of help from the bitnami folks, I learned that my spinning disks on my 4 year old notebook computer are kinda slow (now why this is a problem with Bitnami MySQL and not Bitnami PostreSQL is a mystery).
This works for me:
helm install my-mysql bitnami/mysql \
--set image.debug=true \
--set primary.persistence.enabled=false,secondary.persistence.enabled=false \
--set primary.readinessProbe.enabled=false,primary.livenessProbe.enabled=false \
--set secondary.readinessProbe.enabled=false,secondary.livenessProbe.enabled=false
This turns off the peristent volumes so the data is lost when the pod dies.
Yes this is useful for me for development purposes and no one should be using Docker For Desktop/Kubernetes for production anyway... I just need to populate a tiny database and test my queries and if I need to repopulate database every time I reboot, well, that is not a big problem.
So maybe I need to get a new notebook computer? The price of notebook computers with 4TB of spinning disk space has gone up in the last couple of years.... And I cannot find any SSD drives of that size so even if I purchased a new replacement with spinning disks I might have the same problem? Hmm....
Thanks everyone for your help!
Siegfried
This appears to work just fine for me on windows. Complete the following steps:
helm repo add bitnami https://charts.bitnami.com/bitnami
helm install my-release --set root.password=awesomePassword bitnami/mysql
This is all you need to run the mysql instance. It does not makes a few services and a statefulset. Then, to connect to it, you
Either have to be in another another kubernetes container. Without this, you will not find the dns record for my-release-mysql.default.svc.cluster.local
run my-release-mysql-client --rm --tty -i --image docker.io/bitnami/mysql:8.0.22-debian-10-r0 --namespace default --command -- bash
mysql -h my-release-mysql.default.svc.cluster.local -uroot -p my_database
For the password, it should be 'awesomePassword'
Port forward the service to your local machine.
kubectl port-forward svc/my-release-mysql 3306:3306
As a note, a bitnami container will have issues if you kill it and restart it with only your helm commands and the password is not set. The persistent volume claim will usually stick around - so you would need to set the password to the old password. If you do not specify the password, you can get the password by running the commands bitnami tells you about.
NAME: my-release
LAST DEPLOYED: Thu Oct 29 20:39:23 2020
NAMESPACE: default
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES: Please be patient while the chart is being deployed
Tip:
Watch the deployment status using the command: kubectl get pods -w
--namespace default
Services:
echo Master: my-release-mysql.default.svc.cluster.local:3306 echo
Slave: my-release-mysql-slave.default.svc.cluster.local:3306
Administrator credentials:
echo Username: root echo Password : $(kubectl get secret
--namespace default my-release-mysql -o jsonpath="{.data.mysql-root-password}" | base64 --decode)
To connect to your database:
Run a pod that you can use as a client:
kubectl run my-release-mysql-client --rm --tty -i --restart='Never' --image docker.io/bitnami/mysql:8.0.22-debian-10-r0 --namespace default --command -- bash
To connect to master service (read/write):
mysql -h my-release-mysql.default.svc.cluster.local -uroot -p my_database
To connect to slave service (read-only):
mysql -h my-release-mysql-slave.default.svc.cluster.local -uroot -p my_database
To upgrade this helm chart:
Obtain the password as described on the 'Administrator credentials' section and set the 'root.password' parameter as shown
below:
ROOT_PASSWORD=$(kubectl get secret --namespace default my-release-mysql -o jsonpath="{.data.mysql-root-password}" | base64
--decode)
helm upgrade my-release bitnami/mysql --set root.password=$ROOT_PASSWORD
I'm learning how to work with Docker and Minikube on a Windows 10 Home computer. I've installed the needed software OK. I've installed Docker, minikube, kubectl, and a recent version of MySQL, properly pathed so its CLI can be used. I'm using either the Bash console provided by GIT, and/or the Bash console provided by Cygwin. Both seem to provide the same (bad) results.
I start Docker, and install the MySQL service. The kubectl get all shows everything running OK.
Per the programming book I'm working through, I want to try accessing MySQL through this command:
mysql -h $(minikube service mysql-svc --format "{{.IP}}") -P $(minikube service mysql-svc --format "{{.Port}}") -u root -p
The result should be the MySQL CLI prompt, like mysql> . Instead I get this behavior:
A popup window stating "Windows cannot find '192.168.99.101'. Make sure you typed the name correctly, and then try again."
The console text:
The system cannot find the file 192.168.99.101.
*
X open url failed: 192.168.99.101: exit status 1
*
* minikube is exiting due to an error. If the above message is not useful, open an issue:
- (URL for sending an error message)
A popup window stating "Windows cannot find '31067'. Make sure you typed the name correctly, and then try again."
The console text:
mysql: [ERROR] Unknown suffix '|' used for variable 'port' (value (gives the border fence that surrounds the output from minikube service mysql-svc .)
mysql: [ERROR] (path to mysql.exe): Error while setting value '|-----|--- (etc)' to 'port'
The expected behavior is to insert an IP and Port into the mysql command line, then firing a command like mysql -h http://192.168.99.101 -P 31067 -u root -p .
I think that the problem is with the using the Bash console in a Windows environment. Any explanation is appreciated.
Thanks,
Jerome.
UPDATE ON 8/7/2020:
I'm asked to more thoroughly document my issue. Here we go.
Here is what Docker knows:
$ docker images
REPOSITORY TAG
IMAGE ID CREATED SIZE
logicaltiger/cloudnative-statelessness-posts latest
3a3c66daf7f3 5 days ago 139MB
logicaltiger/cloudnative-statelessness-connections latest
d060e9857f49 5 days ago 139MB
logicaltiger/cloudnative-statelessness-connectionposts-stateful latest
ce33f0966380 5 days ago 123MB
openjdk 8-jdk-alpine
a3562aa0b991 15 months ago 105MB
mysql 8.0.12
ee1e8adfcefb 22 months ago 484MB
Here is my reconfiguring of minikube. Other posters suggested that minikube runs iffy unless given a lot of resources.
$ minikube delete
* Deleting "minikube" in virtualbox ...
* Removed all traces of the "minikube" cluster.
$ minikube start --cpus=4 --memory=4096
* minikube v1.12.1 on Microsoft Windows 10 Home 10.0.18363 Build 18363
* Automatically selected the virtualbox driver
* Starting control plane node minikube in cluster minikube
* Creating virtualbox VM (CPUs=4, Memory=4096MB, Disk=20000MB) ...
* Found network options:
- NO_PROXY=192.168.99.100
- no_proxy=192.168.99.100
* Preparing Kubernetes v1.18.3 on Docker 19.03.12 ...
- env NO_PROXY=192.168.99.100
* Verifying Kubernetes components...
* Enabled addons: default-storageclass, storage-provisioner
* Done! kubectl is now configured to use "minikube"
After starting mysql from its yaml file I have it running:
$ kubectl get all
NAME READY STATUS RESTARTS AGE
pod/mysql-7dbfd4dbc4-b2tmm 1/1 Running 0 2m55s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) A
GE
service/kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 6
m43s
service/mysql-svc NodePort 10.102.7.119 <none> 3306:32235/TCP 2
m55s
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/mysql 1/1 1 1 2m55s
NAME DESIRED CURRENT READY AGE
replicaset.apps/mysql-7dbfd4dbc4 1 1 1 2m55s
Now get the mysql-svc URL:
$ minikube service mysql-svc --url
http://192.168.99.102:32235
Try to run the book example. Again I get the two popup windows and what is shown below in the terminal. I omit the popup window text here...
$ mysql -h $(minikube service mysql-svc --format "{{.IP}}") -P $(minikube service mysql-svc --format "{{.Port}}") -u root -p
The system cannot find the file 192.168.99.102.
*
X open url failed: 192.168.99.102: exit status 1
*
* minikube is exiting due to an error. If the above message is not useful, open
an issue:
- https://github.com/kubernetes/minikube/issues/new/choose
The system cannot find the file 32235.
*
X open url failed: 32235: exit status 1
*
* minikube is exiting due to an error. If the above message is not useful, open
an issue:
- https://github.com/kubernetes/minikube/issues/new/choose
mysql: [ERROR] Unknown suffix '|' used for variable 'port' (value '|-----------|
-----------|-------------|-------|')
mysql: [ERROR] C:\Program Files\MySQL\MySQL Server 8.0\bin\mysql.exe: Error whil
e setting value '|-----------|-----------|-------------|-------|' to 'port'
I'm asked what happens if I put in the IP directly. From above, that IP was http://192.168.99.102:32235
$ mysql -h http://192.168.99.102 -P 32235 -u root -p
Enter password: **********
ERROR 2005 (HY000): Unknown MySQL server host 'http://192.168.99.102' (0)
When directly entering the IP and Port, the MySQL server IS reached (see the "Enter password:" prompt) but the request is refused. I'm thinking that I don't know how to make MySQL use the HTTP request.
But is the MySQL not knowing what to do with the IP, and the way that the indirect method (minikube service mysql -svc ...), related?
Continuing, I edit my cookbook-deployment-posts.yaml file with the MySQL address:
kind: Service
apiVersion: v1
metadata:
name: posts-svc
spec:
selector:
app: posts
ports:
- protocol: "TCP"
port: 80
targetPort: 8080
type: NodePort
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: posts
labels:
app: posts
spec:
replicas: 1
selector:
matchLabels:
app: posts
template:
metadata:
labels:
app: posts
spec:
containers:
- name: posts
image: cdavisafc/cloudnative-statelessness-posts
env:
- name: INSTANCE_IP
valueFrom:
fieldRef:
fieldPath: status.podIP
- name: PORT
value: "8080"
- name: SPRING_APPLICATION_JSON
value: '{"spring":{"datasource":{"url":"jdbc:mysql://192.168.99.102:32235/cookbook"}}}'
Going to kubectl get all, the posts-svc continually starts, errors out and reboots. Don't know what is wrong...
Jerome.
I now see a number of things going wrong.
First, I keep thinking that the mysql call is somehow related to the MySQL installed on my PC. It never is. Just because I manually create a cookbook database on my PC instance doesn't mean that the textbook example thru Docker / Minikube ever references it.
Second, the textbook is missing the --url from its minikube requests. Here is what is happening.
> kubectl create -f mysql-deployment.yaml
> minikube service mysql-svc --url
http://192.168.99.102:31030
> minikube service mysql-svc
This opens the web browser to show the service at 192.168.99.102:31030. The mysql service doesn't render a web page, but that doesn't matter to this example. The console then shows the details of the service (namespace, name, target port, url) in an ASCII box.
minikube service mysql-svc --format "{{.IP}}"
This wants to open the web browser to show the service at http://192.168.99.102, with an implied port of 80. But there is nothing there, and Windows complains at a popup box. The console then complains about not opening that url.
minikube service mysql-svc --format "{{.Port}}"
This wants to open the web browser to show the service at 31030, which isn't a valid URL. Complaints, complaints.
What I really wanted all along is to add the --url to the minikube bits:
mysql -h $(minikube service mysql-svc --format "{{.IP}}" --url) -P $(minikube service mysql-svc --format "{{.Port}}" --url) -u root -p
This connects to the managed mysql in the console, yielding the mysql> prompt. Now I can run 'create database cookbook;'.
Solved!
I've a docker installed in a VM in VirtualBox and I'm attempting to run a container with a dot Net Core application that connects to a MySQL database on the hosts machine. So I've configured the forwarding port for both mysql and my application on Virtual Box. I'm able to access my service through "http://localhost:3131/api/users/login" in the host machine but it throws an error saying that couldn't connect with the MySQL data base. I'm also able to run the app in the host machine when I'm not using docker. I've looked in other threads on the internet but nothing that enlightened me exactly except the last command shown below but I can't run since the MySQL authentication are configured is hard coded in the application not with a config file. The general configuration is as follows:
Program.cs
var host = new WebHostBuilder()
.UseKestrel()
.UseContentRoot(Directory.GetCurrentDirectory())
.UseIISIntegration()
.UseStartup<Startup>()
.UseApplicationInsights()
.UseUrls("http://*:80")
.Build();
Dockerfile
FROM microsoft/aspnetcore
WORKDIR /app
COPY bin/Release/PublishOutput/ .
EXPOSE 80
ENTRYPOINT ["dotnet", "UsersApi.dll"]
Docker Run Command
docker run -d -p 3000:80 user_api
// and also tried
docker run -d -p 3000:80 user_api --net=host
// and also tried
docker run -d -p 3000:80 user_api --add-host localhost:127.0.0.1
VirtualBox fowarding ports:
NAT 3131 -> 3000 tcp
NAT 3306 -> 3306 tcp
NAT 2415 -> 22
localhost (that I thought it would appear the port 3131, but it calls the service anyway.)
Starting Nmap 7.40 ( https://nmap.org ) at 2017-05-22 11:23 E. South America Standard Time
Nmap scan report for localhost (127.0.0.1)
Host is up (0.0013s latency).
Other addresses for localhost (not scanned): ::1
rDNS record for 127.0.0.1: rinaldipc.com
Not shown: 994 closed ports
PORT STATE SERVICE
22/tcp open ssh
135/tcp open msrpc
445/tcp open microsoft-ds
2179/tcp open vmrdp
3306/tcp open mysql
5357/tcp open wsdapi
RUN command in Dockfile that I think I need to add but I'm not sure of the proceedings.
RUN sed -i -e"s/^bind-address\s*=\s*127.0.0.1/bind-address = 0.0.0.0/" /etc/mysql/my.cnf
https://stackoverflow.com/questions/33827342/how-to-connect-mysql-workbench-to-running-mysql-inside-docker/33827463#33827463
Since you're running inside VirtualBox, there's another layer between the VirtualBox host machine and docker. You have a machine hosting VirtualBox (1) -> Linux in VirtualBox (2) -> docker (3).
"localhost" for docker (3) means (2) so it expects mysql to be on (2). In your case, you have mysql on (1).
The only way to access (1) from (3) is by explicitly using the IP of (1) and not the "localhost" alias.
I have Wildfly 10 running on Openshift Origin 3 in AWS with an elastic ip.
I setup a Route in Openshift to map / to the wildfly service. This is working fine. If I go to http://my.ip.address I get the WildFly welcome page.
But if I map a different path, say /wf01, it doesn't work. I get a 404 Not Found error.
My guess is the router is passing along the /wf01 to the service? If that's the case, can I stop it from doing it? Otherwise how can I map http://my.ip.address/wf01 to my wildfly service?
I also want the wildfly console to be accessible from outside (this is a demo server for my own use). I added "-bmanagement","0.0.0.0" to the deploymentconfig but looking at the wildfly logs it is still binding to 127.0.0.1:
02:55:41,483 INFO [org.jboss.as] (Controller Boot Thread) WFLYSRV0051:
Admin console listening on http://127.0.0.1:9990
A router today cannot remap/rewrite the incoming HTTP path to another path value before passing it along. A workaround is to mount another route+service at the root that handles the root and redirects / forwards.
You can also use port-forward :
oc port-forward -h
Forward 1 or more local ports to a pod
Usage:
oc port-forward POD [LOCAL_PORT:]REMOTE_PORT [...[LOCAL_PORT_N:]REMOTE_PORT_N] [options]
Examples:
# Listens on ports 5000 and 6000 locally, forwarding data to/from ports 5000 and 6000 in the pod
$ oc port-forward -p mypod 5000 6000
# Listens on port 8888 locally, forwarding to 5000 in the pod
$ oc port-forward -p mypod 8888:5000
# Listens on a random port locally, forwarding to 5000 in the pod
$ oc port-forward -p mypod :5000
# Listens on a random port locally, forwarding to 5000 in the pod
$ oc port-forward -p mypod 0:5000
On my AWS ec2 server I have docker 1.9.1 installed.
In an image test_image based from ubuntu:trusty official docker image, I have tried to setup the LEMP(Linux, Nginx, MySQL, PHP) architecture.
Following is the docker command i have used to start my container:
docker run --name test_1 -d -p 80:80 -p 3306:3306 test_image /bin/sh -c "while true; do echo daemonized docker container; sleep 5000; done"
I have exposed port 80 and 3306 to the host's network interface and have also allowed AWS's security group to allow inbound connections to these ports. Connection type in security group is: MYSQL/Aurora and protocol is: TCP (I know its not very secure, its only for initial implementation. Production setup will be different)
I followed this DigitalOcean tutorial: https://www.digitalocean.com/community/tutorials/how-to-install-linux-nginx-mysql-php-lemp-stack-on-ubuntu-14-04
After installing Nginx and starting it I am able to test it in the browser via ec2's pubic ip i.e. http://xxx.xxx.xxx.xxx shows the default nginx welcome page.
While installing MySQL, I followed the following commands in the docker container:
apt-get install mysql-server
mysql_install_db
/etc/init.d/mysql start
mysql_secure_installation
I have given a password to my root user and during mysql_secure_installation i had allowed remote access to root user.
mysql -u root -p command from inside the container connects me to the mysql db but not from outside the container.
Also from my local machine:
I tried with mysql-client:
mysql -h xxx.xxx.xxx.xxx -u root -p
I got the following error: ERROR 2003 (HY000): Can't connect to MySQL server on 'xxx.xxx.xxx.xxx' (111)
and also through mysql workbench but I still can't connect to the mysql db.
What am I doing wrong?
In your host mysql's my.cnf set the bind address to 0.0.0.0 so that mysql listens on all network interfaces
bind-address = 0.0.0.0
The default config is:
# Instead of skip-networking the default is now to listen only on
# localhost which is more compatible and is not less secure.
#bind-address = 127.0.0.1