I configured Airflow metadata db as MySQL.
sql_alchemy_conn = sqla+mysql://airflow:airflow#localhost:3306/airflow
After that I used initdb command:
airflow initdb
Tables in the airflow db have been created.
Then I runed command:
airflow shceduler
airflow webserver -p 8080
in logs:
[2020-01-14 18:39:03,457: INFO/MainProcess] Connected to sqla+mysql://airflow:airflow#localhost:3306/airflow
password as plain text.
How can I turn off that? Is sql_alchemy log? It's password leak to logs.
Related
I'm running a mysql 8 server on a custom port using docker and try to connect to it with command line client using the below command
`mysql -uroot -p -P 3305 --protocol=TCP -h localhost`
Error Response
ERROR 2003 (HY000): Can't connect to MySQL server on 'localhost' (61)
Docker Compose File
version: '3'
services:
mysqldb:
image: mysql:8
ports:
- 3305:3306
environment:
- MYSQL_ROOT_PASSWORD=MyRootPass
- MYSQL_USER=myuser
- MYSQL_PASSWORD=myuserpass
- MYSQL_DATABASE=mydb
volumes:
- ../lcdatastore/mysql/data:/var/lib/mysql
But i'm able to connect to the mysql if the mysql is run using docker run command
docker run -e MYSQL_ROOT_PASSWORD=MyRootPass -e MYSQL_USER=myuser -e MYSQL_PASSWORD=myuserpass -e MYSQL_DATABASE=mydb -p 3305:3306 mysql:8
Thanks for any hint
Update
I'm checking this on macOS Catalina (Version 10.15.2)
I see two possible issues here. The first one is not the case for you in particular, I'm just leaving this here for anyone landing here in the future with that problem:
From the mysql docs:
If the host is not specified or is localhost, a connection to the local host occurs:
On Unix, MySQL programs treat the host name localhost specially, in a way that is likely different from what you expect compared to other network-based programs: the client connects using a Unix socket file. The --socket option or the MYSQL_UNIX_PORT environment variable may be used to specify the socket name.
I.e. when using localhost as the host mysql tries to connect via a unix socket and not via network. The former won't work on your macOS host since only the latter will pass the connection to the container.
You can force the connection via network by using 127.0.0.1 as the host or by passing the --protocol=TCP parameter (which you did in your question):
mysql -uroot -p -P 3305 -h 127.0.0.1
mysql -uroot -p -P 3305 --protocol=TCP -h localhost
The second issue may be that the port is not correctly forwarded from the macOS host to the docker host:
Since docker uses Linux namespaces for its containers it does not actually work on macOS natively. What it does instead is to transparently start a Linux VM in the background - which is the actual docker host - and forwards all docker commands to that VM. So the containers are not actually running on macOS but inside a Linux VM.
So when a container exposes a port to the "host" this refers to Linux VM and not the macOS host. From the perspective of the mysql run on macOS localhostrefers to the macOS host and not the docker host (i.e. the Linux VM).
Normally docker will set up respective port forwardings from the macOS host to the Linux VM automatically to make this work as you expect it. But this seems to be broken in your case. To further debug this, first try to connect to mysql on the Linux VM directly:
# start a new container attached to the host network (i.e. the network of the Linux VM)
# "127.0.0.1" will force a network connection
# and "3305" therfor refers to the localhost on the docker host
docker run --network=host -ti mysql:8 mysql -u root -p -P 3305 -h 127.0.0.1
If this succeeds the docker networking is basically working correctly (inside the Linux VM) and there is an issue with forwarding ports from macOS to the VM.
Now:
check if port 3305 on macOS is accepting connections, e.g. with netcat, with and without the mysqldb service running
check which process is listening on 3305, e.g. with netstat or see Who is listening on a given TCP port on Mac OS X?
I'm following theses instructions (page 181) to create a persistent volume & claim, a mysql replica set & service. I specify mysql v5.6 in the yaml file for the replica set.
After viewing the log for the pod, it looks like it is successful. So then I
kubectl run -it --rm --image=mysql --restart=Never mysql-client -- bash
mysql -h mysql -p 3306 -u root
It prompts me for the password and then I get this error:
ERROR 1130 (HY000): Host '10.1.0.17' is not allowed to connect to this MySQL server
Apparently MySQL has a feature where it does not allow remote connections by default and I have to change the configuration files and I don't know how to do that inside a yaml file. Below is my YAML. How do I change it to allow remote connections?
Thanks
Siegfried
cat <<END-OF-FILE | kubectl apply -f -
apiVersion: apps/v1
kind: ReplicaSet
metadata:
name: mysql
# labels so that we can bind a Service to this Pod
labels:
app: mysql
spec:
replicas: 1
selector:
matchLabels:
app: mysql
template:
metadata:
labels:
app: mysql
spec:
containers:
- name: tododata
image: mysql:5.6
resources:
requests:
cpu: 1
memory: 2Gi
env:
# Environment variables are not a best practice for security,
# but we're using them here for brevity in the example.
# See Chapter 11 for better options.
- name: MYSQL_ROOT_PASSWORD
value: some-password-here
livenessProbe:
tcpSocket:
port: 3306
ports:
- containerPort: 3306
volumeMounts:
- name: tododata
# /var/lib/mysql is where MySQL stores its databases
mountPath: "/var/lib/mysql"
volumes:
- name: tododata
persistentVolumeClaim:
claimName: tododata
END-OF-FILE
Sat Oct 24 2020 3PM EDT Update: Try Bitnami MySQL
I like Ben's idea of using bitnami mysql because then I don't have to create my own custom docker image. However, when using bitnami and trying to connect to they mysql server I get
ERROR 2003 (HY000): Can't connect to MySQL server on 'my-release-mysql.default.svc.cluster.local' (111)
This happens after I successfully get a bash shell with this command:
kubectl run my-release-mysql-client --rm --tty -i --restart='Never' --image docker.io/bitnami/mysql:8.0.22-debian-10-r0 --namespace default --command -- bash
Then, as per the instructions, I do this and get the HY000 error above.
mysql -h my-release-mysql.default.svc.cluster.local -uroot -p
Wed Nov 04 2020 Update:
Thanks Ben.. Yes -- I had already tried that on Oct 24 (approx) and when I do a k describe pod I get mysqladmin: connect to server at 'localhost' failed error: 'Can't connect to local MySQL server through socket '/opt/bitnami/mysql/tmp/mysql.sock' (2)' Check that mysqld is running and that the socket: '/opt/bitnami/mysql/tmp/mysql.sock' exists!.
Of course, when I run the mysql client as described in the nicely generated instructions, the client cannot connect because mysqld has died.
This is after having deleted the pvcs and stss and doing helm delete my-release prior to reinstalling via helm.
Unfortunately, when I tried this the first time (a couple of weeks ago) I did not set the root password and used the default generated password and I think it is still trying to use that.
This did work on azure kubernetes after having created a fresh azure kubernetes cluster. How can I reset my kubernetes cluster in my docker for desktop windows? I tried google searching and no luck so far.
Thanks
Siegfried
After a lot of help from the bitnami folks, I learned that my spinning disks on my 4 year old notebook computer are kinda slow (now why this is a problem with Bitnami MySQL and not Bitnami PostreSQL is a mystery).
This works for me:
helm install my-mysql bitnami/mysql \
--set image.debug=true \
--set primary.persistence.enabled=false,secondary.persistence.enabled=false \
--set primary.readinessProbe.enabled=false,primary.livenessProbe.enabled=false \
--set secondary.readinessProbe.enabled=false,secondary.livenessProbe.enabled=false
This turns off the peristent volumes so the data is lost when the pod dies.
Yes this is useful for me for development purposes and no one should be using Docker For Desktop/Kubernetes for production anyway... I just need to populate a tiny database and test my queries and if I need to repopulate database every time I reboot, well, that is not a big problem.
So maybe I need to get a new notebook computer? The price of notebook computers with 4TB of spinning disk space has gone up in the last couple of years.... And I cannot find any SSD drives of that size so even if I purchased a new replacement with spinning disks I might have the same problem? Hmm....
Thanks everyone for your help!
Siegfried
This appears to work just fine for me on windows. Complete the following steps:
helm repo add bitnami https://charts.bitnami.com/bitnami
helm install my-release --set root.password=awesomePassword bitnami/mysql
This is all you need to run the mysql instance. It does not makes a few services and a statefulset. Then, to connect to it, you
Either have to be in another another kubernetes container. Without this, you will not find the dns record for my-release-mysql.default.svc.cluster.local
run my-release-mysql-client --rm --tty -i --image docker.io/bitnami/mysql:8.0.22-debian-10-r0 --namespace default --command -- bash
mysql -h my-release-mysql.default.svc.cluster.local -uroot -p my_database
For the password, it should be 'awesomePassword'
Port forward the service to your local machine.
kubectl port-forward svc/my-release-mysql 3306:3306
As a note, a bitnami container will have issues if you kill it and restart it with only your helm commands and the password is not set. The persistent volume claim will usually stick around - so you would need to set the password to the old password. If you do not specify the password, you can get the password by running the commands bitnami tells you about.
NAME: my-release
LAST DEPLOYED: Thu Oct 29 20:39:23 2020
NAMESPACE: default
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES: Please be patient while the chart is being deployed
Tip:
Watch the deployment status using the command: kubectl get pods -w
--namespace default
Services:
echo Master: my-release-mysql.default.svc.cluster.local:3306 echo
Slave: my-release-mysql-slave.default.svc.cluster.local:3306
Administrator credentials:
echo Username: root echo Password : $(kubectl get secret
--namespace default my-release-mysql -o jsonpath="{.data.mysql-root-password}" | base64 --decode)
To connect to your database:
Run a pod that you can use as a client:
kubectl run my-release-mysql-client --rm --tty -i --restart='Never' --image docker.io/bitnami/mysql:8.0.22-debian-10-r0 --namespace default --command -- bash
To connect to master service (read/write):
mysql -h my-release-mysql.default.svc.cluster.local -uroot -p my_database
To connect to slave service (read-only):
mysql -h my-release-mysql-slave.default.svc.cluster.local -uroot -p my_database
To upgrade this helm chart:
Obtain the password as described on the 'Administrator credentials' section and set the 'root.password' parameter as shown
below:
ROOT_PASSWORD=$(kubectl get secret --namespace default my-release-mysql -o jsonpath="{.data.mysql-root-password}" | base64
--decode)
helm upgrade my-release bitnami/mysql --set root.password=$ROOT_PASSWORD
So im trying to import a MySQL dump that contains two databases with Ansible on a Windows box.
The Ansible playbook should be correct.
- hosts: win
gather_facts: no
tasks:
- name: Import DB
win_command: 'mysql -uroot -p12345678910 -hlocalhost < "C:\sql\dump.sql"'
args:
chdir: C:\Program Files\MySQL\MySQL Server 5.6\bin\
On the windows box if I open CMD and run
cd C:\Program Files\MySQL\MySQL Server 5.6\bin\ and then 'mysql -uroot -p12345678910 -hlocalhost < "C:\sql\dump.sql"' it works perfectly.
Im getting a giant error. What am I doing wrong?
Im able to check the DB with a very similar Playbook:
- hosts: win
gather_facts: no
tasks:
- name: Check DB
win_command: 'mysql -uroot -p12345678910 -e "SHOW DATABASES;"'
args:
chdir: C:\Program Files\MySQL\MySQL Server 5.6\bin\
The problem might be input redirection. Perhaps you can try something like:
win_command: 'Get-Content C:\sql\dump.sql | mysql -uroot -p12345678910 -hlocalhost'
See Redirecting standard input\output in Windows PowerShell
I'm using docker for mac to run laradock and everything is alright, but the phpmyadmin can't not login by message
#2002 - No such file or directory — The server is not responding (or the local server's socket is not correctly configured).
Here is my .env file setting, and docker-compose.yml not change anythings
### PHP MY ADMIN ###
PMA_DB_ENGINE=mysql
PMA_USER=root
PMA_PASSWORD=1234
PMA_ROOT_PASSWORD=1234
PMA_PORT=8080
### MYSQL ##########
MYSQL_VERSION=8.0
MYSQL_DATABASE=test
MYSQL_USER=root
MYSQL_PASSWORD=1234
MYSQL_PORT=3306
MYSQL_ROOT_PASSWORD=1234
MYSQL_ENTRYPOINT_INITDB=./mysql/docker-entrypoint-initdb.d
and $ docker-compose ps
Name Command State Ports
------------------------------------------------------------------------------------------------------------
laradock_apache2_1 /opt/docker/bin/entrypoint ... Up 0.0.0.0:443->443/tcp, 0.0.0.0:80->80/tcp
laradock_applications_1 /true Exit 0
laradock_mysql_1 docker-entrypoint.sh mysqld Up 0.0.0.0:3306->3306/tcp
laradock_php-fpm_1 docker-php-entrypoint php-fpm Up 9000/tcp
laradock_phpmyadmin_1 /run.sh phpmyadmin Up 0.0.0.0:8080->80/tcp
laradock_workspace_1 /sbin/my_init Up 0.0.0.0:2222->22/tcp
Is that wrong with my login server position? or any idea?
Maybe phpmyadmin is trying to connect 3306 inside the phpmyadmin container?
If the image for phpmyadmin is the official one you need to link with host "db"
--link laradock_mysql_1:db
or stablish the host in env variable:
PMA_HOST
On my AWS ec2 server I have docker 1.9.1 installed.
In an image test_image based from ubuntu:trusty official docker image, I have tried to setup the LEMP(Linux, Nginx, MySQL, PHP) architecture.
Following is the docker command i have used to start my container:
docker run --name test_1 -d -p 80:80 -p 3306:3306 test_image /bin/sh -c "while true; do echo daemonized docker container; sleep 5000; done"
I have exposed port 80 and 3306 to the host's network interface and have also allowed AWS's security group to allow inbound connections to these ports. Connection type in security group is: MYSQL/Aurora and protocol is: TCP (I know its not very secure, its only for initial implementation. Production setup will be different)
I followed this DigitalOcean tutorial: https://www.digitalocean.com/community/tutorials/how-to-install-linux-nginx-mysql-php-lemp-stack-on-ubuntu-14-04
After installing Nginx and starting it I am able to test it in the browser via ec2's pubic ip i.e. http://xxx.xxx.xxx.xxx shows the default nginx welcome page.
While installing MySQL, I followed the following commands in the docker container:
apt-get install mysql-server
mysql_install_db
/etc/init.d/mysql start
mysql_secure_installation
I have given a password to my root user and during mysql_secure_installation i had allowed remote access to root user.
mysql -u root -p command from inside the container connects me to the mysql db but not from outside the container.
Also from my local machine:
I tried with mysql-client:
mysql -h xxx.xxx.xxx.xxx -u root -p
I got the following error: ERROR 2003 (HY000): Can't connect to MySQL server on 'xxx.xxx.xxx.xxx' (111)
and also through mysql workbench but I still can't connect to the mysql db.
What am I doing wrong?
In your host mysql's my.cnf set the bind address to 0.0.0.0 so that mysql listens on all network interfaces
bind-address = 0.0.0.0
The default config is:
# Instead of skip-networking the default is now to listen only on
# localhost which is more compatible and is not less secure.
#bind-address = 127.0.0.1