Django - Mysql Database is not created in Docker - mysql

I setup a django project in docker container and every thing is working as expected, except I don't find the project database in mysql image.
Dockerfile
FROM python:3
RUN mkdir /django-website
WORKDIR /django-website
COPY . /django-website
RUN pip install -r requirements.txt
docker-compose.yml
version: '3'
services:
db:
image: mysql:5.7
restart: always
environment:
- MYSQL_ROOT_PASSWORD=root
- MYSQL_DATABASE=mywebsite
- MYSQL_USER=root
- MYSQL_PASSWORD=root
ports:
- '33060:3306'
volumes:
- /var/lib/mysql
web:
build: .
command: python manage.py runserver 0.0.0.0:8000
volumes:
- .:/django-website
ports:
- '8000:8000'
links:
- db
settings.py
DATABASES = {
'default': {
'ENGINE': "django.db.backends.mysql",
'NAME': "mywebsite",
'USER': "root",
'PASSWORD': "root",
'HOST': 'db',
'PORT': '3306',
}
}
I ran migrate and it worked:
docker-compose run web python manage.py migrate
I createdsuperuser:
docker-compose run web python manage.py createsuperuser
The development server is working docker-compose up and the site is working as expected, the issue when I navigate in mysql image I don't find my project related database which is mywebsite .
can you please tell me what is missing? if the database is not created, where has the migration been applied?
Thanks in advance.

I'm not sure what you mean by "I logged in mysql image shell but didn't find mywebsite database"
You are migrated the DB successfully, which means, the DB connections are valid and working.
In your docker-compose.yml file, the port mapping done like this, '33060:3306', which means the db's port 3306 is mapped to host machine's port 33060. So, this may be the issue (it's not an issue, kind of typo)
How to check the DB contents?
METHOD-1: check through django-shell of web container
1. run docker-compose up
2. open a new terminal in the same path and run docker ps
you'll get something like below
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
795093357f78 django_1_11_web "python manage.py ru…" 34 minutes ago Up 11 minutes 0.0.0.0:8000->8000/tcp django_1_11_web_1
4ae48f291e34 mysql:5.7 "docker-entrypoint.s…" 34 minutes ago Up 12 minutes 0.0.0.0:33060->3306/tcp django_1_11_db_1
3.Get into the web container by docker exec -it 795093357f78 bash command, where 795093357f78 is the respective container id
4. now you're inside the container. Then, run the command python manage.py dbshell. Now you will be in MYSQL shell of mywebsite (Screenshot)
5. run the command show tables;. It will display all the tables inside the mywebsite DB
METHOD-2: check through db container
1. repeat the steps 1 and 2 in above section
2. get into db container by docker exec -it 4ae48f291e34 bash
3. Now you'll be in bash terminal of MYSQL. Run the following commmand mysql -u root -p and enter the password when prompt
4. now you're in MYSQL server. run the command, show databases;. This will show all the databases in the server.

Have you tried defining the database image in the dockerfile? The following link is somewhat related to your problem:
https://medium.com/#lvthillo/customize-your-mysql-database-in-docker-723ffd59d8fb

I supposed that ports value of host container should be 3306 not 33060.
Use docker-compose.yml with value 3306 :
version: '3'
services:
db:
image: mysql:5.7
restart: always
environment:
- MYSQL_ROOT_PASSWORD=root
- MYSQL_DATABASE=mywebsite
- MYSQL_USER=root
- MYSQL_PASSWORD=root
ports:
- '3306:3306'
volumes:
- /var/lib/mysql
web:
build: .
command: python manage.py runserver 0.0.0.0:8000
volumes:
- .:/django-website
ports:
- '8000:8000'
links:
- db
Hope this works!

You should change the compose specification to version '2'. Take down the container and bring it back up with docker-compose up -d. Or if you intend to stay with version 3, you can instead use the following specification for database environment parameters
```
environment:
MYSQL_ROOT_PASSWORD: root
MYSQL_DATABASE: mywebsite
MYSQL_USER: root
MYSQL_PASSWORD: root
```
When you have problems with containers not coming up, docker logs <container-name> --tail 25 -f can give you a lot of information about the cause.

Related

How can I use local mysql data in a mysql container created through docker-compose

My company has been working with local setups of our mysql database for years. We have recently decided to adopting a containerized approach to local development, and we want to add the database into being run in a container. The issue is, because all of our data is already set up locally, we want to be able to just use the same data in the mysql container. I have tried using volumes to mount the directory storing all the mysql data into the container to no avail. Has anyone had success with doing this?
db part of docker-compose.yml:
db:
image: mysql:5.6
container_name: mysql
ports:
- "3306:3306"
environment:
MYSQL_ALLOW_EMPTY_PASSWORD: 1
volumes:
- /usr/local/mysql/data:/var/lib/mysql
I am able to get mysql running fine and am able to connect to it easily from my local machine, but when I connect, none of the local databases that already exist are there. Is there something that I'm overlooking?
#yourknightmares,
So I just ran a test and it worked for me. Here is what I did:
docker-compose.yml
version: "3.9"
services:
nginx:
image: nginx:latest
ports:
- "9999:9999"
command: tail -f /dev/null
volumes:
- "/etc/nginx/nginx.conf:/opt/nginx/nginx.conf"
In my host machine, I have the file at /etc/nginx/nginx.conf
, then:
$ docker-compose up -d
$ docker exec -it 02ba7032d699 bash
$ root#02ba7032d699:/# cat /opt/nginx/nginx.conf
#hello
The file was mounted just fine from the host to the container. I would suggest you to do the same exercise just for troubleshooting purposes. Also, have you looked at the container logs with docker logs container_id?

MySQL commands inside Docker container

I have the following docker compose:
version: '3.1'
services:
db:
image: mysql:5.5
container_name: mysql
environment:
- MYSQL_ROOT_PASSWORD=some_root_pw
- MYSQL_DATABASE=some_db
- MYSQL_USER=db_user
- MYSQL_PASSWORD=user_pw
ports:
- 8306:3306
volumes:
- ./db:/var/lib/mysql/some_db
I am accessing the container with the command:
docker-compose exec db /bin/sh
Which drops me at the shell as expected. However, I am unable to access mysql with the information from the docker-compose file.
mysql -u root -p
Prompts for the password, then rejects for bad username or password. I have tried with both root and user. Is there a way I can troubleshoot what is happening with user creation?
NOTE: intentionally older version of MySQL.

Symfony; SQLSTATE[HY000] [2002] No such file or directory while using 127.0.0.1 as database_host

The full error is Doctrine\DBAL\Exception\ConnectionException: An exception occurred in driver: SQLSTATE[HY000] [2002] No such file or directory in /app/vendor/doctrine/dbal/lib/Doctrine/DBAL/Driver/AbstractMySQLDriver.php on line 113, but that's too long for the title.
I'm trying to set up a Symfony project locally, but I'm struggling to get the database connection to work. My parameters.yml looks as follows
parameters:
database_host: 127.0.0.1
database_port: 3306
database_name: database_name
database_user: username
database_password: password
I've been googling this issue a lot and most people seem to solve the issue by changing database_host from localhost to 127.0.0.1, but this doesn't work for me. The app itself runs via Docker, but I've set up the database connection once via Brew and once with a MySQL server for Mac. In both cases I can connect via the command line and with SequelPro/TablePlus, but whenever I try to access the website through the browser I get the "No such file or directory" error.
I've also tried multiple ways of setting up a Docker MySQL container, but can't get it to work. My docker-compose.yml looks like this;
nginx:
build: nginx
ports:
- "8080:80"
links:
- php
volumes:
- ../:/app
php:
build: php-fpm
volumes:
- ../:/app
working_dir: /app
extra_hosts:
- "site.dev: 172.17.0.1"
services:
db:
image: mysql:5.7
restart: always
environment:
MYSQL_DATABASE: 'database_name'
MYSQL_USER: 'username'
MYSQL_PASSWORD: 'password'
MYSQL_ROOT_PASSWORD: 'password_root'
ports:
- '3306:3306'
expose:
- '3306'
volumes:
- my-db:/var/lib/mysql
But whenever I run docker-compose up -d I get the error Unsupported config option for services: 'db'.
Another attempt was adding
mysql:
image: mysql:latest
volumes:
- mysql_data:/var/lib/mysql
environment:
- MYSQL_ROOT_PASSWORD='password'
- MYSQL_DATABASE='database_name'
- MYSQL_USER='username'
- MYSQL_PASSWORD='password'
To the docker-compose file, and while it does build the mysql image, I can't seem to connect to it with SequelPro/TablePlus. I ran docker-inspect on the container to get the IP (172.17.0.3), but can't seem to get access to it. I can exec into it, login using mysql -u root and create the required user and database, but then I'm still struggling to actually connect to it.
Running docker ps does show the sql container running btw;
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
b6de6030791d docker_nginx "nginx -g 'daemon of…" 19 minutes ago Up 14 minutes 0.0.0.0:8080->80/tcp docker_nginx_1
f26b832bb005 docker_php "docker-php-entrypoi…" 19 minutes ago Up 14 minutes 9000/tcp docker_php_1
6c2a9e657435 mysql:latest "docker-entrypoint.s…" 19 minutes ago Up 14 minutes 3306/tcp, 33060/tcp docker_mysql_1
I also thought it might be an issue with changes to the parameters.yml file not properly syncing with the container as I'm using Mac (at my old workplace we had to use docker-sync to make sync changes between our dev environment and the actual container), but when inspecting the container itself using exec I can see the changes in the parameters.yml file.
Could the issue be it trying to connect to a mysql server running outside the Docker container? I'm still very new to Docker so I wouldn't be surprised if that's the mistake. Any tips are appreciated 'cause I'm at a dead end.
Your docker-compose file looks wrong to me, try below docker-compose file.
I removed the links, network is much easier.
version: '3'
services:
nginx:
build: nginx
ports:
- "8080:80"
networks:
- backend
volumes:
- ../:/app
php:
build: php-fpm
volumes:
- ../:/app
working_dir: /app
networks:
- backend
extra_hosts:
- "site.dev: 172.17.0.1"
db:
image: mysql:5.7
restart: always
environment:
MYSQL_DATABASE: 'database_name'
MYSQL_USER: 'username'
MYSQL_PASSWORD: 'password'
MYSQL_ROOT_PASSWORD: 'password_root'
networks:
- backend
ports:
- '3306:3306'
volumes:
- ./my-db:/var/lib/mysql
networks:
backend:
driver: bridge
then use database_host: db in php file.
I would diagnose
Check docker logs in the mysql container => no errors
Login to the mysql container and login to mysql => no errors
Login to mysql from the host (mysql -u username -p since you are mapping to 3306 port of the host)
Make sure mysql.cnf doesn't block connect from outside(check
bind-address in the mysql configuration if it 127.0.0.1 the its only
allow to connect form locally, i would for now make it 0.0.0.0 or
commented that line if exists)
mysqld --verbose --help => you will see all options
mysqld --verbose --help | grep bind-address=> check the bind-address
Make sure the user i tried to login has enough privileges to
connect(SELECT user,host FROM mysql.user;) check your user can
connect from docker network => 172.* or anywhere=> %
I think your issue is with your parameters.yml:
parameters:
database_host: 127.0.0.1
When you run compose, MySQL and PHP will run in their own containers which will have their own IPs: 127.0.0.1 or localhost from the php won't be able to connect to the db container. It's like you deployed PHP on a virtual machine A and MySQL to another virtual machine B, but you try to access MySQL from machine A by using localhost where you should specify machine B IP or hostname.
With Docker Compose the internal DNS will resolve the service name to it's container, so you can use something like:
parameters:
# name of the service in compose should be resolved
database_host: db
The error SQLSTATE[HY000] [2002] No such file or directory may be caused when the client tries to read MySQL socket usually present at /var/lib/mysql/mysql.sock which is probably not present in your PHP container.

Docker MySQL - can't connect from Spring Boot app to MySQL database

What I'm trying to do is, connect from my spring-boot app to mysql database in Docker. Each in their own container.
But I must be having something wrong because I can't do it.
To keep it simple :
application-properties :
# URL for the mysql db
spring.datasource.url=jdbc:mysql://workaround-mysql:3308/workaround?serverTimezone=UTC&max_allowed_packet=15728640
# User name in mysql
spring.datasource.username=springuser
# Password for mysql
spring.datasource.password=admin
#Port at which application runs
server.port=8080
docker-compose for MySQL:
version: '3'
services:
workaround-mysql:
container_name: workaround-mysql
image: mysql
environment:
MYSQL_DATABASE: workaround
MYSQL_USER: springuser
MYSQL_PASSWORD: admin
MYSQL_ROOT_PASSWORD: admin
MYSQL_ROOT_HOST: '%'
ports:
- "3308:3306"
restart: always
So pretty simple right ? Database I start with docker-compose up:
All seems to be working fine so far.
Now that I have db started, to the application, this is its docker-compose.yml:
version: '3'
services:
workaround:
restart: always
# will build ./docker/workaround/Dockerfile
build: ./docker/workaround
working_dir: /workaround
volumes:
- ./:/workaround
- ~/.m2:/root/.m2
expose:
- "8080"
command: "mvn clean spring-boot:run"
For its Dockerfile I use Linux Alpine and Java.
FROM alpine:3.9
....add java...
RUN apk update
RUN apk add dos2unix --update-cache --repository http://dl-3.alpinelinux.org/alpine/edge/community/ --allow-untrusted
RUN apk add bash
RUN apk add maven
Super simple. Now let's start the application :
Unknown host, so let's try the IP then :
docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' workaround-mysql
# URL for the mysql db
spring.datasource.url=jdbc:mysql://172.20.0.2:3308/workaround?serverTimezone=UTC&max_allowed_packet=15728640
Now I get timeout:
As you can see I get error. What is wrong with my setup and how to fix
this? Either I have unknown host exception or Refused to connect or connection timeout.
I have tried:
Using ip of a container in my application.properties, didn't work
Different ports for MySQL and application
Different images and versions of MySQL
Having everything in one docker compose with wait
timer for database.
Minimal setup with
https://github.com/hellokoding/hellokoding-courses/tree/master/docker-examples/dockercompose-springboot-mysql-nginx
Also resulted in communication link failure, Site was accessible but I
doubt that db was connected properly.
Notes:
I run this all on one computer I use port 3308 because I have local
MySQL db at 3306.
Here is docker ps -a
#Vusal ANSWER output :
Only thing different from code in answer I did wait for database to be ready 30 seconds
command: /bin/bash -c "sleep 30;mvn clean spring-boot:run;"
Try this docker-compose.yml:
version: '3'
services:
workaround-mysql:
container_name: workaround-mysql
image: mysql
environment:
MYSQL_DATABASE: workaround
MYSQL_USER: springuser
MYSQL_PASSWORD: admin
MYSQL_ROOT_PASSWORD: admin
MYSQL_ROOT_HOST: '%'
ports:
- "3308:3306"
restart: always
workaround:
depends_on:
- workaround-mysql
restart: always
# will build ./docker/workaround/Dockerfile
build: ./docker/workaround
working_dir: /workaround
volumes:
- ./:/workaround
- ~/.m2:/root/.m2
expose:
- "8080"
command: "mvn clean spring-boot:run"
And update your application.properties to use the next JDBC connection url:
spring.datasource.url=jdbc:mysql://workaround-mysql:3306/workaround?serverTimezone=UTC&max_allowed_packet=15728640
It should work when both containers in the same docker-compose file, because docker-compose creates default network for containers, so they can resolve each other by name.
What you haven't tried so far is running both containers on the same Docker network.
First, forget about IP addressing - using it should be avoided by all means.
Second, launch both compose instances with the same Docker network.
Third, do not expose ports - inside bridge network all ports are accessible to running containers.
Create global network
docker network create foo
Modify both compose files so that they use this network instead of creating each one its own:
version: '3.5'
services:
....
networks:
default:
external: true
name: foo
Remove expose directives from compose files - inside one network all ports are exposed by default
Modify connection strings to use default 3306 port instead of 3308
Enjoy
In order for the service to connect with MySql through docker it has to be in same network, look into Docker network
But for better solution I would suggest you to write a single docker compose file for MySql and Spring boot.The reason is it will easily be linked when you do that.No need any other configuration.
version: "3"
services:
mysql-service:
image: mysql
ports:
- "3306:3306"
environment:
- MYSQL_DATABASE=db
- MYSQL_USER=root
- MYSQL_PASSWORD=pass
- MYSQL_ROOT_PASSWORD=pass
spring-service:
image: springservce:latest
ports:
- "8080:8080"
depends_on:
- mysql-service
Before you try to connect to the Docker container you should stop mysql in your computer then go to the application.properties and type:
spring.datasource.url=jdbc:mysql://localhost:3306/NAME_OF_YOUR_DB_HERE?useSSL=false&allowPublicKeyRetrieval=true
Regarding localhost, you should inspect the mysql container and pick the IP address and use it instead. most likely is 172.17.0.2. If it did not work then use localhost.

Moving Wordpress site to Docker: Error establishing DB connection

Ive been making new sites with Wordpress & Docker recently and have a reasonable grasp of how it all works and Im now looking to move some established sites into Docker.
Ive been following this guide:
https://stephenafamo.com/blog/moving-wordpress-docker-container/
I have everything setup as it should be but when I go to my domain.com:1234 I get the error message 'Error establishing a database connection'. I have changed 'DB HOST' to 'mysql' in wp-config.php as advised and all the DB details from the site Im bringing in are correct.
I have attached to the mysql container and checked that the db is there and with the right user and also made sure the pw is correct via mysql CLI too.
SELinux is set to permissive and I havent changed any dir/file ownership nor permissions and for the latter dirs are all 755 and files 644 as they should be.
Edit: I should mention that database/data and everything under that seem to be owned by user/group 'polkitd input' instead of root.
Docker logs aren't really telling me much either apart from the 500 error messages for the WP container when I browse the site on port 1234 (as expected though).
This is the docker-compose file:
version: '2'
services:
example_db:
image: mysql:latest
container_name: example_db
volumes:
- ./database/data:/var/lib/mysql
- ./database/initdb.d:/docker-entrypoint-initdb.d
restart: always
environment:
MYSQL_ROOT_PASSWORD: password123 # any random string will do
MYSQL_DATABASE: mydomin_db # the name of your mysql database
MYSQL_USER: my domain_me # the name of the database user
MYSQL_PASSWORD: password123 # the password of the mysql user
example:
depends_on:
- example_db
image: wordpress:php7.1 # we're using the image with php7.1
container_name: example
ports:
- "1234:80"
restart: always
links:
- example_db:mysql
volumes:
- ./src:/var/www/html
Suggestions most welcome as Im out of ideas!
With the new version of docker-compose it will look like this (if you don't want to use PhpMyAdmin you can leave it out):
version: '3.7'
volumes:
wp-data:
networks:
wp-back:
services:
db:
image: mysql:5.7
volumes:
- wp-data:/var/lib/mysql
environment:
MYSQL_ROOT_PASSWORD: rootPassword
MYSQL_DATABASE: wordpress
MYSQL_USER: wp-user
MYSQL_PASSWORD: wp-pass
ports:
- 8889:3306
networks:
- wp-back
phpmyadmin:
depends_on:
- db
image: phpmyadmin/phpmyadmin
environment:
PMA_HOST: db
MYSQL_USER: wp-user
MYSQL_PASSWORD: wp-pass
MYSQL_ROOT_PASSWORD: rootPassword
ports:
- 3001:80
networks:
- wp-back
wordpress:
depends_on:
- db
image: wordpress:latest
ports:
- 8888:80
- 443:443
environment:
WORDPRESS_DB_HOST: db
WORDPRESS_DB_USER: wp-user
WORDPRESS_DB_PASSWORD: wp-pass
volumes:
- ./wordpress-files:/var/www/html
container_name: wordpress-site
networks:
- wp-back
The database volume is a named volume wp-data, while the wordpress html is a bind-mount to your current directory ./wordpress-files .
make sure that the wp-config.php file has same credentials defined for db_user, db_password as in docker-composer yml file. I too had similar problem i deleted all the files and re-installed and saw that docker-composer up -d would start everything but the wp-config.php file contents for mysql settings were not defined as in docker. so i changed it accordingly and started working eventually
Please take a look at the following compose script. I tried and tested. It works fine.
version: '2'
services:
db:
image: mysql:latest
container_name: db_server
volumes:
- ./database/data:/var/lib/mysql
- ./database/initdb.d:/docker-entrypoint-initdb.d
restart: always
environment:
MYSQL_ROOT_PASSWORD: password123 # any random string will do
MYSQL_DATABASE: udb_test # the name of your mysql database
MYSQL_USER: me_prname # the name of the database user
MYSQL_PASSWORD: password123 # the password of the mysql user
example:
depends_on:
- db
image: wordpress:php7.1 # we're using the image with php7.1
container_name: wp-web
environment:
WORDPRESS_DB_HOST: db:3306
WORDPRESS_DB_USER: me_prname
WORDPRESS_DB_PASSWORD: password123
WORDPRESS_DB_NAME: udb_test
ports:
- "1234:80"
restart: always
volumes:
- ./src:/var/www/html
Let me know if you encounter further issues.
if you want it all in one container you can refer this repo here,
https://github.com/akshayshikre/lamp-alpine/tree/development
Here from lamp-alpine image is used
Then mysql, php, apache2 (lamp stack) is installed and copied local wordpress demosite and db for demo purpose
if you do not want any kind of continuous integration part ignore .circleci folder
Check docker-compose file and Dockerfile, Environment variables are in .env file
I share with you my approach
Show running version, question to see if all is well on your side!
$ docker --version && docker-compose --version
run Docker Copose file
$ docker-compose -f docker-compose.yml up -d
after you wait fast forward
show running containers and name of the Wordpress Container is listening on port 8000
$ docker ps
you will see the name of your WordPress container on the table as follows if you have followed the steps listed on their site
https://hub.docker.com/_/wordpress
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
xxxxxxxxxxxx wordpress:latest "docker-entrypoint.s…" 8 minutes ago Up 8 minutes 0.0.0.0:8000->80/tcp cms_wordpress_1
xxxxxxxxxxxx mysql:5.7 "docker-entrypoint.s…" 8 minutes ago Up 8 minutes 3306/tcp, 33060/tcp cms_db_1
and if you check your browser with the address : localhost:8000
you will get the message "error establishing DB connection"
launch bash inside the Wordpress container
$ docker exec -it cms_wordpress_1 bash
apt update fails as there is no connectivity
$ apt update
open up new terminal and show current Firewalld configuration
$ sudo cat /etc/firewalld/firewalld-workstation.conf | greb 'FirewallBackend'
currently set to 'nftables'
set value to 'iptables'
$ sudo sed -i 's/FirewallBackend=nftables/FirewallBackend=iptables/g' /etc/firewalld/firewalld-workstation.conf
confirme new value
$ sudo cat /etc/firewalld/firewalld-workstation.conf | grep 'FirewallBackend'
restart Firwalld service to apply change
$ sudo systemctl restart firewalld.service
Refresh the running Wordpress session in your browser and that's good.
good work.
In some cases a probable cause of this issue could be, you have made volumes using docker compose up and then when you did docker compose down you expected the volumes to be deleted as well as the docker images, but this is not how it works.
From the doc you could read this:
For data that needs to persist between updates, use host or named volumes.
It implicitly means that named volumes will not get deleted with down, so what happens is, when you do an up and then add a row to a table and then do a subsequent down, then on the next up you will get the same old volume and so querying the same table would give you the same row you created previously!
What does this have to do with the error Error establishing DB connection, you may ask. To answer your question, let's assume one scenario: What if you changed some MYSQL passwords in the docker compose file in between running the down command and the second up command?
MYSQL keeps its own data just like any other data in its tables, so when you do the second up, Docker loads the old volume (the one created by the first up) and thus the old credential information will be used by MYSQL and Docker will not even have the opportunity to insert your new information (the ones you changed in the docker compose file) in the administration tables. So obviously, you will be rejected.
The solution thus now would be very simple. To fix it, either do:
docker-compose down -v
to remove the named volumes as well as the images when running the down, or do:
docker volume rm [volname]
if you've done the down before, and now you want to delete the named volumes.
If you follow this tutorials ,https://stephenafamo.com/blog/moving-wordpress-docker-container/, your site wil not work properly. Coz It doesn't restore database and you need to restore manually .sql dump file existed in initdb.d dir by using this command.
cat backup.sql | docker exec -i CONTAINER /usr/bin/mysql -u root --password=root DATABASE
I also stuck in this and my CSS are not working properly.
Please let me know when you have new idea .