Docker Compose mysql import .sql - mysql

I'm having trouble importing an .sql dump file with docker-compose. I've followed the docs, which apparently will load the .sql file from docker-entrypoint-initdb.d. However, when I run docker-compose up, the sql file is not copied over to the container.
I've tried stopping the containers with -vf flag, but that didn't work either. Am I doing something wrong in my .yml script?
I have dump.sql in the directory database/db-dump/ in the root where my compose file is.
frontend:
image: myimage
ports:
- "80:80"
links:
- mysql
mysql:
image: mysql
ports:
- "3306:3306"
environment:
MYSQL_ROOT_PASSWORD: rootpass
MYSQL_USER: dbuser
MYSQL_PASSWORD: userpass
MYSQL_DATABASE: myimage_db
volumes:
- ./database/db-dump:/docker-entrypoint-initdb.d

This worked for me,
version: '3.1'
services:
db:
image: mysql
command: --default-authentication-plugin=mysql_native_password
restart: always
volumes:
- ./mysql-dump:/docker-entrypoint-initdb.d
environment:
MYSQL_ROOT_PASSWORD: example
MYSQL_DATABASE: ecommerce
adminer:
image: adminer
restart: always
ports:
- 8080:8080
mysql-dump must be a directory. All the .sql's in the directory will be imported.

After many attempts with the volumes setting i found a workaround
I created another image based on mysql with the following in the Dockerfile
FROM mysql:5.6
ADD dump.sql /docker-entrypoint-initdb.d
Then removed the volumes from compose and ran the new image
frontend:
image: myimage
ports:
- "80:80"
links:
- mysql
mysql:
image: mymysql
ports:
- "3306:3306"
environment:
MYSQL_ROOT_PASSWORD: rootpass
MYSQL_USER: dbuser
MYSQL_PASSWORD: userpass
MYSQL_DATABASE: myimage_db
This way the dump is always copied over and run on startup

This appears on the documentation page of Docker MySQL image: https://hub.docker.com/_/mysql/
Initializing a fresh instance
When a container is started for the first time, a new database with
the specified name will be created and initialized with the provided
configuration variables. Furthermore, it will execute files with
extensions .sh, .sql and .sql.gz that are found in
/docker-entrypoint-initdb.d. Files will be executed in alphabetical
order. You can easily populate your mysql services by mounting a SQL
dump into that
directory
and provide custom
images with contributed
data. SQL files will be imported by default to the database specified
by the MYSQL_DATABASE variable.

Mysql database dump schema.sql is residing in the /mysql-dump/schema.sql directory and it creates tables during the initialization process.
docker-compose.yml:
mysql:
image: mysql:5.7
command: mysqld --user=root
volumes:
- ./mysql-dump:/docker-entrypoint-initdb.d
environment:
MYSQL_DATABASE: ${MYSQL_DATABASE}
MYSQL_USER: ${MYSQL_USER}
MYSQL_PASSWORD: ${MYSQL_PASSWORD}
MYSQL_ROOT_PASSWORD: ${MYSQL_ROOT_PASSWORD}

I was having a similar issue with mysql where I would mount a local directory at /configs/mysql/data containing a mydatabasedump.sql file via docker-compose to the docker-entrypoint-initdb.d volume,
the file would get loaded on to the container but not execute or populate the database when the container initialized. My intial docker-compose.yml looke like this:
#docker-compose.yml
version: '3'
services:
db:
build: ./build/mysql/ #this is pointing to my Dockerfile
container_name: MYSQL_Database
restart: always
environment:
MYSQL_PORT: 3306
MYSQL_ROOT_PASSWORD: admin
MYSQL_DATABASE: my_app_database
MYSQL_USER: admin
MYSQL_PASSWORD: admin
volumes:
- ./configs/mysql/data:/docker-entrypoint-initdb.d:
I found two working solutions for this problem:
The first came after I logged in the running container and confirmed that mydatabasedump.sq file was present and executable in the container's docker-entrypoint-initdb.d directory; I created and added
a bash script to my local /configs/mysql/data directory called dump.sh that excuted after the container was initialized. It contains a single mysql command that copies my_database_dump.sql to my_app_database.
The bash script looks like this
#!/bin/bash
#dump.sh
mysql -uadmin -padmin my_app_database < my_database_dump.sql
#end of dump.sh
I executed this script via my Dockerfile in the ENTRYPOINT directive like this:
#Dockerfile
FROM mysql:5.5
ENTRYPOINT [ "dump.sh" ]
EXPOSE 80
#end of Dockerfile
After realizing the initial issue was due to the volumes being mouted after the cotainer is built and therefore not intilizing the database with the dump file (or executing any scripts in that directory) at boot time, the second solution was simply to
move the volumes directive in my compose-file above the built directive. This worked and allowed me to remove the dump.sh scrip and the DOCKERENTRY directive in my Dockerfile.
The modified docker-compose.yml looks like this
#docker-compose.yml
version: '3'
services:
db:
volumes:
- ./configs/mysql/data:/docker-entrypoint-initdb.d
build: ./build/mysql/ #this is pointing to my Dockerfile
container_name: MYSQL_Database
restart: always
environment:
MYSQL_PORT: 3306
MYSQL_ROOT_PASSWORD: admin
MYSQL_DATABASE: my_app_database
MYSQL_USER: admin
MYSQL_PASSWORD: admin

I also have this problem. I mount a local directory at ./mysql-dump containing a init.sql file via docker-compose to the docker-entrypoint-initdb.d volume, the file would get loaded on to the container but not execute or populate the database when the container initialized.
My intial docker-compose.yml looke like this:
mysqld:
image: mysql
container_name: mysqld
volumes:
- ./mysql/data:/var/lib/mysql
- ./mysql/my.cnf:/etc/my.cnf
- ./init:/docker-entrypoint-initdb.d
env_file: .env
restart: always
environment:
- MYSQL_ROOT_PASSWORD=123456
- MYSQL_DATABASE=fendou
command: --character-set-server=utf8mb4 --collation-server=utf8mb4_unicode_ci
--default-authentication-plugin=mysql_native_password
but it doesn't work for me.
I found another working solutions for this problem:
add --init-file /data/application/init.sql to mysql command.change above configuration like
mysqld:
image: mysql
container_name: mysqld
volumes:
- ./mysql/data:/var/lib/mysql
- ./mysql/my.cnf:/etc/my.cnf
# - ./init:/docker-entrypoint-initdb.d
env_file: .env
restart: always
environment:
- MYSQL_ROOT_PASSWORD=123456
- MYSQL_DATABASE=fendou
command: --character-set-server=utf8mb4 --collation-server=utf8mb4_unicode_ci
--default-authentication-plugin=mysql_native_password
--init-file /docker-entrypoint-initdb.d/init.sql #attention here
hope it help for you

I wanted to keep the original setup of the container, so I tried a restore on the already running container. This seemed to work:
cat dump.sql | docker-compose exec -T db mysql -h localhost -u root -psomewordpress -v
But it was very slow and the verbose output seemed to be buffered, so I tried:
docker-compose cp dump.sql db:/tmp/
docker-compose exec db sh -c "mysql -h localhost -u root -psomewordpress -v < /tmp/dump.sql"
Which at least provided faster feedback.
Might be useful for someone? Looks like it was mainly slow because I used --skip-extended-insert on the dump, without the extended inserts it went faster 🙂

Related

MySQL Docker Compose password not working

What I want to do is:
Create a MySQL8 docker container
The MySQL container should run a dump file
I was successful in creating the basic container, however, that are several issues:
The password that I added in docker-compose.yml is ignored, when I run:
"docker exec -it mysqlDB bash" followed by "MySQL -u admin -p" I get Acess denied, and also with root
I don't know if the dump is being used because I can access the DB
I'm also getting this error:
[ERROR] [MY-000061] [Server] 1105 Input Output error while reading file /docker-entrypoint-initdb.d/, line 0, I/O error code 1
I tried many things for hours and it only got worse like not running at all.
I always run with: "docker-compose --log-level DEBUG -verbose up"
I always retry with the sequence:
ctrl+c
docker-compose down
docker system prune -a
docker volume prune
After running these prunes I need to run twice, or else I got the error:
"The designated data directory /var/lib/mysql/ is unusable. You can remove all files that the server added to it."
Dockerfile(at /MySQL), there's also a LastDump.sql in this directory
EDIT: Later I deleted this file, and got the same result
FROM mysql:8.0.21
RUN chown -R mysql:root /var/lib/mysql/
ENV MYSQL_DATABASE=Olimpo
ENV MYSQL_USER=admin
ENV MYSQL_PASSWORD=senha
ENV MYSQL_ROOT_PASSWORD=senha
ADD LastDump.sql /etc/mysql/LastDump.sql
RUN sed -i 's/MYSQL_DATABASE/'$MYSQL_DATABASE'/g' /etc/mysql/LastDump.sql
RUN cp /etc/mysql/LastDump.sql /docker-entrypoint-initdb.d
EXPOSE 3306
docker-compose.yml(at / main folder)
version: "3.8"
# Define services
services:
# Database Service (Mysql)
mysqldb:
image: mysql:8.0.21
container_name: mysqlDB
command: --default-authentication-plugin=mysql_native_password --init-file /docker-entrypoint-initdb.d/
ports:
- "3307:3306"
restart: always
environment:
MYSQL_DATABASE: Olimpo
MYSQL_USER: admin
MYSQL_PASSWORD: senha
MYSQL_ROOT_PASSWORD: senha
volumes:
- mysql_data:/var/lib/mysql
# next line is commented doesn't run with it
#- ./MySQL/LastDump.sql:/docker-entrypoint-initdb.d
networks:
- backend
# Volumes
volumes:
mysql_data:
driver: local
# Networks to be created to facilitate communication between containers
networks:
backend:
Your problem seem to be the parameter in the command of your yml file. It worked in my machine when I took it out.
Change from command: --default-authentication-plugin=mysql_native_password --init-file /docker-entrypoint-initdb.d/ to command: command: --default-authentication-plugin=mysql_native_password
Fixed file is below:
# Define services
services:
# Database Service (Mysql)
mysqldb:
image: mysql:8.0.21
container_name: mysqlDB
command: --default-authentication-plugin=mysql_native_password
ports:
- "3307:3306"
restart: always
environment:
MYSQL_DATABASE: Olimpo
MYSQL_USER: admin
MYSQL_PASSWORD: senha
MYSQL_ROOT_PASSWORD: senha
volumes:
- mysql_data:/var/lib/mysql
# next line is commented doesn't run with it
#- ./MySQL/LastDump.sql:/docker-entrypoint-initdb.d
networks:
- backend
# Volumes
volumes:
mysql_data:
driver: local
# Networks to be created to facilitate communication between containers
networks:
backend:
The mysql dockerfile is useless.
Create a folder called mysql-dump with the dump.sql inside.
Add the line "USE db_name" to dump.sql
In docker-compose.yml:
remove "--init-file /docker-entrypoint-initdb.d/"
add the line in volumes "- ./mysql-dump:/docker-entrypoint-initdb.d"

Why edit contents of docker-compose.yml are not reflected?

I want to use MySQL by using Docker.
I wrote the following DockerFile and docker-compose.yml.
Dockerfile
FROM mysql:8.0
RUN mkdir /var/log/mysql
RUN touch /var/log/mysql/mysqld.log
docker-compose.yml
version: '3'
services:
dbserver:
build: ./docker/mysql
image: test-db:0.0.1
restart: always
environment:
MYSQL_DATABASE: prototype
MYSQL_USER: user
MYSQL_PASSWORD: password
MYSQL_ROOT_PASSWORD: rootpassword
ports:
- "3306:3306"
volumes:
- ./docker/mysql/initdb.d:/docker-entrypoint-initdb.d
- ./docker/mysql/conf.d:/etc/mysql/conf.d
- ./log/mysql:/var/log/mysql
- ./docker/mysql/data:/var/lib/mysql
volumes:
mysql-bd:
driver: local
I succeeded build and could confirm the database.
Then I wanted to change the database name, so I edited a part of the yml file following.
Before
MYSQL_DATABASE: prototype
After
MYSQL_DATABASE: test_db
Then, I confirmed the database but its name was not changed.
I removed the MySQL container and tried again, but the result was not changed.
Why edit contents of docker-compose.yml are not reflected?
You are using a host volume for your database, meaning that the databases are persisted between containers restarts.
...
volumes:
./docker/mysql/data:/var/lib/mysql
...
Delete the local directory ./docker/mysql/data and restart your services. The database change will be reflected.

logging mysql queries from docker container

i am new to docker and I am still figuring out how it works. I have setup through docker-compose a wordpress site and the yaml file i am using is the following.
version: '3.3'
services:
db:
image: mysql:5.7
volumes:
- ./db_data/db/backup.sql:/docker-entrypoint-initdb.d/backup.sql #it loads an already existing database
#- ./db_data/all.log:/var/log/mysql/all.log # i want here to have the logs of mysql
restart: always
environment:
MYSQL_ROOT_PASSWORD: somewordpress
MYSQL_DATABASE: wordpress
MYSQL_USER: wordpress
MYSQL_PASSWORD: wordpress
wordpress:
depends_on:
- db
image: wordpress:latest
ports:
- "8000:80"
restart: always
environment:
WORDPRESS_DB_HOST: db:3306
WORDPRESS_DB_USER: wordpress
WORDPRESS_DB_PASSWORD: wordpress
volumes:
- ./db_data/wp-content:/var/www/html/wp-content # loads the files from an existing installation of wordpress
volumes:
db_data:
as you can see by this line
- ./db_data/all.log:/var/log/mysql/all.log
i am trying to somehow make it so i can log all the queries made. I have managed to do that by doing the following:
1) docker-compose up, while having commented out the line with all.log from yaml file.
2) logging to the container of the mysql with bash and running the following
mysql -u root -psomewordpress -e "SET global log_output = 'FILE'; SET global general_log_file='/var/log/mysql/all.log'; SET global general_log = 1;"
3) Going to the /var/lib/docker/volumes and getting the file created.
What i am trying to achieve is to make somehow this command part of the yaml file so I can also have the all.log file mounted at the location where the yaml file is.
What I did to solve my problem was using the following:
version: '3.3'
services:
db:
image: mysql:5.7
volumes:
- ./db_data/db/backup.sql:/docker-entrypoint-initdb.d/backup.sql #it loads an already existing database
-./logs/mysql:/var/log/mysql # to get the folder of mysql in my logs/mysql
restart: always
environment:
MYSQL_ROOT_PASSWORD: somewordpress
MYSQL_DATABASE: wordpress
MYSQL_USER: wordpress
MYSQL_PASSWORD: wordpress
wordpress:
depends_on:
- db
image: wordpress:latest
ports:
- "8000:80"
restart: always
environment:
WORDPRESS_DB_HOST: db:3306
WORDPRESS_DB_USER: wordpress
WORDPRESS_DB_PASSWORD: wordpress
volumes:
- ./db_data/wp-content:/var/www/html/wp-content
This code would sync what is inside the docker container of mysql with my folder logs/mysql. So I put a little shell script in there.
#!/bin/bash
if [ ! -f /var/log/mysql/all.log ]; then
touch /var/log/mysql/all.log
fi
chmod 777 /var/log/mysql/all.log
mysql -u root -psomewordpress -e "SET global log_output = 'FILE'; SET global general_log_file='/var/log/mysql/all.log'; SET global general_log = 1;"
It checks if the all.log file is there, if not it creates it and then starts mysql logging. Thats it, after that all queries are logged directly to all.log in which i have direct access.
One more thing, to execute that shell script you might want to use
docker exec 5.7-mysql "./var/log/mysql/initlogs.sh"
where initlogs.sh is the name of the shell script.
I really hope it would help someone!

Docker - how to alter my Docker Compose file to automate a bash script for the MySQL container?

I have the following setup Docker Composer setup and want to run a shell script to automate tasks like importing the DB into the MySQL database.
# Adopt version 2 syntax:
version: '2'
volumes:
database_data:
driver: local
services:
###########################
# Setup the Nginx container
###########################
nginx:
image: nginx:latest
ports:
- 8080:80
volumes:
- ./docker/nginx/default.conf:/etc/nginx/conf.d/default.conf
volumes_from:
- php
###########################
# Setup the PHP container
###########################
php:
build: ./docker/php/
expose:
- 9000
volumes:
- .:/var/www
###########################
# Setup the Database (MySQL) container
###########################
mysql:
image: mysql:latest
expose:
- 3306
volumes:
- database_data:/var/lib/mysql
environment:
MYSQL_ROOT_PASSWORD: secret
MYSQL_DATABASE: project
MYSQL_USER: project
MYSQL_PASSWORD: project
Best solution is to create a custom Dockerfile ,which extends mysql and add a custom shell script, which does what you want. For example:
start.sh
#!/bin/sh
mysqld
mysql -u project -ppropject project < /path/to/backup.sql
Don't forget to add your backup.sql either to your Dockerfile or docker-compose.yml
Now, Dockerfile:
FROM mysql:latest
COPY start.sh /tmp/start.sh
COPY backup.sql /path/to/backup.sql
CMD ["/tmp/start.sh"]
If you change your backup.sql frequently, it makes no sense to add it to Dockerfile. Instead, put it under volumes in docker-compose.yml:
mysql:
build: .
expose:
- 3306
volumes:
- ./backup.sql:/path/to/backup.sql
- database_data:/var/lib/mysql
environment:
MYSQL_ROOT_PASSWORD: secret
MYSQL_DATABASE: project
MYSQL_USER: project
MYSQL_PASSWORD: project
You can keep using a the original image: load your setup script into the container as a Config (using the long-form definition so you can set the execute permission), and then override the Entrypoint to run your script (which should probably run the original entrypoint script once it finishes). So something like:
mysql:
image: mysql:latest
expose:
- 3306
volumes:
- database_data:/var/lib/mysql
environment:
MYSQL_ROOT_PASSWORD: secret
MYSQL_DATABASE: project
MYSQL_USER: project
MYSQL_PASSWORD: project
configs:
- source: ./OverrideScript.sh
target: /OverrideScript.sh
#0777 will work too, but you can't write to it either way
mode: 0555
entrypoint: /OverrideScript.sh
The other answers are right that the "Proper" way would be to make your own image. But TBH if your override script is relatively small and lightweight, the workaround isn't so bad, and it gets you out of having to rebuild your custom image every time MySQL releases a new image.

Docker-compose mysql mount volume turns sql into folder

I am trying to use Docker to create a set of containers (wordpress and MySQL) that will help my local development with Wordpress. As we are running a live database, I want to mount a dump.sql file into the Docker mysql container. Below is my .yml file.
version: '2'
services:
db:
image: mysql:latest
volumes:
- ./data:/docker-entrypoint-initdb.d #./data holders my dump.sql file
environment:
MYSQL_ROOT_PASSWORD: wordpress
MYSQL_DATABASE: wordpress
MYSQL_USER: wordpress
MYSQL_PASSWORD: wordpress
wordpress:
depends_on:
- db
image: wordpress:latest
ports:
- "8000:80"
restart: always
environment:
WORDPRESS_DB_HOST: db:3306
WORDPRESS_DB_PASSWORD: wordpress
volumes:
- ./wp-content/themes/portalV3:/var/www/html/wp-content/themes/portalV3
- ./wp-content/plugins:/var/www/html/wp-content/plugins
- ./wp-content/uploads:/var/www/html/wp-content/uploads
Everything works, but after ~10 seconds the docker container for mysql crashes. Going through the logs, I get the following error:
/usr/local/bin/docker-entrypoint.sh: running /docker-entrypoint-initdb.d/dump.sql
mysql: [Warning] Using a password on the command line interface can be insecure.
ERROR: Can't initialize batch_readline - may be the input source is a directory or a block device.
On closer inspection (attaching to the rebooted mysql container) I see that indeed my dump.sql file wasn't transferred to the container, but a folder with the same name was created in /docker-entrypoint-initdb.d.
Can anyone help me understand how I get docker-compose to copy my dump.sql file and import into the database?
Cheers,
Pieter
The problem you got with docker-entrypoint-initdb.d is that because your source 'data' is a directory and not a file, The destination file (docker-entrypoint-initdb.d) must be a directory too. And vice versa.
So either do
volumes:
- ./data:/docker-entrypoint-initdb.d/
or
volumes:
- ./data/mydump.sql:/docker-entrypoint-initdb.d/mydump.sql
Yes, that is how you should mount the .sql or .sh files i.e by adding a volume by mapping the SQL or .sh files to the docker container's docker-entrypoint-initdb.d folder. But, it's raising an error for some strange reason maybe because the MySQL docker version is old.
You could solve this by creating a custom image i.e,
Dockerfile
FROM mysql:5.7
COPY init.sql /docker-entrypoint-initdb.d/
It creates an image and also helps in running a init script while starting the container.
To use this in a compose file, put your SQL files and Dockerfile in a folder.
database
|---init.sql
|---Dockerfile
docker-compose.yml
version: '3'
services:
mysqldb:
image: mysqldb
build: ./database
container_name: mysql
ports:
- "3306:3306"
environment:
- MYSQL_ROOT_PASSWORD=root
- MYSQL_USER=test
- MYSQL_PASSWORD=root
- MYSQL_DATABASE=test
By this, you could configure the environment variables easily.