I am having trouble to load in my SQL File to create the schema and the associated tables, I have also tried to use the 'volumes' option in the compose file to copy the sql file into the 'docker-entrypoint-initdb.d' directory but that fails to so anything.
Table + Schema: https://pastebin.com/RTBBGZhn
MySQL Dump: https://pastebin.com/6ApQwt1F
Docker Compose File
version: '2'
services:
melissabot:
image: melissabot
build: .
ports:
- 7000:7000
- 7070:7070
depends_on:
- mysqlMelissa
links:
- mysqlMelissa:db
mysqlMelissa:
image: mysql
build: ${PWD}/Docker/DB/
environment:
- MYSQL_ROOT_PASSWORD=root
# - MYSQL_DATABASE=MelissaBot
ports:
- "3306:3306"
phpmyadmin:
image: phpmyadmin/phpmyadmin
container_name: phpmyadmin
restart: always
ports:
- 8080:80
volumes:
- /sessions
links:
- mysqlMelissa:db
Dockerfile
FROM mysql:5.6
ENV MYSQL_ROOT_PASSWORD=root
# ENV MYSQL_DATABASE=MelissaBot
COPY setup.sh /mysql/setup.sh
COPY dump.sql /mysql/Melissa.sql
RUN chmod +x /mysql/setup.sh
RUN /mysql/setup.sh
setup.sh
#!/bin/bash
set -e
service mysql start
mysql -u root MelissaBot < /mysql/Melissa.sql
service mysql stop
You are doing it wrongly. There is no init system inside docker. So you should not be using server start/stop inside a container. Look at the official image documentation always
https://hub.docker.com/_/mysql
Initializing a fresh instance
When a container is started for the first time, a new database with the specified name will be created and initialized with the provided configuration variables. Furthermore, it will execute files with extensions .sh, .sql and .sql.gz that are found in /docker-entrypoint-initdb.d. Files will be executed in alphabetical order. You can easily populate your mysql services by mounting a SQL dump into that directory and provide custom images with contributed data. SQL files will be imported by default to the database specified by the MYSQL_DATABASE variable.
So you can either copy your default dump files to /docker-entrypoint-initdb.d or mount them using a volume mount when running the container. So you chuck your setup.sh and change your Dockerfile to something like below
FROM mysql:5.6
ENV MYSQL_ROOT_PASSWORD=root
ENV MYSQL_DATABASE=MelissaBot
COPY dump.sql /docker-entrypoint-initdb.d/
But I would rather prefer the approach of volume mounting it inside my compose file
mysqlMelissa:
image: mysql:5.6
environment:
- MYSQL_ROOT_PASSWORD=root
- MYSQL_DATABASE=MelissaBot
ports:
- "3306:3306"
volumes:
- dump.sql:/docker-entrypoint-initdb.d/dump.sql
Related
I am having a problem having the mysql container to run my initialisation scripts.
I have two files create.sql and insert.sql, which I use to initialise the database.
I create the images using the command docker-compose.yml and it runs successfully and creates the images.
I am facing two problems.
When I run the docker-compose up command, the mysql container is created and started successfully. However the two initialisation scripts (create.sql and insert.sql) don't run on the database.
I explicitly use the docker run command to run the created mysql container. In this scenario the initialisation scripts run successfully.
I am using Docker version 18.09.0 and docker-compose version 1.23.1 and ubuntu 16.04 LTS
I am new to docker and can't seem to figure out the problem.
The following are the files I am using to create images.
docker-compose.yml file.
version: '3'
services:
demo-mysql:
image: demo-mysql
build: ./demo-mysql
volumes:
- /mnt/data/mysql-data:/var/lib/mysql
ports:
- 3306:3306
environment:
- MYSQL_ROOT_PASSWORD=root
- MYSQL_DATABASE=demo
- MYSQL_PASSWORD=root
demo-api:
image: demo-api-1.0
build: ./api
depends_on:
- demo-mysql
ports:
- 8080:8080
environment:
- DATABASE_HOST=demo-mysql
- DATABASE_USER=root
- DATABASE_PASSWORD=root
- DATABASE_NAME=demo
- DATABASE_PORT=3306
demo1-app:
image: demo1-app-1.0
build: ./demo1
depends_on:
- demo-mysql
ports:
- 8090:8090
environment:
- DATABASE_HOST=demo-mysql
- DATABASE_USER=root
- DATABASE_PASSWORD=root
- DATABASE_NAME=demo
- DATABASE_PORT=3306
The following is the Dockerfile for the spring boot project
FROM java:8
VOLUME /tmp
ARG DATA_PATH=/src/main/resources
ARG APP_PORT=8080
EXPOSE ${APP_PORT}
ADD /build/libs/demo-api.jar demo-api.jar
ENTRYPOINT ["java","-jar","demo-api.jar"]
The following is the Dockerfile I used to create my mysql image
FROM mysql:5.7
ENV MYSQL_DATABASE=demo \
MYSQL_USER=root \
MYSQL_ROOT_PASSWORD=root
ADD ./1.0/create.sql /docker-entrypoint-initdb.d
ADD ./1.0/insert.sql /docker-entrypoint-initdb.d
EXPOSE 3306
From documentation (https://hub.docker.com/_/mysql/)
Initializing a fresh instance
When a container is started for the first time, a new database with
the specified name will be created and initialized with the provided
configuration variables. Furthermore, it will execute files with
extensions .sh, .sql and .sql.gz that are found in
/docker-entrypoint-initdb.d.
I suspect that, because of the persisted volume
volumes:
- /mnt/data/mysql-data:/var/lib/mysql
when docker starts the mysql image, there is already a DB. So the image isn't "fresh" and the scripts are not run.
Update:
we can confirm this suspect looking at the source code of the docker-entrypoint.sh here: https://github.com/docker-library/mysql/blob/696fc899126ae00771b5d87bdadae836e704ae7d/5.7/docker-entrypoint.sh
if [ ! -d "$DATADIR/mysql" ]; then
...
...
ls /docker-entrypoint-initdb.d/ > /dev/null
for f in /docker-entrypoint-initdb.d/*; do
process_init_file "$f" "${mysql[#]}"
done
The scripts run only if the "$DATADIR/mysql" is not present already.
btw, I personally consider a better design to have the "application" create the database schema, preload the required application data, manage schema migrations etc... at startup, but this another topic :)
I'm new to docker. I've been following this tutorial: https://medium.com/coderscorner/connecting-to-mysql-through-docker-997aa2c090cc . I've set up the root password but once I tried to access the mysql command, it throws out this database is uninitialized error. Also, when I do docker-compose up command to pull the needed modules, it gives out an django.db.utils.InternalError: (1049, "Unknown database 'bitpal'"). The command I placed was:
docker run --name=mysql -e MYSQL_USER=root MYSQL_ROOT_PASSWORD=password -d mysql
I reckon I've searched for answers here but I couldn't be sure of what's wrong.
docker-compose.yml
version: '2'
services:
# Redis
mysql:
image: mysql:5.7
restart: always
hostname: mysql
container_name: mysql
environment:
- MYSQL_USER=root
- MYSQL_ROOT_PASSWORD=password
- MYSQL_DB=bitpal
ports:
- "3306:3306"
# Redis
redis:
image: redis:latest
restart: always
hostname: redis
container_name: redis
ports:
- "6379:6379"
# Django web server
bitpal:
image: python:3.5
restart: always
hostname: bitpal
container_name: bitpal
working_dir: /bitpal
command: ./bin/start_dev.sh
volumes:
- ./bitpal:/bitpal
- ./etc/config:/etc/config
- ./log:/log
ports:
- "80:80"
links:
- mysql
- redis
depends_on:
- mysql
environment:
# Database
- DB_NAME=bitpal
- DB_USER=root
- DB_PASSWORD=password
- DB_HOST=mysql
- DB_PORT=3306
# Celery worker
worker:
image: python:3.5
restart: always
container_name: worker
command: bash -c "./bin/install.sh && ./bin/celery_worker.sh"
working_dir: /bitpal
volumes:
- ./bitpal:/bitpal
- ./etc/config:/etc/config
- ./log:/log
links:
- mysql
- redis
depends_on:
- redis
# Bitshares websocket listener
websocket_listener:
image: python:3.5
restart: always
container_name: websocket_listener
command: bash -c "./bin/install.sh && ./bin/websocket_listener.sh"
working_dir: /bitpal
volumes:
- ./bitpal:/bitpal
- ./etc/config:/etc/config
- ./log:/log
links:
- mysql
- redis
depends_on:
- redis
# Nginx
nginx:
image: nginx:1.12.1
container_name: nginx
ports:
- "8000:80"
volumes:
- ./bitpal:/home/bitpal/bitpal/bitpal
- ./nginx:/etc/nginx/conf.d
depends_on:
- bitpal
My directory looks like this.
`**ROOT**
`root: .gitignore, docker-compose.yml, docker-compose-production.yml...
/bitpal /etc /log /nginx /public_html`
**ROOT/bitpal**
`.gitignore, Dockerfile, Makefile, manage.py... /bin /bitpal /media
/static /tests`
All the project's .sh files are stored under root/bitpal/bin. Do I place wait-for-it.sh there instead or place it in bitpal and nginx folders?
This tutorial you were following is incomplete. It didn't tell you that you must wait until the db is initialized if you want to use it.
Just after running the database container via run command, you should check the logs of this container and wait until the DB initialization
process is complete
You can do it with:
$ docker logs -f <container name>
Where container name in your case is mysql. When you see that db is initialized and DB is started, just detach (ctrl+c) from the logs and continue on.
Your DB is ready to use now.
important note considering your compose file
This compose file is not going to work because the other services like bitpal/worker are not waiting for the DB service to initialize.
Initially download a wait-for-it.sh script, that'd allow other servies to wait for your database when using compose file to setup your application. The script, made by vishnubob, is available here, then copy it to all the catalogs where your services requiring database are.
In the same catalogs create a docker-entrypoint.sh files and write them like this:
#!/bin/bash
set -e
sh -c './wait-for-it.sh mysql:3306 -t 30'
exec "$#"
Then, in your compose file add entries in every service that require DB (and where you places wait-for-it.sh script) that will execute the waiting script:
entrypoint: ["./docker-entrypoint.sh"]
Then, your services will wait for the DB until it's initialized and ready to accept connections.
In the edits I'll add straight forward catalog tree so that you can more clearly see how these files should be placed.
This is one of the only efficient methods because depends_on is not waiting for the db service to be initialized as it's clearly stated in the official docs.
edit with files location explanation
root
- bitpal
+ *some service files*
+ wait-for-it.sh
+ docker-entrypoint.sh
- some_service_requiring_db
+ *some service files*
+ wait-for-it.sh
+ docker-entrypoint.sh
- docker-compose.yml
And your compose file should be like:
version: '2'
services:
# MySQL service definition
mysql:
# like you have
# some services
# Django web server
bitpal:
# ...
entrypoint: ["./docker-entrypoint.sh"]
# further declarations
I am trying to use Docker to create a set of containers (wordpress and MySQL) that will help my local development with Wordpress. As we are running a live database, I want to mount a dump.sql file into the Docker mysql container. Below is my .yml file.
version: '2'
services:
db:
image: mysql:latest
volumes:
- ./data:/docker-entrypoint-initdb.d #./data holders my dump.sql file
environment:
MYSQL_ROOT_PASSWORD: wordpress
MYSQL_DATABASE: wordpress
MYSQL_USER: wordpress
MYSQL_PASSWORD: wordpress
wordpress:
depends_on:
- db
image: wordpress:latest
ports:
- "8000:80"
restart: always
environment:
WORDPRESS_DB_HOST: db:3306
WORDPRESS_DB_PASSWORD: wordpress
volumes:
- ./wp-content/themes/portalV3:/var/www/html/wp-content/themes/portalV3
- ./wp-content/plugins:/var/www/html/wp-content/plugins
- ./wp-content/uploads:/var/www/html/wp-content/uploads
Everything works, but after ~10 seconds the docker container for mysql crashes. Going through the logs, I get the following error:
/usr/local/bin/docker-entrypoint.sh: running /docker-entrypoint-initdb.d/dump.sql
mysql: [Warning] Using a password on the command line interface can be insecure.
ERROR: Can't initialize batch_readline - may be the input source is a directory or a block device.
On closer inspection (attaching to the rebooted mysql container) I see that indeed my dump.sql file wasn't transferred to the container, but a folder with the same name was created in /docker-entrypoint-initdb.d.
Can anyone help me understand how I get docker-compose to copy my dump.sql file and import into the database?
Cheers,
Pieter
The problem you got with docker-entrypoint-initdb.d is that because your source 'data' is a directory and not a file, The destination file (docker-entrypoint-initdb.d) must be a directory too. And vice versa.
So either do
volumes:
- ./data:/docker-entrypoint-initdb.d/
or
volumes:
- ./data/mydump.sql:/docker-entrypoint-initdb.d/mydump.sql
Yes, that is how you should mount the .sql or .sh files i.e by adding a volume by mapping the SQL or .sh files to the docker container's docker-entrypoint-initdb.d folder. But, it's raising an error for some strange reason maybe because the MySQL docker version is old.
You could solve this by creating a custom image i.e,
Dockerfile
FROM mysql:5.7
COPY init.sql /docker-entrypoint-initdb.d/
It creates an image and also helps in running a init script while starting the container.
To use this in a compose file, put your SQL files and Dockerfile in a folder.
database
|---init.sql
|---Dockerfile
docker-compose.yml
version: '3'
services:
mysqldb:
image: mysqldb
build: ./database
container_name: mysql
ports:
- "3306:3306"
environment:
- MYSQL_ROOT_PASSWORD=root
- MYSQL_USER=test
- MYSQL_PASSWORD=root
- MYSQL_DATABASE=test
By this, you could configure the environment variables easily.
I'm new in docker, so cant understand - if I want to build container of mysql/postgresql/clickhouse etc - how to create database and schema of database/table? Maybe in Dockerfile or i can do it from docker-compose.yml?
I mean, that I dont know when and where to use CREATE DATABASE; CREATE TABLE ...; queries if I use docker containers of popular databases
You can use both docker and docker-compose. For example with docker compose.
Create a file called docker-compose.yml like:
version: '3'
services:
db:
image: percona:5.7
container_name: whatever_you_want
environment:
- MYSQL_DATABASE=${DATABASE}
- MYSQL_ROOT_PASSWORD=${ROOT_PASSWORD}
- MYSQL_USER=${USER}
- MYSQL_PASSWORD=${PASSWORD}
volumes:
- ./data:/docker-entrypoint-initdb.d
ports:
- "3306:3306"
Additionally you need a file under ./data with whatever SQL commands you want to run and and .env file where you define the environmental variables I used in the docker-compose.yml file above like: ${DATABASE}
Your .env file:
# MySQL
DATABASE=db_name_here
ROOT_USER=root
ROOT_PASSWORD=root
USER=dev
PASSWORD=dev
Your file with SQL commands to execute ./data/init.sql (you can name the file whatever you want)
CREATE DATABASE 'whatever';
DROP DATABASE 'whatever';
-- you can do whatever you want here
This file will be executed each time you do:
docker-compose up -d db
At first you need to create docker a image for your db server, or use an already existing image.
Bellow is an example of mysql docker image.
version: "3"
services:
****************
mysql:
container_name: mysql
image: mysql:5.7
restart: on-failure
environment:
- MYSQL_DATABASE=YOUR_DB_NAME
- MYSQL_ROOT_PASSWORD=YOUR_ROOT_USER_PASSWORD
- MYSQL_USER=YOUR_USER
- MYSQL_PASSWORD=YOUR_USER_PASSWORD
ports:
- "33060:3306"
volumes:
- "./data/db/mysql:/var/lib/mysql"
Let's describe some sections:
volumes:
- "./data/db/mysql:/var/lib/mysql"
This is like "mounting" container's /var/lib/mysql to system's ./data/db/mysql. So your data will be on your system drive, because in debian the default path to MySQL data is /var/lib/mysql.
ports:
- "33060:3306"
This will map port 3306 from container to system's 33060 port, to avoid conflicts if you have installed MySQL server on system as well.
environment:
- MYSQL_DATABASE=YOUR_DB_NAME
- MYSQL_ROOT_PASSWORD=YOUR_ROOT_USER_PASSWORD
- MYSQL_USER=YOUR_USER
- MYSQL_PASSWORD=YOUR_USER_PASSWORD
This will create a database with the defined parameters: name, root password, ..., or if a database already exists it will try to access with the defined credentials. Functionality to check/create database is already defined in the image.
If you want to define your own functionality you can define your image (e.g. dockerfile: ./Dockerfile instead of image: mysql:5.7). Dockerfile can be something like this:
FROM mysql:5.7
ARG MYSQL_DATABASE
ARG MYSQL_USER
ARG MYSQL_PASSWORD
ARG MYSQL_ROOT_PASSWORD
ENV MYSQL_DATABASE=${MYSQL_DATABASE}
ENV MYSQL_USER=${MYSQL_USER}
ENV MYSQL_PASSWORD=${MYSQL_PASSWORD}
ENV MYSQL_ROOT_PASSWORD=${MYSQL_ROOT_PASSWORD}
# copy predefined config file
COPY configs/default.cnf /etc/mysql/conf.d/
# To be sure that MySQL will not ignore configs
RUN chmod og-w /etc/mysql/conf.d/default.cnf
# DO SOMETHING ELSE YOU WANT
EXPOSE 3306
CMD ["mysqld"]
So you can build and up your container with command docker-compose up -d --build
Here is an example I used to initialise SQL Server 2017 database using container.
https://www.handsonarchitect.com/2018/01/build-custom-sql-server-2017-linux.html
The trick is to use a shell script to run which will invoke the database initialisation script. You might have to wait for few seconds for the database engine service to start before executing the initialisation script.
I'm having trouble importing an .sql dump file with docker-compose. I've followed the docs, which apparently will load the .sql file from docker-entrypoint-initdb.d. However, when I run docker-compose up, the sql file is not copied over to the container.
I've tried stopping the containers with -vf flag, but that didn't work either. Am I doing something wrong in my .yml script?
I have dump.sql in the directory database/db-dump/ in the root where my compose file is.
frontend:
image: myimage
ports:
- "80:80"
links:
- mysql
mysql:
image: mysql
ports:
- "3306:3306"
environment:
MYSQL_ROOT_PASSWORD: rootpass
MYSQL_USER: dbuser
MYSQL_PASSWORD: userpass
MYSQL_DATABASE: myimage_db
volumes:
- ./database/db-dump:/docker-entrypoint-initdb.d
This worked for me,
version: '3.1'
services:
db:
image: mysql
command: --default-authentication-plugin=mysql_native_password
restart: always
volumes:
- ./mysql-dump:/docker-entrypoint-initdb.d
environment:
MYSQL_ROOT_PASSWORD: example
MYSQL_DATABASE: ecommerce
adminer:
image: adminer
restart: always
ports:
- 8080:8080
mysql-dump must be a directory. All the .sql's in the directory will be imported.
After many attempts with the volumes setting i found a workaround
I created another image based on mysql with the following in the Dockerfile
FROM mysql:5.6
ADD dump.sql /docker-entrypoint-initdb.d
Then removed the volumes from compose and ran the new image
frontend:
image: myimage
ports:
- "80:80"
links:
- mysql
mysql:
image: mymysql
ports:
- "3306:3306"
environment:
MYSQL_ROOT_PASSWORD: rootpass
MYSQL_USER: dbuser
MYSQL_PASSWORD: userpass
MYSQL_DATABASE: myimage_db
This way the dump is always copied over and run on startup
This appears on the documentation page of Docker MySQL image: https://hub.docker.com/_/mysql/
Initializing a fresh instance
When a container is started for the first time, a new database with
the specified name will be created and initialized with the provided
configuration variables. Furthermore, it will execute files with
extensions .sh, .sql and .sql.gz that are found in
/docker-entrypoint-initdb.d. Files will be executed in alphabetical
order. You can easily populate your mysql services by mounting a SQL
dump into that
directory
and provide custom
images with contributed
data. SQL files will be imported by default to the database specified
by the MYSQL_DATABASE variable.
Mysql database dump schema.sql is residing in the /mysql-dump/schema.sql directory and it creates tables during the initialization process.
docker-compose.yml:
mysql:
image: mysql:5.7
command: mysqld --user=root
volumes:
- ./mysql-dump:/docker-entrypoint-initdb.d
environment:
MYSQL_DATABASE: ${MYSQL_DATABASE}
MYSQL_USER: ${MYSQL_USER}
MYSQL_PASSWORD: ${MYSQL_PASSWORD}
MYSQL_ROOT_PASSWORD: ${MYSQL_ROOT_PASSWORD}
I was having a similar issue with mysql where I would mount a local directory at /configs/mysql/data containing a mydatabasedump.sql file via docker-compose to the docker-entrypoint-initdb.d volume,
the file would get loaded on to the container but not execute or populate the database when the container initialized. My intial docker-compose.yml looke like this:
#docker-compose.yml
version: '3'
services:
db:
build: ./build/mysql/ #this is pointing to my Dockerfile
container_name: MYSQL_Database
restart: always
environment:
MYSQL_PORT: 3306
MYSQL_ROOT_PASSWORD: admin
MYSQL_DATABASE: my_app_database
MYSQL_USER: admin
MYSQL_PASSWORD: admin
volumes:
- ./configs/mysql/data:/docker-entrypoint-initdb.d:
I found two working solutions for this problem:
The first came after I logged in the running container and confirmed that mydatabasedump.sq file was present and executable in the container's docker-entrypoint-initdb.d directory; I created and added
a bash script to my local /configs/mysql/data directory called dump.sh that excuted after the container was initialized. It contains a single mysql command that copies my_database_dump.sql to my_app_database.
The bash script looks like this
#!/bin/bash
#dump.sh
mysql -uadmin -padmin my_app_database < my_database_dump.sql
#end of dump.sh
I executed this script via my Dockerfile in the ENTRYPOINT directive like this:
#Dockerfile
FROM mysql:5.5
ENTRYPOINT [ "dump.sh" ]
EXPOSE 80
#end of Dockerfile
After realizing the initial issue was due to the volumes being mouted after the cotainer is built and therefore not intilizing the database with the dump file (or executing any scripts in that directory) at boot time, the second solution was simply to
move the volumes directive in my compose-file above the built directive. This worked and allowed me to remove the dump.sh scrip and the DOCKERENTRY directive in my Dockerfile.
The modified docker-compose.yml looks like this
#docker-compose.yml
version: '3'
services:
db:
volumes:
- ./configs/mysql/data:/docker-entrypoint-initdb.d
build: ./build/mysql/ #this is pointing to my Dockerfile
container_name: MYSQL_Database
restart: always
environment:
MYSQL_PORT: 3306
MYSQL_ROOT_PASSWORD: admin
MYSQL_DATABASE: my_app_database
MYSQL_USER: admin
MYSQL_PASSWORD: admin
I also have this problem. I mount a local directory at ./mysql-dump containing a init.sql file via docker-compose to the docker-entrypoint-initdb.d volume, the file would get loaded on to the container but not execute or populate the database when the container initialized.
My intial docker-compose.yml looke like this:
mysqld:
image: mysql
container_name: mysqld
volumes:
- ./mysql/data:/var/lib/mysql
- ./mysql/my.cnf:/etc/my.cnf
- ./init:/docker-entrypoint-initdb.d
env_file: .env
restart: always
environment:
- MYSQL_ROOT_PASSWORD=123456
- MYSQL_DATABASE=fendou
command: --character-set-server=utf8mb4 --collation-server=utf8mb4_unicode_ci
--default-authentication-plugin=mysql_native_password
but it doesn't work for me.
I found another working solutions for this problem:
add --init-file /data/application/init.sql to mysql command.change above configuration like
mysqld:
image: mysql
container_name: mysqld
volumes:
- ./mysql/data:/var/lib/mysql
- ./mysql/my.cnf:/etc/my.cnf
# - ./init:/docker-entrypoint-initdb.d
env_file: .env
restart: always
environment:
- MYSQL_ROOT_PASSWORD=123456
- MYSQL_DATABASE=fendou
command: --character-set-server=utf8mb4 --collation-server=utf8mb4_unicode_ci
--default-authentication-plugin=mysql_native_password
--init-file /docker-entrypoint-initdb.d/init.sql #attention here
hope it help for you
I wanted to keep the original setup of the container, so I tried a restore on the already running container. This seemed to work:
cat dump.sql | docker-compose exec -T db mysql -h localhost -u root -psomewordpress -v
But it was very slow and the verbose output seemed to be buffered, so I tried:
docker-compose cp dump.sql db:/tmp/
docker-compose exec db sh -c "mysql -h localhost -u root -psomewordpress -v < /tmp/dump.sql"
Which at least provided faster feedback.
Might be useful for someone? Looks like it was mainly slow because I used --skip-extended-insert on the dump, without the extended inserts it went faster 🙂