Can't connect to db docker container - mysql

currently working on moving our application to start using docker. It's a typical app with backend and frontend. I don't have any troubles with front, while still can't launch back.
I have Docker file for backend:
FROM williamyeh/java8
RUN apt-get -y update && apt-get install -y maven
WORKDIR /explorerbackend
ADD settings.xml /root/.m2/settings.xml
ADD pom.xml /explorerbackend
ADD src /explorerbackend/src
RUN ["mvn", "clean", "install"]
ADD target/explorer-backend-1.0.jar /explorerbackend/app.jar
RUN sh -c 'touch /explorerbackend/app.jar'
ENV JAVA_OPTS=""
ENTRYPOINT [ "sh", "-c", "java $JAVA_OPTS -Djava.security.egd=file:/dev/./urandom -jar /explorerbackend/app.jar" ]
and Docker file for mysql:
FROM mysql
ADD createDB.sql /docker-entrypoint-initdb.d
The reason i'm using a separate Docker file for mysql instead of just using image in docker-compose is necessity to create 2 databases on start (otherwise backend will not launch)
createDB.sql file looks as:
CREATE DATABASE IE;
CREATE DATABASE IE_test;
Now i have docker-compose.yml file which is supposed to start 2 containers and make backend connect to database:
version: "3.0"
services:
database:
environment:
MYSQL_ROOT_PASSWORD: root
build:
context: *PATH_TO_DIR_WITH_DOCKERFILE*
dockerfile: Dockerfile
ports:
- 3306:3306
volumes:
- db_data:/var/lib/mysql
backend:
build:
context: *PATH_TO_DIR_WITH_DOCKERFILE*
dockerfile: Dockerfile
ports:
- 3000:3000
depends_on:
- database
volumes:
db_data:
When I run the command docker-compose up database container is up and running while backend is failing:
backend_1 | java.sql.SQLNonTransientConnectionException: Could not create connection to database server. Attempted reconnect 3 times. Giving up.
However I'm able to log in to database container and I do see databases created:
mysql> show databases;
+--------------------+
| Database |
+--------------------+
| information_schema |
| IE |
| IE_test |
| mysql |
| performance_schema |
| sys |
+--------------------+
6 rows in set (0.00 sec)
The only reason I see might be related to yml property file of backend:
app:
data-base:
name: IE
link: database
port: 3306
.................
From the frontend container I'm able to ping database (but am I allowed to put into property file just link:database):
root#897b187f9042:/frontend# ping database
PING database (172.19.0.2): 56 data bytes
64 bytes from 172.19.0.2: icmp_seq=0 ttl=64 time=0.086 ms
64 bytes from 172.19.0.2: icmp_seq=1 ttl=64 time=0.088 ms
So, I assume it's pingable from backend container as well, but why it's not able to connect to db server?

MySQL takes a few seconds to start-up. In-order to confirm this is a race-condition, try the following:
$ docker-compose up -d database && sleep 5 && docker-compose up
When/if this confirms the race-condition, you can alleviate that with a HEALTHCHECK on your database image.
See: https://github.com/docker-library/healthcheck/tree/master/mysql
Script from above link:
#!/bin/bash
set -eo pipefail
if [ "$MYSQL_RANDOM_ROOT_PASSWORD" ] && [ -z "$MYSQL_USER" ] && [ -z "$MYSQL_PASSWORD" ]; then
# there's no way we can guess what the random MySQL password was
echo >&2 'healthcheck error: cannot determine random root password (and MYSQL_USER and MYSQL_PASSWORD were not set)'
exit 0
fi
host="$(hostname --ip-address || echo '127.0.0.1')"
user="${MYSQL_USER:-root}"
export MYSQL_PWD="${MYSQL_PASSWORD:-$MYSQL_ROOT_PASSWORD}"
args=(
# force mysql to not use the local "mysqld.sock" (test "external" connectibility)
-h"$host"
-u"$user"
--silent
)
if select="$(echo 'SELECT 1' | mysql "${args[#]}")" && [ "$select" = '1' ]; then
exit 0
fi
exit 1

Eventually, we found the problem which is a kind of oversight.
The root cause was backend dockerfile:
FROM williamyeh/java8
RUN apt-get -y update && apt-get install -y maven
WORKDIR /explorerbackend
ADD settings.xml /root/.m2/settings.xml
ADD pom.xml /explorerbackend
ADD src /explorerbackend/src
RUN ["mvn", "clean", "install"]
ADD target/explorer-backend-1.0.jar /explorerbackend/app.jar
RUN sh -c 'touch /explorerbackend/app.jar'
ENV JAVA_OPTS=""
ENTRYPOINT [ "sh", "-c", "java $JAVA_OPTS -Djava.security.egd=file:/dev/./urandom -jar /explorerbackend/app.jar" ]
The idea is pretty simple:
1. Take java image
2. install maven
3. copy src folder of my project from host
4. install with maven in container
5. move jar to workdir inside container
6. launch it
However, option 5. doesn't look correct, as instead of copying jar file what was just created by maven inside container i was copying it from my host.
Issue was resolved simply replacing
ADD target/explorer-backend-1.0.jar /explorerbackend/app.jar
with
RUN cp /explorerbackend/target/explorer-backend-1.0.jar /explorerbackend/app.jar
Thanks Rawcode for looking into it!

Related

CircleCI job creates docker MySQL 8 but nothing can connect

(See UPDATE at end of post for potentially helpful debug info.)
I have a CircleCI job that deploys MySQL 8 via - setup_remote_docker+docker-compose and then attempts to start a Java app to communicate with MySQL 8. Unfortunately, even though docker ps shows the container is up and running, any attempt to communicate with MySQL--either through the Java app or docker exec--fails, saying the container is not running (and Java throws a "Communications Link Failure" exception). It's a bit confusing because the container appears to be up, and the exact same commands work on my local machine.
Here's my CircleCI config.yml:
Build and Test:
<<: *configure_machine
steps:
- *load_repo
- ... other unrelated stuff ...
- *load_gradle_wrapper
- run:
name: Install Docker Compose
environment:
COMPOSE_VERSION: '1.29.2'
command: |
curl -L "https://github.com/docker/compose/releases/download/${COMPOSE_VERSION}/docker-compose-$(uname -s)-$(uname -m)" -o ~/docker-compose
chmod +x ~/docker-compose
sudo mv ~/docker-compose /usr/local/bin/docker-compose
- setup_remote_docker
- run:
name: Start MySQL docker
command: docker-compose up -d
- run:
name: Check Docker MySQL
command: docker ps
- run:
name: Query MySQL #test that fails
command: docker exec -it mysql8_test_mysql mysql mysql -h 127.0.0.1 --port 3306 -u root -prootpass -e "show databases;"
And here's my docker-compose.yml that is run in one of the steps:
version: "3.1"
services:
# MySQL Dev Image
mysql-migrate:
container_name: mysql8_test_mysql
image: mysql:8.0
command:
mysqld --default-authentication-plugin=mysql_native_password
--character-set-server=utf8mb4 --collation-server=utf8mb4_unicode_ci
--log-bin-trust-function-creators=true
environment:
MYSQL_DATABASE: test_db
MYSQL_ROOT_PASSWORD: rootpass
ports:
- "3306:3306"
volumes:
- "./docker/mysql/data:/var/lib/mysql"
- "./docker/mysql/my.cnf:/etc/mysql/conf.d/my.cnf"
- "./mysql_schema_v1.sql:/docker-entrypoint-initdb.d/mysql_schema_v1.sql"
It's a fairly simple setup and the output from CircleCI is positive until it reaches the docker exec, which I added to test the connection. Here is what the output from CircleCI says per step:
Start MySQL Docker:
#!/bin/bash -eo pipefail
docker-compose up -d
Creating network "project_default" with the default driver
Pulling mysql-migrate (mysql:8.0)...
8.0: Pulling from library/mysql
5158dd02: Pulling fs layer
f6778b18: Pulling fs layer
a6c74a04: Pulling fs layer
4028a805: Pulling fs layer
7163f0f6: Pulling fs layer
cb7f57e0: Pulling fs layer
7a431703: Pulling fs layer
5fe86aaf: Pulling fs layer
add93486: Pulling fs layer
960383f3: Pulling fs layer
80965951: Pulling fs layer
Digest: sha256:b17a66b49277a68066559416cf44a185cfee538d0e16b5624781019bc716c122 121B/121BkBBB
Status: Downloaded newer image for mysql:8.0
Creating mysql8_******_mysql ...
Creating mysql8_******_mysql ... done
So we know MySQL 8 was pulled fine (and therefore the previous step worked). Next step is to ask Docker what's running.
Check Docker MySQL:
#!/bin/bash -eo pipefail
docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
cb6b7941ad65 mysql:8.0 "docker-entrypoint.s…" 1 second ago Up Less than a second 0.0.0.0:3306->3306/tcp, 33060/tcp mysql8_test_mysql
CircleCI received exit code 0
Looks good so far. But now let's actually try to run a command against it via docker exec.
Query MySQL:
#!/bin/bash -eo pipefail
docker exec -it mysql8_test_mysql mysql mysql -h 127.0.0.1 --port 3306 -u root -prootpass -e "show databases;"
ERROR 2003 (HY000): Can't connect to MySQL server on '127.0.0.1:3306' (111)
Exited with code exit status 1
CircleCI received exit code 1
So now we can't connect to MySQL even though docker ps showed it up and running. I even tried adding an absurd step to wait in case MySQL needed more time:
- run:
name: Start MySQL docker
command: docker-compose up -d
- run:
name: Check Docker MySQL
command: docker ps
- run:
name: Wait Until Ready
command: sleep 120
- run:
name: Query MySQL
command: docker exec -it mysql8_test_mysql mysql mysql -h 127.0.0.1 --port 3306 -u root -prootpass -e "show databases;"
Of course adding a 2 minute wait for MySQL to spin up didn't help. Any ideas as to why this is so difficult in CircleCI?
Thanks in advance.
UPDATE 1: I can successfully start MySQL if I SSH into the job's server and run the same command myself:
docker-compose up
Then in another terminal run this:
docker exec -it mysql8_test_mysql mysql mysql -h localhost --port 3306 -u root -prootpass -e "show databases;"
+--------------------+
| Database |
+--------------------+
| information_schema |
| test_db |
| mysql |
| performance_schema |
| sys |
+--------------------+
So it is possible to start MySQL. It's just not working right when through job steps.
UPDATE 2: I moved the two minute wait between docker-compose up -d and docker ps and now it shows nothing is running. So the container must be starting then crashing and that's the reason for why it's not available moments later.
The cause of the problem was the volumes entry in my docker-compose.yml with this line:
- "./mysql_schema_v1.sql:/docker-entrypoint-initdb.d/mysql_schema_v1.sql"
The container appeared to be up when I checked immediately after docker-compose up -d but in actuality it would crash seconds later because CircleCI appears to have an issue with Docker volume, potentially related to this: https://discuss.circleci.com/t/docker-compose-doesnt-mount-volumes-with-host-files-with-circle-ci/19099.
To make it work I removed that volume entry and added run commands to copy and import the schema like so:
- run:
name: Start MySQL docker
command: docker-compose up -d
# Manually copy schema file instead of using docker-compose volumes (has issues with CircleCI)
- run:
name: Copy Schema
command: docker cp mysql_schema_v1.sql mysql8_mobile_mysql:docker-entrypoint-initdb.d/mysql_schema_v1.sql
- run:
name: Import Schema
command: docker exec mysql8_mobile_mysql /bin/sh -c 'mysql -u root -prootpass < docker-entrypoint-initdb.d/mysql_schema_v1.sql'
With this new setup I've been able to create the tables and connect to MySQL. However, there appears to be an issue running tests against MySQL causing hangups but that might be unrelated. I will follow up with more information, but at least I hope this can help someone else.

Docker compose: Enter Password access denied error or invalid file error [duplicate]

I am trying to create a container with a MySQL database and add a schema to these database.
My current Dockerfile is:
FROM mysql
MAINTAINER (me) <email>
# Copy the database schema to the /data directory
COPY files/epcis_schema.sql /data/epcis_schema.sql
# Change the working directory
WORKDIR data
CMD mysql -u $MYSQL_USER -p $MYSQL_PASSWORD $MYSQL_DATABASE < epcis_schema.sql
In order to create the container I am following the documentation provided on Docker and executing this command:
docker run --name ${CONTAINER_NAME} -e MYSQL_ROOT_PASSWORD=${DB_ROOT_PASSWORD} -e MYSQL_USER=${DB_USER} -e MYSQL_PASSWORD=${DB_USER_PASSWORD} -e MYSQL_DATABASE=${DB_NAME} -d mvpgomes/epcisdb
But when I execute this command the Container is not created and in the Container status it is possible to see that the CMD was not executed successfully, in fact only the mysql command is executed.
Anyway, is there a way to initialize the database with the schema or do I need to perform these operations manually?
I had this same issue where I wanted to initialize my MySQL Docker instance's schema, but I ran into difficulty getting this working after doing some Googling and following others' examples. Here's how I solved it.
1) Dump your MySQL schema to a file.
mysqldump -h <your_mysql_host> -u <user_name> -p --no-data <schema_name> > schema.sql
2) Use the ADD command to add your schema file to the /docker-entrypoint-initdb.d directory in the Docker container. The docker-entrypoint.sh file will run any files in this directory ending with ".sql" against the MySQL database.
Dockerfile:
FROM mysql:5.7.15
MAINTAINER me
ENV MYSQL_DATABASE=<schema_name> \
MYSQL_ROOT_PASSWORD=<password>
ADD schema.sql /docker-entrypoint-initdb.d
EXPOSE 3306
3) Start up the Docker MySQL instance.
docker-compose build
docker-compose up
Thanks to Setting up MySQL and importing dump within Dockerfile for clueing me in on the docker-entrypoint.sh and the fact that it runs both SQL and shell scripts!
I am sorry for this super long answer, but, you have a little way to go to get where you want. I will say that normally you wouldn't put the storage for the database in the same container as the database itself, you would either mount a host volume so that the data persists on the docker host, or, perhaps a container could be used to hold the data (/var/lib/mysql). Also, I am new to mysql, so, this might not be super efficient. That said...
I think there may be a few issues here. The Dockerfile is used to create an image. You need to execute the build step. At a minimum, from the directory that contains the Dockerfile you would do something like :
docker build .
The Dockerfile describes the image to create. I don't know much about mysql (I am a postgres fanboy), but, I did a search around the interwebs for 'how do i initialize a mysql docker container'. First I created a new directory to work in, I called it mdir, then I created a files directory which I deposited a epcis_schema.sql file which creates a database and a single table:
create database test;
use test;
CREATE TABLE testtab
(
id INTEGER AUTO_INCREMENT,
name TEXT,
PRIMARY KEY (id)
) COMMENT='this is my test table';
Then I created a script called init_db in the files directory:
#!/bin/bash
# Initialize MySQL database.
# ADD this file into the container via Dockerfile.
# Assuming you specify a VOLUME ["/var/lib/mysql"] or `-v /var/lib/mysql` on the `docker run` command…
# Once built, do e.g. `docker run your_image /path/to/docker-mysql-initialize.sh`
# Again, make sure MySQL is persisting data outside the container for this to have any effect.
set -e
set -x
mysql_install_db
# Start the MySQL daemon in the background.
/usr/sbin/mysqld &
mysql_pid=$!
until mysqladmin ping >/dev/null 2>&1; do
echo -n "."; sleep 0.2
done
# Permit root login without password from outside container.
mysql -e "GRANT ALL ON *.* TO root#'%' IDENTIFIED BY '' WITH GRANT OPTION"
# create the default database from the ADDed file.
mysql < /tmp/epcis_schema.sql
# Tell the MySQL daemon to shutdown.
mysqladmin shutdown
# Wait for the MySQL daemon to exit.
wait $mysql_pid
# create a tar file with the database as it currently exists
tar czvf default_mysql.tar.gz /var/lib/mysql
# the tarfile contains the initialized state of the database.
# when the container is started, if the database is empty (/var/lib/mysql)
# then it is unpacked from default_mysql.tar.gz from
# the ENTRYPOINT /tmp/run_db script
(most of this script was lifted from here: https://gist.github.com/pda/9697520)
Here is the files/run_db script I created:
# start db
set -e
set -x
# first, if the /var/lib/mysql directory is empty, unpack it from our predefined db
[ "$(ls -A /var/lib/mysql)" ] && echo "Running with existing database in /var/lib/mysql" || ( echo 'Populate initial db'; tar xpzvf default_mysql.tar.gz )
/usr/sbin/mysqld
Finally, the Dockerfile to bind them all:
FROM mysql
MAINTAINER (me) <email>
# Copy the database schema to the /data directory
ADD files/run_db files/init_db files/epcis_schema.sql /tmp/
# init_db will create the default
# database from epcis_schema.sql, then
# stop mysqld, and finally copy the /var/lib/mysql directory
# to default_mysql_db.tar.gz
RUN /tmp/init_db
# run_db starts mysqld, but first it checks
# to see if the /var/lib/mysql directory is empty, if
# it is it is seeded with default_mysql_db.tar.gz before
# the mysql is fired up
ENTRYPOINT "/tmp/run_db"
So, I cd'ed to my mdir directory (which has the Dockerfile along with the files directory). I then run the command:
docker build --no-cache .
You should see output like this:
Sending build context to Docker daemon 7.168 kB
Sending build context to Docker daemon
Step 0 : FROM mysql
---> 461d07d927e6
Step 1 : MAINTAINER (me) <email>
---> Running in 963e8de55299
---> 2fd67c825c34
Removing intermediate container 963e8de55299
Step 2 : ADD files/run_db files/init_db files/epcis_schema.sql /tmp/
---> 81871189374b
Removing intermediate container 3221afd8695a
Step 3 : RUN /tmp/init_db
---> Running in 8dbdf74b2a79
+ mysql_install_db
2015-03-19 16:40:39 12 [Note] InnoDB: Using atomics to ref count buffer pool pages
...
/var/lib/mysql/ib_logfile0
---> 885ec2f1a7d5
Removing intermediate container 8dbdf74b2a79
Step 4 : ENTRYPOINT "/tmp/run_db"
---> Running in 717ed52ba665
---> 7f6d5215fe8d
Removing intermediate container 717ed52ba665
Successfully built 7f6d5215fe8d
You now have an image '7f6d5215fe8d'. I could run this image:
docker run -d 7f6d5215fe8d
and the image starts, I see an instance string:
4b377ac7397ff5880bc9218abe6d7eadd49505d50efb5063d6fab796ee157bd3
I could then 'stop' it, and restart it.
docker stop 4b377
docker start 4b377
If you look at the logs, the first line will contain:
docker logs 4b377
Populate initial db
var/lib/mysql/
...
Then, at the end of the logs:
Running with existing database in /var/lib/mysql
These are the messages from the /tmp/run_db script, the first one indicates that the database was unpacked from the saved (initial) version, the second one indicates that the database was already there, so the existing copy was used.
Here is a ls -lR of the directory structure I describe above. Note that the init_db and run_db are scripts with the execute bit set:
gregs-air:~ gfausak$ ls -Rl mdir
total 8
-rw-r--r-- 1 gfausak wheel 534 Mar 19 11:13 Dockerfile
drwxr-xr-x 5 gfausak staff 170 Mar 19 11:24 files
mdir/files:
total 24
-rw-r--r-- 1 gfausak staff 126 Mar 19 11:14 epcis_schema.sql
-rwxr-xr-x 1 gfausak staff 1226 Mar 19 11:16 init_db
-rwxr-xr-x 1 gfausak staff 284 Mar 19 11:23 run_db
Another way based on a merge of serveral responses here before :
docker-compose file :
version: "3"
services:
db:
container_name: db
image: mysql
ports:
- "3306:3306"
environment:
- MYSQL_ROOT_PASSWORD=mysql
- MYSQL_DATABASE=db
volumes:
- /home/user/db/mysql/data:/var/lib/mysql
- /home/user/db/mysql/init:/docker-entrypoint-initdb.d/:ro
where /home/user.. is a shared folder on the host
And in the /home/user/db/mysql/init folder .. just drop one sql file, with any name, for example init.sql containing :
CREATE DATABASE mydb;
GRANT ALL PRIVILEGES ON mydb.* TO 'myuser'#'%' IDENTIFIED BY 'mysql';
GRANT ALL PRIVILEGES ON mydb.* TO 'myuser'#'localhost' IDENTIFIED BY 'mysql';
USE mydb
CREATE TABLE CONTACTS (
[ ... ]
);
INSERT INTO CONTACTS VALUES ...
[ ... ]
According to the official mysql documentation, you can put more than one sql file in the docker-entrypoint-initdb.d, they are executed in the alphabetical order
The other simple way, use docker-compose with the following lines:
mysql:
from: mysql:5.7
volumes:
- ./database:/tmp/database
command: mysqld --init-file="/tmp/database/install_db.sql"
Put your database schema into the ./database/install_db.sql. Every time when you build up your container, the install_db.sql will be executed.
I've tried Greg's answer with zero success, I must have done something wrong since my database had no data after all the steps: I was using MariaDB's latest image, just in case.
Then I decided to read the entrypoint for the official MariaDB image, and used that to generate a simple docker-compose file:
database:
image: mariadb
ports:
- 3306:3306
expose:
- 3306
volumes:
- ./docker/mariadb/data:/var/lib/mysql:rw
- ./database/schema.sql:/docker-entrypoint-initdb.d/schema.sql:ro
environment:
MYSQL_ALLOW_EMPTY_PASSWORD: "yes"
Now I'm able to persist my data AND generate a database with my own schema!
After Aug. 4, 2015, if you are using the official mysql Docker image, you can just ADD/COPY a file into the /docker-entrypoint-initdb.d/ directory and it will run with the container is initialized. See github: https://github.com/docker-library/mysql/commit/14f165596ea8808dfeb2131f092aabe61c967225 if you want to implement it on other container images
The easiest solution is to use tutum/mysql
Step1
docker pull tutum/mysql:5.5
Step2
docker run -d -p 3306:3306 -v /tmp:/tmp -e STARTUP_SQL="/tmp/to_be_imported.mysql" tutum/mysql:5.5
Step3
Get above CONTAINER_ID and then execute command docker logs to see the generated password information.
docker logs #<CONTAINER_ID>
Since I struggled with this problem recently, I'm adding a docker-compose file that really helped me:
version: '3.5'
services:
db:
image: mysql:5.7
container_name: db-container
command: mysqld --character-set-server=utf8mb4 --collation-server=utf8mb4_unicode_ci
volumes:
- "./scripts/schema.sql:/docker-entrypoint-initdb.d/1.sql"
- "./scripts/data.sql:/docker-entrypoint-initdb.d/2.sql"
environment:
MYSQL_ROOT_PASSWORD: password
MYSQL_DATABASE: test
MYSQL_USER: test-user
MYSQL_PASSWORD: password
ports:
- '3306:3306'
healthcheck:
test: "/usr/bin/mysql --user=root --password=password --execute \"SHOW DATABASES;\""
interval: 2s
timeout: 20s
retries: 10
You just need to create a scripts folder in the same location as the docker-compose.yml file above.
The scripts folder will have 2 files:
schema.sql: DDL scripts (create table...etc)
data.sql: Insert statements that you want to be executed right after schema creation.
After this, you can run the command below to erase any previous database info (for a fresh start):
docker-compose rm -v -f db && docker-compose up
For the ones not wanting to create an entrypoint script like me, you actually can start mysqld at build-time and then execute the mysql commands in your Dockerfile like so:
RUN mysqld_safe & until mysqladmin ping; do sleep 1; done && \
mysql -e "CREATE DATABASE somedb;" && \
mysql -e "CREATE USER 'someuser'#'localhost' IDENTIFIED BY 'somepassword';" && \
mysql -e "GRANT ALL PRIVILEGES ON somedb.* TO 'someuser'#'localhost';"
or source a prepopulated sql dump:
COPY dump.sql /SQL
RUN mysqld_safe & until mysqladmin ping; do sleep 1; done && \
mysql -e "SOURCE /SQL;"
RUN mysqladmin shutdown
The key here is to send mysqld_safe to background with the single & sign.
After to struggle a little bit with that, take a look the Dockerfile using named volumes (db-data).
It's important declare a plus at final part, where I mentioned that volume is [external]
All worked great this way!
version: "3"
services:
database:
image: mysql:5.7
container_name: mysql
ports:
- "3306:3306"
volumes:
- db-data:/docker-entrypoint-initdb.d
environment:
- MYSQL_DATABASE=sample
- MYSQL_ROOT_PASSWORD=root
volumes:
db-data:
external: true
Below is the Dockerfile I used successfully to install xampp, create a MariaDB with scheme and pre populated with the info used on local server(usrs,pics orders,etc..)
FROM ubuntu:14.04
COPY Ecommerce.sql /root
RUN apt-get update \
&& apt-get install wget -yq \
&& apt-get install nano \
&& wget https://www.apachefriends.org/xampp-files/7.1.11/xampp-linux-x64-7.1.11-0-installer.run \
&& mv xampp-linux-x64-7.1.11-0-installer.run /opt/ \
&& cd /opt/ \
&& chmod +x xampp-linux-x64-7.1.11-0-installer.run \
&& printf 'y\n\y\n\r\n\y\n\r\n' | ./xampp-linux-x64-7.1.11-0-installer.run \
&& cd /opt/lampp/bin \
&& /opt/lampp/lampp start \
&& sleep 5s \
&& ./mysql -uroot -e "CREATE DATABASE Ecommerce" \
&& ./mysql -uroot -D Ecommerce < /root/Ecommerce.sql \
&& cd / \
&& /opt/lampp/lampp reload \
&& mkdir opt/lampp/htdocs/Ecommerce
COPY /Ecommerce /opt/lampp/htdocs/Ecommerce
EXPOSE 80

Docker MySQL container unable to wait for MySQL server ready

I am attempting to create a docker container using the mysql:5 docker image. Once the MySQL server is up and running I want to create some databases, users and tables.
My Dockerfile looks like this;
FROM mysql:5
# Add db.data with correct permissions
RUN mkdir /server_data
WORKDIR /server_data
ADD --chown="root:root" ./db.data .
# Copy setup directory
COPY ./setup setup
COPY ./config /etc/mysql/conf.d
CMD ["./setup/setup.sh", "mysql", "-u", "root", "<", "./setup/schema.sql"]
My ./setup/setup.sh script looks like this;
#!/bin/bash
# wait-for-mysql.sh
set -e
shift
cmd="$#"
until mysql -uroot -c '\q'; do
>&2 echo "mysql is unavailable - sleeping"
sleep 1
done
>&2 echo "mysql is up - executing command"
exec $cmd
My docker-compose.yml looks like this;
version: "3"
services:
db:
build: ./db
volumes:
- data-db:/var/lib/mysql
environment:
- MYSQL_ROOT_PASSWORD=password
restart: always
container_name: db
volumes:
data-db:
When I run 'docker-compose up --build' I get the following output;
Building db
Step 1/7 : FROM mysql:5
---> 0d16d0a97dd1
Step 2/7 : RUN mkdir /server_data
---> Using cache
---> 087b5ded3a53
Step 3/7 : WORKDIR /server_data
---> Using cache
---> 5a32ea1b0a49
Step 4/7 : ADD --chown="root:root" ./db.data .
---> Using cache
---> 5d453c52a9f1
Step 5/7 : COPY ./setup setup
---> 9c5359818748
Step 6/7 : COPY ./config /etc/mysql/conf.d
---> b663a380813f
Step 7/7 : CMD ["./setup/setup.sh", "mysql", "-u", "root", "<", "./setup/schema.sql"]
---> Running in 4535b2620141
Removing intermediate container 4535b2620141
---> 2d2fb7e308ad
Successfully built 2d2fb7e308ad
Successfully tagged wasdbsandbox_db:latest
Recreating db ... done
Attaching to db
db | ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2)
db | mysql is unavailable - sleeping
db | ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2)
db | mysql is unavailable - sleeping
This goes on interminably until I press Ctrl + c.
If I comment out the CMD line in my Dockerfile the output from running 'docker-compose up --build' is the output of the ENTRYPOINT command that is defined in the official mysql Dockerfile.
Why is mysql never starting when I use my own CMD command?
This is supported already by the official mysql image. No need to make your own custom solution.
Look at the Docker Hub README under "Initializing a fresh instance".
You can see in the official image under the 5.7 Dockerfile (for example) that it copies in a ENTRYPOINT script. That script doesn't run at build time, but at run-time right before the CMD starts the daemon.
In that existing ENTRYPOINT script you'll see that it will process any files you put in /docker-entrypoint-initdb.d/
So in short, when you start a new container from that existing official image it will:
start the mysqld in local-only mode
creates default user, db, pw, etc.
runs any scripts you put in /docker-entrypoint-initdb.d/
stops the mysqld and hands off to the Dockerfile CMD
CMD will run the mysqld to listen on the network

Import into dockerized mariadb on initial build with script

I'm using MariaDB, but I think this could probably apply to MySQL as well.
I have a project that works off of MariaDB, and there is some initial setup for the database that needs to be done to create tables, insert initial data, etc. Based on other answers, I could normally do ADD dump.sql /docker-entrypoint-initdb.d, but I don't have a dump.sql -- instead what I have is a python script that connects to MariaDB directly and creates the tables and data.
I have a docker-compose.yml
version: '3'
services:
db:
build: ./db
ports:
- "3306:3306"
container_name: db
environment:
- MYSQL_ROOT_PASSWORD=root
web:
build: ./web
command: node /app/src/index.js
ports:
- "3000:3000"
links:
- db
"Web" is not so important right now since I just want to get db working.
The Dockerfile I've attempted for DB is:
# db/Dockerfile
FROM mariadb:10.3.2
RUN apt-get update && apt-get install -y python python-pip python-dev libmariadbclient-dev
RUN pip install requests mysql-python
ADD pricing_import.py /scripts/
RUN ["/bin/sh", "-c", "python /scripts/pricing_import.py"]
However this doesn't work for various reasons. I've gotten up to the point where pip install mysql-python doesn't compile:
_mysql.c:2005:41: error: 'MYSQL' has no member named 'reconnect'
if ( reconnect != -1 ) self->connection.reconnect = reconnect;
I think this has to do with the installation of mysql-python.
However before I go down the hole too far, I want to make sure my approach even makes sense since I don't even think the database will be started once I get to the ./pricing_import.py script and since it tries to connect to the database and runs queries, it probably won't work.
Since I can't get the python installation to work on the mariadb container anyway, I was also thinking about creating another docker-compose entry that depends on db and runs the python script on build to do the initial import during docker-compose build.
Are either of these approaches correct, or is there a better way to handle running an initialization script against MariaDB?
We use docker-compose healthcheck with combination of makefile and bash to handle running an initialization script. So your docker-compose.yml would look something like that:
version: '3'
services:
db:
build: ./db
ports:
- "3306:3306"
container_name: db
environment:
- MYSQL_ROOT_PASSWORD=root
healthcheck:
test: mysqlshow --defaults-extra-file=./database/my.cnf
interval: 5s
timeout: 60s
Where ./database/my.cnf is the config with credentials:
[client]
host = db
user = root
password = password
Then you can use this health-check.bash script to check the health:
#!/usr/bin/env bash
DATABASE_DOCKER_CONTAINER=$1
# Check on health database server before continuing
function get_service_health {
# https://gist.github.com/mixja/1ed1314525ba4a04807303dad229f2e1
docker inspect -f '{{if .State.Running}}{{ .State.Health.Status }}{{end}}' $DATABASE_DOCKER_CONTAINER
}
until [[ $(get_service_health) != starting ]];
do echo "database: ... Waiting on database Docker Instance to Start";
sleep 5;
done;
# Instance has finished starting, will be unhealthy until database finishes startup
MYSQL_HEALTH_CHECK_ATTEMPTS=12
until [[ $(get_service_health) == healthy ]]; do
echo "database: ... Waiting on database service"
sleep 5
if [[ $MYSQL_HEALTH_CHECK_ATTEMPTS == 0 ]];
then echo $DATABASE_DOCKER_CONTAINER ' failed health check (not running or unhealthy) - ' $(get_service_mysql_health)
exit 1
fi;
MYSQL_HEALTH_CHECK_ATTEMPTS=$((MYSQL_HEALTH_CHECK_ATTEMPTS-1))
done;
echo "Database is healthy"
Finally, you can use makefile to connect all things together. Something like that:
docker-up:
docker-compose up -d
db-health-check:
db/health-check.bash db
load-database:
docker run --rm --interactive --tty --network your_docker_network_name -v `pwd`:/application -w /application your_docker_db_image_name python /application/pricing_import.py
start: docker-up db-health-check load-database
Then start your app with make start.

How can I initialize a MySQL database with schema in a Docker container?

I am trying to create a container with a MySQL database and add a schema to these database.
My current Dockerfile is:
FROM mysql
MAINTAINER (me) <email>
# Copy the database schema to the /data directory
COPY files/epcis_schema.sql /data/epcis_schema.sql
# Change the working directory
WORKDIR data
CMD mysql -u $MYSQL_USER -p $MYSQL_PASSWORD $MYSQL_DATABASE < epcis_schema.sql
In order to create the container I am following the documentation provided on Docker and executing this command:
docker run --name ${CONTAINER_NAME} -e MYSQL_ROOT_PASSWORD=${DB_ROOT_PASSWORD} -e MYSQL_USER=${DB_USER} -e MYSQL_PASSWORD=${DB_USER_PASSWORD} -e MYSQL_DATABASE=${DB_NAME} -d mvpgomes/epcisdb
But when I execute this command the Container is not created and in the Container status it is possible to see that the CMD was not executed successfully, in fact only the mysql command is executed.
Anyway, is there a way to initialize the database with the schema or do I need to perform these operations manually?
I had this same issue where I wanted to initialize my MySQL Docker instance's schema, but I ran into difficulty getting this working after doing some Googling and following others' examples. Here's how I solved it.
1) Dump your MySQL schema to a file.
mysqldump -h <your_mysql_host> -u <user_name> -p --no-data <schema_name> > schema.sql
2) Use the ADD command to add your schema file to the /docker-entrypoint-initdb.d directory in the Docker container. The docker-entrypoint.sh file will run any files in this directory ending with ".sql" against the MySQL database.
Dockerfile:
FROM mysql:5.7.15
MAINTAINER me
ENV MYSQL_DATABASE=<schema_name> \
MYSQL_ROOT_PASSWORD=<password>
ADD schema.sql /docker-entrypoint-initdb.d
EXPOSE 3306
3) Start up the Docker MySQL instance.
docker-compose build
docker-compose up
Thanks to Setting up MySQL and importing dump within Dockerfile for clueing me in on the docker-entrypoint.sh and the fact that it runs both SQL and shell scripts!
I am sorry for this super long answer, but, you have a little way to go to get where you want. I will say that normally you wouldn't put the storage for the database in the same container as the database itself, you would either mount a host volume so that the data persists on the docker host, or, perhaps a container could be used to hold the data (/var/lib/mysql). Also, I am new to mysql, so, this might not be super efficient. That said...
I think there may be a few issues here. The Dockerfile is used to create an image. You need to execute the build step. At a minimum, from the directory that contains the Dockerfile you would do something like :
docker build .
The Dockerfile describes the image to create. I don't know much about mysql (I am a postgres fanboy), but, I did a search around the interwebs for 'how do i initialize a mysql docker container'. First I created a new directory to work in, I called it mdir, then I created a files directory which I deposited a epcis_schema.sql file which creates a database and a single table:
create database test;
use test;
CREATE TABLE testtab
(
id INTEGER AUTO_INCREMENT,
name TEXT,
PRIMARY KEY (id)
) COMMENT='this is my test table';
Then I created a script called init_db in the files directory:
#!/bin/bash
# Initialize MySQL database.
# ADD this file into the container via Dockerfile.
# Assuming you specify a VOLUME ["/var/lib/mysql"] or `-v /var/lib/mysql` on the `docker run` command…
# Once built, do e.g. `docker run your_image /path/to/docker-mysql-initialize.sh`
# Again, make sure MySQL is persisting data outside the container for this to have any effect.
set -e
set -x
mysql_install_db
# Start the MySQL daemon in the background.
/usr/sbin/mysqld &
mysql_pid=$!
until mysqladmin ping >/dev/null 2>&1; do
echo -n "."; sleep 0.2
done
# Permit root login without password from outside container.
mysql -e "GRANT ALL ON *.* TO root#'%' IDENTIFIED BY '' WITH GRANT OPTION"
# create the default database from the ADDed file.
mysql < /tmp/epcis_schema.sql
# Tell the MySQL daemon to shutdown.
mysqladmin shutdown
# Wait for the MySQL daemon to exit.
wait $mysql_pid
# create a tar file with the database as it currently exists
tar czvf default_mysql.tar.gz /var/lib/mysql
# the tarfile contains the initialized state of the database.
# when the container is started, if the database is empty (/var/lib/mysql)
# then it is unpacked from default_mysql.tar.gz from
# the ENTRYPOINT /tmp/run_db script
(most of this script was lifted from here: https://gist.github.com/pda/9697520)
Here is the files/run_db script I created:
# start db
set -e
set -x
# first, if the /var/lib/mysql directory is empty, unpack it from our predefined db
[ "$(ls -A /var/lib/mysql)" ] && echo "Running with existing database in /var/lib/mysql" || ( echo 'Populate initial db'; tar xpzvf default_mysql.tar.gz )
/usr/sbin/mysqld
Finally, the Dockerfile to bind them all:
FROM mysql
MAINTAINER (me) <email>
# Copy the database schema to the /data directory
ADD files/run_db files/init_db files/epcis_schema.sql /tmp/
# init_db will create the default
# database from epcis_schema.sql, then
# stop mysqld, and finally copy the /var/lib/mysql directory
# to default_mysql_db.tar.gz
RUN /tmp/init_db
# run_db starts mysqld, but first it checks
# to see if the /var/lib/mysql directory is empty, if
# it is it is seeded with default_mysql_db.tar.gz before
# the mysql is fired up
ENTRYPOINT "/tmp/run_db"
So, I cd'ed to my mdir directory (which has the Dockerfile along with the files directory). I then run the command:
docker build --no-cache .
You should see output like this:
Sending build context to Docker daemon 7.168 kB
Sending build context to Docker daemon
Step 0 : FROM mysql
---> 461d07d927e6
Step 1 : MAINTAINER (me) <email>
---> Running in 963e8de55299
---> 2fd67c825c34
Removing intermediate container 963e8de55299
Step 2 : ADD files/run_db files/init_db files/epcis_schema.sql /tmp/
---> 81871189374b
Removing intermediate container 3221afd8695a
Step 3 : RUN /tmp/init_db
---> Running in 8dbdf74b2a79
+ mysql_install_db
2015-03-19 16:40:39 12 [Note] InnoDB: Using atomics to ref count buffer pool pages
...
/var/lib/mysql/ib_logfile0
---> 885ec2f1a7d5
Removing intermediate container 8dbdf74b2a79
Step 4 : ENTRYPOINT "/tmp/run_db"
---> Running in 717ed52ba665
---> 7f6d5215fe8d
Removing intermediate container 717ed52ba665
Successfully built 7f6d5215fe8d
You now have an image '7f6d5215fe8d'. I could run this image:
docker run -d 7f6d5215fe8d
and the image starts, I see an instance string:
4b377ac7397ff5880bc9218abe6d7eadd49505d50efb5063d6fab796ee157bd3
I could then 'stop' it, and restart it.
docker stop 4b377
docker start 4b377
If you look at the logs, the first line will contain:
docker logs 4b377
Populate initial db
var/lib/mysql/
...
Then, at the end of the logs:
Running with existing database in /var/lib/mysql
These are the messages from the /tmp/run_db script, the first one indicates that the database was unpacked from the saved (initial) version, the second one indicates that the database was already there, so the existing copy was used.
Here is a ls -lR of the directory structure I describe above. Note that the init_db and run_db are scripts with the execute bit set:
gregs-air:~ gfausak$ ls -Rl mdir
total 8
-rw-r--r-- 1 gfausak wheel 534 Mar 19 11:13 Dockerfile
drwxr-xr-x 5 gfausak staff 170 Mar 19 11:24 files
mdir/files:
total 24
-rw-r--r-- 1 gfausak staff 126 Mar 19 11:14 epcis_schema.sql
-rwxr-xr-x 1 gfausak staff 1226 Mar 19 11:16 init_db
-rwxr-xr-x 1 gfausak staff 284 Mar 19 11:23 run_db
Another way based on a merge of serveral responses here before :
docker-compose file :
version: "3"
services:
db:
container_name: db
image: mysql
ports:
- "3306:3306"
environment:
- MYSQL_ROOT_PASSWORD=mysql
- MYSQL_DATABASE=db
volumes:
- /home/user/db/mysql/data:/var/lib/mysql
- /home/user/db/mysql/init:/docker-entrypoint-initdb.d/:ro
where /home/user.. is a shared folder on the host
And in the /home/user/db/mysql/init folder .. just drop one sql file, with any name, for example init.sql containing :
CREATE DATABASE mydb;
GRANT ALL PRIVILEGES ON mydb.* TO 'myuser'#'%' IDENTIFIED BY 'mysql';
GRANT ALL PRIVILEGES ON mydb.* TO 'myuser'#'localhost' IDENTIFIED BY 'mysql';
USE mydb
CREATE TABLE CONTACTS (
[ ... ]
);
INSERT INTO CONTACTS VALUES ...
[ ... ]
According to the official mysql documentation, you can put more than one sql file in the docker-entrypoint-initdb.d, they are executed in the alphabetical order
The other simple way, use docker-compose with the following lines:
mysql:
from: mysql:5.7
volumes:
- ./database:/tmp/database
command: mysqld --init-file="/tmp/database/install_db.sql"
Put your database schema into the ./database/install_db.sql. Every time when you build up your container, the install_db.sql will be executed.
I've tried Greg's answer with zero success, I must have done something wrong since my database had no data after all the steps: I was using MariaDB's latest image, just in case.
Then I decided to read the entrypoint for the official MariaDB image, and used that to generate a simple docker-compose file:
database:
image: mariadb
ports:
- 3306:3306
expose:
- 3306
volumes:
- ./docker/mariadb/data:/var/lib/mysql:rw
- ./database/schema.sql:/docker-entrypoint-initdb.d/schema.sql:ro
environment:
MYSQL_ALLOW_EMPTY_PASSWORD: "yes"
Now I'm able to persist my data AND generate a database with my own schema!
After Aug. 4, 2015, if you are using the official mysql Docker image, you can just ADD/COPY a file into the /docker-entrypoint-initdb.d/ directory and it will run with the container is initialized. See github: https://github.com/docker-library/mysql/commit/14f165596ea8808dfeb2131f092aabe61c967225 if you want to implement it on other container images
The easiest solution is to use tutum/mysql
Step1
docker pull tutum/mysql:5.5
Step2
docker run -d -p 3306:3306 -v /tmp:/tmp -e STARTUP_SQL="/tmp/to_be_imported.mysql" tutum/mysql:5.5
Step3
Get above CONTAINER_ID and then execute command docker logs to see the generated password information.
docker logs #<CONTAINER_ID>
Since I struggled with this problem recently, I'm adding a docker-compose file that really helped me:
version: '3.5'
services:
db:
image: mysql:5.7
container_name: db-container
command: mysqld --character-set-server=utf8mb4 --collation-server=utf8mb4_unicode_ci
volumes:
- "./scripts/schema.sql:/docker-entrypoint-initdb.d/1.sql"
- "./scripts/data.sql:/docker-entrypoint-initdb.d/2.sql"
environment:
MYSQL_ROOT_PASSWORD: password
MYSQL_DATABASE: test
MYSQL_USER: test-user
MYSQL_PASSWORD: password
ports:
- '3306:3306'
healthcheck:
test: "/usr/bin/mysql --user=root --password=password --execute \"SHOW DATABASES;\""
interval: 2s
timeout: 20s
retries: 10
You just need to create a scripts folder in the same location as the docker-compose.yml file above.
The scripts folder will have 2 files:
schema.sql: DDL scripts (create table...etc)
data.sql: Insert statements that you want to be executed right after schema creation.
After this, you can run the command below to erase any previous database info (for a fresh start):
docker-compose rm -v -f db && docker-compose up
For the ones not wanting to create an entrypoint script like me, you actually can start mysqld at build-time and then execute the mysql commands in your Dockerfile like so:
RUN mysqld_safe & until mysqladmin ping; do sleep 1; done && \
mysql -e "CREATE DATABASE somedb;" && \
mysql -e "CREATE USER 'someuser'#'localhost' IDENTIFIED BY 'somepassword';" && \
mysql -e "GRANT ALL PRIVILEGES ON somedb.* TO 'someuser'#'localhost';"
or source a prepopulated sql dump:
COPY dump.sql /SQL
RUN mysqld_safe & until mysqladmin ping; do sleep 1; done && \
mysql -e "SOURCE /SQL;"
RUN mysqladmin shutdown
The key here is to send mysqld_safe to background with the single & sign.
After to struggle a little bit with that, take a look the Dockerfile using named volumes (db-data).
It's important declare a plus at final part, where I mentioned that volume is [external]
All worked great this way!
version: "3"
services:
database:
image: mysql:5.7
container_name: mysql
ports:
- "3306:3306"
volumes:
- db-data:/docker-entrypoint-initdb.d
environment:
- MYSQL_DATABASE=sample
- MYSQL_ROOT_PASSWORD=root
volumes:
db-data:
external: true
Below is the Dockerfile I used successfully to install xampp, create a MariaDB with scheme and pre populated with the info used on local server(usrs,pics orders,etc..)
FROM ubuntu:14.04
COPY Ecommerce.sql /root
RUN apt-get update \
&& apt-get install wget -yq \
&& apt-get install nano \
&& wget https://www.apachefriends.org/xampp-files/7.1.11/xampp-linux-x64-7.1.11-0-installer.run \
&& mv xampp-linux-x64-7.1.11-0-installer.run /opt/ \
&& cd /opt/ \
&& chmod +x xampp-linux-x64-7.1.11-0-installer.run \
&& printf 'y\n\y\n\r\n\y\n\r\n' | ./xampp-linux-x64-7.1.11-0-installer.run \
&& cd /opt/lampp/bin \
&& /opt/lampp/lampp start \
&& sleep 5s \
&& ./mysql -uroot -e "CREATE DATABASE Ecommerce" \
&& ./mysql -uroot -D Ecommerce < /root/Ecommerce.sql \
&& cd / \
&& /opt/lampp/lampp reload \
&& mkdir opt/lampp/htdocs/Ecommerce
COPY /Ecommerce /opt/lampp/htdocs/Ecommerce
EXPOSE 80