Docker compose MySql initialisation scripts not executing - mysql

I am having a problem having the mysql container to run my initialisation scripts.
I have two files create.sql and insert.sql, which I use to initialise the database.
I create the images using the command docker-compose.yml and it runs successfully and creates the images.
I am facing two problems.
When I run the docker-compose up command, the mysql container is created and started successfully. However the two initialisation scripts (create.sql and insert.sql) don't run on the database.
I explicitly use the docker run command to run the created mysql container. In this scenario the initialisation scripts run successfully.
I am using Docker version 18.09.0 and docker-compose version 1.23.1 and ubuntu 16.04 LTS
I am new to docker and can't seem to figure out the problem.
The following are the files I am using to create images.
docker-compose.yml file.
version: '3'
services:
demo-mysql:
image: demo-mysql
build: ./demo-mysql
volumes:
- /mnt/data/mysql-data:/var/lib/mysql
ports:
- 3306:3306
environment:
- MYSQL_ROOT_PASSWORD=root
- MYSQL_DATABASE=demo
- MYSQL_PASSWORD=root
demo-api:
image: demo-api-1.0
build: ./api
depends_on:
- demo-mysql
ports:
- 8080:8080
environment:
- DATABASE_HOST=demo-mysql
- DATABASE_USER=root
- DATABASE_PASSWORD=root
- DATABASE_NAME=demo
- DATABASE_PORT=3306
demo1-app:
image: demo1-app-1.0
build: ./demo1
depends_on:
- demo-mysql
ports:
- 8090:8090
environment:
- DATABASE_HOST=demo-mysql
- DATABASE_USER=root
- DATABASE_PASSWORD=root
- DATABASE_NAME=demo
- DATABASE_PORT=3306
The following is the Dockerfile for the spring boot project
FROM java:8
VOLUME /tmp
ARG DATA_PATH=/src/main/resources
ARG APP_PORT=8080
EXPOSE ${APP_PORT}
ADD /build/libs/demo-api.jar demo-api.jar
ENTRYPOINT ["java","-jar","demo-api.jar"]
The following is the Dockerfile I used to create my mysql image
FROM mysql:5.7
ENV MYSQL_DATABASE=demo \
MYSQL_USER=root \
MYSQL_ROOT_PASSWORD=root
ADD ./1.0/create.sql /docker-entrypoint-initdb.d
ADD ./1.0/insert.sql /docker-entrypoint-initdb.d
EXPOSE 3306

From documentation (https://hub.docker.com/_/mysql/)
Initializing a fresh instance
When a container is started for the first time, a new database with
the specified name will be created and initialized with the provided
configuration variables. Furthermore, it will execute files with
extensions .sh, .sql and .sql.gz that are found in
/docker-entrypoint-initdb.d.
I suspect that, because of the persisted volume
volumes:
- /mnt/data/mysql-data:/var/lib/mysql
when docker starts the mysql image, there is already a DB. So the image isn't "fresh" and the scripts are not run.
Update:
we can confirm this suspect looking at the source code of the docker-entrypoint.sh here: https://github.com/docker-library/mysql/blob/696fc899126ae00771b5d87bdadae836e704ae7d/5.7/docker-entrypoint.sh
if [ ! -d "$DATADIR/mysql" ]; then
...
...
ls /docker-entrypoint-initdb.d/ > /dev/null
for f in /docker-entrypoint-initdb.d/*; do
process_init_file "$f" "${mysql[#]}"
done
The scripts run only if the "$DATADIR/mysql" is not present already.
btw, I personally consider a better design to have the "application" create the database schema, preload the required application data, manage schema migrations etc... at startup, but this another topic :)

Related

Error connecting Mysql from Go REST API with Docker Compose

I'm very new to Docker, and I'm trying to dockerize a Go REST API and MySQL database to communicate with each other using Docker Compose. I am getting the error [main] Error 1049: Unknown database 'puapp'
Docker compose:
version: '3'
services:
db:
build: ./mysql/
restart: always
environment:
- MYSQL_ROOT_PASSWORD=root
volumes:
- db_volume:/var/lib/mysql
api-service:
restart: always
build: ./
ports:
- "8080:80"
environment:
- DB_USER=root
- DB_PASS=root
- DB_ADDRESS=db:3306
- DB_PROTOCOL=tcp
- DB_NAME=puapp
depends_on:
- db
links:
- db
volumes:
db_volume:
Dockerfile for go service:
# syntax=docker/dockerfile:1
# Build stage
FROM golang:1.16-alpine AS builder
WORKDIR /app
COPY . .
RUN go mod download
WORKDIR /app/src/main
RUN go build -o restserv
# Run stage
FROM alpine:3.13
WORKDIR /app
COPY --from=builder /app/src/main/restserv .
EXPOSE 8080
CMD "./restserv"
Dockerfile for MySQL:
FROM mysql:latest
ADD dump.sql /docker-entrypoint-initdb.d
Full code - https://github.com/bens-schreiber/restservproj
Let me know if I need to add anything
Containers will be having their own ip addresses, so API container won't be able to access mysql container over 127.0.0.1. As mentioned in the comments, you want to utilize container's names to addresses from container from another. See this page for details.

MariaDB volume in docker-compose clears out data

I'm using the yobasystems/alpine-mariadb docker image to run an instance for a development environment. I'm mounting the data directory for MySQL to a docker volume and this has worked in the past. Every so often I lose data but not the table structure and I cannot work out why.
db:
image: yobasystems/alpine-mariadb
restart: always
environment:
- MYSQL_ROOT_PASSWORD=password
- MYSQL_DATABASE=database
- MYSQL_USER=user
- MYSQL_PASSWORD=password
ports:
- "33333:3306"
volumes:
- mariadb:/var/lib/mysql
I suspect that in your case the volume is getting removed(may be via docker-compose down -v or dockere-compose rm -v).
Please specify that the volume is external using -
volumes:
mariadb:
external: true
From docker docs - external: If set to true, specifies that this volume has been created outside of Compose. docker-compose up does not attempt to create it, and raises an error if it doesn’t exist.
You may create the volume prior to docker-compose up with docker volume create mariadb

docker-compose mysql init sql is not executed

I am trying to set up a mysql docker container and execute init sql script. Unfortunately the sql script is not executed. What am I doing wrong?
version: '3.3'
services:
api:
container_name: 'api'
build: './api'
ports:
- target: 8080
published: 8888
protocol: tcp
mode: host
volumes:
- './api:/go/src/app'
depends_on:
- 'mysql'
mysql:
image: 'mysql:latest'
container_name: 'mysql'
volumes:
- ./db_data:/var/lib/mysql:rw
- ./database/init.sql:/docker-entrypoint-initdb.d/init.sql:ro
restart: always
environment:
MYSQL_USER: test
MYSQL_PASSWORD: test
MYSQL_ROOT_PASSWORD: test
MYSQL_DATABASE: test
ports:
- '3306:3306'
volumes:
db_data:
I execute file with docker-compose up -d --build
The docker-entrypoint-initdb.d folder will only be run once while the container is created (instantiated) so you actually have to do a docker-compose down -v to re-activate this for the next run.
If you want to be able to add sql files at any moment you can look here at a specialized MySql docker image... http://ivo2u.nl/o4
Update for M1 arch:
Here an almost drop-in replacement in MariaDB: http://ivo2u.nl/V1
Many containerized applications, especially stateful ones, have a way of running init scripts (like the sql scripts here) and they are supposed to run only once.
And since they are stateful, the volumes are a source of truth for the containers on whether to run the init scripts or not on container restart.
Like in your case, deleting the folder used for bind mount or using a new named volume should re-run any init scripts present.
These scripts run when you create the container, not every time you start it.
You can docker-compose up --force-recreate mysql to force those scripts to re-run.
Additionally, if you have a volume like this ./db_data:/var/lib/mysql:rw, then you also need to remove ./db_data before recreating the container.
I'm not a docker expert, but this worked for me.

Docker-compose mysql mount volume turns sql into folder

I am trying to use Docker to create a set of containers (wordpress and MySQL) that will help my local development with Wordpress. As we are running a live database, I want to mount a dump.sql file into the Docker mysql container. Below is my .yml file.
version: '2'
services:
db:
image: mysql:latest
volumes:
- ./data:/docker-entrypoint-initdb.d #./data holders my dump.sql file
environment:
MYSQL_ROOT_PASSWORD: wordpress
MYSQL_DATABASE: wordpress
MYSQL_USER: wordpress
MYSQL_PASSWORD: wordpress
wordpress:
depends_on:
- db
image: wordpress:latest
ports:
- "8000:80"
restart: always
environment:
WORDPRESS_DB_HOST: db:3306
WORDPRESS_DB_PASSWORD: wordpress
volumes:
- ./wp-content/themes/portalV3:/var/www/html/wp-content/themes/portalV3
- ./wp-content/plugins:/var/www/html/wp-content/plugins
- ./wp-content/uploads:/var/www/html/wp-content/uploads
Everything works, but after ~10 seconds the docker container for mysql crashes. Going through the logs, I get the following error:
/usr/local/bin/docker-entrypoint.sh: running /docker-entrypoint-initdb.d/dump.sql
mysql: [Warning] Using a password on the command line interface can be insecure.
ERROR: Can't initialize batch_readline - may be the input source is a directory or a block device.
On closer inspection (attaching to the rebooted mysql container) I see that indeed my dump.sql file wasn't transferred to the container, but a folder with the same name was created in /docker-entrypoint-initdb.d.
Can anyone help me understand how I get docker-compose to copy my dump.sql file and import into the database?
Cheers,
Pieter
The problem you got with docker-entrypoint-initdb.d is that because your source 'data' is a directory and not a file, The destination file (docker-entrypoint-initdb.d) must be a directory too. And vice versa.
So either do
volumes:
- ./data:/docker-entrypoint-initdb.d/
or
volumes:
- ./data/mydump.sql:/docker-entrypoint-initdb.d/mydump.sql
Yes, that is how you should mount the .sql or .sh files i.e by adding a volume by mapping the SQL or .sh files to the docker container's docker-entrypoint-initdb.d folder. But, it's raising an error for some strange reason maybe because the MySQL docker version is old.
You could solve this by creating a custom image i.e,
Dockerfile
FROM mysql:5.7
COPY init.sql /docker-entrypoint-initdb.d/
It creates an image and also helps in running a init script while starting the container.
To use this in a compose file, put your SQL files and Dockerfile in a folder.
database
|---init.sql
|---Dockerfile
docker-compose.yml
version: '3'
services:
mysqldb:
image: mysqldb
build: ./database
container_name: mysql
ports:
- "3306:3306"
environment:
- MYSQL_ROOT_PASSWORD=root
- MYSQL_USER=test
- MYSQL_PASSWORD=root
- MYSQL_DATABASE=test
By this, you could configure the environment variables easily.

Load SQL Create DB file to Docker

I am having trouble to load in my SQL File to create the schema and the associated tables, I have also tried to use the 'volumes' option in the compose file to copy the sql file into the 'docker-entrypoint-initdb.d' directory but that fails to so anything.
Table + Schema: https://pastebin.com/RTBBGZhn
MySQL Dump: https://pastebin.com/6ApQwt1F
Docker Compose File
version: '2'
services:
melissabot:
image: melissabot
build: .
ports:
- 7000:7000
- 7070:7070
depends_on:
- mysqlMelissa
links:
- mysqlMelissa:db
mysqlMelissa:
image: mysql
build: ${PWD}/Docker/DB/
environment:
- MYSQL_ROOT_PASSWORD=root
# - MYSQL_DATABASE=MelissaBot
ports:
- "3306:3306"
phpmyadmin:
image: phpmyadmin/phpmyadmin
container_name: phpmyadmin
restart: always
ports:
- 8080:80
volumes:
- /sessions
links:
- mysqlMelissa:db
Dockerfile
FROM mysql:5.6
ENV MYSQL_ROOT_PASSWORD=root
# ENV MYSQL_DATABASE=MelissaBot
COPY setup.sh /mysql/setup.sh
COPY dump.sql /mysql/Melissa.sql
RUN chmod +x /mysql/setup.sh
RUN /mysql/setup.sh
setup.sh
#!/bin/bash
set -e
service mysql start
mysql -u root MelissaBot < /mysql/Melissa.sql
service mysql stop
You are doing it wrongly. There is no init system inside docker. So you should not be using server start/stop inside a container. Look at the official image documentation always
https://hub.docker.com/_/mysql
Initializing a fresh instance
When a container is started for the first time, a new database with the specified name will be created and initialized with the provided configuration variables. Furthermore, it will execute files with extensions .sh, .sql and .sql.gz that are found in /docker-entrypoint-initdb.d. Files will be executed in alphabetical order. You can easily populate your mysql services by mounting a SQL dump into that directory and provide custom images with contributed data. SQL files will be imported by default to the database specified by the MYSQL_DATABASE variable.
So you can either copy your default dump files to /docker-entrypoint-initdb.d or mount them using a volume mount when running the container. So you chuck your setup.sh and change your Dockerfile to something like below
FROM mysql:5.6
ENV MYSQL_ROOT_PASSWORD=root
ENV MYSQL_DATABASE=MelissaBot
COPY dump.sql /docker-entrypoint-initdb.d/
But I would rather prefer the approach of volume mounting it inside my compose file
mysqlMelissa:
image: mysql:5.6
environment:
- MYSQL_ROOT_PASSWORD=root
- MYSQL_DATABASE=MelissaBot
ports:
- "3306:3306"
volumes:
- dump.sql:/docker-entrypoint-initdb.d/dump.sql