I am new to docker. I am on a windows 7 machine and using docker toolbox.
I am trying to write a docker-compose.yml for MySQL which creates a database and runs 2 scripts (create table and insert)
version: '3'
services:
mysql-image:
image: mysql:5.7
environment:
MYSQL_ROOT_PASSWORD: root
MYSQL_DATABASE: myDatabase
MYSQL_USER: test
MYSQL_PASSWORD: pwtest
ports:
- "3306:3306"
volumes:
- ./sqlscripts/:/docker-entrypoint-initdb.d/
volumes:
sqlscripts:
I can connect to the database, but the problem I have is running the scripts.
mostly I run into the following error:
mysql-image_1 | mysql: [Warning] Using a password on the command line
interface can be insecure.
mysql-image_1 | ERROR: Can't initialize
batch_readline - may be the input source is a directory or a block
device. docker_mysql-image_1 exited with code 1
I searched for a while, trying to get it work but at this point I do not know what I am doing wrong. This is one of my .sql scripts which I am trying to run it when the docker container starts.
createTablePerson.sql
CREATE TABLE `myDatabase`.`Person` (
`idPerson` INT NOT NULL AUTO_INCREMENT,
`Name` VARCHAR(45) NOT NULL,
`age` INT NOT NULL,
PRIMARY KEY (`idPerson`));
Thanks.
I investigated further and it seems is because I am on Windows 7 and I have to use docker-toolbox which runs in a VirtualBox VM. It seems that the location I had my scripts, is not shared between windows host and docker-toolbox. I noticed the files were copied in the container but they where considered as empty directories. I moved my scripts to user location and it seems the scripts are copied to the container and run perfectly. Now I am searching for a way to try make other paths visible. I prefer to have my scripts under my java project.
Related
Hi I am building a service in which I need a Mysql/MariaDB database. I have been googling different solutions and I got the db started with a database created thanks to a guide a was following (never found the link again unfortunately).
Problem
The problem I am having is that the tables are not being created. I added the sql-scema file to /docker-entrypoint-initdb.d/ (you can check it down in the docker file) but it doesnt seem to be executing it (I have tried with both copy and ADD commands).
Current output
This is my current console output from the container:
[![image][1]][1]
The database is created but the SOW TABLES; command returns Empty Set.
Desired output
Since this db is going to be a service differents scripts connect to (currently python), I need to be able to create the db and the sql schema (tables, triggers, etc...) so my team can work with same configuration.
Some of the solutions I have tried (I cant find all the links i have visited only a few)
How to import a mysql dump file into a Docker mysql container
mysql:5.7 docker allow access from all hosts and create DB
Can't connect to mariadb outside of docker container
Mariadb tables are deleted when use volume in docker-compose
Project structure
The structure is pretty simple I am using the following docker-compose.yml
Docker-compose
I still have to try if the MARIADB_ enviroment variables are necessary here.
version: '3'
services:
db-mysql:
#image: mysql/mysql-server:latest
build: ./mysql-db
restart: always
container_name : db-music
ports:
- '3306:3306'
environment:
MYSQL_ROOT_PASSWORD: pwd
MYSQL_DATABASE : audio_service
MYSQL_USER : user
MYSQL_PASSWORD : password
environment:
MARIADB_ROOT_PASSWORD: pwd
MARIADB_DATABASE : audio_service
MARIADB_USER : user
MARIADB_PASSWORD : password
#https://stackoverflow.com/questions/29145370/how-can-i-initialize-a-mysql-database-with-schema-in-a-docker-container?rq=1
expose:
- '3306:3306'
volumes:
- type: bind
source : E:\python-code\Rockstar\volume\mysql
target : /var/lib/mysql
#- type: bind
#source : E:\python-code\Rockstar\mysql-db\sql_scripts\tables.sql
#target : /docker-entrypoint-initdb.d/init.sql
networks:
net:
ipam:
driver: default
config:
- subnet: 212.172.1.0/30
host:
name: host
external: true
Dockerfile
FROM mariadb:latest as builder
# That file does the DB initialization but also runs mysql daemon, by removing the last line it will only init
RUN ["sed", "-i", "s/exec \"$#\"/echo \"not running $#\"/", "/usr/local/bin/docker-entrypoint.sh"]
# needed for intialization
ENV MYSQL_ROOT_PASSWORD=root
ENV MYSQL_ROOT_PASSWORD = pwd
ENV MYSQL_DATABASE = audio_service
ENV MYSQL_USER = user
ENV MYSQL_PASSWORD = password
COPY sql_scripts/tables.sql /docker-entrypoint-initdb.d/
# Need to change the datadir to something else that /var/lib/mysql because the parent docker file defines it as a volume.
# https://docs.docker.com/engine/reference/builder/#volume :
# Changing the volume from within the Dockerfile: If any build steps change the data within the volume after
# it has been declared, those changes will be discarded.
RUN ["/usr/local/bin/docker-entrypoint.sh", "mysqld", "--datadir", "/initialized-db", "--aria-log-dir-path", "/initialized-db"]
FROM mariadb:latest
# needed for intialization
ENV MARIADB_ROOT_PASSWORD=root
ENV MARIADB_ROOT_PASSWORD = pwd
ENV MARIADB_DATABASE = audio_service
ENV MARIADB_USER = user
ENV MARIADB_PASSWORD = password
COPY --from=builder /initialized-db /var/lib/mysql
EXPOSE 3306
SQL schema file
create database audio_service;
use audio_service;
CREATE TABLE audio
(
audio_id BINARY(16),
title TEXT NOT NULL UNIQUE,
content MEDIUMBLOB NOT NULL,
PRIMARY KEY (audio_id)
) COMMENT='this table stores sons';
DELIMITER ;;
CREATE TRIGGER `audio_before_insert`
BEFORE INSERT ON `audio` FOR EACH ROW
BEGIN
IF new.audio_id IS NULL THEN
SET new.audio_id = UUID_TO_BIN(UUID(), TRUE);
END IF;
END;;
DELIMITER ;
There is no need to build your own image since the official mysql / mariadb images are already well suited. You only need to run them with the following as explained in their image documentations:
environment variables to initialize an new database with a respective user on the first run
a volume at /var/lib/mysql to persist the data
any initialization/sql scripts mounted into /docker-entrypoint-initdb.d
So storing your SQL* into a schema.sql file right next to the docker-compose.yml the following is enough to achieve what you want:
# docker-compose.yml
services:
db:
image: mariadb
environment:
MARIADB_ROOT_PASSWORD: pwd
MARIADB_DATABASE: audio_service
MARIADB_USER: user
MARIADB_PASSWORD: password
volumes:
# persist data files into `datadir` volume managed by docker
- datadir:/var/lib/mysql
# bind-mount any sql files that should be run while initializing
- ./schema.sql:/docker-entrypoint-initdb.d/schema.sql
volumes:
datadir:
*note that you can remove the CREATE DATABASE and USE statements from your schema.sql since these will be automatically done by the init script for you anyway
There are two reasons that your own setup isn't working as expected:
the line COPY --from=builder /initialized-db /var/lib/mysql won't work as expected for the same reason you described in your comment a bit above it: /var/lib/mysql is a volume and thus no new files a stored in it in the build steps after it was defined.
you are bind-mounting E:\python-code\Rockstar\volume\mysql to /var/lib/mysql in your docker-compose.yml.
But this will effectively override any contents of /var/lib/mysql of the image, i.e. although your own image built from your Dockerfile does include an initialized database this is overwritten by the contents of E:\python-code\Rockstar\volume\mysql when starting the service.
I found out about the /docker-entrypoint-initdb.d directory from this answer, and also read the "Initializing a fresh instance" section of the "How to use this image" MySQL documentation. But when I run docker-compose up in the directory containing the docker-compose.yml file below, my database isn't initialized.
services:
# Use root/root as MySQL user/password credentials
db:
image: mysql
restart: always
environment:
MYSQL_ROOT_PASSWORD: root
MYSQL_USER: root
MYSQL_PASSWORD: root
MYSQL_DATABASE: db
volumes:
- ./mysql/data:/var/lib/mysql
- ./mysql/init:/docker-entrypoint-initdb.d/:ro
adminer:
image: adminer
restart: always
ports:
- 8080:8080
I confirmed the ./mysql/init directory contains a file named init.sql. And I confirmed that after I empty the ./mysql/data directory and run docker-compose up, that a db database is created. But the database is not populated, unless I manually execute the script in Adminer. (I click on "Import", then choose the file and press the Execute button.)
I looked for messages in the console output after running docker-compose up that indicate an attempt to run init.sql and can't find anything.
Update: The MySQL version is 8.0.19.
Devil hides in details...
You have a double definition of root in your env vars. root user is created by default with password from MYSQL_ROOT_PASSWORD. You then ask to create a second "normal" user... with the exact same name and password (i.e. with MYSQL_USER and MYSQL_PASSWORD)
If you look carefully at your startup log, you will see an error
db_1 | ERROR 1396 (HY000) at line 1: Operation CREATE USER failed for 'root'#'%'
This actually stops further processing of your init files in docker-entrypoint-initdb.d and goes on with the rest of the image startup process (i.e. restarting mysql after initialization on temporary server).
Simply drop MYSQL_USER and MYSQL_PASSWORD in your env vars, or set a different user than root and you will immediately see your init files processed (don't forget to empty your data dir again).
I've been working with a simple MySQL table using Docker Compose that only included ID and NAME column. I attempted to update my myDb.sql file that initially creates the table like this:
CREATE TABLE `Person` (
`id` int(11) NOT NULL,
`firstName` varchar(50) NOT NULL, // updated this column
`lastName` varchar(50) NOT NULL // added this column
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
I updated the NAME column to firstName, and added a lastName column.
I then stopped the Docker containers by running DOCKER-COMPOSE STOP.
I then restarted the Docker containers by running DOCKER-COMPOSE UP. I even tried DOCKER-COMPOSE RESTART.
The error message I was able to print in the console was this:
SELECT '', id, `firstName` From Person<br>Unknown column 'firstName' in 'field list'
This leads me to believe that I did not restart Docker Compose correctly.
How do I restart Docker Compose so that it runs the CREATE TABLE command?
Edit
Here is my docker-compose.yml file:
version: "3.7"
services:
www:
build: .
ports:
- "8001:80"
volumes:
- ./www:/var/www/html/
links:
- db
networks:
- default
db:
image: mysql:5.7.13
ports:
- "3306:3306"
environment:
MYSQL_DATABASE: myDb
MYSQL_USER: user
MYSQL_PASSWORD: test
MYSQL_ROOT_PASSWORD: test
volumes:
- ./dump:/docker-entrypoint-initdb.d
- persistent:/var/lib/mysql
networks:
- default
phpmyadmin:
image: phpmyadmin/phpmyadmin
links:
- db:db
ports:
- 8000:80
environment:
MYSQL_USER: user
MYSQL_PASSWORD: test
MYSQL_ROOT_PASSWORD: test
volumes:
persistent:
My Dockerfile looks like this:
FROM php:7.0.30-apache
RUN docker-php-ext-install mysqli
The docker-entrypoint-initdb.d mechanism only runs the first time a database container starts up, with an empty database. You'll need to explicitly docker-compose rm your containers after you docker-compose stop them to cause the current database to be deleted, and then a new empty table will be created in a new empty database.
If you need to preserve the data in an existing database, you are looking for a mechanism called a migration. The various Docker database images don't directly have migration support; this is almost always something that is packaged with your application-level database library (Ruby on Rails and Python's SQLAlchemy both have migration facilities, for instance).
Once you have a migration system anyways, it's probably better to just use that to create the initial database tables. docker-entrypoint-initdb.d makes a little more sense for database-level setup like creating initial users or loading a seed database dump, but in practice you will always need a migration system for changes like what you're describing.
Do docker-compose down whenever you update your schema, it will remove containers and docker network and do docker-compose up to bring your environment with your new schema. Hope it helps. If not, try updating to latest mysql image, the image you are using is almost 3 years old.
So I was thinking that restarting docker-compose would automatically run the CREATE TABLE query in the myDb.sql file. As indicated above, I changed the name of 'name' column to 'firstName' and added a column called 'lastName'.
Again, I am not sure if there was a command that actually does this, but in the end, I was able to alter the table in phpmyadmin. Once I altered the table there, now I am getting data back to the page with no errors.
I'm very new to Docker and after reading about data volumes I'm still somewhat confused by the behaviour I'm seeing.
In my compose file I had an entry for mysql like this:
db:
image: mysql
restart: always
volumes:
- ./database:/var/lib/mysql
environment:
MYSQL_ROOT_PASSWORD: p4ssw0rd!
networks:
- back
This mapped the /database directory to /var/lib/mysql. The database files where created and I could start Wordpress, install, add a post. The problem as it never persisted any created data. If I restarted Docker and executed:
docker-compose up -d
The database was empty.
Changing this to:
db:
image: mysql
restart: always
volumes:
- db_data:/var/lib/mysql
environment:
MYSQL_ROOT_PASSWORD: p4ssw0rd!
networks:
- back
And adding in a volume like this:
volumes:
db_data:
Now persists the data in the Docker data volume and restarting works. Any data created during the last run is still present.
How would I get this to work using the host mapped directory?
Am I right in thinking the second example using volumes is the way to go?
Docker volumes on windows work a bit different way than Linux. Basically on Windows, docker runs a VM and the docker is setup inside the VM. So it seems to you that you run docker commands locally on Windows but the actual stuff happens in background inside a VM.
docker run -v d:/data:/data alpine ls /data
First you need to make share the D: in docker settings. You can find a detailed article explaining the steps for doing so
https://rominirani.com/docker-on-windows-mounting-host-directories-d96f3f056a2c
I am trying to create a mysql database/schema if it doesn't already exist.
Here is what I have tried:
docker-compose.yml
mysql:
image: mysql:5.6.26
environment:
- MYSQL_ROOT_PASSWORD=root
command: "mysql -uroot -proot < createDB.sql"
ports:
- "3306:3306"
createDB.sql
CREATE DATABASE IF NOT EXISTS bignibou;
It does not work. What would be the best way to use docker/docker-compose in order to create a schema if it does not exist?
I finally found the beginning of a solution.
The MySQL image takes an environment variable i.e. MYSQL_DATABASE that initialize the container with the name of the database on image startup See here for full documentation.
Or read the excerpt below:
MYSQL_DATABASE
This variable is optional and allows you to specify the name of a
database to be created on image startup. If a user/password was
supplied (see below) then that user will be granted superuser access
(corresponding to GRANT ALL) to this database.
Here is what I came up with:
mysql:
image: mysql:5.6.26
environment:
- MYSQL_ROOT_PASSWORD=root
- MYSQL_DATABASE=bignibou
ports:
- "3306:3306"
I now need a way to specify the default collation but that is another story...
edit: For those interested in specifying a different collation from the default, here are the instructions to use another config file that will override the default one. See below:
Using a custom MySQL configuration file The MySQL startup
configuration is specified in the file /etc/mysql/my.cnf, and that
file in turn includes any files found in the /etc/mysql/conf.d
directory that end with .cnf. Settings in files in this directory will
augment and/or override settings in /etc/mysql/my.cnf. If you want to
use a customized MySQL configuration, you can create your alternative
configuration file in a directory on the host machine and then mount
that directory location as /etc/mysql/conf.d inside the mysql
container.
If /my/custom/config-file.cnf is the path and name of your custom
configuration file, you can start your mysql container like this (note
that only the directory path of the custom config file is used in this
command):
$ docker run --name some-mysql -v /my/custom:/etc/mysql/conf.d -e
MYSQL_ROOT_PASSWORD=my-secret-pw -d mysql:tag This will start a new
container some-mysql where the MySQL instance uses the combined
startup settings from /etc/mysql/my.cnf and
/etc/mysql/conf.d/config-file.cnf, with settings from the latter
taking precedence.
To not lost your data better use volumes as well:
version: '3'
services:
db:
image: mysql:5.7
volumes:
- mysql-db:/var/lib/mysql
environment:
MYSQL_ROOT_PASSWORD: root
MYSQL_DATABASE: my_db_name
ports:
- "3307:3306"
volumes:
mysql-db:
probably what you are trying to do needs an additional script. So if building an image instead of directly using a prebuilt image is an option for you, you need to use a Dockerfile and use a script file which first imports the script in MySql and then runs the service itself.
take a look at this answer: Docker - Initialize mysql database with schema
From the docker-compose documentation - see Define Services - you can tell which Dockerfile it will use to build the image. Therefore you can create a Dockerfile based on the mysql image and create the database inside it using standard Dockerfile commands.
This might be useful in case someone lands here in future. The real issue appears to be the "command" statement in the docker-compose file. Once the command finishes successfully the container will get destroyed. This sql script must be run only after docker-compose has run and containers have been created. docker-compose "command" is really to start a service in the container. In this case you overrode the mysql service with your command.