I created the following Dockerfile:
FROM mysql:5.7
ADD assets/geograph.cnf /etc/mysql/conf.d
ENV MYSQL_DATABASE=geograph \
MYSQL_RANDOM_ROOT_PASSWORD=yes
ADD http://data.geograph.org.uk/dumps/gridimage_base_sample.mysql.gz /docker-entrypoint-initdb.d/gridimage_base_sample.sql.gz
I then created an image from this Dockerfile:
docker build --tag geograph:latest .
I then created a container from this image:
docker run --name geograph -e MYSQL_USER=geograph -e MYSQL_PASSWORD=geograph --detach geograph:latest
However, I've noticed some unexpected behaviour:
The container starts and then stops. (I've compared docker ps and docker ps --all.)
The geograph database doesn't contain the data from gridimage_base_sample.sql.gz, which was created by mysqldump. (I've verified the database dump.)
I expected the container to behave like the base MySQL image (mysql:5.7), with some additional configuration and some data. What am I doing wrong?
Some context: I'd like to use the database for analysis, ensuring that the results of my analysis are repeatable. I'm not going to be creating/deleting records and host/guest will probably be on the same machine. I've experimented with using another image/container to curl the database dump into a volume and then mounting this volume when creating the geograph container. This solves #2 but not #1. (It also seems a bit unnecessary to have two images/containers but that's not terribly important: if two are required, then two are required!)
Thanks in advance for any help.
Whilst the above image could ADD the database dump, the MySQL image couldn't unzip it because of a permissions error. (The user didn't have permission to write to /docker-entrypoint-initdb.d.) Seemingly this was enough to cause the container to stop (#1) and explains why the geograph database didn't contain the data from gridimage_base_sample.sql.gz (#2).
Related
I'm new to Docker and learning about it. I'm using a Docker container of MySQL and I have created two databases with populated tables.
I've pushed the image to Docker Hub so I can use it on another device but I've tried several times whenever I pull my MySQL repository and run it I don't see any of my databases. I think I'm doing it the wrong way.
Mysql Databases from the pulled image
How can I push the MySQL image with its two databases to Docker Hub the right way?
Rather than have the database included in your image, you can have SQL scripts in your image that creates the database and populates it with initial data.
If you put files ending in .sh, .sql or .sql.gz in the /docker-entrypoint-initdb.d directory in the image, they will be run the first time the container is run.
If you have an SQL script to initialize your database, you can include it in the image by having a Dockerfile like this
FROM mysql:latest
COPY initialize-database.sql /docker-entrypoint-initdb.d/
Then you can run the container and map /var/lib/mysql to a docker volume that will store the database like this
docker run --rm -e MYSQL_ROOT_PASSWORD=password -v mysql:/var/lib/mysql <my-image-name>
I'm copy previous mariadb container's data(/var/lib/mysql) and paste data to new container image.
this is Dockerfile
FROM mariadb:latest
ENV MYSQL_ROOT_PASSWORD tt
ENV MYSQL_DATABASE tt
ENV MYSQL_USER tt
ENV MYSQL_PASSWORD tt
# copy other database data
ADD mysql /var/lib/mysql
RUN chown -R mysql:mysql /var/lib/mysql
VOLUME /var/lib/mysql
EXPOSE 3306
CMD ["mysqld"]
when I build docker image, all table remained
but run docker image by using volume, all table disappear, just db.opt remains.
how can i get database's data with using volume?
Your problem it's related on how VOLUMES work, what you are doing it's adding stuff inside /var/lib/mysql at build time, and then when you run the container, a new EMPTY VOLUME is created and linked at the /var/lib/mysql, which pretty much overwrites anything you put there before.
The correct way to go about it for an existing table space would be to create a volume with the docker volume create syntax (ref: https://docs.docker.com/storage/volumes/) and then using this trick ( https://github.com/moby/moby/issues/25245#issuecomment-365980572 ) you can add the table data to your volume, and then you run your mariadb container mounting said volume docker run --mount source=myvolume,target=/var/lib/mysql mariadb:latest
I will add that you shouldn't build an image with the tables added at the docker build layer, it makes the docker image huge and it's a wrong use of it all around except for some niche cases like QA databases that you destroy afterwards or read-only databases. The usual way about it it's either using VOLUMES as you stated or with bindings from the host OS.
Then, you shouldn't use mariadb:latest either, choose a particular version and stick with it, as using mariadb:latest can upgrade/downgrade your version and cause all kind of funny bugs.
Documentation for the mysql docker image says:
When a container is started for the first time [...] it will execute files with extensions .sh and .sql that are found in /docker-entrypoint-initdb.d. You can easily populate your mysql services by mounting a SQL dump into that directory and provide custom images with contributed data.
So at first I did this in my docker-compose.yml:
version: '2'
services:
db:
image: mysql:5.7
volumes:
- .:/docker-entrypoint-initdb.d:ro
When I ran docker-compose build and docker-compose up the container was created and the sql files in the current directory were executed. So far all good.
But if I want to deploy these containers to another machine (using docker-machine), mounting /docker-entrypoint-initdb.d as a volume won't work, since that machine won't have access to my machine's . directory.
So then I tried to extend the mysql:5.7 image:
FROM mysql:5.7
COPY ./*.sql /docker-entrypoint-initdb.d/
And do this in my docker-compose.yml
version: '2'
services:
db:
build:
context: .
dockerfile: Dockerfile
However, when I then run docker-compose build and docker-compose up on the second machine and try to run my application, the *.sql files in the current directory aren't executed. None of my tables are created.
Why doesn't my second approach work?
EDIT:
Ah, wait. I have asked the wrong question. The problem is not that the second approach doesn't work, it is that the second approach doesn't work when running it on the local docker-machine running in Virtualbox. The second approach actually works when I use it on my host machine (i.e. not using docker-machine).
I found the issue. The problem was that I thought docker-compose rm -f destroyed any volumes attached to the containers, but I was wrong. So what I thought was the first up:ed containers were in fact using the database created by an earlier up. So the sql-files weren't run because it wasn't actually the first time the containers started. Duh. Thanks Ken for pointing me in the right direction.
Turns out that not even using docker-compose rm -v removes the volumes. I had to list them with docker volume ls and then remove them manually with docker volume rm <volume>.
Another Docker-specific way to clean up volumes:
docker system prune
This will remove dangling images, containers, volumes, and networks. Adding -a will also remove expired containers and currently unused images.
Using the official MySQL Docker image, I don't understand how to mount the data directory to a specifc point on the host. The Dockerfile of the image sets
VOLUME /var/lib/mysql
so database data should be stored "somewhere" on the host. I want to be more specific in my docker-compose file, so I tried the following:
mysql:
image: mysql:5.7
environment:
- MYSQL_ROOT_PASSWORD=password
- MYSQL_DATABASE=mydb
volumes:
- ./database/mysql:/var/lib/mysql
Starting with docker-compose up everything works fine, but the ./database/mysql directory on the host stays empty, whereas /var/lib/mysql in the container contains data. Is there a problem in my configuration? Or do I misunderstand how to use volumes?
docker-compose will always try to preserve data volumes, so that you don't lose any data within them. If you started with a data volume, then changed to a host volume, you may still get the data volume.
To correct this, run docker-compose stop && docker-compose rm -f, which will remove your containers and your data volumes (this will erase any data in your data volumes). On the next docker-compose up, you should see it using the host volume.
Edit: As of Compose 1.6 you can run docker-compose stop -v instead of the two commands above.
I am trying to automate the installation and running of set of linked docker containers using fig. The configuration is composed of a container running RStudio linked to a container running MySQL, such that I can query the MySQL database from RStudio.
On first run, I would like to create the MySQL container from the base MySQL image, and populate it with a user and database. From the command line, something like this:
#Get the latest database file
wget -P /tmp http://ergast.com/downloads/f1db.sql.gz && gunzip -f /tmp/f1db.sql.gz
#Create the database container with user, password and database
docker run --name ergastdb -e MYSQL_USER=ergast -e MYSQL_ROOT_PASSWORD=mrd -e MYSQL_DATABASE=f1db -d mysql
#Populate the database
docker run -it --link=ergastdb:mysql -v /tmp:/tmp/import --rm mysql sh -c 'exec mysql -h$MYSQL_PORT_3306_TCP_ADDR -P$MYSQL_PORT_3306_TCP_PORT -uergast -pmrd f1db < /tmp/import/f1db.sql'
#Fire up RStudio and link to the MySQL db
docker run --name f1djd -p 8788:8787 --link ergastdb:db -d rocker/hadleyverse
If I could get hold of a database image with the data preloaded, I guess that something like the following fig.yml script could link the elements?
gdrive:
command: echo created
image: busybox
volumes:
- "~/Google Drive/shareddata:/gdrive"
dbdata:
image: mysql_preloaded
environment:
MYSQL_USER=ergast
MYSQL_ROOT_PASSWORD=mrd
MYSQL_DATABASE=f1db
rstudio:
image: rocker/hadleyverse
links:
- dbdata:db
ports:
- "8788:8787"
volumes_from:
- gdrive
My question is, can I use a one-shot fig step to create the dbdata container, then perhaps mount a persistent volume, link to it and initialise the database, presumably as part of an initial fig up. If I then start and stop containers, I don't want to run the db initialisation step again, just link to the data volume container that contains the data I previously installed.
I also notice that the MySQL docker image looks like it will support arbitrary datadir definitions (Update entrypoints to read DATADIR from the MySQL configuration directly instead of assuming /var/lib/docker). As I understand it, the current definition of the MySQL image prevents mounting (and hence persisting) the database contents within the database container. I guess this might make it possible to create a mysql_preloaded image, but I don't think the latest version of the MySQL docker script has been pushed to dockerhub just yet and I can't quite think my way to how fig might then be able to make use of this alternative pathway?
Some options:
Edit the fig.yml to run a custom command that is different than the default image command/entrypoint.
From http://www.fig.sh/yml.html (example):
command: bundle exec thin -p 3000
Start the container locally, modify it and then commit it as a new image.
Modify the MySQL image docker-entrypoint.sh file to do your custom initialization.
https://github.com/docker-library/mysql/blob/567028d4e177238c58760bcd69a8766a8f026e2a/5.7/docker-entrypoint.sh
Couldn't you just roll your own version of the MySQL docker image? The official one from MySQL "upstream" is available at https://github.com/mysql/mysql-docker/blob/mysql-server/5.7/Dockerfile
What if you simply make your own copy of that, remove the VOLUME line (line 11) and then you can
docker build -t my_mysql .
docker run -d --name=empty_db my_mysql ...
# add data to the database running in the container
docker commit empty_db primed_db
docker rm -v empty_db
docker run -d --name=instance1 primed_db
docker run -d --name=instance2 primed_db
which should leave you with two running "identical" but fully isolated instances.