This is taken from the official docker website Dockerfile best practices ..
VOLUME
The VOLUME instruction should be used to expose any database storage
area, configuration storage, or files/folders created by your docker
container. You are strongly encouraged to use VOLUME for any mutable
and/or user-serviceable parts of your image.
What is meant by using volume for any mutable and user serviceable parts of the image? Are there times when I should/shouldnt and use a volume for databases? If so why? Is this where you mount the actual data contents of the database separate from the docker container..
Not a complete answer, but I found an example which might help. From the book "Build your own PAAS with Docker" by Oskar Hane, where he creates a container used only to host files for other containers, such as a MySQL container:
There is a VOLUME instruction for the Dockerfile, where you can define which directories to expose to other containers when this data volume container is added using --volumes-from attribute. In our data volume containers, we first need to add a directory for MySQL data. Let's take a look inside the MySQL image we will be using to see which directory is used for the data storage, and expose that directory to our data volume container so that we can own it:
RUN mkdir –p /var/lib/mysql
VOLUME ["/var/lib/mysql"]
Related
I am working on logging of my application on Persistent Volume.
I am using OpenShift, I created storage(Persistent volume under nas-thin class) and allocated 64Gib to it. I added mount path to this PV for one of my pods where my application is running and generating logs in one of the folder named "logs".
My mount path is "/logs". Hence anything inside this folder will be root for my PVC.
I am appending my logs inside logs folder in a single file.
I tried to read about expanding PV but couldn't understand much.
What would happen if my log file size exceeds allocated PV size(which is 64Gib)?
That will depend on the persistent storage actually being used by the cluster. Some persistent volume providers will let you write more data than you actually defined. So you'll have to test how your storage and your application actually behave on your particular cluster.
That being said, it is generally a bad idea to have container workload log to a persistent volume. I would strongly recommend to log to STDOUT and then use an external logging system to manage your logs instead of writing to a file.
How will you deal with multiple replicas of your application running? Do you really want to go into each container to get the log files? How will you correlate logs between different replicas?
Applications running on OpenShift / Kubernetes should not manage their logs in files but write to STDOUT.
I have a big static database where no data is inserted / updated.
I created a Dockerfile in the following manner which works well to create the database in the docker container. There are a bunch of insert .sql scripts in data/ which populates the database.
FROM mysql:5.7
ADD data/* /docker-entrypoint-initdb.d/
EXPOSE 3306
The only problem with this is that it takes a long time to start up, so I ran the container locally and adjusted it slightly so that the data persisted within the container, and I removed the startup scripts from the startup script directory. I then committed this container to a new docker image (committed-image:v1).
This worked, however the only problem I have is that the password for the mysql root user for this database is fixed to the password which I used when I committed the image.
My question is: Is there any way to run another startup script on top of this new image (which already has an existing database)? I need to modify the password of the root user to match an environment variable in the container (cant use MYSQL_ROOT_PASSWORD now since the startup script no longer runs in the container spun up from the new committed image).
The trouble is your entrypoint script for mysql:5.7 (the default) is quite elaborate. DATABASE_ALREADY_EXISTS is based on the existence of the mysql/database directory and therefore the ROOT doesn't get reinitialized. Its hard to get a loaded database to exist without a system tables include a root user being set.
Solutions:
a) the default configuration of mysql instance isn't particularly conducive to large database loads. Including a configuration item that sets innodb_buffer_pool_size to several gig of ram/80% of available RAM will help the load and run.
and/or:
b) set innodb_file_per_table=1 in the configuration and then for your SQL, use CREATE TABLESPACE / ALTER TABLE .. TABLESPACE = to load the data from a pre-geneated idb file. See these instructions from the manual on extracting a tablespace.
To create/set configuration file settings see the readme notes from dockerhub. You could also just drop in a file like /etc/mysql/conf.d/config-file.cnf in your Dockerfile.
I have a container with MySQL that is configured to start with "-v /data:/var/lib/mysql" and therefore persists data between container restarts in the separate folder. Although this approach has some drawbacks, in particular, the user may not have write permissions for a specified directory. How exactly container should be reconfigured in order to use Docker's implicit per-container storage to save MySQL data in the /var/lib/docker/volumes in order to reuse it after the container is stopped and started again? Or is it better to consider another persistence options?
What you show is called bind mounts.
What you request is called volumes.
Just create volume and connect it
docker volume create foo
docker run ... -v foo:/var/lib/mysql <image> <command>
And you've done it! You can connect it to many containers at will.
I've setup a basic MariaDB instance running in Docker - basically from starting the container using the Kitematic UI, changing the settings, and letting it run.
Today, I wanted to make a backup, so I used Kitematic to change the port so I could access it from a machine to make automated backups. After changing the port in Kitematic, it seems to have started a fresh MariaDB container (i.e. all my data seems to be removed).
Is that the expected behavior? And, more importantly, is there any way to recover the seemingly missing data, or has it been completely removed?
Also, if the data is actually removed, what is the preferred way to change settings—such as the exposed ports—without losing all changes? docker commit?
Notes:
running docker 1.12.0 beta for OS X
docker -ps a shows the database status as "Up for X minutes" when the original had been up for several days
Thanks in advance!
UPDATE:
It looks like the recommended procedure to retain data (without creating a volume or similar) is to:
commit changes (e.g. docker commit <containerid> <name/tag>)
take the container offline
update settings such as exposed port or whatever else
run the image with committed changes
...taken from this answer.
Yes, this is expected behavior. If you want your data to be persistant you should mount volume from host (via --volume option for docker run) or from another container and store your database files at this volume.
docker run --volume /path/on/your/host/machine:/var/lib/mysql mariadb
Losing changes are actually core feature of containers so it can not be omitted. This way you can be sure that between every docker run you get fresh environment without any changes. If you want your changes to be permanent you should do them in your image's Dockerfile, not in container itself.
For more information please visit official documentation: https://docs.docker.com/engine/tutorials/dockervolumes/.
it looks like you dont mount container volume into certain path. You can read about volumes and storing data into container here
you need run container with volume option
$ docker run --name some-mariadb -v /my/own/datadir:/var/lib/mysql -e MYSQL_ROOT_PASSWORD=my-secret-pw -d mariadb:tag
where /my/own/datadir is directory on host machine
I have the following Docker containers:
web (nginx)
db (mysql)
The web container is linked to the db container. All standard stuff.
For data persistence, I want to take the data volume container approach.
I want to be able to run several websites using these 2 containers as the main application containers (well, technically, the web container runs the main user-facing application and the db sits behind that).
Let's say I have siteA, siteB, and siteC using databases A, B, and C respectively in mysql.
I would like to be able to partition these site's database data into 3 different data volume containers (dataA, dataB, dataC) for portability, and bring them together in a single deployment (one host running web, db, dataA, dataB, dataC all linked appropriately) when needed.
Is there a way to partition the separate mysql databases into their own data volume containers for portability? AFAIK, mysql stores all of it's database data in /var/lib/mysql in some fashion that is not transparent in terms of the databases that mysql is storing.
For example, it would be nice if the different databases being stored in mysql were mapped to known directories - in this case, /var/lib/mysql/A, /var/lib/mysql/B, and /var/lib/mysql/C. This way, all I would have to do persist those directories and mount them in the db container. I don't think this is the case though.
Why not just run multiple instances of the db container? Mapping the dir's as needed.
docker run -d -v /var/lib/mysql/A:/var/lib/mysql --name dbA my_docker/my_awesome_mysql
docker run -d -v /var/lib/mysql/B:/var/lib/mysql --name dbB my_docker/my_awesome_mysql
docker run -d -v /var/lib/mysql/C:/var/lib/mysql --name dbC my_docker/my_awesome_mysql
This would help if you ever needed to move dbA only to another host.