i am building an app stack where mysql is one of the services. I want all services to run as uid 1000:1000, so i dont have any trouble with permissions on the host system.
Now i have problems doing this with the official mysql docker image.
Here is what i do:
FROM mysql:5.7
RUN addgroup --gid 1000 app \
&& adduser --gecos "" --home /home/app --shell /bin/bash --disabled-password --uid 1000 --gid 1000 app \
&& adduser app mysql \
&& adduser mysql app
RUN chown -R app:app /var/lib/mysql && chmod -R g+wrx /var/lib/mysql
COPY mysql.cnf /etc/mysql/conf.d/99-docker.cnf
COPY .my.cnf /home/app/.my.cnf
COPY .root.cnf /root/.my.cnf
RUN chmod 0664 /etc/mysql/conf.d/99-docker.cnf
USER app
EXPOSE 3306
And the service in my compose file:
version: "3"
services:
mysql:
build:
context: ./docker/mysql
dockerfile: Dockerfile
env_file:
- .env
volumes:
- ./var/mysql:/var/lib/mysql
user: 1000:1000
ports:
- 3307:3306
command: ['mysqld', '--character-set-server=utf8mb4', '--collation-server=utf8mb4_unicode_ci']
The env file contains some vars for mysql:
MYSQL_ROOT_PASSWORD=root
MYSQL_DATABASE=app
MYSQL_USER=app
MYSQL_PASSWORD=app
the .my.cnf files contain credentials
the mysql config contains some cache settings, sql-mode and:
[mysqld]
user=app
when starting the service with: docker-compose up --build mysql i get the following error:
mysql_1 | Initializing database
mysql_1 | mysqld: Can't create/write to file '/var/lib/mysql/is_writable' (Errcode: 13 - Permission denied)
mysql_1 | 2020-04-09T11:11:25.664285Z 0 [Warning] TIMESTAMP with implicit DEFAULT value is deprecated. Please use --explicit_defaults_for_timestamp server option (see documentation for more details).
mysql_1 | 2020-04-09T11:11:25.664345Z 0 [Warning] 'NO_AUTO_CREATE_USER' sql mode was not set.
mysql_1 | 2020-04-09T11:11:25.666725Z 0 [ERROR] --initialize specified but the data directory exists and is not writable. Aborting.
mysql_1 | 2020-04-09T11:11:25.666744Z 0 [ERROR] Aborting
mysql_1 |
pixel3_mysql_1 exited with code 1
When running docker-compose run mysql bash and inspecting the files i see, that the /var/lib/mysql directory is root-owned.
In the documentation (https://hub.docker.com/_/mysql) it says i can run the image with another user but to me it seems this is not possible.
Do i have to override the entrypoint to chown the directories over and over again or what is my mistake?
Related
I had a spring-boot project that used mysql docker-image so I didn't need to download the mysql benchwork. For other reasons I had to start over so I created a new project that uses the same mysql docker image I previously used.
My docker-compose.yml mysql service looks like this
version: "3.7"
services:
db:
image: mysql:5.7
command: --lower_case_table_names=1
environment:
MYSQL_ROOT_PASSWORD: root
MYSQL_DATABASE: farming_db
MYSQL_USER: root
MYSQL_PASSWORD: root
restart: always
volumes:
- "./database/farming_db/:/var/lib/mysql" #local
- farming_db:/var/lib/mysql/data #docker
ports:
- "3306:3306"
container_name: farming_mysql
networks:
- backend-network
When I run
docker-compose up
This is the error :
Attaching to farming_mysql, farming_server_springboot_1
farming_mysql | 2021-03-18 07:03:20+00:00 [Note] [Entrypoint]: Entrypoint script for MySQL Server 5.7.33-1debian10 started.
farming_mysql | 2021-03-18 07:03:20+00:00 [Note] [Entrypoint]: Switching to dedicated user 'mysql'
farming_mysql | 2021-03-18 07:03:20+00:00 [Note] [Entrypoint]: Entrypoint script for MySQL Server 5.7.33-1debian10 started.
farming_mysql | 2021-03-18 07:03:21+00:00 [Note] [Entrypoint]: Initializing database files
farming_mysql | 2021-03-18T07:03:21.058436Z 0 [Warning] TIMESTAMP with implicit DEFAULT value is deprecated. Please use --explicit_defaults_for_timestamp server opti
on (see documentation for more details).
farming_mysql | 2021-03-18T07:03:21.063630Z 0 [ERROR] --initialize specified but the data directory has files in it. Aborting.
farming_mysql | 2021-03-18T07:03:21.063710Z 0 [ERROR] Aborting
farming_mysql |
farming_mysql exited with code 1
springboot_1 |
I understood that my directory is not empty. I am trying to use "./database/farming_db/:/var/lib/mysql" and "farming_db:/var/lib/mysql/data" both as the volume directories. I think the problem is with the latter directory because the prior directory is empty. I'm having a problem deleting the contents in the latter directory because I don't know how to access it.
So this is what I've tried :
I deleted all the containers and then deleted all the volumes.docker volume prune but didn't work.
I searched that I could do rm -rf /usr/local/var/mysql but I don't know where I can execute this command since the container won't run properly at all.
I deleted the mysql image and just ran docker-compose up again. This seems to pull a new mysql image from somewhere? but I still get the same error. I guess volume directory has nothing do with the docker image itself.
I deleted the "- farming_db:/var/lib/mysql/data #docker" line from the docker-compose. But the same error is still occuring!
I'm using Windows10.
My question :
How can I access the directory? I don't know where to use the rm -rf command.
Why does this error still occur even when I erase "- farming_db:/var/lib/mysql/data #docker" from the docker-compose?
And also could anyone explain what I am doing? I'm new to docker and I don't really understand these volume problems.
Run docker system prune --volumes
This frees up the memory by removing all unused containers. Sometimes, the mentioned issue can occur due to memory limitations
Generally I emptied the volume's data directory and just changed the versions of the MySQL.
So in steps:
empty volume directory content
modify docker-compose.yml mysql version from 5.7 to 5.7.16
This line indicate that mysql container is storing the data inside a directory database in the same directory than your docker-compose.yml:
volumes:
- "./database/farming_db/:/var/lib/mysql" #local
This kind of volume isn't managed by Docker, it's just a directory in your filesystem, this is why docker volume prune doesn't work. I know that, because it starts with a "path" relative or absolute.
The other volume, farming_db, are managed by Docker. I know that because it starts with a simple name. This kinds of volume are managed by Docker and are removed with prune.
So, answering:
In the same directory than your docker-compose.yml you can remove that database folder.
Because the first volume, the one with /var/lib/mysql still exists. MySQL keeps all files inside this directory and any other child directory are a database.
You're just trying to put a container running and docker-compose hides a lot of details.
This is just a detail, but MYSQL_USER should be different than root.
You can let Docker manage the entire volume, creating a single volume to hold all data, in this case I named it as mysql_data:
volumes:
- mysql_data:/var/lib/mysql
Or, you can explore a bit more the docker run equivalent command to get used with it:
docker run -d --name mysql \
-e MYSQL_ROOT_PASSWORD=root \
-e MYSQL_DATABASE=farming_db \
-e MYSQL_USER=myuser \
-e MYSQL_PASSWORD=mypass \
-v mysql_data:/var/lib/mysql \
-p 3306:3306 \
mysql:5.7
As vencedor's answer, it worked for me. If anyone need stay with mysql 5.7, you can add these lines to your db service in docker-compose.yml:
- /etc/group:/etc/group:ro
- /etc/passwd:/etc/passwd:ro
user: "1000:1000"
I used docker-compose to run mysql image and encountered the error.
I use the following configuration to set volume.
- ./mysql/data:/var/lib/mysql/data
Then I changed it to the following and the error was solved.
- ./mysql:/var/lib/mysql
I'm running docker in Ubuntu and trying to create and run a MySql container. I want to use a mounted network share for the data directory. I am trying the following docker run command, but I'm having issues with permissions. How do I fix this?
root#jarvis:/mnt/wayne/mysql-data$ sudo docker run -it -p 3306:3306 -e MYSQL_ROOT_PASSWORD=admin -v /mnt/wayne/mysql:/var/lib/mysql/ --name mysqlserver mysql/mysql-server
[Entrypoint] MySQL Docker Image 8.0.20-1.1.16
[Entrypoint] Initializing database
2020-06-08T21:43:25.253898Z 0 [System] [MY-013169] [Server] /usr/sbin/mysqld (mysqld 8.0.20) initializing of server in progress as process 22
2020-06-08T21:43:25.281460Z 1 [System] [MY-013576] [InnoDB] InnoDB initialization has started.
2020-06-08T21:43:27.815075Z 1 [System] [MY-013577] [InnoDB] InnoDB initialization has ended.
mysqld: Cannot change permissions of the file 'ca.pem' (OS errno 1 - Operation not permitted)
2020-06-08T21:43:29.851875Z 0 [ERROR] [MY-010295] [Server] Could not set file permission for ca.pem
2020-06-08T21:43:29.852970Z 0 [ERROR] [MY-013236] [Server] The designated data directory /var/lib/mysql/ is unusable. You can remove all files that the server added to it.
2020-06-08T21:43:29.854806Z 0 [ERROR] [MY-010119] [Server] Aborting
2020-06-08T21:43:31.947298Z 0 [System] [MY-010910] [Server] /usr/sbin/mysqld: Shutdown complete (mysqld 8.0.20) MySQL Community Server - GPL.
You use CIFs for network mount means the remote server is windows right? My answer is based on this assumption.
The latest mysql docker image has a user named mysql and its uid=27,gid=27
You verify this by mounting an empty folder as data_dir. You will see that the files created by mysql container has user and group is as 27.
Hence the mysql container expects files with uid/gid(owner userid and owner group id) as 27 in its data_dir. But the files that you mounted from the windows share has uid/gid which belongs to the user that executes mount command in ubuntu. This is the default behavior of mount command.
To solve this you need to pass "uid=27,gid=27" parameters to the Linux mount command.
For instance
sudo mount -t cifs -o
username=windows-username,uid=27,gid=27
//WIN_SHARE_IP/ /mnt/wayne
You can have look here for further details
I must say it is unlikely to run mysql over a network share. It won't perform well.
This is not exactly with MySQL but I hope it can give you an idea, I basically use this for testing against a MySQL database from my local environment, for this I use docker-compose and MariaDB, I configure the "data-dir" as a volume so that I can stop/start the docker container without the need to "seed" every time the database.
This is the content of the /your/path/docker-compose.yml file:
---
version: '3'
services:
mariadb:
image: mariadb:10.4.13
container_name: mariadb
restart: always
ports:
- 13306:3306
environment:
MYSQL_DATABASE: world
MYSQL_ROOT_PASSWORD: test
command: --transaction-isolation=READ-COMMITTED --binlog-format=ROW
volumes:
- ${PWD}/mariadb/db/:/var/lib/mysql
In the same directory, I have the volume directory /your/path/mariadb/db
Then to bring up the container I use:
$ docker-compose up
From the docker-compose.yml has you can see I use port 13306 therefore for testing/connection I use:
$ mysql -h 127.0.0.1 -P13306 -uroot -p
All the data (databases) will be in /your/path/mariadb/db
If you run into the same "permissions" problem:
mysqld: Cannot change permissions of the file 'ca.pem' (OS errno 1 - Operation not permitted)
Try to change the permissions of your volume/mount point, for example:
chmod -R 777 /your/volume/mount_point
okay, I tried this and google also, what I found is
https://github.com/docker-library/mysql/issues/302#issuecomment-308745834
So basically if you are using mysql:5.7 then upgrade to mysql:5.7.16.
And if this doesn't help then I have one more solution.
Basically the problem is you are sharing dir to container -v /mnt/wayne/mysql:/var/lib/mysql/ but you ubuntu is not giving permission to access the /mnt/wayne/mysql dir. so give admin permission to this location or you can create a docker user chown and chmode.
Basically give permission to the host machine directory. so that docker container can access it.
and One more thing give permission to the docker container dir also, that is showing in your error
The designated data directory /var/lib/mysql/ is unusable. You can remove all files that the server added to it.
Create a user in a docker container which have chown and chmod permissions to the dir /var/lib/mysql/.
if you are using dockerfile to create mysql container then use these following 2 lines in it
FROM mysql:5.7.16
WORKDIR /app
RUN chown -R admin:admin /app
RUN chmod 755 /app
USER admin
CMD ["Your command"]
To operate normally, MariaDB or MySQL needs to set some permissions on their own files. Some external file systems (such as FTP and many others) do not support these features. You need to use a file system which supports these features.
there is a permission issue to access the mounted volume. Please read the documentation about use volumes:
https://docs.docker.com/storage/volumes/#use-a-volume-driver
For NFSv3 Partition:
$ docker service create -d \
--name nfs-service \
--mount 'type=volume,source=nfsvolume,target=/app,volume-driver=local,volume-opt=type=nfs,volume-opt=device=:/var/docker-nfs,volume-opt=o=addr=10.0.0.10' \
nginx:latest
Or check the CA.pem file permissions (use chmod 777 /path/to/ca.pem)
For NFSv4 Partition:
docker service create -d \
--name nfs-service \
--mount 'type=volume,source=nfsvolume,target=/app,volume-driver=local,volume-opt=type=nfs,volume-opt=device=:/var/docker-nfs,"volume-opt=o=10.0.0.10,rw,nfsvers=4,async"' \
nginx:latest
Check https://docs.docker.com/storage/volumes/#use-a-volume-driver
Ok, i'm following official Mysql docs to enable encryption on a mysql database on docker:
Using the keyring_file File-Based Plugin
Keyring Plugin Installation
keyring_file_data
So here's what i've done:
added early-plugin-load and keyring_file_data to /etc/mysql/my.cnf (i used echo stuff >> file since mysql docker image has no text editor), so now it is:
[mysqld]
pid-file = /var/run/mysqld/mysqld.pid
socket = /var/run/mysqld/mysqld.sock
datadir = /var/lib/mysql
secure-file-priv= NULL
# Disabling symbolic-links is recommended to prevent assorted security risks
symbolic-links=0
# Custom config should go here
!includedir /etc/mysql/conf.d/
early-plugin-load=keyring_file.so
keyring_file_data=/usr/local/mysql/mysql-keyring/keyring
created keyring file with
cd /usr/local/mysql
mkdir mysql-keyring
chmod 750 mysql-keyring
chown mysql mysql-keyring
chgrp mysql mysql-keyring
restarted container to restart mysql
connected to mysql and checked plugin availability (whith no luck) with
mysql> SELECT PLUGIN_NAME, PLUGIN_STATUS
FROM INFORMATION_SCHEMA.PLUGINS
WHERE PLUGIN_NAME LIKE 'keyring%';
Checked the logs for errors:
2020-03-15T12:30:08.669015Z 0 [ERROR] [MY-011370] [Server] Plugin keyring_file reported: 'File '/usr/local/mysql/mysql-keyring/keyring' not found (OS errno 20 - Not a directory)'
2020-03-15T12:30:08.669036Z 0 [ERROR] [MY-011355] [Server] Plugin keyring_file reported: 'keyring_file initialization failure. Please check if the keyring_file_data points to readable keyring file or keyring file can be created in the specified location. The keyring_file will stay unusable until correct path to the keyring file gets provided'
2020-03-15T12:30:08.669053Z 0 [ERROR] [MY-010202] [Server] Plugin 'keyring_file' init function returned error.
So it look like that i correctly enabled the plugin, but something is wrong with the file.
Am i missing some steps?
keyring file
root#8c3670db35d4:/# ls -la /usr/local/mysql/mysql-keyring/
total 8
drwxr-s--- 2 mysql mysql 4096 Mar 15 12:34 .
drwxr-sr-x 3 root staff 4096 Mar 15 12:33 ..
-rw-r----- 1 mysql mysql 0 Mar 15 12:34 keyring
Are you sure you created the keyring file correctly inside the container ? This is how I was able to achieve the above with a correctly crafted Dockerfile.
Create a folder for your image project (use whatever folder you like)
mkdir /tmp/testMysqlKeyring
cd /tmp/testMysqlKeyring
Create a mysql keyring dropin configuration file keyring.cnf with the following content:
[mysqld]
early-plugin-load=keyring_file.so
keyring_file_data=/usr/local/mysql/mysql-keyring/keyring
Create a Dockerfile with the following content
FROM mysql:8
# Place the dropin config file in the relevant folder
COPY keyring.cnf /etc/mysql/conf.d/
# Create the keyring folder and adapt perms
RUN mkdir -p /usr/local/mysql/mysql-keyring && \
chmod 750 /usr/local/mysql/mysql-keyring && \
chown mysql.mysql /usr/local/mysql/mysql-keyring
Build image from the above configuration:
docker build -t file_keyringed_mysql:latest .
Run a container from that image (you will adapt with your exact volumes and environment later...)
docker run -d --rm --name my_keyring_test -e MYSQL_ALLOW_EMPTY_PASSWORD=true file_keyringed_mysql:latest
Check that plugin is correctly installed inside the container
$ docker exec my_keyring_test mysql -e "SELECT PLUGIN_NAME, PLUGIN_STATUS FROM INFORMATION_SCHEMA.PLUGINS WHERE PLUGIN_NAME LIKE 'keyring%';"
PLUGIN_NAME PLUGIN_STATUS
keyring_file ACTIVE
I use below command to start MySQL container:
docker run --name mysql-for-teamcity \
-e MYSQL_ROOT_PASSWORD=FAKE-ROOT-PWD \
-v ~/MySQL/var_lib_mysql:/var/lib/mysql \
-v ~/MySQL/etc:/etc \
-p 3306:3306 \
-p 33060:33060 \
-it mysql
But MySQL won't start and complains:
ERROR: mysqld failed while attempting to check config
command was: "mysqld --verbose --help"
mysqld: Error on realpath() on '/var/lib/mysql-files' (Error 2 - No such file or directory)
2018-12-05T07:33:50.856816Z 0 [ERROR] [MY-010095] [Server] Failed to access directory for --secure-file-priv. Please make sure that directory exists and is accessible by MySQL Server. Supplied value : /var/lib/mysql-files
2018-12-05T07:33:50.859460Z 0 [ERROR] [MY-010119] [Server] Aborting
If I remove -v ~/MySQL/etc:/etc \ then MySQL would start correctlly.
Why can't I mount /etc to my host in Docker?
This is my silly mistake; it should be an RTFM question.
According to Docker manual, Bind mounts are used to bind directory into a container, not out.
So when I specified -v ~/MySQL/etc:/etc, the contents in my "~/MySQL/etc" would obscure the original contents in "/etc" of the MySQL image.
That's why mysqld failed while attempting to check config. Because the config does not exist in my "~/MySQL/etc/".
I leave the answer here in case someone else does not read the manual:(
i am trying to start the db container from this docker-compose file :
db:
image: mysql
environment:
MYSQL_ROOT_PASSWORD: root
volumes:
- "./testsql:/var/lib/mysql"
phpmyadmin:
image: phpmyadmin/phpmyadmin
container_name: phpmyadmin
volumes:
- /sessions
environment:
- PMA_ARBITRARY=1
restart: always
ports:
- "8082:80"
depends_on:
- db
i've got this error on the container logs :
2018-04-20T21:36:42.409677Z 0 [Warning] TIMESTAMP with implicit DEFAULT value is deprecated. Please use --explicit_defaults_for_timestamp server option (see documentation for more details).
2018-04-20T21:36:42.420057Z 0 [ERROR] --initialize specified but the data directory has files in it. Aborting.
2018-04-20T21:36:42.420341Z 0 [ERROR] Aborting
2018-04-20T21:36:42.420428809Z
thanks
Is there a subdirectory in testsql?
If there is a subdirectory, the error will be reproduced as shown below.
$ mkdir -p testsql/aaa
$ docker run -e MYSQL_ROOT_PASSWORD=r00t -v `pwd`/testsql:/var/lib/mysql mysql
Initializing database
2018-04-20T23:59:23.465762Z 0 [Warning] TIMESTAMP with implicit DEFAULT value is deprecated. Please use --explicit_defaults_for_timestamp server option (see documentation for more details).
2018-04-20T23:59:23.468193Z 0 [ERROR] --initialize specified but the data directory has files in it. Aborting.
2018-04-20T23:59:23.468245Z 0 [ERROR] Aborting
$
If there is no subdirectory, no error will be issued.
$ rm -rf testsql/
$ mkdir testsql
$ docker run -e MYSQL_ROOT_PASSWORD=r00t -v `pwd`/testsql:/var/lib/mysql mysql
Initializing database
2018-04-21T00:04:06.012172Z 0 [Warning] TIMESTAMP with implicit DEFAULT value is deprecated. Please use --explicit_defaults_for_timestamp server option (see documentation for more details).
2018-04-21T00:04:06.533290Z 0 [Warning] InnoDB: New log files created, LSN=45790
2018-04-21T00:04:06.629547Z 0 [Warning] InnoDB: Creating foreign key constraint system tables.
2018-04-21T00:04:06.692854Z 0 [Warning] No existing UUID has been found, so we assume that this is the first time that this server has been started. Generating a new UUID: 823515e3-44f7-11e8-b4cf-0242ac110003.
...