docker mysql - mis-matched host uid/gid - mysql

I have a mysql docker container that has its data and logs dirs separately mapped to host folders for performance reasons.
I'm using docker-compose to start the container with a group of other related services.
--datadir=/var/lib/mysql/innodb-data
--innodb_log_group_home_dir=/var/lib/mysql/innodb-logs
The container dirs are mapped to the host files system via:
volumes:
- /db/mysql-innodb-data:/var/lib/mysql/innodb-data
- /db/mysql-innodb-logs:/var/lib/mysql/innodb-logs
My problem is that the MySQL container is setting the owner uid to 999.
On the host system this maps to the user 'systemd-coredump'.
Instead I want the container to apply the uid for the hosts 'mysql' user.
I've looked at the MySQL docker container and it has the following logic:
docker_create_db_directories() {
local user; user="$(id -u)"
# TODO other directories that are used by default? like /var/lib/mysql-files
# see https://github.com/docker-library/mysql/issues/562
mkdir -p "$DATADIR"
if [ "$user" = "0" ]; then
# this will cause less disk access than `chown -R`
find "$DATADIR" \! -user mysql -exec chown mysql '{}' +
fi
}
We can see that the above script applies the uid user the container runs under to the data directory. By default the container runs as root.
Given that root is uid 0 I don't actually see how this code is change the data-dirs directory to 999 and as such I suspect this code isn't actually the problem.
So I tried changing the user the container runs as to 'mysql'
mysql:
container_name: mysql
image: mysql:8.0
user: mysql
This changes the container user as expected but then MySQL couldn't start up as there are a number of config files that it can no longer read as it's not running as root.
Here is the full service section from my docker-compose:
mysql:
container_name: mysql
image: mysql:8.0
restart: on-failure
environment:
MYSQL_ROOT_PASSWORD: ${MYSQL_ADMIN_PASSWORD}
MYSQL_DATABASE: ${MYSQL_SCHEMA}
command: >
--user=mysql
--lower-case-table-names=1
--datadir=/var/lib/mysql/innodb-data
--innodb_log_group_home_dir=/var/lib/mysql/innodb-logs
--default-authentication-plugin=mysql_native_password
--max-allowed-packet=512M
--innodb_buffer_pool_instances=${MYSQL_INNODB_BUFFER_POOL_INSTANCES-32}
--innodb_buffer_pool_chunk_size=${MYSQL_INNODB_BUFFER_POOL_CHUNK_SIZE-8M}
--innodb_buffer_pool_size=${MYSQL_INNODB_BUFFER_POOL_SIZE-512M}
--table_open_cache=${MYSQL_TABLE_OPEN_CACHE-512}
--max_connections=${MYSQL_MAX_CONNECTIONS-98}
--innodb_flush_neighbors=0
--innodb_fast_shutdown=2
--innodb_flush_log_at_trx_commit=1
--innodb_flush_method=fsync
--innodb_doublewrite=0
--innodb_use_native_aio=0
--innodb_read_io_threads=10
--innodb_write_io_threads=10
--slow_query_log_file=/tmp/mysql-slow.log --long-query-time=1
--slow_query_log
# mem_limit: ${MYSQL_MEMORY}
volumes:
- /db/mysql-innodb-data:/var/lib/mysql/innodb-data
- /db/mysql-innodb-logs:/var/lib/mysql/innodb-logs
network_mode: "host"
logging:
driver: "journald"

Related

Docker LAMP server - keep mysql between two operating systems

I have one windows and one ubuntu 22.04 OS on my device. I have a special shared partition on my harddisk that I can enter from both of the OS'es.
On the shared partition I have a docker setup and I would like it to store the mysql data (volume) in a folder inside the host folder (./mysql_data)
The disk is mounted ntfs and thus, has as permission for all files and folders 1000:1000. Now, that is not docker (gid=999) so I have a hard time getting all permissions correct. Even when I set the -user 1000:1000 flag, issues keep piling up all the time:
corrupt files
not being able to login into phpmyadmin
permission issues
What would be a correct configuration for both the partition mount and the docker file?
partition mount (/etc/fstab):
UUID=xxx /mnt/share ntfs defaults,uid=1000,gid=999,fmask=0022,dmask=0000 0 0
docker-compose.yml:
version: "3.1"
services:
www:
build:
context: .
dockerfile: Dockerfile.lamp
user: 1000:1000
ports:
- "${WEBSERVER_PORT}:80"
volumes:
- ./www:/var/www/html/
links:
- db
networks:
- default
db:
image: mysql:8.0
user: 1000:1000
ports:
- "3306:3306"
command: --default-authentication-plugin=mysql_native_password
environment:
MYSQL_DATABASE: ${MYSQL_DATABASE}
MYSQL_USER: ${MYSQL_USER}
MYSQL_PASSWORD: ${MYSQL_PASSWORD}
MYSQL_ROOT_PASSWORD: ${MYSQL_PASSWORD}
volumes:
- ./sql:/docker-entrypoint-initdb.d
- ./conf:/etc/mysql/conf.d
- mysql_data:/var/lib/mysql
networks:
- default
phpmyadmin:
image: phpmyadmin/phpmyadmin
links:
- db:db
ports:
- ${PHPMYADMIN_PORT}:80
networks:
- default
environment:
MYSQL_USER: ${MYSQL_USER}
MYSQL_PASSWORD: ${MYSQL_PASSWORD}
MYSQL_ROOT_PASSWORD: ${MYSQL_PASSWORD}
UPLOAD_LIMIT: 64M
volumes:
mysql_data:
driver: local
driver_opts:
type: 'none'
o: 'bind'
device: './mysql_data'
networks:
default:
Any suggestions to how I could successfully achieve this? given that a mount has issues with permissions and will only set permissions on all files as given by the /etc/fstab file or default 1000:1000
Why default is 1000:1000?
The reason it set 1000:1000 as default is used to prevent docker created a file/folder own by root, once it own by root lot of situation will cause permission error if any process want to create or write file/folder, but if own by 1000:1000, you can change owner easily to which you want.
And also, set to 1000:1000 doesn't means you or any user have permissions to acceess with it, it still followed Linux permission rules, 1000:1000 is just Linux normal level user gid:uid count where it start with.
How mysql work around?
For example, Here is the mysql8.0 image (https://github.com/docker-library/mysql/blob/master/8.0/Dockerfile.oracle).
You can see it create a mysql:mysql user at beginning.
RUN set -eux; \
groupadd --system --gid 999 mysql; \
useradd --system --uid 999 --gid 999 --home-dir /var/lib/mysql --no-create-home mysql
And it changed these folders owner to mysql user on line 85, and it's the folder where mysql data stored by.
# ensure these directories exist and have useful permissions
# the rpm package has different opinions on the mode of `/var/run/mysqld`, so this needs to be after install
mkdir -p /var/lib/mysql /var/run/mysqld; \
chown mysql:mysql /var/lib/mysql /var/run/mysqld; \
# ensure that /var/run/mysqld (used for socket and lock files) is writable regardless of the UID our mysqld instance ends up having at runtime
chmod 1777 /var/lib/mysql /var/run/mysqld; \
What should you do?
Change your volume folder owner to which the container user, in mysql container you should change owner to mysql:mysql, or mysql won't able to write.
In other container do as same, change owner to which user your use to write/read, but docker compose can't volume as a specific user, so you might need try this: https://stackoverflow.com/a/56990338.
Notice that in your case user flag is not needed.

Building mysql / mariadb database with scema in docker

Hi I am building a service in which I need a Mysql/MariaDB database. I have been googling different solutions and I got the db started with a database created thanks to a guide a was following (never found the link again unfortunately).
Problem
The problem I am having is that the tables are not being created. I added the sql-scema file to /docker-entrypoint-initdb.d/ (you can check it down in the docker file) but it doesnt seem to be executing it (I have tried with both copy and ADD commands).
Current output
This is my current console output from the container:
[![image][1]][1]
The database is created but the SOW TABLES; command returns Empty Set.
Desired output
Since this db is going to be a service differents scripts connect to (currently python), I need to be able to create the db and the sql schema (tables, triggers, etc...) so my team can work with same configuration.
Some of the solutions I have tried (I cant find all the links i have visited only a few)
How to import a mysql dump file into a Docker mysql container
mysql:5.7 docker allow access from all hosts and create DB
Can't connect to mariadb outside of docker container
Mariadb tables are deleted when use volume in docker-compose
Project structure
The structure is pretty simple I am using the following docker-compose.yml
Docker-compose
I still have to try if the MARIADB_ enviroment variables are necessary here.
version: '3'
services:
db-mysql:
#image: mysql/mysql-server:latest
build: ./mysql-db
restart: always
container_name : db-music
ports:
- '3306:3306'
environment:
MYSQL_ROOT_PASSWORD: pwd
MYSQL_DATABASE : audio_service
MYSQL_USER : user
MYSQL_PASSWORD : password
environment:
MARIADB_ROOT_PASSWORD: pwd
MARIADB_DATABASE : audio_service
MARIADB_USER : user
MARIADB_PASSWORD : password
#https://stackoverflow.com/questions/29145370/how-can-i-initialize-a-mysql-database-with-schema-in-a-docker-container?rq=1
expose:
- '3306:3306'
volumes:
- type: bind
source : E:\python-code\Rockstar\volume\mysql
target : /var/lib/mysql
#- type: bind
#source : E:\python-code\Rockstar\mysql-db\sql_scripts\tables.sql
#target : /docker-entrypoint-initdb.d/init.sql
networks:
net:
ipam:
driver: default
config:
- subnet: 212.172.1.0/30
host:
name: host
external: true
Dockerfile
FROM mariadb:latest as builder
# That file does the DB initialization but also runs mysql daemon, by removing the last line it will only init
RUN ["sed", "-i", "s/exec \"$#\"/echo \"not running $#\"/", "/usr/local/bin/docker-entrypoint.sh"]
# needed for intialization
ENV MYSQL_ROOT_PASSWORD=root
ENV MYSQL_ROOT_PASSWORD = pwd
ENV MYSQL_DATABASE = audio_service
ENV MYSQL_USER = user
ENV MYSQL_PASSWORD = password
COPY sql_scripts/tables.sql /docker-entrypoint-initdb.d/
# Need to change the datadir to something else that /var/lib/mysql because the parent docker file defines it as a volume.
# https://docs.docker.com/engine/reference/builder/#volume :
# Changing the volume from within the Dockerfile: If any build steps change the data within the volume after
# it has been declared, those changes will be discarded.
RUN ["/usr/local/bin/docker-entrypoint.sh", "mysqld", "--datadir", "/initialized-db", "--aria-log-dir-path", "/initialized-db"]
FROM mariadb:latest
# needed for intialization
ENV MARIADB_ROOT_PASSWORD=root
ENV MARIADB_ROOT_PASSWORD = pwd
ENV MARIADB_DATABASE = audio_service
ENV MARIADB_USER = user
ENV MARIADB_PASSWORD = password
COPY --from=builder /initialized-db /var/lib/mysql
EXPOSE 3306
SQL schema file
create database audio_service;
use audio_service;
CREATE TABLE audio
(
audio_id BINARY(16),
title TEXT NOT NULL UNIQUE,
content MEDIUMBLOB NOT NULL,
PRIMARY KEY (audio_id)
) COMMENT='this table stores sons';
DELIMITER ;;
CREATE TRIGGER `audio_before_insert`
BEFORE INSERT ON `audio` FOR EACH ROW
BEGIN
IF new.audio_id IS NULL THEN
SET new.audio_id = UUID_TO_BIN(UUID(), TRUE);
END IF;
END;;
DELIMITER ;
There is no need to build your own image since the official mysql / mariadb images are already well suited. You only need to run them with the following as explained in their image documentations:
environment variables to initialize an new database with a respective user on the first run
a volume at /var/lib/mysql to persist the data
any initialization/sql scripts mounted into /docker-entrypoint-initdb.d
So storing your SQL* into a schema.sql file right next to the docker-compose.yml the following is enough to achieve what you want:
# docker-compose.yml
services:
db:
image: mariadb
environment:
MARIADB_ROOT_PASSWORD: pwd
MARIADB_DATABASE: audio_service
MARIADB_USER: user
MARIADB_PASSWORD: password
volumes:
# persist data files into `datadir` volume managed by docker
- datadir:/var/lib/mysql
# bind-mount any sql files that should be run while initializing
- ./schema.sql:/docker-entrypoint-initdb.d/schema.sql
volumes:
datadir:
*note that you can remove the CREATE DATABASE and USE statements from your schema.sql since these will be automatically done by the init script for you anyway
There are two reasons that your own setup isn't working as expected:
the line COPY --from=builder /initialized-db /var/lib/mysql won't work as expected for the same reason you described in your comment a bit above it: /var/lib/mysql is a volume and thus no new files a stored in it in the build steps after it was defined.
you are bind-mounting E:\python-code\Rockstar\volume\mysql to /var/lib/mysql in your docker-compose.yml.
But this will effectively override any contents of /var/lib/mysql of the image, i.e. although your own image built from your Dockerfile does include an initialized database this is overwritten by the contents of E:\python-code\Rockstar\volume\mysql when starting the service.

How do I set the perms of a my.cnf file to be readonly from within Docker (not at the client's OS level)?

I'm using docker-compose v 1.27 and Docker v 19.03. I have this in my docker-compose.yml file ...
version: '3'
services:
mysql:
restart: always
image: mysql:8.0
cap_add:
- SYS_NICE # CAP_SYS_NICE
environment:
MYSQL_DATABASE: 'directory_data'
# So you don't have to use root, but you can if you like
MYSQL_USER: 'root'
# You can use whatever password you like
MYSQL_PASSWORD: 'password'
# Password for root access
MYSQL_ROOT_PASSWORD: 'password'
MYSQL_ROOT_HOST: '%'
ports:
- "3406:3306"
volumes:
- my-db:/var/lib/mysql
- ./mysql/mysqlconf:/etc/mysql/conf.d
command: ['mysqld', '--character-set-server=utf8mb4', '--collation-server=utf8mb4_unicode_ci']
Note that I have no Dockerfile (didn't think I needed it). My "my.cnf" file, is below
davea$ cat mysql/mysqlconf/my.cnf
bind-address = 0.0.0.0
From Docker, how do I set the permissions of the my.cnf file to be read-only? This comes into play on Windows 10 in which running "docker-compose up" results in this warning
mysqld: [Warning] World-writable config file '/etc/mysql/conf.d/my.cnf' is ignored.
Note, this answer -- https://stackoverflow.com/questions/64327260/in-docker-compose-how-do-i-set-perms-on-a-my-cnf-file-if-i-dont-have-a-dockerf, doesn't cut it, because it relies on setting th
I think the underlying problem here is that you are mounting a NTFS directory volume inside of an ext filesystem. Below are some possible solutions that may be helpful.
Docker-level: Use Read-only Volume Mounts
You can use a read-only volume mount instead of the default read-write setting.
For example, add :ro (read-only) to the end of the volume specification:
volumes:
- ...
- ./mysql/mysqlconf:/etc/mysql/conf.d:ro
Container-level: chmod the configuration file
If you want to suppress the warning, you can try setting the permissions of the files at run-time to read-only by expanding the command configuration to several commands. I think this is what you are referring to as the client's OS level, though. The mount would not be read-only.
For example:
command: bash -c "
chmod -R 0444 /etc/mysql/conf.d/ &&
mysqld --user=root --character-set-server=utf8mb4 --collation-server=utf8mb4_unicode_ci
"
Note that this is incompatible with read-only mounts, as you cannot adjust the permissions since it is a read-only filesystem.
In your docker-compose yaml file, you can define the read-only access to the mounted volume
by adding :ro at the end of the volume definition.
version: '3'
services:
mysql:
restart: always
image: mysql:8.0
cap_add:
- SYS_NICE # CAP_SYS_NICE
environment:
MYSQL_DATABASE: 'directory_data'
# So you don't have to use root, but you can if you like
MYSQL_USER: 'root'
# You can use whatever password you like
MYSQL_PASSWORD: 'password'
# Password for root access
MYSQL_ROOT_PASSWORD: 'password'
MYSQL_ROOT_HOST: '%'
ports:
- "3406:3306"
volumes:
- my-db:/var/lib/mysql
- ./mysql/mysqlconf:/etc/mysql/conf.d:ro
command: ['mysqld', '--character-set-server=utf8mb4', '--collation-server=utf8mb4_unicode_ci']
I suggest you set the permissions in a custom entrypoint script. This ensures they are adjusted on every container start, plays nicely with the "official" mysql image (it has custom entry point script support baked in) and does not collide with docker best practices (keeps mysqld running as pid 1).
It's three steps.
Create a script that makes all necessary adjustments and make it executable:
cat <<EOF > ./adjust-permissions.sh
#!/bin/sh
set -ex
chown -R root:root /etc/mysql/conf.d/
chmod -R 0644 /etc/mysql/conf.d/
EOF
chmod +x ./adjust-permissions.sh
You might want to leave out chown, personally, I like to ensure there are no surprises with mounted files.
Mount it into /docker-entrypoint-initdb.d/ inside the container (see docker-entrypoint.sh):
volumes:
[...]
- ./adjust-permissions.sh:/docker-entrypoint-initdb.d/adjust-permissions.sh
Enjoy.

Moving Wordpress site to Docker: Error establishing DB connection

Ive been making new sites with Wordpress & Docker recently and have a reasonable grasp of how it all works and Im now looking to move some established sites into Docker.
Ive been following this guide:
https://stephenafamo.com/blog/moving-wordpress-docker-container/
I have everything setup as it should be but when I go to my domain.com:1234 I get the error message 'Error establishing a database connection'. I have changed 'DB HOST' to 'mysql' in wp-config.php as advised and all the DB details from the site Im bringing in are correct.
I have attached to the mysql container and checked that the db is there and with the right user and also made sure the pw is correct via mysql CLI too.
SELinux is set to permissive and I havent changed any dir/file ownership nor permissions and for the latter dirs are all 755 and files 644 as they should be.
Edit: I should mention that database/data and everything under that seem to be owned by user/group 'polkitd input' instead of root.
Docker logs aren't really telling me much either apart from the 500 error messages for the WP container when I browse the site on port 1234 (as expected though).
This is the docker-compose file:
version: '2'
services:
example_db:
image: mysql:latest
container_name: example_db
volumes:
- ./database/data:/var/lib/mysql
- ./database/initdb.d:/docker-entrypoint-initdb.d
restart: always
environment:
MYSQL_ROOT_PASSWORD: password123 # any random string will do
MYSQL_DATABASE: mydomin_db # the name of your mysql database
MYSQL_USER: my domain_me # the name of the database user
MYSQL_PASSWORD: password123 # the password of the mysql user
example:
depends_on:
- example_db
image: wordpress:php7.1 # we're using the image with php7.1
container_name: example
ports:
- "1234:80"
restart: always
links:
- example_db:mysql
volumes:
- ./src:/var/www/html
Suggestions most welcome as Im out of ideas!
With the new version of docker-compose it will look like this (if you don't want to use PhpMyAdmin you can leave it out):
version: '3.7'
volumes:
wp-data:
networks:
wp-back:
services:
db:
image: mysql:5.7
volumes:
- wp-data:/var/lib/mysql
environment:
MYSQL_ROOT_PASSWORD: rootPassword
MYSQL_DATABASE: wordpress
MYSQL_USER: wp-user
MYSQL_PASSWORD: wp-pass
ports:
- 8889:3306
networks:
- wp-back
phpmyadmin:
depends_on:
- db
image: phpmyadmin/phpmyadmin
environment:
PMA_HOST: db
MYSQL_USER: wp-user
MYSQL_PASSWORD: wp-pass
MYSQL_ROOT_PASSWORD: rootPassword
ports:
- 3001:80
networks:
- wp-back
wordpress:
depends_on:
- db
image: wordpress:latest
ports:
- 8888:80
- 443:443
environment:
WORDPRESS_DB_HOST: db
WORDPRESS_DB_USER: wp-user
WORDPRESS_DB_PASSWORD: wp-pass
volumes:
- ./wordpress-files:/var/www/html
container_name: wordpress-site
networks:
- wp-back
The database volume is a named volume wp-data, while the wordpress html is a bind-mount to your current directory ./wordpress-files .
make sure that the wp-config.php file has same credentials defined for db_user, db_password as in docker-composer yml file. I too had similar problem i deleted all the files and re-installed and saw that docker-composer up -d would start everything but the wp-config.php file contents for mysql settings were not defined as in docker. so i changed it accordingly and started working eventually
Please take a look at the following compose script. I tried and tested. It works fine.
version: '2'
services:
db:
image: mysql:latest
container_name: db_server
volumes:
- ./database/data:/var/lib/mysql
- ./database/initdb.d:/docker-entrypoint-initdb.d
restart: always
environment:
MYSQL_ROOT_PASSWORD: password123 # any random string will do
MYSQL_DATABASE: udb_test # the name of your mysql database
MYSQL_USER: me_prname # the name of the database user
MYSQL_PASSWORD: password123 # the password of the mysql user
example:
depends_on:
- db
image: wordpress:php7.1 # we're using the image with php7.1
container_name: wp-web
environment:
WORDPRESS_DB_HOST: db:3306
WORDPRESS_DB_USER: me_prname
WORDPRESS_DB_PASSWORD: password123
WORDPRESS_DB_NAME: udb_test
ports:
- "1234:80"
restart: always
volumes:
- ./src:/var/www/html
Let me know if you encounter further issues.
if you want it all in one container you can refer this repo here,
https://github.com/akshayshikre/lamp-alpine/tree/development
Here from lamp-alpine image is used
Then mysql, php, apache2 (lamp stack) is installed and copied local wordpress demosite and db for demo purpose
if you do not want any kind of continuous integration part ignore .circleci folder
Check docker-compose file and Dockerfile, Environment variables are in .env file
I share with you my approach
Show running version, question to see if all is well on your side!
$ docker --version && docker-compose --version
run Docker Copose file
$ docker-compose -f docker-compose.yml up -d
after you wait fast forward
show running containers and name of the Wordpress Container is listening on port 8000
$ docker ps
you will see the name of your WordPress container on the table as follows if you have followed the steps listed on their site
https://hub.docker.com/_/wordpress
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
xxxxxxxxxxxx wordpress:latest "docker-entrypoint.s…" 8 minutes ago Up 8 minutes 0.0.0.0:8000->80/tcp cms_wordpress_1
xxxxxxxxxxxx mysql:5.7 "docker-entrypoint.s…" 8 minutes ago Up 8 minutes 3306/tcp, 33060/tcp cms_db_1
and if you check your browser with the address : localhost:8000
you will get the message "error establishing DB connection"
launch bash inside the Wordpress container
$ docker exec -it cms_wordpress_1 bash
apt update fails as there is no connectivity
$ apt update
open up new terminal and show current Firewalld configuration
$ sudo cat /etc/firewalld/firewalld-workstation.conf | greb 'FirewallBackend'
currently set to 'nftables'
set value to 'iptables'
$ sudo sed -i 's/FirewallBackend=nftables/FirewallBackend=iptables/g' /etc/firewalld/firewalld-workstation.conf
confirme new value
$ sudo cat /etc/firewalld/firewalld-workstation.conf | grep 'FirewallBackend'
restart Firwalld service to apply change
$ sudo systemctl restart firewalld.service
Refresh the running Wordpress session in your browser and that's good.
good work.
In some cases a probable cause of this issue could be, you have made volumes using docker compose up and then when you did docker compose down you expected the volumes to be deleted as well as the docker images, but this is not how it works.
From the doc you could read this:
For data that needs to persist between updates, use host or named volumes.
It implicitly means that named volumes will not get deleted with down, so what happens is, when you do an up and then add a row to a table and then do a subsequent down, then on the next up you will get the same old volume and so querying the same table would give you the same row you created previously!
What does this have to do with the error Error establishing DB connection, you may ask. To answer your question, let's assume one scenario: What if you changed some MYSQL passwords in the docker compose file in between running the down command and the second up command?
MYSQL keeps its own data just like any other data in its tables, so when you do the second up, Docker loads the old volume (the one created by the first up) and thus the old credential information will be used by MYSQL and Docker will not even have the opportunity to insert your new information (the ones you changed in the docker compose file) in the administration tables. So obviously, you will be rejected.
The solution thus now would be very simple. To fix it, either do:
docker-compose down -v
to remove the named volumes as well as the images when running the down, or do:
docker volume rm [volname]
if you've done the down before, and now you want to delete the named volumes.
If you follow this tutorials ,https://stephenafamo.com/blog/moving-wordpress-docker-container/, your site wil not work properly. Coz It doesn't restore database and you need to restore manually .sql dump file existed in initdb.d dir by using this command.
cat backup.sql | docker exec -i CONTAINER /usr/bin/mysql -u root --password=root DATABASE
I also stuck in this and my CSS are not working properly.
Please let me know when you have new idea .

Enable logging in docker mysql container

I'm trying to get familiar with the docker ecosystem and tried to setup a mysql database container. With docker-compose this looks like:
version: '2'
services:
db:
image: mysql:5.6.33#sha256:31ad2efd094a1336ef1f8efaf40b88a5019778e7d9b8a8579a4f95a6be88eaba
volumes:
- "./db/data:/var/lib/mysql"
- "./db/log:/var/log/mysql"
- "./db/conf:/etc/mysql/conf.d"
restart: "yes"
environment:
MYSQL_ROOT_PASSWORD: rootpw
MYSQL_DATABASE: db
MYSQL_USER: db
MYSQL_PASSWORD: dbpw
My conf directory contains one file:
[mysqld]
log_error =/var/log/mysql/mysql_error.log
general_log_file=/var/log/mysql/mysql.log
general_log =1
slow_query_log =1
slow_query_log_file=/var/log/mysql/mysql_slow.log
long_query_time =2
log_queries_not_using_indexes = 1
Unfortunately I don't get any log files that way. The setup itself is correct and the cnf file is used. After connecting to the container and creating the 3 files, chown them to mysql and restarting the container, the logging is working as expected.
I'm pretty sure that this is a common scenario, and my current way to get it running seems really stupid. What is the correct way to do it?
I could improve my approach by moving all this stuff in a Dockerfile, but this still seem strange to me.
I was looking for the exact same thing, and now, there is a better way to do it.
The docker mysql writes:
Many configuration options can be passed as flags to mysqld. This will
give you the flexibility to customize the container without needing a
cnf file. For example, if you want to change the default encoding and
collation for all tables to use UTF-8 (utf8mb4) just run the
following:
$ docker run --name some-mysql -e MYSQL_ROOT_PASSWORD=my-secret-pw -d mysql:tag --character-set-server=utf8mb4 --collation-server=utf8mb4_unicode_ci
In a docker-compose world, one could pass these arguments through the "command" section of the service:
command: mysqld --character-set-server=utf8mb4 --collation-server=utf8mb4_unicode_ci
In my use case I just wanted to turn on the logs and specify the path to the log file :
command: mysqld --general-log=1 --general-log-file=/var/lib/mysql/general-log.log
With the adequate volumes (e.g. - ./logs/mysql.log:/var/lib/mysql/general-log.log), it becomes easy to reach them.
This is pretty straight forward and avoid dealing with a local configuration. It will works with any MySQL Docker images and will keep the my.cnf as shipped by the image.
Edit: change path from /var/log/mysql/ to /var/lib/mysql/ to ensure a MySQL writable folder.
After connecting to the container and creating the 3 files, chown them to mysql and restarting the container, the logging is working as expected.
That points to a host volume permission issue. When you map from a container to the host, no mappings are made on user id's, and the name attached to the uid inside the container may be very different from outside. You need to initialize the directory permissions with something the container user can write to. One simple method is to create a group that has access to write to the files on both the host and container, and then add the various users to this group on both your image and host OS. Another option is to use a named filesystem that you don't access directly from your host and initialize it with the image's directory permissions.
Edit: An example of a named volume with your docker-compose.yml is as simple as:
version: '2'
volumes:
mysql-data:
driver: local
mysql-log:
driver: local
mysql-conf:
driver: local
services:
db:
image: mysql:5.6.33
volumes:
- "mysql-data:/var/lib/mysql"
- "mysql-log:/var/log/mysql"
- "mysql-conf:/etc/mysql/conf.d"
restart: unless-stopped
environment:
MYSQL_ROOT_PASSWORD: rootpw
MYSQL_DATABASE: db
MYSQL_USER: db
MYSQL_PASSWORD: dbpw
Note that I also removed the sha256 from your image name, this reference would block you from being able to pull patched versions of the image. I also prefer the "unless-stopped" restart policy so that Docker does expected things on a reboot.
I needed to temporarily enable logging due to a weird PDO binding issue and I wanted to see the actual query being executed. This question was the top search result and I wasn't satisfied with any of the answers. Assuming you already have volumes setup for the container, I got it working the following way:
Run the following queries on the database:
SET global general_log = on;
SET global general_log_file='/var/log/mysql/mysql.log';
SET global log_output = 'file';
Get the container ID using docker ps.
Run docker exec -it <id> /usr/bin/tail -f /var/log/mysql/mysql.log
Run the following query once you're done: SET global general_log = off;
If you're having problems setting the general_log_file variable, you probably need to /bin/bash into the container then manually create the log file with the correct permissions.
In addition to ponsfrilus answer
you can do the same from docker run command:
docker run --name some-mysql --restart always -v //d/MySqlDockerData:/var/lib/mysql -e MYSQL_ROOT_PASSWORD=my-secret-pw -d mysql:latest --character-set-server=utf8mb4 --collation-server=utf8mb4_unicode_ci --log_error=/var/lib/mysql/mysql_error.log --general_log_file=/var/lib/mysql/mysql.log --general_log=1 --slow_query_log=1 --slow_query_log_file=/var/lib/mysql/mysql_slow.log --long_query_time=2 --log_queries_not_using_indexes=1