I have a mysql 5.7 docker container. When I run the mysql command:
SELECT now();
It shows the time -3 hours to my current time (which is logical). I want to set the time zone in a config file. Following the documentation in https://hub.docker.com/_/mysql/ I create a volume in my docker-compose.yml file like the following:
mysqldb:
image: mysql:5.7.21
container_name: mysql_container
ports:
- "3306:3306"
volumes:
- ./.docker/etc/mysql/custom.cnf:/etc/mysql/conf.d/custom.cnf
When I browse the files inside the container the file custom.cnf is there.
In that file, I tried some of the ways I found as solutions like:
[mysqld]
default_time_zone='Europe/Sofia'
or a compromise solution which is less elegant as the zone will have to be changed twice a year (summer / winter):
[mysqld]
default_time_zone='+03:00'
but none works. I have the sensation the this file is not loaded by mysql at all because if I try to put invalid configuration in there nothing happens either (the container starts normally).
Any suggestions on that?
So you need to use a Dockerfile in this case and handle it like below
FROM mysql:5.7.21
RUN echo "USE mysql;" > /docker-entrypoint-initdb.d/timezones.sql && mysql_tzinfo_to_sql /usr/share/zoneinfo >> /docker-entrypoint-initdb.d/timezones.sql
This makes sure that when the mysql container loads it will load all the timezone info. Now you can use it using environment variables
Environment variable
mysqldb:
#image: mysql:5.7.21
build: .
#container_name: mysql_container
ports:
- "3306:3306"
environment:
- MYSQL_ROOT_PASSWORD=root
- TZ=Europe/Sofia
To use it with a config file is a problem. When you start the DB it will give your an error
mysqldb_1 | 2018-04-24T12:29:43.169214Z 0 [Warning] InnoDB: New log files created, LSN=45790
mysqldb_1 | 2018-04-24T12:29:43.215187Z 0 [Warning] InnoDB: Creating foreign key constraint system tables.
mysqldb_1 | 2018-04-24T12:29:43.281229Z 0 [Warning] No existing UUID has been found, so we assume that this is the first time that this server has been started. Generating a new UUID: 2a87fec6-47bb-11e8-9f1e-0242ac110002.
mysqldb_1 | 2018-04-24T12:29:43.284010Z 0 [Warning] Gtid table is not ready to be used. Table 'mysql.gtid_executed' cannot be opened.
mysqldb_1 | 2018-04-24T12:29:43.284404Z 0 [ERROR] Fatal error: Illegal or unknown default time zone 'Europe/Sofia'
mysqldb_1 | 2018-04-24T12:29:43.284567Z 0 [ERROR] Aborting
This is because it needs the timezone to have been already loaded. It is possible to fix this also, but too much of hassle. I will go with the environment variable only as that means when the container starts the timezone is already setup.
The mariadb:10.3 docker container interprets the TZ environment variable, although it's not mentioned on the Readme.md for the docker image. In my case providing the value of America\New_York worked perfectly.
value should be
default_time_zone=Europe/Sofia
check details
But this could give you error like this while restarting the mysql service so you should use below command before editing custom.cnf file.
mysql_tzinfo_to_sql /usr/share/zoneinfo | mysql -u root mysql -p
For me, I'm using Keramatic https://kitematic.com configure by click on MySQL container General and add TZ specify with your current time show list in https://en.wikipedia.org/wiki/List_of_tz_database_time_zones
Cheers
Related
I had a spring-boot project that used mysql docker-image so I didn't need to download the mysql benchwork. For other reasons I had to start over so I created a new project that uses the same mysql docker image I previously used.
My docker-compose.yml mysql service looks like this
version: "3.7"
services:
db:
image: mysql:5.7
command: --lower_case_table_names=1
environment:
MYSQL_ROOT_PASSWORD: root
MYSQL_DATABASE: farming_db
MYSQL_USER: root
MYSQL_PASSWORD: root
restart: always
volumes:
- "./database/farming_db/:/var/lib/mysql" #local
- farming_db:/var/lib/mysql/data #docker
ports:
- "3306:3306"
container_name: farming_mysql
networks:
- backend-network
When I run
docker-compose up
This is the error :
Attaching to farming_mysql, farming_server_springboot_1
farming_mysql | 2021-03-18 07:03:20+00:00 [Note] [Entrypoint]: Entrypoint script for MySQL Server 5.7.33-1debian10 started.
farming_mysql | 2021-03-18 07:03:20+00:00 [Note] [Entrypoint]: Switching to dedicated user 'mysql'
farming_mysql | 2021-03-18 07:03:20+00:00 [Note] [Entrypoint]: Entrypoint script for MySQL Server 5.7.33-1debian10 started.
farming_mysql | 2021-03-18 07:03:21+00:00 [Note] [Entrypoint]: Initializing database files
farming_mysql | 2021-03-18T07:03:21.058436Z 0 [Warning] TIMESTAMP with implicit DEFAULT value is deprecated. Please use --explicit_defaults_for_timestamp server opti
on (see documentation for more details).
farming_mysql | 2021-03-18T07:03:21.063630Z 0 [ERROR] --initialize specified but the data directory has files in it. Aborting.
farming_mysql | 2021-03-18T07:03:21.063710Z 0 [ERROR] Aborting
farming_mysql |
farming_mysql exited with code 1
springboot_1 |
I understood that my directory is not empty. I am trying to use "./database/farming_db/:/var/lib/mysql" and "farming_db:/var/lib/mysql/data" both as the volume directories. I think the problem is with the latter directory because the prior directory is empty. I'm having a problem deleting the contents in the latter directory because I don't know how to access it.
So this is what I've tried :
I deleted all the containers and then deleted all the volumes.docker volume prune but didn't work.
I searched that I could do rm -rf /usr/local/var/mysql but I don't know where I can execute this command since the container won't run properly at all.
I deleted the mysql image and just ran docker-compose up again. This seems to pull a new mysql image from somewhere? but I still get the same error. I guess volume directory has nothing do with the docker image itself.
I deleted the "- farming_db:/var/lib/mysql/data #docker" line from the docker-compose. But the same error is still occuring!
I'm using Windows10.
My question :
How can I access the directory? I don't know where to use the rm -rf command.
Why does this error still occur even when I erase "- farming_db:/var/lib/mysql/data #docker" from the docker-compose?
And also could anyone explain what I am doing? I'm new to docker and I don't really understand these volume problems.
Run docker system prune --volumes
This frees up the memory by removing all unused containers. Sometimes, the mentioned issue can occur due to memory limitations
Generally I emptied the volume's data directory and just changed the versions of the MySQL.
So in steps:
empty volume directory content
modify docker-compose.yml mysql version from 5.7 to 5.7.16
This line indicate that mysql container is storing the data inside a directory database in the same directory than your docker-compose.yml:
volumes:
- "./database/farming_db/:/var/lib/mysql" #local
This kind of volume isn't managed by Docker, it's just a directory in your filesystem, this is why docker volume prune doesn't work. I know that, because it starts with a "path" relative or absolute.
The other volume, farming_db, are managed by Docker. I know that because it starts with a simple name. This kinds of volume are managed by Docker and are removed with prune.
So, answering:
In the same directory than your docker-compose.yml you can remove that database folder.
Because the first volume, the one with /var/lib/mysql still exists. MySQL keeps all files inside this directory and any other child directory are a database.
You're just trying to put a container running and docker-compose hides a lot of details.
This is just a detail, but MYSQL_USER should be different than root.
You can let Docker manage the entire volume, creating a single volume to hold all data, in this case I named it as mysql_data:
volumes:
- mysql_data:/var/lib/mysql
Or, you can explore a bit more the docker run equivalent command to get used with it:
docker run -d --name mysql \
-e MYSQL_ROOT_PASSWORD=root \
-e MYSQL_DATABASE=farming_db \
-e MYSQL_USER=myuser \
-e MYSQL_PASSWORD=mypass \
-v mysql_data:/var/lib/mysql \
-p 3306:3306 \
mysql:5.7
As vencedor's answer, it worked for me. If anyone need stay with mysql 5.7, you can add these lines to your db service in docker-compose.yml:
- /etc/group:/etc/group:ro
- /etc/passwd:/etc/passwd:ro
user: "1000:1000"
I used docker-compose to run mysql image and encountered the error.
I use the following configuration to set volume.
- ./mysql/data:/var/lib/mysql/data
Then I changed it to the following and the error was solved.
- ./mysql:/var/lib/mysql
I have a docker-compose.yml with a MariaDB database service that looks like this:
version: '2'
services:
db:
container_name: misp_db
image: mariadb
restart: unless-stopped
volumes:
- ${DATA_DIR}/db:/var/lib/mysql
environment:
- MYSQL_DATABASE=${MYSQL_DATABASE}
- MYSQL_USER=${MYSQL_USER}
- MYSQL_PASSWORD=${MYSQL_PASSWORD}
- MYSQL_ROOT_PASSWORD=${MYSQL_ROOT_PASSWORD}
When spinning up my containers, I noticed these errors:
[ERROR] Missing system table mysql.roles_mapping; please run mysql_upgrade to create it
[ERROR] Incorrect definition of table mysql.event: expected column 'sql_mode' at position 14 to have type set('REAL_AS_FLOAT','PIPES_AS_CONCAT','ANSI_QUOTES','IGNORE_SPACE','IGNORE_BAD_TABLE_OPTIONS','ONLY_FULL_GROUP_BY','NO_UNSIGNED_SUBTRACTION','NO_DIR_IN_CREATE','POSTGRESQL','ORACLE','MSSQL','DB2','MAXDB','NO_KEY_OPTIONS','NO_TABLE_OPTIONS','NO_FIELD_OPTIONS','MYSQL323','MYSQL40','ANSI','NO_AUTO_VALUE_ON_ZERO','NO_BACKSLASH_ESCAPES','STRICT_TRANS_TABLES','STRICT_ALL_TABLES','NO_ZERO_IN_DATE','NO_ZERO_DATE','INVALID_DATES','ERROR_FOR_DIVISION_BY_ZERO','TRADITIONAL','NO_AUTO_CREATE_USER','HIGH_NOT_PRECEDENCE','NO_ENGINE_SUBSTITUTION','PAD_CHAR_TO_FULL_LENGTH','EMPTY_STRING_IS_NULL','SIMULTANEOUS_ASSIGNMENT'), found type set('REAL_AS_FLOAT','PIPES_AS_CONCAT','ANSI_QUOTES','IGNORE_SPACE','NOT_USED','ONLY_FULL_GROUP_BY','NO_UNSIGNED_SUBTRACTION','NO_DIR_IN_CREATE','POSTGRESQL','ORACLE','MSSQL','DB2','MAXDB','NO_KEY_OPTIONS','NO_TABLE_OPTIONS','NO_FIELD_OPTIONS','MYSQL323','MYSQL40','ANSI','NO_AUTO_VALUE_ON_ZERO','NO_B
[ERROR] mysqld: Event Scheduler: An error occurred when initializing system tables. Disabling the Event Scheduler.
I was able to work around this problem by running docker exec -it db /bin/bash to get a shell, and then running mysql_upgrade -p and entering my configured MySQL root password, but this should be automated.
How can I do this in Docker Compose?
You can't run it directly from the docker-compose but it can be done rather simply using a one-line Dockerfile called from docker-compose.
This can be done by simply adding the following to your service definition in the docker-compose.yaml file:
build: .
This will cause docker-compose to run the Dockerfile located in the working directory ".".
In the Dockerfile, you can then run the upgrade script itself.
RUN mysql_upgrade -p
This will cause the upgrade to be run once; while building the image. Subsequently, your mySQL version will be frozen at the time you build the image; not when you start the container. This seems like a reasonable approach to me.
You could run an entrypoint.sh script from the Dockerfile (see here for more details), which gets called when you start the container. That would perform the upgrade when you start the container as you originally requested but, I agree with #blokje5, upgrading when starting the container seems a bit risky. If they accidentally break something (it happens), then an image that once worked would no longer be working and that can be a bit of a nightmare to debug; especially since it's happening before the service is considered ready.
I would like to containerize an existing mysql database linking the data directory inside my container to my local file system.
This docker run command works fine, i.e. the database is accessible and data is persisted.
docker run -v ./mydata:/var/lib/mysql mysql:5.7
However, I am unable to make it work with docker-compose.
I am getting the following errors
[ERROR] Can't open the mysql.plugin table. Please run mysql_upgrade to create it.
...
Fatal error: Can't open and lock privilege tables: Table 'mysql.user' doesn't exist
My docker-compose file looks like this:
version: "3.7"
services:
database :
restart: always
image: mysql:5.7
command: --port=4406
volumes:
- ./mydata:/var/lib/mysql
What is wrong with my compose-file?
UPDATE:
The content of mydata is
auto.cnf
ib_buffer
pool
ib_logfile0
ib_logfile1
ibdata1
mysql
I also tried ./mydata/mysql:/var/lib/mysql for volumes. That lets me connect to the db (through mysql workbench for instance), but I get the another error (DESCRIBE mysql.db File not found) when I try to see the users.
I am building an application that uses nodeJS and backend and mySQL as backend, and currently, my steps to bring up the app (without docker) is by:
Install NodeJS
Install MYSQL
Launch mysqld on port 3306
Manually create a MYSQL user dedicated for the NodeJS backend. This
user should have only basic previliges to only my desired schema.
Run sequelize commands to perform data migration and seeding using
the user generated in 4)
npm install and npm start to launch NodeJS on port 8080
Now I want to dokerize my application, and I already have the following Dockerfile:
#node version: carbon
#app version: 1.0.0
FROM node:8.11.2
WORKDIR /usr/src/app
COPY package*.json ./
RUN npm install
COPY . .
EXPOSE 8080
I have put a init.sql file within ./docker_db folder which does the following:
CREATE USER 'app_user'#'%' IDENTIFIED BY 'password';
CREATE SCHEMA `myapp` DEFAULT CHARACTER SET utf8;
GRANT INSERT, CREATE, ALTER, UPDATE, SELECT, REFERENCES on myapp.*
TO 'app_user'#'%' IDENTIFIED BY 'password'
WITH GRANT OPTION;
and the following docker-compose.yaml:
version: '3.6'
services:
mysql1:
image: mysql/mysql-server:5.7
environment:
MYSQL_ROOT_PASSWORD: password
ports:
- "127.0.0.1:3306:3306"
volumes:
- type: bind
source: ./docker_db
target: /docker-entrypoint-initdb.d
expose:
- "3306"
networks:
- app-network
myapp:
build:
context: .
dockerfile: Dockerfile
command: npm start
depends_on:
- mysql1
ports:
- "127.0.0.1:8080:8080"
expose:
- "8080"
links:
- mysql1
networks:
- app-network
command: ["./wait-for-db.sh"]
networks:
app-network:
driver: bridge
where my ./wait-for-db.sh does the following:
#!/bin/bash
until mysql -h mysql1 -u app_user -p password -e 'select 1'; do
echo "still waiting for mysql"; sleep 1; done
exec node ./db/scripts/generateSequelizeCLIConfig.js
exec node_modules/sequelize-cli/bin/sequelize db:migrate
exec node_modules/sequelize-cli/bin/sequelize db:seed:all
exec npm start
(BTW I do want to expose 3306 to host machine so that I can use workbench to connect to the mysql server, which I have successfully connected.)
In my sequelize config file I do have:
"username": "app_user",
"password": "password",
"database": "myapp",
"host": "mysql1",
"port": "3306"
With the above setting, I executed docker-compose up, and then I got the following lines:
mysql1_1 | [Entrypoint] MySQL Docker Image 5.7.22-1.1.5
mysql1_1 | [Entrypoint] Initializing database
myapp_1 | standard_init_linux.go:190: exec user process caused "no such file or directory"
myapp_myapp_1 exited with code 1
mysql1_1 | [Entrypoint] Database initialized
mysql1_1 | Warning: Unable to load '/usr/share/zoneinfo/iso3166.tab' as time zone. Skipping it.
mysql1_1 | Warning: Unable to load '/usr/share/zoneinfo/leapseconds' as time zone. Skipping it.
mysql1_1 | Warning: Unable to load '/usr/share/zoneinfo/tzdata.zi' as time zone. Skipping it.
mysql1_1 | Warning: Unable to load '/usr/share/zoneinfo/zone.tab' as time zone. Skipping it.
mysql1_1 | Warning: Unable to load '/usr/share/zoneinfo/zone1970.tab' as time zone. Skipping it.
mysql1_1 |
mysql1_1 | [Entrypoint] running /docker-entrypoint-initdb.d/init.sql
mysql1_1 |
mysql1_1 |
mysql1_1 | [Entrypoint] Server shut down
mysql1_1 |
mysql1_1 | [Entrypoint] MySQL init process done. Ready for start up.
mysql1_1 |
mysql1_1 | [Entrypoint] Starting MySQL 5.7.22-1.1.5
The problems now I face are:
1) The script's execution is hanging on the last line of Starting MySQL 5.7.22-1.1.5 and not going anywhere.
2) In the output, the 3rd and 4th lines shows an error about exec user process caused "no such file or directory". I don't think it is caused by the commands in the wait-for-db.sh because if I removed the lines after the until command, the problem still persist. In fact, I doubt the command execution ever reaching those lines and it feels like it is still within the until command.
I think it's really close to the final solution though :)
Use the name of your db service, which is mysql, as your database host. Docker will resolve it to the actually IP. Also why do you have FROM mysql:5.7 in your Dockerfile, I don't think it is of any uses.
Updated
Alright, seems like myapp runs db scripts before the db is ready. See here for solution https://docs.docker.com/compose/startup-order/
The problem is probably related to timing. Both containers will start at the same time and your node-app will try to connect to mysql almost immediately, while the MySQL server is still starting.
docker-compose doesn't have any kind of structure for this so you will have to build an entrypoint in your node-app that first waits for mysql to respond.
So, in your case, the entrypoint would be something like
#!/bin/bash
until mysql -h mysql1 -uapp_user -ppassword -e'select 1'; do echo "still waiting for mysql"; sleep 1; done
exec npm start
Using Laradock
System Info:
Docker version: 17.10.0-ce, build f4ffd25
OS: Windows 10 Home
When I run docker-compose up -d mysql I'm getting error. Following is the docker logs
[Note] Basedir set to /usr/
[Warning] The syntax '--symbolic-links/-s' is deprecated and will be removed in a future release
[Warning] 'NO_ZERO_DATE', 'NO_ZERO_IN_DATE' and 'ERROR_FOR_DIVISION_BY_ZERO' sql modes should be used with strict mode. They will be merged with strict mode in a future release.
[ERROR] --initialize specified but the data directory has files in it. Aborting.
[ERROR] Aborting
I have tried deleting mysql folder under ~/.laradock\data and didn't work.
Update 1
MySQL Container under laradock Dockerfile
mysql:
build:
context: ./mysql
args:
- MYSQL_VERSION=${MYSQL_VERSION}
environment:
- MYSQL_DATABASE=${MYSQL_DATABASE}
- MYSQL_USER=${MYSQL_USER}
- MYSQL_PASSWORD=${MYSQL_PASSWORD}
- MYSQL_ROOT_PASSWORD=${MYSQL_ROOT_PASSWORD}
- TZ=${WORKSPACE_TIMEZONE}
volumes:
- ${DATA_SAVE_PATH}/mysql:/var/lib/mysql
- ${MYSQL_ENTRYPOINT_INITDB}:/docker-entrypoint-initdb.d
ports:
- "${MYSQL_PORT}:3306"
networks:
- backend
MySQL Dockerfile
ARG MYSQL_VERSION=8.0
FROM mysql:${MYSQL_VERSION}
MAINTAINER Mahmoud Zalt <mahmoud#zalt.me>
#####################################
# Set Timezone
#####################################
ARG TZ=UTC
ENV TZ ${TZ}
RUN ln -snf /usr/share/zoneinfo/$TZ /etc/localtime && echo $TZ > /etc/timezone
RUN chown -R mysql:root /var/lib/mysql/
ADD my.cnf /etc/mysql/conf.d/my.cnf
CMD ["mysqld"]
EXPOSE 3306
Update 2
After I delete mysql folder under ~/.laradock/data I'm getting following error. After the command it generates the files in below image. When I rerun giving back the previous error mentioned above.
[Note] Basedir set to /usr/
[Warning] The syntax '--symbolic-links/-s' is deprecated and will be removed in a future release
[Warning] 'NO_ZERO_DATE', 'NO_ZERO_IN_DATE' and 'ERROR_FOR_DIVISION_BY_ZERO' sql modes should be used with strict mode. They will be merged with strict mode in a future release.
[Warning] Setting lower_case_table_names=2 because file system for /var/lib/mysql/ is case insensitive
[Warning] You need to use --log-bin to make --log-slave-updates work.
libnuma: Warning: /sys not mounted or invalid. Assuming one node: No such file or directory
mbind: Operation not permitted
[ERROR]
InnoDB: Operating system error number 22 in a file operation.
[ERROR] InnoDB: Error number 22 means
'Invalid argument'
[ERROR] InnoDB: File
./ib_logfile101: 'aio write' returned OS error 122. Cannot continue
operation
[ERROR] InnoDB: Cannot
continue operation.
** I tried in a windows 7 machine and its working.
Disable AIO
This fixed it for me when I got the AIO error as you did when I was starting a container from a guest Debian OS from Virtualbox and creating the database files on a shared folder on Windows 10.
The issue seems to be that AIO is not supported on shared folders, or at least on some versions of Windows. It seems to have occurred for me after I moved from Windows 10 Pro to Home after my main machine crashed.
For details:
aio
disable aio in MySQL for zfs
Here are some options:
Option 1 - start the container like this :
docker run -it mysql --innodb_use_native_aio=0
Option 2 - add the command to your docker-compose file:
command: --innodb_use_native_aio=0
In context, this is the relevant portion of my working docker-compose.yml:
services:
db:
image: ${MYSQL_IMAGE}
command: "--innodb_use_native_aio=0"
volumes:
- ${DB_DATA_PATH}:/var/lib/mysql
ports:
- ${MYSQL_PORT}:3306
Option 3 -- add an option to your my.cnf file in your build
innodb_use_native_aio=0
Option 4 - Don't persist your DB on the local file system.(Can destroy your db, Not Recommended)
Simply remove the volume in your docker configuration that contains your mysql db. Of course, your DB will be deleted if you do a docker-compose down or otherwise destroy your container, so there's that.
I am not sure but Try this step.This is because of data folder.
Try remove the docker image. List all images using docker images then remove the mysql & laradock mysql using docker rmi imagename. Dont forget remove the docker volume using docker volume rm volumename
Also go to cd ~/.laradock/data remove the mysql folder.
Then try docker-compose up mysql for debuging. If no errors, you could try docker-compose up -d mysql
I had the same problem with my Windows 10 Enterprise, I couldn't find perfect solution around it because it seems to be my Windows version issue - I have ran the same image on other windows (8 & 10 professional) successfully. My temporary work around was to remove the /var/lib/mysql mount entirely from the docker-compose file allowing the database data files to be created and modified in the container itself.
I use kitematics which will restart and reattach my created container anytime I want to work on the container, like that I don't loose my data when the container exit. If you don't use kitematics this what it does.
After running docker-compose the image will be created and a container will be created to run it as well, I avoid using run on the created image because this will create new container
I stick to running the following sequentially
docker ps -a # this command will get container_id of all container, those that are running and those that are not
docker start <container_id> # start container from background
docker attach <container_id> # attach container to standard input
With these docker commands my data were preserved in the container even after exiting and restarting
BACK UP
Now whenever I want to move the data, I will commit my container into an image and save the image to local storage
docker commit <container_id> <backup_image_name>:<1_31_2017> # I use date to tag it
docker save -o <local_storage_tar_name> <backup_image_name>
RESTORE
Whenever I need to restore the mysql container either on my computer or for a new programming intern
cd <dir_containing_the_tar_file>
docker load -o <local_storage_tar_name>
This solution is for your update
File ./ib_logfile101: 'aio write' returned OS error 122. Cannot continue operation
I hope this helps somebody
In addition to the response above by AndrewD, you should change the permission on the my.cnf so that it does not get ignored when booting up. You can run the below command,
sudo chmod 400 my.cnf
Do this right before running docker-compose up -d