Unable to start mysql docker client on Mac M1 - mysql

I have a docker-compose configuration as follows:
mysql:
image: mysql/mysql-server:8.0.23
ports:
- ${DOCKER_HOST_MYSQL_PORT:-3306}:3306
command: --default-authentication-plugin=mysql_native_password
environment:
MYSQL_ROOT_PASSWORD: secret
MYSQL_DATABASE: ${DB_DATABASE}
volumes:
- ./storage/docker/mysql:/var/lib/mysql
networks:
then I ran the following: docker-compose up mysql
I'm getting the following error:
[+] Running 5/5
⠿ mysql Pulled 14.1s
⠿ 517a40cb4717 Pull complete 5.9s
⠿ 4a26b26e2152 Pull complete 10.0s
⠿ 14794e2450ed Pull complete 10.0s
⠿ 754b01a0f0ff Pull complete 10.0s
[+] Running 1/1
⠿ Container client-app-mysql-1 Recreated 0.4s
Attaching to client-app-mysql-1
client-app-mysql-1 | [Entrypoint] MySQL Docker Image 8.0.23-1.1.19
client-app-mysql-1 | [Entrypoint] Starting MySQL 8.0.23-1.1.19
client-app-mysql-1 | 2022-03-29T08:41:12.591047Z 0 [System] [MY-010116] [Server] /usr/sbin/mysqld (mysqld 8.0.23) starting as process 26
client-app-mysql-1 | 2022-03-29T08:41:12.597713Z 0 [Warning] [MY-010159] [Server] Setting lower_case_table_names=2 because file system for /var/lib/mysql/ is case insensitive
client-app-mysql-1 | 2022-03-29T08:41:12.611927Z 1 [System] [MY-011012] [Server] Starting upgrade of data directory.
client-app-mysql-1 | 2022-03-29T08:41:12.612425Z 1 [System] [MY-013576] [InnoDB] InnoDB initialization has started.
client-app-mysql-1 | 2022-03-29T08:41:13.343343Z 1 [ERROR] [MY-012224] [InnoDB] Tablespace flags are invalid in datafile: ./ibdata1, Space ID:0, Flags: 21. Please refer to http://dev.mysql.com/doc/refman/8.0/en/innodb-troubleshooting-datadict.html for how to resolve the issue.
client-app-mysql-1 | 2022-03-29T08:41:13.344157Z 1 [ERROR] [MY-012237] [InnoDB] Corrupted page [page id: space=0, page number=0] of datafile './ibdata1' could not be found in the doublewrite buffer.
client-app-mysql-1 | 2022-03-29T08:41:13.344888Z 1 [ERROR] [MY-012930] [InnoDB] Plugin initialization aborted with error Data structure corruption.
client-app-mysql-1 | 2022-03-29T08:41:13.833188Z 1 [ERROR] [MY-011013] [Server] Failed to initialize DD Storage Engine.
client-app-mysql-1 | 2022-03-29T08:41:13.835836Z 0 [ERROR] [MY-010020] [Server] Data Dictionary initialization failed.
client-app-mysql-1 | 2022-03-29T08:41:13.837260Z 0 [ERROR] [MY-010119] [Server] Aborting
client-app-mysql-1 | 2022-03-29T08:41:13.838870Z 0 [System] [MY-010910] [Server] /usr/sbin/mysqld: Shutdown complete (mysqld 8.0.23) MySQL Community Server - GPL.
client-app-mysql-1 exited with code 1

I got into the same problem once moved to a M1 machine.
Didn't dig much into the issue, but got it working by attaching a docker volume and in this way the DB data are persistent
version: '3.7'
services:
dcm-mysql:
hostname: dcm-mysql
container_name: dcm-mysql
image: mysql:8.0.29-oracle
restart: always
environment:
MYSQL_DATABASE: dcm
MYSQL_USER: dcm
MYSQL_PASSWORD: dcm
MYSQL_ROOT_PASSWORD: root
ports:
- 3306:3306
volumes:
# - ../local_data/mdb/data:/var/lib/mysql
- dcm-mysql-data:/var/lib/mysql
networks:
- mariadb
volumes:
dcm-mysql-data:

Related

bind mount to mysql docker container not working

I have a mysql docker container with container_name=mysql1. When I run docker exec -it mysqlsh and look at the files using ls command, i see the following files.
bin boot dev docker-entrypoint-initdb.d entrypoint.sh etc healthcheck.sh home lib lib64 media mnt opt proc root run sbin srv sys tmp usr var
I want to bind mount some of these folders in my host machine, inside the directory where the docker-compose file is being run from.
version: '3.9'
services:
mysql1:
container_name: mysql1
image: mysql/mysql-server
ports:
- '3306:3306'
environment:
- MYSQL_ROOT_PASSWORD = 'root'
volumes:
- ./db:/etc
But tis is giving me the following error:
[+] Running 2/0
⠿ Network poc_1_default Created 0.0s
⠿ Container mysql1 Created 0.0s
Attaching to mysql1
mysql1 | [Entrypoint] MySQL Docker Image 8.0.28-1.2.7-server
mysql1 | [Entrypoint] Initializing database
mysql1 | [Entrypoint] No password option specified for new database.
mysql1 | [Entrypoint] A random onetime password will be generated.
mysql1 | 2022-02-05T16:36:18.356881Z 0 [System] [MY-013169] [Server] /usr/sbin/mysqld (mysqld 8.0.28) initializing of server in progress as process 18
mysql1 | 2022-02-05T16:36:18.359651Z 0 [ERROR] [MY-010124] [Server] Fatal error: Can't change to run as user 'mysql' ; Please check that the user exists!
mysql1 | 2022-02-05T16:36:18.359664Z 0 [ERROR] [MY-013236] [Server] The designated data directory /var/lib/mysql/ is unusable. You can remove all files that the server added to it.
mysql1 | 2022-02-05T16:36:18.359700Z 0 [ERROR] [MY-010119] [Server] Aborting
mysql1 | 2022-02-05T16:36:18.359754Z 0 [System] [MY-010910] [Server] /usr/sbin/mysqld: Shutdown complete (mysqld 8.0.28) MySQL Community Server - GPL.
mysql1 exited with code 1
What is happening and how do I fix this?

lower_case_table_names error for Docker + MySQL 8.0.26 based project on Ubuntu 20.04 Server(AWS EC2)

I am trying to configure my Django + MySQL(8.0.26) project on Ubuntu 20.04 based AWS EC2 server with Docker Compose. However, MySQL database container is not working because of lower_case_table_names. These are a few solutions solution1, solution2, solution3 which I've already tried but have not been able to find the solution.
I have created a sample project based on this YouTube tutorial which is successfully running on my local machine(macOS 11.4).
docker-compose.yml
version: '3.8'
services:
db:
image: mysql:8.0.26
restart: always
command: --lower_case_table_names=1
environment:
- MYSQL_DATABASE=tutorialdb
- MYSQL_USER=chitrang
- MYSQL_PASSWORD=test123
- MYSQL_ROOT_PASSWORD=test123
ports:
- "3307:3306"
volumes:
- ./dbdata:/var/lib/mysql
backend:
build: .
command: python manage.py runserver 0.0.0.0:8000
ports:
- 8000:8000
volumes:
- .:/app
depends_on:
- db
Error Log:
db_1 | 2021-08-20 01:01:48+00:00 [Note] [Entrypoint]: Entrypoint script for MySQL Server 8.0.26-1debian10 started.
db_1 | 2021-08-20 01:01:50+00:00 [Note] [Entrypoint]: Switching to dedicated user 'mysql'
db_1 | 2021-08-20 01:01:50+00:00 [Note] [Entrypoint]: Entrypoint script for MySQL Server 8.0.26-1debian10 started.
db_1 | 2021-08-20T01:01:51.247142Z 0 [System] [MY-010116] [Server] /usr/sbin/mysqld (mysqld 8.0.26) starting as process 1
db_1 | 2021-08-20T01:01:52.809303Z 1 [System] [MY-013576] [InnoDB] InnoDB initialization has started.
db_1 | 2021-08-20T01:01:56.262851Z 1 [System] [MY-013577] [InnoDB] InnoDB initialization has ended.
db_1 | 2021-08-20T01:01:56.320513Z 1 [ERROR] [MY-011087] [Server] Different lower_case_table_names settings for server ('1') and data dictionary ('2').
db_1 | 2021-08-20T01:01:56.321126Z 0 [ERROR] [MY-010020] [Server] Data Dictionary initialization failed.
db_1 | 2021-08-20T01:01:56.321630Z 0 [ERROR] [MY-010119] [Server] Aborting
db_1 | 2021-08-20T01:01:56.867684Z 0 [System] [MY-010910] [Server] /usr/sbin/mysqld: Shutdown complete (mysqld 8.0.26) MySQL Community Server - GPL.
I have tried to set all the three possible values 0|1|2 for lower_case_table_names in a docker-compose.yml file but error message is similar each time.
I guess solution could be simple but since this my first Backend + DevOps project I might not able to figure it out easily. Moreover, it seems this issue is common and has been faced by many developers so in my opinion either MySQL or Ubuntu team should have resolved it.
Try this, It worked for me:
https://gist.github.com/feltnerm/bb6e23f531803896ca1e
version: '3.8'
services:
db:
image: mysql:8.0.26
restart: always
command: --lower_case_table_names=0
environment:
- MYSQL_DATABASE=tutorialdb
- MYSQL_USER=chitrang
- MYSQL_PASSWORD=test123
- MYSQL_ROOT_PASSWORD=test123
ports:
- "3307:3306"
volumes:
- ./data/initdb.d:/docker-entrypoint-initdb.d
- ./data/mysql:/var/lib/mysql
backend:
build: .
command: python manage.py runserver 0.0.0.0:8000
ports:
- 8000:8000
volumes:
- .:/app
depends_on:
- db

Initialising a mysql database and persist data using Docker compose

I am trying to run a web application that uses two docker containers: one contains the application code and the other container is a mysql database instance.
I am trying to populate the db with an sql file and also try to persist the database data through a volume. I am encountering an issue with the sql init file being copied into the mysql container but not being executed.
This is how my docker-compose.yml looks like:
version: "3.7"
services:
db:
image: mysql:8.0.19
command: '--default-authentication-plugin=mysql_native_password'
restart: always
volumes:
- db-data:/var/lib/mysql
- ./db:/docker-entrypoint-initdb.d/:ro
environment:
- MYSQL_ROOT_PASSWORD=password
networks:
- backend
app:
build: app
restart: always
depends_on:
- db
ports:
- 5000:5000
networks:
- backend
- frontend
volumes:
- ./app/src:/code
volumes:
db-data:
networks:
backend:
frontend:
And this is the log when starting the mysql container
2021-04-17 12:34:01+00:00 [Note] [Entrypoint]: Entrypoint script for MySQL Server 8.0.19-1debian10 started.
2021-04-17 12:34:01+00:00 [Note] [Entrypoint]: Switching to dedicated user 'mysql'
2021-04-17 12:34:01+00:00 [Note] [Entrypoint]: Entrypoint script for MySQL Server 8.0.19-1debian10 started.
2021-04-17T12:34:01.466444Z 0 [Warning] [MY-011070] [Server] 'Disabling symbolic links using --skip-symbolic-links (or equivalent) is the default. Consider not using this option as it' is deprecated and will be removed in a future release.
2021-04-17T12:34:01.466615Z 0 [System] [MY-010116] [Server] /usr/sbin/mysqld (mysqld 8.0.19) starting as process 1
2021-04-17T12:34:01.891144Z 0 [Warning] [MY-010068] [Server] CA certificate ca.pem is self signed.
2021-04-17T12:34:01.895163Z 0 [Warning] [MY-011810] [Server] Insecure configuration for --pid-file: Location '/var/run/mysqld' in the path is accessible to all OS users. Consider choosing a different directory.
2021-04-17T12:34:01.916128Z 0 [System] [MY-010931] [Server] /usr/sbin/mysqld: ready for connections. Version: '8.0.19' socket: '/var/run/mysqld/mysqld.sock' port: 3306 MySQL Community Server - GPL.
2021-04-17T12:34:02.051164Z 0 [System] [MY-011323] [Server] X Plugin ready for connections. Socket: '/var/run/mysqld/mysqlx.sock' bind-address: '::' port: 33060
It does not look like the init.sql file is being executed. Also from the mysql container CLI I can see that the 'init.sql' file has been copied.
# cd docker-entrypoint-initdb.d
# ls
init.sql
# mysql -u root -p
mysql> show databases;
+--------------------+
| Database |
+--------------------+
| example |
| information_schema |
| mysql |
| performance_schema |
| sys |
+--------------------+
5 rows in set (0.01 sec)
I am quite new to Docker and Docker compose but I cannot figure out how to make this work.

How to connect to docker-compose mysql after container has initilized?

I have a project that I have been working on and now I am experimenting with docker-compose to build it.
Before running my own custom scripts to create tables and such I thought I would just give it a go and see if it builds and I can connect to it.
I found a suitable docker-compose for mysql:
version: "3.8"
services:
mysql:
image: mysql:latest
ports:
- 3307:3307
environment:
MYSQL_ROOT_PASSWORD: SomeRootPassword1!
MYSQL_USER: someuser
MYSQL_PASSWORD: Password1!
MYSQL_DATABASE: wedding
After running docker-compose-up in gitbash I see:
mysql_1 | 2020-08-26 20:00:14+00:00 [Note] [Entrypoint]: Entrypoint script for MySQL Server 8.0.21-1debian10 started.
mysql_1 | 2020-08-26 20:00:14+00:00 [Note] [Entrypoint]: Switching to dedicated user 'mysql'
mysql_1 | 2020-08-26 20:00:14+00:00 [Note] [Entrypoint]: Entrypoint script for MySQL Server 8.0.21-1debian10 started.
mysql_1 | 2020-08-26 20:00:14+00:00 [Note] [Entrypoint]: Initializing database files
mysql_1 | 2020-08-26T20:00:14.502438Z 0 [System] [MY-013169] [Server] /usr/sbin/mysqld (mysqld 8.0.21) initializing of server in progress as process 43
mysql_1 | 2020-08-26T20:00:14.507639Z 1 [System] [MY-013576] [InnoDB] InnoDB initialization has started.
mysql_1 | 2020-08-26T20:00:15.519301Z 1 [System] [MY-013577] [InnoDB] InnoDB initialization has ended.
mysql_1 | 2020-08-26T20:00:17.309017Z 6 [Warning] [MY-010453] [Server] root#localhost is created with an empty password ! Please consider switching off the --initialize-insecure option.
mysql_1 | 2020-08-26 20:00:21+00:00 [Note] [Entrypoint]: Database files initialized
mysql_1 | 2020-08-26 20:00:21+00:00 [Note] [Entrypoint]: Starting temporary server
mysql_1 | 2020-08-26T20:00:22.149193Z 0 [System] [MY-010116] [Server] /usr/sbin/mysqld (mysqld 8.0.21) starting as process 90
mysql_1 | 2020-08-26T20:00:22.170767Z 1 [System] [MY-013576] [InnoDB] InnoDB initialization has started.
mysql_1 | 2020-08-26T20:00:22.378726Z 1 [System] [MY-013577] [InnoDB] InnoDB initialization has ended.
mysql_1 | 2020-08-26T20:00:22.457272Z 0 [System] [MY-011323] [Server] X Plugin ready for connections. Socket: /var/run/mysqld/mysqlx.sock
mysql_1 | 2020-08-26T20:00:22.549270Z 0 [Warning] [MY-010068] [Server] CA certificate ca.pem is self signed.
mysql_1 | 2020-08-26T20:00:22.549394Z 0 [System] [MY-013602] [Server] Channel mysql_main configured to support TLS. Encrypted connections are now supported for this channel.
mysql_1 | 2020-08-26T20:00:22.553380Z 0 [Warning] [MY-011810] [Server] Insecure configuration for --pid-file: Location '/var/run/mysqld' in the path is accessible to all OS users. Consider choosing a different directory.
mysql_1 | 2020-08-26T20:00:22.564741Z 0 [System] [MY-010931] [Server] /usr/sbin/mysqld: ready for connections. Version: '8.0.21' socket: '/var/run/mysqld/mysqld.sock' port: 0 MySQL Community Server - GPL.
mysql_1 | 2020-08-26 20:00:22+00:00 [Note] [Entrypoint]: Temporary server started.
mysql_1 | Warning: Unable to load '/usr/share/zoneinfo/iso3166.tab' as time zone. Skipping it.
mysql_1 | Warning: Unable to load '/usr/share/zoneinfo/leap-seconds.list' as time zone. Skipping it.
mysql_1 | Warning: Unable to load '/usr/share/zoneinfo/zone.tab' as time zone. Skipping it.
mysql_1 | Warning: Unable to load '/usr/share/zoneinfo/zone1970.tab' as time zone. Skipping it.
mysql_1 | 2020-08-26 20:00:24+00:00 [Note] [Entrypoint]: Creating database wedding
mysql_1 | 2020-08-26 20:00:24+00:00 [Note] [Entrypoint]: Creating user someuser
mysql_1 | 2020-08-26 20:00:24+00:00 [Note] [Entrypoint]: Giving user someuser access to schema wedding
mysql_1 |
mysql_1 | 2020-08-26 20:00:24+00:00 [Note] [Entrypoint]: Stopping temporary server
mysql_1 | 2020-08-26T20:00:24.697408Z 14 [System] [MY-013172] [Server] Received SHUTDOWN from user root. Shutting down mysqld (Version: 8.0.21).
mysql_1 | 2020-08-26T20:00:27.792197Z 0 [System] [MY-010910] [Server] /usr/sbin/mysqld: Shutdown complete (mysqld 8.0.21) MySQL Community Server - GPL.
mysql_1 | 2020-08-26 20:00:28+00:00 [Note] [Entrypoint]: Temporary server stopped
mysql_1 |
mysql_1 | 2020-08-26 20:00:28+00:00 [Note] [Entrypoint]: MySQL init process done. Ready for start up.
mysql_1 |
mysql_1 | 2020-08-26T20:00:28.931847Z 0 [System] [MY-010116] [Server] /usr/sbin/mysqld (mysqld 8.0.21) starting as process 1
mysql_1 | 2020-08-26T20:00:28.947655Z 1 [System] [MY-013576] [InnoDB] InnoDB initialization has started.
mysql_1 | 2020-08-26T20:00:29.162873Z 1 [System] [MY-013577] [InnoDB] InnoDB initialization has ended.
mysql_1 | 2020-08-26T20:00:29.250374Z 0 [System] [MY-011323] [Server] X Plugin ready for connections. Bind-address: '::' port: 33060, socket: /var/run/mysqld/mysqlx.sock
mysql_1 | 2020-08-26T20:00:29.322041Z 0 [Warning] [MY-010068] [Server] CA certificate ca.pem is self signed.
mysql_1 | 2020-08-26T20:00:29.322181Z 0 [System] [MY-013602] [Server] Channel mysql_main configured to support TLS. Encrypted connections are now supported for this channel.
mysql_1 | 2020-08-26T20:00:29.328435Z 0 [Warning] [MY-011810] [Server] Insecure configuration for --pid-file: Location '/var/run/mysqld' in the path is accessible to all OS users. Consider choosing a different directory.
mysql_1 | 2020-08-26T20:00:29.343021Z 0 [System] [MY-010931] [Server] /usr/sbin/mysqld: ready for connections. Version: '8.0.21' socket: '/var/run/mysqld/mysqld.sock' port: 3306 MySQL Community Server - GPL.
The problem
Now that above is finished(Not sure if all is correct) I thought I would try and connect to it via mysql workbench but Im getting the following:
I also tried changing my binding address to :
What am I doing wrong?
In the compose file, under ports: you creating a forwarding rule from localhost:3307 to container:3307.
While you can choose any source port that is not in use on your host machine, at the other end you must hit the port on which the container is listening: in this case it would be 3306.
Your docker-compose.yml file should look like this:
version: "3.8"
services:
mysql:
image: mysql:latest
ports:
- 3307:3306
environment:
MYSQL_ROOT_PASSWORD: SomeRootPassword1!
MYSQL_USER: someuser
MYSQL_PASSWORD: Password1!
MYSQL_DATABASE: wedding
...then you should be able to connect at localhost:3307
You need to use docker internal network to bind address, instead of localhost (if you are not exposing the port), replacing 0.0.0.0 for host.docker.internal. This should do the trick.

mysql docker container not getting started ,mysqld: Can't create/write to file '/var/lib/mysql/is_writable' (OS errno 13 - Permission denied)

i've created a docker compose file to run the mysql docker container and using the database scripts initializing the database.
While starting the container i am getting the below error.
account-service-db_1 | Initializing database
account-service-db_1 | mysqld: Can't create/write to file '/var/lib/mysql/is_writable' (OS errno 13 - Permission denied)
account-service-db_1 | 2019-05-11T16:58:35.265463Z 0 [Warning] [MY-011070] [Server] 'Disabling symbolic links using --skip-symbolic-links (or equivalent) is the default. Consider not using this option as it' is deprecated and will be removed in a future release.
account-service-db_1 | 2019-05-11T16:58:35.265550Z 0 [System] [MY-013169] [Server] /usr/sbin/mysqld (mysqld 8.0.16) initializing of server in progress as process 15
account-service-db_1 | 2019-05-11T16:58:35.268464Z 0 [ERROR] [MY-010460] [Server] --initialize specified but the data directory exists and is not writable. Aborting.
account-service-db_1 | 2019-05-11T16:58:35.268472Z 0 [ERROR] [MY-013236] [Server] Newly created data directory /var/lib/mysql/ is unusable. You can safely remove it.
account-service-db_1 | 2019-05-11T16:58:35.269284Z 0 [ERROR] [MY-010119] [Server] Aborting
account-service-db_1 | 2019-05-11T16:58:35.270510Z 0 [System] [MY-010910] [Server] /usr/sbin/mysqld: Shutdown complete (mysqld 8.0.16) MySQL Community Server - GPL.
docker-swarm-account-service-mysql-demo_account-service-db_1 exited with code 1
After research many people suggested to add user: "1000:50" in compose file will solve the issue .but after adding also i am getting the same error.
please find the my docker-compose file below
version: '3.7'
services:
account-service-db:
user: "1000:50"
image: mysql
environment:
- MYSQL_DATABASE=account-service
- MYSQL_USER=mysqluser
- MYSQL_PASSWORD=mysqlpassword
- MYSQL_ROOT_PASSWORD=mysqlpassword
volumes:
- "/Users/admin/dockerfiles/docker-swarm-account-service-mysql-demo/scripts/:/docker-entrypoint-initdb.d/"
- "/Users/admin/dockerfiles/docker-swarm-account-service-mysql-demo/account:/var/lib/mysql"
ports:
- "3306:3306"
deploy:
replicas: 1
placement:
constraints:
- node.labels.mysql.node == true
restart_policy:
condition: on-failure
delay: 10s
#max-attempts: 3
i am executing the above scenario in mac machine. Please help me why its behaving like this after giving all necessary permissions to volume and scripts folder.
The problem seems to be with your account folder. It might already be in use or corrupted. Create a new folder and test.
Remove double quotes from environment values in compose file.
If the folder on host that is being mounted does not have the required write permissions:
ssh to your host/node
chmod -R a+rwx /path/to/account/folder