Docker-compose mysql: import .sql - mysql

My docker-compose script successfully manages to run a mysql/mariadb service, and copies my "init.sql" script containing the database schema to "/docker-entrypoint-initdb.d". However, the sql script is never executed like it should according to the docs.
There is no error, nothing in the logs. I checked that the sql file is copied at the right place in the container. I checked that the script runs with no errors on an empty database.
What am I missing? Is it even visible in the logs if the script was executed (and for some reason did nothing)?
(Edit: docker-compose version: 1.8.1, image mariadb:10.1.21)
# docker-compose.yml
version: '2'
volumes:
data-volume: {}
services:
mysql:
image: mariadb
ports:
- "3306:3306"
environment:
MYSQL_ROOT_PASSWORD: pwd
MYSQL_DATABASE: users_db
volumes:
- data-volume:/var/lib/mysql
- ./resources/docker-sql/init.sql:/docker-entrypoint-initdb.d/init.sql
backend:
image: myapp
ports:
- "8000:80"
depends_on:
- mysql
links:
- mysql

Looking at the source of the docker-entrypoint.sh file of the mariadb image, the script which is supposed to execute your init.sql, it becomes clear that
the script will only be executed when the container didn't contain a database before (line 76 of the script), and
there will a log output informing you that you script was called.
To achieve this with docker-compose you have to stop the service, remove the container (to get rid of the database) and start it again:
docker-compose stop
docker-compose rm
docker-compose start
Here is an example of the image's behavior, only with a .sh file instead of .sql:
$ cat hello.sh
echo "This is output of the hello script"
$ docker run -it -v `pwd`/hello.sh:/docker-entrypoint-initdb.d/hello.sh -v mariadb_test:/var/lib/mysql -e MYSQL_RANDOM_ROOT_PASSWORD=1 mariadb
Initializing database
2017-02-09 14:13:54 140617005938624 [Note] /usr/sbin/mysqld (mysqld 10.1.21-MariaDB-1~jessie) starting as process 63 ...
2017-02-09 14:13:54 140617005938624 [Note] InnoDB: Using mutexes to ref count buffer pool pages
2017-02-09 14:13:54 140617005938624 [Note] InnoDB: The InnoDB memory heap is disabled
2017-02-09 14:13:54 140617005938624 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
[...]
2017-02-09 14:14:03 139836212086528 [Note] InnoDB: Dumping buffer pool(s) not yet started
2017-02-09 14:14:03 139836971521984 [Note] Plugin 'FEEDBACK' is disabled.
2017-02-09 14:14:03 139836971521984 [Warning] 'user' entry 'root#830dbd0908f3' ignored in --skip-name-resolve mode.
2017-02-09 14:14:03 139836971521984 [Warning] 'proxies_priv' entry '#% root#830dbd0908f3' ignored in --skip-name-resolve mode.
2017-02-09 14:14:03 139836971521984 [Note] mysqld: ready for connections.
Version: '10.1.21-MariaDB-1~jessie' socket: '/var/run/mysqld/mysqld.sock' port: 0 mariadb.org binary distribution
Warning: Unable to load '/usr/share/zoneinfo/leap-seconds.list' as time zone. Skipping it.
GENERATED ROOT PASSWORD: dau6voh4eej2jooRohpiop4eh6ahl7Uz
2017-02-09 14:14:05 139836970654464 [Warning] 'proxies_priv' entry '#% root#830dbd0908f3' ignored in --skip-name-resolve mode.
/usr/local/bin/docker-entrypoint.sh: running /docker-entrypoint-initdb.d/hello.sh
This is output of the hello script
2017-02-09 14:14:05 139836970351360 [Note] mysqld: Normal shutdown
2017-02-09 14:14:05 139836970351360 [Note] Event Scheduler: Purging the queue. 0 events
2017-02-09 14:14:05 139836195301120 [Note] InnoDB: FTS optimize thread exiting.
2017-02-09 14:14:05 139836970351360 [Note] InnoDB: Starting shutdown...
2017-02-09 14:14:05 139836970351360 [Note] InnoDB: Waiting for page_cleaner to finish flushing of buffer pool
2017-02-09 14:14:07 139836970351360 [Note] InnoDB: Shutdown completed; log sequence number 1616829
2017-02-09 14:14:07 139836970351360 [Note] mysqld: Shutdown complete
MySQL init process done. Ready for start up.
[...]
You see somewhere in the log output the call and its stdout of the hello.sh script is buried. On subsequent starts of the container the script is not executed because the database is already created (in the local mariadb volume):
$ docker run -it -v `pwd`/hello.sh:/docker-entrypoint-initdb.d/hello.sh -v mariadb_test:/var/lib/mysql -e MYSQL_RANDOM_ROOT_PASSWORD=1 mariadb
2017-02-09 14:19:13 140155189532608 [Note] mysqld (mysqld 10.1.21-MariaDB-1~jessie) starting as process 1 ...
2017-02-09 14:19:13 140155189532608 [Note] InnoDB: Using mutexes to ref count buffer pool pages
2017-02-09 14:19:13 140155189532608 [Note] InnoDB: The InnoDB memory heap is disabled
2017-02-09 14:19:13 140155189532608 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
2017-02-09 14:19:13 140155189532608 [Note] InnoDB: GCC builtin __atomic_thread_fence() is used for memory barrier
2017-02-09 14:19:13 140155189532608 [Note] InnoDB: Compressed tables use zlib 1.2.8
2017-02-09 14:19:13 140155189532608 [Note] InnoDB: Using Linux native AIO
2017-02-09 14:19:13 140155189532608 [Note] InnoDB: Using SSE crc32 instructions
2017-02-09 14:19:13 140155189532608 [Note] InnoDB: Initializing buffer pool, size = 256.0M
2017-02-09 14:19:13 140155189532608 [Note] InnoDB: Completed initialization of buffer pool
2017-02-09 14:19:13 140155189532608 [Note] InnoDB: Highest supported file format is Barracuda.
2017-02-09 14:19:13 140155189532608 [Note] InnoDB: 128 rollback segment(s) are active.
2017-02-09 14:19:13 140155189532608 [Note] InnoDB: Waiting for purge to start
2017-02-09 14:19:13 140155189532608 [Note] InnoDB: Percona XtraDB (http://www.percona.com) 5.6.34-79.1 started; log sequence number 1616839
2017-02-09 14:19:13 140154429736704 [Note] InnoDB: Dumping buffer pool(s) not yet started
2017-02-09 14:19:13 140155189532608 [Note] Plugin 'FEEDBACK' is disabled.
2017-02-09 14:19:13 140155189532608 [Note] Server socket created on IP: '::'.
2017-02-09 14:19:13 140155189532608 [Warning] 'proxies_priv' entry '#% root#830dbd0908f3' ignored in --skip-name-resolve mode.
2017-02-09 14:19:13 140155189532608 [Note] mysqld: ready for connections.
Version: '10.1.21-MariaDB-1~jessie' socket: '/var/run/mysqld/mysqld.sock' port: 3306 mariadb.org binary distribution

Maybe a little late, but I have found another more accurate solution.
Through depends_on and healthcheck, don't run backend until mysql dump has finished. The only detail is that you must run it with version 2.1.
# docker-compose.yml
version: '2.1'
volumes:
data-volume: {}
services:
mysql:
image: mariadb
ports:
- "3306:3306"
environment:
MYSQL_ROOT_PASSWORD: pwd
MYSQL_DATABASE: users_db
depends_on:
backend:
condition: service_healthy
volumes:
- data-volume:/var/lib/mysql
- ./resources/docker-sql/init.sql:/docker-entrypoint-initdb.d/init.sql
backend:
image: myapp
ports:
- "8000:80"
depends_on:
- mysql
links:
- mysql
healthcheck:
test: "/usr/bin/mysql --user=root --password=you_pass --database=your_db --execute \"SELECT * FROM last_table_of_your_dump;\""
interval: 10s
timeout: 3s
retries: 10

Related

Access denied for user 'root' Mysql container

I m trying to dockerize a laravel project a small problem with my MySQL Container.
I cant connect to the container my DB with root or simple user.
I only used docker-compose build && docker-compose up -d
docker exec -it ica-backoffice_attendize_db_1 /bin/bash
mysql -u root -p
Enter password: ERROR 1045 (28000): Access denied for user 'root'#'localhost' (using password: YES)
This is my docker log : docker logs ica-backoffice_attendize_db_1
2021-03-30 12:20:44+00:00 [Note] [Entrypoint]: Entrypoint script for MySQL Server 5.7.33-1debian10 started.
2021-03-30 12:20:44+00:00 [Note] [Entrypoint]: Switching to dedicated user 'mysql'
2021-03-30 12:20:44+00:00 [Note] [Entrypoint]: Entrypoint script for MySQL Server 5.7.33-1debian10 started.
2021-03-30T12:20:44.525563Z 0 [Warning] TIMESTAMP with implicit DEFAULT value is deprecated. Please use --explicit_defaults_for_timestamp server option (see documentation for more details).
2021-03-30T12:20:44.527413Z 0 [Note] mysqld (mysqld 5.7.33) starting as process 1 ...
2021-03-30T12:20:44.532377Z 0 [Note] InnoDB: PUNCH HOLE support available
2021-03-30T12:20:44.532396Z 0 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
2021-03-30T12:20:44.532402Z 0 [Note] InnoDB: Uses event mutexes
2021-03-30T12:20:44.532407Z 0 [Note] InnoDB: GCC builtin __atomic_thread_fence() is used for memory barrier
2021-03-30T12:20:44.532425Z 0 [Note] InnoDB: Compressed tables use zlib 1.2.11
2021-03-30T12:20:44.532430Z 0 [Note] InnoDB: Using Linux native AIO
2021-03-30T12:20:44.532739Z 0 [Note] InnoDB: Number of pools: 1
2021-03-30T12:20:44.532879Z 0 [Note] InnoDB: Using CPU crc32 instructions
2021-03-30T12:20:44.535236Z 0 [Note] InnoDB: Initializing buffer pool, total size = 128M, instances = 1, chunk size = 128M
2021-03-30T12:20:44.546880Z 0 [Note] InnoDB: Completed initialization of buffer pool
2021-03-30T12:20:44.549449Z 0 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().
2021-03-30T12:20:44.561014Z 0 [Note] InnoDB: Highest supported file format is Barracuda.
2021-03-30T12:20:44.562673Z 0 [Note] InnoDB: Log scan progressed past the checkpoint lsn 12665801
2021-03-30T12:20:44.562685Z 0 [Note] InnoDB: Doing recovery: scanned up to log sequence number 12665810
2021-03-30T12:20:44.562690Z 0 [Note] InnoDB: Database was not shutdown normally!
2021-03-30T12:20:44.562695Z 0 [Note] InnoDB: Starting crash recovery.
2021-03-30T12:20:44.713371Z 0 [Note] InnoDB: Removed temporary tablespace data file: "ibtmp1"
2021-03-30T12:20:44.713417Z 0 [Note] InnoDB: Creating shared tablespace for temporary tables
2021-03-30T12:20:44.713562Z 0 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...
2021-03-30T12:20:45.454122Z 0 [Note] InnoDB: File './ibtmp1' size is now 12 MB.
2021-03-30T12:20:45.456067Z 0 [Note] InnoDB: 96 redo rollback segment(s) found. 96 redo rollback segment(s) are active.
2021-03-30T12:20:45.456099Z 0 [Note] InnoDB: 32 non-redo rollback segment(s) are active.
2021-03-30T12:20:45.456761Z 0 [Note] InnoDB: Waiting for purge to start
2021-03-30T12:20:45.507075Z 0 [Note] InnoDB: 5.7.33 started; log sequence number 12665810
2021-03-30T12:20:45.507543Z 0 [Note] InnoDB: Loading buffer pool(s) from /var/lib/mysql/ib_buffer_pool
2021-03-30T12:20:45.507941Z 0 [Note] Plugin 'FEDERATED' is disabled.
2021-03-30T12:20:45.511918Z 0 [Note] InnoDB: Buffer pool(s) load completed at 210330 12:20:45
2021-03-30T12:20:45.524384Z 0 [Note] Found ca.pem, server-cert.pem and server-key.pem in data directory. Trying to enable SSL support using them.
2021-03-30T12:20:45.524418Z 0 [Note] Skipping generation of SSL certificates as certificate files are present in data directory.
2021-03-30T12:20:45.526363Z 0 [Warning] CA certificate ca.pem is self signed.
2021-03-30T12:20:45.526435Z 0 [Note] Skipping generation of RSA key pair as key files are present in data directory.
2021-03-30T12:20:45.527538Z 0 [Note] Server hostname (bind-address): '*'; port: 3306
2021-03-30T12:20:45.527717Z 0 [Note] IPv6 is available.
2021-03-30T12:20:45.527762Z 0 [Note] - '::' resolves to '::';
2021-03-30T12:20:45.527810Z 0 [Note] Server socket created on IP: '::'.
2021-03-30T12:20:45.578895Z 0 [Warning] Insecure configuration for --pid-file: Location '/var/run/mysqld' in the path is accessible to all OS users. Consider choosing a different directory.
2021-03-30T12:20:45.602259Z 0 [Note] Event Scheduler: Loaded 0 events
2021-03-30T12:20:45.602848Z 0 [Note] mysqld: ready for connections.
Version: '5.7.33' socket: '/var/run/mysqld/mysqld.sock' port: 3306 MySQL Community Server (GPL)
2021-03-30T12:20:56.582691Z 2 [Note] Access denied for user 'root'#'localhost' (using password: NO)
2021-03-30T12:21:22.098439Z 3 [Note] Access denied for user 'dhia'#'localhost' (using password: YES)
2021-03-30T12:21:26.702866Z 4 [Note] Access denied for user 'root'#'localhost' (using password: NO)
2021-03-30T12:21:56.174284Z 5 [Note] Access denied for user 'root'#'localhost' (using password: YES)
.env file :
DB_CONNECTION=mysql
DB_HOST=127.0.0.1
DB_PORT=3306
DB_DATABASE=preprodica
DB_USERNAME=root
DB_PASSWORD=Str0ngDBP#ssw0rd
Docker compose :
services:
attendize_app:
build:
context: . # The path of the Dockerfile
volumes:
- .:/usr/src/app
depends_on:
- attendize_db
working_dir: /usr/src/app
environment:
SERVICE_NAME: attendize-app
DB_CONNECTION: mysql
DB_DRIVER: mysql
DB_HOST: attendize_db
DB_PORT: 3306
networks:
- app_network
attendize_nginx:
build:
context: ./docker-nginx/. # The path of the Dockerfile
volumes:
- "./public:/usr/src/app/public"
environment:
NGINX_FPM_HOST: attendize_app
NGINX_ROOT: /usr/src/app/public
ports:
- "8000:80"
depends_on:
- attendize_app
networks:
- app_network
attendize_db:
image: mysql:5
restart: always
environment:
MYSQL_DATABASE: "preprodica"
MYSQL_ROOT_PASSWORD: "Str0ngDBP#ssw0rd"
MYSQL_USER: "dhia"
MYSQL_PASSWORD: "dhia"
MYSQL_ALLOW_EMPTY_PASSWORD: "yes"
ports:
- "3306:3306"
volumes:
- "./storage/db-data:/var/lib/mysql"
networks:
- app_network
networks:
app_network:
driver: bridge
Docker-compose does extra work to preserve volumes between runs (thus preserving the database); you may want to try docker-compose rm -v to delete everything and try starting it up again. The command docker-compose rm -v reflects in the containers, but NOT in the volume mapped. So, the following command will remove the containers and since it doesn't delete the volumes, the command files to create a new user.
$ docker-compose rm -v
Going to remove backendphp_qgPhpMyAdmin_1, backendphp_qgMySQLServer_1, backendphp_qgMysqlData_1
Are you sure? [yN] y
Removing backendphp_qgMysqlData_1...
Removing backendphp_qgMySQLServer_1...
Removing backendphp_qgPhpMyAdmin_1...
At this point, the only way to make it right is to delete the directory that you mapped your mysql data.
$ sudo rm -rf mysql-data
Only after that, the command created a new database with the users correctly set!
You can check this link too
https://www.programmersought.com/article/16512825562/
I Fixed using :
sudo rm -r db-data/
docker-compose up -d
docker-compose down -v
Good Luck!

Docker mysql container does not allow remote connection

I have an error Access denied for user 'bonus_user'#'my-ip' (using password: YES) when trying to connect from my mysql client to dockerized mysql server on remote machine
My docker-compose file:
bonus_db:
container_name: bonus_db
image: mysql:5.7
command: --default-authentication-plugin=mysql_native_password
restart: unless-stopped
ports:
- "3306:3306"
environment:
MYSQL_USER: bonus_user
MYSQL_PASSWORD: bonus_pass
MYSQL_DATABASE: bonus_db
MYSQL_ROOT_PASSWORD: root_pass
volumes:
- /var/lib/volumes/bonusdata:/var/lib/mysql
Here's also log of container starting:
bonus_db | 2020-11-14 15:06:05+00:00 [Note] [Entrypoint]: Entrypoint script for MySQL Server 5.7.31-1debian10 started.
bonus_db | 2020-11-14 15:06:05+00:00 [Note] [Entrypoint]: Switching to dedicated user 'mysql'
bonus_db | 2020-11-14 15:06:05+00:00 [Note] [Entrypoint]: Entrypoint script for MySQL Server 5.7.31-1debian10 started.
bonus_db | 2020-11-14T15:06:06.470897Z 0 [Warning] TIMESTAMP with implicit DEFAULT value is deprecated. Please use --explicit_defaults_for_timestamp server option (see documentation for more details).
bonus_db | 2020-11-14T15:06:06.480712Z 0 [Note] mysqld (mysqld 5.7.31) starting as process 1 ...
bonus_db | 2020-11-14T15:06:06.502124Z 0 [Note] InnoDB: PUNCH HOLE support available
bonus_db | 2020-11-14T15:06:06.504807Z 0 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
bonus_db | 2020-11-14T15:06:06.504851Z 0 [Note] InnoDB: Uses event mutexes
bonus_db | 2020-11-14T15:06:06.504878Z 0 [Note] InnoDB: GCC builtin __atomic_thread_fence() is used for memory barrier
bonus_db | 2020-11-14T15:06:06.504894Z 0 [Note] InnoDB: Compressed tables use zlib 1.2.11
bonus_db | 2020-11-14T15:06:06.504906Z 0 [Note] InnoDB: Using Linux native AIO
bonus_db | 2020-11-14T15:06:06.505684Z 0 [Note] InnoDB: Number of pools: 1
bonus_db | 2020-11-14T15:06:06.506021Z 0 [Note] InnoDB: Using CPU crc32 instructions
bonus_db | 2020-11-14T15:06:06.522229Z 0 [Note] InnoDB: Initializing buffer pool, total size = 128M, instances = 1, chunk size = 128M
bonus_db | 2020-11-14T15:06:06.548073Z 0 [Note] InnoDB: Completed initialization of buffer pool
bonus_db | 2020-11-14T15:06:06.554315Z 0 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().
bonus_db | 2020-11-14T15:06:06.568903Z 0 [Note] InnoDB: Highest supported file format is Barracuda.
bonus_db | 2020-11-14T15:06:06.603166Z 0 [Note] InnoDB: Creating shared tablespace for temporary tables
bonus_db | 2020-11-14T15:06:06.605892Z 0 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...
bonus_db | 2020-11-14T15:06:06.654737Z 0 [Note] InnoDB: File './ibtmp1' size is now 12 MB.
bonus_db | 2020-11-14T15:06:06.656736Z 0 [Note] InnoDB: 96 redo rollback segment(s) found. 96 redo rollback segment(s) are active.
bonus_db | 2020-11-14T15:06:06.656771Z 0 [Note] InnoDB: 32 non-redo rollback segment(s) are active.
bonus_db | 2020-11-14T15:06:06.657961Z 0 [Note] InnoDB: Waiting for purge to start
bonus_db | 2020-11-14T15:06:06.708340Z 0 [Note] InnoDB: 5.7.31 started; log sequence number 13750567
bonus_db | 2020-11-14T15:06:06.709023Z 0 [Note] Plugin 'FEDERATED' is disabled.
bonus_db | 2020-11-14T15:06:06.711205Z 0 [Note] InnoDB: Loading buffer pool(s) from /var/lib/mysql/ib_buffer_pool
bonus_db | 2020-11-14T15:06:06.715836Z 0 [Note] InnoDB: Buffer pool(s) load completed at 201114 15:06:06
bonus_db | 2020-11-14T15:06:06.721862Z 0 [Note] Found ca.pem, server-cert.pem and server-key.pem in data directory. Trying to enable SSL support using them.
bonus_db | 2020-11-14T15:06:06.721918Z 0 [Note] Skipping generation of SSL certificates as certificate files are present in data directory.
bonus_db | 2020-11-14T15:06:06.723943Z 0 [Warning] CA certificate ca.pem is self signed.
bonus_db | 2020-11-14T15:06:06.724051Z 0 [Note] Skipping generation of RSA key pair as key files are present in data directory.
bonus_db | 2020-11-14T15:06:06.725417Z 0 [Note] Server hostname (bind-address): '*'; port: 3306
bonus_db | 2020-11-14T15:06:06.725502Z 0 [Note] IPv6 is available.
bonus_db | 2020-11-14T15:06:06.725525Z 0 [Note] - '::' resolves to '::';
bonus_db | 2020-11-14T15:06:06.725562Z 0 [Note] Server socket created on IP: '::'.
bonus_db | 2020-11-14T15:06:06.740164Z 0 [Warning] Insecure configuration for --pid-file: Location '/var/run/mysqld' in the path is accessible to all OS users. Consider choosing a different directory.
bonus_db | 2020-11-14T15:06:06.765251Z 0 [Note] Event Scheduler: Loaded 0 events
bonus_db | 2020-11-14T15:06:06.765978Z 0 [Note] mysqld: ready for connections.
bonus_db | Version: '5.7.31' socket: '/var/run/mysqld/mysqld.sock' port: 3306 MySQL Community Server (GPL)
I tried to log in into container on remote machine like this:
docker exec -i bonus_db mysql -ubonus_user -pbonus_pass
And it seems working, because if I pass wrong password here, it will give the same error Access denied. I've also tried to import data from sql file
docker exec -i bonus_db mysql -ubonus_user -pbonus_pass bonus_db < dump.sql
and it also seems working, it's processing a couple of seconds and then exits to the terminal. But I cant access database from my local machine and from other containers too.

Access denied for user 'root'#'localhost' with mariadb 10.4.8 docker container using docker compose and issue while attaching external volume

I am new to Docker, I was trying to crate docker container of mariadb for my application but when I start running mariadb container it shows Access denied for user 'root'#'localhost' (using password: YES) dockerfile
Following is the docker compose I am using.
version: '3'
services:
mysql:
image: mariadb
container_name: mariadb
volumes:
- dbvolume:/var/lib/mysql
- ./AppDatabase.sql:/docker-entrypoint-initdb.d/AppDatabase.sql
environment:
MYSQL_ROOT_PASSWORD: root123
MYSQL_ROOT_USER: root
MYSQL_USER: root
MYSQL_PASSWORD: root123
MYSQL_DATABASE: appdata
ports:
- "3333:3306"
volumes:
dbvolume:
After trying for multiple times by referring few links I was able to connect my application to docker container but it failed to import AppDatabase.sql script at the time of creating docker container.
But now by using same docker compose file I am not able to connect mariadb to my application and I think even it's not importing SQL script to the database (based on logs I have observed).
Following is the docker log generated while running docker compose:
$ docker logs 3fde358ff015
2019-09-24 17:40:37 0 [Note] mysqld (mysqld 10.4.8-MariaDB-1:10.4.8+maria~bionic) starting as process 1 ...
2019-09-24 17:40:37 0 [Note] InnoDB: Using Linux native AIO
2019-09-24 17:40:37 0 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
2019-09-24 17:40:37 0 [Note] InnoDB: Uses event mutexes
2019-09-24 17:40:37 0 [Note] InnoDB: Compressed tables use zlib 1.2.11
2019-09-24 17:40:37 0 [Note] InnoDB: Number of pools: 1
2019-09-24 17:40:37 0 [Note] InnoDB: Using SSE2 crc32 instructions
2019-09-24 17:40:37 0 [Note] mysqld: O_TMPFILE is not supported on /tmp (disabling future attempts)
2019-09-24 17:40:37 0 [Note] InnoDB: Initializing buffer pool, total size = 256M, instances = 1, chunk size = 128M
2019-09-24 17:40:37 0 [Note] InnoDB: Completed initialization of buffer pool
2019-09-24 17:40:37 0 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().
2019-09-24 17:40:37 0 [Note] InnoDB: Upgrading redo log: 2*50331648 bytes; LSN=21810033
2019-09-24 17:40:38 0 [Note] InnoDB: Starting to delete and rewrite log files.
2019-09-24 17:40:38 0 [Note] InnoDB: Setting log file ./ib_logfile101 size to 50331648 bytes
2019-09-24 17:40:38 0 [Note] InnoDB: Setting log file ./ib_logfile1 size to 50331648 bytes
2019-09-24 17:40:38 0 [Note] InnoDB: Renaming log file ./ib_logfile101 to ./ib_logfile0
2019-09-24 17:40:38 0 [Note] InnoDB: New log files created, LSN=21810033
2019-09-24 17:40:38 0 [Note] InnoDB: 128 out of 128 rollback segments are active.
2019-09-24 17:40:38 0 [Note] InnoDB: Creating shared tablespace for temporary tables
2019-09-24 17:40:38 0 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...
2019-09-24 17:40:38 0 [Note] InnoDB: File './ibtmp1' size is now 12 MB.
2019-09-24 17:40:38 0 [Note] InnoDB: Waiting for purge to start
2019-09-24 17:40:38 0 [Note] InnoDB: 10.4.8 started; log sequence number 21810033; transaction id 14620
2019-09-24 17:40:38 0 [Note] InnoDB: Loading buffer pool(s) from /var/lib/mysql/ib_buffer_pool
2019-09-24 17:40:38 0 [Note] Plugin 'FEEDBACK' is disabled.
2019-09-24 17:40:38 0 [Note] Server socket created on IP: '::'.
2019-09-24 17:40:38 0 [Warning] 'proxies_priv' entry '#% root#c980daa43351' ignored in --skip-name-resolve mode.
2019-09-24 17:40:38 0 [Note] InnoDB: Buffer pool(s) load completed at 190924 17:40:38
2019-09-24 17:40:38 0 [Note] Reading of all Master_info entries succeeded
2019-09-24 17:40:38 0 [Note] Added new Master_info '' to hash table
2019-09-24 17:40:38 0 [Note] mysqld: ready for connections.
Version: '10.4.8-MariaDB-1:10.4.8+maria~bionic' socket: '/var/run/mysqld/mysqld.sock' port: 3306 mariadb.org binary distribution
SQL Script I am trying to import:
create database appdata;
use appdata;
CREATE TABLE `appdatadetails` (
`Name` varchar(8) NOT NULL,
`appIndex` int(11) NOT NULL,
`connector` varchar(16) DEFAULT NULL,
`intName` varchar(12) DEFAULT NULL,
`intIndex` int(11) DEFAULT NULL,
PRIMARY KEY (`Name`,`appIndex`)
)
Please help me to understand what I am doing wrong, I have tried all possible solution posted on different blogs.
Update:
Latest Update:
I was able to up and running mariadb docker image with 10.1. But if I attach volume then still I am facing issue.
Docker Compose:
version: '3'
services:
mysql:
image: mariadb:10.1
container_name: mariadb
volumes:
- container-volume:/var/lib/mysql
- ./AppDatabase.sql:/docker-entrypoint-initdb.d/AppDatabase.sql
environment:
MYSQL_ROOT_PASSWORD: root123
MYSQL_DATABASE: appdata
ports:
- "3333:3306"
volumes:
container-volume:
And the log error message, If I attach container-volume volume.
Creating mariadb ... done
Attaching to mariadb
mariadb | 2019-09-25 6:56:26 140542855440384 [Note] mysqld (mysqld 10.1.41-MariaDB-1~bionic) starting as process 1 ...
mariadb | 2019-09-25 6:56:26 140542855440384 [Note] InnoDB: Using mutexes to ref count buffer pool pages
mariadb | 2019-09-25 6:56:26 140542855440384 [Note] InnoDB: The InnoDB memory heap is disabled
mariadb | 2019-09-25 6:56:26 140542855440384 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
mariadb | 2019-09-25 6:56:26 140542855440384 [Note] InnoDB: GCC builtin __atomic_thread_fence() is used for memory barrier
mariadb | 2019-09-25 6:56:26 140542855440384 [Note] InnoDB: Compressed tables use zlib 1.2.11
mariadb | 2019-09-25 6:56:26 140542855440384 [Note] InnoDB: Using Linux native AIO
mariadb | 2019-09-25 6:56:26 140542855440384 [Note] InnoDB: Using SSE crc32 instructions
mariadb | 2019-09-25 6:56:26 140542855440384 [Note] InnoDB: Initializing buffer pool, size = 256.0M
mariadb | 2019-09-25 6:56:26 140542855440384 [Note] InnoDB: Completed initialization of buffer pool
mariadb | 2019-09-25 6:56:26 140542855440384 [Note] InnoDB: Highest supported file format is Barracuda.
mariadb | InnoDB: No valid checkpoint found.
mariadb | InnoDB: A downgrade from MariaDB 10.2.2 or later is not supported.
mariadb | InnoDB: If this error appears when you are creating an InnoDB database,
mariadb | InnoDB: the problem may be that during an earlier attempt you managed
mariadb | InnoDB: to create the InnoDB data files, but log file creation failed.
mariadb | InnoDB: If that is the case, please refer to
mariadb | InnoDB: http://dev.mysql.com/doc/refman/5.6/en/error-creating-innodb.html
mariadb | 2019-09-25 6:56:26 140542855440384 [ERROR] Plugin 'InnoDB' init function returned error.
mariadb | 2019-09-25 6:56:26 140542855440384 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed.
mariadb | 2019-09-25 6:56:26 140542855440384 [Note] Plugin 'FEEDBACK' is disabled.
mariadb | 2019-09-25 6:56:26 140542855440384 [ERROR] Unknown/unsupported storage engine: InnoDB
mariadb | 2019-09-25 6:56:26 140542855440384 [ERROR] Aborting
mariadb |
mariadb exited with code 1
If I remove container-volume then It is importing .sql script and running well and good.
Updated with working script: Before I was using mariadb 10.4.8 or latest and facing issue(s) to access DB and attaching external volume.
Now I have downgraded (As suggested by #Adiii) and tried. It runs perfectlly and we no need to specify external: true in volumes service
version: '3'
services:
mysql:
image: mariadb:10.1
container_name: mariadb
volumes:
- ./dbvolume:/var/lib/mysql
- ./AppDatabase.sql:/docker-entrypoint-initdb.d/AppDatabase.sql
environment:
MYSQL_ROOT_PASSWORD: root123
MYSQL_DATABASE: appdata
ports:
- "3333:3306"
It will affect the new user and keys even if you create fresh container.
Remove mount location, as it will pick a user name and password from this location, also will not run your DB init script too.
volumes:
- dbvolume:/var/lib/mysql
Also you do not need
create database appdata;
As database already created at
MYSQL_ROOT_PASSWORD: root123
MYSQL_ROOT_USER: root
MYSQL_DATABASE: appdata
this step.
update:
remove the user root, as root already defined. you can try with
version: '3.7'
services:
mysql:
environment:
MYSQL_ROOT_PASSWORD: root123
MYSQL_DATABASE: appdata
image: mariadb
Or the second user should be different
version: '3.7'
services:
mysql:
environment:
MYSQL_ROOT_PASSWORD: root123
MYSQL_DATABASE: appdata
MYSQL_USER: test
MYSQL_PASSWORD: root123
image: mariadb
you can try with
docker exec -it container_name bash -c "mysql -u test -proot123"
or
docker exec -it mysql bash -c "mysql -u root -proot123"
If still issue, remove the DB image, pull a new one.
Or try image tag 10.1
If I remove container-volume then It is importing .sql script and
running well and good.
if mount the location then the init script will not run as the container is expecting that there is already DB or try to remove named volume and create new one.
So mount the location, import the DB using MySQL command and use the mount location for next time.

Docker - MySQL mount/bind with local machine mysql data /var/lib/mysql

i have installed mysql on my local machine and i have docker installed in my machine i am converting my projects to Docker successfully i have deployed flask on container but i want to install mysql on container but mysql container should use my local machine data because data almost 100+Gb in size.
I have to do Same for mongodb.
Am i missing something for dockerize the existing mysql and mongodb.
OS: Ubuntu and default mysql path: /var/lib/mysql
I dont want to do import/export.
Below is my docker-compose.yml
version: "3"
services:
app:
build:
context: ./app
dockerfile: ./dockerfile
container_name: flaskPython
links:
- mysqldb
- mongodb
ports:
- "5000:5000"
myadmin:
image: phpmyadmin/phpmyadmin
container_name: phpmyadmin
ports:
- "8282:80"
environment:
- PMA_ARBITRARY=1
- PMA_HOST=${MYSQL_HOST}
restart: always
depends_on:
- mysqldb
- mongodb
mysqldb:
image: mysql:${MYSQL_VERSION}
container_name: ${MYSQL_HOST}
restart: 'always'
expose:
- '3306'
env_file:
- ".env"
environment:
- MYSQL_DATABASE=${MYSQL_DATABASE}
- MYSQL_ROOT_PASSWORD=${MYSQL_ROOT_PASSWORD}
- MYSQL_USER=${MYSQL_USER}
- MYSQL_PASSWORD=${MYSQL_PASSWORD}
ports:
- "8989:3306"
volumes:
#- ./data/db/mysql:/var/lib/mysql
- /var/lib/mysql:/var/lib/mysql
mongodb:
image: mongo
container_name: mongodb
restart: always
environment:
- MONGO_INITDB_ROOT_USERNAME=root
- MONGO_INITDB_ROOT_PASSWORD=root
# if you wish to setup additional user accounts specific per DB or with different roles you can use following entry point
#no --auth is needed here as presence of username and password add this option automatically
command: mongod
ports:
- "27017:27017"
volumes:
#- ./data/db/mongo/001_users.js:/docker-entrypoint-initdb.d/001_users.js:ro
- ./mongo-entrypoint/:/docker-entrypoint-initdb.d/
- ./data/db/mongo:/usr/data/db
Logs:
2019-02-01T16:08:20.100825Z 0 [Warning] TIMESTAMP with implicit DEFAULT value is deprecated. Please use --explicit_defaults_for_timestamp server option (see documentation for more details).
2019-02-01T16:08:20.101919Z 0 [Note] mysqld (mysqld 5.7.22) starting as process 1 ...
2019-02-01T16:08:20.104031Z 0 [Note] InnoDB: PUNCH HOLE support available
2019-02-01T16:08:20.104043Z 0 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
2019-02-01T16:08:20.104046Z 0 [Note] InnoDB: Uses event mutexes
2019-02-01T16:08:20.104048Z 0 [Note] InnoDB: GCC builtin __atomic_thread_fence() is used for memory barrier
2019-02-01T16:08:20.104050Z 0 [Note] InnoDB: Compressed tables use zlib 1.2.3
2019-02-01T16:08:20.104052Z 0 [Note] InnoDB: Using Linux native AIO
2019-02-01T16:08:20.104214Z 0 [Note] InnoDB: Number of pools: 1
2019-02-01T16:08:20.104285Z 0 [Note] InnoDB: Using CPU crc32 instructions
2019-02-01T16:08:20.105476Z 0 [Note] InnoDB: Initializing buffer pool, total size = 128M, instances = 1, chunk size = 128M
2019-02-01T16:08:20.110790Z 0 [Note] InnoDB: Completed initialization of buffer pool
2019-02-01T16:08:20.112288Z 0 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().
2019-02-01T16:08:20.123892Z 0 [Note] InnoDB: Highest supported file format is Barracuda.
2019-02-01T16:08:20.139921Z 0 [Note] InnoDB: Creating shared tablespace for temporary tables
2019-02-01T16:08:20.139996Z 0 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...
2019-02-01T16:08:20.170786Z 0 [Note] InnoDB: File './ibtmp1' size is now 12 MB.
2019-02-01T16:08:20.173104Z 0 [Note] InnoDB: 96 redo rollback segment(s) found. 96 redo rollback segment(s) are active.
2019-02-01T16:08:20.173143Z 0 [Note] InnoDB: 32 non-redo rollback segment(s) are active.
2019-02-01T16:08:20.174483Z 0 [Note] InnoDB: Waiting for purge to start
2019-02-01T16:08:20.224878Z 0 [Note] InnoDB: 5.7.22 started; log sequence number 1529088986
2019-02-01T16:08:20.225403Z 0 [Note] InnoDB: Loading buffer pool(s) from /var/lib/mysql/ib_buffer_pool
2019-02-01T16:08:20.225887Z 0 [Note] Plugin 'FEDERATED' is disabled.
2019-02-01T16:08:20.231185Z 0 [Note] InnoDB: Buffer pool(s) load completed at 190201 16:08:20
2019-02-01T16:08:20.233231Z 0 [Warning] Failed to set up SSL because of the following SSL library error: SSL context is not usable without certificate and private key
2019-02-01T16:08:20.233502Z 0 [Note] Server hostname (bind-address): '*'; port: 3306
2019-02-01T16:08:20.233535Z 0 [Note] IPv6 is available.
2019-02-01T16:08:20.233544Z 0 [Note] - '::' resolves to '::';
2019-02-01T16:08:20.233556Z 0 [Note] Server socket created on IP: '::'.
2019-02-01T16:08:20.236451Z 0 [Warning] Insecure configuration for --pid-file: Location '/var/run/mysqld' in the path is accessible to all OS users. Consider choosing a different directory.
2019-02-01T16:08:20.248740Z 0 [Warning] 'user' entry 'root#localhost' ignored in --skip-name-resolve mode.
2019-02-01T16:08:20.248832Z 0 [Warning] 'user' entry 'mysql.session#localhost' ignored in --skip-name-resolve mode.
2019-02-01T16:08:20.248857Z 0 [Warning] 'user' entry 'mysql.sys#localhost' ignored in --skip-name-resolve mode.
2019-02-01T16:08:20.248875Z 0 [Warning] 'user' entry 'debian-sys-maint#localhost' ignored in --skip-name-resolve mode.
2019-02-01T16:08:20.248921Z 0 [Warning] 'db' entry 'performance_schema mysql.session#localhost' ignored in --skip-name-resolve mode.
2019-02-01T16:08:20.248935Z 0 [Warning] 'db' entry 'sys mysql.sys#localhost' ignored in --skip-name-resolve mode.
2019-02-01T16:08:20.249216Z 0 [Warning] 'proxies_priv' entry '# root#localhost' ignored in --skip-name-resolve mode.
2019-02-01T16:08:20.256300Z 0 [Warning] 'tables_priv' entry 'user mysql.session#localhost' ignored in --skip-name-resolve mode.
2019-02-01T16:08:20.256349Z 0 [Warning] 'tables_priv' entry 'sys_config mysql.sys#localhost' ignored in --skip-name-resolve mode.
2019-02-01T16:08:20.301149Z 0 [Note] Event Scheduler: Loaded 0 events
2019-02-01T16:08:20.301811Z 0 [Note] mysqld: ready for connections.
Version: '5.7.22' socket: '/var/run/mysqld/mysqld.sock' port: 3306 MySQL Community Server (GPL)
I want to use machine mysql data in mysql container so that i dont have to migrate 100+ gb data to mysql container, so that my mysql container will use my local machine mysql data for read and write purpose.
PS: i just want i dont migrate machine mysql data to mysql container so that my mysql container somehow link to my machine and use my local machine installed mysql data folder.

Custom docker official mariadb image

I intended to create a database based on mariadb within a Dockerfile but failed.
Dockerfile content as following:
FROM mariadb
RUN mysql -uroot --password="test" -e "CREATE DATABASE IF NOT EXISTS mymariadb;"
then, when I build customed image with command docker build -t testdb .
it fails with following error:
docker#docker-virtual-machine:~$ docker build -t tttt .
Sending build context to Docker daemon 32.26kB
Step 1/2 : FROM mariadb
---> 2bdd97ca79d9
Step 2/2 : RUN mysql -uroot --password="test" -e "CREATE DATABASE IF NOT EXISTS mymariadb;"
---> Running in 80339dc97c2e
ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2)
The command '/bin/sh -c mysql -uroot --password="test" -e "CREATE DATABASE IF NOT EXISTS mymariadb;"' returned a non-zero code: 1
docker#docker-virtual-machine:~$
And then I follow #Sebastian advices but find the database is not created, details as following:
docker#docker-virtual-machine:~/test$ ls
Dockerfile init.sql
docker#docker-virtual-machine:~/test$ cat init.sql
CREATE DATABASE IF NOT EXISTS mydb;
docker#docker-virtual-machine:~/test$
docker#docker-virtual-machine:~/test$ cat Dockerfile
FROM mariadb
COPY init.sql /docker-entrypoint-initdb.d/
CMD ["mysqld"]
docker#docker-virtual-machine:~/test$ docker build -t mydb .
Sending build context to Docker daemon 3.072kB
Step 1/3 : FROM mariadb
---> 2bdd97ca79d9
Step 2/3 : COPY init.sql /docker-entrypoint-initdb.d/
---> 59fa0f173d93
Step 3/3 : CMD ["mysqld"]
---> Running in e605ae1fda5b
Removing intermediate container e605ae1fda5b
---> 94b98e45dc05
Successfully built 94b98e45dc05
Successfully tagged mydb:latest
docker#docker-virtual-machine:~/test$ docker run -p 3306:3306/tcp -v /my/own/dat adir1:/var/lib/mysql -e MYSQL_ROOT_PASSWORD=test123 mydb
2019-01-26 17:30:21 0 [Note] mysqld (mysqld 10.3.12-MariaDB-1:10.3.12+maria~bion ic) starting as process 1 ...
2019-01-26 17:30:21 0 [Note] InnoDB: Using Linux native AIO
2019-01-26 17:30:21 0 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtin s
2019-01-26 17:30:21 0 [Note] InnoDB: Uses event mutexes
2019-01-26 17:30:21 0 [Note] InnoDB: Compressed tables use zlib 1.2.11
2019-01-26 17:30:21 0 [Note] InnoDB: Number of pools: 1
2019-01-26 17:30:21 0 [Note] InnoDB: Using SSE2 crc32 instructions
2019-01-26 17:30:21 0 [Note] InnoDB: Initializing buffer pool, total size = 256M , instances = 1, chunk size = 128M
2019-01-26 17:30:21 0 [Note] InnoDB: Completed initialization of buffer pool
2019-01-26 17:30:21 0 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().
2019-01-26 17:30:21 0 [Note] InnoDB: 128 out of 128 rollback segments are active .
2019-01-26 17:30:21 0 [Note] InnoDB: Creating shared tablespace for temporary ta bles
2019-01-26 17:30:21 0 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Phys ically writing the file full; Please wait ...
2019-01-26 17:30:21 0 [Note] InnoDB: File './ibtmp1' size is now 12 MB.
2019-01-26 17:30:21 0 [Note] InnoDB: 10.3.12 started; log sequence number 163095 0; transaction id 21
2019-01-26 17:30:21 0 [Note] InnoDB: Loading buffer pool(s) from /var/lib/mysql/ ib_buffer_pool
2019-01-26 17:30:21 0 [Note] Plugin 'FEEDBACK' is disabled.
2019-01-26 17:30:21 0 [Note] InnoDB: Buffer pool(s) load completed at 190126 17: 30:21
2019-01-26 17:30:21 0 [Note] Server socket created on IP: '::'.
2019-01-26 17:30:21 0 [Warning] 'proxies_priv' entry '#% root#566af8cb98ef' igno red in --skip-name-resolve mode.
2019-01-26 17:30:21 0 [Note] Reading of all Master_info entries succeded
2019-01-26 17:30:21 0 [Note] Added new Master_info '' to hash table
2019-01-26 17:30:21 0 [Note] mysqld: ready for connections.
Version: '10.3.12-MariaDB-1:10.3.12+maria~bionic' socket: '/var/run/mysqld/mysq ld.sock' port: 3306 mariadb.org binary distribution
The official MariaDB image uses CMD ["mysqld"] instead of RUN mysql. So
FROM mariadb
COPY ./my-initial.sql /docker-entrypoint-initdb.d/
CMD ["mysqld"]
should do the trick.
To execute initial SQL see this section of the image docs.