What I want to do is:
Create a MySQL8 docker container
The MySQL container should run a dump file
I was successful in creating the basic container, however, that are several issues:
The password that I added in docker-compose.yml is ignored, when I run:
"docker exec -it mysqlDB bash" followed by "MySQL -u admin -p" I get Acess denied, and also with root
I don't know if the dump is being used because I can access the DB
I'm also getting this error:
[ERROR] [MY-000061] [Server] 1105 Input Output error while reading file /docker-entrypoint-initdb.d/, line 0, I/O error code 1
I tried many things for hours and it only got worse like not running at all.
I always run with: "docker-compose --log-level DEBUG -verbose up"
I always retry with the sequence:
ctrl+c
docker-compose down
docker system prune -a
docker volume prune
After running these prunes I need to run twice, or else I got the error:
"The designated data directory /var/lib/mysql/ is unusable. You can remove all files that the server added to it."
Dockerfile(at /MySQL), there's also a LastDump.sql in this directory
EDIT: Later I deleted this file, and got the same result
FROM mysql:8.0.21
RUN chown -R mysql:root /var/lib/mysql/
ENV MYSQL_DATABASE=Olimpo
ENV MYSQL_USER=admin
ENV MYSQL_PASSWORD=senha
ENV MYSQL_ROOT_PASSWORD=senha
ADD LastDump.sql /etc/mysql/LastDump.sql
RUN sed -i 's/MYSQL_DATABASE/'$MYSQL_DATABASE'/g' /etc/mysql/LastDump.sql
RUN cp /etc/mysql/LastDump.sql /docker-entrypoint-initdb.d
EXPOSE 3306
docker-compose.yml(at / main folder)
version: "3.8"
# Define services
services:
# Database Service (Mysql)
mysqldb:
image: mysql:8.0.21
container_name: mysqlDB
command: --default-authentication-plugin=mysql_native_password --init-file /docker-entrypoint-initdb.d/
ports:
- "3307:3306"
restart: always
environment:
MYSQL_DATABASE: Olimpo
MYSQL_USER: admin
MYSQL_PASSWORD: senha
MYSQL_ROOT_PASSWORD: senha
volumes:
- mysql_data:/var/lib/mysql
# next line is commented doesn't run with it
#- ./MySQL/LastDump.sql:/docker-entrypoint-initdb.d
networks:
- backend
# Volumes
volumes:
mysql_data:
driver: local
# Networks to be created to facilitate communication between containers
networks:
backend:
Your problem seem to be the parameter in the command of your yml file. It worked in my machine when I took it out.
Change from command: --default-authentication-plugin=mysql_native_password --init-file /docker-entrypoint-initdb.d/ to command: command: --default-authentication-plugin=mysql_native_password
Fixed file is below:
# Define services
services:
# Database Service (Mysql)
mysqldb:
image: mysql:8.0.21
container_name: mysqlDB
command: --default-authentication-plugin=mysql_native_password
ports:
- "3307:3306"
restart: always
environment:
MYSQL_DATABASE: Olimpo
MYSQL_USER: admin
MYSQL_PASSWORD: senha
MYSQL_ROOT_PASSWORD: senha
volumes:
- mysql_data:/var/lib/mysql
# next line is commented doesn't run with it
#- ./MySQL/LastDump.sql:/docker-entrypoint-initdb.d
networks:
- backend
# Volumes
volumes:
mysql_data:
driver: local
# Networks to be created to facilitate communication between containers
networks:
backend:
The mysql dockerfile is useless.
Create a folder called mysql-dump with the dump.sql inside.
Add the line "USE db_name" to dump.sql
In docker-compose.yml:
remove "--init-file /docker-entrypoint-initdb.d/"
add the line in volumes "- ./mysql-dump:/docker-entrypoint-initdb.d"
Related
I don't know how unique my problem is but I see similar issues throughout the web but I have the scenario that doesn't match any of the problems I've seen.
I have a pretty standard Express app that uses SequelizeJS and MySQL 8. Sequlize has migration which I run using entrypoint.sh that runs sequelize migration command.
# Run migrations
npx sequelize-cli db:migrate
# Preload initial data
npx sequelize-cli db:seed:all
When I run docker compose command the project builds without any issues and sequelize migration and seeds runs. Database connection from sequelize command line works perfectly!
But when I load my homepage the SQL query from the application throws error. The error itself is quite common
{
code: 'ER_NOT_SUPPORTED_AUTH_MODE',
errno: 1251,
sqlMessage: 'Client does not support authentication protocol requested by server; consider upgrading MySQL client',
sqlState: '08004',
fatal: true
}
I have seen this error while mysql 8 and the package mysql used together with sequlizejs. The common solution is to use mysql2 which I am using in my project. I've also tried to run command --default-authentication-plugin=mysql_native_password inside mysql container which didn't help.
Docker Compose
version: '3.9'
services:
linkfy-dashboard:
container_name: linkfy-dashboard
restart: always
depends_on:
- linkfy-mysql
build:
context: .
dockerfile: Dockerfile.dev
ports:
- '10200:3000'
environment:
APP_NAME: Linkfy
DB_HOST: linkfy-mysql
DB_USER: linkfy
DB_PASS: password
DB_NAME: linkfy
COOKIE_SECRET: secret
NONCE_SECRET: secret
COOKIE_SECURE: 'false'
volumes:
- /app/node_modules
- .:/app
networks:
- linkfy
linkfy-mysql:
container_name: linkfy-mysql
command: --default-authentication-plugin=mysql_native_password
restart: always
image: mysql:8
environment:
MYSQL_ROOT_PASSWORD: password
MYSQL_DATABASE: linkfy
MYSQL_USER: linkfy
MYSQL_PASSWORD: password
volumes:
- database:/var/lib/mysql
networks:
- linkfy
adminer:
image: adminer
restart: always
ports:
- 10210:8080
networks:
- linkfy
networks:
linkfy:
name: linkfy
volumes:
database:
Dockerfile
FROM node:14.17-alpine
# source directory
WORKDIR /app
# global dependancy
RUN yarn global add sequelize-cli nodemon
# dependency file (package.json and package-lock.json)
COPY package.json .
RUN apk add --update python2 make g++ && rm -rf /var/cache/apk/*
RUN yarn install
# copy the sources
COPY . .
# listener
EXPOSE 3000
ENTRYPOINT ["./entrypoint.sh"]
# command to run
CMD [ "npm", "run", "dev" ]
I need some direction how to troubleshoot this.
I need to execute these commands on every startup since it looks to be overwritten every time I set and restart it
mysql -uroot -padmin;
set global general_log = 1;
I start the docker container with docker-compose for development purposes only and it looks like this.
version: "3.8"
services:
mysql_service:
container_name: db_container
build:
context: .
dockerfile: ./db/Dockerfile.dev
# needed for mysql 8+
command: --default-authentication-plugin=mysql_native_password
restart: always
hostname: db
ports:
- target: 3306
published: 3306
protocol: tcp
mode: host
environment:
- MYSQL_ROOT_PASSWORD=admin
- MYSQL_DATABASE=example
- MYSQL_ROOT_HOST=localhost
- MYSQL_ALLOW_EMPTY_PASSWORD=yes
volumes:
- ./db/:/docker-entrypoint-initdb.d/
- data_volume:/var/lib/mysql/
cap_add:
- ALL
volumes:
data_volume:
driver: local
and the Dockerfile
FROM mysql:8.0
COPY ./DevOps/Docker/db/set_logging.sh /usr/local/bin
ENTRYPOINT ["docker-entrypoint.sh", "./usr/local/bin/set_logging.sh"]
However the copy goes through but the script is never executed.
Where the script looks like
#!/bin/bash
mysql -uroot -padmin -e "set global general_log = 1"
Any suggestions on getting this command to go through? This is only for development
In order to tell MYSQL container to run that script once it starts you can either mount the script into the image's /docker-entrypoint-initdb.d folder using docker file, or docker-compose file using bind mount volume.
Dockerfile
FROM mysql:8.0
COPY ./DevOps/Docker/db/script.sh /docker-entrypoint-initdb.d/script.sh
ENTRYPOINT ["docker-entrypoint.sh"]
docker-compose
version: "3.8"
services:
mysql_service:
container_name: db_container
build:
context: .
dockerfile: ./db/Dockerfile.dev
# needed for mysql 8+
command: --default-authentication-plugin=mysql_native_password
restart: always
hostname: db
ports:
- target: 3306
published: 3306
protocol: tcp
mode: host
environment:
- MYSQL_ROOT_PASSWORD=admin
- MYSQL_DATABASE=example
- MYSQL_ROOT_HOST=localhost
- MYSQL_ALLOW_EMPTY_PASSWORD=yes
volumes:
- "./DevOps/Docker/db/script.sh:/docker-entrypoint-initdb.d/script.sh"
- data_volume:/var/lib/mysql/
cap_add:
- ALL
volumes:
data_volume:
driver: local
You can check how scripts are launched in /docker-entrypoint-initdb.d read the Initializing a fresh instance Section
Pulled MySQL image using command
docker pull mysql
$ docker run --name some-mysql -e MYSQL_ROOT_PASSWORD=my-secret-pw -d mysql:tag
Then made a stack.yml file in my root for mysql
stack.yml :
version: '3.1'
services:
db:
image: mysql
command: --default-authentication-plugin=mysql_native_password
restart: always
environment:
MYSQL_ROOT_PASSWORD: example
adminer:
image: adminer
restart: always
ports:
- 8080:8080
And lastly composed it up.
docker-compose -f stack.yml up (after making the stack.yml file)
After it stopped running I accessed the localhost:8080 page and the mysql database login page was loaded.
Superset setup
git clone https://github.com/apache/incubator-superset.git
cd incubator-superset
docker-compose up
Accessed superset on localhost:8088 page.
How do I frame the SQLAlchemy URI for connecting the Database and Superset?
You can access both applications individually due to it has a different bridge network. But when it comes to connecting each other, You have to connect through the same network for both applications. Superset already running on incubator-superset_default and MySQL running on default bridge network.
Here tested stack.yaml
version: '3.7'
services:
db:
image: mysql
command: --default-authentication-plugin=mysql_native_password
container_name: mysql
restart: always
environment:
MYSQL_ROOT_PASSWORD: sample
networks:
- proxynet
networks:
proxynet:
name: incubator-superset_default
Note:
As a user-defined network so use can use the service container name or container IP address as follows
mysql://root:sample#mysql/mysql
mysql://root:sample#172.19.0.5/mysql
I'm new to docker. I've been following this tutorial: https://medium.com/coderscorner/connecting-to-mysql-through-docker-997aa2c090cc . I've set up the root password but once I tried to access the mysql command, it throws out this database is uninitialized error. Also, when I do docker-compose up command to pull the needed modules, it gives out an django.db.utils.InternalError: (1049, "Unknown database 'bitpal'"). The command I placed was:
docker run --name=mysql -e MYSQL_USER=root MYSQL_ROOT_PASSWORD=password -d mysql
I reckon I've searched for answers here but I couldn't be sure of what's wrong.
docker-compose.yml
version: '2'
services:
# Redis
mysql:
image: mysql:5.7
restart: always
hostname: mysql
container_name: mysql
environment:
- MYSQL_USER=root
- MYSQL_ROOT_PASSWORD=password
- MYSQL_DB=bitpal
ports:
- "3306:3306"
# Redis
redis:
image: redis:latest
restart: always
hostname: redis
container_name: redis
ports:
- "6379:6379"
# Django web server
bitpal:
image: python:3.5
restart: always
hostname: bitpal
container_name: bitpal
working_dir: /bitpal
command: ./bin/start_dev.sh
volumes:
- ./bitpal:/bitpal
- ./etc/config:/etc/config
- ./log:/log
ports:
- "80:80"
links:
- mysql
- redis
depends_on:
- mysql
environment:
# Database
- DB_NAME=bitpal
- DB_USER=root
- DB_PASSWORD=password
- DB_HOST=mysql
- DB_PORT=3306
# Celery worker
worker:
image: python:3.5
restart: always
container_name: worker
command: bash -c "./bin/install.sh && ./bin/celery_worker.sh"
working_dir: /bitpal
volumes:
- ./bitpal:/bitpal
- ./etc/config:/etc/config
- ./log:/log
links:
- mysql
- redis
depends_on:
- redis
# Bitshares websocket listener
websocket_listener:
image: python:3.5
restart: always
container_name: websocket_listener
command: bash -c "./bin/install.sh && ./bin/websocket_listener.sh"
working_dir: /bitpal
volumes:
- ./bitpal:/bitpal
- ./etc/config:/etc/config
- ./log:/log
links:
- mysql
- redis
depends_on:
- redis
# Nginx
nginx:
image: nginx:1.12.1
container_name: nginx
ports:
- "8000:80"
volumes:
- ./bitpal:/home/bitpal/bitpal/bitpal
- ./nginx:/etc/nginx/conf.d
depends_on:
- bitpal
My directory looks like this.
`**ROOT**
`root: .gitignore, docker-compose.yml, docker-compose-production.yml...
/bitpal /etc /log /nginx /public_html`
**ROOT/bitpal**
`.gitignore, Dockerfile, Makefile, manage.py... /bin /bitpal /media
/static /tests`
All the project's .sh files are stored under root/bitpal/bin. Do I place wait-for-it.sh there instead or place it in bitpal and nginx folders?
This tutorial you were following is incomplete. It didn't tell you that you must wait until the db is initialized if you want to use it.
Just after running the database container via run command, you should check the logs of this container and wait until the DB initialization
process is complete
You can do it with:
$ docker logs -f <container name>
Where container name in your case is mysql. When you see that db is initialized and DB is started, just detach (ctrl+c) from the logs and continue on.
Your DB is ready to use now.
important note considering your compose file
This compose file is not going to work because the other services like bitpal/worker are not waiting for the DB service to initialize.
Initially download a wait-for-it.sh script, that'd allow other servies to wait for your database when using compose file to setup your application. The script, made by vishnubob, is available here, then copy it to all the catalogs where your services requiring database are.
In the same catalogs create a docker-entrypoint.sh files and write them like this:
#!/bin/bash
set -e
sh -c './wait-for-it.sh mysql:3306 -t 30'
exec "$#"
Then, in your compose file add entries in every service that require DB (and where you places wait-for-it.sh script) that will execute the waiting script:
entrypoint: ["./docker-entrypoint.sh"]
Then, your services will wait for the DB until it's initialized and ready to accept connections.
In the edits I'll add straight forward catalog tree so that you can more clearly see how these files should be placed.
This is one of the only efficient methods because depends_on is not waiting for the db service to be initialized as it's clearly stated in the official docs.
edit with files location explanation
root
- bitpal
+ *some service files*
+ wait-for-it.sh
+ docker-entrypoint.sh
- some_service_requiring_db
+ *some service files*
+ wait-for-it.sh
+ docker-entrypoint.sh
- docker-compose.yml
And your compose file should be like:
version: '2'
services:
# MySQL service definition
mysql:
# like you have
# some services
# Django web server
bitpal:
# ...
entrypoint: ["./docker-entrypoint.sh"]
# further declarations
I'm having trouble importing an .sql dump file with docker-compose. I've followed the docs, which apparently will load the .sql file from docker-entrypoint-initdb.d. However, when I run docker-compose up, the sql file is not copied over to the container.
I've tried stopping the containers with -vf flag, but that didn't work either. Am I doing something wrong in my .yml script?
I have dump.sql in the directory database/db-dump/ in the root where my compose file is.
frontend:
image: myimage
ports:
- "80:80"
links:
- mysql
mysql:
image: mysql
ports:
- "3306:3306"
environment:
MYSQL_ROOT_PASSWORD: rootpass
MYSQL_USER: dbuser
MYSQL_PASSWORD: userpass
MYSQL_DATABASE: myimage_db
volumes:
- ./database/db-dump:/docker-entrypoint-initdb.d
This worked for me,
version: '3.1'
services:
db:
image: mysql
command: --default-authentication-plugin=mysql_native_password
restart: always
volumes:
- ./mysql-dump:/docker-entrypoint-initdb.d
environment:
MYSQL_ROOT_PASSWORD: example
MYSQL_DATABASE: ecommerce
adminer:
image: adminer
restart: always
ports:
- 8080:8080
mysql-dump must be a directory. All the .sql's in the directory will be imported.
After many attempts with the volumes setting i found a workaround
I created another image based on mysql with the following in the Dockerfile
FROM mysql:5.6
ADD dump.sql /docker-entrypoint-initdb.d
Then removed the volumes from compose and ran the new image
frontend:
image: myimage
ports:
- "80:80"
links:
- mysql
mysql:
image: mymysql
ports:
- "3306:3306"
environment:
MYSQL_ROOT_PASSWORD: rootpass
MYSQL_USER: dbuser
MYSQL_PASSWORD: userpass
MYSQL_DATABASE: myimage_db
This way the dump is always copied over and run on startup
This appears on the documentation page of Docker MySQL image: https://hub.docker.com/_/mysql/
Initializing a fresh instance
When a container is started for the first time, a new database with
the specified name will be created and initialized with the provided
configuration variables. Furthermore, it will execute files with
extensions .sh, .sql and .sql.gz that are found in
/docker-entrypoint-initdb.d. Files will be executed in alphabetical
order. You can easily populate your mysql services by mounting a SQL
dump into that
directory
and provide custom
images with contributed
data. SQL files will be imported by default to the database specified
by the MYSQL_DATABASE variable.
Mysql database dump schema.sql is residing in the /mysql-dump/schema.sql directory and it creates tables during the initialization process.
docker-compose.yml:
mysql:
image: mysql:5.7
command: mysqld --user=root
volumes:
- ./mysql-dump:/docker-entrypoint-initdb.d
environment:
MYSQL_DATABASE: ${MYSQL_DATABASE}
MYSQL_USER: ${MYSQL_USER}
MYSQL_PASSWORD: ${MYSQL_PASSWORD}
MYSQL_ROOT_PASSWORD: ${MYSQL_ROOT_PASSWORD}
I was having a similar issue with mysql where I would mount a local directory at /configs/mysql/data containing a mydatabasedump.sql file via docker-compose to the docker-entrypoint-initdb.d volume,
the file would get loaded on to the container but not execute or populate the database when the container initialized. My intial docker-compose.yml looke like this:
#docker-compose.yml
version: '3'
services:
db:
build: ./build/mysql/ #this is pointing to my Dockerfile
container_name: MYSQL_Database
restart: always
environment:
MYSQL_PORT: 3306
MYSQL_ROOT_PASSWORD: admin
MYSQL_DATABASE: my_app_database
MYSQL_USER: admin
MYSQL_PASSWORD: admin
volumes:
- ./configs/mysql/data:/docker-entrypoint-initdb.d:
I found two working solutions for this problem:
The first came after I logged in the running container and confirmed that mydatabasedump.sq file was present and executable in the container's docker-entrypoint-initdb.d directory; I created and added
a bash script to my local /configs/mysql/data directory called dump.sh that excuted after the container was initialized. It contains a single mysql command that copies my_database_dump.sql to my_app_database.
The bash script looks like this
#!/bin/bash
#dump.sh
mysql -uadmin -padmin my_app_database < my_database_dump.sql
#end of dump.sh
I executed this script via my Dockerfile in the ENTRYPOINT directive like this:
#Dockerfile
FROM mysql:5.5
ENTRYPOINT [ "dump.sh" ]
EXPOSE 80
#end of Dockerfile
After realizing the initial issue was due to the volumes being mouted after the cotainer is built and therefore not intilizing the database with the dump file (or executing any scripts in that directory) at boot time, the second solution was simply to
move the volumes directive in my compose-file above the built directive. This worked and allowed me to remove the dump.sh scrip and the DOCKERENTRY directive in my Dockerfile.
The modified docker-compose.yml looks like this
#docker-compose.yml
version: '3'
services:
db:
volumes:
- ./configs/mysql/data:/docker-entrypoint-initdb.d
build: ./build/mysql/ #this is pointing to my Dockerfile
container_name: MYSQL_Database
restart: always
environment:
MYSQL_PORT: 3306
MYSQL_ROOT_PASSWORD: admin
MYSQL_DATABASE: my_app_database
MYSQL_USER: admin
MYSQL_PASSWORD: admin
I also have this problem. I mount a local directory at ./mysql-dump containing a init.sql file via docker-compose to the docker-entrypoint-initdb.d volume, the file would get loaded on to the container but not execute or populate the database when the container initialized.
My intial docker-compose.yml looke like this:
mysqld:
image: mysql
container_name: mysqld
volumes:
- ./mysql/data:/var/lib/mysql
- ./mysql/my.cnf:/etc/my.cnf
- ./init:/docker-entrypoint-initdb.d
env_file: .env
restart: always
environment:
- MYSQL_ROOT_PASSWORD=123456
- MYSQL_DATABASE=fendou
command: --character-set-server=utf8mb4 --collation-server=utf8mb4_unicode_ci
--default-authentication-plugin=mysql_native_password
but it doesn't work for me.
I found another working solutions for this problem:
add --init-file /data/application/init.sql to mysql command.change above configuration like
mysqld:
image: mysql
container_name: mysqld
volumes:
- ./mysql/data:/var/lib/mysql
- ./mysql/my.cnf:/etc/my.cnf
# - ./init:/docker-entrypoint-initdb.d
env_file: .env
restart: always
environment:
- MYSQL_ROOT_PASSWORD=123456
- MYSQL_DATABASE=fendou
command: --character-set-server=utf8mb4 --collation-server=utf8mb4_unicode_ci
--default-authentication-plugin=mysql_native_password
--init-file /docker-entrypoint-initdb.d/init.sql #attention here
hope it help for you
I wanted to keep the original setup of the container, so I tried a restore on the already running container. This seemed to work:
cat dump.sql | docker-compose exec -T db mysql -h localhost -u root -psomewordpress -v
But it was very slow and the verbose output seemed to be buffered, so I tried:
docker-compose cp dump.sql db:/tmp/
docker-compose exec db sh -c "mysql -h localhost -u root -psomewordpress -v < /tmp/dump.sql"
Which at least provided faster feedback.
Might be useful for someone? Looks like it was mainly slow because I used --skip-extended-insert on the dump, without the extended inserts it went faster 🙂