I'm new to docker. I've been following this tutorial: https://medium.com/coderscorner/connecting-to-mysql-through-docker-997aa2c090cc . I've set up the root password but once I tried to access the mysql command, it throws out this database is uninitialized error. Also, when I do docker-compose up command to pull the needed modules, it gives out an django.db.utils.InternalError: (1049, "Unknown database 'bitpal'"). The command I placed was:
docker run --name=mysql -e MYSQL_USER=root MYSQL_ROOT_PASSWORD=password -d mysql
I reckon I've searched for answers here but I couldn't be sure of what's wrong.
docker-compose.yml
version: '2'
services:
# Redis
mysql:
image: mysql:5.7
restart: always
hostname: mysql
container_name: mysql
environment:
- MYSQL_USER=root
- MYSQL_ROOT_PASSWORD=password
- MYSQL_DB=bitpal
ports:
- "3306:3306"
# Redis
redis:
image: redis:latest
restart: always
hostname: redis
container_name: redis
ports:
- "6379:6379"
# Django web server
bitpal:
image: python:3.5
restart: always
hostname: bitpal
container_name: bitpal
working_dir: /bitpal
command: ./bin/start_dev.sh
volumes:
- ./bitpal:/bitpal
- ./etc/config:/etc/config
- ./log:/log
ports:
- "80:80"
links:
- mysql
- redis
depends_on:
- mysql
environment:
# Database
- DB_NAME=bitpal
- DB_USER=root
- DB_PASSWORD=password
- DB_HOST=mysql
- DB_PORT=3306
# Celery worker
worker:
image: python:3.5
restart: always
container_name: worker
command: bash -c "./bin/install.sh && ./bin/celery_worker.sh"
working_dir: /bitpal
volumes:
- ./bitpal:/bitpal
- ./etc/config:/etc/config
- ./log:/log
links:
- mysql
- redis
depends_on:
- redis
# Bitshares websocket listener
websocket_listener:
image: python:3.5
restart: always
container_name: websocket_listener
command: bash -c "./bin/install.sh && ./bin/websocket_listener.sh"
working_dir: /bitpal
volumes:
- ./bitpal:/bitpal
- ./etc/config:/etc/config
- ./log:/log
links:
- mysql
- redis
depends_on:
- redis
# Nginx
nginx:
image: nginx:1.12.1
container_name: nginx
ports:
- "8000:80"
volumes:
- ./bitpal:/home/bitpal/bitpal/bitpal
- ./nginx:/etc/nginx/conf.d
depends_on:
- bitpal
My directory looks like this.
`**ROOT**
`root: .gitignore, docker-compose.yml, docker-compose-production.yml...
/bitpal /etc /log /nginx /public_html`
**ROOT/bitpal**
`.gitignore, Dockerfile, Makefile, manage.py... /bin /bitpal /media
/static /tests`
All the project's .sh files are stored under root/bitpal/bin. Do I place wait-for-it.sh there instead or place it in bitpal and nginx folders?
This tutorial you were following is incomplete. It didn't tell you that you must wait until the db is initialized if you want to use it.
Just after running the database container via run command, you should check the logs of this container and wait until the DB initialization
process is complete
You can do it with:
$ docker logs -f <container name>
Where container name in your case is mysql. When you see that db is initialized and DB is started, just detach (ctrl+c) from the logs and continue on.
Your DB is ready to use now.
important note considering your compose file
This compose file is not going to work because the other services like bitpal/worker are not waiting for the DB service to initialize.
Initially download a wait-for-it.sh script, that'd allow other servies to wait for your database when using compose file to setup your application. The script, made by vishnubob, is available here, then copy it to all the catalogs where your services requiring database are.
In the same catalogs create a docker-entrypoint.sh files and write them like this:
#!/bin/bash
set -e
sh -c './wait-for-it.sh mysql:3306 -t 30'
exec "$#"
Then, in your compose file add entries in every service that require DB (and where you places wait-for-it.sh script) that will execute the waiting script:
entrypoint: ["./docker-entrypoint.sh"]
Then, your services will wait for the DB until it's initialized and ready to accept connections.
In the edits I'll add straight forward catalog tree so that you can more clearly see how these files should be placed.
This is one of the only efficient methods because depends_on is not waiting for the db service to be initialized as it's clearly stated in the official docs.
edit with files location explanation
root
- bitpal
+ *some service files*
+ wait-for-it.sh
+ docker-entrypoint.sh
- some_service_requiring_db
+ *some service files*
+ wait-for-it.sh
+ docker-entrypoint.sh
- docker-compose.yml
And your compose file should be like:
version: '2'
services:
# MySQL service definition
mysql:
# like you have
# some services
# Django web server
bitpal:
# ...
entrypoint: ["./docker-entrypoint.sh"]
# further declarations
Related
I need to execute these commands on every startup since it looks to be overwritten every time I set and restart it
mysql -uroot -padmin;
set global general_log = 1;
I start the docker container with docker-compose for development purposes only and it looks like this.
version: "3.8"
services:
mysql_service:
container_name: db_container
build:
context: .
dockerfile: ./db/Dockerfile.dev
# needed for mysql 8+
command: --default-authentication-plugin=mysql_native_password
restart: always
hostname: db
ports:
- target: 3306
published: 3306
protocol: tcp
mode: host
environment:
- MYSQL_ROOT_PASSWORD=admin
- MYSQL_DATABASE=example
- MYSQL_ROOT_HOST=localhost
- MYSQL_ALLOW_EMPTY_PASSWORD=yes
volumes:
- ./db/:/docker-entrypoint-initdb.d/
- data_volume:/var/lib/mysql/
cap_add:
- ALL
volumes:
data_volume:
driver: local
and the Dockerfile
FROM mysql:8.0
COPY ./DevOps/Docker/db/set_logging.sh /usr/local/bin
ENTRYPOINT ["docker-entrypoint.sh", "./usr/local/bin/set_logging.sh"]
However the copy goes through but the script is never executed.
Where the script looks like
#!/bin/bash
mysql -uroot -padmin -e "set global general_log = 1"
Any suggestions on getting this command to go through? This is only for development
In order to tell MYSQL container to run that script once it starts you can either mount the script into the image's /docker-entrypoint-initdb.d folder using docker file, or docker-compose file using bind mount volume.
Dockerfile
FROM mysql:8.0
COPY ./DevOps/Docker/db/script.sh /docker-entrypoint-initdb.d/script.sh
ENTRYPOINT ["docker-entrypoint.sh"]
docker-compose
version: "3.8"
services:
mysql_service:
container_name: db_container
build:
context: .
dockerfile: ./db/Dockerfile.dev
# needed for mysql 8+
command: --default-authentication-plugin=mysql_native_password
restart: always
hostname: db
ports:
- target: 3306
published: 3306
protocol: tcp
mode: host
environment:
- MYSQL_ROOT_PASSWORD=admin
- MYSQL_DATABASE=example
- MYSQL_ROOT_HOST=localhost
- MYSQL_ALLOW_EMPTY_PASSWORD=yes
volumes:
- "./DevOps/Docker/db/script.sh:/docker-entrypoint-initdb.d/script.sh"
- data_volume:/var/lib/mysql/
cap_add:
- ALL
volumes:
data_volume:
driver: local
You can check how scripts are launched in /docker-entrypoint-initdb.d read the Initializing a fresh instance Section
I'm very new to Docker, and I'm trying to dockerize a Go REST API and MySQL database to communicate with each other using Docker Compose. I am getting the error [main] Error 1049: Unknown database 'puapp'
Docker compose:
version: '3'
services:
db:
build: ./mysql/
restart: always
environment:
- MYSQL_ROOT_PASSWORD=root
volumes:
- db_volume:/var/lib/mysql
api-service:
restart: always
build: ./
ports:
- "8080:80"
environment:
- DB_USER=root
- DB_PASS=root
- DB_ADDRESS=db:3306
- DB_PROTOCOL=tcp
- DB_NAME=puapp
depends_on:
- db
links:
- db
volumes:
db_volume:
Dockerfile for go service:
# syntax=docker/dockerfile:1
# Build stage
FROM golang:1.16-alpine AS builder
WORKDIR /app
COPY . .
RUN go mod download
WORKDIR /app/src/main
RUN go build -o restserv
# Run stage
FROM alpine:3.13
WORKDIR /app
COPY --from=builder /app/src/main/restserv .
EXPOSE 8080
CMD "./restserv"
Dockerfile for MySQL:
FROM mysql:latest
ADD dump.sql /docker-entrypoint-initdb.d
Full code - https://github.com/bens-schreiber/restservproj
Let me know if I need to add anything
Containers will be having their own ip addresses, so API container won't be able to access mysql container over 127.0.0.1. As mentioned in the comments, you want to utilize container's names to addresses from container from another. See this page for details.
What I want to do is:
Create a MySQL8 docker container
The MySQL container should run a dump file
I was successful in creating the basic container, however, that are several issues:
The password that I added in docker-compose.yml is ignored, when I run:
"docker exec -it mysqlDB bash" followed by "MySQL -u admin -p" I get Acess denied, and also with root
I don't know if the dump is being used because I can access the DB
I'm also getting this error:
[ERROR] [MY-000061] [Server] 1105 Input Output error while reading file /docker-entrypoint-initdb.d/, line 0, I/O error code 1
I tried many things for hours and it only got worse like not running at all.
I always run with: "docker-compose --log-level DEBUG -verbose up"
I always retry with the sequence:
ctrl+c
docker-compose down
docker system prune -a
docker volume prune
After running these prunes I need to run twice, or else I got the error:
"The designated data directory /var/lib/mysql/ is unusable. You can remove all files that the server added to it."
Dockerfile(at /MySQL), there's also a LastDump.sql in this directory
EDIT: Later I deleted this file, and got the same result
FROM mysql:8.0.21
RUN chown -R mysql:root /var/lib/mysql/
ENV MYSQL_DATABASE=Olimpo
ENV MYSQL_USER=admin
ENV MYSQL_PASSWORD=senha
ENV MYSQL_ROOT_PASSWORD=senha
ADD LastDump.sql /etc/mysql/LastDump.sql
RUN sed -i 's/MYSQL_DATABASE/'$MYSQL_DATABASE'/g' /etc/mysql/LastDump.sql
RUN cp /etc/mysql/LastDump.sql /docker-entrypoint-initdb.d
EXPOSE 3306
docker-compose.yml(at / main folder)
version: "3.8"
# Define services
services:
# Database Service (Mysql)
mysqldb:
image: mysql:8.0.21
container_name: mysqlDB
command: --default-authentication-plugin=mysql_native_password --init-file /docker-entrypoint-initdb.d/
ports:
- "3307:3306"
restart: always
environment:
MYSQL_DATABASE: Olimpo
MYSQL_USER: admin
MYSQL_PASSWORD: senha
MYSQL_ROOT_PASSWORD: senha
volumes:
- mysql_data:/var/lib/mysql
# next line is commented doesn't run with it
#- ./MySQL/LastDump.sql:/docker-entrypoint-initdb.d
networks:
- backend
# Volumes
volumes:
mysql_data:
driver: local
# Networks to be created to facilitate communication between containers
networks:
backend:
Your problem seem to be the parameter in the command of your yml file. It worked in my machine when I took it out.
Change from command: --default-authentication-plugin=mysql_native_password --init-file /docker-entrypoint-initdb.d/ to command: command: --default-authentication-plugin=mysql_native_password
Fixed file is below:
# Define services
services:
# Database Service (Mysql)
mysqldb:
image: mysql:8.0.21
container_name: mysqlDB
command: --default-authentication-plugin=mysql_native_password
ports:
- "3307:3306"
restart: always
environment:
MYSQL_DATABASE: Olimpo
MYSQL_USER: admin
MYSQL_PASSWORD: senha
MYSQL_ROOT_PASSWORD: senha
volumes:
- mysql_data:/var/lib/mysql
# next line is commented doesn't run with it
#- ./MySQL/LastDump.sql:/docker-entrypoint-initdb.d
networks:
- backend
# Volumes
volumes:
mysql_data:
driver: local
# Networks to be created to facilitate communication between containers
networks:
backend:
The mysql dockerfile is useless.
Create a folder called mysql-dump with the dump.sql inside.
Add the line "USE db_name" to dump.sql
In docker-compose.yml:
remove "--init-file /docker-entrypoint-initdb.d/"
add the line in volumes "- ./mysql-dump:/docker-entrypoint-initdb.d"
I have a spring boot project which I'd like to containerize using docker.
I have a couple of spring boot applications which connect to same MySql server.
My spring applications requires the database to be completely setup (i.e. all the tables to be created and some data to be inserted in some of the tables) in order to start.
I am using Docker version 18.09.0 and docker-compose version 1.23.1 and ubuntu 16.04 LTS
I have two files create.sql and insert.sql, which I use to initialise the database to be used by the application.
I create the images using the command docker-compose.yml and it runs successfully and creates the images.
I have the following questions.
I assume when using docker-compose, a container starts as soon as all its dependent containers have started. Is there a way to wait for the mysql server to be up and ready to accept connections, before my API container gets started?
If I chose to create containers separately for the applications and mysql, and not use docker-compose, how do I make sure that my applications connect to the mysql container?
Is there any other tool which might help me achieve this?
Note:
I have tried to use docker inspect <container_id> to find the the IpAddress for the mysql container and use it to connect, but it doesn't work as well.
The following are the files I am using to create images.
docker-compose.yml file.
version: '3'
services:
demo-mysql:
image: demo-mysql
build: ./demo-mysql
volumes:
- /mnt/data/mysql-data:/var/lib/mysql
ports:
- 3306:3306
environment:
- MYSQL_ROOT_PASSWORD=root
- MYSQL_DATABASE=demo
- MYSQL_PASSWORD=root
demo-api:
image: demo-api-1.0
build: ./api
depends_on:
- demo-mysql
ports:
- 8080:8080
environment:
- DATABASE_HOST=demo-mysql
- DATABASE_USER=root
- DATABASE_PASSWORD=root
- DATABASE_NAME=demo
- DATABASE_PORT=3306
demo1-app:
image: demo1-app-1.0
build: ./demo1
depends_on:
- demo-mysql
ports:
- 8090:8090
environment:
- DATABASE_HOST=demo-mysql
- DATABASE_USER=root
- DATABASE_PASSWORD=root
- DATABASE_NAME=demo
- DATABASE_PORT=3306
The following is the Dockerfile for the spring boot project
FROM java:8
VOLUME /tmp
ARG DATA_PATH=/src/main/resources
ARG APP_PORT=8080
EXPOSE ${APP_PORT}
ADD /build/libs/demo-api.jar demo-api.jar
ENTRYPOINT ["java","-jar","demo-api.jar"]
The following is the Dockerfile I used to create my mysql image
FROM mysql:5.7
ENV MYSQL_DATABASE=demo \
MYSQL_USER=root \
MYSQL_ROOT_PASSWORD=root
ADD ./1.0/create.sql /docker-entrypoint-initdb.d
ADD ./1.0/insert.sql /docker-entrypoint-initdb.d
EXPOSE 3306
Use the healthcheck feature of docker-compose (https://docs.docker.com/compose/compose-file/#healthcheck).
Something like this:
services:
demo-mysql:
image: demo-mysql
build: ./demo-mysql
volumes:
- /mnt/data/mysql-data:/var/lib/mysql
ports:
- 3306:3306
environment:
- MYSQL_ROOT_PASSWORD=root
- MYSQL_DATABASE=demo
- MYSQL_PASSWORD=root
healthcheck:
test: ["CMD-SHELL", 'mysqladmin ping']
interval: 10s
timeout: 2s
retries: 10
The depending containers will not start until the demo-mysql container is healthy
After trying several approaches, IMO the simplest and most elegant option is using the jwilder/dockerize utility image with its -wait flag. Here is a simple example where I need a MySQL database to be ready before starting my app:
version: "3.8"
services:
# Start MySQL.
db:
image: mysql
# Wait for MySQL to be joinable.
check-db-started:
image: jwilder/dockerize:0.6.1
depends_on:
- db
command: 'dockerize -wait=tcp://db:3306'
# Only start myapp once MySQL is joinable.
myapp:
image: myapp:latest
depends_on:
- check-db-started
I have the mysql database stored in /home/mysql instead of /var/lib/mysql. The directory used to be owned by mysql. However, when I run the command docker-compose up with this yml file:
version: '3'
services:
mariadb:
image: mariadb
restart: always
volumes:
- /home/mysql:/var/lib/mysql
elasticsearch:
image: docker.elastic.co/elasticsearch/elasticsearch:5.6.4
environment:
- "ES_JAVA_OPTS=-Xms750m -Xmx750m"
- bootstrap.memory_lock=false
site:
build: .
volumes:
- "./app:/app"
links:
- mariadb:mysql
environment:
- DOCKER_IP=172.19.0.2
depends_on: ['elasticsearch','mariadb']
ports:
- "3000:3000"
The docker container is able to run, but the entire folder and files in /home/mysql are owned by systemd-journal-remote, which causes the node server fails to connect to mariadb. I have to stop the docker instance, restore the mysql folder ownership and delete ib_logfile0 and ib_logfile1.
Why does mounting /home/mysql cause such a fatal problem?
Update:
My solution is to add user: "mysql":
version: '3'
services:
mariadb:
image: mariadb
restart: always
volumes:
- /home/mysql:/var/lib/mysql
user: "mysql"
elasticsearch:
image: docker.elastic.co/elasticsearch/elasticsearch:5.6.4
environment:
- "ES_JAVA_OPTS=-Xms750m -Xmx750m"
- bootstrap.memory_lock=false
site:
build: .
volumes:
- "./app:/app"
links:
- mariadb:mysql
environment:
- DOCKER_IP=172.19.0.2
depends_on: ['elasticsearch','mariadb']
ports:
- "3000:3000"
You should start Docker's container with --user parameter. If you do this and set the same uid:gid as owner of the MySQL storage you will no have problems with permissions. You have to check how exactly to do this in Docker Compose because I show you example for normal command line execution
Most likely, uid of your user systemd-journal-remote is the same as uid of user mysqld in container. Check with ls -n. To avoid confusion, either use common uids, perhaps test as root:root with chmod o+rwx.