I am trying to run a one time command on my application container using the command
docker-compose run --entrypoint="/usr/src/app/migrate.sh" app
app is the name of my service and the said entrypoint contains the one-time command that I'm trying to run.
Here's my docker-compose.yml file
version: '3'
services:
app:
build: .
# mount the current directory (on the host) to /usr/src/app on the container, any changes in either would be reflected in both the host and the container
volumes:
- .:/usr/src/app
# expose application on localhost:36081
ports:
- "36081:36081"
# application restarts if stops for any reason - required for the container to restart when the application fails to start due to the database containers not being ready
restart: always
depends_on:
- db1
- db2
# the environment variables are used in docker/config/env_config.rb to connect to different database containers
environment:
MYSQL_DB1_HOST: db1
MYSQL_DB1_PORT: 3306
MYSQL_DB2_HOST: db2
MYSQL_DB2_PORT: 3306
db1:
image: mysql/mysql-server:5.7
environment:
MYSQL_DATABASE: test1
MYSQL_ALLOW_EMPTY_PASSWORD: 'yes'
# mount volume of the schema script to /docker-entrypoint-initdb.d to execute the script on startup
volumes:
- ./docker/seed/db1:/docker-entrypoint-initdb.d
- db1-volume:/var/lib/mysql
restart: always
# to connect locally from SequelPro
ports:
- "1200:3306"
db2:
image: mysql/mysql-server:5.7
environment:
MYSQL_DATABASE: test2
MYSQL_ALLOW_EMPTY_PASSWORD: 'yes'
# mount volume of the schema script to /docker-entrypoint-initdb.d to execute the script on startup
volumes:
- ./docker/seed/db2:/docker-entrypoint-initdb.d
- db2-volume:/var/lib/mysql
restart: always
# to connect locally from SequelPro
ports:
- "1201:3306"
Everything works as expected when I start docker-compose up, but when I invoke docker-compose run, the dependent services db1 and db2 containers are up, but they are not initialised with the entrypoint script(as a result the mySQL database is not created). The volume is attached though.
How can I ensure that the entrypoint script of the dependent containers is invoked as well?
Related
What I want to do is:
Create a MySQL8 docker container
The MySQL container should run a dump file
I was successful in creating the basic container, however, that are several issues:
The password that I added in docker-compose.yml is ignored, when I run:
"docker exec -it mysqlDB bash" followed by "MySQL -u admin -p" I get Acess denied, and also with root
I don't know if the dump is being used because I can access the DB
I'm also getting this error:
[ERROR] [MY-000061] [Server] 1105 Input Output error while reading file /docker-entrypoint-initdb.d/, line 0, I/O error code 1
I tried many things for hours and it only got worse like not running at all.
I always run with: "docker-compose --log-level DEBUG -verbose up"
I always retry with the sequence:
ctrl+c
docker-compose down
docker system prune -a
docker volume prune
After running these prunes I need to run twice, or else I got the error:
"The designated data directory /var/lib/mysql/ is unusable. You can remove all files that the server added to it."
Dockerfile(at /MySQL), there's also a LastDump.sql in this directory
EDIT: Later I deleted this file, and got the same result
FROM mysql:8.0.21
RUN chown -R mysql:root /var/lib/mysql/
ENV MYSQL_DATABASE=Olimpo
ENV MYSQL_USER=admin
ENV MYSQL_PASSWORD=senha
ENV MYSQL_ROOT_PASSWORD=senha
ADD LastDump.sql /etc/mysql/LastDump.sql
RUN sed -i 's/MYSQL_DATABASE/'$MYSQL_DATABASE'/g' /etc/mysql/LastDump.sql
RUN cp /etc/mysql/LastDump.sql /docker-entrypoint-initdb.d
EXPOSE 3306
docker-compose.yml(at / main folder)
version: "3.8"
# Define services
services:
# Database Service (Mysql)
mysqldb:
image: mysql:8.0.21
container_name: mysqlDB
command: --default-authentication-plugin=mysql_native_password --init-file /docker-entrypoint-initdb.d/
ports:
- "3307:3306"
restart: always
environment:
MYSQL_DATABASE: Olimpo
MYSQL_USER: admin
MYSQL_PASSWORD: senha
MYSQL_ROOT_PASSWORD: senha
volumes:
- mysql_data:/var/lib/mysql
# next line is commented doesn't run with it
#- ./MySQL/LastDump.sql:/docker-entrypoint-initdb.d
networks:
- backend
# Volumes
volumes:
mysql_data:
driver: local
# Networks to be created to facilitate communication between containers
networks:
backend:
Your problem seem to be the parameter in the command of your yml file. It worked in my machine when I took it out.
Change from command: --default-authentication-plugin=mysql_native_password --init-file /docker-entrypoint-initdb.d/ to command: command: --default-authentication-plugin=mysql_native_password
Fixed file is below:
# Define services
services:
# Database Service (Mysql)
mysqldb:
image: mysql:8.0.21
container_name: mysqlDB
command: --default-authentication-plugin=mysql_native_password
ports:
- "3307:3306"
restart: always
environment:
MYSQL_DATABASE: Olimpo
MYSQL_USER: admin
MYSQL_PASSWORD: senha
MYSQL_ROOT_PASSWORD: senha
volumes:
- mysql_data:/var/lib/mysql
# next line is commented doesn't run with it
#- ./MySQL/LastDump.sql:/docker-entrypoint-initdb.d
networks:
- backend
# Volumes
volumes:
mysql_data:
driver: local
# Networks to be created to facilitate communication between containers
networks:
backend:
The mysql dockerfile is useless.
Create a folder called mysql-dump with the dump.sql inside.
Add the line "USE db_name" to dump.sql
In docker-compose.yml:
remove "--init-file /docker-entrypoint-initdb.d/"
add the line in volumes "- ./mysql-dump:/docker-entrypoint-initdb.d"
I have a requirement where I need to wait for a few commands before I seed the data for the database:
I have some Migration scripts that create the schema in the database (this command runs from my app container). After this executes, I want to seed data to the database.
As I read, the docker-entrypoint-initdb scripts is executed when the container is initialized. If I mount my seed.sql script to it, the data is seeded before the Migrate scripts. (The Migrate scripts actually drop all tables and create them from scratch). The seeded data is therefore lost.
How can I achieve this? (I cannot change the Migrate scripts)
Here's my docker-compose.yml file
version: '3'
services:
app:
build: .
# mount the current directory (on the host) to /usr/src/app on the container, any changes in either would be reflected in both the host and the container
volumes:
- .:/usr/src/app
# expose application on localhost:36081
ports:
- "36081:36081"
# application restarts if stops for any reason - required for the container to restart when the application fails to start due to the database containers not being ready
restart: always
environment:
MIGRATE: Y
<some env variables here>
config-dev:
image: mysql/mysql-server:5.7
environment:
MYSQL_DATABASE: config_dev
MYSQL_ALLOW_EMPTY_PASSWORD: 'yes'
volumes:
# to persist data
- config-dev-volume:/var/lib/mysql
restart: always
# to connect locally from SequelPro
ports:
- "1200:3306"
<other database containers>
My Dockerfile for app container has the following ENTRYPOINT
# start the application
ENTRYPOINT /usr/src/app/docker-entrypoint.sh
Here's the docker-entrypoint.sh file
#!/bin/bash
if [ "$MIGRATE" = "Y" ];
then
<command to start migration scripts>
echo "------------starting application--------------"
<command to start application>
else
echo "------------starting application--------------"
<command to start application>
fi
Edit: Is there a way I can run a script in config-db container from the docker-entrypoint.sh file in app container?
This can be solved in two steps:
You need to wait until your db container is started and is ready.
Wait until started can be handled by adding depends_on in docker-compose file:
version: '3'
services:
app:
build: .
# mount the current directory (on the host) to /usr/src/app on the container, any changes in either would be reflected in both the host and the container
depends_on:
- config-dev
- <other containers (if any)>
volumes:
- .:/usr/src/app
# expose application on localhost:36081
ports:
- "36081:36081"
# application restarts if stops for any reason - required for the container to restart when the application fails to start due to the database containers not being ready
restart: always
environment:
MIGRATE: Y
<some env variables here>
config-dev:
image: mysql/mysql-server:5.7
environment:
MYSQL_DATABASE: config_dev
MYSQL_ALLOW_EMPTY_PASSWORD: 'yes'
volumes:
# to persist data
- config-dev-volume:/var/lib/mysql
restart: always
# to connect locally from SequelPro
ports:
- "1200:3306"
<other database containers>
Wait until db is ready is another case because sometimes it takes time for the db process to start listening on the tcp port.
Unfortunately, Docker does not provide a way to hook onto container state. There are many tools and scripts to have a workaround this.
You can go through this to implement the workaround.
https://docs.docker.com/compose/startup-order/
TL;DR
Download https://raw.githubusercontent.com/vishnubob/wait-for-it/master/wait-for-it.sh inside the container and delete the ENTRYPOINT field (Not required for your use case) and use CMD field instead:
CMD ["./wait-for-it.sh", "<db_service_name_as_per_compose_file>:<port>", "--", "/usr/src/app/docker-entrypoint.sh"]
Now, That this is complete. Next part is to execute your seed.sql script.
That is easy and can be executed by adding following line into your /usr/src/app/docker-entrypoint.sh script.
sqlcmd -S -U -P -i inputquery_file_name -o outputfile_name
Place above command after migrate script in /usr/src/app/docker-entrypoint.sh
I have a project that runs when ./entrypoint.sh or docker-compose up is run from the root directory of project and generates the swagger API interface, but the calls return entry response no data.
If I run with MySQL on localhost without docker, works perfectly fine. How do I load the data?
entrypoint.sh
#!/bin/bash
docker network create turingmysql
docker container run -p 3306:3306 --name mysqldb --network turingmysql -e MYSQL_ROOT_PASSWORD=root -e MYSQL_DATABASE=tshirtshop -d mysql:5.7
docker-compose build
docker-compose up
DockerFile
FROM mysql:5.7
ADD ./database/tshirtshop.sql /docker-entrypoint-initdb.d
#### Stage 1: Build the application
FROM openjdk:8-jdk-alpine as build
# Set the current working directory inside the image
WORKDIR /app
# Copy maven executable to the image
COPY mvnw .
COPY .mvn .mvn
# Copy the pom.xml file
COPY pom.xml .
# Build all the dependencies in preparation to go offline.
# This is a separate step so the dependencies will be cached unless
# the pom.xml file has changed.
RUN ./mvnw dependency:go-offline -B
# Copy the project source
COPY src src
# Package the application
RUN ./mvnw package -DskipTests
RUN mkdir -p target/dependency && (cd target/dependency; jar -xf ../*.jar)
#### Stage 2: A minimal docker image with command to run the app
FROM openjdk:8-jre-alpine
ARG DEPENDENCY=/app/target/dependency
# Copy project dependencies from the build stage
COPY --from=build ${DEPENDENCY}/BOOT-INF/lib /app/lib
COPY --from=build ${DEPENDENCY}/META-INF /app/META-INF
COPY --from=build ${DEPENDENCY}/BOOT-INF/classes /app
ENTRYPOINT ["java","-cp","app:app/lib/*","com.turing.ecommerce.TuringApplication"]
docker-compose.yml
version: '3.7'
# Define services
services:
# App backend service
app-server:
# Configuration for building the docker image for the backend service
build:
context: .
dockerfile: Dockerfile
ports:
- "8080:8080" # Forward the exposed port 8080 on the container to port 8080 on the host machine
restart: always
depends_on:
- mysqldb # This service depends on mysql. Start that first.
environment: # Pass environment variables to the service
SPRING_DATASOURCE_URL: jdbc:mysql://mysqldb:3306/tshirtshop?useSSL=false&useLegacyDatetimeCode=false&serverTimezone=UTC
SPRING_DATASOURCE_USERNAME: root
SPRING_DATASOURCE_PASSWORD: root
networks: # Networks to join (Services on the same network can communicate with each other using their name)
- turingmysql
# Database Service (Mysql)
mysqldb:
image: mysql:5.7
ports:
- "3306:3306"
restart: always
environment:
MYSQL_DATABASE: tshirtshop
MYSQL_USER: root
MYSQL_ROOT_PASSWORD: root
volumes:
- db-data:/var/lib/mysql
networks:
- turingmysql
# Volumes
volumes:
db-data:
# Networks to be created to facilitate communication between containers
networks:
turingmysql:
Do you have two Dockerfiles? Looks like you built your own MySQL container?
Otherwise, these shouldn't be part of your Java multi-stage build
FROM mysql:5.7
ADD ./database/tshirtshop.sql /docker-entrypoint-initdb.d
Assuming that you did build a separate image for mysql, in the Docker-Compose, you're not using it, as you're still referring to image: mysql:5.7
Rather than building your own, you should mount the SQL script into it
For example
mysqldb:
image: mysql:5.7
...
volumes:
- db-data:/var/lib/mysql
- ./database/tshirtshop.sql:/docker-entrypoint-initdb.d/0_init.sql
Then, forget the Java service for a minute and use MySQL workbench or the mysql CLI to verify that data is actually there. Once you do, then startup the API
If you copying sql scipt already to docker build then you do not need to mapped it again in the docker-compose, if you have docker-compose then you do not the bash script single command docker-compose up --build will do the job.
So modify your docker-compose as per your Dockerfile.
Dockerfile
FROM mysql
ADD init.sql /docker-entrypoint-initdb.d
docker-compose
version: '3.7'
services:
# App backend service
app-server:
# Configuration for building the docker image for the backend service
build:
context: .
dockerfile: Dockerfile
ports:
- "8080:8080" # Forward the exposed port 8080 on the container to port 8080 on the host machine
restart: always
depends_on:
- mysqldb # This service depends on mysql. Start that first.
environment: # Pass environment variables to the service
SPRING_DATASOURCE_URL: jdbc:mysql://mysqldb:3306/tshirtshop?useSSL=false&useLegacyDatetimeCode=false&serverTimezone=UTC
SPRING_DATASOURCE_USERNAME: root
SPRING_DATASOURCE_PASSWORD: root
networks: # Networks to join (Services on the same network can communicate with each other using container name)
- uringmysql
# Database Service (Mysql)
mysql:
build: .
environment:
MYSQL_ROOT_PASSWORD: root123
MYSQL_DATABASE: appdata
MYSQL_USER: test
MYSQL_PASSWORD: root123
volumes:
- db-data:/var/lib/mysql
tty: true
# Volumes
volumes:
db-data:
# Networks to be created to facilitate communication between containers
networks:
turingmysql:
Now just run
docker-compose up --build
this will build and up the container and you will not need to mapped the host init script, as it already in Docker image.
The directory structure will look like
Now you application will able to access DB using jdbc:mysql://mysqldb:3306/tshirtshop? this endpoint as both are in same network and can refer eacher other using name.
Thank you cricket_007 and Adii for the responses. They put me in the right direction. I want to document my experience and how the issue was resolved. New to dockerization so I was learning by practice. For anyone new to dockerization and having same issues in Spring Boot, MySQL and docker, this would surely help
First, my entrypoint.sh changed below. The docker-compose down is for restarts.
#!/bin/bash
docker-compose down -v
docker-compose up --build
Second, I had to use an existing mysql image instead of building one.
version: '3.7'
# Define services
services:
# App backend service
app-server:
# Configuration for building the docker image for the backend service
build:
context: .
dockerfile: Dockerfile
ports:
- "8080:8080" # Forward the exposed port 8080 on the container to port 8080 on the host machine
restart: always
depends_on:
- mysql # This service depends on mysql. Start that first.
environment: # Pass environment variables to the service
SPRING_DATASOURCE_URL: jdbc:mysql://mysql:3306/tshirtshop?useSSL=false&allowPublicKeyRetrieval=true&useLegacyDatetimeCode=false&serverTimezone=UTC
SPRING_DATASOURCE_USERNAME: turing
SPRING_DATASOURCE_PASSWORD: pass
networks: # Networks to join (Services on the same network can communicate with each other using their name)
- turingmysql
# Database Service (Mysql)
mysql:
image: mysql/mysql-server
ports:
- "3306:3306"
environment:
MYSQL_ROOT_PASSWORD: root
MYSQL_DATABASE: tshirtshop
MYSQL_USER: turing
MYSQL_PASSWORD: pass
volumes:
- db-data:/var/lib/mysql
- ./database:/docker-entrypoint-initdb.d
tty: true
networks: # Networks to join (Services on the same network can communicate with each other using their name)
- turingmysql
# Volumes
volumes:
db-data:
# Networks to be created to facilitate communication between containers
networks:
turingmysql:
driver: bridge
Needed to specify that the network is a bridge. My sql file was mounted from a folder relative to docker-compose.yml. Also had to add allowPublicKeyRetrieval=true to my jdbc url. Created a user to access the database tshirtshop.
And here is the Dockerfile.
#### Stage 1: Build the application
FROM openjdk:8-jdk-alpine as build
# Set the current working directory inside the image
WORKDIR /app
# Copy maven executable to the image
COPY mvnw .
COPY .mvn .mvn
# Copy the pom.xml file
COPY pom.xml .
# Build all the dependencies in preparation to go offline.
# This is a separate step so the dependencies will be cached unless
# the pom.xml file has changed.
RUN ./mvnw dependency:go-offline -B
# Copy the project source
COPY src src
# Package the application
RUN ./mvnw package -DskipTests
RUN mkdir -p target/dependency && (cd target/dependency; jar -xf ../*.jar)
#### Stage 2: A minimal docker image with command to run the app
FROM openjdk:8-jre-alpine
ARG DEPENDENCY=/app/target/dependency
# Copy project dependencies from the build stage
COPY --from=build ${DEPENDENCY}/BOOT-INF/lib /app/lib
COPY --from=build ${DEPENDENCY}/META-INF /app/META-INF
COPY --from=build ${DEPENDENCY}/BOOT-INF/classes /app
ENTRYPOINT ["java","-cp","app:app/lib/*","com.turing.ecommerce.TuringApplication"]
to run, from root directory of project ./entrypoint.sh on mac and the rest is history.
What I'm trying to do is, connect from my spring-boot app to mysql database in Docker. Each in their own container.
But I must be having something wrong because I can't do it.
To keep it simple :
application-properties :
# URL for the mysql db
spring.datasource.url=jdbc:mysql://workaround-mysql:3308/workaround?serverTimezone=UTC&max_allowed_packet=15728640
# User name in mysql
spring.datasource.username=springuser
# Password for mysql
spring.datasource.password=admin
#Port at which application runs
server.port=8080
docker-compose for MySQL:
version: '3'
services:
workaround-mysql:
container_name: workaround-mysql
image: mysql
environment:
MYSQL_DATABASE: workaround
MYSQL_USER: springuser
MYSQL_PASSWORD: admin
MYSQL_ROOT_PASSWORD: admin
MYSQL_ROOT_HOST: '%'
ports:
- "3308:3306"
restart: always
So pretty simple right ? Database I start with docker-compose up:
All seems to be working fine so far.
Now that I have db started, to the application, this is its docker-compose.yml:
version: '3'
services:
workaround:
restart: always
# will build ./docker/workaround/Dockerfile
build: ./docker/workaround
working_dir: /workaround
volumes:
- ./:/workaround
- ~/.m2:/root/.m2
expose:
- "8080"
command: "mvn clean spring-boot:run"
For its Dockerfile I use Linux Alpine and Java.
FROM alpine:3.9
....add java...
RUN apk update
RUN apk add dos2unix --update-cache --repository http://dl-3.alpinelinux.org/alpine/edge/community/ --allow-untrusted
RUN apk add bash
RUN apk add maven
Super simple. Now let's start the application :
Unknown host, so let's try the IP then :
docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' workaround-mysql
# URL for the mysql db
spring.datasource.url=jdbc:mysql://172.20.0.2:3308/workaround?serverTimezone=UTC&max_allowed_packet=15728640
Now I get timeout:
As you can see I get error. What is wrong with my setup and how to fix
this? Either I have unknown host exception or Refused to connect or connection timeout.
I have tried:
Using ip of a container in my application.properties, didn't work
Different ports for MySQL and application
Different images and versions of MySQL
Having everything in one docker compose with wait
timer for database.
Minimal setup with
https://github.com/hellokoding/hellokoding-courses/tree/master/docker-examples/dockercompose-springboot-mysql-nginx
Also resulted in communication link failure, Site was accessible but I
doubt that db was connected properly.
Notes:
I run this all on one computer I use port 3308 because I have local
MySQL db at 3306.
Here is docker ps -a
#Vusal ANSWER output :
Only thing different from code in answer I did wait for database to be ready 30 seconds
command: /bin/bash -c "sleep 30;mvn clean spring-boot:run;"
Try this docker-compose.yml:
version: '3'
services:
workaround-mysql:
container_name: workaround-mysql
image: mysql
environment:
MYSQL_DATABASE: workaround
MYSQL_USER: springuser
MYSQL_PASSWORD: admin
MYSQL_ROOT_PASSWORD: admin
MYSQL_ROOT_HOST: '%'
ports:
- "3308:3306"
restart: always
workaround:
depends_on:
- workaround-mysql
restart: always
# will build ./docker/workaround/Dockerfile
build: ./docker/workaround
working_dir: /workaround
volumes:
- ./:/workaround
- ~/.m2:/root/.m2
expose:
- "8080"
command: "mvn clean spring-boot:run"
And update your application.properties to use the next JDBC connection url:
spring.datasource.url=jdbc:mysql://workaround-mysql:3306/workaround?serverTimezone=UTC&max_allowed_packet=15728640
It should work when both containers in the same docker-compose file, because docker-compose creates default network for containers, so they can resolve each other by name.
What you haven't tried so far is running both containers on the same Docker network.
First, forget about IP addressing - using it should be avoided by all means.
Second, launch both compose instances with the same Docker network.
Third, do not expose ports - inside bridge network all ports are accessible to running containers.
Create global network
docker network create foo
Modify both compose files so that they use this network instead of creating each one its own:
version: '3.5'
services:
....
networks:
default:
external: true
name: foo
Remove expose directives from compose files - inside one network all ports are exposed by default
Modify connection strings to use default 3306 port instead of 3308
Enjoy
In order for the service to connect with MySql through docker it has to be in same network, look into Docker network
But for better solution I would suggest you to write a single docker compose file for MySql and Spring boot.The reason is it will easily be linked when you do that.No need any other configuration.
version: "3"
services:
mysql-service:
image: mysql
ports:
- "3306:3306"
environment:
- MYSQL_DATABASE=db
- MYSQL_USER=root
- MYSQL_PASSWORD=pass
- MYSQL_ROOT_PASSWORD=pass
spring-service:
image: springservce:latest
ports:
- "8080:8080"
depends_on:
- mysql-service
Before you try to connect to the Docker container you should stop mysql in your computer then go to the application.properties and type:
spring.datasource.url=jdbc:mysql://localhost:3306/NAME_OF_YOUR_DB_HERE?useSSL=false&allowPublicKeyRetrieval=true
Regarding localhost, you should inspect the mysql container and pick the IP address and use it instead. most likely is 172.17.0.2. If it did not work then use localhost.
I am trying to set up a mysql docker container and execute init sql script. Unfortunately the sql script is not executed. What am I doing wrong?
version: '3.3'
services:
api:
container_name: 'api'
build: './api'
ports:
- target: 8080
published: 8888
protocol: tcp
mode: host
volumes:
- './api:/go/src/app'
depends_on:
- 'mysql'
mysql:
image: 'mysql:latest'
container_name: 'mysql'
volumes:
- ./db_data:/var/lib/mysql:rw
- ./database/init.sql:/docker-entrypoint-initdb.d/init.sql:ro
restart: always
environment:
MYSQL_USER: test
MYSQL_PASSWORD: test
MYSQL_ROOT_PASSWORD: test
MYSQL_DATABASE: test
ports:
- '3306:3306'
volumes:
db_data:
I execute file with docker-compose up -d --build
The docker-entrypoint-initdb.d folder will only be run once while the container is created (instantiated) so you actually have to do a docker-compose down -v to re-activate this for the next run.
If you want to be able to add sql files at any moment you can look here at a specialized MySql docker image... http://ivo2u.nl/o4
Update for M1 arch:
Here an almost drop-in replacement in MariaDB: http://ivo2u.nl/V1
Many containerized applications, especially stateful ones, have a way of running init scripts (like the sql scripts here) and they are supposed to run only once.
And since they are stateful, the volumes are a source of truth for the containers on whether to run the init scripts or not on container restart.
Like in your case, deleting the folder used for bind mount or using a new named volume should re-run any init scripts present.
These scripts run when you create the container, not every time you start it.
You can docker-compose up --force-recreate mysql to force those scripts to re-run.
Additionally, if you have a volume like this ./db_data:/var/lib/mysql:rw, then you also need to remove ./db_data before recreating the container.
I'm not a docker expert, but this worked for me.