How to set 'spring.datasource.url' inside a Docker Container - mysql

I created a Spring Boot application which uses a MySQL database. I use a docker-compose to launch the database.
services:
adminer:
image: adminer
restart: always
ports:
- 8888:8080
db:
image: mysql:latest
restart: always
environment:
MYSQL_ROOT_PASSWORD: 'example' # TODO: Change this
volumes:
- "./config/my.conf:/etc/mysql/conf.d/config-file.cnf"
- "./data:/var/lib/mysql:rw"
The Spring Boot Application (Backend) currently does not use Docker, I run it inside Eclipse. Before launching the Backend I have to grep the Docker Container for IPAddress:
docker inspect mysql_ex_db_1 | grep 'IPAddress'
which results something like this (this exact address changes time-to time)
"IPAddress": "",
"IPAddress": "172.21.0.2",
Then I take this value and I set spring.datasource.url inside Eclipse in the file Application.properties with it.
spring.datasource.url=jdbc:mysql://172.21.0.2:3306/employee_management_system?allowPublicKeyRetrieval=true&useSSL=false&createDatabaseIfNotExist=true
After this I can launch the Backend in Eclipse the Connection to database is there, everything works.
Now I want to move the launching of Backend from Eclipse to the same docker-compose file I use to launch the database. Therefore I built an image, and appended the docker-compose file:
version: '3.1'
services:
adminer:
image: adminer
restart: always
ports:
- 8888:8080
db:
image: mysql:latest
restart: always
environment:
MYSQL_ROOT_PASSWORD: 'example' # TODO: Change this
volumes:
- "./config/my.conf:/etc/mysql/conf.d/config-file.cnf"
- "./data:/var/lib/mysql:rw"
backend:
image: backend:latest
restart: always
ports:
- 8090:8080
In this case how can I configure the IPAddress in spring.datasource.url?
The exact IPAddress changes whenever I re-launch the mysql containers.
spring.datasource.url=jdbc:mysql://172.21.0.2:3306/employee_management_system?allowPublicKeyRetrieval=true&useSSL=false&createDatabaseIfNotExist=true
So what should I write instead of '172.21.0.2' ?
I tried localhost here but it
does not seem to work.

First of all, you can set environment variables like spring.datasource.url outside of your docker image. This allows you to dynamically set these variables according to your deployment needs (like connecting to a dev or prod database).
All docker containers running from your docker-compose file run in the same virtual network and their service names correspond to their hostnames within this network. When you want to access your database from your dockerized spring backend the hostname and port will be db:3306. You can overwrite spring.datasource.url in your docker-compose file by introducing an environment variable like:
version: '3.1'
services:
adminer:
image: adminer
restart: always
ports:
- 8888:8080
db:
image: mysql:latest
restart: always
environment:
MYSQL_ROOT_PASSWORD: 'example' # TODO: Change this
volumes:
- "./config/my.conf:/etc/mysql/conf.d/config-file.cnf"
- "./data:/var/lib/mysql:rw"
backend:
image: backend:latest
restart: always
ports:
- 8090:8080
environment:
spring.datasource.url: "jdbc:mysql://db:3306/employee_management_system?allowPublicKeyRetrieval=true&useSSL=false&createDatabaseIfNotExist=true"

In your Spring Boot app, your spring.datasource.url must be like this:
spring.datasource.url=jdbc:postgresql://${DATABASE_HOST}:5432/my_db
(here I am connecting to a postgres db). Then you set the variable in your docker compose:
...
environment:
DATABASE_HOST:container_name
you can also test outside a docker-compose, by command line like this:
docker run -it -p 8080:8080 -e "JAVA_OPTS=-Xmx128m" --network=my_network -e DATABASE_HOST=postgres_container_name --name myapp myregistry/myimage:version

Add this env variable to your backend in docker-compose:
backend:
...
environment:
- SPRING_DATASOURCE_URL=jdbc:mysql://db:3306/employee_management_system?allowPublicKeyRetrieval=true&useSSL=false&createDatabaseIfNotExist=true

Related

Can't connect to MySQL container from other container

I'm trying to connect my FASTAPI app container to a MySQL database container using the docker-compose file. In the Docker documentation it says that docker creates a default network for both containers. However, I would like to use a pre-existing network that I've created(app-net).
This is my docker-compose file:
version: '3.4'
services:
mysql:
image: mysql:latest
command: mysqld --default-authentication-plugin=mysql_native_password
container_name: mysql
restart: always
ports:
- 3307:3306
environment:
MYSQL_ROOT_PASSWORD: password
volumes:
- mysql-data:/var/lib/mysql
app:
build: .
image: app:1.0
container_name: app
ports:
- 8000:8000
environment:
PASSWORD: password
USERNAME: root
networks:
default:
external: true
name: app-net
volumes:
mysql-data:
driver: local
this is the output I get when i run docker inspect mysql -f "{{json .NetworkSettings.Networks }}":
{"app-net":{"IPAMConfig":null,"Links":null,"Aliases":["mysql","5e998f9fb646"],"NetworkID":"7f60c83e4c88d25e674461521446ec9fa98baca8639c782c79671c4fcb77ba88","EndpointID":"","Gateway":"","IPAddress":"","IPPrefixLen":0,"IPv6Gateway":"","GlobalIPv6Address":"","GlobalIPv6PrefixLen":0,"MacAddress":"","DriverOpts":null}}
However, when I run each container individually using CMD with the --network app-net the output is different:
{"app-net":{"IPAMConfig":null,"Links":null,"Aliases":["46157e588c87"],"NetworkID":"7f60c83e4c88d25e674461521446ec9fa98baca8639c782c79671c4fcb77ba88","EndpointID":"6a6922a9a6ea8f9d113447decbbb927cb93ddffd3b9563ee882fa2e44970cde5","Gateway":"172.20.0.1","IPAddress":"172.20.0.2","IPPrefixLen":16,"IPv6Gateway":"","GlobalIPv6Address":"","GlobalIPv6PrefixLen":0,"MacAddress":"02:42:ac:14:00:02","DriverOpts":null}}
In my app code in order to connect the mysql server, I specified the container name as the hostname since they are supposed to share the same network. But, as I mentioned it seems both containers can't talk to each other.
I'm pretty sure that is the reason I can't connect the database through my app and get that error when I run:
docker-compose -f docker-compose.yml up
I get this error:
sqlalchemy.exc.OperationalError: (_mysql_exceptions.OperationalError) (2005, "Unknown MySQL server host 'mysql' (0)")
What am I missing?
If the error you're getting is specifically "unknown host" or something similar, this can happen if your application container starts before the database container exists. You can work around this situation by telling Compose about the dependency:
version: '3.8'
services:
mysql: { ... }
app:
depends_on:
- mysql
This has two key effects. If you try to start only the application container docker-compose up app, it will also start the mysql container, following the depends_on: chain. When Compose does start containers, it at least creates the mysql container before creating the app container.
Note that this doesn't guarantee the database will actually be running by the time your application starts up. If you do encounter this, you will get a different error like "connection refused". You can work around this by embedding a script in your image that waits for the database to be available, or by using a version 2 Compose file with health-check support; see Docker Compose wait for container X before starting Y for more details on this specific problem.
You could add the key networks to both containers, this way:
version: '3.4'
services:
mysql:
image: mysql:latest
command: mysqld --default-authentication-plugin=mysql_native_password
container_name: mysql
restart: always
ports:
- 3307:3306
environment:
MYSQL_ROOT_PASSWORD: password
volumes:
- mysql-data:/var/lib/mysql
networks:
- app-net
app:
build: .
image: app:1.0
container_name: app
ports:
- 8000:8000
environment:
PASSWORD: password
USERNAME: root
networks:
- app-net
networks:
default:
external: true
name: app-net
volumes:
mysql-data:
driver: local

SQLSTATE[HY000] [2002] Connection refused (SQL: select * from information_schema.tables where table_schema = ms_api_shop

I am using Lumen with Docker to create simple API for authentication. After installing LumenPassport, I cannot migrate the database. I can easily connect to the MySQL db with Dbeaver.
I have already created one Lumen Docker project for the same purpose, it is the second. The first one worked without a problem. Moreover, I have checked the MySQL databases, ms_api_shop was there
Errors:
Here is my docker-compose
services:
nginx:
build:
context: .
dockerfile: docker/Nginx.Dockerfile
image: nginx
ports:
- 8092:80
depends_on:
- fpm
volumes:
- ./:/var/www/lumen-docker
links:
- mysql
fpm:
build:
context: .
dockerfile: docker/fpm.Dockerfile
volumes:
- ./:/var/www/lumen-docker
depends_on:
- mysql
links:
- mysql
mysql:
image: mysql:5.7
ports:
- 33006:3306
environment:
- MYSQL_ROOT_PASSWORD=
- MYSQL_DATABASE=ms_api_shop
- MYSQL_ROOT_USER=
volumes:
- mysql-data:/var/lib/mysql
volumes:
mysql-data:
And env:
DB_HOST=mysql
DB_PORT=33006
DB_DATABASE=ms_api_shop
DB_USERNAME=
DB_PASSWORD=
In your docker-file, you are binding the 33006 container port to the 3306 of the host port. In case you want to access the MySQL, you should use 3306 not 33006 as you did in your .env
I have a Laravel app running in Docker and a part of my docker config looks like:
But I personally feel that they should be within a docker network, look at the last two lines of the code in my docker-compose.yml below:
mysql:
image: mysql:5.7.29
container_name: mysql
restart: unless-stopped
tty: true
ports:
- "3306:3306"
environment:
MYSQL_DATABASE: homestead
MYSQL_USER: homestead
MYSQL_PASSWORD: secret
MYSQL_ROOT_PASSWORD: secret
SERVICE_TAGS: dev
SERVICE_NAME: mysql
volumes:
- ./mysql:/var/lib/mysql
networks:
- laravel
According to my knowledge, docker containers can communicate with each other when they are on the same network. You seem to have not connected these two docker in docker-compose yet. To get network information from a container, you can use the below command.
$ docker inspect --format='{{json .NetworkSettings.Networks}}' <docker_container_name>
In case you need to connect these two containers, follow these steps:
First, you create a network
$ docker network create my-net
Second, you connect container to the network.
$ docker network connect my-net <docker_container_name>
Don't forget to connect these two containers to the network.

Spring Boot + MySQL + Docker Compose - Cannot make Spring Boot connect to MySQL

I've been trying to set up a connection between a backend (runs on Spring Boot) container and a pre-built MySQL container. However, I cannot get it to connect. My docker compose file is:
version: '3.7'
services:
test-mysql:
image: mysql
restart: always
volumes:
- db_data:/var/lib/mysql
environment:
MYSQL_DATABASE: testdb
MYSQL_USER: test
MYSQL_PASSWORD: test
MYSQL_ROOT_PASSWORD: root
backend:
depends_on:
- test-mysql
build:
context: backend
dockerfile: Dockerfile
ports:
- "8080:8080"
restart: always
volumes:
db_data: {}
my application.properties:
spring.jpa.hibernate.ddl-auto=update
spring.jpa.properties.hibernate.dialect = org.hibernate.dialect.MySQL5InnoDBDialect
spring.datasource.url=jdbc:mysql://test-mysql:3306/testdb?autoReconnect=true&failOverReadOnly=false&maxReconnects=10
spring.datasource.driver-class-name=com.mysql.cj.jdbc.Driver
spring.datasource.username=test
spring.datasource.password=test
When I use docker-compose up, Spring Boot is not able to recognize the container name test-mysql. It throws: java.net.UnknownHostException
When I change it to an IP, it says connection refused. I have been looking everywhere and couldn't come with a fix. I hope anyone can help me out. Thank you!
You have to mention the backend mysql properties in the composer file like below,
backend:
depends_on:
- test-mysql
build:
context: backend
dockerfile: Dockerfile
ports:
- "8080:8080"
restart: always
environment:
SPRING_DATASOURCE_URL: jdbc:mysql://test-
mysql:3306/testdbautoReconnect=true&failOverReadOnly=false&maxReconnects=10
SPRING_DATASOURCE_USERNAME: test
SPRING_DATASOURCE_PASSWORD: test
links:
- test-mysql:test-mysql
If this wouldn't work try to create a common docker network and add it to your composer file like below,
backend:
depends_on:
- test-mysql
build:
context: backend
dockerfile: Dockerfile
ports:
- "8080:8080"
restart: always
environment:
SPRING_DATASOURCE_URL: jdbc:mysql://test-
mysql:3306/testdbautoReconnect=true&failOverReadOnly=false&maxReconnects=10
SPRING_DATASOURCE_USERNAME: test
SPRING_DATASOURCE_PASSWORD: test
networks:
-common-network
test-mysql:
image: mysql
restart: always
volumes:
- db_data:/var/lib/mysql
environment:
MYSQL_DATABASE: testdb
MYSQL_USER: test
MYSQL_PASSWORD: test
MYSQL_ROOT_PASSWORD: root
networks:
-common-network
#Docker Networks
networks:
common-network:
driver: bridge
#Volumes
volumes:
dbdata:
driver: local
You can define a common network on which both the application server and the database can connect. Please check the file (docker-compose.yml) below where I have defined a common network: backend
# Docker Compose file Reference (https://docs.docker.com/compose/compose-file/)
version: '3.7'
# Define services
services:
# App backend service
app-server:
# Configuration for building the docker image for the backend service
build:
context: . # Use an image built from the specified dockerfile in the `springboot-app-server` directory.
dockerfile: Dockerfile
ports:
- "8080:8080" # Forward the exposed port 4000 on the container to port 4000 on the host machine
restart: always
depends_on:
- db # This service depends on mysql. Start that first.
environment: # Pass environment variables to the service
SPRING_DATASOURCE_URL: jdbc:mysql://test-mysql:3306/testdb?useSSL=false&serverTimezone=UTC&useLegacyDatetimeCode=false
SPRING_DATASOURCE_USERNAME: test
SPRING_DATASOURCE_PASSWORD: test
networks: # Networks to join (Services on the same network can communicate with each other using their name)
- backend
# Database Service (Mysql)
db:
image: mysql:5.7
ports:
- "3306:3306"
restart: always
environment:
MYSQL_DATABASE: testdb
MYSQL_USER: test
MYSQL_PASSWORD: test
MYSQL_ROOT_PASSWORD: root
volumes:
- db-data:/var/lib/mysql
networks:
- backend
# Volumes
volumes:
db-data:
# Networks to be created to facilitate communication between containers
networks:
backend:
I have written a blog and a simple working Spring Boot MySQL application on GitHub which tells about using Docker Compose. Please check: http://softwaredevelopercentral.blogspot.com/2020/10/spring-boot-mysql-docker-compose-example.html
If you would like to use this test-mysql in your spring config
spring.datasource.url=jdbc:mysql://test-mysql:3306/testdb?autoReconnect=true&failOverReadOnly=false&maxReconnects=10
Then add the hostname attribute at service test-mysql
version: '3.7'
services:
test-mysql:
image: mysql
hostname: test-mysql
...
I hope this has been already solved but in case it hasn't yet, the problem lies in mysql docker container lagging behind the start up.
Another problem is that you might need to build the jar file and then copy it the container. That's a big problem because when you build the jar file, the database with db as hostname is unavailable. So when you are building the jar file, skip the test.
This is bash script i created but you can run command one by one:
#!/bin/bash
cd storage-service
rm -rf target/
mvn clean compile package -Dmaven.test.skip=true
cd ..
docker-compose up
In case you want to initialize db in the container. That file is in the folder env where i have a file database.env
-- create the databases
CREATE DATABASE IF NOT EXISTS model_storage;
-- create the users for each database
CREATE USER 'arsene'#'localhost' IDENTIFIED BY 'arsene';
GRANT CREATE, ALTER, INDEX, LOCK TABLES, REFERENCES, UPDATE, DELETE, DROP, SELECT, INSERT ON `model_storage`.* TO 'arsene'#'localhost';
FLUSH PRIVILEGES;
The backend service Dockerfile looks like this:
FROM adoptopenjdk/openjdk11
COPY target/*.jar storage.jar
ENV JAVA_OPTS=""
ENTRYPOINT [ "sh", "-c", "java $JAVA_OPTS -Djava.security.egd=file:/dev/./urandom -jar /storage.jar" ]
EXPOSE 8089
The database env file looks like this:
MYSQL_ROOT_PASSWORD=arsene
MYSQL_DATABASE=model_storage
MYSQL_USER=arsene
MYSQL_PASSWORD=arsene
DATABASE_HOST=model_storage
DATABASE_USER=arsene
DATABASE_PASSWORD=arsene
DATABASE_NAME=model_storage
DATABASE_PORT=3306
In case you intend to pass JAVA_OPTS env in the image. These can be used later as seen in docker-compose.yml below
Your backend (the service that depends on the mysql db) needs to restart until the docker-compose is able to resolve the the container name of mysql, in my case its name is db. And don't forget to include datasource connection properties in docker-compose backend service image as i did below. I am not an expert in spring boot and neither in docker but for now it works!
Below is the way mine is structured:
I am using docker version: "3.8"
Storage service
storage-service:
container_name: storage-service
restart: always
build:
context: storage-service
image: "service_storage_image"
depends_on:
- db
ports:
- "8089:8089"
links:
- db
env_file:
- env/database.env
environment:
WAIT_HOSTS: db:3306
SPRING_DATASOURCE_URL: jdbc:mysql://db:3306/model_storage?allowPublicKeyRetrieval=true&useSSL=false
SPRING_DATASOURCE_USERNAME: root
SPRING_DATASOURCE_PASSWORD: arsene
healthcheck:
test: "/usr/bin/mysql --user=arsene --password=arsene--execute \"SHOW DATABASES;\""
interval: 2s
timeout: 20s
retries: 10
environment:
- JAVA_OPTS=
-DEUREKA_SERVER=http://eureka-registry-server:7070/eureka
-DZIPKIN_SERVER=http://zipkin:9411/
networks:
- private-network-mms
My db in docker-compose is structured this way:
Mysql database
db:
hostname: db
container_name: db
image: "mysql:latest"
env_file:
- env/database.env
volumes:
- type: bind
source: ./env/setup.sql
target: /docker-entrypoint-initdb.d/setup.sql
- db_volume:/var/lib/mysql
ports:
- 3307:3306
networks:
- private-network-mms

Docker-compose MySQL link failure

EDIT: I've managed to get it to work
I'm currently using docker in combination with a Dockerfile and a docker-compose.yml i'm trying to run my backend so i can use Postman to get data from the database. However I keep getting Communication Link Failure
com.mysql.cj.jdbc.exceptions.CommunicationsException: Communications link failure
Dockerfile
FROM openjdk:8-jdk-alpine
EXPOSE 8080
ADD /build/libs/assignment_4-0.0.1-SNAPSHOT.jar spring-docker.jar
ENTRYPOINT ["java", "-jar", "spring-docker.jar"]
docker-compose.yml
version: '3'
services:
docker-mysql:
restart: always
container_name: docker-mysql
image: mysql:5.7
environment:
MYSQL_DATABASE: database
MYSQL_ROOT_PASSWORD: root
MYSQL_ROOT_HOST: '%'
volumes:
- ./sql:/docker-entrypoint-initdb.d
ports:
- "3306:3306"
app:
image: springio/docker
expose:
- "8080"
ports:
- 8080:8080
environment:
WAIT_HOSTS: mysql:3306
depends_on:
- docker-mysql
application.properties
spring.jpa.hibernate.ddl-auto=none
spring.datasource.url=jdbc:mysql://docker-mysql:3306/database
spring.datasource.username=root
spring.datasource.password=root
# To keep the database connection alive while idle for a long time
spring.datasource.testWhileIdle = true
spring.datasource.validationQuery = SELECT 1
The API is made with SpringBoot using Kotlin. Can someone help me figure out how to resolve this problem?
To achieve the correct order of services to start you need to add 2 things:
Set container's names to let them ping each other by these names (by means of docker bridge)
Enforce your app to wait for DB fully up before connect.
From point of this your Dockerfile could look like this (let we choose wait-for-it)
FROM openjdk:8-jdk-alpine
EXPOSE 8080
RUN apk add --no-cache bash
RUN wget -q https://github.com/vishnubob/wait-for-it/raw/master/wait-for-it.sh -O /usr/bin/wait-for-it && \
chmod +x /usr/bin/wait-for-it
ADD /build/libs/assignment_4-0.0.1-SNAPSHOT.jar spring-docker.jar
ENTRYPOINT /usr/bin/wait-for-it docker-mysql-container:3306 -t 120 ; java -jar spring-docker.jar
While docker-compose.yml should include
version: '3'
networks:
database:
services:
docker-mysql:
networks:
- database
container_name: docker-mysql-container
...
app:
networks:
- database
container_name: app-container
...
And replace in your application.properties service's name docker-mysql with container's name docker-mysql-container

Docker-compose up for a spring service and a mysql service works, but init.sql does is not loaded and Postman return empty data

I am working for the first time with Docker. In my job, they asked me to deploy a complete application using docker-compose and I am learning right now.
I have a docker-compose.yml that starts two services: one for spring and another one for MySQL.
The one for Spring uses a Dockerfile to build the image.
The one for MySQL uses the official MySQL image.
I don´t know how to load the init.sql to initialize and load data into the database of the MySQL container.
I´ve tried to use another Dockerfile in order to copy init.sql into the MySQL container but,
how do I tell the compose to use it when creating the image or running the container?
This is for a Windows 10 OS Desktop. Docker version 18.09.2. Docker-compose version 1.23.2
Docker Desktop
Docker-compose.yml
version: '3.7'
services:
mysql-docker-container:
image: mysql
container_name: mysql-docker-container
environment:
- MYSQL_ROOT_PASSWORD=root
- MYSQL_DATABASE=database
- MYSQL_USER=user
- MYSQL_PASSWORD=user
ports:
- 2012:3306
tty: true
spring-jpa-app:
build:
context: .
dockerfile: Dockerfile
container_name: spring-jpa-app
links:
- mysql-docker-container:mysql-docker-container
depends_on:
- mysql-docker-container
ports:
- 8087:8080
tty: true
restart: always
Dockerfile
FROM java:8
VOLUME /tmp
EXPOSE 8080
ADD app-1.0.0.jar app-1.0.0.jar
ENTRYPOINT ["java","-jar","app-1.0.0.jar"]
application.properties
spring.datasource.driver-class-name=com.mysql.jdbc.Driver
spring.datasource.url=jdbc:mysql://mysql-docker-container:2012/database?autoReconnect=true&useSSL=false
spring.datasource.username=user
spring.datasource.password=user
spring.jpa.database=database
spring.jpa.hibernate.ddl-auto=create-drop
spring.jpa.properties.hibernate.dialect=org.hibernate.dialect.MySQL5Dialect
logging.file=log/log.log
logging.level.org.springframework=info
logging.level.orghibernate=info
Docker Desktop running
Opened Git Bash as administrator
docker-machine ls
NAME ACTIVE DRIVER STATE URL DOCKER
app * hyperv Running tcp://172.18.67.35:2376 v19.03.0
In the app folder, where is located /src
mvn clean install -DskipTests
BUILD SUCCESS
In the app folder where is located application.properties, app.jar, docker-compose.yml, Dockerfile and init.sql
docker-compose up --build
Postman
URL = http://172.18.67.35:8087
POST {{url}}/login
{
"Authorization": "gnioengrlnkwejnR",
"username": "Administrator",
"role": "ADMIN",
"id": "1"
}
GET {{url}}/listAllCenters
[]
Returns empty because init.sql is not being loaded
PD: Also, if I open a new git Bash as Administrator and do
docker ps -a
I do not get the containers running so I cannot know container's ID to enter the MySQL container.
you need to add a volume to copy the sql.init to your db continer:
version: '3.7'
services:
mysql-docker-container:
image: mysql
container_name: mysql-docker-container
environment:
- MYSQL_ROOT_PASSWORD=root
- MYSQL_DATABASE=database
- MYSQL_USER=user
- MYSQL_PASSWORD=user
ports:
- 2012:3306
volumes:
- /bath/to/file/on/host/init.sql:/docker-entrypoint-initdb.d/init.sql
tty: true
network_mode: "host"
spring-jpa-app:
build:
context: .
dockerfile: Dockerfile
container_name: spring-jpa-app
links:
- mysql-docker-container:mysql-docker-container
depends_on:
- mysql-docker-container
ports:
- 8087:8080
tty: true
restart: always
network_mode: "host"
see this and search for "Initializing a fresh instance"
on the other hand you need to use network to setup the connection between the containers - see the compose file above-
and you need to change you configs to
spring.datasource.url=jdbc:mysql://localhost:2012/database?autoReconnect=true&useSSL=false