Spring Boot + MySQL + Docker Compose - Cannot make Spring Boot connect to MySQL - mysql

I've been trying to set up a connection between a backend (runs on Spring Boot) container and a pre-built MySQL container. However, I cannot get it to connect. My docker compose file is:
version: '3.7'
services:
test-mysql:
image: mysql
restart: always
volumes:
- db_data:/var/lib/mysql
environment:
MYSQL_DATABASE: testdb
MYSQL_USER: test
MYSQL_PASSWORD: test
MYSQL_ROOT_PASSWORD: root
backend:
depends_on:
- test-mysql
build:
context: backend
dockerfile: Dockerfile
ports:
- "8080:8080"
restart: always
volumes:
db_data: {}
my application.properties:
spring.jpa.hibernate.ddl-auto=update
spring.jpa.properties.hibernate.dialect = org.hibernate.dialect.MySQL5InnoDBDialect
spring.datasource.url=jdbc:mysql://test-mysql:3306/testdb?autoReconnect=true&failOverReadOnly=false&maxReconnects=10
spring.datasource.driver-class-name=com.mysql.cj.jdbc.Driver
spring.datasource.username=test
spring.datasource.password=test
When I use docker-compose up, Spring Boot is not able to recognize the container name test-mysql. It throws: java.net.UnknownHostException
When I change it to an IP, it says connection refused. I have been looking everywhere and couldn't come with a fix. I hope anyone can help me out. Thank you!

You have to mention the backend mysql properties in the composer file like below,
backend:
depends_on:
- test-mysql
build:
context: backend
dockerfile: Dockerfile
ports:
- "8080:8080"
restart: always
environment:
SPRING_DATASOURCE_URL: jdbc:mysql://test-
mysql:3306/testdbautoReconnect=true&failOverReadOnly=false&maxReconnects=10
SPRING_DATASOURCE_USERNAME: test
SPRING_DATASOURCE_PASSWORD: test
links:
- test-mysql:test-mysql
If this wouldn't work try to create a common docker network and add it to your composer file like below,
backend:
depends_on:
- test-mysql
build:
context: backend
dockerfile: Dockerfile
ports:
- "8080:8080"
restart: always
environment:
SPRING_DATASOURCE_URL: jdbc:mysql://test-
mysql:3306/testdbautoReconnect=true&failOverReadOnly=false&maxReconnects=10
SPRING_DATASOURCE_USERNAME: test
SPRING_DATASOURCE_PASSWORD: test
networks:
-common-network
test-mysql:
image: mysql
restart: always
volumes:
- db_data:/var/lib/mysql
environment:
MYSQL_DATABASE: testdb
MYSQL_USER: test
MYSQL_PASSWORD: test
MYSQL_ROOT_PASSWORD: root
networks:
-common-network
#Docker Networks
networks:
common-network:
driver: bridge
#Volumes
volumes:
dbdata:
driver: local

You can define a common network on which both the application server and the database can connect. Please check the file (docker-compose.yml) below where I have defined a common network: backend
# Docker Compose file Reference (https://docs.docker.com/compose/compose-file/)
version: '3.7'
# Define services
services:
# App backend service
app-server:
# Configuration for building the docker image for the backend service
build:
context: . # Use an image built from the specified dockerfile in the `springboot-app-server` directory.
dockerfile: Dockerfile
ports:
- "8080:8080" # Forward the exposed port 4000 on the container to port 4000 on the host machine
restart: always
depends_on:
- db # This service depends on mysql. Start that first.
environment: # Pass environment variables to the service
SPRING_DATASOURCE_URL: jdbc:mysql://test-mysql:3306/testdb?useSSL=false&serverTimezone=UTC&useLegacyDatetimeCode=false
SPRING_DATASOURCE_USERNAME: test
SPRING_DATASOURCE_PASSWORD: test
networks: # Networks to join (Services on the same network can communicate with each other using their name)
- backend
# Database Service (Mysql)
db:
image: mysql:5.7
ports:
- "3306:3306"
restart: always
environment:
MYSQL_DATABASE: testdb
MYSQL_USER: test
MYSQL_PASSWORD: test
MYSQL_ROOT_PASSWORD: root
volumes:
- db-data:/var/lib/mysql
networks:
- backend
# Volumes
volumes:
db-data:
# Networks to be created to facilitate communication between containers
networks:
backend:
I have written a blog and a simple working Spring Boot MySQL application on GitHub which tells about using Docker Compose. Please check: http://softwaredevelopercentral.blogspot.com/2020/10/spring-boot-mysql-docker-compose-example.html

If you would like to use this test-mysql in your spring config
spring.datasource.url=jdbc:mysql://test-mysql:3306/testdb?autoReconnect=true&failOverReadOnly=false&maxReconnects=10
Then add the hostname attribute at service test-mysql
version: '3.7'
services:
test-mysql:
image: mysql
hostname: test-mysql
...

I hope this has been already solved but in case it hasn't yet, the problem lies in mysql docker container lagging behind the start up.
Another problem is that you might need to build the jar file and then copy it the container. That's a big problem because when you build the jar file, the database with db as hostname is unavailable. So when you are building the jar file, skip the test.
This is bash script i created but you can run command one by one:
#!/bin/bash
cd storage-service
rm -rf target/
mvn clean compile package -Dmaven.test.skip=true
cd ..
docker-compose up
In case you want to initialize db in the container. That file is in the folder env where i have a file database.env
-- create the databases
CREATE DATABASE IF NOT EXISTS model_storage;
-- create the users for each database
CREATE USER 'arsene'#'localhost' IDENTIFIED BY 'arsene';
GRANT CREATE, ALTER, INDEX, LOCK TABLES, REFERENCES, UPDATE, DELETE, DROP, SELECT, INSERT ON `model_storage`.* TO 'arsene'#'localhost';
FLUSH PRIVILEGES;
The backend service Dockerfile looks like this:
FROM adoptopenjdk/openjdk11
COPY target/*.jar storage.jar
ENV JAVA_OPTS=""
ENTRYPOINT [ "sh", "-c", "java $JAVA_OPTS -Djava.security.egd=file:/dev/./urandom -jar /storage.jar" ]
EXPOSE 8089
The database env file looks like this:
MYSQL_ROOT_PASSWORD=arsene
MYSQL_DATABASE=model_storage
MYSQL_USER=arsene
MYSQL_PASSWORD=arsene
DATABASE_HOST=model_storage
DATABASE_USER=arsene
DATABASE_PASSWORD=arsene
DATABASE_NAME=model_storage
DATABASE_PORT=3306
In case you intend to pass JAVA_OPTS env in the image. These can be used later as seen in docker-compose.yml below
Your backend (the service that depends on the mysql db) needs to restart until the docker-compose is able to resolve the the container name of mysql, in my case its name is db. And don't forget to include datasource connection properties in docker-compose backend service image as i did below. I am not an expert in spring boot and neither in docker but for now it works!
Below is the way mine is structured:
I am using docker version: "3.8"
Storage service
storage-service:
container_name: storage-service
restart: always
build:
context: storage-service
image: "service_storage_image"
depends_on:
- db
ports:
- "8089:8089"
links:
- db
env_file:
- env/database.env
environment:
WAIT_HOSTS: db:3306
SPRING_DATASOURCE_URL: jdbc:mysql://db:3306/model_storage?allowPublicKeyRetrieval=true&useSSL=false
SPRING_DATASOURCE_USERNAME: root
SPRING_DATASOURCE_PASSWORD: arsene
healthcheck:
test: "/usr/bin/mysql --user=arsene --password=arsene--execute \"SHOW DATABASES;\""
interval: 2s
timeout: 20s
retries: 10
environment:
- JAVA_OPTS=
-DEUREKA_SERVER=http://eureka-registry-server:7070/eureka
-DZIPKIN_SERVER=http://zipkin:9411/
networks:
- private-network-mms
My db in docker-compose is structured this way:
Mysql database
db:
hostname: db
container_name: db
image: "mysql:latest"
env_file:
- env/database.env
volumes:
- type: bind
source: ./env/setup.sql
target: /docker-entrypoint-initdb.d/setup.sql
- db_volume:/var/lib/mysql
ports:
- 3307:3306
networks:
- private-network-mms

Related

How to set 'spring.datasource.url' inside a Docker Container

I created a Spring Boot application which uses a MySQL database. I use a docker-compose to launch the database.
services:
adminer:
image: adminer
restart: always
ports:
- 8888:8080
db:
image: mysql:latest
restart: always
environment:
MYSQL_ROOT_PASSWORD: 'example' # TODO: Change this
volumes:
- "./config/my.conf:/etc/mysql/conf.d/config-file.cnf"
- "./data:/var/lib/mysql:rw"
The Spring Boot Application (Backend) currently does not use Docker, I run it inside Eclipse. Before launching the Backend I have to grep the Docker Container for IPAddress:
docker inspect mysql_ex_db_1 | grep 'IPAddress'
which results something like this (this exact address changes time-to time)
"IPAddress": "",
"IPAddress": "172.21.0.2",
Then I take this value and I set spring.datasource.url inside Eclipse in the file Application.properties with it.
spring.datasource.url=jdbc:mysql://172.21.0.2:3306/employee_management_system?allowPublicKeyRetrieval=true&useSSL=false&createDatabaseIfNotExist=true
After this I can launch the Backend in Eclipse the Connection to database is there, everything works.
Now I want to move the launching of Backend from Eclipse to the same docker-compose file I use to launch the database. Therefore I built an image, and appended the docker-compose file:
version: '3.1'
services:
adminer:
image: adminer
restart: always
ports:
- 8888:8080
db:
image: mysql:latest
restart: always
environment:
MYSQL_ROOT_PASSWORD: 'example' # TODO: Change this
volumes:
- "./config/my.conf:/etc/mysql/conf.d/config-file.cnf"
- "./data:/var/lib/mysql:rw"
backend:
image: backend:latest
restart: always
ports:
- 8090:8080
In this case how can I configure the IPAddress in spring.datasource.url?
The exact IPAddress changes whenever I re-launch the mysql containers.
spring.datasource.url=jdbc:mysql://172.21.0.2:3306/employee_management_system?allowPublicKeyRetrieval=true&useSSL=false&createDatabaseIfNotExist=true
So what should I write instead of '172.21.0.2' ?
I tried localhost here but it
does not seem to work.
First of all, you can set environment variables like spring.datasource.url outside of your docker image. This allows you to dynamically set these variables according to your deployment needs (like connecting to a dev or prod database).
All docker containers running from your docker-compose file run in the same virtual network and their service names correspond to their hostnames within this network. When you want to access your database from your dockerized spring backend the hostname and port will be db:3306. You can overwrite spring.datasource.url in your docker-compose file by introducing an environment variable like:
version: '3.1'
services:
adminer:
image: adminer
restart: always
ports:
- 8888:8080
db:
image: mysql:latest
restart: always
environment:
MYSQL_ROOT_PASSWORD: 'example' # TODO: Change this
volumes:
- "./config/my.conf:/etc/mysql/conf.d/config-file.cnf"
- "./data:/var/lib/mysql:rw"
backend:
image: backend:latest
restart: always
ports:
- 8090:8080
environment:
spring.datasource.url: "jdbc:mysql://db:3306/employee_management_system?allowPublicKeyRetrieval=true&useSSL=false&createDatabaseIfNotExist=true"
In your Spring Boot app, your spring.datasource.url must be like this:
spring.datasource.url=jdbc:postgresql://${DATABASE_HOST}:5432/my_db
(here I am connecting to a postgres db). Then you set the variable in your docker compose:
...
environment:
DATABASE_HOST:container_name
you can also test outside a docker-compose, by command line like this:
docker run -it -p 8080:8080 -e "JAVA_OPTS=-Xmx128m" --network=my_network -e DATABASE_HOST=postgres_container_name --name myapp myregistry/myimage:version
Add this env variable to your backend in docker-compose:
backend:
...
environment:
- SPRING_DATASOURCE_URL=jdbc:mysql://db:3306/employee_management_system?allowPublicKeyRetrieval=true&useSSL=false&createDatabaseIfNotExist=true

SQLSTATE[HY000] [2002] Connection refused (SQL: select * from information_schema.tables where table_schema = ms_api_shop

I am using Lumen with Docker to create simple API for authentication. After installing LumenPassport, I cannot migrate the database. I can easily connect to the MySQL db with Dbeaver.
I have already created one Lumen Docker project for the same purpose, it is the second. The first one worked without a problem. Moreover, I have checked the MySQL databases, ms_api_shop was there
Errors:
Here is my docker-compose
services:
nginx:
build:
context: .
dockerfile: docker/Nginx.Dockerfile
image: nginx
ports:
- 8092:80
depends_on:
- fpm
volumes:
- ./:/var/www/lumen-docker
links:
- mysql
fpm:
build:
context: .
dockerfile: docker/fpm.Dockerfile
volumes:
- ./:/var/www/lumen-docker
depends_on:
- mysql
links:
- mysql
mysql:
image: mysql:5.7
ports:
- 33006:3306
environment:
- MYSQL_ROOT_PASSWORD=
- MYSQL_DATABASE=ms_api_shop
- MYSQL_ROOT_USER=
volumes:
- mysql-data:/var/lib/mysql
volumes:
mysql-data:
And env:
DB_HOST=mysql
DB_PORT=33006
DB_DATABASE=ms_api_shop
DB_USERNAME=
DB_PASSWORD=
In your docker-file, you are binding the 33006 container port to the 3306 of the host port. In case you want to access the MySQL, you should use 3306 not 33006 as you did in your .env
I have a Laravel app running in Docker and a part of my docker config looks like:
But I personally feel that they should be within a docker network, look at the last two lines of the code in my docker-compose.yml below:
mysql:
image: mysql:5.7.29
container_name: mysql
restart: unless-stopped
tty: true
ports:
- "3306:3306"
environment:
MYSQL_DATABASE: homestead
MYSQL_USER: homestead
MYSQL_PASSWORD: secret
MYSQL_ROOT_PASSWORD: secret
SERVICE_TAGS: dev
SERVICE_NAME: mysql
volumes:
- ./mysql:/var/lib/mysql
networks:
- laravel
According to my knowledge, docker containers can communicate with each other when they are on the same network. You seem to have not connected these two docker in docker-compose yet. To get network information from a container, you can use the below command.
$ docker inspect --format='{{json .NetworkSettings.Networks}}' <docker_container_name>
In case you need to connect these two containers, follow these steps:
First, you create a network
$ docker network create my-net
Second, you connect container to the network.
$ docker network connect my-net <docker_container_name>
Don't forget to connect these two containers to the network.

Docker-compose MySQL link failure

EDIT: I've managed to get it to work
I'm currently using docker in combination with a Dockerfile and a docker-compose.yml i'm trying to run my backend so i can use Postman to get data from the database. However I keep getting Communication Link Failure
com.mysql.cj.jdbc.exceptions.CommunicationsException: Communications link failure
Dockerfile
FROM openjdk:8-jdk-alpine
EXPOSE 8080
ADD /build/libs/assignment_4-0.0.1-SNAPSHOT.jar spring-docker.jar
ENTRYPOINT ["java", "-jar", "spring-docker.jar"]
docker-compose.yml
version: '3'
services:
docker-mysql:
restart: always
container_name: docker-mysql
image: mysql:5.7
environment:
MYSQL_DATABASE: database
MYSQL_ROOT_PASSWORD: root
MYSQL_ROOT_HOST: '%'
volumes:
- ./sql:/docker-entrypoint-initdb.d
ports:
- "3306:3306"
app:
image: springio/docker
expose:
- "8080"
ports:
- 8080:8080
environment:
WAIT_HOSTS: mysql:3306
depends_on:
- docker-mysql
application.properties
spring.jpa.hibernate.ddl-auto=none
spring.datasource.url=jdbc:mysql://docker-mysql:3306/database
spring.datasource.username=root
spring.datasource.password=root
# To keep the database connection alive while idle for a long time
spring.datasource.testWhileIdle = true
spring.datasource.validationQuery = SELECT 1
The API is made with SpringBoot using Kotlin. Can someone help me figure out how to resolve this problem?
To achieve the correct order of services to start you need to add 2 things:
Set container's names to let them ping each other by these names (by means of docker bridge)
Enforce your app to wait for DB fully up before connect.
From point of this your Dockerfile could look like this (let we choose wait-for-it)
FROM openjdk:8-jdk-alpine
EXPOSE 8080
RUN apk add --no-cache bash
RUN wget -q https://github.com/vishnubob/wait-for-it/raw/master/wait-for-it.sh -O /usr/bin/wait-for-it && \
chmod +x /usr/bin/wait-for-it
ADD /build/libs/assignment_4-0.0.1-SNAPSHOT.jar spring-docker.jar
ENTRYPOINT /usr/bin/wait-for-it docker-mysql-container:3306 -t 120 ; java -jar spring-docker.jar
While docker-compose.yml should include
version: '3'
networks:
database:
services:
docker-mysql:
networks:
- database
container_name: docker-mysql-container
...
app:
networks:
- database
container_name: app-container
...
And replace in your application.properties service's name docker-mysql with container's name docker-mysql-container

docker compose spring boot logs

I'm trying to run both a spring boot app and mysql in separate docker containers and I'm having trouble debugging issues because I can't see any logs. When I run docker-compose up I see the start up logs (Spring Boot banner) and see the app start, but after that no more logging. I'm getting a 404 hitting one of my end points but I can't debug it without seeing the logs.
docker-compose.yml:
version: "3.3"
services:
database:
build:
context: ./database
image: pensionator_db
# set default mysql root password, change as needed
environment:
MYSQL_USER: pensionatoruser
MYSQL_DATABASE: pensionatordb
# Expose port 3306 to host. Not for the application but
# handy to inspect the database from the host machine.
ports:
- "3306:3306"
restart: always
appserver:
build:
context: .
dockerfile: app/src/main/docker/Dockerfile
image: pensionator_app
# mount point for application in tomcat
# open ports for tomcat and remote debugging
ports:
- "8080:8080"
- "8000:8000"
restart: always
How do I get logging to work?
There was nothing wrong with the logging, the issue was with my docker-compose.yml file. I needed to link the database correctly.
docker-compose.yml:
version: '3'
services:
database:
image: mysql:5.7
environment:
MYSQL_ALLOW_EMPTY_PASSWORD: 'yes'
MYSQL_USER: root
MYSQL_DATABASE: pensionator
ports:
- '3307:3306'
restart: always
appserver:
build:
context: .
dockerfile: src/main/docker/Dockerfile
depends_on:
- database
image: pensionator_app
environment:
SPRING_DATASOURCE_URL: 'jdbc:mysql://database:3306/pensionator'
links:
- database
ports:
- '8080:8080'
- '8000:8000'
restart: always

docker nodejs container cant connect mysql container

I'm running Docker server in Digital Ocean. There I have two containers Nodejs and Mysql. Mysql container has open port to 3306.
When trying to access mysql via nodejs by Docker Server ip + port. I get Error: connect ETIMEDOUT.
When I run same nodejs docker setup in my local computer it works fine. Is there something i'm missing?
Here is nodejs docker-composer.yml:
version: '2'
services:
test-web-install:
image: example-nodejs:latest
working_dir: /home/app
volumes:
- ./:/home/app
command: sh -c 'nodemon'
environment:
- NODE_ENV=development
- DB_HOST=192.168.11.207 #or public ip in internet
- DB_PORT=3036
- DB_PASSWORD=root
- DB_USER=root
- DB_DATABASE=root
ports:
- "3000:3000"
Here is docker-composer.yml for mysql
mysql:
container_name: flask_mysql
restart: always
image: mysql:5.6
environment:
MYSQL_ROOT_PASSWORD: 'root' # TODO: Change this
MYSQL_USER: 'root'
MYSQL_PASS: 'root'
MYSQL_DATABASE: 'root'
volumes:
- ./data:/var/lib/mysql
ports:
- "3036:3306"
restart: always
I'll modify answer as we advance - Following your comments, while I can not access to your env, lets try to solve this incrementally:
Let's make the db visible to the node.js server
See how it works and then probably dive into env networking configuration.
There 2 ways to solve 1st and may be 2nd problem as i see without being able to touch your env:
1st one will ensure that the server sees the database, but if you can not connect to the db from outside seems there firewall/droplet networking configuration issue, and you can try 2nd way (wont likely to change, but it's good to try). This assumes you use same docker compose and same bridge cusom network:
version: '2'
services:
test-web-install:
image: example-nodejs:latest
working_dir: /home/app
volumes:
- ./:/home/app
command: sh -c 'nodemon'
environment:
- NODE_ENV=development
- DB_HOST= mysql
- DB_PORT=3036
- DB_PASSWORD=root
- DB_USER=root
- DB_DATABASE=root
ports:
- "3000:3000"
networks:
inner:
alias: server
mysql:
container_name: flask_mysql
restart: always
image: mysql:5.6
environment:
MYSQL_ROOT_PASSWORD: 'root' # TODO: Change this
MYSQL_USER: 'root'
MYSQL_PASS: 'root'
MYSQL_DATABASE: 'root'
volumes:
- ./data:/var/lib/mysql
ports:
- "<externalEnvIp>:3036:3306"
restart: always
networks:
inner:
alias: mysql
networks:
inner:
driver: bridge
driver_opts:
com.docker.network.enable_ipv6: "true"
com.docker.network.bridge.enable_ip_masquerade: "true"
ipam:
driver: default
config:
- subnet: 172.16.100.0/24
gateway: 172.16.100.1
Option 2 :
version: '2'
services:
test-web-install:
image: example-nodejs:latest
working_dir: /home/app
volumes:
- ./:/home/app
command: sh -c 'nodemon'
environment:
- NODE_ENV=development
- DB_HOST= mysql
- DB_PORT=3036
- DB_PASSWORD=root
- DB_USER=root
- DB_DATABASE=root
ports:
- "3000:3000"
network_mode: "host"
mysql:
container_name: flask_mysql
restart: always
image: mysql:5.6
environment:
MYSQL_ROOT_PASSWORD: 'root' # TODO: Change this
MYSQL_USER: 'root'
MYSQL_PASS: 'root'
MYSQL_DATABASE: 'root'
volumes:
- ./data:/var/lib/mysql
ports:
- "3036:3306"
restart: always
network_mode: "host"
More precise solution (to find the roots of the problem) would involve into deep digging into your env network configuration, docker networking settings etc., but those solutions may help and fix your problem for now.
Pleasse after you try please output the results.
The docker networking doesn't allow you to go from inside a container back out to the host IP to connect to a port exposed by another container. I haven't dug into this enough to see if that's due to the iptables rules or perhaps something inside of docker-proxy. Either way, it's never been worth investigating since container-to-container networking is a built in feature of docker.
To use docker's networking, the containers need to be on the same docker network, and you reference them by their container name in DNS. With docker-compose, normally you can use the service name in place of the container name (e.g. test-web-install and mysql from your examples) since compose creates an alias for these. However, since you've overridden the container name for mysql, use your flask_mysql container name instead.
In your scenario, since you've split up the startup with two separate docker-compose.yml files, you'll be on separate networks created by compose. You have two options to resolve this:
Merge the two into a single docker-compose.yml (BlackStork gave an example of this).
Use an externally defined network that you create in advance.
To do the latter, first create your network:
docker network create dbnet
Then update your docker-compose.yml for the app to look like:
version: '2'
networks:
dbnet:
external: true
services:
test-web-install:
image: example-nodejs:latest
working_dir: /home/app
volumes:
- ./:/home/app
command: sh -c 'nodemon'
environment:
- NODE_ENV=development
- DB_HOST=flask_mysql
- DB_PORT=3036
- DB_PASSWORD=root
- DB_USER=root
- DB_DATABASE=root
ports:
- "3000:3000"
networks:
- dbnet
And the docker-compose.yml for mysql:
version: '2'
networks:
dbnet:
external: true
services:
mysql:
container_name: flask_mysql
restart: always
image: mysql:5.6
environment:
MYSQL_ROOT_PASSWORD: 'root' # TODO: Change this
MYSQL_USER: 'root'
MYSQL_PASS: 'root'
MYSQL_DATABASE: 'root'
volumes:
- ./data:/var/lib/mysql
ports:
- "3036:3306"
restart: always
networks:
- dbnet