What I'm trying to do is, connect from my spring-boot app to mysql database in Docker. Each in their own container.
But I must be having something wrong because I can't do it.
To keep it simple :
application-properties :
# URL for the mysql db
spring.datasource.url=jdbc:mysql://workaround-mysql:3308/workaround?serverTimezone=UTC&max_allowed_packet=15728640
# User name in mysql
spring.datasource.username=springuser
# Password for mysql
spring.datasource.password=admin
#Port at which application runs
server.port=8080
docker-compose for MySQL:
version: '3'
services:
workaround-mysql:
container_name: workaround-mysql
image: mysql
environment:
MYSQL_DATABASE: workaround
MYSQL_USER: springuser
MYSQL_PASSWORD: admin
MYSQL_ROOT_PASSWORD: admin
MYSQL_ROOT_HOST: '%'
ports:
- "3308:3306"
restart: always
So pretty simple right ? Database I start with docker-compose up:
All seems to be working fine so far.
Now that I have db started, to the application, this is its docker-compose.yml:
version: '3'
services:
workaround:
restart: always
# will build ./docker/workaround/Dockerfile
build: ./docker/workaround
working_dir: /workaround
volumes:
- ./:/workaround
- ~/.m2:/root/.m2
expose:
- "8080"
command: "mvn clean spring-boot:run"
For its Dockerfile I use Linux Alpine and Java.
FROM alpine:3.9
....add java...
RUN apk update
RUN apk add dos2unix --update-cache --repository http://dl-3.alpinelinux.org/alpine/edge/community/ --allow-untrusted
RUN apk add bash
RUN apk add maven
Super simple. Now let's start the application :
Unknown host, so let's try the IP then :
docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' workaround-mysql
# URL for the mysql db
spring.datasource.url=jdbc:mysql://172.20.0.2:3308/workaround?serverTimezone=UTC&max_allowed_packet=15728640
Now I get timeout:
As you can see I get error. What is wrong with my setup and how to fix
this? Either I have unknown host exception or Refused to connect or connection timeout.
I have tried:
Using ip of a container in my application.properties, didn't work
Different ports for MySQL and application
Different images and versions of MySQL
Having everything in one docker compose with wait
timer for database.
Minimal setup with
https://github.com/hellokoding/hellokoding-courses/tree/master/docker-examples/dockercompose-springboot-mysql-nginx
Also resulted in communication link failure, Site was accessible but I
doubt that db was connected properly.
Notes:
I run this all on one computer I use port 3308 because I have local
MySQL db at 3306.
Here is docker ps -a
#Vusal ANSWER output :
Only thing different from code in answer I did wait for database to be ready 30 seconds
command: /bin/bash -c "sleep 30;mvn clean spring-boot:run;"
Try this docker-compose.yml:
version: '3'
services:
workaround-mysql:
container_name: workaround-mysql
image: mysql
environment:
MYSQL_DATABASE: workaround
MYSQL_USER: springuser
MYSQL_PASSWORD: admin
MYSQL_ROOT_PASSWORD: admin
MYSQL_ROOT_HOST: '%'
ports:
- "3308:3306"
restart: always
workaround:
depends_on:
- workaround-mysql
restart: always
# will build ./docker/workaround/Dockerfile
build: ./docker/workaround
working_dir: /workaround
volumes:
- ./:/workaround
- ~/.m2:/root/.m2
expose:
- "8080"
command: "mvn clean spring-boot:run"
And update your application.properties to use the next JDBC connection url:
spring.datasource.url=jdbc:mysql://workaround-mysql:3306/workaround?serverTimezone=UTC&max_allowed_packet=15728640
It should work when both containers in the same docker-compose file, because docker-compose creates default network for containers, so they can resolve each other by name.
What you haven't tried so far is running both containers on the same Docker network.
First, forget about IP addressing - using it should be avoided by all means.
Second, launch both compose instances with the same Docker network.
Third, do not expose ports - inside bridge network all ports are accessible to running containers.
Create global network
docker network create foo
Modify both compose files so that they use this network instead of creating each one its own:
version: '3.5'
services:
....
networks:
default:
external: true
name: foo
Remove expose directives from compose files - inside one network all ports are exposed by default
Modify connection strings to use default 3306 port instead of 3308
Enjoy
In order for the service to connect with MySql through docker it has to be in same network, look into Docker network
But for better solution I would suggest you to write a single docker compose file for MySql and Spring boot.The reason is it will easily be linked when you do that.No need any other configuration.
version: "3"
services:
mysql-service:
image: mysql
ports:
- "3306:3306"
environment:
- MYSQL_DATABASE=db
- MYSQL_USER=root
- MYSQL_PASSWORD=pass
- MYSQL_ROOT_PASSWORD=pass
spring-service:
image: springservce:latest
ports:
- "8080:8080"
depends_on:
- mysql-service
Before you try to connect to the Docker container you should stop mysql in your computer then go to the application.properties and type:
spring.datasource.url=jdbc:mysql://localhost:3306/NAME_OF_YOUR_DB_HERE?useSSL=false&allowPublicKeyRetrieval=true
Regarding localhost, you should inspect the mysql container and pick the IP address and use it instead. most likely is 172.17.0.2. If it did not work then use localhost.
Related
I have a Java Spring Boot app which works with a Postgres database. I want to use Docker for both of them. I initially put just the Postgres in Docker, and I had a docker-compose.yml file defined like this:
version: '2'
services:
db:
container_name: sample_db
image: postgres:9.5
volumes:
- sample_db:/var/lib/postgresql/data
environment:
- POSTGRES_PASSWORD=sample
- POSTGRES_USER=sample
- POSTGRES_DB=sample
- PGDATA=/var/lib/postgresql/data/pgdata
ports:
- 5432:5432
volumes:
sample_db: {}
Then, when I issued the commands sudo dockerd and sudo docker-compose -f docker-compose.yml up, it was starting the database. I could connect using pgAdmin for example, by using localhost as server and port 5432. Then, in my Spring Boot app, inside the application.properties file I defined the following properties.
spring.datasource.url=jdbc:postgresql://localhost:5432/sample
spring.datasource.username=sample
spring.datasource.password=sample
spring.jpa.generate-ddl=true
At this point I could run my Spring Boot app locally through Spring Suite, and it all was working fine. Then, I wanted to also add my Spring Boot app as Docker image. I first of all created a Dockerfile in my project directory, which looks like this:
FROM java:8
EXPOSE 8080
ADD /target/manager.jar manager.jar
ENTRYPOINT ["java","-jar","manager.jar"]
Then, I entered to the directory of the project issued mvn clean followed by mvn install. Next, issued docker build -f Dockerfile -t manager . followed by docker tag 9c6b1e3f1d5e myuser/manager:latest (the id is correct). Finally, I edited my existing docker-compose.yml file to look like this:
version: '2'
services:
web:
image: myuser/manager:latest
ports:
- 8080:8080
depends_on:
- db
db:
container_name: sample_db
image: postgres:9.5
volumes:
- sample_db:/var/lib/postgresql/data
environment:
- POSTGRES_PASSWORD=sample
- POSTGRES_USER=sample
- POSTGRES_DB=sample
- PGDATA=/var/lib/postgresql/data/pgdata
ports:
- 5432:5432
volumes:
sample_db: {}
But, now if I issue sudo docker-compose -f docker-compose.yml up command, the database again starts correctly, but I get errors and exit code 1 for the web app part. The problem is the connection string. I believe I have to change it to something else, but I don't know what it should be. I get the following error messages:
web_1 | 2017-06-27 22:11:54.418 ERROR 1 --- [ main] o.a.tomcat.jdbc.pool.ConnectionPool : Unable to create initial connections of pool.
web_1 |
web_1 | org.postgresql.util.PSQLException: Connection to localhost:5432 refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections
Any ideas?
Each container has its own network interface with its own localhost. So change how Java points to Postgres:
spring.datasource.url=jdbc:postgresql://localhost:5432/sample
To:
spring.datasource.url=jdbc:postgresql://db:5432/sample
db will resolve to the proper Postgres IP.
Bonus. With docker-compose you don't need to build your image by hand. So change:
web:
image: myuser/manager:latest
To:
web:
build: .
I had the same problem and I lost some time to understand and solve this problem:
org.postgresql.util.PSQLException: Connection to localhost:5432 refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections.
I show all the properties so that everyone understands.
application.properties:
spring.datasource.url=jdbc:postgresql://localhost:5432/testdb
spring.datasource.driver-class-name=org.postgresql.Driver
spring.datasource.username=postgres
spring.datasource.password=postgres
spring.jpa.properties.hibernate.dialect=org.hibernate.dialect.PostgreSQL82Dialect
spring.jpa.hibernate.ddl-auto=update
docker-compose.yml:
version: "3"
services:
springapp:
build: .
container_name: springapp
environment:
SPRING_DATASOURCE_URL: jdbc:postgresql://db:5432/testdb
ports:
- 8000:8080
restart: always
depends_on:
- db
db:
image: postgres
container_name: db
environment:
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=postgres
- POSTGRES_DB=testdb
- PGDATA=/var/lib/postgresql/data/pgdata
ports:
- 5000:5432
volumes:
- pgdata:/var/lib/postgresql/data
restart: always
volumes:
pgdata:
For start spring application with local database we use url localhost.
For connect to container with database we need change 'localhost' on your database service, in my case 'localhost' to 'db'.
Solution: add SPRING_DATASOURCE_URL environment in docker-compose.yml wich rewrite spring.datasource.url value for connect:
environment:
SPRING_DATASOURCE_URL: jdbc:postgresql://db:5432/testdb
I hope this helps someone save his time.
You can use this.
version: "2"
services:
sample_db-postgresql:
image: postgres:9.5
ports:
- 5432:5432
environment:
- POSTGRES_PASSWORD=sample
- POSTGRES_USER=sample
- POSTGRES_DB=sample
volumes:
- sample_db:/var/lib/postgresql/data
volumes:
sample_db:
You can use ENV variable to change the db address in your docker-compose.
Dockerfile:
FROM java:8
EXPOSE 8080
ENV POSTGRES localhost
ADD /target/manager.jar manager.jar
ENTRYPOINT exec java $JAVA_OPTS -jar manager.jar --spring.datasource.url=jdbc:postgresql://$POSTGRES:5432/sample
docker-compose:
`
container_name: springapp
environment:
- POSTGRES=db`
Below is my docker-compose.yml file
version: '3.2'
services:
mysql-server:
container_name: mysql
ports:
- "13306:3306"
environment:
MYSQL_ROOT_PASSWORD: 12345
MYSQL_DATABASE: wordpress
MYSQL_USER: wordpress_user
MYSQL_PASSWORD: secret
image: mysql/mysql-server
grafana:
image: grafana/grafana
container_name: grafana
environment:
DB_HOST: mysql-server:3306
ports:
- "3000:3000"
depends_on:
- mysql-server
I would like to connect to grafana and use mysql to be my datasource.
However, when I connect to grafana using 'container ip address + port' 172.19.0.3 (retrieved by using 'docker inspect containerID'):3306. It cannot connect to grafana. I tried to close my firewall but still failed. Or my method of connection is wrong?
Besides, when I set MySQL as datasource in grafana, is the host be: mysql-server:3306?
So far you have not cared about networking of the two containers. Yes, both of them expose a port on the host - but you are not trying to access that one. Instead you are trying to make a connection from one container directly to the other container.
First step is to create a bridged network.
Then define both containers to be connected to that network.
Finally you can use the name of one container as hostname to connect from the other container.
Pulled MySQL image using command
docker pull mysql
$ docker run --name some-mysql -e MYSQL_ROOT_PASSWORD=my-secret-pw -d mysql:tag
Then made a stack.yml file in my root for mysql
stack.yml :
version: '3.1'
services:
db:
image: mysql
command: --default-authentication-plugin=mysql_native_password
restart: always
environment:
MYSQL_ROOT_PASSWORD: example
adminer:
image: adminer
restart: always
ports:
- 8080:8080
And lastly composed it up.
docker-compose -f stack.yml up (after making the stack.yml file)
After it stopped running I accessed the localhost:8080 page and the mysql database login page was loaded.
Superset setup
git clone https://github.com/apache/incubator-superset.git
cd incubator-superset
docker-compose up
Accessed superset on localhost:8088 page.
How do I frame the SQLAlchemy URI for connecting the Database and Superset?
You can access both applications individually due to it has a different bridge network. But when it comes to connecting each other, You have to connect through the same network for both applications. Superset already running on incubator-superset_default and MySQL running on default bridge network.
Here tested stack.yaml
version: '3.7'
services:
db:
image: mysql
command: --default-authentication-plugin=mysql_native_password
container_name: mysql
restart: always
environment:
MYSQL_ROOT_PASSWORD: sample
networks:
- proxynet
networks:
proxynet:
name: incubator-superset_default
Note:
As a user-defined network so use can use the service container name or container IP address as follows
mysql://root:sample#mysql/mysql
mysql://root:sample#172.19.0.5/mysql
Attempting to dockerize a Symfony 3.1 application, I run Docker for Windows:
Docker Desktop Version: 2.0.0.3
Engine Version: 18.09.2
Compose Version: 1.23.2
Here is my current docker-compose.yaml:
version: "3.6"
services:
nginx:
build:
context: ./../..
dockerfile: ./docker/dev/nginx/Dockerfile
volumes:
- ./../..:/app
ports:
- 80:80
links:
- mariadb:mariadb
- php:php
php:
build:
context: ./../..
dockerfile: ./docker/dev/php/Dockerfile
volumes:
- ./../..:/app
links:
- mariadb:mariadb
mariadb:
build:
context: ./../..
dockerfile: ./docker/dev/mariadb/Dockerfile
volumes:
- database:/var/lib/mysql
ports:
- 3306:3306
environment:
- MYSQL_ROOT_PASSWORD=root
- MYSQL_DATABASE=todo_list
- MYSQL_USER=todo_list
- MYSQL_PASSWORD=todo_list
volumes:
database:
Here are my Dockerfiles for each of the 3 services above:
nginx:
FROM nginx:1.16.0-alpine
COPY docker/dev/nginx/config/nginx.conf /etc/nginx/conf.d/default.conf
COPY . /app
php:
FROM php:7.3.6-fpm-alpine3.10
RUN docker-php-ext-install -j$(nproc) pdo_mysql
COPY docker/dev/php/config/php.conf ${PHP_INI_DIR}/conf.d/default.ini
COPY . /app
mariadb:
FROM mariadb:10.3
COPY docker/dev/mariadb/config/mariadb.conf /etc/mysql/conf.d/default.cnf
RUN chmod 0444 /etc/mysql/conf.d/default.cnf
Running docker-compose up, everything seems fine, no inconsistencies or error/warning detected in the services logs…
The Symfony parameters.yml in which is defined database access and credentials looks like this:
parameters:
database_host: 127.0.0.1
database_port: 3306
database_name: todo_list
database_user: todo_list
database_password: todo_list
With this configuration I am able to run successfully the Symfony console commands:
php bin/console doctrine:schema:update
php bin/console doctrine:fixtures:load
Which means that in this context, the app has access to the database.
But when I interact with the app via HTTP (http://localhost in web browser), so via the docker service nginx, I get a SQLSTATE[HY000] [2002] Connection refused.
So I change the parameters:database_host entry in the parameters.yml from 127.0.0.1 to 172.31.0.1 (which is the current IP address of the mariadb container) and this time I can interact with the app via HTTP, without any problem for it to access to the database, but instead, I get a SQLSTATE[HY000] [2002] when attempting to access to the database via the Symfony console.
I suspected a Docker Compose misconfiguration but after checking and rechecking the Docker doc, tutorials, example GitHub repos, and similar SE questions, for hours, I don't get what I'd be doing wrong…
As #DavidMaze said in the comment, remove the links from the docker-compose file and use the name of the service instead of the address (mariadb instead of 127.0.0.1). This way it will work for connections coming from inside docker.
This way, as you have already seen, it stops working for connections coming from outside docker (neve used Symfony console but I assume it's outside of docker). To work around it you have at least two ways:
quick and dirty, add a line in your hosts file (on windows, the file should be at C:\Windows\System32\Drivers\etc) like
this: 127.0.0.1 mariadb, this will instruct windows to resolve
mariadb as 127.0.0.1
configure your project to use different config files depending on if you are connecting from inside or outside docker (can't tell you
exactly how, it really depends on your project structure)
IMHO option 1 is acceptable for development machines.
You can't use 127.0.0.1 in your parameter.yml but you can use 'mariadb' instead since you defined it in your links section.
Now links is described as deprecated so it will work but you'll probably want to go with networking.
version: "3.6"
services:
nginx:
build:
context: ./../..
dockerfile: ./docker/dev/nginx/Dockerfile
networks:
- frontend
- backend
volumes:
- ./../..:/app
ports:
- 80:80
php:
build:
context: ./../..
dockerfile: ./docker/dev/php/Dockerfile
networks:
- frontend
- backend
volumes:
- ./../..:/app
mariadb:
build:
context: ./../..
dockerfile: ./docker/dev/mariadb/Dockerfile
networks:
- backend
volumes:
- database:/var/lib/mysql
ports:
- 3306:3306
environment:
- MYSQL_ROOT_PASSWORD=root
- MYSQL_DATABASE=todo_list
- MYSQL_USER=todo_list
- MYSQL_PASSWORD=todo_list
volumes:
database:
networks:
frontend:
driver: bridge
name: frontend #or whatever custom name since 3.5
backend:
driver: bridge
name: backend #or whatever custom name since 3.5
I am trying to set up a mysql docker container and execute init sql script. Unfortunately the sql script is not executed. What am I doing wrong?
version: '3.3'
services:
api:
container_name: 'api'
build: './api'
ports:
- target: 8080
published: 8888
protocol: tcp
mode: host
volumes:
- './api:/go/src/app'
depends_on:
- 'mysql'
mysql:
image: 'mysql:latest'
container_name: 'mysql'
volumes:
- ./db_data:/var/lib/mysql:rw
- ./database/init.sql:/docker-entrypoint-initdb.d/init.sql:ro
restart: always
environment:
MYSQL_USER: test
MYSQL_PASSWORD: test
MYSQL_ROOT_PASSWORD: test
MYSQL_DATABASE: test
ports:
- '3306:3306'
volumes:
db_data:
I execute file with docker-compose up -d --build
The docker-entrypoint-initdb.d folder will only be run once while the container is created (instantiated) so you actually have to do a docker-compose down -v to re-activate this for the next run.
If you want to be able to add sql files at any moment you can look here at a specialized MySql docker image... http://ivo2u.nl/o4
Update for M1 arch:
Here an almost drop-in replacement in MariaDB: http://ivo2u.nl/V1
Many containerized applications, especially stateful ones, have a way of running init scripts (like the sql scripts here) and they are supposed to run only once.
And since they are stateful, the volumes are a source of truth for the containers on whether to run the init scripts or not on container restart.
Like in your case, deleting the folder used for bind mount or using a new named volume should re-run any init scripts present.
These scripts run when you create the container, not every time you start it.
You can docker-compose up --force-recreate mysql to force those scripts to re-run.
Additionally, if you have a volume like this ./db_data:/var/lib/mysql:rw, then you also need to remove ./db_data before recreating the container.
I'm not a docker expert, but this worked for me.