I have to deploy a web app with a Jetty Server. This app need a database, running on MariaDB. Here the docker-compose file used to deploy the app:
version: '3.0'
services:
jetty:
build:
context: .
dockerfile: docker/jetty/Dockerfile
container_name: app-jetty
ports:
- "8080:8080"
depends_on:
- mariadb
networks:
- app
links:
- "mariadb:mariadb"
mariadb:
image: mariadb:10.7
container_name: app-mariadb
restart: always
environment:
MARIADB_ROOT_PASSWORD: myPassword
MARIADB_DATABASE: APPBD
ports:
- "3307:3306"
networks:
- app
adminer:
image: adminer
container_name: app-adminer
restart: always
ports:
- "2002:8080"
depends_on:
- mariadb
networks:
- app
networks:
app:
driver: bridge
The Dockerfile used by the Jetty container:
FROM gradle:7.2-jdk17 as grad
WORKDIR /tmp
RUN mkdir src
RUN ls -la
COPY src /tmp/src
COPY build.gradle /tmp/build.gradle
RUN ls -la
RUN gradle --warning-mode all war
RUN ls -la /tmp/build/libs/
FROM jetty:latest
COPY --from=grad /tmp/build/libs/tmp.war /var/lib/jetty/webapps/ROOT.war
EXPOSE 8080
The app is build with Gradle before the initialisation of Jetty.
The problem is the database: it takes several minutes to initialize the MariaDB database. So, when I want to connect to the DB from Adminer, I have to wait until the DB is ready before I can log in. But for my app, I got this: Could not connect to address=(host=mariadb)(port=3306)(type=master) : Socket fail to connect to host:mariadb, port:3306. Connection refused, even if the database is ready. I have to stop the app-jetty container and restart it to use the database. I thought that depends_on will ran the app-jetty when the database was ready, but it ran the container when the app-mariadb was ran.
I use JDBC to establish connection to the DB : jdbc:mariadb://mariadb:3306/APPBD?user=root&password=myPassword. I successfully establish connection on local execution with gradle appRun and use localhost:3306 instead of mariadb:3306, so I think I don't have any mistakes on my code.
How can I indicate to the app-jetty container to start only when MariaDB is ready?
Compose (2.1) supports healthchecks:
Under the mariadb service:
healthcheck:
test: [ "CMD", "mariadb-admin", "--protocol", "tcp" ,"ping" ]
timeout: 3m
interval: 10s
retries: 10
ref: tip.
Related
I have a Java Spring Boot app which works with a Postgres database. I want to use Docker for both of them. I initially put just the Postgres in Docker, and I had a docker-compose.yml file defined like this:
version: '2'
services:
db:
container_name: sample_db
image: postgres:9.5
volumes:
- sample_db:/var/lib/postgresql/data
environment:
- POSTGRES_PASSWORD=sample
- POSTGRES_USER=sample
- POSTGRES_DB=sample
- PGDATA=/var/lib/postgresql/data/pgdata
ports:
- 5432:5432
volumes:
sample_db: {}
Then, when I issued the commands sudo dockerd and sudo docker-compose -f docker-compose.yml up, it was starting the database. I could connect using pgAdmin for example, by using localhost as server and port 5432. Then, in my Spring Boot app, inside the application.properties file I defined the following properties.
spring.datasource.url=jdbc:postgresql://localhost:5432/sample
spring.datasource.username=sample
spring.datasource.password=sample
spring.jpa.generate-ddl=true
At this point I could run my Spring Boot app locally through Spring Suite, and it all was working fine. Then, I wanted to also add my Spring Boot app as Docker image. I first of all created a Dockerfile in my project directory, which looks like this:
FROM java:8
EXPOSE 8080
ADD /target/manager.jar manager.jar
ENTRYPOINT ["java","-jar","manager.jar"]
Then, I entered to the directory of the project issued mvn clean followed by mvn install. Next, issued docker build -f Dockerfile -t manager . followed by docker tag 9c6b1e3f1d5e myuser/manager:latest (the id is correct). Finally, I edited my existing docker-compose.yml file to look like this:
version: '2'
services:
web:
image: myuser/manager:latest
ports:
- 8080:8080
depends_on:
- db
db:
container_name: sample_db
image: postgres:9.5
volumes:
- sample_db:/var/lib/postgresql/data
environment:
- POSTGRES_PASSWORD=sample
- POSTGRES_USER=sample
- POSTGRES_DB=sample
- PGDATA=/var/lib/postgresql/data/pgdata
ports:
- 5432:5432
volumes:
sample_db: {}
But, now if I issue sudo docker-compose -f docker-compose.yml up command, the database again starts correctly, but I get errors and exit code 1 for the web app part. The problem is the connection string. I believe I have to change it to something else, but I don't know what it should be. I get the following error messages:
web_1 | 2017-06-27 22:11:54.418 ERROR 1 --- [ main] o.a.tomcat.jdbc.pool.ConnectionPool : Unable to create initial connections of pool.
web_1 |
web_1 | org.postgresql.util.PSQLException: Connection to localhost:5432 refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections
Any ideas?
Each container has its own network interface with its own localhost. So change how Java points to Postgres:
spring.datasource.url=jdbc:postgresql://localhost:5432/sample
To:
spring.datasource.url=jdbc:postgresql://db:5432/sample
db will resolve to the proper Postgres IP.
Bonus. With docker-compose you don't need to build your image by hand. So change:
web:
image: myuser/manager:latest
To:
web:
build: .
I had the same problem and I lost some time to understand and solve this problem:
org.postgresql.util.PSQLException: Connection to localhost:5432 refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections.
I show all the properties so that everyone understands.
application.properties:
spring.datasource.url=jdbc:postgresql://localhost:5432/testdb
spring.datasource.driver-class-name=org.postgresql.Driver
spring.datasource.username=postgres
spring.datasource.password=postgres
spring.jpa.properties.hibernate.dialect=org.hibernate.dialect.PostgreSQL82Dialect
spring.jpa.hibernate.ddl-auto=update
docker-compose.yml:
version: "3"
services:
springapp:
build: .
container_name: springapp
environment:
SPRING_DATASOURCE_URL: jdbc:postgresql://db:5432/testdb
ports:
- 8000:8080
restart: always
depends_on:
- db
db:
image: postgres
container_name: db
environment:
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=postgres
- POSTGRES_DB=testdb
- PGDATA=/var/lib/postgresql/data/pgdata
ports:
- 5000:5432
volumes:
- pgdata:/var/lib/postgresql/data
restart: always
volumes:
pgdata:
For start spring application with local database we use url localhost.
For connect to container with database we need change 'localhost' on your database service, in my case 'localhost' to 'db'.
Solution: add SPRING_DATASOURCE_URL environment in docker-compose.yml wich rewrite spring.datasource.url value for connect:
environment:
SPRING_DATASOURCE_URL: jdbc:postgresql://db:5432/testdb
I hope this helps someone save his time.
You can use this.
version: "2"
services:
sample_db-postgresql:
image: postgres:9.5
ports:
- 5432:5432
environment:
- POSTGRES_PASSWORD=sample
- POSTGRES_USER=sample
- POSTGRES_DB=sample
volumes:
- sample_db:/var/lib/postgresql/data
volumes:
sample_db:
You can use ENV variable to change the db address in your docker-compose.
Dockerfile:
FROM java:8
EXPOSE 8080
ENV POSTGRES localhost
ADD /target/manager.jar manager.jar
ENTRYPOINT exec java $JAVA_OPTS -jar manager.jar --spring.datasource.url=jdbc:postgresql://$POSTGRES:5432/sample
docker-compose:
`
container_name: springapp
environment:
- POSTGRES=db`
I'm trying to connect my FASTAPI app container to a MySQL database container using the docker-compose file. In the Docker documentation it says that docker creates a default network for both containers. However, I would like to use a pre-existing network that I've created(app-net).
This is my docker-compose file:
version: '3.4'
services:
mysql:
image: mysql:latest
command: mysqld --default-authentication-plugin=mysql_native_password
container_name: mysql
restart: always
ports:
- 3307:3306
environment:
MYSQL_ROOT_PASSWORD: password
volumes:
- mysql-data:/var/lib/mysql
app:
build: .
image: app:1.0
container_name: app
ports:
- 8000:8000
environment:
PASSWORD: password
USERNAME: root
networks:
default:
external: true
name: app-net
volumes:
mysql-data:
driver: local
this is the output I get when i run docker inspect mysql -f "{{json .NetworkSettings.Networks }}":
{"app-net":{"IPAMConfig":null,"Links":null,"Aliases":["mysql","5e998f9fb646"],"NetworkID":"7f60c83e4c88d25e674461521446ec9fa98baca8639c782c79671c4fcb77ba88","EndpointID":"","Gateway":"","IPAddress":"","IPPrefixLen":0,"IPv6Gateway":"","GlobalIPv6Address":"","GlobalIPv6PrefixLen":0,"MacAddress":"","DriverOpts":null}}
However, when I run each container individually using CMD with the --network app-net the output is different:
{"app-net":{"IPAMConfig":null,"Links":null,"Aliases":["46157e588c87"],"NetworkID":"7f60c83e4c88d25e674461521446ec9fa98baca8639c782c79671c4fcb77ba88","EndpointID":"6a6922a9a6ea8f9d113447decbbb927cb93ddffd3b9563ee882fa2e44970cde5","Gateway":"172.20.0.1","IPAddress":"172.20.0.2","IPPrefixLen":16,"IPv6Gateway":"","GlobalIPv6Address":"","GlobalIPv6PrefixLen":0,"MacAddress":"02:42:ac:14:00:02","DriverOpts":null}}
In my app code in order to connect the mysql server, I specified the container name as the hostname since they are supposed to share the same network. But, as I mentioned it seems both containers can't talk to each other.
I'm pretty sure that is the reason I can't connect the database through my app and get that error when I run:
docker-compose -f docker-compose.yml up
I get this error:
sqlalchemy.exc.OperationalError: (_mysql_exceptions.OperationalError) (2005, "Unknown MySQL server host 'mysql' (0)")
What am I missing?
If the error you're getting is specifically "unknown host" or something similar, this can happen if your application container starts before the database container exists. You can work around this situation by telling Compose about the dependency:
version: '3.8'
services:
mysql: { ... }
app:
depends_on:
- mysql
This has two key effects. If you try to start only the application container docker-compose up app, it will also start the mysql container, following the depends_on: chain. When Compose does start containers, it at least creates the mysql container before creating the app container.
Note that this doesn't guarantee the database will actually be running by the time your application starts up. If you do encounter this, you will get a different error like "connection refused". You can work around this by embedding a script in your image that waits for the database to be available, or by using a version 2 Compose file with health-check support; see Docker Compose wait for container X before starting Y for more details on this specific problem.
You could add the key networks to both containers, this way:
version: '3.4'
services:
mysql:
image: mysql:latest
command: mysqld --default-authentication-plugin=mysql_native_password
container_name: mysql
restart: always
ports:
- 3307:3306
environment:
MYSQL_ROOT_PASSWORD: password
volumes:
- mysql-data:/var/lib/mysql
networks:
- app-net
app:
build: .
image: app:1.0
container_name: app
ports:
- 8000:8000
environment:
PASSWORD: password
USERNAME: root
networks:
- app-net
networks:
default:
external: true
name: app-net
volumes:
mysql-data:
driver: local
I am working for the first time with Docker. In my job, they asked me to deploy a complete application using docker-compose and I am learning right now.
I have a docker-compose.yml that starts two services: one for spring and another one for MySQL.
The one for Spring uses a Dockerfile to build the image.
The one for MySQL uses the official MySQL image.
I don´t know how to load the init.sql to initialize and load data into the database of the MySQL container.
I´ve tried to use another Dockerfile in order to copy init.sql into the MySQL container but,
how do I tell the compose to use it when creating the image or running the container?
This is for a Windows 10 OS Desktop. Docker version 18.09.2. Docker-compose version 1.23.2
Docker Desktop
Docker-compose.yml
version: '3.7'
services:
mysql-docker-container:
image: mysql
container_name: mysql-docker-container
environment:
- MYSQL_ROOT_PASSWORD=root
- MYSQL_DATABASE=database
- MYSQL_USER=user
- MYSQL_PASSWORD=user
ports:
- 2012:3306
tty: true
spring-jpa-app:
build:
context: .
dockerfile: Dockerfile
container_name: spring-jpa-app
links:
- mysql-docker-container:mysql-docker-container
depends_on:
- mysql-docker-container
ports:
- 8087:8080
tty: true
restart: always
Dockerfile
FROM java:8
VOLUME /tmp
EXPOSE 8080
ADD app-1.0.0.jar app-1.0.0.jar
ENTRYPOINT ["java","-jar","app-1.0.0.jar"]
application.properties
spring.datasource.driver-class-name=com.mysql.jdbc.Driver
spring.datasource.url=jdbc:mysql://mysql-docker-container:2012/database?autoReconnect=true&useSSL=false
spring.datasource.username=user
spring.datasource.password=user
spring.jpa.database=database
spring.jpa.hibernate.ddl-auto=create-drop
spring.jpa.properties.hibernate.dialect=org.hibernate.dialect.MySQL5Dialect
logging.file=log/log.log
logging.level.org.springframework=info
logging.level.orghibernate=info
Docker Desktop running
Opened Git Bash as administrator
docker-machine ls
NAME ACTIVE DRIVER STATE URL DOCKER
app * hyperv Running tcp://172.18.67.35:2376 v19.03.0
In the app folder, where is located /src
mvn clean install -DskipTests
BUILD SUCCESS
In the app folder where is located application.properties, app.jar, docker-compose.yml, Dockerfile and init.sql
docker-compose up --build
Postman
URL = http://172.18.67.35:8087
POST {{url}}/login
{
"Authorization": "gnioengrlnkwejnR",
"username": "Administrator",
"role": "ADMIN",
"id": "1"
}
GET {{url}}/listAllCenters
[]
Returns empty because init.sql is not being loaded
PD: Also, if I open a new git Bash as Administrator and do
docker ps -a
I do not get the containers running so I cannot know container's ID to enter the MySQL container.
you need to add a volume to copy the sql.init to your db continer:
version: '3.7'
services:
mysql-docker-container:
image: mysql
container_name: mysql-docker-container
environment:
- MYSQL_ROOT_PASSWORD=root
- MYSQL_DATABASE=database
- MYSQL_USER=user
- MYSQL_PASSWORD=user
ports:
- 2012:3306
volumes:
- /bath/to/file/on/host/init.sql:/docker-entrypoint-initdb.d/init.sql
tty: true
network_mode: "host"
spring-jpa-app:
build:
context: .
dockerfile: Dockerfile
container_name: spring-jpa-app
links:
- mysql-docker-container:mysql-docker-container
depends_on:
- mysql-docker-container
ports:
- 8087:8080
tty: true
restart: always
network_mode: "host"
see this and search for "Initializing a fresh instance"
on the other hand you need to use network to setup the connection between the containers - see the compose file above-
and you need to change you configs to
spring.datasource.url=jdbc:mysql://localhost:2012/database?autoReconnect=true&useSSL=false
Attempting to dockerize a Symfony 3.1 application, I run Docker for Windows:
Docker Desktop Version: 2.0.0.3
Engine Version: 18.09.2
Compose Version: 1.23.2
Here is my current docker-compose.yaml:
version: "3.6"
services:
nginx:
build:
context: ./../..
dockerfile: ./docker/dev/nginx/Dockerfile
volumes:
- ./../..:/app
ports:
- 80:80
links:
- mariadb:mariadb
- php:php
php:
build:
context: ./../..
dockerfile: ./docker/dev/php/Dockerfile
volumes:
- ./../..:/app
links:
- mariadb:mariadb
mariadb:
build:
context: ./../..
dockerfile: ./docker/dev/mariadb/Dockerfile
volumes:
- database:/var/lib/mysql
ports:
- 3306:3306
environment:
- MYSQL_ROOT_PASSWORD=root
- MYSQL_DATABASE=todo_list
- MYSQL_USER=todo_list
- MYSQL_PASSWORD=todo_list
volumes:
database:
Here are my Dockerfiles for each of the 3 services above:
nginx:
FROM nginx:1.16.0-alpine
COPY docker/dev/nginx/config/nginx.conf /etc/nginx/conf.d/default.conf
COPY . /app
php:
FROM php:7.3.6-fpm-alpine3.10
RUN docker-php-ext-install -j$(nproc) pdo_mysql
COPY docker/dev/php/config/php.conf ${PHP_INI_DIR}/conf.d/default.ini
COPY . /app
mariadb:
FROM mariadb:10.3
COPY docker/dev/mariadb/config/mariadb.conf /etc/mysql/conf.d/default.cnf
RUN chmod 0444 /etc/mysql/conf.d/default.cnf
Running docker-compose up, everything seems fine, no inconsistencies or error/warning detected in the services logs…
The Symfony parameters.yml in which is defined database access and credentials looks like this:
parameters:
database_host: 127.0.0.1
database_port: 3306
database_name: todo_list
database_user: todo_list
database_password: todo_list
With this configuration I am able to run successfully the Symfony console commands:
php bin/console doctrine:schema:update
php bin/console doctrine:fixtures:load
Which means that in this context, the app has access to the database.
But when I interact with the app via HTTP (http://localhost in web browser), so via the docker service nginx, I get a SQLSTATE[HY000] [2002] Connection refused.
So I change the parameters:database_host entry in the parameters.yml from 127.0.0.1 to 172.31.0.1 (which is the current IP address of the mariadb container) and this time I can interact with the app via HTTP, without any problem for it to access to the database, but instead, I get a SQLSTATE[HY000] [2002] when attempting to access to the database via the Symfony console.
I suspected a Docker Compose misconfiguration but after checking and rechecking the Docker doc, tutorials, example GitHub repos, and similar SE questions, for hours, I don't get what I'd be doing wrong…
As #DavidMaze said in the comment, remove the links from the docker-compose file and use the name of the service instead of the address (mariadb instead of 127.0.0.1). This way it will work for connections coming from inside docker.
This way, as you have already seen, it stops working for connections coming from outside docker (neve used Symfony console but I assume it's outside of docker). To work around it you have at least two ways:
quick and dirty, add a line in your hosts file (on windows, the file should be at C:\Windows\System32\Drivers\etc) like
this: 127.0.0.1 mariadb, this will instruct windows to resolve
mariadb as 127.0.0.1
configure your project to use different config files depending on if you are connecting from inside or outside docker (can't tell you
exactly how, it really depends on your project structure)
IMHO option 1 is acceptable for development machines.
You can't use 127.0.0.1 in your parameter.yml but you can use 'mariadb' instead since you defined it in your links section.
Now links is described as deprecated so it will work but you'll probably want to go with networking.
version: "3.6"
services:
nginx:
build:
context: ./../..
dockerfile: ./docker/dev/nginx/Dockerfile
networks:
- frontend
- backend
volumes:
- ./../..:/app
ports:
- 80:80
php:
build:
context: ./../..
dockerfile: ./docker/dev/php/Dockerfile
networks:
- frontend
- backend
volumes:
- ./../..:/app
mariadb:
build:
context: ./../..
dockerfile: ./docker/dev/mariadb/Dockerfile
networks:
- backend
volumes:
- database:/var/lib/mysql
ports:
- 3306:3306
environment:
- MYSQL_ROOT_PASSWORD=root
- MYSQL_DATABASE=todo_list
- MYSQL_USER=todo_list
- MYSQL_PASSWORD=todo_list
volumes:
database:
networks:
frontend:
driver: bridge
name: frontend #or whatever custom name since 3.5
backend:
driver: bridge
name: backend #or whatever custom name since 3.5
I wish to set up 2 docker containers: grafana and mysql and to allow queries from the grafana to the mysql db.
I have an AWS machine, in which I built the following folders structure:
- docker
- docker-compose.yml
- grafana:
- config.ini
- dashboards:
- grafana_dashboard.json
- provisioning:
- dashboards:
- all.yml
- datasources:
- all.yml
- Dockerfile
- mysql:
- dbcreation.sql
- Dockerfile
- dashboards:
- import.sh
the content of the docker-compose.yml is:
version: '2'
services:
db-service:
build: './mysql'
container_name: mysql
restart: always
ports:
- "3306:3306"
networks:
net:
ipv4_address: 172.16.1.3
grafana-service:
build: './grafana'
container_name: grafana
restart: always
ports:
- "3000:3000"
environment:
GF_SECURITY_ADMIN_PASSWORD: "XXX1"
GF_AUTH_PROXY_ENABLED: "true"
GF_SECURITY_DATA_SOURCE_PROXY_WHITELIST: 172.16.1.3:3306
GF_AUTH_ANONYMOUS_ENABLED: "true"
GF_LOG_LEVEL: "debug"
depends_on:
- db-service
networks:
net:
ipv4_address: 172.16.1.4
networks:
net:
external: true
volumes:
grafanadata:
driver: local
mysqldata:
The dockerfile for the grafana:
FROM grafana/grafana:5.2.2
ADD ./provisioning /etc/grafana/provisioning
ADD ./dashboards /var/lib/grafana/dashboards
ENV DS_DB "grafana"
The content of the mysql/Dockerfile is:
FROM mysql:8.0.12
ENV MYSQL_ROOT_PASSWORD="XXX2"
ENV MYSQL_DATABASE="grafana"
ADD ./dbcreation.sql /docker-entrypoint-initdb.d/dbcreation.sql
EXPOSE 3306
The grafana_dashboard.json file has the exported json from the Grafana I had set up locally on my own computer.
The dbcreation.sql file has the exported data from the local DB I had set up locally on my own computer.
I'm running the following commands:
docker network create --gateway 172.16.1.1 --subnet 172.16.1.0/24 net
docker-compose up --build
I'm getting an error: "The authentication plugin is not supported"
when turning the log level of Grafana to debug I'm seeing this:
t=2018-08-19T10:55:20+0000 lvl=dbug msg=getEngine logger=tsdb.mysql connection="root:XXX2#tcp(172.16.1.3:3306)/grafana?collation=utf8mb4_unicode_ci&parseTime=true&loc=UTC&allowNativePasswords=true"
t=2018-08-19T10:55:47+0000 lvl=eror msg="Request Completed" logger=context userId=1 orgId=1 uname=admin method=POST path=/api/tsdb/query status=500 remote_addr=XX.XXX.XXX.XXX time_ms=2 size=195 referer=http://XX.XXX.XXX.XXX:3000/datasources/edit/1
I have used the following sources already to set this up:
https://ops.tips/blog/initialize-grafana-with-preconfigured-dashboards/
https://storage.pardot.com/138181/61672/mysql_on_docker_how_to_containerize_your_database.pdf
Any help would be appreciated!
Thanks
As stated here:
go-mysql: authentication plugin not supported while connecting from go app container to mysql container
The problem was compatibility between the Grafana versions and the MySQL version.
Once moving to docker image mysql:5.7 (had to migrate the data too) - the issue was resolved (need to also change the COLLATION and CHAR_SET in the DB to downgrade from version 8.0.12 to 5.7).