With docker-compose, database container is not up and running - mysql

Question

I am following this tutorial to setup spring boot project with docker-compose. Here is what I did:
My application.properties:
spring.datasource.url = jdbc:mysql://mysql-demo-container:3306/demo_db?useSSL=false
spring.datasource.username = root
spring.datasource.password =root
spring.datasource.tomcat.testWhileIdle = true
spring.datasource.tomcat.timeBetweenEvictionRunsMillis = 60000
spring.datasource.tomcat.validationQuery = SELECT 1
spring.jpa.properties.hibernate.dialect = org.hibernate.dialect.MySQL5Dialect
# Hibernate ddl auto (create, create-drop, validate, update)
spring.jpa.hibernate.ddl-auto = update
My Dockerfile:
`FROM java:8
LABEL maintainer=“foo.bar#gmail.com”
VOLUME /tmp
EXPOSE 8080
ADD target/spring-boot-data-jpa-example-0.0.1-SNAPSHOT.jar spring-boot-data-jpa-example-0.0.1-SNAPSHOT.jar
ENTRYPOINT ["java","-jar","spring-boot-data-jpa-example-0.0.1-SNAPSHOT.jar"]`
I have this docker-compose.yml:
version: '3'
services:
mysql-demo-container:
image: mysql:latest
environment:
- MYSQL_ROOT_PASSWORD=root
- MYSQL_DATABASE=demo_db
- MYSQL_PASSWORD=myPass
ports:
- 2012:3306
volumes:
- /data/mysql
spring-boot-jpa-app:
image: spring-boot-jpa-image
build:
context: ./
dockerfile: Dockerfile
depends_on:
- mysql-demo-container
ports:
- 8087:8080
volumes:
- /data/spring-boot-app
I firstly mvn clean install to build the project jar file, then I run docker-compose up to build images & hopefully run containers.
The logs:
$ docker-compose up
Creating network "spring-boot-data-jpa-example-master_default" with the default driver
Creating spring-boot-data-jpa-example-master_mysql-demo-container_1 ... done
Creating spring-boot-data-jpa-example-master_spring-boot-jpa-app_1 ... done
Attaching to spring-boot-data-jpa-example-master_mysql-demo-container_1, spring-boot-data-jpa-example-master_spring-boot-jpa-app_1
mysql-demo-container_1 | Initializing database
mysql-demo-container_1 | 2019-07-02T13:03:05.097872Z 0 [Warning] [MY-011070] [Server] 'Disabling symbolic links using --skip-symbolic-links (or equivalent) is the default. Consider not using this option as it' is deprecated and will be removed in a future release.
mysql-demo-container_1 | 2019-07-02T13:03:05.098037Z 0 [System] [MY-013169] [Server] /usr/sbin/mysqld (mysqld 8.0.16) initializing of server in progress as process 29
mysql-demo-container_1 | 2019-07-02T13:03:09.171833Z 5 [Warning] [MY-010453] [Server] root#localhost is created with an empty password ! Please consider switching off the --initialize-insecure option.
spring-boot-jpa-app_1 |
spring-boot-jpa-app_1 | . ____ _ __ _ _
spring-boot-jpa-app_1 | /\\ / ___'_ __ _ _(_)_ __ __ _ \ \ \ \
spring-boot-jpa-app_1 | ( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
spring-boot-jpa-app_1 | \\/ ___)| |_)| | | | | || (_| | ) ) ) )
spring-boot-jpa-app_1 | ' |____| .__|_| |_|_| |_\__, | / / / /
spring-boot-jpa-app_1 | =========|_|==============|___/=/_/_/_/
spring-boot-jpa-app_1 | :: Spring Boot :: (v1.5.9.RELEASE)
spring-boot-jpa-app_1 |
spring-boot-jpa-app_1 | 2019-07-02 13:03:11.516 INFO 1 --- [ main] .s.e.SpringBootDataJpaExampleApplication : Starting SpringBootDataJpaExampleApplication v0.0.1-SNAPSHOT on 3c0d94b76fd6 with PID 1 (/spring-boot-data-jpa-example-0.0.1-SNAPSHOT.jar started by root in /)
spring-boot-jpa-app_1 | 2019-07-02 13:03:11.556 INFO 1 --- [ main] .s.e.SpringBootDataJpaExampleApplication : No active profile set, falling back to default profiles: default
spring-boot-jpa-app_1 | 2019-07-02 13:03:11.979 INFO 1 --- [ main] ationConfigEmbeddedWebApplicationContext : Refreshing org.springframework.boot.context.embedded.AnnotationConfigEmbeddedWebApplicationContext#6d5380c2: startup date [Tue Jul 02 13:03:11 UTC 2019]; root of context hierarchy
mysql-demo-container_1 | 2019-07-02T13:03:13.525348Z 0 [System] [MY-013170] [Server] /usr/sbin/mysqld (mysqld 8.0.16) initializing of server has completed
mysql-demo-container_1 | Database initialized
mysql-demo-container_1 | MySQL init process in progress...
mysql-demo-container_1 | MySQL init process in progress...
mysql-demo-container_1 | MySQL init process in progress...
mysql-demo-container_1 | 2019-07-02T13:03:15.636380Z 0 [Warning] [MY-011070] [Server] 'Disabling symbolic links using --skip-symbolic-links (or equivalent) is the default. Consider not using this option as it' is deprecated and will be removed in a future release.
mysql-demo-container_1 | 2019-07-02T13:03:15.636539Z 0 [System] [MY-010116] [Server] /usr/sbin/mysqld (mysqld 8.0.16) starting as process 80
mysql-demo-container_1 | 2019-07-02T13:03:18.070695Z 0 [Warning] [MY-010068] [Server] CA certificate ca.pem is self signed.
mysql-demo-container_1 | 2019-07-02T13:03:18.081045Z 0 [Warning] [MY-011810] [Server] Insecure configuration for --pid-file: Location '/var/run/mysqld' in the path is accessible to all OS users. Consider choosing a different directory.
mysql-demo-container_1 | 2019-07-02T13:03:18.152808Z 0 [System] [MY-010931] [Server] /usr/sbin/mysqld: ready for connections. Version: '8.0.16' socket: '/var/run/mysqld/mysqld.sock' port: 0 MySQL Community Server - GPL.
mysql-demo-container_1 | 2019-07-02T13:03:18.311442Z 0 [System] [MY-011323] [Server] X Plugin ready for connections. Socket: '/var/run/mysqld/mysqlx.sock'
spring-boot-jpa-app_1 | 2019-07-02 13:03:20.039 INFO 1 --- [ main] s.b.c.e.t.TomcatEmbeddedServletContainer : Tomcat initialized with port(s): 8080 (http)
spring-boot-jpa-app_1 | 2019-07-02 13:03:20.221 INFO 1 --- [ main] o.apache.catalina.core.StandardService : Starting service [Tomcat]
spring-boot-jpa-app_1 | 2019-07-02 13:03:20.232 INFO 1 --- [ main] org.apache.catalina.core.StandardEngine : Starting Servlet Engine: Apache Tomcat/8.5.23
spring-boot-jpa-app_1 | 2019-07-02 13:03:21.210 INFO 1 --- [ost-startStop-1] o.a.c.c.C.[Tomcat].[localhost].[/] : Initializing Spring embedded WebApplicationContext
spring-boot-jpa-app_1 | 2019-07-02 13:03:21.213 INFO 1 --- [ost-startStop-1] o.s.web.context.ContextLoader : Root WebApplicationContext: initialization completed in 9251 ms
spring-boot-jpa-app_1 | 2019-07-02 13:03:22.153 INFO 1 --- [ost-startStop-1] o.s.b.w.servlet.ServletRegistrationBean : Mapping servlet: 'dispatcherServlet' to [/]
spring-boot-jpa-app_1 | 2019-07-02 13:03:22.169 INFO 1 --- [ost-startStop-1] o.s.b.w.servlet.FilterRegistrationBean : Mapping filter: 'characterEncodingFilter' to: [/*]
spring-boot-jpa-app_1 | 2019-07-02 13:03:22.178 INFO 1 --- [ost-startStop-1] o.s.b.w.servlet.FilterRegistrationBean : Mapping filter: 'hiddenHttpMethodFilter' to: [/*]
spring-boot-jpa-app_1 | 2019-07-02 13:03:22.178 INFO 1 --- [ost-startStop-1] o.s.b.w.servlet.FilterRegistrationBean : Mapping filter: 'httpPutFormContentFilter' to: [/*]
spring-boot-jpa-app_1 | 2019-07-02 13:03:22.181 INFO 1 --- [ost-startStop-1] o.s.b.w.servlet.FilterRegistrationBean : Mapping filter: 'requestContextFilter' to: [/*]
mysql-demo-container_1 | Warning: Unable to load '/usr/share/zoneinfo/iso3166.tab' as time zone. Skipping it.
mysql-demo-container_1 | Warning: Unable to load '/usr/share/zoneinfo/leap-seconds.list' as time zone. Skipping it.
spring-boot-jpa-app_1 | 2019-07-02 13:03:25.161 ERROR 1 --- [ main] o.a.tomcat.jdbc.pool.ConnectionPool : Unable to create initial connections of pool.
spring-boot-jpa-app_1 |
spring-boot-jpa-app_1 | com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure
spring-boot-jpa-app_1 |
spring-boot-jpa-app_1 | The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
spring-boot-jpa-app_1 | at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[na:1.8.0_111]
spring-boot-jpa-app_1 | at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[na:1.8.0_111]
spring-boot-jpa-app_1 | at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[na:1.8.0_111]
spring-boot-jpa-app_1 | at java.lang.reflect.Constructor.newInstance(Constructor.java:423) ~[na:1.8.0_111]
spring-boot-jpa-app_1 | at com.mysql.jdbc.Util.handleNewInstance(Util.java:425) ~[mysql-connector-java-5.1.44.jar!/:5.1.44]
spring-boot-jpa-app_1 | at com.mysql.jdbc.SQLError.createCommunicationsException(SQLError.java:989) ~[mysql-connector-java-5.1.44.jar!/:5.1.44]
spring-boot-jpa-app_1 | at com.mysql.jdbc.MysqlIO.<init>(MysqlIO.java:341) ~[mysql-connector-java-5.1.44.jar!/:5.1.44]
spring-boot-jpa-app_1 | at com.mysql.jdbc.ConnectionImpl.coreConnect(ConnectionImpl.java:2189) ~[mysql-connector-java-5.1.44.jar!/:5.1.44]
spring-boot-jpa-app_1 | at com.mysql.jdbc.ConnectionImpl.connectOneTryOnly(ConnectionImpl.java:2222) ~[mysql-connector-java-5.1.44.jar!/:5.1.44]
spring-boot-jpa-app_1 | at com.mysql.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:2017) ~[mysql-connector-java-5.1.44.jar!/:5.1.44]
spring-boot-jpa-app_1 | at com.mysql.jdbc.ConnectionImpl.<init>(ConnectionImpl.java:779) ~[mysql-connector-java-5.1.44.jar!/:5.1.44]
spring-boot-jpa-app_1 | at com.mysql.jdbc.JDBC4Connection.<init>(JDBC4Connection.java:47) ~[mysql-connector-java-5.1.44.jar!/:5.1.44]
spring-boot-jpa-app_1 | at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[na:1.8.0_111]
spring-boot-jpa-app_1 | at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[na:1.8.0_111]
spring-boot-jpa-app_1 | at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[na:1.8.0_111]
spring-boot-jpa-app_1 | at java.lang.reflect.Constructor.newInstance(Constructor.java:423) ~[na:1.8.0_111]
spring-boot-jpa-app_1 | at com.mysql.jdbc.Util.handleNewInstance(Util.java:425) ~[mysql-connector-java-5.1.44.jar!/:5.1.44]
spring-boot-jpa-app_1 | at com.mysql.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:389) ~[mysql-connector-java-5.1.44.jar!/:5.1.44]
spring-boot-jpa-app_1 | at com.mysql.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:330) ~[mysql-connector-java-5.1.44.jar!/:5.1.44]
spring-boot-jpa-app_1 | at org.apache.tomcat.jdbc.pool.PooledConnection.connectUsingDriver(PooledConnection.java:310) ~[tomcat-jdbc-8.5.23.jar!/:na]
spring-boot-jpa-app_1 | at org.apache.tomcat.jdbc.pool.PooledConnection.connect(PooledConnection.java:203) ~[tomcat-jdbc-8.5.23.jar!/:na]
spring-boot-jpa-app_1 | at org.apache.tomcat.jdbc.pool.ConnectionPool.createConnection(ConnectionPool.java:735) [tomcat-jdbc-8.5.23.jar!/:na]
spring-boot-jpa-app_1 | at org.apache.tomcat.jdbc.pool.ConnectionPool.borrowConnection(ConnectionPool.java:667) [tomcat-jdbc-8.5.23.jar!/:na]
spring-boot-jpa-app_1 | at org.apache.tomcat.jdbc.pool.ConnectionPool.init(ConnectionPool.java:482) [tomcat-jdbc-8.5.23.jar!/:na]
spring-boot-jpa-app_1 | at org.apache.tomcat.jdbc.pool.ConnectionPool.<init>(ConnectionPool.java:154) [tomcat-jdbc-8.5.23.jar!/:na]
spring-boot-jpa-app_1 | at org.apache.tomcat.jdbc.pool.DataSourceProxy.pCreatePool(DataSourceProxy.java:118) [tomcat-jdbc-8.5.23.jar!/:na]
spring-boot-jpa-app_1 | at org.apache.tomcat.jdbc.pool.DataSourceProxy.createPool(DataSourceProxy.java:107) [tomcat-jdbc-8.5.23.jar!/:na]
spring-boot-jpa-app_1 | at org.apache.tomcat.jdbc.pool.DataSourceProxy.getConnection(DataSourceProxy.java:131) [tomcat-jdbc-8.5.23.jar!/:na]
spring-boot-jpa-app_1 | at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:111) [spring-jdbc-4.3.13.RELEASE.jar!/:4.3.13.RELEASE]
spring-boot-jpa-app_1 | at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:77) [spring-jdbc-4.3.13.RELEASE.jar!/:4.3.13.RELEASE]
spring-boot-jpa-app_1 | at org.springframework.jdbc.support.JdbcUtils.extractDatabaseMetaData(JdbcUtils.java:326) [spring-jdbc-4.3.13.RELEASE.jar!/:4.3.13.RELEASE]
spring-boot-jpa-app_1 | at org.springframework.jdbc.support.JdbcUtils.extractDatabaseMetaData(JdbcUtils.java:366) [spring-jdbc-4.3.13.RELEASE.jar!/:4.3.13.RELEASE]
spring-boot-jpa-app_1 | at org.springframework.boot.autoconfigure.orm.jpa.DatabaseLookup.getDatabase(DatabaseLookup.java:72) [spring-boot-autoconfigure-1.5.9.RELEASE.jar!/:1.5.9.RELEASE]
spring-boot-jpa-app_1 | at org.springframework.boot.autoconfigure.orm.jpa.JpaProperties.determineDatabase(JpaProperties.java:139) [spring-boot-autoconfigure-1.5.9.
As you can see from the beginning of the log, the containers seem created successfully.
Run docker ps shows me:
docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
8fa64acb363d mysql:latest "docker-entrypoint.s…" 30 minutes ago Up 30 minutes 33060/tcp, 0.0.0.0:2012->3306/tcp spring-boot-data-jpa-example-master_mysql-demo-container_1
What could be the reason for the error?

Answer 1

You should be aware of this very important footnote in the reference documentation of Docker Compose:
depends_on does not wait for db and redis to be “ready” before starting web - only until they have been started. If you need to wait for a service to be ready, see Controlling startup order for more on this problem and strategies for solving it.
In your case, your Spring boot container starts right after your MySQL container is being started. There's no guarantee though that MySQL will be able to take connections when Spring boot is setting up its connection pool.
This means that your Spring boot connection will fail to startup, because MySQL isn't ready yet. This results in your application container to be killed.
The solution to this problem is mentioned within the documentation I quoted. Namely, you have to use some kind of polling mechanism to see if you can connect to your database or not.
If you don't want to use tools like wait-for-it or wait-for, you could write a simple Shell script that does something like this:
while ! exec 6<>/dev/tcp/mysql-demo-container/3306; do
echo "Trying to connect to MySQL..."
sleep 10
done
exec java -jar spring-boot-data-jpa-example-0.0.1-SNAPSHOT.jar
This also means you have to change the ENTRYPOINT within your Dockerfile to refer to such shell script rather than directly running your JAR file.