I have a springboot application that is connected to a mysql database. I want to use the .gitlab-ci.yml to handle the pipeline. I created the following config file.
before_script:
- echo "Execute scripts which are required to bootstrap the application. !"
after_script:
- echo "Clean up activity can be done here !."
services:
- mysql
stages:
- build
- connect
- test
- package
- deploy
variables:
MAVEN_CLI_OPTS: "--batch-mode"
MAVEN_OPTS: "-Dmaven.repo.local=.m2/repository"
MYSQL_DATABASE_NAME: gyyconsortiumdb
MYSQL_DATABASE_SCHEMA: "$CI_PROJECT_DIR/src/main/resources/static/sql/gyyconsortiumdb.sql"
MYSQL_ROOT_PASSWORD: mysql
cache:
paths:
- .m2/repository/
- target/
build:
stage: build
image: maven:latest
script:
- mvn $MAVEN_CLI_OPTS clean compile
connect:
stage: connect
image: mysql
before_script:
- mysql --version
script:
- echo "create database $MYSQL_DATABASE_NAME;" | mysql --user=root --password="$MYSQL_ROOT_PASSWORD" --host=mysql
- mysql --user=root --password="$MYSQL_ROOT_PASSWORD" --host=mysql $MYSQL_DATABASE_NAME < $MYSQL_DATABASE_SCHEMA
- mysql --user=root --password="$MYSQL_ROOT_PASSWORD" --host=mysql -e "show databases; use $MYSQL_DATABASE_NAME; show tables;"
test:
stage: test
image: maven:latest
script:
- mvn $MAVEN_CLI_OPTS test
package:
stage: package
image: maven:latest
script:
- mvn $MAVEN_CLI_OPTS package
artifacts:
paths: [target/basecamp-0.0.1.war]
deploy_test:
stage: deploy
script:
- echo "######## To be defined ########"
environment: staging
deploy_prod:
stage: deploy
script:
- echo "######## To be defined ########"
only:
- master
environment: production
When the first job is running, it is pulling the mysql image from docker hub, and I don't know why? I thought the mysql should be built only on the connect job.
Also, after the connect job is success, the test job pull the mysql image again. And my test job doesn't recognize my database however the service was create successfully and I was able to see the database also created.
Build job console output:
Running with gitlab-runner 10.3.0 (5cf5e19a)
on docker-auto-scale (fa6cab46)
Using Docker executor with image maven:latest ...
Starting service mysql:latest ...
Pulling docker image mysql:latest ...
Using docker image mysql:latest ID=sha256:7d83a47ab2d2d0f803aa230fdac1c4e53d251bfafe9b7265a3777bcc95163755 for mysql service...
Waiting for services to be up and running...
Using docker image sha256:d6e999707ab00f954f4ab77c8ced4efce186099fb8318e1a0bc2f4fd8bb7bf6b for predefined container...
Pulling docker image maven:latest ...
Using docker image maven:latest ID=sha256:1f858e89a5843b0804a9e1498476a135f45e23a42fe673914f977e0882a6789e for build container...
Running on runner-fa6cab46-project-4899225-concurrent-0 via runner-fa6cab46-srm-1514081729-3d654341...
Cloning repository...
Cloning into '/builds/user/xxx'...
Checking out b9efb2da as 24-create-the-gitlab-ci-yml-file...
Skipping Git submodules setup
Checking cache for default...
Downloading cache.zip from http://runners-cache-3-internal.gitlab.com:444/runner/project/4899225/default
Successfully extracted cache
$ echo "Execute scripts which are required to bootstrap the application. !"
Execute scripts which are required to bootstrap the application. !
$ mvn $MAVEN_CLI_OPTS clean compile
[INFO] Scanning for projects...
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building xxx 0.0.1-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- maven-clean-plugin:2.6.1:clean (default-clean) # xxx ---
[INFO] Deleting /builds/user/xxx/target
[INFO]
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) # xxx ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 4 resources
[INFO] Copying 108 resources
[INFO]
[INFO] --- maven-compiler-plugin:3.1:compile (default-compile) # xxx ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 35 source files to /builds/user/xxx/target/classes
[WARNING] /builds/user/xxx/src/main/java/com//www/xxx/domain/Project.java: Some input files use unchecked or unsafe operations.
[WARNING] /builds/user/xxx/src/main/java/com/xxx/www/xxx/domain/Project.java: Recompile with -Xlint:unchecked for details.
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 6.111 s
[INFO] Finished at: 2017-12-24T02:18:07Z
[INFO] Final Memory: 30M/71M
[INFO] ------------------------------------------------------------------------
Running after script...
$ echo "Clean up activity can be done here !."
Clean up activity can be done here !.
Creating cache default...
.m2/repository/: found 1775 matching files
target/: found 197 matching files
Uploading cache.zip to http://runners-cache-3-
internal.gitlab.com:444/runner/project/4899225/default
Created cache
Job succeeded
Connect job console output
Running with gitlab-runner 10.3.0 (5cf5e19a)
on docker-auto-scale (e11ae361)
Using Docker executor with image mysql ...
Starting service mysql:latest ...
Pulling docker image mysql:latest ...
Using docker image mysql:latest ID=sha256:7d83a47ab2d2d0f803aa230fdac1c4e53d251bfafe9b7265a3777bcc95163755 for mysql service...
Waiting for services to be up and running...
Using docker image sha256:295a9e80fe6ae475bfeef8e318eb11db267d317fa6bc1ed8d72185dfd2adb8b7 for predefined container...
Pulling docker image mysql ...
Using docker image mysql ID=sha256:7d83a47ab2d2d0f803aa230fdac1c4e53d251bfafe9b7265a3777bcc95163755 for build container...
Running on runner-e11ae361-project-4899225-concurrent-0 via runner-e11ae361-srm-1514081811-5092ac06...
Cloning repository...
Cloning into '/builds/yimengael/basecamp'...
Checking out b9efb2da as 24-create-the-gitlab-ci-yml-file...
Skipping Git submodules setup
Checking cache for default...
Downloading cache.zip from http://runners-cache-5-internal.gitlab.com:444/runner/project/4899225/default
Successfully extracted cache
$ mysql --version
mysql Ver 14.14 Distrib 5.7.20, for Linux (x86_64) using EditLine wrapper
$ echo "create database $MYSQL_DATABASE_NAME;" | mysql --user=root --password="$MYSQL_ROOT_PASSWORD" --host=mysql
mysql: [Warning] Using a password on the command line interface can be insecure.
$ mysql --user=root --password="$MYSQL_ROOT_PASSWORD" --host=mysql $MYSQL_DATABASE_NAME < $MYSQL_DATABASE_SCHEMA
mysql: [Warning] Using a password on the command line interface can be insecure.
$ mysql --user=root --password="$MYSQL_ROOT_PASSWORD" --host=mysql -e "show databases; use $MYSQL_DATABASE_NAME; show tables;"
mysql: [Warning] Using a password on the command line interface can be insecure.
Database
information_schema
mydb
mysql
performance_schema
sys
Tables_in_mydb
message
persistent_logins
project
project_description
role
user
user_role
Running after script...
$ echo "Clean up activity can be done here !."
Clean up activity can be done here !.
Creating cache default...
.m2/repository/: found 1775 matching files
target/: found 197 matching files
Archive is up to date!
Created cache
Job succeeded
Test job failed :
2017-12-24 02:21:43.827 INFO 62 --- [ main] c.g.w.b.xxxWebApplicationTests : Starting xxxWebApplicationTests on runner-4e4528ca-project-4899225-concurrent-0 with PID 62 (started by root in /builds/yimengael/xxx)
2017-12-24 02:21:43.829 DEBUG 62 --- [ main] c.g.w.b.xxxWebApplicationTests : Running with Spring Boot v1.5.8.RELEASE, Spring v4.3.12.RELEASE
2017-12-24 02:21:43.829 INFO 62 --- [ main] c.g.w.b.xxxWebApplicationTests : The following profiles are active: dev
2017-12-24 02:21:45.101 INFO 62 --- [ main] o.s.w.c.s.GenericWebApplicationContext : Refreshing org.springframework.web.context.support.GenericWebApplicationContext#62e20a76: startup date [Sun Dec 24 02:21:45 UTC 2017]; root of context hierarchy
2017-12-24 02:21:49.515 INFO 62 --- [ main] f.a.AutowiredAnnotationBeanPostProcessor : JSR-330 'javax.inject.Inject' annotation found and supported for autowiring
2017-12-24 02:21:51.533 WARN 62 --- [ main] o.a.tomcat.jdbc.pool.ConnectionPool : maxIdle is larger than maxActive, setting maxIdle to: 50
2017-12-24 02:21:52.448 ERROR 62 --- [ main] o.a.tomcat.jdbc.pool.ConnectionPool : Unable to create initial connections of pool.
com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Unknown database 'mydb'
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
.....
2017-12-24 02:21:52.477 WARN 62 --- [ main] o.s.b.a.orm.jpa.DatabaseLookup : Unable to determine jdbc url from datasource
org.springframework.jdbc.support.MetaDataAccessException: Could not get Connection for extracting meta data; nested exception is
org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get JDBC Connection; nested exception is com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Unknown database 'mydb'
at org.springframework.jdbc.support.JdbcUtils.extractDatabaseMetaData(JdbcUtils.java:339)
at org.springframework.jdbc.support.JdbcUtils.extractDatabaseMetaData(JdbcUtils.java:366)
at org.springframework.boot.autoconfigure.orm.jpa.DatabaseLookup.getDatabase(DatabaseLookup.java:72)
at org.springframework.boot.autoconfigure.orm.jpa.JpaProperties.determineDatabase(JpaProperties.java:139)
Could you please tell me why the test job does not pick up the mysql service the right way ?
Best,
G
When the first job is running, it is pulling the mysql image from
docker hub, and I don't know why?
Because you defined Mysql as a service. GitLab CI uses the services keyword to define what docker containers should be linked with your base image. So, it will be run for every stages.
Could you please tell me why the test job does not pick up the mysql
service the right way ?
Your database name in your gitlab config file is 'gyyconsortiumdb' which is different from what you defined in your spring config file as log shows you are trying to connect to 'mydb'
jdbc4.MySQLSyntaxErrorException: Unknown database 'mydb'
follow the link for more info
https://docs.gitlab.com/ce/ci/services/mysql.html
Related
This is not my first time creating an ndb cluster but I hadn't received such a problem.
I am following this manual by team mysql.
I am using the default configuration echoed in this GitHub repository.
When I start my ndb_mgmd container, it returns the following error:
$ docker run -d --net=cluster --name=management1 --ip=192.168.0.2 mysql/mysql-cluster ndb_mgmd
The log I receive is:
$ docker logs management1
It returns the following lines
[Entrypoint] MySQL Docker Image 8.0.24-1.2.2-cluster
[Entrypoint] Starting ndb_mgmd
Failed to open /sys/devices/system/cpu/cpu0/cache/index3/shared_cpu_list: No such file or directory
ndb_mgmd: [ERROR] unknown variable 'user='.
MySQL Cluster Management Server mysql-8.0.24 ndb-8.0.24
and the container exits immediately. I am not sure if it is a bug, thus I write it in SO.
It is a bug.
I run plain echo and --user= is appended to the commandline by docker image.
$ docker run -d --net=cluster --name=management1 --ip=192.168.0.2 mysql/mysql-cluster echo
55b11ea72989fad50b29fe199ad54ebe2a919079770d0188512a465699e8a256
$ docker logs management1
[Entrypoint] MySQL Docker Image 8.0.24-1.2.2-cluster
--user=
Probably some workaround that works starting the MySQL server, but not suitable for Ndb programs.
Edit#1:
Until new docker images are fixed and out you can try the below workaround.
It overrides the faulty entrypoint script when starting the ndb processes with --entrypoint=/usr/bin/env (note, must be before image mysql/mysql-cluster).
And using explicit command line options extracted from entrypoint script at https://github.com/mysql/mysql-docker/blob/main/mysql-cluster/8.0/docker-entrypoint.sh .
For management server override entrypoint and add -f /etc/mysql-cluster.cnf --nodaemon:
$ docker run -d --net=cluster --name=management1 --ip=192.168.0.2 --entrypoint=/usr/bin/env mysql/mysql-cluster ndb_mgmd -f /etc/mysql-cluster.cnf --nodaemon
The two data nodes (I choose ndbmtd instead of ndbd) override entrypoint and add --nodaemon:
$ docker run -d --net=cluster --name=ndb1 --ip=192.168.0.3 --entrypoint=/usr/bin/env mysql/mysql-cluster ndbmtd --nodaemon
$ docker run -d --net=cluster --name=ndb2 --ip=192.168.0.4 --entrypoint=/usr/bin/env mysql/mysql-cluster ndbmtd --nodaemon
When starting the mysqld you should not override the entrypoint, stick to the manual.
Edit#2:
The mysql-cluster docker image is now fixed, pull the new image and recreate containers.
$ docker pull mysql/mysql-cluster
Using default tag: latest
latest: Pulling from mysql/mysql-cluster
Digest: sha256:a8ae8a4358f0c2f07aa39df046eb81e8f88cb2bebcaaf436c67663b300a1e1fe
Status: Image is up to date for mysql/mysql-cluster:latest
docker.io/mysql/mysql-cluster:latest
$ docker run -d --net=cluster --name=management1 --ip=192.168.0.2 mysql/mysql-cluster ndb_mgmd
715ad773b51b3d8fefcf6230460b6149a0a0226ee604752352b9e88d8dfa5bb8
$ docker logs management1
[Entrypoint] MySQL Docker Image 8.0.25-1.2.3-cluster
[Entrypoint] Starting ndb_mgmd
MySQL Cluster Management Server mysql-8.0.25 ndb-8.0.25
2021-05-12 07:59:21 [MgmtSrvr] INFO -- The default config directory '/usr/mysql-cluster' does not exist. Trying to create it...
2021-05-12 07:59:21 [MgmtSrvr] INFO -- Sucessfully created config directory
2021-05-12 07:59:21 [MgmtSrvr] WARNING -- at line 19: [DB] IndexMemory is deprecated, will use Number bytes on each ndbd(DB) node allocated for storing indexes instead
2021-05-12 07:59:21 [MgmtSrvr] INFO -- Got initial configuration from '/etc/mysql-cluster.cnf', will try to set it when all
ndb_mgmd(s) started
2021-05-12 07:59:21 [MgmtSrvr] INFO -- Node 1: Node 1 Connected
2021-05-12 07:59:21 [MgmtSrvr] INFO -- Id: 1, Command port: *:1186
==INITIAL==
2021-05-12 07:59:21 [MgmtSrvr] INFO -- MySQL Cluster Management Server mysql-8.0.25 ndb-8.0.25 started
2021-05-12 07:59:22 [MgmtSrvr] INFO -- Node 1 connected
2021-05-12 07:59:22 [MgmtSrvr] INFO -- Starting initial configuration change
2021-05-12 07:59:22 [MgmtSrvr] INFO -- Configuration 1 commited
2021-05-12 07:59:22 [MgmtSrvr] INFO -- Config change completed! New generation: 1
==CONFIRMED==
This proved to be a bug. Until being fixed, one can do the following:
1- Clone the mysql-docker
git clone https://github.com/mysql/mysql-docker.git
2- Checkout an earlier tag (1.1.19-cluster checked and verified)
git checkout 1.1.19-cluster
3- build the image and use it later.
docker build -t mysql-cluster-customized 8.0/
4- run management cluster
docker run -d --net=cluster --name=management1 --ip=192.168.0.2 -it mysql-cluster-customized ndb_mgmd
I hope it helps while mysql team is fixing the issue with the repository head.
I'm trying to run confluence with mysql using Docker Desktop on Windows 10 in a multi-container application using a docker-compose.yml file like this:
services:
confluence:
image: atlassian/confluence-server:7.4.6
build: ./confluence
container_name: confluence
hostname: confluence
volumes:
- ./confluence/data/confluence:/var/atlassian/application-data/confluence # for the confluence.cfg.xml
- ./confluence/lib:/opt/atlassian/confluence/confluence/WEB-INF/lib # for the mysql driver jar-file
ports:
- 8090:8090
- 8091:8091
mysql:
image: mysql:5.7
build: ./mysql
container_name: mysql
hostname: mysql
ports:
- 3306:3306
environment:
- MYSQL_ROOT_PASSWORD=root
command: [mysqld, --character-set-server=utf8, --collation-server=utf8_bin, --default-storage-engine=INNODB, --max_allowed_packet=256M, --innodb_log_file_size=2GB, --transaction-isolation=READ-COMMITTED, --binlog_format=row]
However when I run this with docker-compose up --build --force-recreate I get
INFO [main] org.apache.catalina.core.StandardEngine.startInternal Starting Servlet engine: [Apache Tomcat/9.0.33]
confluence | Handler error
confluence | java.lang.ClassNotFoundException: org.slf4j.bridge.SLF4JBridgeHandler
confluence | at ... at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
confluence | at java.base/java.lang.Thread.run(Thread.java:834)
confluence | SEVERE [Catalina-utility-1] org.apache.catalina.core.StandardContext.startInternal One or more listeners failed to start. Full details will be found in the appropriate container log file
confluence | SEVERE [Catalina-utility-1] org.apache.catalina.core.StandardContext.startInternal Context [] startup failed due to previous errors
confluence | INFO [Catalina-utility-2] org.apache.jasper.servlet.TldScanner.scanJars At least one JAR was scanned for TLDs yet contained no TLDs. Enable debug logging for this logger for a complete list of JARs that were scanned but no TLDs were found in them. Skipping unneeded JARs during scanning can improve startup time and JSP compilation time.
confluence | SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
confluence | SLF4J: Defaulting to no-operation (NOP) logger implementation
confluence | SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
When I run docker run -v d:/docker/conf-mysql/confluence/data/confluence:/var/atlassian/application-data/confluence --name="confluence" -p 8090:8090 -p 8091:8091 atlassian/confluence-server:7.4.6 I get
INFO [main] org.apache.catalina.core.StandardEngine.startInternal Starting Servlet engine: [Apache Tomcat/9.0.33]
INFO [Catalina-utility-2] org.apache.jasper.servlet.TldScanner.scanJars At least one JAR was scanned for TLDs yet contained no TLDs. Enable debug logging for this logger for a complete list of JARs that were scanned but no TLDs were found in them. Skipping unneeded JARs during scanning can improve startup time and JSP compilation time.
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
So: In both cases SLF4J falls back to NOP-mode but when I use docker-compose up I get a severe error and a 404 on localhost:8090 because Tomcat could not load SLF4J and when I use docker run confluence starts up on the URI, but then has no DBMS to connect to.
Any ideas? Please help! Thank you!
OK, that was easy:
I simply had to remove the line
./confluence/lib:/opt/atlassian/confluence/confluence/WEB-INF/lib
My initial understanding was that files bound to the container like that, would be added to and accessible through the folder ( like a link ).
Obviously "volumes" reroutes the whole path to the host and if there are files in the container's folder, they are not accessible any more, while the volume is bound to the container.
Hope this helps someone! :-)
I'm deploying my Java8 SpringBoot App to a Google Compute Engine instance and trying to connect it to a Debian9 CloudSQL instance. I'm trying to get the instance to run with my startup-script.sh, but when it tries to boot up the SpringBoot Application, according to the daemon.log, when the .war is ran by "java -jar order-routing-0.0.1-SNAPSHOT.war" the startup fails with a "Unable to obtain connection from database: Communications link failure", with SQL State:08S01 and error code 0.
I mapped the GCE instance to a static external IP, as well as whitelisting that IP on the CloudSQL instance's connections configs. I also verify that the war file runs locally with "java -jar order-routing.war".
Here is my startup script.sh:
#!/usr/bin/env bash
# This script is passed to the GCE instance by the setup script. It is run on the instance when it is spun up.
# Derived from GCE Tutorial at https://cloud.google.com/java/docs/tutorials/bookshelf-on-compute-engine
# [START script]
set -e
set -v
# Talk to the metadata server to get the project id
PROJECTID=$(curl -s "http://metadata.google.internal/computeMetadata/v1/project/project-id" -H "Metadata-Flavor: Google")
BUCKET=$(curl -s "http://metadata.google.internal/computeMetadata/v1/instance/attributes/BUCKET" -H "Metadata-Flavor: Google")
echo "Project ID: ${PROJECTID}"
# get our file(s)
gsutil cp "gs://order-routing-install/gce/"** .
# Install dependencies from apt
apt-get update
apt-get install mysql-client -y
wget https://dl.google.com/cloudsql/cloud_sql_proxy.linux.amd64
mv cloud_sql_proxy.linux.amd64 cloud_sql_proxy
chmod +x cloud_sql_proxy
./cloud_sql_proxy -instances=instance-qa1:us-central1:instance-qa1-cloudsql-0=tcp:3307 &
apt-get install -yq default-jre
apt-get install -yq default-jdk
java -jar order-routing-0.0.1-SNAPSHOT.war
# [END script]
Here is the failing error log from the daemon.log:
ConfigServletWebServerApplicationContext : Exception encountered during context initialization - cancelling refresh attempt: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'flywayInitializer' defined in class path resource [org/springframework/boot/autoconfigure/flyway/FlywayAutoConfiguration$FlywayConfiguration.class]: Invocation of init method failed; nested exception is org.flywaydb.core.internal.exception.FlywaySqlException:
Jul 29 03:39:30 order-routing-group-hk66 startup-script: INFO startup-script: Unable to obtain connection from database: Communications link failure
Jul 29 03:39:30 order-routing-group-hk66 startup-script: INFO startup-script: 2019-07-29 03:39:30.435 INFO 9051 --- [ main] ConditionEvaluationReportLoggingListener :
Jul 29 03:39:30 order-routing-group-hk66 startup-script: INFO startup-script:n from database: Communications link failure
Jul 29 03:39:30 order-routing-group-hk66 startup-script: INFO startup-script: The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
Jul 29 03:39:30 order-routing-group-hk66 startup-script: INFO startup-script: SQL State : 08S01
Jul 29 03:39:30 order-routing-group-hk66 startup-script: INFO startup-script: Error Code : 0
Jul 29 03:39:30 order-routing-group-hk66 startup-script: INFO startup-script: Message : Communications link failure
Expected Result:
The GCE instance starts up the war file successfully and I can access my app using the external IP.
Actual Result:
Getting CommunicationsLinkFailures upon starting up Spring Boot Java app.
I am building an application that uses nodeJS and backend and mySQL as backend, and currently, my steps to bring up the app (without docker) is by:
Install NodeJS
Install MYSQL
Launch mysqld on port 3306
Manually create a MYSQL user dedicated for the NodeJS backend. This
user should have only basic previliges to only my desired schema.
Run sequelize commands to perform data migration and seeding using
the user generated in 4)
npm install and npm start to launch NodeJS on port 8080
Now I want to dokerize my application, and I already have the following Dockerfile:
#node version: carbon
#app version: 1.0.0
FROM node:8.11.2
WORKDIR /usr/src/app
COPY package*.json ./
RUN npm install
COPY . .
EXPOSE 8080
I have put a init.sql file within ./docker_db folder which does the following:
CREATE USER 'app_user'#'%' IDENTIFIED BY 'password';
CREATE SCHEMA `myapp` DEFAULT CHARACTER SET utf8;
GRANT INSERT, CREATE, ALTER, UPDATE, SELECT, REFERENCES on myapp.*
TO 'app_user'#'%' IDENTIFIED BY 'password'
WITH GRANT OPTION;
and the following docker-compose.yaml:
version: '3.6'
services:
mysql1:
image: mysql/mysql-server:5.7
environment:
MYSQL_ROOT_PASSWORD: password
ports:
- "127.0.0.1:3306:3306"
volumes:
- type: bind
source: ./docker_db
target: /docker-entrypoint-initdb.d
expose:
- "3306"
networks:
- app-network
myapp:
build:
context: .
dockerfile: Dockerfile
command: npm start
depends_on:
- mysql1
ports:
- "127.0.0.1:8080:8080"
expose:
- "8080"
links:
- mysql1
networks:
- app-network
command: ["./wait-for-db.sh"]
networks:
app-network:
driver: bridge
where my ./wait-for-db.sh does the following:
#!/bin/bash
until mysql -h mysql1 -u app_user -p password -e 'select 1'; do
echo "still waiting for mysql"; sleep 1; done
exec node ./db/scripts/generateSequelizeCLIConfig.js
exec node_modules/sequelize-cli/bin/sequelize db:migrate
exec node_modules/sequelize-cli/bin/sequelize db:seed:all
exec npm start
(BTW I do want to expose 3306 to host machine so that I can use workbench to connect to the mysql server, which I have successfully connected.)
In my sequelize config file I do have:
"username": "app_user",
"password": "password",
"database": "myapp",
"host": "mysql1",
"port": "3306"
With the above setting, I executed docker-compose up, and then I got the following lines:
mysql1_1 | [Entrypoint] MySQL Docker Image 5.7.22-1.1.5
mysql1_1 | [Entrypoint] Initializing database
myapp_1 | standard_init_linux.go:190: exec user process caused "no such file or directory"
myapp_myapp_1 exited with code 1
mysql1_1 | [Entrypoint] Database initialized
mysql1_1 | Warning: Unable to load '/usr/share/zoneinfo/iso3166.tab' as time zone. Skipping it.
mysql1_1 | Warning: Unable to load '/usr/share/zoneinfo/leapseconds' as time zone. Skipping it.
mysql1_1 | Warning: Unable to load '/usr/share/zoneinfo/tzdata.zi' as time zone. Skipping it.
mysql1_1 | Warning: Unable to load '/usr/share/zoneinfo/zone.tab' as time zone. Skipping it.
mysql1_1 | Warning: Unable to load '/usr/share/zoneinfo/zone1970.tab' as time zone. Skipping it.
mysql1_1 |
mysql1_1 | [Entrypoint] running /docker-entrypoint-initdb.d/init.sql
mysql1_1 |
mysql1_1 |
mysql1_1 | [Entrypoint] Server shut down
mysql1_1 |
mysql1_1 | [Entrypoint] MySQL init process done. Ready for start up.
mysql1_1 |
mysql1_1 | [Entrypoint] Starting MySQL 5.7.22-1.1.5
The problems now I face are:
1) The script's execution is hanging on the last line of Starting MySQL 5.7.22-1.1.5 and not going anywhere.
2) In the output, the 3rd and 4th lines shows an error about exec user process caused "no such file or directory". I don't think it is caused by the commands in the wait-for-db.sh because if I removed the lines after the until command, the problem still persist. In fact, I doubt the command execution ever reaching those lines and it feels like it is still within the until command.
I think it's really close to the final solution though :)
Use the name of your db service, which is mysql, as your database host. Docker will resolve it to the actually IP. Also why do you have FROM mysql:5.7 in your Dockerfile, I don't think it is of any uses.
Updated
Alright, seems like myapp runs db scripts before the db is ready. See here for solution https://docs.docker.com/compose/startup-order/
The problem is probably related to timing. Both containers will start at the same time and your node-app will try to connect to mysql almost immediately, while the MySQL server is still starting.
docker-compose doesn't have any kind of structure for this so you will have to build an entrypoint in your node-app that first waits for mysql to respond.
So, in your case, the entrypoint would be something like
#!/bin/bash
until mysql -h mysql1 -uapp_user -ppassword -e'select 1'; do echo "still waiting for mysql"; sleep 1; done
exec npm start
I used docker-machine with Azure as the driver to spin up a VM. I then deployed a simple nginx test container on to the host. My issue is that when I try to set and endpoint I am getting the following error:
azure vm endpoint create huldra 80 32769
info: Executing command vm endpoint create
+ Getting virtual machines
+ Reading network configuration
+ Updating network configuration
error: Parameter 'ConsoleScreenshotBlobUri' should not be set.
info: Error information has been recorded to /Users/ryan/.azure/azure.err
error: vm endpoint create command failed
When I look at the error log it pretty much repeats what the console said Parameter 'ConsoleScreenshotBlobUri' should not be set.
Here are my docker and azure environment details:
❯ docker info
Containers: 1
Running: 1
Paused: 0
Stopped: 0
Images: 3
Server Version: 1.10.2
Storage Driver: aufs
Root Dir: /var/lib/docker/aufs
Backing Filesystem: extfs
Dirs: 21
Dirperm1 Supported: true
Execution Driver: native-0.2
Logging Driver: json-file
Plugins:
Volume: local
Network: bridge null host
Kernel Version: 4.2.0-18-generic
Operating System: Ubuntu 15.10
OSType: linux
Architecture: x86_64
CPUs: 1
Total Memory: 1.636 GiB
Name: huldra
ID: PHUY:JRE3:DOJO:NNWO:JBBH:42H2:56ZO:HVSB:MZDE:QLOI:GO6F:SCC5
WARNING: No swap limit support
Labels:
provider=azure
~/Projects/dockerswarm master*
❯ docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
ce51127b2bb8 nginx "nginx -g 'daemon off" 11 minutes ago Up 11 minutes 0.0.0.0:32769->80/tcp, 0.0.0.0:32768->443/tcp machinenginx
❯ azure --version
0.9.17 (node: 5.8.0)
❯ azure vm list
info: Executing command vm list
+ Getting virtual machines
data: Name Status Location DNS Name IP Address
data: ------ --------- -------- ------------------- -------------
data: huldra ReadyRole West US huldra.cloudapp.net x.x.x.x
info: vm list command OK