Hello I have a configuration that builds docker containers for a flask app and a mysql instance.
I create a new VM with
docker-machine create -d amazonec2 --....... production
and then (after setting the correct environment)
docker-compose build -> docker-compose up -d
The problem is that all these happen whilst CWD is a local repo with the files I need. It turns out these files are not copied over.
I have looked at docker cp and docker scp but it seems they do not solve the problem. E.g. with SCP I cannot reference the specific machine I need to copy the repo over (xow_web_1)
Here is the .yml
web:
restart: always
volumes:
- .:/xow
build: .
ports:
- "80:80"
links:
- db
hostname: xowflask
command: python xow.py
db:
restart: always
hostname: xowmysql
image: mysql:latest
ports:
- "3306:3306"
environment:
MYSQL_ROOT_PASSWORD: somepasswordhere
MYSQL_DATABASE: somedatabase
data:
restart: always
image: mysql:latest
volumes:
- /var/lib/mysql
command: "true"
How would be the most appropriate way to solve this? Is docker-compose the right approach? Looks awesome, but it doesn't solve an issue like this
The way we solved it in our organization is by using the COPY command to copy all of the data in the folder to the container.
For example, copying all of the files from the current dir to the container /src folder will look like this -
### Copy Code
COPY . /src
It looks like you should add this line into the web container in your docker-compose configuration.
Related
I have a project that runs when ./entrypoint.sh or docker-compose up is run from the root directory of project and generates the swagger API interface, but the calls return entry response no data.
If I run with MySQL on localhost without docker, works perfectly fine. How do I load the data?
entrypoint.sh
#!/bin/bash
docker network create turingmysql
docker container run -p 3306:3306 --name mysqldb --network turingmysql -e MYSQL_ROOT_PASSWORD=root -e MYSQL_DATABASE=tshirtshop -d mysql:5.7
docker-compose build
docker-compose up
DockerFile
FROM mysql:5.7
ADD ./database/tshirtshop.sql /docker-entrypoint-initdb.d
#### Stage 1: Build the application
FROM openjdk:8-jdk-alpine as build
# Set the current working directory inside the image
WORKDIR /app
# Copy maven executable to the image
COPY mvnw .
COPY .mvn .mvn
# Copy the pom.xml file
COPY pom.xml .
# Build all the dependencies in preparation to go offline.
# This is a separate step so the dependencies will be cached unless
# the pom.xml file has changed.
RUN ./mvnw dependency:go-offline -B
# Copy the project source
COPY src src
# Package the application
RUN ./mvnw package -DskipTests
RUN mkdir -p target/dependency && (cd target/dependency; jar -xf ../*.jar)
#### Stage 2: A minimal docker image with command to run the app
FROM openjdk:8-jre-alpine
ARG DEPENDENCY=/app/target/dependency
# Copy project dependencies from the build stage
COPY --from=build ${DEPENDENCY}/BOOT-INF/lib /app/lib
COPY --from=build ${DEPENDENCY}/META-INF /app/META-INF
COPY --from=build ${DEPENDENCY}/BOOT-INF/classes /app
ENTRYPOINT ["java","-cp","app:app/lib/*","com.turing.ecommerce.TuringApplication"]
docker-compose.yml
version: '3.7'
# Define services
services:
# App backend service
app-server:
# Configuration for building the docker image for the backend service
build:
context: .
dockerfile: Dockerfile
ports:
- "8080:8080" # Forward the exposed port 8080 on the container to port 8080 on the host machine
restart: always
depends_on:
- mysqldb # This service depends on mysql. Start that first.
environment: # Pass environment variables to the service
SPRING_DATASOURCE_URL: jdbc:mysql://mysqldb:3306/tshirtshop?useSSL=false&useLegacyDatetimeCode=false&serverTimezone=UTC
SPRING_DATASOURCE_USERNAME: root
SPRING_DATASOURCE_PASSWORD: root
networks: # Networks to join (Services on the same network can communicate with each other using their name)
- turingmysql
# Database Service (Mysql)
mysqldb:
image: mysql:5.7
ports:
- "3306:3306"
restart: always
environment:
MYSQL_DATABASE: tshirtshop
MYSQL_USER: root
MYSQL_ROOT_PASSWORD: root
volumes:
- db-data:/var/lib/mysql
networks:
- turingmysql
# Volumes
volumes:
db-data:
# Networks to be created to facilitate communication between containers
networks:
turingmysql:
Do you have two Dockerfiles? Looks like you built your own MySQL container?
Otherwise, these shouldn't be part of your Java multi-stage build
FROM mysql:5.7
ADD ./database/tshirtshop.sql /docker-entrypoint-initdb.d
Assuming that you did build a separate image for mysql, in the Docker-Compose, you're not using it, as you're still referring to image: mysql:5.7
Rather than building your own, you should mount the SQL script into it
For example
mysqldb:
image: mysql:5.7
...
volumes:
- db-data:/var/lib/mysql
- ./database/tshirtshop.sql:/docker-entrypoint-initdb.d/0_init.sql
Then, forget the Java service for a minute and use MySQL workbench or the mysql CLI to verify that data is actually there. Once you do, then startup the API
If you copying sql scipt already to docker build then you do not need to mapped it again in the docker-compose, if you have docker-compose then you do not the bash script single command docker-compose up --build will do the job.
So modify your docker-compose as per your Dockerfile.
Dockerfile
FROM mysql
ADD init.sql /docker-entrypoint-initdb.d
docker-compose
version: '3.7'
services:
# App backend service
app-server:
# Configuration for building the docker image for the backend service
build:
context: .
dockerfile: Dockerfile
ports:
- "8080:8080" # Forward the exposed port 8080 on the container to port 8080 on the host machine
restart: always
depends_on:
- mysqldb # This service depends on mysql. Start that first.
environment: # Pass environment variables to the service
SPRING_DATASOURCE_URL: jdbc:mysql://mysqldb:3306/tshirtshop?useSSL=false&useLegacyDatetimeCode=false&serverTimezone=UTC
SPRING_DATASOURCE_USERNAME: root
SPRING_DATASOURCE_PASSWORD: root
networks: # Networks to join (Services on the same network can communicate with each other using container name)
- uringmysql
# Database Service (Mysql)
mysql:
build: .
environment:
MYSQL_ROOT_PASSWORD: root123
MYSQL_DATABASE: appdata
MYSQL_USER: test
MYSQL_PASSWORD: root123
volumes:
- db-data:/var/lib/mysql
tty: true
# Volumes
volumes:
db-data:
# Networks to be created to facilitate communication between containers
networks:
turingmysql:
Now just run
docker-compose up --build
this will build and up the container and you will not need to mapped the host init script, as it already in Docker image.
The directory structure will look like
Now you application will able to access DB using jdbc:mysql://mysqldb:3306/tshirtshop? this endpoint as both are in same network and can refer eacher other using name.
Thank you cricket_007 and Adii for the responses. They put me in the right direction. I want to document my experience and how the issue was resolved. New to dockerization so I was learning by practice. For anyone new to dockerization and having same issues in Spring Boot, MySQL and docker, this would surely help
First, my entrypoint.sh changed below. The docker-compose down is for restarts.
#!/bin/bash
docker-compose down -v
docker-compose up --build
Second, I had to use an existing mysql image instead of building one.
version: '3.7'
# Define services
services:
# App backend service
app-server:
# Configuration for building the docker image for the backend service
build:
context: .
dockerfile: Dockerfile
ports:
- "8080:8080" # Forward the exposed port 8080 on the container to port 8080 on the host machine
restart: always
depends_on:
- mysql # This service depends on mysql. Start that first.
environment: # Pass environment variables to the service
SPRING_DATASOURCE_URL: jdbc:mysql://mysql:3306/tshirtshop?useSSL=false&allowPublicKeyRetrieval=true&useLegacyDatetimeCode=false&serverTimezone=UTC
SPRING_DATASOURCE_USERNAME: turing
SPRING_DATASOURCE_PASSWORD: pass
networks: # Networks to join (Services on the same network can communicate with each other using their name)
- turingmysql
# Database Service (Mysql)
mysql:
image: mysql/mysql-server
ports:
- "3306:3306"
environment:
MYSQL_ROOT_PASSWORD: root
MYSQL_DATABASE: tshirtshop
MYSQL_USER: turing
MYSQL_PASSWORD: pass
volumes:
- db-data:/var/lib/mysql
- ./database:/docker-entrypoint-initdb.d
tty: true
networks: # Networks to join (Services on the same network can communicate with each other using their name)
- turingmysql
# Volumes
volumes:
db-data:
# Networks to be created to facilitate communication between containers
networks:
turingmysql:
driver: bridge
Needed to specify that the network is a bridge. My sql file was mounted from a folder relative to docker-compose.yml. Also had to add allowPublicKeyRetrieval=true to my jdbc url. Created a user to access the database tshirtshop.
And here is the Dockerfile.
#### Stage 1: Build the application
FROM openjdk:8-jdk-alpine as build
# Set the current working directory inside the image
WORKDIR /app
# Copy maven executable to the image
COPY mvnw .
COPY .mvn .mvn
# Copy the pom.xml file
COPY pom.xml .
# Build all the dependencies in preparation to go offline.
# This is a separate step so the dependencies will be cached unless
# the pom.xml file has changed.
RUN ./mvnw dependency:go-offline -B
# Copy the project source
COPY src src
# Package the application
RUN ./mvnw package -DskipTests
RUN mkdir -p target/dependency && (cd target/dependency; jar -xf ../*.jar)
#### Stage 2: A minimal docker image with command to run the app
FROM openjdk:8-jre-alpine
ARG DEPENDENCY=/app/target/dependency
# Copy project dependencies from the build stage
COPY --from=build ${DEPENDENCY}/BOOT-INF/lib /app/lib
COPY --from=build ${DEPENDENCY}/META-INF /app/META-INF
COPY --from=build ${DEPENDENCY}/BOOT-INF/classes /app
ENTRYPOINT ["java","-cp","app:app/lib/*","com.turing.ecommerce.TuringApplication"]
to run, from root directory of project ./entrypoint.sh on mac and the rest is history.
I am trying to set up a mysql docker container and execute init sql script. Unfortunately the sql script is not executed. What am I doing wrong?
version: '3.3'
services:
api:
container_name: 'api'
build: './api'
ports:
- target: 8080
published: 8888
protocol: tcp
mode: host
volumes:
- './api:/go/src/app'
depends_on:
- 'mysql'
mysql:
image: 'mysql:latest'
container_name: 'mysql'
volumes:
- ./db_data:/var/lib/mysql:rw
- ./database/init.sql:/docker-entrypoint-initdb.d/init.sql:ro
restart: always
environment:
MYSQL_USER: test
MYSQL_PASSWORD: test
MYSQL_ROOT_PASSWORD: test
MYSQL_DATABASE: test
ports:
- '3306:3306'
volumes:
db_data:
I execute file with docker-compose up -d --build
The docker-entrypoint-initdb.d folder will only be run once while the container is created (instantiated) so you actually have to do a docker-compose down -v to re-activate this for the next run.
If you want to be able to add sql files at any moment you can look here at a specialized MySql docker image... http://ivo2u.nl/o4
Update for M1 arch:
Here an almost drop-in replacement in MariaDB: http://ivo2u.nl/V1
Many containerized applications, especially stateful ones, have a way of running init scripts (like the sql scripts here) and they are supposed to run only once.
And since they are stateful, the volumes are a source of truth for the containers on whether to run the init scripts or not on container restart.
Like in your case, deleting the folder used for bind mount or using a new named volume should re-run any init scripts present.
These scripts run when you create the container, not every time you start it.
You can docker-compose up --force-recreate mysql to force those scripts to re-run.
Additionally, if you have a volume like this ./db_data:/var/lib/mysql:rw, then you also need to remove ./db_data before recreating the container.
I'm not a docker expert, but this worked for me.
I am trying to use Docker to create a set of containers (wordpress and MySQL) that will help my local development with Wordpress. As we are running a live database, I want to mount a dump.sql file into the Docker mysql container. Below is my .yml file.
version: '2'
services:
db:
image: mysql:latest
volumes:
- ./data:/docker-entrypoint-initdb.d #./data holders my dump.sql file
environment:
MYSQL_ROOT_PASSWORD: wordpress
MYSQL_DATABASE: wordpress
MYSQL_USER: wordpress
MYSQL_PASSWORD: wordpress
wordpress:
depends_on:
- db
image: wordpress:latest
ports:
- "8000:80"
restart: always
environment:
WORDPRESS_DB_HOST: db:3306
WORDPRESS_DB_PASSWORD: wordpress
volumes:
- ./wp-content/themes/portalV3:/var/www/html/wp-content/themes/portalV3
- ./wp-content/plugins:/var/www/html/wp-content/plugins
- ./wp-content/uploads:/var/www/html/wp-content/uploads
Everything works, but after ~10 seconds the docker container for mysql crashes. Going through the logs, I get the following error:
/usr/local/bin/docker-entrypoint.sh: running /docker-entrypoint-initdb.d/dump.sql
mysql: [Warning] Using a password on the command line interface can be insecure.
ERROR: Can't initialize batch_readline - may be the input source is a directory or a block device.
On closer inspection (attaching to the rebooted mysql container) I see that indeed my dump.sql file wasn't transferred to the container, but a folder with the same name was created in /docker-entrypoint-initdb.d.
Can anyone help me understand how I get docker-compose to copy my dump.sql file and import into the database?
Cheers,
Pieter
The problem you got with docker-entrypoint-initdb.d is that because your source 'data' is a directory and not a file, The destination file (docker-entrypoint-initdb.d) must be a directory too. And vice versa.
So either do
volumes:
- ./data:/docker-entrypoint-initdb.d/
or
volumes:
- ./data/mydump.sql:/docker-entrypoint-initdb.d/mydump.sql
Yes, that is how you should mount the .sql or .sh files i.e by adding a volume by mapping the SQL or .sh files to the docker container's docker-entrypoint-initdb.d folder. But, it's raising an error for some strange reason maybe because the MySQL docker version is old.
You could solve this by creating a custom image i.e,
Dockerfile
FROM mysql:5.7
COPY init.sql /docker-entrypoint-initdb.d/
It creates an image and also helps in running a init script while starting the container.
To use this in a compose file, put your SQL files and Dockerfile in a folder.
database
|---init.sql
|---Dockerfile
docker-compose.yml
version: '3'
services:
mysqldb:
image: mysqldb
build: ./database
container_name: mysql
ports:
- "3306:3306"
environment:
- MYSQL_ROOT_PASSWORD=root
- MYSQL_USER=test
- MYSQL_PASSWORD=root
- MYSQL_DATABASE=test
By this, you could configure the environment variables easily.
I am having trouble to load in my SQL File to create the schema and the associated tables, I have also tried to use the 'volumes' option in the compose file to copy the sql file into the 'docker-entrypoint-initdb.d' directory but that fails to so anything.
Table + Schema: https://pastebin.com/RTBBGZhn
MySQL Dump: https://pastebin.com/6ApQwt1F
Docker Compose File
version: '2'
services:
melissabot:
image: melissabot
build: .
ports:
- 7000:7000
- 7070:7070
depends_on:
- mysqlMelissa
links:
- mysqlMelissa:db
mysqlMelissa:
image: mysql
build: ${PWD}/Docker/DB/
environment:
- MYSQL_ROOT_PASSWORD=root
# - MYSQL_DATABASE=MelissaBot
ports:
- "3306:3306"
phpmyadmin:
image: phpmyadmin/phpmyadmin
container_name: phpmyadmin
restart: always
ports:
- 8080:80
volumes:
- /sessions
links:
- mysqlMelissa:db
Dockerfile
FROM mysql:5.6
ENV MYSQL_ROOT_PASSWORD=root
# ENV MYSQL_DATABASE=MelissaBot
COPY setup.sh /mysql/setup.sh
COPY dump.sql /mysql/Melissa.sql
RUN chmod +x /mysql/setup.sh
RUN /mysql/setup.sh
setup.sh
#!/bin/bash
set -e
service mysql start
mysql -u root MelissaBot < /mysql/Melissa.sql
service mysql stop
You are doing it wrongly. There is no init system inside docker. So you should not be using server start/stop inside a container. Look at the official image documentation always
https://hub.docker.com/_/mysql
Initializing a fresh instance
When a container is started for the first time, a new database with the specified name will be created and initialized with the provided configuration variables. Furthermore, it will execute files with extensions .sh, .sql and .sql.gz that are found in /docker-entrypoint-initdb.d. Files will be executed in alphabetical order. You can easily populate your mysql services by mounting a SQL dump into that directory and provide custom images with contributed data. SQL files will be imported by default to the database specified by the MYSQL_DATABASE variable.
So you can either copy your default dump files to /docker-entrypoint-initdb.d or mount them using a volume mount when running the container. So you chuck your setup.sh and change your Dockerfile to something like below
FROM mysql:5.6
ENV MYSQL_ROOT_PASSWORD=root
ENV MYSQL_DATABASE=MelissaBot
COPY dump.sql /docker-entrypoint-initdb.d/
But I would rather prefer the approach of volume mounting it inside my compose file
mysqlMelissa:
image: mysql:5.6
environment:
- MYSQL_ROOT_PASSWORD=root
- MYSQL_DATABASE=MelissaBot
ports:
- "3306:3306"
volumes:
- dump.sql:/docker-entrypoint-initdb.d/dump.sql
Using docker-compose and a build file, I am unable to COPY files to a mysql container. Here's the compose.yml:
version: '3'
services:
db:
#image: mysql:5.7
build:
context: .
dockerfile: Dockerfile-mysql
environment:
MYSQL_ROOT_PASSWORD: drupal
MYSQL_DATABASE: drupal
MYSQL_USER: drupal
MYSQL_PASSWORD: drupal
#volumes:
# - /var/lib/mysql
ports:
- "3300:3306"
Then the Dockerfile-mysql:
FROM mysql:5.7
COPY ./drupal.sql /docker-entrypoint-initdb.d/
I see no errors, and the file isn't there. The container starts up, mysql is good, all that, but NO FILE! Can someone point out what I've missed? I'm assuming after the .sql file is transferred to that directory, the .sql file will run as well?
Thanks!
Figured it out. You have to run docker-compose build first to get docker to run any kind of COPY commands. If I understand it all correctly docker-compose build will create the containers, and docker-compose up will launch said containers.