I´m trying to built a docker-compose file to deploy locally my NodeJS app that connects to a mysql server. I´ve tried everything ( read a lot of tutorials and some questions here in Stackoverflow ) but I keep getting the ECONNREFUSED error.
This is my Dockerfile from NodeJS:
##Nodejs
FROM node:latest
RUN useradd --user-group --create-home --shell /bin/false app
ENV HOME=/home/app
COPY package.json npm-shrinkwrap.json $HOME/playerground/
RUN chown -R app:app $HOME/*
USER app
WORKDIR $HOME/playerground
RUN npm cache clean && npm install --silent --progress=false
USER root
COPY . $HOME/playerground
RUN chown -R app:app $HOME/*
USER app
This is my Mysql Dockerfile:
FROM mysql:latest
ENV MYSQL_ROOT_PASSWORD root
ENV MYSQL_DATABASE playerground
ENV MYSQL_USER root
ENV MYSQL_PASSWORD root
And this is my docker-compose:
version: '2'
services:
db:
build: ./database
ports:
- "3307:3306"
playerground:
build:
context: .
dockerfile: Dockerfile
command: node_modules/.bin/nodemon --exec npm start
environment:
ENVIRONMENT: development
ports:
- "9090:9090"
links:
- db
volumes:
- .:/home/app/playerground
- /home/app/playerground/node_modules
Another thing is my configuration file from Nodejs:
//Database
'development' : {
'host' : process.env.HOSTNAME || 'localhost',
'user' : 'root',
'password' : 'root',
'database' : 'playerground'
},
//Server
'development' : {
'host' : process.env.HOSTNAME || '127.0.0.1',
'port' : 3000
},
Can you help me?
Thanks
In your app config, the database host isn't pointing at your MySql container. You're pointing it at the local host, which is the app container. In the Compose file you can explicitly name the link to the db container using:
links:
- db:db
And then your database container can be reached at the hostname db. You can hard-code that host in your app config, because it will be the same for all environments, as long as you use the same Compose file.
Also, if you're using a recent version of Docker you don't need to publish ports between containers in the same Docker network - so you can remove this from your Compose file:
ports:
- "3307:3306"
Then the db container will be accessible by the app container, but not externally.
Related
I've been following with the docker-compose tutorial here (linking django and postgres container). Although I was able to go through with the tutorial I'm however not able to proceed with repeating the same
using a mysql container.
The following are my dockerfile and docker-compose.yml
`
db:
image: mysql
web:
build: .
command: python manage.py runserver 0.0.0.0:8000
volumes:
- .:/code
ports:
- "8000:8000"
links:
- db:db
`
dockerfile
FROM python:2.7
RUN mkdir /code
WORKDIR /code
RUN pip install mysql-python
RUN pip install django
They both build fine when I do docker-compose up but it seems the db environment variables are not passed to the django container since when I run os.environ.keys() in one of my django views I can't see any of the expected DB_* environment variables.
So does mysql require a different setup or am I missing something.
Thank you.
[EDIT]
Docker compose version
docker-compose version: 1.3.0
CPython version: 2.7.9
OpenSSL version: OpenSSL 1.0.1e 11 Feb 2013
Docker version
Docker version 1.6.2, build 7c8fca2
In Django settings.py file make sure you have something like:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.mysql',
'NAME': 'django1',
'USER': 'django',
'PASSWORD': 'password',
'HOST': 'db',
'PORT': 3306,
}
}
then in your docker-compose.yml file make sure you have something along the lines of:
db:
image: mysql
environment:
MYSQL_ROOT_PASSWORD: docker
MYSQL_DATABASE: docker
MYSQL_USER: docker
MYSQL_PASSWORD: docker
then as per the docker/django tutorial you are following run the following again to rebuild everything and things should start working
docker-compose run web django-admin.py startproject composeexample .
In response to a further question, the mysql root password variable is required by docker when creating new databases.
EDIT: added run to docker-compose above; see edit comment
you don't need to worry about environment variable. When linking containers together you just use the container alias defined by the link as if it was the hostname.
for instance if your docker-compose.yml file were:
db:
image: postgres
web:
build: .
command: python manage.py runserver 0.0.0.0:8000
volumes:
- .:/code
ports:
- "8000:8000"
links:
- db:mydb
In your django settings you would have to set the database host to mydb.
First you need to modify the settings file...
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql_psycopg2',
'NAME': 'postgres',
'USER': 'postgres',
'HOST': 'db',
'PORT': 5432,
} }
Then if you used the docker-compose command properly, the containers should be linked, and it should resolve the hostname db properly based on the links in the docker-compose.yml file.
Still, if you want to check the environment...
~/django-example: docker-compose run web env
Starting djangoexample_db_1...
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
HOSTNAME=66ff09ed8632
TERM=xterm
DJANGOEXAMPLE_DB_1_PORT=tcp://172.17.0.35:5432
DJANGOEXAMPLE_DB_1_PORT_5432_TCP=tcp://172.17.0.35:5432
DJANGOEXAMPLE_DB_1_PORT_5432_TCP_ADDR=172.17.0.35
DJANGOEXAMPLE_DB_1_PORT_5432_TCP_PORT=5432
DJANGOEXAMPLE_DB_1_PORT_5432_TCP_PROTO=tcp
DJANGOEXAMPLE_DB_1_NAME=/djangoexample_web_run_2/djangoexample_db_1
DJANGOEXAMPLE_DB_1_ENV_affinity:container==52c78c810792b0e7b9a231eab7ab7a3d50c95b76faf0abb8ec38a7d1ff0c7e5f
DJANGOEXAMPLE_DB_1_ENV_LANG=en_US.utf8
DJANGOEXAMPLE_DB_1_ENV_PG_MAJOR=9.4
DJANGOEXAMPLE_DB_1_ENV_PG_VERSION=9.4.4-1.pgdg70+1
DJANGOEXAMPLE_DB_1_ENV_PGDATA=/var/lib/postgresql/data
DB_PORT=tcp://172.17.0.35:5432
DB_PORT_5432_TCP=tcp://172.17.0.35:5432
DB_PORT_5432_TCP_ADDR=172.17.0.35
DB_PORT_5432_TCP_PORT=5432
DB_PORT_5432_TCP_PROTO=tcp
DB_NAME=/djangoexample_web_run_2/db
DB_ENV_affinity:container==52c78c810792b0e7b9a231eab7ab7a3d50c95b76faf0abb8ec38a7d1ff0c7e5f
DB_ENV_LANG=en_US.utf8
DB_ENV_PG_MAJOR=9.4
DB_ENV_PG_VERSION=9.4.4-1.pgdg70+1
DB_ENV_PGDATA=/var/lib/postgresql/data
DB_1_PORT=tcp://172.17.0.35:5432
DB_1_PORT_5432_TCP=tcp://172.17.0.35:5432
DB_1_PORT_5432_TCP_ADDR=172.17.0.35
DB_1_PORT_5432_TCP_PORT=5432
DB_1_PORT_5432_TCP_PROTO=tcp
DB_1_NAME=/djangoexample_web_run_2/db_1
DB_1_ENV_affinity:container==52c78c810792b0e7b9a231eab7ab7a3d50c95b76faf0abb8ec38a7d1ff0c7e5f
DB_1_ENV_LANG=en_US.utf8
DB_1_ENV_PG_MAJOR=9.4
DB_1_ENV_PG_VERSION=9.4.4-1.pgdg70+1
DB_1_ENV_PGDATA=/var/lib/postgresql/data
LANG=C.UTF-8
PYTHON_VERSION=2.7.10
PYTHON_PIP_VERSION=7.0.3
PYTHONUNBUFFERED=1
HOME=/root
I have a problem with mysql 5.7 container denying access to my nodeJS container. I'm using docker-compose and I'm running docker on Ubuntu 18.04 lts.
I apologize for the mistakes in English. This is not my mother language.
I replaced all confidential information with *** and in the images I put a red line
Here's my docker-compose.yml
version: '3.1'
services:
db:
container_name: '***-db'
image: mysql:5.7
command: --default-authentication-plugin=mysql_native_password
restart: always
environment:
- MYSQL_ROOT_PASSWORD=***
- MYSQL_DATABASE=***
- MYSQL_USER=***
- MYSQL_PASSWORD=***
- TZ=America/Sao_Paulo
command: mysqld --sql_mode="" --character-set-server=utf8 --collation-server=utf8_slovenian_ci --init-connect='SET NAMES UTF8;' --innodb-flush-log-at-trx-commit=0
ports:
- 3306:3306
volumes:
- ./db_data:/var/lib/mysql
network_mode: "host"
server:
image: server
restart: always
ports:
- "2000:22"
- "3000:3000"
network_mode: "host"
front:
image: portal
restart: always
links:
- server:server
ports:
- 4200:80
I am using a volume with the database for my mysql container. In this volume I have my user and password set and I also have my schema with every table. I copy everything from this path:
/var/lib/mysql
that I have on my configuration (I am not using docker on this one) in another computer with the same version of mysql and that I know it is working.
Here is my server Dockerfile:
FROM ubuntu:18.04
LABEL maintainer = "***"
LABEL build_date="2020-04-24"
RUN apt-get update && apt-get install -y curl
RUN curl -sL https://deb.nodesource.com/setup_8.x | bash -
RUN apt-get update && apt-get install -y nodejs
#COPY AN CONFIGURATION OF APP
COPY ***-server /home/root/***-server
WORKDIR /home/root/***-server
RUN npm install
EXPOSE 22
EXPOSE 3000
CMD ["npm", "start"]
Here is my nodeJS file to connect with the mysql:
"use strict";
module.exports = {
config: {
connectionLimit : 200,
host: 'localhost',
user : '***',
password : '***',
database: '***'
}
}
this is the error mesage that docker compose send to me, when I try to connect:
And this is image of the container of the mysql that I am trying to access. I make access to the mysql inside the container using the same user that I am using on the server.
I setup a django project in docker container and every thing is working as expected, except I don't find the project database in mysql image.
Dockerfile
FROM python:3
RUN mkdir /django-website
WORKDIR /django-website
COPY . /django-website
RUN pip install -r requirements.txt
docker-compose.yml
version: '3'
services:
db:
image: mysql:5.7
restart: always
environment:
- MYSQL_ROOT_PASSWORD=root
- MYSQL_DATABASE=mywebsite
- MYSQL_USER=root
- MYSQL_PASSWORD=root
ports:
- '33060:3306'
volumes:
- /var/lib/mysql
web:
build: .
command: python manage.py runserver 0.0.0.0:8000
volumes:
- .:/django-website
ports:
- '8000:8000'
links:
- db
settings.py
DATABASES = {
'default': {
'ENGINE': "django.db.backends.mysql",
'NAME': "mywebsite",
'USER': "root",
'PASSWORD': "root",
'HOST': 'db',
'PORT': '3306',
}
}
I ran migrate and it worked:
docker-compose run web python manage.py migrate
I createdsuperuser:
docker-compose run web python manage.py createsuperuser
The development server is working docker-compose up and the site is working as expected, the issue when I navigate in mysql image I don't find my project related database which is mywebsite .
can you please tell me what is missing? if the database is not created, where has the migration been applied?
Thanks in advance.
I'm not sure what you mean by "I logged in mysql image shell but didn't find mywebsite database"
You are migrated the DB successfully, which means, the DB connections are valid and working.
In your docker-compose.yml file, the port mapping done like this, '33060:3306', which means the db's port 3306 is mapped to host machine's port 33060. So, this may be the issue (it's not an issue, kind of typo)
How to check the DB contents?
METHOD-1: check through django-shell of web container
1. run docker-compose up
2. open a new terminal in the same path and run docker ps
you'll get something like below
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
795093357f78 django_1_11_web "python manage.py ru…" 34 minutes ago Up 11 minutes 0.0.0.0:8000->8000/tcp django_1_11_web_1
4ae48f291e34 mysql:5.7 "docker-entrypoint.s…" 34 minutes ago Up 12 minutes 0.0.0.0:33060->3306/tcp django_1_11_db_1
3.Get into the web container by docker exec -it 795093357f78 bash command, where 795093357f78 is the respective container id
4. now you're inside the container. Then, run the command python manage.py dbshell. Now you will be in MYSQL shell of mywebsite (Screenshot)
5. run the command show tables;. It will display all the tables inside the mywebsite DB
METHOD-2: check through db container
1. repeat the steps 1 and 2 in above section
2. get into db container by docker exec -it 4ae48f291e34 bash
3. Now you'll be in bash terminal of MYSQL. Run the following commmand mysql -u root -p and enter the password when prompt
4. now you're in MYSQL server. run the command, show databases;. This will show all the databases in the server.
Have you tried defining the database image in the dockerfile? The following link is somewhat related to your problem:
https://medium.com/#lvthillo/customize-your-mysql-database-in-docker-723ffd59d8fb
I supposed that ports value of host container should be 3306 not 33060.
Use docker-compose.yml with value 3306 :
version: '3'
services:
db:
image: mysql:5.7
restart: always
environment:
- MYSQL_ROOT_PASSWORD=root
- MYSQL_DATABASE=mywebsite
- MYSQL_USER=root
- MYSQL_PASSWORD=root
ports:
- '3306:3306'
volumes:
- /var/lib/mysql
web:
build: .
command: python manage.py runserver 0.0.0.0:8000
volumes:
- .:/django-website
ports:
- '8000:8000'
links:
- db
Hope this works!
You should change the compose specification to version '2'. Take down the container and bring it back up with docker-compose up -d. Or if you intend to stay with version 3, you can instead use the following specification for database environment parameters
```
environment:
MYSQL_ROOT_PASSWORD: root
MYSQL_DATABASE: mywebsite
MYSQL_USER: root
MYSQL_PASSWORD: root
```
When you have problems with containers not coming up, docker logs <container-name> --tail 25 -f can give you a lot of information about the cause.
I am trying set up a Django development environment using docker. Though I can connect to the mysql on host machine. But the web container failed to connect to mysql container with following error:
django.db.utils.OperationalError: (2003, "Can't connect to MySQL server on 'db' (111)")
Below are the docker configurations and django configuration:
---------docker-compose.yml---------
version: '2'
services:
web:
build: .
volumes:
- ".:/code/current"
ports:
- "8000:8000"
depends_on:
- db
- redis
links:
- db
- redis
command: ./manage.py runserver 0.0.0.0:8000
db:
image: mysql:5.7
volumes:
- "./.data/db:/var/lib/mysql"
environment:
- MYSQL_ROOT_PASSWORD=root
- MYSQL_DATABASE=goat_db
restart: always
ports:
- "3306:3306"
redis:
restart: always
image: redis:latest
ports:
- "6379:6379"
--------------Dockerfile for web image----------------------
FROM ubuntu:14.04
MAINTAINER Alice
RUN apt-get update
RUN apt-get install -y tar git curl wget emacs build-essential python python-dev python-distribute python-pip libpcre3 libpcre3-dev libmysqlclient-dev python-m2crypto openssl libssl-dev swig freetds-dev python-pymssql nginx subversion
RUN mkdir -p var/www/goat.garenanow.com/current/log
WORKDIR /var/www/goat.garenanow.com/current
ADD requirements.txt /var/www/goat.garenanow.com/current
RUN pip install -r requirements.txt
ADD . /var/www/goat.garenanow.com/current
EXPOSE 8000
------django database configuration---
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.mysql',
'NAME': 'goat_db',
'USER': 'root',
'PASSWORD': 'root',
'HOST': 'db',
'PORT': '3306',
},
If Django tries to connect to your database and MySql is not ready, then you will probably see this error:
django.db.utils.OperationalError: (2003, "Can't connect to MySQL server on 'db' (111)")
The solution for this case is wait for the db server to be ready before run the web server.
How we can do that?
Instead of running the web server directly we create and run a script that it will wait for the db server to be ready. After that the script will run the web server.
docker-compose.yml file (The relevant part is when we overwrite the web:command directive and we run the script instead of run the web server immediately).
version: '3'
services:
db:
restart: always
image: mysql:5.7
container_name: my_db
env_file: env_db
ports:
- "3306:3306"
web:
restart: always
build: ./web
env_file: env_web
command: ./wait_for_db_and_start_server.sh
ports:
- "8000:8000"
depends_on:
- db
Now env_web file (The relevant part is START_CMD variable).
# Add Environment Variables
DB_NAME=docker
DB_USER=docker_user
DB_PASS=docker_password
DB_SERVICE=db
DB_HOST=db
DB_PORT=3306
START_CMD=python manage.py runserver 0.0.0.0:8000
Now, our web Dockerfile (The relevant part is when we add wait_for_db_and_start_server.sh).
FROM python:2.7
ADD wait_for_db_and_start_server.sh .
ADD requirements.txt .
RUN pip install -r requirements.txt
ADD . .
Now wait_for_db_and_start_server.sh file. (Basically, while we cannot connect to our db server on the specified $DB_HOST:$DB_PORT defined in our env_web file, we wait 3 seconds and try again for M_LOOPS times. If we are able to connect to the server, then we run $START_CMD that was defined in our env_web file.)
#!/bin/sh
# Wait for database to get available
M_LOOPS="10"
#wait for mysql
i=0
# http://stackoverflow.com/a/19956266/4848859
while ! curl $DB_HOST:$DB_PORT >/dev/null 2>&1 < /dev/null; do
i=`expr $i + 1`
if [ $i -ge $M_LOOPS ]; then
echo "$(date) - ${DB_HOST}:${DB_PORT} still not reachable, giving up"
exit 1
fi
echo "$(date) - waiting for ${DB_HOST}:${DB_PORT}..."
sleep 3
done
echo "$(date) - ${DB_HOST}:${DB_PORT} Reachable ! - Starting Daemon"
#start the daemon
exec $START_CMD
What's your my.cnf file look like for the db service?
Check the bind-address value and also make sure the grant permissions are correct connecting from the web container.
To test, you could docker exec ... into the web container and try to connect to db host from there.
If you are trying to keep your web container "thin", you may not want to add mysql or postgres client packages, just to check if your db is ready. Some of the popular wait scripts out there have client dependencies.
I just created a simple Python script which makes native calls to see if the DB is functional. In this case I avoid adding valid credentials and just look for an authentication failure message. The example here is for Postgres, but should be easily adapted for MySQL.
wait_for_db.py
import psycopg2.extras
from time import sleep
connect_str = "dbname='postgres' user='nobody' host='db' " + \
"password='foobar'"
while True:
try:
conn = psycopg2.connect(connect_str)
except psycopg2.OperationalError as e:
if "authentication failed" in e.args[0]:
print("Postgres is ready. Launching site.")
break
else:
print("Waiting for postgres...", e.args[0])
sleep(1)
Then set your web container in your dev environment docker-compose.yml so that you have something like
command: bash -c "python wait-for-postgres.py; python manage.py runserver 0.0.0.0:8000"
I've been following with the docker-compose tutorial here (linking django and postgres container). Although I was able to go through with the tutorial I'm however not able to proceed with repeating the same
using a mysql container.
The following are my dockerfile and docker-compose.yml
`
db:
image: mysql
web:
build: .
command: python manage.py runserver 0.0.0.0:8000
volumes:
- .:/code
ports:
- "8000:8000"
links:
- db:db
`
dockerfile
FROM python:2.7
RUN mkdir /code
WORKDIR /code
RUN pip install mysql-python
RUN pip install django
They both build fine when I do docker-compose up but it seems the db environment variables are not passed to the django container since when I run os.environ.keys() in one of my django views I can't see any of the expected DB_* environment variables.
So does mysql require a different setup or am I missing something.
Thank you.
[EDIT]
Docker compose version
docker-compose version: 1.3.0
CPython version: 2.7.9
OpenSSL version: OpenSSL 1.0.1e 11 Feb 2013
Docker version
Docker version 1.6.2, build 7c8fca2
In Django settings.py file make sure you have something like:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.mysql',
'NAME': 'django1',
'USER': 'django',
'PASSWORD': 'password',
'HOST': 'db',
'PORT': 3306,
}
}
then in your docker-compose.yml file make sure you have something along the lines of:
db:
image: mysql
environment:
MYSQL_ROOT_PASSWORD: docker
MYSQL_DATABASE: docker
MYSQL_USER: docker
MYSQL_PASSWORD: docker
then as per the docker/django tutorial you are following run the following again to rebuild everything and things should start working
docker-compose run web django-admin.py startproject composeexample .
In response to a further question, the mysql root password variable is required by docker when creating new databases.
EDIT: added run to docker-compose above; see edit comment
you don't need to worry about environment variable. When linking containers together you just use the container alias defined by the link as if it was the hostname.
for instance if your docker-compose.yml file were:
db:
image: postgres
web:
build: .
command: python manage.py runserver 0.0.0.0:8000
volumes:
- .:/code
ports:
- "8000:8000"
links:
- db:mydb
In your django settings you would have to set the database host to mydb.
First you need to modify the settings file...
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql_psycopg2',
'NAME': 'postgres',
'USER': 'postgres',
'HOST': 'db',
'PORT': 5432,
} }
Then if you used the docker-compose command properly, the containers should be linked, and it should resolve the hostname db properly based on the links in the docker-compose.yml file.
Still, if you want to check the environment...
~/django-example: docker-compose run web env
Starting djangoexample_db_1...
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
HOSTNAME=66ff09ed8632
TERM=xterm
DJANGOEXAMPLE_DB_1_PORT=tcp://172.17.0.35:5432
DJANGOEXAMPLE_DB_1_PORT_5432_TCP=tcp://172.17.0.35:5432
DJANGOEXAMPLE_DB_1_PORT_5432_TCP_ADDR=172.17.0.35
DJANGOEXAMPLE_DB_1_PORT_5432_TCP_PORT=5432
DJANGOEXAMPLE_DB_1_PORT_5432_TCP_PROTO=tcp
DJANGOEXAMPLE_DB_1_NAME=/djangoexample_web_run_2/djangoexample_db_1
DJANGOEXAMPLE_DB_1_ENV_affinity:container==52c78c810792b0e7b9a231eab7ab7a3d50c95b76faf0abb8ec38a7d1ff0c7e5f
DJANGOEXAMPLE_DB_1_ENV_LANG=en_US.utf8
DJANGOEXAMPLE_DB_1_ENV_PG_MAJOR=9.4
DJANGOEXAMPLE_DB_1_ENV_PG_VERSION=9.4.4-1.pgdg70+1
DJANGOEXAMPLE_DB_1_ENV_PGDATA=/var/lib/postgresql/data
DB_PORT=tcp://172.17.0.35:5432
DB_PORT_5432_TCP=tcp://172.17.0.35:5432
DB_PORT_5432_TCP_ADDR=172.17.0.35
DB_PORT_5432_TCP_PORT=5432
DB_PORT_5432_TCP_PROTO=tcp
DB_NAME=/djangoexample_web_run_2/db
DB_ENV_affinity:container==52c78c810792b0e7b9a231eab7ab7a3d50c95b76faf0abb8ec38a7d1ff0c7e5f
DB_ENV_LANG=en_US.utf8
DB_ENV_PG_MAJOR=9.4
DB_ENV_PG_VERSION=9.4.4-1.pgdg70+1
DB_ENV_PGDATA=/var/lib/postgresql/data
DB_1_PORT=tcp://172.17.0.35:5432
DB_1_PORT_5432_TCP=tcp://172.17.0.35:5432
DB_1_PORT_5432_TCP_ADDR=172.17.0.35
DB_1_PORT_5432_TCP_PORT=5432
DB_1_PORT_5432_TCP_PROTO=tcp
DB_1_NAME=/djangoexample_web_run_2/db_1
DB_1_ENV_affinity:container==52c78c810792b0e7b9a231eab7ab7a3d50c95b76faf0abb8ec38a7d1ff0c7e5f
DB_1_ENV_LANG=en_US.utf8
DB_1_ENV_PG_MAJOR=9.4
DB_1_ENV_PG_VERSION=9.4.4-1.pgdg70+1
DB_1_ENV_PGDATA=/var/lib/postgresql/data
LANG=C.UTF-8
PYTHON_VERSION=2.7.10
PYTHON_PIP_VERSION=7.0.3
PYTHONUNBUFFERED=1
HOME=/root