I'm trying to connect Django project with MySQL using docker.
I have the problem when upping docker-compose. It says the next error:
super(Connection, self).init(*args, **kwargs2)
django.db.utils.OperationalError: (2002, "Can't connect to MySQL server on 'db' (115)")
I'm using port 3307, should i create first new database schema in my local? Or how can I do it because my default port is 3306 but if i try to use it, it says that is busy.
My code here:
Dockerfile
FROM python:3.7
ENV PYTHONUNBUFFERED 1
ENV LANG=C.UTF-8
RUN mkdir /code
WORKDIR /code
ADD requirements.txt /code/
RUN apt-get update
RUN pip install -r requirements.txt
ADD . /code/
Docker-compose
version: '2'
services:
app:
container_name: container_app
build:
context: .
dockerfile: Dockerfile
restart: always
command: bash -c "python3 manage.py migrate && python manage.py shell < backend/scripts/setup.py && python3 manage.py runserver 0.0.0.0:8000"
links:
- db
depends_on:
- db
ports:
- "8000:8000"
db:
container_name: container_database
image: mariadb
restart: always
environment:
MYSQL_ROOT_HOST: 'host.docker.internal'
MYSQL_DATABASE: 'container_develop'
MYSQL_USER: 'root'
MYSQL_PASSWORD: 'password'
MYSQL_ROOT_PASSWORD: 'password'
ports:
- "3307:3307"
settings.py:
DATABASES = {
'default' : {
'ENGINE': 'django.db.backends.mysql',
'NAME': 'database_develop',
'USER': 'root',
'PASSWORD': 'password',
'HOST': 'db',
'PORT': 3307,
'CHARSET': 'utf8',
'COLLATION': 'utf8_bin',
'OPTIONS': {
'use_unicode' : True,
'init_command': 'SET SESSION TRANSACTION ISOLATION LEVEL READ COMMITTED',
},
}
}
The thing is that I do not have the database anywhere so I'm using it locally, do I have to upload the db to a server and then use the port that server provides to me?
Is there any way to use it locally from docker? Or installing it to docker? So I can share that docker to a friend and he can use the same DB?
In the "db" part, in your docker-compose :
ports:
- "3307:3306"
The #Nico answer my fixed your service accessibility from outside of the container, mean if try to access it from the host it should work.
But the error will arise during service to service communication. You should not use publish port in service to service communication or another word you must use container port in service to service communication.
Change the port in connection from 3307 to 3306
DATABASES = {
'default' : {
'ENGINE': 'django.db.backends.mysql',
'NAME': 'database_develop',
'USER': 'root',
'PASSWORD': 'password',
'HOST': 'db',
'PORT': 3306,
'CHARSET': 'utf8',
'COLLATION': 'utf8_bin',
'OPTIONS': {
'use_unicode' : True,
'init_command': 'SET SESSION TRANSACTION ISOLATION LEVEL READ COMMITTED',
},
}
}
Related
Before you flag this question as a duplicate, please note that I did read other answers, but it didn't solve my problem.
I have a Docker compose file consisting of two services:
version: "3"
services:
mysql:
image: mysql:5.7
environment:
MYSQL_HOST: localhost
MYSQL_DATABASE: mydb
MYSQL_USER: mysql
MYSQL_PASSWORD: 1234
MYSQL_ROOT_PASSWORD: root
ports:
- "3307:3306"
expose:
- 3307
volumes:
- /var/lib/mysql
- ./mysql/migrations:/docker-entrypoint-initdb.d
restart: unless-stopped
web:
build:
context: .
dockerfile: web/Dockerfile
volumes:
- ./:/web
ports:
- "3000:3000"
environment:
NODE_ENV: development
PORT: 3000
links:
- mysql:mysql
depends_on:
- mysql
expose:
- 3000
command: ["./wait-for-it.sh", "mysql:3307"]
/web/Dockerfile:
FROM node:6.11.1
RUN mkdir -p /usr/src/app
WORKDIR /usr/src/app
COPY package.json /usr/src/app/
RUN npm install
COPY . /usr/src/app
CMD [ "npm", "start" ]
After docker-compose up --build the services start up, however the "wait-for-it.sh" script times out when waiting for mySQL to start (so temporarily I am not using it when testing for DB connectivity, I just wait until the console shows that MySQL is ready for accepting incoming connections)
When MySQL is running from the host machine I can login using Sequel Pro and query the DB and get the sample records from ./mysql/migrations
I can also SSH into the running MySQL container and do the same.
However, my Node.js app yields ECONNREFUSED 127.0.0.1:3307 when connecting
MySQL init:
import * as mysql from 'promise-mysql'
const config = {
host: 'localhost',
database: 'mydb',
port: '3307',
user: 'mysql',
password: '1234',
connectionLimit: 10
}
export let db = mysql.createPool(config);
MySQL query:
import { db } from '../db/client'
export let get = () => {
return db.query('SELECT * FROM users', [])
.then((results) => {
return results
})
.catch((e) => {
return Promise.reject(e)
})
}
Route invoked when hitting url /
import { Router } from 'express';
import * as repository from '../repository'
export let router = Router();
router.get('/', async (req, res) => {
let users;
try{
users = await repository.users.get();
} catch(e){
// ECONNREFUSED 127.0.0.1:3307
}
res.render('index', {
users: users
});
});
It's unlikely to be a race condition because at the same time when Node.js fails I can query using Sequel Pro or SSH into the running Docker container and query. So it's probably a case of Node.js not being able to access to MySQL container?
{
error: connect ECONNREFUSED 127.0.0.1:3307
code: 'ECONNREFUSED',
errno: 'ECONNREFUSED',
syscall: 'connect',
address: '127.0.0.1',
port: 3307,
fatal: true
}
This:
mysql:
image: mysql:5.7
environment:
...
ports:
- "3307:3306"
Means that Docker will map the 3307 port of the host to the 3306 port of the container. So you can access from Sequel to localhost:3307.
However, it does not mean that the container is listenting to 3307; the container is in fact still listening to 3306. When others containers tries to access the mysql DNS, it gets translated to the internal container IP, therefore you must connect to 3306.
So your node config should look like:
const config = {
host: 'mysql',
database: 'mydb',
port: '3306',
user: 'mysql',
password: '1234',
connectionLimit: 10
}
And this in your docker-compose.yml:
command: ["./wait-for-it.sh", "mysql:3306"]
Note: wait-for-it.sh script comes from: https://github.com/vishnubob/wait-for-it
I am trying to write a script that will spin up a docker container MySQL DB that can connect with my Nest.JS API. The goal is to use this across multiple people for easy setup of environment.
I plan to run
nest start --watch
will start the API. I am just struggling with setting up/connecting to the DB.
I have a docker-compose.yml that looks like this:
version: "3"
services:
# MySQL database
db:
image: mysql:8
container_name: db
command: --default-authentication-plugin=mysql_native_password --character-set-server=utf8mb4 --collation-server=utf8mb4_0900_ai_ci
restart: always
ports:
- 3306:3306
environment:
MYSQL_ROOT_PASSWORD: root
MYSQL_PASSWORD: root
MYSQL_DATABASE: db_name
and my databaseProvider looks like :
export const databaseProviders = [
{
provide: DATABASE_CONNECTION,
useFactory: async () => await createConnection({
type: 'mysql',
host: 'localhost',
port: 3306,
username: 'root',
password: 'root',
database: 'db_name',
entities: [
__dirname + '/../**/*.entity{.ts,.js}',
],
synchronize: true,
}),
},
];
I get the error :
Error: connect ECONNREFUSED 127.0.0.1:3306
Can anyone help me out with this?
I'm trying to connect Django project with MySQL using docker.
I have the problem when upping docker-compose. It says the next error:
super(Connection, self).init(*args, **kwargs2)
django.db.utils.OperationalError: (2002, "Can't connect to MySQL server on 'db' (115)")
I'm using port 3307, should i create first new database schema in my local? Or how can I do it because my default port is 3306 but if i try to use it, it says that is busy.
My code here:
Dockerfile
FROM python:3.7
ENV PYTHONUNBUFFERED 1
ENV LANG=C.UTF-8
RUN mkdir /code
WORKDIR /code
ADD requirements.txt /code/
RUN apt-get update
RUN pip install -r requirements.txt
ADD . /code/
Docker-compose
version: '2'
services:
app:
container_name: container_app
build:
context: .
dockerfile: Dockerfile
restart: always
command: bash -c "python3 manage.py migrate && python manage.py shell < backend/scripts/setup.py && python3 manage.py runserver 0.0.0.0:8000"
links:
- db
depends_on:
- db
ports:
- "8000:8000"
db:
container_name: container_database
image: mariadb
restart: always
environment:
MYSQL_ROOT_HOST: 'host.docker.internal'
MYSQL_DATABASE: 'container_develop'
MYSQL_USER: 'root'
MYSQL_PASSWORD: 'password'
MYSQL_ROOT_PASSWORD: 'password'
ports:
- "3307:3307"
settings.py:
DATABASES = {
'default' : {
'ENGINE': 'django.db.backends.mysql',
'NAME': 'database_develop',
'USER': 'root',
'PASSWORD': 'password',
'HOST': 'db',
'PORT': 3307,
'CHARSET': 'utf8',
'COLLATION': 'utf8_bin',
'OPTIONS': {
'use_unicode' : True,
'init_command': 'SET SESSION TRANSACTION ISOLATION LEVEL READ COMMITTED',
},
}
}
The thing is that I do not have the database anywhere so I'm using it locally, do I have to upload the db to a server and then use the port that server provides to me?
Is there any way to use it locally from docker? Or installing it to docker? So I can share that docker to a friend and he can use the same DB?
In the "db" part, in your docker-compose :
ports:
- "3307:3306"
The #Nico answer my fixed your service accessibility from outside of the container, mean if try to access it from the host it should work.
But the error will arise during service to service communication. You should not use publish port in service to service communication or another word you must use container port in service to service communication.
Change the port in connection from 3307 to 3306
DATABASES = {
'default' : {
'ENGINE': 'django.db.backends.mysql',
'NAME': 'database_develop',
'USER': 'root',
'PASSWORD': 'password',
'HOST': 'db',
'PORT': 3306,
'CHARSET': 'utf8',
'COLLATION': 'utf8_bin',
'OPTIONS': {
'use_unicode' : True,
'init_command': 'SET SESSION TRANSACTION ISOLATION LEVEL READ COMMITTED',
},
}
}
I'm very new for docker, now I am trying to run django with mariadb in docker through docker-compose, but I always get this error:
I use Docker version 17.09.1-ce, build 19e2cf6, docker-compose version 1.18.0, build 8dd22a9
django.db.utils.OperationalError: (2003, 'Can\'t connect to MySQL
server on \'mariadb55\' (111 "Connection refused")')
I can connect db correctly after run docker-compose up db in local or remote, and I even can run python manage.py runserver 0.0.0.0:6001 correctly in anaconda virtual environment to connect db service in docker by setting parameters of settings.py file like below:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.mysql',
'NAME': 'test',
'USER': 'belter',
# 'HOST': 'mariadb55',
'HOST': '127.0.0.1',
'PORT': '3302',
'PASSWORD': 'belter_2017',
'default-character-set': 'utf8',
'OPTIONS': {
'sql_mode': 'traditional',
}
}
}
This is my docker-compose.yml file
version: '3'
services:
db:
image: mariadb:5.5
restart: always
environment:
- MYSQL_HOST=localhost
- MYSQL_PORT=3306
- MYSQL_ROOT_HOST=%
- MYSQL_DATABASE=test
- MYSQL_USER=belter
- MYSQL_PASSWORD=belter_2017
- MYSQL_ROOT_PASSWORD=123456_abc
volumes:
- /home/belter/mdbdata/mdb55:/var/lib/mysql
ports:
- "3302:3306"
web:
image: onlybelter/django_py35
command: python3 manage.py runserver 0.0.0.0:6001
volumes:
- /mnt/data/www/mysite:/djcode
ports:
- "6001:6001"
depends_on:
- db
links:
- db:mariadb55
I almost tried everything I can find, but still cannot figure it out, any help would be nice!
What I have tried:
Docker compose mysql connection failing
Linking django and mysql containers using docker-compose
Django connection to postgres by docker-compose
Finally, I figured it out!
The key point is, just as #SangminKim said, I need to use 3306 not 3302 in settings.py, and use db as HOST not 127.0.0.1.
So this is my docker-compose.yml file now:
version: '3'
services:
db:
image: mariadb:5.5
restart: always
environment:
- MYSQL_HOST=localhost
- MYSQL_PORT=3306 # cannot change this port to other number
- MYSQL_ROOT_HOST=%
- MYSQL_DATABASE=test
- MYSQL_USER=belter
- MYSQL_PASSWORD=belter_2017
- MYSQL_ROOT_PASSWORD=123456_abc
volumes:
- /home/belter/mdbdata/mdb55:/var/lib/mysql
ports:
- "3302:3306"
web:
image: onlybelter/django_py35
command: python3 manage.py runserver 0.0.0.0:6001
volumes:
- .:/djcode
ports:
- "6001:6001"
depends_on:
- db
So now we can connect this docker-mysql by mysql -h 127.0.0.1 -P 3302 -u root -p in shell directly, but we have to use db and 3306 in django settings.py file:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.mysql',
'NAME': 'test',
'USER': 'belter',
# 'HOST': 'mariadb55',
'HOST': 'db', #<---
'PORT': '3306', #<---
'PASSWORD': 'belter_2017',
'default-character-set': 'utf8',
'OPTIONS': {
'sql_mode': 'traditional',
}
}
}
And we can still check if this port is open, by running extra command in docker-compose.yml file:
...
web:
image: onlybelter/django_py35
command: /bin/sh -c "python check_db.py --service-name mysql --ip db --port 3306"
volumes:
- .:/djcode
...
Here is check_db.py file:
# check_db.py
import socket
import time
import argparse
""" Check if port is open, avoid docker-compose race condition """
parser = argparse.ArgumentParser(description='Check if port is open, avoid\
docker-compose race condition')
parser.add_argument('--service-name', required=True)
parser.add_argument('--ip', required=True)
parser.add_argument('--port', required=True)
args = parser.parse_args()
# Get arguments
service_name = str(args.service_name)
port = int(args.port)
ip = str(args.ip)
# Infinite loop
while True:
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
result = sock.connect_ex((ip, port))
if result == 0:
print("{0} port is open! Bye!".format(service_name))
break
else:
print("{0} port is not open! I'll check it soon!".format(service_name))
time.sleep(3)
By the way, this is my Dockerfile for build django-py35:
FROM python:3.5-alpine
MAINTAINER Xin Xiong "xiongxin20008#126.com"
ENV PYTHONUNBUFFERED 1
RUN set -e; \
apk add --no-cache --virtual .build-deps \
gcc \
libc-dev \
linux-headers \
mariadb-dev \
python3-dev \
postgresql-dev \
freetype-dev \
libpng-dev \
g++ \
;
RUN mkdir /djcode
WORKDIR /djcode
ENV REFRESHED_AT 2017-12-25
ADD requirements.txt /djcode/
RUN pip install --no-cache-dir -r /djcode/requirements.txt
RUN pip install uwsgi
ADD . /djcode/ # copy . to /djcode/
EXPOSE 6001
See more details from here: https://github.com/OnlyBelter/django-compose
You should use the container name instead of localhost (or 127.0.0.1) in your settings.py file. Try providing a container name to the db service in the docker-compose.yml file using container_name attribute and replace the host name in the settings.py by the value of the container_name. (Make sure that they are in the same network that docker compose creates for you.)
Build container with this:
docker run --name mysql-latest \
-p 3306:3306 -p 33060:33060 \
-e MYSQL_ROOT_HOST='%' -e MYSQL_ROOT_PASSWORD='strongpassword' \
-d mysql/mysql-server:latest
Make sure MYSQL_ROOT_HOST='%', that means root can connect from any IP.
I am attempting to create a dev environment for my company's app using docker. For some reason I am able to connect to the database if I use docker run web "bash" then run mysql from the shell, but Django wont connect. The Dockerfile for the web/worker nodes looks like:
FROM python:2.7
ENV PYTHONUNBUFFERED 1
ENV DOCKER 1
ENV REDIS_HOST "redis"
ENV CASSANDRA_HOST "cassandra"
RUN mkdir /code
WORKDIR /code
ADD . /code/
RUN pip install --default-timeout=100 numpy==1.9.1
RUN pip install --default-timeout=100 scipy==0.15.1
RUN pip install --default-timeout=100 -r requirements_docker.txt
#RUN python /code/app/manage.py migrate
And the docker-compose file looks like this:
version: "2"
services:
cassandra:
image:
cassandra:latest
redis:
image:
redis:latest
db:
image: mysql
environment:
MYSQL_DATABASE: appdb
MYSQL_USER: dbuser
MYSQL_PASSWORD: dbpass
web:
build: .
command: python app/manage.py runserver 0.0.0.0:8000
environment:
DOCKER: 1
DATABASE_HOST: db
DATABASE_NAME: appdb
DATABASE_USER: dbuser
DATABASE_PASSWORD: dbpass
REDIS_HOST: redis
CASSANDRA_HOST: cassandra
volumes:
- .:/code
ports:
- "8000:8000"
depends_on:
- db
- redis
- cassandra
worker:
build: .
environment:
DOCKER: 1
DATABASE_HOST: db
DATABASE_NAME: appdb
DATABASE_USER: dbuser
DATABASE_PASSWORD: dbpass
REDIS_HOST: redis
CASSANDRA_HOST: cassandra
command: python /code/app/manage.py celery worker -Q celery,offline,periodic --broker=redis://redis:6379/4
depends_on:
- db
- redis
- cassandra
Finally the database settings are (consolidated from multiple settings files):
if not os.environ.has_key('DOCKER'):
DATABASES = {
"default": {
'ENGINE': 'django.db.backends.mysql',
'OPTIONS': {
'read_default_file': os.path.join(PROJECT_ROOT, 'mysql.cnf'),
"init_command": "SET foreign_key_checks = 0;",
},
}
}
else: # Using Docker
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.mysql',
'NAME': os.environ['DATABASE_NAME'],
'USER': os.environ['DATABASE_USER'],
'PASSWORD': os.environ['DATABASE_PASSWORD'],
'HOST': os.environ["DATABASE_HOST"],
'PORT': '5432',
'OPTIONS': {
'init_command': "SET foreign_key_checks = 0;"
}
}
}
if 'test' in sys.argv:
SLAVE_DATABASES = []
elif os.environ.has_key('DOCKER'):
DATABASES['replica_1'] = {
'ENGINE': 'django.db.backends.mysql',
'NAME': os.environ['DATABASE_NAME'],
'USER': os.environ['DATABASE_USER'],
'PASSWORD': os.environ['DATABASE_PASSWORD'],
'HOST': os.environ["DATABASE_HOST"],
'PORT': '5432',
'OPTIONS': {
"init_command": "SET foreign_key_checks = 0;"
}
}
SLAVE_DATABASES = ['replica_1', ]
DATABASE_ROUTERS = ('multidb.PinningMasterSlaveRouter',)
MIDDLEWARE_CLASSES = ['multidb.middleware.PinningRouterMiddleware', ] + settings.MIDDLEWARE_CLASSES
else:
DATABASES["replica_1"] = {
'ENGINE': 'django.db.backends.mysql',
'OPTIONS': {
'read_default_file': os.path.join(settings.PROJECT_ROOT, 'mysql.cnf'),
"init_command": "SET foreign_key_checks = 0;",
},
'TEST': {
'MIRROR': 'default',
},
}
SLAVE_DATABASES = ['replica_1', ]
DATABASE_ROUTERS = ('multidb.PinningMasterSlaveRouter',)
MIDDLEWARE_CLASSES = ['multidb.middleware.PinningRouterMiddleware', ] + settings.MIDDLEWARE_CLASSES
if os.environ.has_key("DOCKER"):
from cassandra import ConsistencyLevel
DATABASES['cassandra'] = {
'ENGINE': 'django_cassandra_engine',
'NAME': 'verificient_local',
'TEST_NAME': 'test_verificient_local',
'HOST': os.environ['CASSANDRA_HOST'],
'OPTIONS': {
'replication': {
'strategy_class': 'SimpleStrategy',
'replication_factor': 1
},
'connection': {
'consistency': ConsistencyLevel.LOCAL_ONE,
'retry_connect': True
},
'session': {
'default_timeout': 30,
'default_fetch_size': 10000
}
}
}
The weird part is that even though running python manage.py migrate returns an error (django.db.utils.OperationalError: (2003, "Can't connect to MySQL server on 'db' (111)")) I am able to connect to the database with no issue using the command mysql -h$DATABASE_HOST -u$DATABASE_USER -p$DATABASE_PASSWORD $DATABASE_NAME
What could be going wrong in my django setup?
MySQL default port is 3306, so change:
'PORT': '5432',
to
'PORT': '3306',
It does work manually with your mysql command because that command uses the correct port by default