I am attempting to create a dev environment for my company's app using docker. For some reason I am able to connect to the database if I use docker run web "bash" then run mysql from the shell, but Django wont connect. The Dockerfile for the web/worker nodes looks like:
FROM python:2.7
ENV PYTHONUNBUFFERED 1
ENV DOCKER 1
ENV REDIS_HOST "redis"
ENV CASSANDRA_HOST "cassandra"
RUN mkdir /code
WORKDIR /code
ADD . /code/
RUN pip install --default-timeout=100 numpy==1.9.1
RUN pip install --default-timeout=100 scipy==0.15.1
RUN pip install --default-timeout=100 -r requirements_docker.txt
#RUN python /code/app/manage.py migrate
And the docker-compose file looks like this:
version: "2"
services:
cassandra:
image:
cassandra:latest
redis:
image:
redis:latest
db:
image: mysql
environment:
MYSQL_DATABASE: appdb
MYSQL_USER: dbuser
MYSQL_PASSWORD: dbpass
web:
build: .
command: python app/manage.py runserver 0.0.0.0:8000
environment:
DOCKER: 1
DATABASE_HOST: db
DATABASE_NAME: appdb
DATABASE_USER: dbuser
DATABASE_PASSWORD: dbpass
REDIS_HOST: redis
CASSANDRA_HOST: cassandra
volumes:
- .:/code
ports:
- "8000:8000"
depends_on:
- db
- redis
- cassandra
worker:
build: .
environment:
DOCKER: 1
DATABASE_HOST: db
DATABASE_NAME: appdb
DATABASE_USER: dbuser
DATABASE_PASSWORD: dbpass
REDIS_HOST: redis
CASSANDRA_HOST: cassandra
command: python /code/app/manage.py celery worker -Q celery,offline,periodic --broker=redis://redis:6379/4
depends_on:
- db
- redis
- cassandra
Finally the database settings are (consolidated from multiple settings files):
if not os.environ.has_key('DOCKER'):
DATABASES = {
"default": {
'ENGINE': 'django.db.backends.mysql',
'OPTIONS': {
'read_default_file': os.path.join(PROJECT_ROOT, 'mysql.cnf'),
"init_command": "SET foreign_key_checks = 0;",
},
}
}
else: # Using Docker
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.mysql',
'NAME': os.environ['DATABASE_NAME'],
'USER': os.environ['DATABASE_USER'],
'PASSWORD': os.environ['DATABASE_PASSWORD'],
'HOST': os.environ["DATABASE_HOST"],
'PORT': '5432',
'OPTIONS': {
'init_command': "SET foreign_key_checks = 0;"
}
}
}
if 'test' in sys.argv:
SLAVE_DATABASES = []
elif os.environ.has_key('DOCKER'):
DATABASES['replica_1'] = {
'ENGINE': 'django.db.backends.mysql',
'NAME': os.environ['DATABASE_NAME'],
'USER': os.environ['DATABASE_USER'],
'PASSWORD': os.environ['DATABASE_PASSWORD'],
'HOST': os.environ["DATABASE_HOST"],
'PORT': '5432',
'OPTIONS': {
"init_command": "SET foreign_key_checks = 0;"
}
}
SLAVE_DATABASES = ['replica_1', ]
DATABASE_ROUTERS = ('multidb.PinningMasterSlaveRouter',)
MIDDLEWARE_CLASSES = ['multidb.middleware.PinningRouterMiddleware', ] + settings.MIDDLEWARE_CLASSES
else:
DATABASES["replica_1"] = {
'ENGINE': 'django.db.backends.mysql',
'OPTIONS': {
'read_default_file': os.path.join(settings.PROJECT_ROOT, 'mysql.cnf'),
"init_command": "SET foreign_key_checks = 0;",
},
'TEST': {
'MIRROR': 'default',
},
}
SLAVE_DATABASES = ['replica_1', ]
DATABASE_ROUTERS = ('multidb.PinningMasterSlaveRouter',)
MIDDLEWARE_CLASSES = ['multidb.middleware.PinningRouterMiddleware', ] + settings.MIDDLEWARE_CLASSES
if os.environ.has_key("DOCKER"):
from cassandra import ConsistencyLevel
DATABASES['cassandra'] = {
'ENGINE': 'django_cassandra_engine',
'NAME': 'verificient_local',
'TEST_NAME': 'test_verificient_local',
'HOST': os.environ['CASSANDRA_HOST'],
'OPTIONS': {
'replication': {
'strategy_class': 'SimpleStrategy',
'replication_factor': 1
},
'connection': {
'consistency': ConsistencyLevel.LOCAL_ONE,
'retry_connect': True
},
'session': {
'default_timeout': 30,
'default_fetch_size': 10000
}
}
}
The weird part is that even though running python manage.py migrate returns an error (django.db.utils.OperationalError: (2003, "Can't connect to MySQL server on 'db' (111)")) I am able to connect to the database with no issue using the command mysql -h$DATABASE_HOST -u$DATABASE_USER -p$DATABASE_PASSWORD $DATABASE_NAME
What could be going wrong in my django setup?
MySQL default port is 3306, so change:
'PORT': '5432',
to
'PORT': '3306',
It does work manually with your mysql command because that command uses the correct port by default
Related
I am trying to write a script that will spin up a docker container MySQL DB that can connect with my Nest.JS API. The goal is to use this across multiple people for easy setup of environment.
I plan to run
nest start --watch
will start the API. I am just struggling with setting up/connecting to the DB.
I have a docker-compose.yml that looks like this:
version: "3"
services:
# MySQL database
db:
image: mysql:8
container_name: db
command: --default-authentication-plugin=mysql_native_password --character-set-server=utf8mb4 --collation-server=utf8mb4_0900_ai_ci
restart: always
ports:
- 3306:3306
environment:
MYSQL_ROOT_PASSWORD: root
MYSQL_PASSWORD: root
MYSQL_DATABASE: db_name
and my databaseProvider looks like :
export const databaseProviders = [
{
provide: DATABASE_CONNECTION,
useFactory: async () => await createConnection({
type: 'mysql',
host: 'localhost',
port: 3306,
username: 'root',
password: 'root',
database: 'db_name',
entities: [
__dirname + '/../**/*.entity{.ts,.js}',
],
synchronize: true,
}),
},
];
I get the error :
Error: connect ECONNREFUSED 127.0.0.1:3306
Can anyone help me out with this?
I'm trying to connect Django project with MySQL using docker.
I have the problem when upping docker-compose. It says the next error:
super(Connection, self).init(*args, **kwargs2)
django.db.utils.OperationalError: (2002, "Can't connect to MySQL server on 'db' (115)")
I'm using port 3307, should i create first new database schema in my local? Or how can I do it because my default port is 3306 but if i try to use it, it says that is busy.
My code here:
Dockerfile
FROM python:3.7
ENV PYTHONUNBUFFERED 1
ENV LANG=C.UTF-8
RUN mkdir /code
WORKDIR /code
ADD requirements.txt /code/
RUN apt-get update
RUN pip install -r requirements.txt
ADD . /code/
Docker-compose
version: '2'
services:
app:
container_name: container_app
build:
context: .
dockerfile: Dockerfile
restart: always
command: bash -c "python3 manage.py migrate && python manage.py shell < backend/scripts/setup.py && python3 manage.py runserver 0.0.0.0:8000"
links:
- db
depends_on:
- db
ports:
- "8000:8000"
db:
container_name: container_database
image: mariadb
restart: always
environment:
MYSQL_ROOT_HOST: 'host.docker.internal'
MYSQL_DATABASE: 'container_develop'
MYSQL_USER: 'root'
MYSQL_PASSWORD: 'password'
MYSQL_ROOT_PASSWORD: 'password'
ports:
- "3307:3307"
settings.py:
DATABASES = {
'default' : {
'ENGINE': 'django.db.backends.mysql',
'NAME': 'database_develop',
'USER': 'root',
'PASSWORD': 'password',
'HOST': 'db',
'PORT': 3307,
'CHARSET': 'utf8',
'COLLATION': 'utf8_bin',
'OPTIONS': {
'use_unicode' : True,
'init_command': 'SET SESSION TRANSACTION ISOLATION LEVEL READ COMMITTED',
},
}
}
The thing is that I do not have the database anywhere so I'm using it locally, do I have to upload the db to a server and then use the port that server provides to me?
Is there any way to use it locally from docker? Or installing it to docker? So I can share that docker to a friend and he can use the same DB?
In the "db" part, in your docker-compose :
ports:
- "3307:3306"
The #Nico answer my fixed your service accessibility from outside of the container, mean if try to access it from the host it should work.
But the error will arise during service to service communication. You should not use publish port in service to service communication or another word you must use container port in service to service communication.
Change the port in connection from 3307 to 3306
DATABASES = {
'default' : {
'ENGINE': 'django.db.backends.mysql',
'NAME': 'database_develop',
'USER': 'root',
'PASSWORD': 'password',
'HOST': 'db',
'PORT': 3306,
'CHARSET': 'utf8',
'COLLATION': 'utf8_bin',
'OPTIONS': {
'use_unicode' : True,
'init_command': 'SET SESSION TRANSACTION ISOLATION LEVEL READ COMMITTED',
},
}
}
I have a docker-compose like a bellow:
version: '3'
services:
mysql:
image: mysql
ports:
- 3306:3306
env_file: ./src/.environment
volumes:
- mysql_data:/var/lib/mysql
web:
build:
context: ./src
dockerfile: Dockerfile
command: bash -c "python manage.py collectstatic --no-input && python manage.py makemigrations --no-input && python manage.py migrate --no-input"
env_file: ./src/.environment
ports:
- 8001:8001
depends_on:
- mysql
volumes:
- ./src:/var/www/project/src
- staticfiles:/var/www/project/src/statics
.environment content as follow:
DATABASE_MYSQL_NAME=dataCollection
DATABASE_MYSQL_HOST=mysql
DATABASE_MYSQL_USER=user1
DATABASE_MYSQL_PASSWORD=user123456
DATABASE_MYSQL_PORT=3306
Then in django setting I have follow database configuration:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.mysql',
'NAME': os.getenv('DATABASE_MYSQL_NAME'),
'HOST': os.getenv('DATABASE_MYSQL_HOST'),
'USER': os.getenv('DATABASE_MYSQL_USER'),
'PASSWORD': os.getenv('DATABASE_MYSQL_PASSWORD'),
'PORT': os.getenv('DATABASE_MYSQL_PORT'),
}
}
But, when I run docker-comose up it throws the following error:
MySQLdb._exceptions.OperationalError: (2002, "Can't connect to MySQL server on 'mysql' (115)")
I don't know what the problem is?
I'm trying to connect Django project with MySQL using docker.
I have the problem when upping docker-compose. It says the next error:
super(Connection, self).init(*args, **kwargs2)
django.db.utils.OperationalError: (2002, "Can't connect to MySQL server on 'db' (115)")
I'm using port 3307, should i create first new database schema in my local? Or how can I do it because my default port is 3306 but if i try to use it, it says that is busy.
My code here:
Dockerfile
FROM python:3.7
ENV PYTHONUNBUFFERED 1
ENV LANG=C.UTF-8
RUN mkdir /code
WORKDIR /code
ADD requirements.txt /code/
RUN apt-get update
RUN pip install -r requirements.txt
ADD . /code/
Docker-compose
version: '2'
services:
app:
container_name: container_app
build:
context: .
dockerfile: Dockerfile
restart: always
command: bash -c "python3 manage.py migrate && python manage.py shell < backend/scripts/setup.py && python3 manage.py runserver 0.0.0.0:8000"
links:
- db
depends_on:
- db
ports:
- "8000:8000"
db:
container_name: container_database
image: mariadb
restart: always
environment:
MYSQL_ROOT_HOST: 'host.docker.internal'
MYSQL_DATABASE: 'container_develop'
MYSQL_USER: 'root'
MYSQL_PASSWORD: 'password'
MYSQL_ROOT_PASSWORD: 'password'
ports:
- "3307:3307"
settings.py:
DATABASES = {
'default' : {
'ENGINE': 'django.db.backends.mysql',
'NAME': 'database_develop',
'USER': 'root',
'PASSWORD': 'password',
'HOST': 'db',
'PORT': 3307,
'CHARSET': 'utf8',
'COLLATION': 'utf8_bin',
'OPTIONS': {
'use_unicode' : True,
'init_command': 'SET SESSION TRANSACTION ISOLATION LEVEL READ COMMITTED',
},
}
}
The thing is that I do not have the database anywhere so I'm using it locally, do I have to upload the db to a server and then use the port that server provides to me?
Is there any way to use it locally from docker? Or installing it to docker? So I can share that docker to a friend and he can use the same DB?
In the "db" part, in your docker-compose :
ports:
- "3307:3306"
The #Nico answer my fixed your service accessibility from outside of the container, mean if try to access it from the host it should work.
But the error will arise during service to service communication. You should not use publish port in service to service communication or another word you must use container port in service to service communication.
Change the port in connection from 3307 to 3306
DATABASES = {
'default' : {
'ENGINE': 'django.db.backends.mysql',
'NAME': 'database_develop',
'USER': 'root',
'PASSWORD': 'password',
'HOST': 'db',
'PORT': 3306,
'CHARSET': 'utf8',
'COLLATION': 'utf8_bin',
'OPTIONS': {
'use_unicode' : True,
'init_command': 'SET SESSION TRANSACTION ISOLATION LEVEL READ COMMITTED',
},
}
}
I'm very new for docker, now I am trying to run django with mariadb in docker through docker-compose, but I always get this error:
I use Docker version 17.09.1-ce, build 19e2cf6, docker-compose version 1.18.0, build 8dd22a9
django.db.utils.OperationalError: (2003, 'Can\'t connect to MySQL
server on \'mariadb55\' (111 "Connection refused")')
I can connect db correctly after run docker-compose up db in local or remote, and I even can run python manage.py runserver 0.0.0.0:6001 correctly in anaconda virtual environment to connect db service in docker by setting parameters of settings.py file like below:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.mysql',
'NAME': 'test',
'USER': 'belter',
# 'HOST': 'mariadb55',
'HOST': '127.0.0.1',
'PORT': '3302',
'PASSWORD': 'belter_2017',
'default-character-set': 'utf8',
'OPTIONS': {
'sql_mode': 'traditional',
}
}
}
This is my docker-compose.yml file
version: '3'
services:
db:
image: mariadb:5.5
restart: always
environment:
- MYSQL_HOST=localhost
- MYSQL_PORT=3306
- MYSQL_ROOT_HOST=%
- MYSQL_DATABASE=test
- MYSQL_USER=belter
- MYSQL_PASSWORD=belter_2017
- MYSQL_ROOT_PASSWORD=123456_abc
volumes:
- /home/belter/mdbdata/mdb55:/var/lib/mysql
ports:
- "3302:3306"
web:
image: onlybelter/django_py35
command: python3 manage.py runserver 0.0.0.0:6001
volumes:
- /mnt/data/www/mysite:/djcode
ports:
- "6001:6001"
depends_on:
- db
links:
- db:mariadb55
I almost tried everything I can find, but still cannot figure it out, any help would be nice!
What I have tried:
Docker compose mysql connection failing
Linking django and mysql containers using docker-compose
Django connection to postgres by docker-compose
Finally, I figured it out!
The key point is, just as #SangminKim said, I need to use 3306 not 3302 in settings.py, and use db as HOST not 127.0.0.1.
So this is my docker-compose.yml file now:
version: '3'
services:
db:
image: mariadb:5.5
restart: always
environment:
- MYSQL_HOST=localhost
- MYSQL_PORT=3306 # cannot change this port to other number
- MYSQL_ROOT_HOST=%
- MYSQL_DATABASE=test
- MYSQL_USER=belter
- MYSQL_PASSWORD=belter_2017
- MYSQL_ROOT_PASSWORD=123456_abc
volumes:
- /home/belter/mdbdata/mdb55:/var/lib/mysql
ports:
- "3302:3306"
web:
image: onlybelter/django_py35
command: python3 manage.py runserver 0.0.0.0:6001
volumes:
- .:/djcode
ports:
- "6001:6001"
depends_on:
- db
So now we can connect this docker-mysql by mysql -h 127.0.0.1 -P 3302 -u root -p in shell directly, but we have to use db and 3306 in django settings.py file:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.mysql',
'NAME': 'test',
'USER': 'belter',
# 'HOST': 'mariadb55',
'HOST': 'db', #<---
'PORT': '3306', #<---
'PASSWORD': 'belter_2017',
'default-character-set': 'utf8',
'OPTIONS': {
'sql_mode': 'traditional',
}
}
}
And we can still check if this port is open, by running extra command in docker-compose.yml file:
...
web:
image: onlybelter/django_py35
command: /bin/sh -c "python check_db.py --service-name mysql --ip db --port 3306"
volumes:
- .:/djcode
...
Here is check_db.py file:
# check_db.py
import socket
import time
import argparse
""" Check if port is open, avoid docker-compose race condition """
parser = argparse.ArgumentParser(description='Check if port is open, avoid\
docker-compose race condition')
parser.add_argument('--service-name', required=True)
parser.add_argument('--ip', required=True)
parser.add_argument('--port', required=True)
args = parser.parse_args()
# Get arguments
service_name = str(args.service_name)
port = int(args.port)
ip = str(args.ip)
# Infinite loop
while True:
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
result = sock.connect_ex((ip, port))
if result == 0:
print("{0} port is open! Bye!".format(service_name))
break
else:
print("{0} port is not open! I'll check it soon!".format(service_name))
time.sleep(3)
By the way, this is my Dockerfile for build django-py35:
FROM python:3.5-alpine
MAINTAINER Xin Xiong "xiongxin20008#126.com"
ENV PYTHONUNBUFFERED 1
RUN set -e; \
apk add --no-cache --virtual .build-deps \
gcc \
libc-dev \
linux-headers \
mariadb-dev \
python3-dev \
postgresql-dev \
freetype-dev \
libpng-dev \
g++ \
;
RUN mkdir /djcode
WORKDIR /djcode
ENV REFRESHED_AT 2017-12-25
ADD requirements.txt /djcode/
RUN pip install --no-cache-dir -r /djcode/requirements.txt
RUN pip install uwsgi
ADD . /djcode/ # copy . to /djcode/
EXPOSE 6001
See more details from here: https://github.com/OnlyBelter/django-compose
You should use the container name instead of localhost (or 127.0.0.1) in your settings.py file. Try providing a container name to the db service in the docker-compose.yml file using container_name attribute and replace the host name in the settings.py by the value of the container_name. (Make sure that they are in the same network that docker compose creates for you.)
Build container with this:
docker run --name mysql-latest \
-p 3306:3306 -p 33060:33060 \
-e MYSQL_ROOT_HOST='%' -e MYSQL_ROOT_PASSWORD='strongpassword' \
-d mysql/mysql-server:latest
Make sure MYSQL_ROOT_HOST='%', that means root can connect from any IP.