dockerized django application cannot connect to mysql server on localhost - mysql

I have a Dockerized django application I am running and I am trying to connect it to a mysql server I have that is port forwarded from another docker container. I have done a sanity test already and confirmed that I can connect to my mysql server using mysql workbench on my localhost.
I have my dockerized django application running on network_mode: host so I thought I would be able to simply connect. Sadly I currently error out on docker-compose build with the error django.db.utils.OperationalError: (2002, "Can't connect to MySQL server on '127.0.0.1' (115)")
An accepted resolution to this issue means that my dockerized django application would be able to connect successfully to my mysql server running localhost:29998
SETTINGS.PY (Django Application)
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.mysql',
'NAME': 'mytestdb',
'USER': 'userone',
'PASSWORD': 'password',
'HOST': '127.0.0.1',
'PORT': '29998',
}
}
DJANGO App compose file
version: '3.3'
services:
mydjangoapp:
container_name: mydjangoapp
restart: always
env_file: .env
build: .
volumes:
- ./apps:/apps
- ./core:/core
network_mode: host
Django app dockerfile:
FROM python:3.9
COPY . .
# set environment variables
ENV PYTHONDONTWRITEBYTECODE 1
ENV PYTHONUNBUFFERED 1
COPY requirements.txt .
# install python dependencies
RUN pip install --upgrade pip
RUN pip install --no-cache-dir -r requirements.txt
# running migrations
RUN python manage.py migrate
# gunicorn
CMD ["gunicorn", "--config", "gunicorn-cfg.py", "core.wsgi"]
Dockerized mysql server (port forwarded to localhost)
version: '3.3'
services:
mysql:
image: mysql:latest
container_name: mymysqlserver
environment:
- MYSQL_DATABASE=mytestdb
- MYSQL_USER=userone
- MYSQL_ROOT_PASSWORD=password
- MYSQL_PASSWORD=password
ports:
- 29998:3306
Do I need to create some sort of docker network / bridge for this to work? (never tried that before).
I have already attempted the following solutions: sol1 (network_mode=host), sol2,

You django app is not working because:
You are running the containers in separated docker-compose files, this causes django container runs in different network than mysql container.
You are trying to connect to localhost (127.0.0.1) inside the django container. This localhost is different to 'localhost' of your computer and is different to the 'localhost' of mysql container. There are 3 different networks. If you want to connect django container with mysql container use the same network (docker network or your computer IP assigned by a router also will works).
You are trying to connect to the exposed port 29998, but this port is exposed from mysql container to your computer. If you are trying to make an internal connection you should use 3306. (If you are using an internal connection, then you don't need to expose the port)

Why not put the two services in same docker compose file and run it from there, like this:
version: '3.3'
services:
mydjangoapp:
container_name: mydjangoapp
restart: always
env_file: .env
build: .
volumes:
- ./apps:/apps
- ./core:/core
network_mode: host
depends_on: mysql
mysql:
image: mysql:latest
container_name: mymysqlserver
environment:
- MYSQL_DATABASE=mytestdb
- MYSQL_USER=userone
- MYSQL_ROOT_PASSWORD=password
- MYSQL_PASSWORD=password
ports:
- 29998:3306
And then update the settings like this:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.mysql',
'NAME': 'mytestdb',
'USER': 'userone',
'PASSWORD': 'password',
'HOST': 'mysql',
'PORT': '3306',
}
}
In that way, the communication between Django and MySQL will be done through docker network, rather than accessing the host machine network.
Apart from that, you need to change the Dockerfile, so that the migration runs after the MySQL server is running. To ensure that, you can add the migration command in the CMD part:
CMD ["sh", "-c", "python manage.py migrate;gunicorn --config gunicorn-cfg.py core.wsgi"]

Related

docker mysql ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) [duplicate]

I've been following with the docker-compose tutorial here (linking django and postgres container). Although I was able to go through with the tutorial I'm however not able to proceed with repeating the same
using a mysql container.
The following are my dockerfile and docker-compose.yml
`
db:
image: mysql
web:
build: .
command: python manage.py runserver 0.0.0.0:8000
volumes:
- .:/code
ports:
- "8000:8000"
links:
- db:db
`
dockerfile
FROM python:2.7
RUN mkdir /code
WORKDIR /code
RUN pip install mysql-python
RUN pip install django
They both build fine when I do docker-compose up but it seems the db environment variables are not passed to the django container since when I run os.environ.keys() in one of my django views I can't see any of the expected DB_* environment variables.
So does mysql require a different setup or am I missing something.
Thank you.
[EDIT]
Docker compose version
docker-compose version: 1.3.0
CPython version: 2.7.9
OpenSSL version: OpenSSL 1.0.1e 11 Feb 2013
Docker version
Docker version 1.6.2, build 7c8fca2
In Django settings.py file make sure you have something like:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.mysql',
'NAME': 'django1',
'USER': 'django',
'PASSWORD': 'password',
'HOST': 'db',
'PORT': 3306,
}
}
then in your docker-compose.yml file make sure you have something along the lines of:
db:
image: mysql
environment:
MYSQL_ROOT_PASSWORD: docker
MYSQL_DATABASE: docker
MYSQL_USER: docker
MYSQL_PASSWORD: docker
then as per the docker/django tutorial you are following run the following again to rebuild everything and things should start working
docker-compose run web django-admin.py startproject composeexample .
In response to a further question, the mysql root password variable is required by docker when creating new databases.
EDIT: added run to docker-compose above; see edit comment
you don't need to worry about environment variable. When linking containers together you just use the container alias defined by the link as if it was the hostname.
for instance if your docker-compose.yml file were:
db:
image: postgres
web:
build: .
command: python manage.py runserver 0.0.0.0:8000
volumes:
- .:/code
ports:
- "8000:8000"
links:
- db:mydb
In your django settings you would have to set the database host to mydb.
First you need to modify the settings file...
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql_psycopg2',
'NAME': 'postgres',
'USER': 'postgres',
'HOST': 'db',
'PORT': 5432,
} }
Then if you used the docker-compose command properly, the containers should be linked, and it should resolve the hostname db properly based on the links in the docker-compose.yml file.
Still, if you want to check the environment...
~/django-example: docker-compose run web env
Starting djangoexample_db_1...
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
HOSTNAME=66ff09ed8632
TERM=xterm
DJANGOEXAMPLE_DB_1_PORT=tcp://172.17.0.35:5432
DJANGOEXAMPLE_DB_1_PORT_5432_TCP=tcp://172.17.0.35:5432
DJANGOEXAMPLE_DB_1_PORT_5432_TCP_ADDR=172.17.0.35
DJANGOEXAMPLE_DB_1_PORT_5432_TCP_PORT=5432
DJANGOEXAMPLE_DB_1_PORT_5432_TCP_PROTO=tcp
DJANGOEXAMPLE_DB_1_NAME=/djangoexample_web_run_2/djangoexample_db_1
DJANGOEXAMPLE_DB_1_ENV_affinity:container==52c78c810792b0e7b9a231eab7ab7a3d50c95b76faf0abb8ec38a7d1ff0c7e5f
DJANGOEXAMPLE_DB_1_ENV_LANG=en_US.utf8
DJANGOEXAMPLE_DB_1_ENV_PG_MAJOR=9.4
DJANGOEXAMPLE_DB_1_ENV_PG_VERSION=9.4.4-1.pgdg70+1
DJANGOEXAMPLE_DB_1_ENV_PGDATA=/var/lib/postgresql/data
DB_PORT=tcp://172.17.0.35:5432
DB_PORT_5432_TCP=tcp://172.17.0.35:5432
DB_PORT_5432_TCP_ADDR=172.17.0.35
DB_PORT_5432_TCP_PORT=5432
DB_PORT_5432_TCP_PROTO=tcp
DB_NAME=/djangoexample_web_run_2/db
DB_ENV_affinity:container==52c78c810792b0e7b9a231eab7ab7a3d50c95b76faf0abb8ec38a7d1ff0c7e5f
DB_ENV_LANG=en_US.utf8
DB_ENV_PG_MAJOR=9.4
DB_ENV_PG_VERSION=9.4.4-1.pgdg70+1
DB_ENV_PGDATA=/var/lib/postgresql/data
DB_1_PORT=tcp://172.17.0.35:5432
DB_1_PORT_5432_TCP=tcp://172.17.0.35:5432
DB_1_PORT_5432_TCP_ADDR=172.17.0.35
DB_1_PORT_5432_TCP_PORT=5432
DB_1_PORT_5432_TCP_PROTO=tcp
DB_1_NAME=/djangoexample_web_run_2/db_1
DB_1_ENV_affinity:container==52c78c810792b0e7b9a231eab7ab7a3d50c95b76faf0abb8ec38a7d1ff0c7e5f
DB_1_ENV_LANG=en_US.utf8
DB_1_ENV_PG_MAJOR=9.4
DB_1_ENV_PG_VERSION=9.4.4-1.pgdg70+1
DB_1_ENV_PGDATA=/var/lib/postgresql/data
LANG=C.UTF-8
PYTHON_VERSION=2.7.10
PYTHON_PIP_VERSION=7.0.3
PYTHONUNBUFFERED=1
HOME=/root

Bitbucket pipeline with docker and mysql

I want to implement some CI/CD for a Django web using the bitbucket pipeline. The goal is: Test the Docker builds correctly and next run Django test.
But I get this error:
django.db.utils.OperationalError: (2003, 'Can\'t connect to MySQL server on \'127.0.0.1\' (111 "Connection refused")')
Here is the bitbucket-pipelines.yml:
options:
docker: true
definitions:
services:
mysql:
image: mysql:5.7
variables:
MYSQL_DATABASE: 'foo'
MYSQL_USER: 'default'
MYSQL_PASSWORD: 'default'
MYSQL_RANDOM_ROOT_PASSWORD: 'yes'
steps:
- step: &test-docker
name: "docker builds"
services:
- docker
- mysql
caches:
- docker
script:
- export IMAGE_NAME=foo:$BITBUCKET_COMMIT
- export CONTAINER_NAME=test-foo
- docker build -t $IMAGE_NAME .
- docker run -p 8080:8080 --name $CONTAINER_NAME --rm -d $IMAGE_NAME
- docker exec $CONTAINER_NAME python /app/manage.py test tests --noinput
- docker stop $CONTAINER_NAME
pipelines:
default:
- step: *test-docker
I already tried some solutions and workarounds like. Linking the ports from the services and the docker; using volumes; and test the Django outside docker. This one have more problems because it needs 2 DB (original and test) and a user with full access, and using the entrypoint didn't work.
The database address specified in your django application i.e., 127.0.0.1 only works if you have mysql running in you local system and not the container.
As you are using a docker image of mysql, it has it's own IP address and not 127.0.0.1. You can use docker inspect <container id> to get the IP address.
check this in you django settings.py file:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.mysql',
'NAME': 'foo',
'USER': 'default',
'PASSWORD': 'default',
'HOST': <mysql_ip>, #this is 127.0.0.1 in your case
'PORT': '3306',
}
}
If you are internally using docker services through docker-compose, you can simply access the mysql container using it's service name in the yml file.

Docker django mysql.sock in different location

I'm trying to containerize my django file, and I keep running into the issue:(2006, ’Can\‘t connect to local MySQL server through socket \‘/var/run/mysqld/mysqld.sock\’ (2 “No such file or directory”)
I found out later mysql.sock is in this location:/tmp/mysql.sock instead of /var/run/mysqld/mysqld.sock, how do I change the location for docker to see /tmp/mysql.sock
Here is my docker-composr.yml:
version: '3'
services:
db:
image: mysql
command: --default-authentication-plugin=mysql_native_password
restart: always
environment:
MYSQL_ROOT_PASSWORD: somepassword
adminer:
image: adminer
restart: always
ports:
- 8080:8080
web:
build: .
command: python manage.py runserver 0.0.0.0:8000
volumes:
- .:/code
ports:
- "8000:8000"
depends_on:
- db
I have followed the instructions on the mysql docker website to link mysql instance to a container
EDIT: I read another stack overflow similar to this, I changed my django code to 'HOST': '127.0.0.1' in DATABASES now I get : (2006, 'Can\'t connect to MySQL server on \'127.0.0.1\' (111 "Connection refused")')
Your host should be db. When using docker-compose, you address different servers by their service name.
So, in settings.py, you should have:
DATABASES = {
'default': {
'HOST': 'db',
...
}
}
If you want to connect to your containerized MySQL server both inside and outside of the container, you'll first need to make sure the port is mapped on the host machine:
services:
db:
image: mysql
ports:
- "3306:3306"
...
That will allow you to access MySQL using localhost or 127.0.0.1 directly on your host machine.
If you want to be able to run Django in both the web container and also on the host, you'll need to override the DATABASES setting depending upon the scenario. The web container will need to use a HOST value of db, whereas your local machine will need a value of localhost.

Connecting NodeJS Docker container to a MySQL Docker container - docker-compose 3

When I try to connect two docker containers on the same machine, one running a node.js server and the other running mysql dbms
I get the following error:
(node:32) UnhandledPromiseRejectionWarning: Error: getaddrinfo ENOTFOUND jdbc:mysql://localhost:3306 jdbc:mysql://localhost:3306:3306
mysql driver connection config
const connection= mysql.createConnection({
host: 'jdbc:mysql://topsectiondb:3306',
user: 'root',
password: 'rootpass'
})
docker-compose.yml
version: '3'
services:
topsection:
container_name: topsection-server
restart: always
build: .
ports:
- '7777:7777'
depends_on:
- topsectiondb
environment:
- PORT=7777
topsectiondb:
container_name: topsectiondb
image: mysql:8.0.3
ports:
- '3306:3306'
environment:
- MYSQL_ROOT_PASSWORD=rootpass
Dockerfile
FROM node:10
RUN mkdir serviceFolder
WORKDIR /usr/app/
COPY . .
RUN npm install
EXPOSE 7777
CMD ["npm", "start"]
for a more complete stack trace
https://gist.github.com/armouti/877a8b4405330c44e4009ebae3df822c
First of all there are a couple of problems here.
Firstly you need to look at networking your docker containers in your docker-compose file together. Basically each container doesn't know of the other until you specify which network they are in. Each container can be in multiple but if you want two containers to connect they need to be in the same network. https://docs.docker.com/compose/networking/#configure-the-default-network
Then Secondly you can't use localhost as a url. Basically a docker container is an isolated environment so it has it's own "localhost" so when you use that address it's actually trying to connect to the mysql on the nodejs container. So when networked properly you will be able to connect to the mysql container using their names you gave them in the docker-compose something like topsectiondb:3306 or something like this. This tells docker your using networking to connect one docker container to another and will allow you to make the initial connection. Connect to another container using Docker compose
===================================================
Actual Answer:
var mysql = require('mysql');
var connection = mysql.createConnection({
host : 'localhost',
user : 'me',
password : 'secret',
database : 'my_db'
});
Basically the mysql library requires a hostname, which in most cases is localhost, but as your linking containers with docker-compose you use the alias of the container. The port number is 3306 by default
So:
host: "topsectiondb" will suffice!

Linking django and mysql containers using docker-compose

I've been following with the docker-compose tutorial here (linking django and postgres container). Although I was able to go through with the tutorial I'm however not able to proceed with repeating the same
using a mysql container.
The following are my dockerfile and docker-compose.yml
`
db:
image: mysql
web:
build: .
command: python manage.py runserver 0.0.0.0:8000
volumes:
- .:/code
ports:
- "8000:8000"
links:
- db:db
`
dockerfile
FROM python:2.7
RUN mkdir /code
WORKDIR /code
RUN pip install mysql-python
RUN pip install django
They both build fine when I do docker-compose up but it seems the db environment variables are not passed to the django container since when I run os.environ.keys() in one of my django views I can't see any of the expected DB_* environment variables.
So does mysql require a different setup or am I missing something.
Thank you.
[EDIT]
Docker compose version
docker-compose version: 1.3.0
CPython version: 2.7.9
OpenSSL version: OpenSSL 1.0.1e 11 Feb 2013
Docker version
Docker version 1.6.2, build 7c8fca2
In Django settings.py file make sure you have something like:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.mysql',
'NAME': 'django1',
'USER': 'django',
'PASSWORD': 'password',
'HOST': 'db',
'PORT': 3306,
}
}
then in your docker-compose.yml file make sure you have something along the lines of:
db:
image: mysql
environment:
MYSQL_ROOT_PASSWORD: docker
MYSQL_DATABASE: docker
MYSQL_USER: docker
MYSQL_PASSWORD: docker
then as per the docker/django tutorial you are following run the following again to rebuild everything and things should start working
docker-compose run web django-admin.py startproject composeexample .
In response to a further question, the mysql root password variable is required by docker when creating new databases.
EDIT: added run to docker-compose above; see edit comment
you don't need to worry about environment variable. When linking containers together you just use the container alias defined by the link as if it was the hostname.
for instance if your docker-compose.yml file were:
db:
image: postgres
web:
build: .
command: python manage.py runserver 0.0.0.0:8000
volumes:
- .:/code
ports:
- "8000:8000"
links:
- db:mydb
In your django settings you would have to set the database host to mydb.
First you need to modify the settings file...
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql_psycopg2',
'NAME': 'postgres',
'USER': 'postgres',
'HOST': 'db',
'PORT': 5432,
} }
Then if you used the docker-compose command properly, the containers should be linked, and it should resolve the hostname db properly based on the links in the docker-compose.yml file.
Still, if you want to check the environment...
~/django-example: docker-compose run web env
Starting djangoexample_db_1...
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
HOSTNAME=66ff09ed8632
TERM=xterm
DJANGOEXAMPLE_DB_1_PORT=tcp://172.17.0.35:5432
DJANGOEXAMPLE_DB_1_PORT_5432_TCP=tcp://172.17.0.35:5432
DJANGOEXAMPLE_DB_1_PORT_5432_TCP_ADDR=172.17.0.35
DJANGOEXAMPLE_DB_1_PORT_5432_TCP_PORT=5432
DJANGOEXAMPLE_DB_1_PORT_5432_TCP_PROTO=tcp
DJANGOEXAMPLE_DB_1_NAME=/djangoexample_web_run_2/djangoexample_db_1
DJANGOEXAMPLE_DB_1_ENV_affinity:container==52c78c810792b0e7b9a231eab7ab7a3d50c95b76faf0abb8ec38a7d1ff0c7e5f
DJANGOEXAMPLE_DB_1_ENV_LANG=en_US.utf8
DJANGOEXAMPLE_DB_1_ENV_PG_MAJOR=9.4
DJANGOEXAMPLE_DB_1_ENV_PG_VERSION=9.4.4-1.pgdg70+1
DJANGOEXAMPLE_DB_1_ENV_PGDATA=/var/lib/postgresql/data
DB_PORT=tcp://172.17.0.35:5432
DB_PORT_5432_TCP=tcp://172.17.0.35:5432
DB_PORT_5432_TCP_ADDR=172.17.0.35
DB_PORT_5432_TCP_PORT=5432
DB_PORT_5432_TCP_PROTO=tcp
DB_NAME=/djangoexample_web_run_2/db
DB_ENV_affinity:container==52c78c810792b0e7b9a231eab7ab7a3d50c95b76faf0abb8ec38a7d1ff0c7e5f
DB_ENV_LANG=en_US.utf8
DB_ENV_PG_MAJOR=9.4
DB_ENV_PG_VERSION=9.4.4-1.pgdg70+1
DB_ENV_PGDATA=/var/lib/postgresql/data
DB_1_PORT=tcp://172.17.0.35:5432
DB_1_PORT_5432_TCP=tcp://172.17.0.35:5432
DB_1_PORT_5432_TCP_ADDR=172.17.0.35
DB_1_PORT_5432_TCP_PORT=5432
DB_1_PORT_5432_TCP_PROTO=tcp
DB_1_NAME=/djangoexample_web_run_2/db_1
DB_1_ENV_affinity:container==52c78c810792b0e7b9a231eab7ab7a3d50c95b76faf0abb8ec38a7d1ff0c7e5f
DB_1_ENV_LANG=en_US.utf8
DB_1_ENV_PG_MAJOR=9.4
DB_1_ENV_PG_VERSION=9.4.4-1.pgdg70+1
DB_1_ENV_PGDATA=/var/lib/postgresql/data
LANG=C.UTF-8
PYTHON_VERSION=2.7.10
PYTHON_PIP_VERSION=7.0.3
PYTHONUNBUFFERED=1
HOME=/root