How to migrate from docker-compose to vagrant? - mysql

Here is my docker-compose file that works fine and I want to reproduce the same results using Vagrant:
version: '3.7'
services:
db:
image: mysql:5.7.36
restart: always
environment:
MYSQL_ROOT_PASSWORD: root
MYSQL_DATABASE: test_db
ports:
- "3308:3306"
phpmyadmin:
image: phpmyadmin/phpmyadmin:latest
restart: always
environment:
PMA_HOST: db
PMA_USER: root
PMA_PASSWORD: root
ports:
- "8080:80"
Execute docker-compose up and visit localhost:8080 phpmyadmin works fine.
When I try to the same with vagrant containers are built and they are running, but phpmyadmin is unable to communicate with mysql container.
Here is my Vagrantfile:
# -*- mode: ruby -*-
# vi: set ft=ruby :
Vagrant.configure("2") do |config|
config.vm.define "mysql" do |db|
db.vm.network "forwarded_port", guest: 3306, host: 3308
db.vm.hostname = "mysqldb"
db.vm.provider "docker" do |d|
d.image = "mysql:5.7.36"
d.env = {
:MYSQL_ROOT_PASSWORD => "root",
:MYSQL_DATBASE => "test_db"
}
d.remains_running = "true"
end
end
config.vm.define "phpmyadmin" do |pa|
pa.vm.network "forwarded_port", guest: 80, host: 8080
pa.vm.hostname = "phpmyadmin"
pa.vm.provider "docker" do |d|
d.image = "phpmyadmin/phpmyadmin:latest"
d.env = {
:PMA_HOST => "mysqldb",
:PMA_USER => "root",
:PMA_PASSWORD => "root"
}
d.remains_running = "true"
end
end
end
How can I get the phpmyadmin and MySQL working together with vagrant?

I got the solution. I need to define the network at the top level.
# -*- mode: ruby -*-
# vi: set ft=ruby :
# This file works exactly like the docker-compose.yml file.
Vagrant.configure("2") do |config|
# Define the network, using which the containers can communicate
config.vm.network :private_network, type: "dhcp"
config.vm.define "mysql" do |db|
# Without prot forwardign vagrant entwork doesnt work
db.vm.network "forwarded_port", guest: 3306, host: 3306
db.vm.hostname = "mysqldb"
db.vm.provider "docker" do |d|
d.image = "mysql:5.7.36"
d.env = {
:MYSQL_ROOT_PASSWORD => "root",
:MYSQL_DATBASE => "test_db"
}
d.remains_running = "true"
end
end
config.vm.define "phpmyadmin" do |pa|
pa.vm.network "forwarded_port", guest: 80, host: 8080
pa.vm.hostname = "phpmyadmin"
pa.vm.provider "docker" do |d|
d.image = "phpmyadmin/phpmyadmin:latest"
d.env = {
:PMA_HOST => "mysqldb",
:PMA_USER => "root",
:PMA_PASSWORD => "root",
# Without specifying the specific port phpadmin container doesn't work
:PMA_PORT => "3306",
}
d.remains_running = "true"
end
end
end
I still need to solve one more problem - how to define a dependency between vms. Build the VM phpmyadmin only after mysql is up and running.
Unlike docker-compose it is not possible to set a direct dependency in vagrant. But it is possible to do a sequential build by setting the option --no-parallel eg:- vagrant up --no-parallel.
Even though docker-compose has the dependency set, the dependency is only to the extent of waiting for the dependent container to be built but not waiting for the services in the container to be up and running.
Hence even after building sequentially in vagrant, the dependent services failed to connect. It did not matter vagrant or docker-compose. Services with in the containers need to wait for their dependent services to be up.
So I added a delay in the dependent containers in a timed loop, the dependent service will attempt to connect and upon failure will wait for 30 seconds to connect again, after 10 attempts it gives up.

Related

ECONNREFUSED 3306 in Node.js connect to MySQL Container using Docker-Compose [duplicate]

Before you flag this question as a duplicate, please note that I did read other answers, but it didn't solve my problem.
I have a Docker compose file consisting of two services:
version: "3"
services:
mysql:
image: mysql:5.7
environment:
MYSQL_HOST: localhost
MYSQL_DATABASE: mydb
MYSQL_USER: mysql
MYSQL_PASSWORD: 1234
MYSQL_ROOT_PASSWORD: root
ports:
- "3307:3306"
expose:
- 3307
volumes:
- /var/lib/mysql
- ./mysql/migrations:/docker-entrypoint-initdb.d
restart: unless-stopped
web:
build:
context: .
dockerfile: web/Dockerfile
volumes:
- ./:/web
ports:
- "3000:3000"
environment:
NODE_ENV: development
PORT: 3000
links:
- mysql:mysql
depends_on:
- mysql
expose:
- 3000
command: ["./wait-for-it.sh", "mysql:3307"]
/web/Dockerfile:
FROM node:6.11.1
RUN mkdir -p /usr/src/app
WORKDIR /usr/src/app
COPY package.json /usr/src/app/
RUN npm install
COPY . /usr/src/app
CMD [ "npm", "start" ]
After docker-compose up --build the services start up, however the "wait-for-it.sh" script times out when waiting for mySQL to start (so temporarily I am not using it when testing for DB connectivity, I just wait until the console shows that MySQL is ready for accepting incoming connections)
When MySQL is running from the host machine I can login using Sequel Pro and query the DB and get the sample records from ./mysql/migrations
I can also SSH into the running MySQL container and do the same.
However, my Node.js app yields ECONNREFUSED 127.0.0.1:3307 when connecting
MySQL init:
import * as mysql from 'promise-mysql'
const config = {
host: 'localhost',
database: 'mydb',
port: '3307',
user: 'mysql',
password: '1234',
connectionLimit: 10
}
export let db = mysql.createPool(config);
MySQL query:
import { db } from '../db/client'
export let get = () => {
return db.query('SELECT * FROM users', [])
.then((results) => {
return results
})
.catch((e) => {
return Promise.reject(e)
})
}
Route invoked when hitting url /
import { Router } from 'express';
import * as repository from '../repository'
export let router = Router();
router.get('/', async (req, res) => {
let users;
try{
users = await repository.users.get();
} catch(e){
// ECONNREFUSED 127.0.0.1:3307
}
res.render('index', {
users: users
});
});
It's unlikely to be a race condition because at the same time when Node.js fails I can query using Sequel Pro or SSH into the running Docker container and query. So it's probably a case of Node.js not being able to access to MySQL container?
{
error: connect ECONNREFUSED 127.0.0.1:3307
code: 'ECONNREFUSED',
errno: 'ECONNREFUSED',
syscall: 'connect',
address: '127.0.0.1',
port: 3307,
fatal: true
}
This:
mysql:
image: mysql:5.7
environment:
...
ports:
- "3307:3306"
Means that Docker will map the 3307 port of the host to the 3306 port of the container. So you can access from Sequel to localhost:3307.
However, it does not mean that the container is listenting to 3307; the container is in fact still listening to 3306. When others containers tries to access the mysql DNS, it gets translated to the internal container IP, therefore you must connect to 3306.
So your node config should look like:
const config = {
host: 'mysql',
database: 'mydb',
port: '3306',
user: 'mysql',
password: '1234',
connectionLimit: 10
}
And this in your docker-compose.yml:
command: ["./wait-for-it.sh", "mysql:3306"]
Note: wait-for-it.sh script comes from: https://github.com/vishnubob/wait-for-it

Dockerize Django project with MySQL container [duplicate]

I'm trying to connect Django project with MySQL using docker.
I have the problem when upping docker-compose. It says the next error:
super(Connection, self).init(*args, **kwargs2)
django.db.utils.OperationalError: (2002, "Can't connect to MySQL server on 'db' (115)")
I'm using port 3307, should i create first new database schema in my local? Or how can I do it because my default port is 3306 but if i try to use it, it says that is busy.
My code here:
Dockerfile
FROM python:3.7
ENV PYTHONUNBUFFERED 1
ENV LANG=C.UTF-8
RUN mkdir /code
WORKDIR /code
ADD requirements.txt /code/
RUN apt-get update
RUN pip install -r requirements.txt
ADD . /code/
Docker-compose
version: '2'
services:
app:
container_name: container_app
build:
context: .
dockerfile: Dockerfile
restart: always
command: bash -c "python3 manage.py migrate && python manage.py shell < backend/scripts/setup.py && python3 manage.py runserver 0.0.0.0:8000"
links:
- db
depends_on:
- db
ports:
- "8000:8000"
db:
container_name: container_database
image: mariadb
restart: always
environment:
MYSQL_ROOT_HOST: 'host.docker.internal'
MYSQL_DATABASE: 'container_develop'
MYSQL_USER: 'root'
MYSQL_PASSWORD: 'password'
MYSQL_ROOT_PASSWORD: 'password'
ports:
- "3307:3307"
settings.py:
DATABASES = {
'default' : {
'ENGINE': 'django.db.backends.mysql',
'NAME': 'database_develop',
'USER': 'root',
'PASSWORD': 'password',
'HOST': 'db',
'PORT': 3307,
'CHARSET': 'utf8',
'COLLATION': 'utf8_bin',
'OPTIONS': {
'use_unicode' : True,
'init_command': 'SET SESSION TRANSACTION ISOLATION LEVEL READ COMMITTED',
},
}
}
The thing is that I do not have the database anywhere so I'm using it locally, do I have to upload the db to a server and then use the port that server provides to me?
Is there any way to use it locally from docker? Or installing it to docker? So I can share that docker to a friend and he can use the same DB?
In the "db" part, in your docker-compose :
ports:
- "3307:3306"
The #Nico answer my fixed your service accessibility from outside of the container, mean if try to access it from the host it should work.
But the error will arise during service to service communication. You should not use publish port in service to service communication or another word you must use container port in service to service communication.
Change the port in connection from 3307 to 3306
DATABASES = {
'default' : {
'ENGINE': 'django.db.backends.mysql',
'NAME': 'database_develop',
'USER': 'root',
'PASSWORD': 'password',
'HOST': 'db',
'PORT': 3306,
'CHARSET': 'utf8',
'COLLATION': 'utf8_bin',
'OPTIONS': {
'use_unicode' : True,
'init_command': 'SET SESSION TRANSACTION ISOLATION LEVEL READ COMMITTED',
},
}
}

Docker-Compose can't connect to MySQL

I'm trying to connect Django project with MySQL using docker.
I have the problem when upping docker-compose. It says the next error:
super(Connection, self).init(*args, **kwargs2)
django.db.utils.OperationalError: (2002, "Can't connect to MySQL server on 'db' (115)")
I'm using port 3307, should i create first new database schema in my local? Or how can I do it because my default port is 3306 but if i try to use it, it says that is busy.
My code here:
Dockerfile
FROM python:3.7
ENV PYTHONUNBUFFERED 1
ENV LANG=C.UTF-8
RUN mkdir /code
WORKDIR /code
ADD requirements.txt /code/
RUN apt-get update
RUN pip install -r requirements.txt
ADD . /code/
Docker-compose
version: '2'
services:
app:
container_name: container_app
build:
context: .
dockerfile: Dockerfile
restart: always
command: bash -c "python3 manage.py migrate && python manage.py shell < backend/scripts/setup.py && python3 manage.py runserver 0.0.0.0:8000"
links:
- db
depends_on:
- db
ports:
- "8000:8000"
db:
container_name: container_database
image: mariadb
restart: always
environment:
MYSQL_ROOT_HOST: 'host.docker.internal'
MYSQL_DATABASE: 'container_develop'
MYSQL_USER: 'root'
MYSQL_PASSWORD: 'password'
MYSQL_ROOT_PASSWORD: 'password'
ports:
- "3307:3307"
settings.py:
DATABASES = {
'default' : {
'ENGINE': 'django.db.backends.mysql',
'NAME': 'database_develop',
'USER': 'root',
'PASSWORD': 'password',
'HOST': 'db',
'PORT': 3307,
'CHARSET': 'utf8',
'COLLATION': 'utf8_bin',
'OPTIONS': {
'use_unicode' : True,
'init_command': 'SET SESSION TRANSACTION ISOLATION LEVEL READ COMMITTED',
},
}
}
The thing is that I do not have the database anywhere so I'm using it locally, do I have to upload the db to a server and then use the port that server provides to me?
Is there any way to use it locally from docker? Or installing it to docker? So I can share that docker to a friend and he can use the same DB?
In the "db" part, in your docker-compose :
ports:
- "3307:3306"
The #Nico answer my fixed your service accessibility from outside of the container, mean if try to access it from the host it should work.
But the error will arise during service to service communication. You should not use publish port in service to service communication or another word you must use container port in service to service communication.
Change the port in connection from 3307 to 3306
DATABASES = {
'default' : {
'ENGINE': 'django.db.backends.mysql',
'NAME': 'database_develop',
'USER': 'root',
'PASSWORD': 'password',
'HOST': 'db',
'PORT': 3306,
'CHARSET': 'utf8',
'COLLATION': 'utf8_bin',
'OPTIONS': {
'use_unicode' : True,
'init_command': 'SET SESSION TRANSACTION ISOLATION LEVEL READ COMMITTED',
},
}
}

Node.js connect to MySQL Docker container ECONNREFUSED

Before you flag this question as a duplicate, please note that I did read other answers, but it didn't solve my problem.
I have a Docker compose file consisting of two services:
version: "3"
services:
mysql:
image: mysql:5.7
environment:
MYSQL_HOST: localhost
MYSQL_DATABASE: mydb
MYSQL_USER: mysql
MYSQL_PASSWORD: 1234
MYSQL_ROOT_PASSWORD: root
ports:
- "3307:3306"
expose:
- 3307
volumes:
- /var/lib/mysql
- ./mysql/migrations:/docker-entrypoint-initdb.d
restart: unless-stopped
web:
build:
context: .
dockerfile: web/Dockerfile
volumes:
- ./:/web
ports:
- "3000:3000"
environment:
NODE_ENV: development
PORT: 3000
links:
- mysql:mysql
depends_on:
- mysql
expose:
- 3000
command: ["./wait-for-it.sh", "mysql:3307"]
/web/Dockerfile:
FROM node:6.11.1
RUN mkdir -p /usr/src/app
WORKDIR /usr/src/app
COPY package.json /usr/src/app/
RUN npm install
COPY . /usr/src/app
CMD [ "npm", "start" ]
After docker-compose up --build the services start up, however the "wait-for-it.sh" script times out when waiting for mySQL to start (so temporarily I am not using it when testing for DB connectivity, I just wait until the console shows that MySQL is ready for accepting incoming connections)
When MySQL is running from the host machine I can login using Sequel Pro and query the DB and get the sample records from ./mysql/migrations
I can also SSH into the running MySQL container and do the same.
However, my Node.js app yields ECONNREFUSED 127.0.0.1:3307 when connecting
MySQL init:
import * as mysql from 'promise-mysql'
const config = {
host: 'localhost',
database: 'mydb',
port: '3307',
user: 'mysql',
password: '1234',
connectionLimit: 10
}
export let db = mysql.createPool(config);
MySQL query:
import { db } from '../db/client'
export let get = () => {
return db.query('SELECT * FROM users', [])
.then((results) => {
return results
})
.catch((e) => {
return Promise.reject(e)
})
}
Route invoked when hitting url /
import { Router } from 'express';
import * as repository from '../repository'
export let router = Router();
router.get('/', async (req, res) => {
let users;
try{
users = await repository.users.get();
} catch(e){
// ECONNREFUSED 127.0.0.1:3307
}
res.render('index', {
users: users
});
});
It's unlikely to be a race condition because at the same time when Node.js fails I can query using Sequel Pro or SSH into the running Docker container and query. So it's probably a case of Node.js not being able to access to MySQL container?
{
error: connect ECONNREFUSED 127.0.0.1:3307
code: 'ECONNREFUSED',
errno: 'ECONNREFUSED',
syscall: 'connect',
address: '127.0.0.1',
port: 3307,
fatal: true
}
This:
mysql:
image: mysql:5.7
environment:
...
ports:
- "3307:3306"
Means that Docker will map the 3307 port of the host to the 3306 port of the container. So you can access from Sequel to localhost:3307.
However, it does not mean that the container is listenting to 3307; the container is in fact still listening to 3306. When others containers tries to access the mysql DNS, it gets translated to the internal container IP, therefore you must connect to 3306.
So your node config should look like:
const config = {
host: 'mysql',
database: 'mydb',
port: '3306',
user: 'mysql',
password: '1234',
connectionLimit: 10
}
And this in your docker-compose.yml:
command: ["./wait-for-it.sh", "mysql:3306"]
Note: wait-for-it.sh script comes from: https://github.com/vishnubob/wait-for-it

Connect to MySQL in a docker container (Vagrant on Windows/VirtualBox)

I am trying to create a virtualised dev environment on Windows using Vagrant and Docker (as are a lot of people). The problem I have is that I cannot connect (or I dont understand how to) from MySQL Workbench running on my Windows laptop to my MySQL DB in a Docker container in Boot2Docker. This is how I visualise the connection:
MySQL Work bench -> 3306 -> Boot2Docker -> 3306 -> Docker -> MySql
However I cannot connect to the DB from MySQLWorkbench. I have tried connection to the Boot2Docker host 10.0.2.15 on 3306 and tcp over ssh using the private key of the Boot2Docker box ".vagrant\machines\dockerhost\virtualbox\id"
What am I doing wrong/what have I misunderstood.
My Vagrantfile:
ENV['VAGRANT_DEFAULT_PROVIDER'] = 'docker'
DOCKER_HOST_NAME = "dockerhost"
DOCKER_HOST_VAGRANTFILE = "./host/Vagrantfile"
Vagrant.configure("2") do |config|
config.vm.network "forwarded_port", guest: 3306, host: 3306
config.vm.define "mysql" do |v|
v.vm.provider "docker" do |d|
d.image = "mysql"
d.env = {
:MYSQL_ROOT_PASSWORD => "root",
:MYSQL_DATABASE => "dockertest",
:MYSQL_USER => "dockertest",
:MYSQL_PASSWORD => "docker"
}
d.ports =["3306:3306"]
d.remains_running = "true"
d.vagrant_machine = "#{DOCKER_HOST_NAME}"
d.vagrant_vagrantfile = "#{DOCKER_HOST_VAGRANTFILE}"
end
end
end
My hosts/Vagrantfile (describing my Boot2docker host) is:
FORWARD_DOCKER_PORTS='true'
Vagrant.configure(2) do |config|
config.vm.provision "docker"
# The following line terminates all ssh connections. Therefore
# Vagrant will be forced to reconnect.
# That's a workaround to have the docker command in the PATH
#Clear any existing ssh connections
####NOTE: ps aux seems to give variable results depending on run -> process number can be ####first #or second causing provision to fail!!!
config.vm.provision "clear-ssh", type: "shell", inline:
"ps aux | grep 'sshd:' | awk '{print $1}' | xargs kill"
# "ps aux | grep 'sshd:' | awk '{print $2}' | xargs kill"
config.vm.define "dockerhost"
config.vm.box = "dduportal/boot2docker"
config.vm.network "forwarded_port",guest: 8080, host: 8080
config.vm.provider "virtualbox" do |vb|
vb.name = "dockerhost"
end
end
Solved. As I suspected the ports were not getting forwarded between the host machine and the docker host. The solution is to move the port forwarding config line:
config.vm.network "forwarded_port", guest: 3306, host: 3306
into the docker host Vagrant file. MySQL Workbench on the Windows host can then connect on localhost:3306.