I'm trying to use jest codes before deploy to dev. So i made docker-compose.yml and put "npm test (ENV=test jest --runInBand --forceExit test/**.test.ts -u)" but it has error.
This is my local.yml file (for docker-compose.yml)
version: "3"
services:
my-node:
image: my-api-server:dev
container_name: my_node
# sleep 10 sec for db init
command: bash -c "sleep 10; pwd; cd packages/server; yarn orm schema:sync -f ormconfig.dev.js; yarn db:migrate:run -f ormconfig.dev.js; npm test; cross-env ENV=dev node lib/server.js"
ports:
- "8082:8082"
depends_on:
- my-mysql
- my-redis
my-mysql:
image: mysql:5.7
container_name: my_mysql
command: --character-set-server=utf8mb4 --sql_mode="NO_ENGINE_SUBSTITUTION"
ports:
- "33079:3306"
volumes:
- ./init/:/docker-entrypoint-initdb.d/
environment:
- MYSQL_ROOT_PASSWORD=test
- MYSQL_DATABASE=test
- MYSQL_USER=test
- MYSQL_PASSWORD=test
my-redis:
image: redis:6.2-alpine
container_name: my_redis
ports:
- 6379:6379
command: redis-server --requirepass test
networks:
default:
external:
name: my_nginx_default
and it's my ormconfig.dev.js file
module.exports = {
type: 'mysql',
host: 'my_mysql',
port: 3306,
username: 'test',
password: 'test',
database: 'test',
entities: ['./src/modules/**/entities/*'],
migrations: ['migration/dev/*.ts'],
cli: { "migrationsDir": "migration/dev" }
}
But after i use docker-compose -f res/docker/local.yml up, it throws error after whole build and then jests. and then it opens server which does not have error.
Errors are like these below.
connect ECONNREFUSED 127.0.0.1:3306
and then
my_node | TypeError: Cannot read property 'subscriptionsPath' of undefined
my_node |
my_node | 116 | ): Promise<TestResponse<T>> => {
my_node | 117 | const req = request(server.app)
my_node | > 118 | .post(server.apolloServer!.subscriptionsPath!)
my_node | | ^
my_node | 119 | .set('Accept', 'application/json')
my_node | 120 |
my_node | 121 | if (token) {
I've tried to change entities path.
entities: ['./src/modules/**/entities/*']
entities: ['src/modules/**/entities/*']
entities: [__dirname + '/src/modules/**/entities/*']
My entities are in the right path.
Here is my whole file structure
Can anyone help this problem?
in your module.export host:my-mysql
Looking at the documentation, it could be an environment variable issue. As per the docs, the orm config file used is -
From the environment variables. Typeorm will attempt to load the .env file using dotEnv if it exists. If the environment variables TYPEORM_CONNECTION or TYPEORM_URL are set, Typeorm will use this method.
From the ormconfig.env.
From the other ormconfig.[format] files, in this order: [js, ts, json, yml, yaml, xml].
Since you have not defined the first two, it must be defaulting to use the ormconfig.js file. There is no reason it should pick ormconfig.dev.js.
If you can, change the ormconfig.js file to be this -
module.exports = {
type: 'mysql',
host: 'my_mysql',
port: 3306,
username: 'test',
password: 'test',
database: 'test',
entities: ['./src/modules/**/entities/*'],
migrations: ['migration/dev/*.ts'],
cli: { "migrationsDir": "migration/dev" }
}
Related
I am in the process of Dockerizing my project, and the following is my docker-compose.yml:
version: '3.8'
services:
server:
image: server
environment:
DB_HOST: 0.0.0.0
DB_DATABASE: dev
DB_PORT: 3306
DB_USER: dev
DB_PASSWORD: dev
ports:
- "4000:4000"
restart: on-failure
depends_on:
database:
condition: service_started
database:
image: mysql:latest
command: --default-authentication-plugin=mysql_native_password
ports:
- "3306:3306"
environment:
MYSQL_ROOT_PASSWORD: root
MYSQL_USER: dev
MYSQL_PASSWORD: dev
MYSQL_DATABASE: dev
volumes:
- mysql_data:/var/lib/mysql
volumes:
mysql_data:
and the Dockerfile for server:
FROM elixir:latest
EXPOSE 4000
RUN mkdir /server
COPY . /server
WORKDIR /server
RUN mix local.rebar --force
RUN mix local.hex --force
RUN mix deps.get
RUN mix do compile
CMD [ "sh", "/server/entry.sh" ]
where entry.sh is:
# Exit on failure
set -e
mix ecto.create
mix ecto.migrate
mix run priv/repo/seeds.exs
exec mix phx.server
When I run this using docker compose up, I get an error that Phoenix can't connect to the MySQL database. I have tried using hosts 0.0.0.0 and localhost, however both don't work.
This is the corresponding section of my config/dev.exs:
config :app, App.Repo
username: System.get_env("DB_USER"),
password: System.get_env("DB_PASSWORD"),
hostname: System.get_env("DB_HOST"),
database: System.get_env("DB_DATABASE"),
port: 3306,
stacktrace: true,
pool_size: 10
The following is the exact error message I get:
server-1 | 11:23:07.417 [error] GenServer #PID<0.272.0> terminating
server-1 | ** (DBConnection.ConnectionError) (0.0.0.0:3306) connection refused - :econnrefused
server-1 | (db_connection 2.4.3) lib/db_connection/connection.ex:100: DBConnection.Connection.connect/2
server-1 | (connection 1.1.0) lib/connection.ex:622: Connection.enter_connect/5
server-1 | (stdlib 4.2) proc_lib.erl:240: :proc_lib.init_p_do_apply/3
server-1 | Last message: nil
server-1 | State: MyXQL.Connection
server-1 |
server-1 | 11:23:07.430 [error] GenServer #PID<0.278.0> terminating
server-1 | ** (DBConnection.ConnectionError) (0.0.0.0:3306) connection refused - :econnrefused
server-1 | (db_connection 2.4.3) lib/db_connection/connection.ex:100: DBConnection.Connection.connect/2
server-1 | (connection 1.1.0) lib/connection.ex:622: Connection.enter_connect/5
server-1 | (stdlib 4.2) proc_lib.erl:240: :proc_lib.init_p_do_apply/3
server-1 | Last message: nil
server-1 | State: MyXQL.Connection
server-1 | ** (Mix) The database for App.Repo couldn't be created: %RuntimeError{message: "killed"}
server-1 exited with code 1
Before you flag this question as a duplicate, please note that I did read other answers, but it didn't solve my problem.
I have a Docker compose file consisting of two services:
version: "3"
services:
mysql:
image: mysql:5.7
environment:
MYSQL_HOST: localhost
MYSQL_DATABASE: mydb
MYSQL_USER: mysql
MYSQL_PASSWORD: 1234
MYSQL_ROOT_PASSWORD: root
ports:
- "3307:3306"
expose:
- 3307
volumes:
- /var/lib/mysql
- ./mysql/migrations:/docker-entrypoint-initdb.d
restart: unless-stopped
web:
build:
context: .
dockerfile: web/Dockerfile
volumes:
- ./:/web
ports:
- "3000:3000"
environment:
NODE_ENV: development
PORT: 3000
links:
- mysql:mysql
depends_on:
- mysql
expose:
- 3000
command: ["./wait-for-it.sh", "mysql:3307"]
/web/Dockerfile:
FROM node:6.11.1
RUN mkdir -p /usr/src/app
WORKDIR /usr/src/app
COPY package.json /usr/src/app/
RUN npm install
COPY . /usr/src/app
CMD [ "npm", "start" ]
After docker-compose up --build the services start up, however the "wait-for-it.sh" script times out when waiting for mySQL to start (so temporarily I am not using it when testing for DB connectivity, I just wait until the console shows that MySQL is ready for accepting incoming connections)
When MySQL is running from the host machine I can login using Sequel Pro and query the DB and get the sample records from ./mysql/migrations
I can also SSH into the running MySQL container and do the same.
However, my Node.js app yields ECONNREFUSED 127.0.0.1:3307 when connecting
MySQL init:
import * as mysql from 'promise-mysql'
const config = {
host: 'localhost',
database: 'mydb',
port: '3307',
user: 'mysql',
password: '1234',
connectionLimit: 10
}
export let db = mysql.createPool(config);
MySQL query:
import { db } from '../db/client'
export let get = () => {
return db.query('SELECT * FROM users', [])
.then((results) => {
return results
})
.catch((e) => {
return Promise.reject(e)
})
}
Route invoked when hitting url /
import { Router } from 'express';
import * as repository from '../repository'
export let router = Router();
router.get('/', async (req, res) => {
let users;
try{
users = await repository.users.get();
} catch(e){
// ECONNREFUSED 127.0.0.1:3307
}
res.render('index', {
users: users
});
});
It's unlikely to be a race condition because at the same time when Node.js fails I can query using Sequel Pro or SSH into the running Docker container and query. So it's probably a case of Node.js not being able to access to MySQL container?
{
error: connect ECONNREFUSED 127.0.0.1:3307
code: 'ECONNREFUSED',
errno: 'ECONNREFUSED',
syscall: 'connect',
address: '127.0.0.1',
port: 3307,
fatal: true
}
This:
mysql:
image: mysql:5.7
environment:
...
ports:
- "3307:3306"
Means that Docker will map the 3307 port of the host to the 3306 port of the container. So you can access from Sequel to localhost:3307.
However, it does not mean that the container is listenting to 3307; the container is in fact still listening to 3306. When others containers tries to access the mysql DNS, it gets translated to the internal container IP, therefore you must connect to 3306.
So your node config should look like:
const config = {
host: 'mysql',
database: 'mydb',
port: '3306',
user: 'mysql',
password: '1234',
connectionLimit: 10
}
And this in your docker-compose.yml:
command: ["./wait-for-it.sh", "mysql:3306"]
Note: wait-for-it.sh script comes from: https://github.com/vishnubob/wait-for-it
I'm trying to docerize my NodeJS API together with a MySQL image. Before the initial run, I want to run Sequelize migrations and seeds to have the tables up and ready to be served.
Here's my docker-compose.yaml:
version: '3.8'
services:
mysqldb:
image: mysql
restart: unless-stopped
environment:
MYSQL_ROOT_USER: myuser
MYSQL_ROOT_PASSWORD: mypassword
MYSQL_DATABASE: mydb
ports:
- '3306:3306'
networks:
- app-connect
volumes:
- db-config:/etc/mysql
- db-data:/var/lib/mysql
- ./db/backup/files/:/data_backup/data
app:
build:
context: .
dockerfile: ./Dockerfile
image: node-mysql-app
depends_on:
- mysqldb
ports:
- '3030:3030'
networks:
- app-connect
stdin_open: true
tty: true
volumes:
db-config:
db-data:
networks:
app-connect:
driver: bridge
Here's my app's Dockerfile:
FROM node:lts-alpine
WORKDIR /app
COPY package*.json ./
RUN npm install
COPY . .
EXPOSE 3030
ENV PORT 3030
ENV NODE_ENV docker
RUN npm run db:migrate:up
RUN npm run db:seeds:up
CMD [ "npm", "start" ]
And here's my default.db.json that the Sequelize migration uses (shortened):
{
"development": {
},
"production": {
},
"docker": {
"username": "myuser",
"password": "mypassword",
"database": "mydb",
"host": "mysqldb",
"port": "3306",
"dialect": "mysql"
}
}
Upon running compose up the DB installs well, the image deploys, but when it reaches the RUN npm run db:migrate:up (which translates into npx sequelize-cli db:migrate) I get the error:
npx: installed 81 in 13.108s
Sequelize CLI [Node: 14.17.0, CLI: 6.2.0, ORM: 6.6.2]
Loaded configuration file "default.db.json".
Using environment "docker".
ERROR: getaddrinfo EAI_AGAIN mysqldb
npm ERR! code ELIFECYCLE
npm ERR! errno 1
If I change the "host" in the default.db.json to "127.0.0.1", I get ERROR: connect ECONNREFUSED 127.0.0.1:3306 in place of the ERROR: getaddrinfo EAI_AGAIN mysqldb.
What am i doing wrong, and what host should I specify so the app can see the MySQL container? Should I remove the network? Should I change ports? (I tried combinations of both to no avail, so far).
I solved my issue by using Docker Compose Wait. Essentially, it adds a wait loop that samples the DB container, and only when it's up, runs migrations and seeds the DB.
My next problem was: those seeds ran every time the container was run - I solved that by instead running a script that runs the seeds, and touchs a semaphore file. If the file exists already, it skips the seeds.
The following configuration worked for me, I am adding the .env, sequelize configuration along with mysql database and docker. And finally don't forget to run docker-compose up --build cheers 🎁 🎁 🎁
.env
DB_NAME="testdb"
DB_USER="root"
DB_PASS="root"
DB_HOST="mysql"
.sequelizerc now we can use config.js rather than config.json for sequelize
const path = require('path');
module.exports = {
'config': path.resolve('config', 'config.js')
}
config.js
require("dotenv").config();
module.exports = {
development: {
username: process.env.DB_USER,
password: process.env.DB_PASS,
database: process.env.DB_NAME,
host: process.env.DB_HOST,
dialect: "mysql"
},
test: {
username: process.env.DB_USER,
password: process.env.DB_PASS,
database: process.env.DB_NAME,
host: process.env.DB_HOST,
dialect: "mysql"
},
production: {
username: process.env.DB_USER,
password: process.env.DB_PASS,
database: process.env.DB_NAME,
host: process.env.DB_HOST,
dialect: "mysql"
}
}
database-connection with sequelize
import Sequelize from 'sequelize';
import dbConfig from './config/config';
const conf = dbConfig.development;
const sequelize = new Sequelize(
conf.database,
conf.username,
conf.password,
{
host: conf.host,
dialect: "mysql",
operatorsAliases: 0,
logging: 0
}
);
sequelize.sync();
(async () => {
try {
await sequelize.authenticate();
console.log("Database connection setup successfully!");
} catch (error) {
console.log("Unable to connect to the database", error);
}
})();
export default sequelize;
global.sequelize = sequelize;
docker-compose.yaml
version: "3.8"
networks:
proxy:
name: proxy
services:
mysql:
image: mysql
networks:
- proxy
ports:
- 3306:3306
environment:
- MYSQL_ROOT_PASSWORD=root
- MYSQL_DATABASE=testdb
healthcheck:
test: "mysql -uroot -p$$MYSQL_ROOT_PASSWORD -e 'SHOW databases'"
interval: 10s
retries: 3
api:
build: ./node-backend
networks:
- proxy
ports:
- 3000:3000
depends_on:
mysql:
condition: service_healthy
Dockerfile
FROM node:16
WORKDIR /api
COPY . /api
RUN npm i
EXPOSE 3000
RUN chmod +x startup.sh
RUN npm i -g sequelize-cli
RUN npm i -g nodemon
ENTRYPOINT [ "./startup.sh" ]
startup.sh
#!/bin/bash
npm run migrate-db
npm run start
I found a really clean solution wanted to share it. First of all I used docker-compose, so if you are using only docker, it might not help.
First thig first, I created a docker file which looks like that.I am using typescript, so if you are using js, you don't need to download typescript and build it!
FROM node:current-alpine
WORKDIR /app
COPY . ./
COPY .env.development ./.env
RUN npm install
RUN npm install -g typescript
RUN npm install -g sequelize-cli
RUN npm install -g nodemon
RUN npm run build
RUN rm -f .npmrc
RUN cp -R res/ dist/
RUN chmod 755 docker/entrypoint.sh
EXPOSE 8000
EXPOSE 3000
EXPOSE 9229
CMD ["sh", "-c","--","echo 'started';while true; do sleep 1000; done"]
Till here it is standart. In order to do things in right order, I need a docker compose and entrypoint file. Entrypoint file is a file, that runs when your containers start. Here is docker-compose file.
version: '3'
services:
app:
build:
context: ..
dockerfile: docker/Dockerfile.development
entrypoint: docker/development-entrypoint.sh
ports:
- 3000:3000
env_file:
- ../.env.development
depends_on:
- postgres
postgres:
image: postgres:alpine
environment:
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=test
volumes:
- ./docker_postgres_init.sql:/docker-entrypoint-initdb.d/docker_postgres_init.sql
As you can see, I am using postgresql for db. My docker file, docker-compose and also entrypoint files are in a folder called docker, thats why the paths starts wtih docker, change it according to your file structure. Last and the best part is the entrypoint file. It is really simple.
#!/bin/sh
echo "Starting get ready!!!"
sequelize db:migrate
nodemon ./dist/index.js
Ofcourse change the path of the index.js file according to your settings.
Hope it helps!
I am trying to setup some containers for my NestJS + TypeORM + MySQL environment by using Docker Compose in a Windows 10 host, but I am getting an ECONNREFUSED error:
connect ECONNREFUSED 127.0.0.1:3306 +2ms
backend_1 | Error: connect ECONNREFUSED 127.0.0.1:3306
backend_1 | at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1145:16)
backend_1 | --------------------
backend_1 | at Protocol._enqueue (/usr/src/app/node_modules/mysql/lib/protocol/Protocol.js:144:48)
backend_1 | at Protocol.handshake (/usr/src/app/node_modules/mysql/lib/protocol/Protocol.js:51:23)
backend_1 | at PoolConnection.connect (/usr/src/app/node_modules/mysql/lib/Connection.js:116:18)
backend_1 | at Pool.getConnection (/usr/src/app/node_modules/mysql/lib/Pool.js:48:16)
backend_1 | at /usr/src/app/node_modules/typeorm/driver/mysql/MysqlDriver.js:793:18
backend_1 | at new Promise (<anonymous>)
backend_1 | at MysqlDriver.createPool (/usr/src/app/node_modules/typeorm/driver/mysql/MysqlDriver.js:790:16)
backend_1 | at MysqlDriver.<anonymous> (/usr/src/app/node_modules/typeorm/driver/mysql/MysqlDriver.js:278:51)
backend_1 | at step (/usr/src/app/node_modules/typeorm/node_modules/tslib/tslib.js:141:27)
backend_1 | at Object.next (/usr/src/app/node_modules/typeorm/node_modules/tslib/tslib.js:122:57)
I have created the following Dockerfile to configure the NestJS API container:
FROM node:12-alpine
WORKDIR /usr/src/app
COPY package.json .
RUN npm install
EXPOSE 3000
#CMD ["npm", "start"]
CMD /wait-for-it.sh db:3306 -- npm start
COPY . .
And then I reference this from Docker Compose with the following docker-compose.yml:
version: "3.8"
networks:
app-tier:
driver: bridge
services:
db:
image: mysql
command: --default-authentication-plugin=mysql_native_password
restart: always
expose:
- "3306"
ports:
- "3306:3306"
networks:
- app-tier
environment:
MYSQL_DATABASE: school
MYSQL_ALLOW_EMPTY_PASSWORD: ok
MYSQL_ROOT_PASSWORD: root
MYSQL_USER: dbuser
MYSQL_PASSWORD: dbuser
MYSQL_ROOT_HOST: '%'
backend:
depends_on:
- db
build: .
ports:
- "3000:3000"
networks:
- app-tier
Finally, I set the TypeORM configuration to match with the Docker Compose file:
export const DB_CONFIG: TypeOrmModuleOptions = {
type: 'mysql',
host: 'db',
port: 3306,
username: 'dbuser',
password: 'dbuser',
database: 'school',
entities: [], // We specify the entities in the App Module.
synchronize: true,
};
I am kind of new to Docker Compose, but I have tried many things like changing the output port to 3307, setting an explicit network... and the port 3306 is free in my host OS when I run it. Any help?
Edit 1
I have included MYSQL_ROOT_HOST and wait-for-it.sh as suggested, but still no results.
I think your db is taking more time to start and your app is starting before the db, try something like this for your docker-compose.yml file:
version: "3.8"
networks:
app-tier:
driver: bridge
services:
db:
image: mysql
command: --default-authentication-plugin=mysql_native_password
restart: always
expose:
- "3306"
ports:
- "3306:3306"
networks:
- app-tier
environment:
MYSQL_DATABASE: school
MYSQL_ALLOW_EMPTY_PASSWORD: ok
MYSQL_ROOT_PASSWORD: root
MYSQL_USER: dbuser
MYSQL_PASSWORD: dbuser
MYSQL_ROOT_HOST: '%'
backend:
depends_on:
- db
build: .
command: bash -c 'while !</dev/tcp/db/3306; do sleep 1; done; npm start'
ports:
- "3000:3000"
networks:
- app-tier
I try to dockerlized my own node application, but can't connect the mysql container. my codes:
docker-compose.yml
version: '3.2'
services:
node:
build: ./
ports:
- "8787:8787"
depends_on:
- db
networks:
- docker_xxx
environment:
- PORT=8787
- DATABASE_HOST=db
- DATABASE_PASSWORD=xxx
- EGG_SERVER_ENV=local
- NODE_ENV=development
# command: ["./wait-for-it.sh", "db:3306", "--", "npm run docker"]
db:
build: ./db
networks:
- docker_xxx
environment:
- MYSQL_ROOT_PASSWORD=xxx
- MYSQL_DATABASE=database
- MYSQL_USER=user
- MYSQL_PASSWORD=passwd
networks:
docker_xxx:
driver: bridge
./Dockerfile (for nodejs)
FROM node:8.9.4-alpine
RUN mkdir -p /usr/src/app
WORKDIR /usr/src/app
COPY package.json /usr/src/app/
RUN npm install --production
COPY . /usr/src/app
# COPY wait-for-it.sh /usr/src/app
EXPOSE 8787
CMD npm run docker
db/Dockerfile (for mysql)
FROM mysql:5.6
ADD honggang.sql /docker-entrypoint-initdb.d
config/config.default.js
config.mysql = {
// mysql settings
client: {
// host
host: process.env.DATABASE_HOST || '127.0.0.1',
// port
port: '3306',
// username
user: 'root',
// password
password: 'xxx',
database: 'xxx',
charset: 'utf8',
dialectOptions: {
collate: 'utf8_general_ci',
},
},
app: true,
agent: false,
};
I run docker-compose up -d , there's only db running.
I run docker logs hash to find errors, it shows the following info:
2018-04-30 14:43:51,334 ERROR 54 nodejs.ECONNREFUSEDError: connect ECONNREFUSED 172.19.0.2:3306
at Object._errnoException (util.js:1022:11)
at _exceptionWithHostPort (util.js:1044:20)
at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1182:14)
--------------------
at Protocol._enqueue (/usr/src/app/node_modules/mysql/lib/protocol/Protocol.js:145:48)
at Protocol.handshake (/usr/src/app/node_modules/mysql/lib/protocol/Protocol.js:52:23)
at PoolConnection.connect (/usr/src/app/node_modules/mysql/lib/Connection.js:130:18)
at Pool.getConnection (/usr/src/app/node_modules/mysql/lib/Pool.js:48:16)
at /usr/src/app/node_modules/ali-rds/node_modules/pify/index.js:29:7
at new Promise (<anonymous>)
at Pool.<anonymous> (/usr/src/app/node_modules/ali-rds/node_modules/pify/index.js:12:10)
at Pool.ret [as getConnection] (/usr/src/app/node_modules/ali-rds/node_modules/pify/index.js:56:34)
at Pool.query (/usr/src/app/node_modules/mysql/lib/Pool.js:202:8)
at /usr/src/app/node_modules/ali-rds/node_modules/pify/index.js:29:7
sql: select now() as currentTime;
code: 'ECONNREFUSED'
errno: 'ECONNREFUSED'
syscall: 'connect'
address: '172.19.0.2'
port: 3306
fatal: true
name: 'ECONNREFUSEDError'
pid: 54
hostname: d9cd95667a5d
I added CMD ping db, it responded.
I tried to use wait-for-it.sh (the code just commented out), but got the error info:
env: can't execute 'bash': No such file or directory
I solved https://github.com/ycjcl868/eggjs-mysql-docker
there are the points:
1. apk add --no-cache bash to run the wait-for-it.sh waiting the mysql server is ok
2. hostname is 0.0.0.0 not localhost/127.0.0.1