Node MySQL Connection Pool - wait for database to start - mysql

How do I check if a MySQL database is ready for some queries from a Node MySQL Connection Pool?
I have a Docker environment consisting of thee containers:
container 1: web server
container 2: api
container 3: database
The database container runs a MySQL database. The api container connects to that database. All three containers are started at the same time. The web server container is up after 0,5s. The api container is up after 2s. The database server is up after 20s.
Currently, the api tries to access the tables of the database before the database is up and running. This leads to errors like connection refused. The following code segment always ends up with the message "Error querying database!" when the MySQL database is not yet up:
const sql: string = 'SELECT * FROM sometable;';
MySQL.createPool({
connectionLimit: 10,
acquireTimeout: 30000,
waitForConnections: true,
database: 'mydatabase',
host: 'localhost',
multipleStatements: true,
password: 'mypassword',
user: 'root',
}).query(sql, (err, result) => {
if (result) {
console.log('Successfully queried database.');
} else {
console.log('Error querying database!');
}
});
Versions in use:
OS: Ubuntu 19.10
Node: v13.6.0
MySQL (Node API): "#types/mysql": "2.15.8" and "mysql": "2.17.1"
MySQL (Docker Database): mysql:5.7.28
TypeScript: 3.7.4
I would like to check (and wait) the database readiness out of the api, possibly using the Connection Pool I use for queries. Is that possible?

Retry to connect with setTimeout():
(answer in Javascript rather than typescript)
'use strict';
const dbpool = require('mysql').createPool({
connectionLimit: 10,
acquireTimeout: 30000,
waitForConnections: true,
database: 'mydatabase',
host: 'localhost',
multipleStatements: true,
password: 'mypassword',
user: 'root',
});
const sql = 'SELECT * FROM sometable;';
const attemptConnection = () =>
dbpool.getConnection((err, connection) => {
if (err) {
console.log('error connecting. retrying in 1 sec');
setTimeout(attemptConnection, 1000);
} else {
connection.query(sql, (errQuery, results) => {
connection.release();
if (errQuery) {
console.log('Error querying database!');
} else {
console.log('Successfully queried database.');
}
});
}
});
attemptConnection();
Here is my test run:
$ sudo service mysql stop; node test.js & sudo service mysql start
[1] 24737
error connecting. retrying in 1 sec
error connecting. retrying in 1 sec
$ Successfully queried database.
FYI, The program never ends because it needs dbpool.end();

Your API should try to connect to the database with a timeout and a certain threshold of connection attempts. However, there are readily available solutions for this scenario.
Try using wait-for-mysql module.
waitForMy = require 'wait-for-mysql'
config =
username: user
password: pass
quiet: true
query: 'SELECT 1'
waitForMy.wait(config)

Here you have a variation but with no need to mysql pooling. I'm using this on my server and it does work:
const mysql = require('mysql')
var db // global, to use later to db.query
var dbInfo = {
host : 'example.org',
database : 'some_database',
user : 'bob',
password : 'secret'
}
function connectToDb(callback) {
const attemptConnection = () => {
console.log('Attempting to connect to db')
dbInfo.connectTimeout = 2000 // same as setTimeout to avoid server overload
db = mysql.createConnection(dbInfo)
db.connect(function (err) {
if (err) {
console.log('Error connecting to database, try again in 1 sec...')
db.destroy() // end immediately failed connection before creating new one
setTimeout(attemptConnection, 2000)
} else {
callback()
}
})
}
attemptConnection()
}
// now you simply call it with normal callback
connectToDb( () => {
console.log('Connection successfully')
// do some queries
db.query(..)
})

Related

errno: 1203, code: 'ER_TOO_MANY_USER_CONNECTIONS' in nodejs mysql code?

Hello Everyone,
I'm a beginner in Node.js Mysql. I have connected to Node.js with mysql. While starting the Node.js server, I got the error like " code: 'ER_TOO_MANY_USER_CONNECTIONS', " further I will attach the mysql db connection code below. Any type of help would be appreciated. Thanks in advance...
var db = mysql.createPool({
host: 'xxxxxxxxxxxxxxxxx',
port: 'xxx',
user: 'xxxx',
password: 'xxx',
database: 'xxx'
});
db.getConnection((err, tempConn) => {
if (err) {
console.log(err);
}
else {
tempConn.release();
console.log('Mysql Connected');
}
});
module.exports={db};
If you're creating a pool you don't need to use getConnection. There is a shortcut that allows you to use it directly. If you do use getConnection you must follow it with a query, then you may release the connection. Your example is missing a query.
Here is a helpful template for using a pool config:
// in your application initialization file such as app.js
//
// other require items here as well like express maybe?
//
const mysql = require('mysql');
const connection = mysql.createPool({
connectionLimit: 10,
host: process.env.DB_HOST || '127.0.0.1',
user: process.env.DB_USER || 'local_user',
password: process.env.DB_PASSWORD || 'local_password',
database: process.env.DB_NAME || 'local_database',
multipleStatements: true,
charset: 'utf8mb4' // necessary if you might need support for emoji characters
});
connection.on('connection', function (connection) {
// handy for testing
console.log('Pool id %d connected', connection.threadId);
});
connection.on('enqueue', function () {
// handy for testing
console.log('Waiting for available connection slot');
});
global.db = connection;
//
// other app setup stuff here like app.set, app.engine, app.use, module.exports = app and all that good stuff
//
// later…
// everywhere else in your app, use the global db variable when running queries
// ../new_users.js or similar maybe?
const _create_user = (user_payload) => {
db.query(
'INSERT INTO users SET ?', user_payload, function(error, results, fields) {
if (error) throw error;
console.log(results);
});
}
// maybe we are in a module that has access to
// the request object so we can use something
// that has come via POST
//
// here is a manual object as a placeholder…
let new_user = {
first_name: 'John',
last_name: 'Smith',
email: 'j.smith#example.com',
password: 'keyboard_cat'
}
_create_user(new_user);

non-Google MySQL database connection with firebase cloud function

When i try to connect to my MySQL, server sends me an error Error: connect ETIMEDOUT
Here is sample code
const mysql = require('mysql');
const mysqlConfig = {
connectionLimit: 1,
host: "remote_host_ip",
user: "server_user",
password: "server_pass",
database: "server_db",
port: 3306
};
mysql.createConnection(mysqlConfig).connect(function (err) {
if (!err) {
console.log("Database is connected");
} else {
console.log("Database is not connected " + err);
}
});
I am on firebase blaze(pay as you go) plan.
I know the question title already states it, but you are not using GCP Cloud SQL, right?
In that case, there's a great possibility of your MySQL server is not reachable from Cloud Functions. Are you sure network connectivity is OK?
Besides, even in Cloud Functions, it's a good idea to use connection pools. Consider using it, it will be something like that:
var config = {
user: 'root',
password: 'akdaskdasdaE',
database: 'database1'
}
config.connectionLimit = 10
config.multipleStatements = true
// needed in GCP Cloud SQL, but it seems it's not your case
// config.socketPath = `/cloudsql/__INSTANCE_CONNECTION_NAME__`
var connectionPool = mysql.createPool(config)
connectionPool.on('connection', () => {
console.log(`[connectionPool] new connection opened`)
})
// then you use connectionPool.getConnection(..)

Server NodeJs crashes in idle time and how do I fix it?

My server api is on alwayse alwaysdata.
After x time the server crash.
events.js:183
throw er;
// Unhandled 'error' eventError: Connection lost: The server closed the connection.
at Protocol.end (/home/ec2-user/node_modules/mysql/lib/protocol/Protocol.js:112:13)
at Socket.<anonymous> (/home/ec2-user/node_modules/mysql/lib/Connection.js:97:28)
at Socket.<anonymous> (/home/ec2-user/node_modules/mysql/lib/Connection.js:502:10)
at emitNone (events.js:111:20)
at Socket.emit (events.js:208:7)
at endReadableNT (_stream_readable.js:1064:12)
at _combinedTickCallback (internal/process/next_tick.js:139:11)
at process._tickCallback (internal/process/next_tick.js:181:9)
I'm looking at whether this could not be related to a mysql error. but pre-existing posts do not help me. I think the server mysql cut the connection I do not know why.
here I establish the connection:
let con = mysql.createConnection({
host: "alwaysdata.net",
user: "user",
password: "",
database: "database"
});
try {
con.query(check, (err, customer) => {
if (err){
console.log("%s Error on check query",Date());
throw err;
}
try connection pool:
const mysql = require('mysql');
let pool = mysql.createPool(
{
connectionLimit : 100,
host : '172.17.0.1',
port : 3306,
user : 'test',
password : 'test',
database : 'test',
multipleStatements: true
}
);
...
pool.query(sql, params, function(err, rows) {
...
it works stably on my versions of mysql 5.7 and 8
I believe there are two ways you can handle this.
1) Force MySQL to keep the connection alive (not official, but I believe will do the trick).
2) Handle the mysql server disconnect from the Node's point of
view.
For both there is an excellent example here.
Server disconnects
You may lose the connection to a MySQL server due to network problems,
the server timing you out, the server being restarted, or crashing.
All of these events are considered fatal errors, and will have the
err.code = 'PROTOCOL_CONNECTION_LOST'. See the Error Handling section
for more information.
Re-connecting a connection is done by establishing a new connection.
Once terminated, an existing connection object cannot be re-connected
by design.
With Pool, disconnected connections will be removed from the pool
freeing up space for a new connection to be created on the next
getConnection call.
let connection=null;
function handleDisconnect() {
connection = mysql.createConnection({
host: "alwaysdata.net",
user: "user",
password: "",
database: "database"
}); // Recreate the connection, since
// the old one cannot be reused.
connection.connect(function (err) { // The server is either down
if (err) { // or restarting (takes a while sometimes).
console.log('error when connecting to db:', err);
setTimeout(handleDisconnect, 2000); // We introduce a delay before attempting to reconnect,
} // to avoid a hot loop, and to allow our node script to
}); // process asynchronous requests in the meantime.
// If you're also serving http, display a 503 error.
connection.on('error', function (err) {
console.log('db error', err);
if (err.code === 'PROTOCOL_CONNECTION_LOST') { // Connection to the MySQL server is usually
handleDisconnect(); // lost due to either server restart, or a
} else { // connnection idle timeout (the wait_timeout
throw err; // server variable configures this)
}
});
}
handleDisconnect();
setInterval(function () {
connection.query('SELECT 1');
}, 5000);
module.exports = connection;
you export the connection object and use this for the other connection queries.
I have called a Query every 5sec to keep the connection alive, i have tried all other approaches and this works like a charm.
Manish's answer worked for me!
I've been struggling with this for the past two days. I had a nodejs server with mysql db running on my localhost and after migrating to heroku with cleardb addon I came across several issues. I had this code in a config file:
const mysql = require('mysql');
const db = mysql.createConnection({
host: 'host',
database: 'database',
user: 'user',
password: 'password', });
module.exports = db;
I changed it to what Manish mentioned to handle the disconnect.

Error connecting to MySQL from Node-JS server

I am learning to develop server on node-js and have developed a basic get function which retrieves data from MySQL DB hosted on a ubuntu server at digitalocean.
Here's my code:
const express = require('express')
const app = express()
const mysql = require('mysql')
const Client = require('ssh2').Client;
const ssh = new Client();
const db = new Promise(function(resolve, reject){
ssh.on('ready', function() {
ssh.forwardOut(
// source address, this can usually be any valid address
'localhost',
// source port, this can be any valid port number
3333,
// destination address (localhost here refers to the SSH server)
'xxx.xxx.xxx.xxx',
// destination port
22,
function (err, stream) {
if (err) throw err; // SSH error: can also send error in promise ex.
reject(err)
// use `sql` connection as usual
connection = mysql.createConnection({
host : '127.0.0.1',
user : 'user',
password : 'pass',
database: 'mysql',
stream: stream
});
// send connection back in variable depending on success or not
connection.connect(function(err){
if (!err) {
resolve(connection)
} else {
reject(err)
}
});
});
}).connect({
host: 'xxx.xxx.xxx.xxx', //IP address where DB is hosted
port: 22, //Port refering to the IP
username: 'user', //username to loginto the host
password: 'pass' //password to log into host
});
});
//Retrieve route
app.get('/users', (req, res) => {
//console.log("Fetching user with id: " + req.params.id)
const queryString = "SELECT * FROM user"
connection.query(queryString, (err, rows, fields) => {
if(err){
console.log("Failed to query " + err)
res.sendStatus(500)
return
}
console.log("Fetch Succesful")
res.json(rows)
})
})
app.listen(3000, () => {
console.log("Server is up and listerning on port 3000")
})
When I run this code on my local machine it is able to connect to external DB and fetch the data. I have created another server at digitalocean and hosted the same code. However upon running it I get error at connection stating: UnhandledPromiseRejectionWarning: Error: connect ECONNREFUSED 127.0.0.1:3306
I tried various solutions available on the platform but could not suceed.
I have written the code accoring to the documentation but still clueless what's causing the error.
Thank You.

Failed to connect to localhost:1433 - connect ECONNREFUSED 127.0.0.1:1433

I'm trying to connect to my local MySql server from node.js/express application using mssql with following config:
// Database connection
var sql = require('mssql');
var config = {
user: 'db_user',
password: 'db_pass',
server: 'localhost',
database: 'project_db'
};
sql.connect(config, function (err) {
if (err) {
console.log(err);
}
});
I get this error:
{ [ConnectionError: Failed to connect to localhost:1433 - connect ECONNREFUSED 127.0.0.1:1433]
name: 'ConnectionError',
message: 'Failed to connect to localhost:1433 - connect ECONNREFUSED 127.0.0.1:1433',
code: 'ESOCKET' }
I know I have to enable TCP/IP connections for my local MySql server, but I can not find how to do it on OSX El Capitan. There is nothing like control panel with settings. That's all available at System Preferences > MySql:
Any ideas how to fix this, guys? Thanks for trying to help.
mssql and mysql are two very different things... You should use the node-mysql or some other mysql client library if you are going to be running mysql on your dev machine
https://github.com/felixge/node-mysql
var mysql = require('mysql');
var connection = mysql.createConnection({
host : 'localhost',
user : 'me',
password : 'secret',
database : 'my_db'
});
connection.connect();
connection.query('SELECT 1 + 1 AS solution', function(err, rows, fields) {
if (err) throw err;
console.log('The solution is: ', rows[0].solution);
});
connection.end();