First of all, I'm a beginner on NodeJS. Well, I'm using a shared hosting to my project and when the database reaches 1 minute of inactivity, NodeJS crashes and disconnects me from MySQL. Since I'm using a shared hosting, I can't edit the idle time on the MySQL config and I'll need to handle it in code.
I'm using module.exports to handle my connection, as shown below. So how can I make an auto-reconnection script to take care of my issue? Thank you.
var mysql = require('mysql');
module.exports =
{
handle: null,
connect: function(call){
this.handle = mysql.createConnection({
host : 'localhost',
user : 'root',
password : '',
database : 'test',
timezone: 'utc',
charset : 'utf8'
});
this.handle.connect(function (err) {
if(err) {
console.log("[MySQL] Connection error: " + err.code);
} else {
console.log("[MySQL] Successfully connected");
}
});
}
};
The node mysql module that you are using also has a connection pooling mechanism.
Check out the docs at https://github.com/mysqljs/mysql#pooling-connections
Connection pools will make you task easier. You can then store the connection pool object and use its getConnection method to obtain a connection. Make sure that you release the connection when you are done with it.
If for some reason you cant use connection pooling then you will have to listen for error event on the connection and handle it accordingly. But I strongly recommend that you use connection pool.
Related
function Connect() {
var connection = mysql.createConnection({
host : 'hidden',
user : 'hidden',
password : 'hidden',
database : 'hidden'
});
connection.connect(function(err) {
if (err) {
console.log('Error connecting to Database'.red);
setInterval(Connect, 5000);
connection.end();
}else{
console.log('Connected to Database'.green);
}
});
}
Connect();
I am attempting to make it so my code won't crash if the database loses connection.
It will be connected to a database which is prone to many restarts and updates, meaning that the program will crash upon losing connection.
I tried adding setInterval(Connect, 5000); so it would attempt to reconnect if it loses connection but to no luck.
Can someone help me out where I am going wrong?
Assuming you're using the mysql npm package you can use connection pooling provided by the library:
https://github.com/mysqljs/mysql#pooling-connections
When a previous connection is retrieved from the pool, a ping packet is sent to the server to check if the connection is still good.
It's rather easy to use, instead of using createConnection you can use createPool
var pool = mysql.createPool({
connectionLimit : 2,
host : 'example.org',
user : 'bob',
password : 'secret',
database : 'my_db'
});
And then use the pool to query the database.
It has many additional benefits as well e.g. load balancing connections.
I'm building an Node application which will query simple and more complex (multiple joins) queries. I'm looking for suggestions on how I should manage the mySQL connections.
I have the following elements:
server.js : Express
router1.js (fictive name) : Express Router middleware
router2.js (fictive name) : Express Router middleware
//this is router1
router.get('/', function (req, res){
connection.connect(function(Err){...});
connection.query('SELECT* FROM table WHERE id = "blah"', function(err,results,fields){
console.log(results);
});
...
connection.end();
})
Should I connect to mysql everytime '/router1/' is requested, like in this example, or it's better to leave one connection open one at start up? As: connection.connect(); outside of: router.get('/',function(req,res){
...
}); ?
I am using mysql2 for this, it is basicly mysql but with promises. If you use mysql you can also do this.
Create a seperate file called connection.js or something.
const mysql = require('mysql2');
const connection = mysql.createPool({
host: "localhost",
user: "",
password: "",
database: ""
// here you can set connection limits and so on
});
module.exports = connection;
Then it is probaly better you create some models and call these from within your controllers, within your router.get('/', (req, res) => {here});
A model would look like this:
const connection = require('../util/connection');
async function getAll() {
const sql = "SELECT * FROM tableName";
const [rows] = await connection.promise().query(sql);
return rows;
}
exports.getAll = getAll;
You can do this with or without promises, it doesn't matter.
Your connection to the pool is automatically released when the query is finished.
Then you should call getAll from your router or app.
I hope this helped, sorry if not.
Connection pooling is how it should be done. Opening a new connection for every request slows down the application and it can sooner or later become a bottleneck, as node does not automatically closes the connections unlike PHP. Thus connection pool ensures that a fixed number of connections are always available and it handles the closing of unnecessary connections as and when required.
This is how I start my express app using Sequelize. For Mongoose, it is more or less simlar except the library API.
const sequelize = new Sequelize('database', 'username', 'password', {
host: 'localhost',
dialect: 'mysql',
pool: {
max: 5,
min: 0,
acquire: 30000,
idle: 10000
}
});
sequelize.authenticate()
.then(
// On successfull connection, open a port
// and listen to requests. This is where the application
// starts listening to requests.
() => {
const server = http.createServer(app);
server.listen(port);
},
)
.catch(err => {
console.error('Unable to connect to the database:', err);
console.error('Cancelling app server launch');
});
The app is started only after a database connection has been established. This ensures that the server won't be active without any database connection. Connection pool will keep the connections open by default, and use a connection out of the pool for all queries.
If you use createPool mysql will manage opening and closing connections and you will have better performance. It doesn't matter if you use mysql or mysql2 or sequlize. use a separate file for createPool and export it. You can use it everywhere. Don't use classes and just do it functionally for better performance in nodejs.
> npm install mysql
mysql is a great module which makes working with MySQL very easy and it provides all the capabilities you might need.
Once you have mysql installed, all you have to do to connect to your database is
var mysql = require('mysql')
var conn = mysql.createConnection({
host: 'localhost',
user: 'username',
password: 'password',
database: 'database'
})
conn.connect(function(err) {
if (err) throw err
console.log('connected')
})
Now you are ready to begin writing and reading from your database.
I already have WAMP server installed on my machine. Can I be able to access MySQL databases created on WAMP's MySQL using node-mysql module?
Actually, I tried this code, its running without errors but unable to fetch the database(or tables):
var http = require('http'),
mysql = require("mysql");
var connection = mysql.createConnection({
host: "localhost",
user: "root",
password: "",
database: "database_name"
});
http.createServer(function (request, response) {
request.on('end', function () {
connection.query('SELECT * FROM table_name', function (error, rows, fields) {
console.log('The first field is: ', rows[0].field);
});
});
}).listen(8001);
console.log("running on localhost:8001");
Try adding request.resume(); before your 'end' event handler.
In node v0.10+, streams start out in a "paused" state that allow you to .read() specific sized chunks or you can use them like the old streams by attaching a 'data' event handler which causes the stream to be continuously read from.
Calling request.resume(); will also switch to the old stream mode, effectively discarding the request data (because there are no 'data' event handlers) so that your 'end' event handler will be called.
when I use node mysql, an error is appear between 12:00 to 2:00 that the TCP connection is shutdown by the server. This is the full message:
Error: Connection lost: The server closed the connection.
at Protocol.end (/opt/node-v0.10.20-linux-x64/IM/node_modules/mysql/lib/protocol/Protocol.js:73:13)
at Socket.onend (stream.js:79:10)
at Socket.EventEmitter.emit (events.js:117:20)
at _stream_readable.js:920:16
at process._tickCallback (node.js:415:13)
There is the solution. However, after I try by this way, the problem also appear. now I do not know how to do. Does anyone meet this problem?
Here is the way I wrote follow the solution:
var handleKFDisconnect = function() {
kfdb.on('error', function(err) {
if (!err.fatal) {
return;
}
if (err.code !== 'PROTOCOL_CONNECTION_LOST') {
console.log("PROTOCOL_CONNECTION_LOST");
throw err;
}
log.error("The database is error:" + err.stack);
kfdb = mysql.createConnection(kf_config);
console.log("kfid");
console.log(kfdb);
handleKFDisconnect();
});
};
handleKFDisconnect();
Try to use this code to handle server disconnect:
var db_config = {
host: 'localhost',
user: 'root',
password: '',
database: 'example'
};
var connection;
function handleDisconnect() {
connection = mysql.createConnection(db_config); // Recreate the connection, since
// the old one cannot be reused.
connection.connect(function(err) { // The server is either down
if(err) { // or restarting (takes a while sometimes).
console.log('error when connecting to db:', err);
setTimeout(handleDisconnect, 2000); // We introduce a delay before attempting to reconnect,
} // to avoid a hot loop, and to allow our node script to
}); // process asynchronous requests in the meantime.
// If you're also serving http, display a 503 error.
connection.on('error', function(err) {
console.log('db error', err);
if(err.code === 'PROTOCOL_CONNECTION_LOST') { // Connection to the MySQL server is usually
handleDisconnect(); // lost due to either server restart, or a
} else { // connnection idle timeout (the wait_timeout
throw err; // server variable configures this)
}
});
}
handleDisconnect();
In your code i am missing the parts after connection = mysql.createConnection(db_config);
I do not recall my original use case for this mechanism. Nowadays, I cannot think of any valid use case.
Your client should be able to detect when the connection is lost and allow you to re-create the connection. If it important that part of program logic is executed using the same connection, then use transactions.
tl;dr; Do not use this method.
A pragmatic solution is to force MySQL to keep the connection alive:
setInterval(function () {
db.query('SELECT 1');
}, 5000);
I prefer this solution to connection pool and handling disconnect because it does not require to structure your code in a way thats aware of connection presence. Making a query every 5 seconds ensures that the connection will remain alive and PROTOCOL_CONNECTION_LOST does not occur.
Furthermore, this method ensures that you are keeping the same connection alive, as opposed to re-connecting. This is important. Consider what would happen if your script relied on LAST_INSERT_ID() and mysql connection have been reset without you being aware about it?
However, this only ensures that connection time out (wait_timeout and interactive_timeout) does not occur. It will fail, as expected, in all others scenarios. Therefore, make sure to handle other errors.
better solution is to use the pool - it will handle this for you.
const pool = mysql.createPool({
host: 'localhost',
user: '--',
database: '---',
password: '----'
});
// ... later
pool.query('select 1 + 1', (err, rows) => { /* */ });
https://github.com/sidorares/node-mysql2/issues/836
To simulate a dropped connection try
connection.destroy();
More information here: https://github.com/felixge/node-mysql/blob/master/Readme.md#terminating-connections
Creating and destroying the connections in each query maybe complicated, i had some headaches with a server migration when i decided to install MariaDB instead MySQL. For some reason in the file etc/my.cnf the parameter wait_timeout had a default value of 10 sec (it causes that the persistence can't be implemented). Then, the solution was set it in 28800, that's 8 hours. Well, i hope help somebody with this "güevonada"... excuse me for my bad english.
ANy idea how can I connect from NodeJs to the Mysql ClearDB on Heroku?
I was able to connect to the ClearDB Mysql runing on Heroku from Navicat, I even created a table called t_users. But I'm having problems connecting from NodeJs from Heroku.
This is my code, it is very simple: everything works find until it tries to connect to MySQL
web.js:
var express = require("express");
var app = express();
app.use(express.logger());
var mysql = require('mysql');
var connection = mysql.createConnection({
host : 'us-cdbr-east-04.cleardb.com',
user : '',
password : '',
database : ''
});
connection.connect();
app.get('/', function(request, response) {
response.send('Hello World!!!! HOLA MUNDOOOOO!!!');
connection.query('SELECT 1 + 1 AS solution', function(err, rows, fields) {
if (err) throw err;
response.send('The solution is: ', rows[0].solution);
});
});
var port = process.env.PORT || 5000;
app.listen(port, function() {
console.log("Listening on " + port);
});
This is what I got when I run on the command line: Heroku config
C:\wamp\www\Nodejs_MySql_Heroku>heroku config
=== frozen-wave-8356 Config Vars
CLEARDB_DATABASE_URL: mysql://b32fa719432e40:87de815a#us-cdbr-east-04.cleardb.com/heroku_28437b49d76cc53?reconnect=true
PATH: bin:node_modules/.bin:/usr/local/bin:/usr/bin:/bin
This is the LOG:
http://d.pr/i/cExL
ANy idea how can I connect from NodeJs to the Mysql ClearDB on Heroku?
Thanks
Try this. Hope this will help you
mysql://b32fa719432e40:87de815a#us-cdbr-east-04.cleardb.com/heroku_28437b49d76cc53?reconnect=true
var connection = mysql.createConnection({
host : 'us-cdbr-east-04.cleardb.com',
user : 'b32fa719432e40',
password : '87de815a',
database : 'heroku_28437b49d76cc53'
});
Use this details and connect it in mySQL work bench, and import your localhost db
The base code was ok, I missed some NodeJS code.
I did a video explaining how to connect to MySqlusing NodeJS on a Heroku server, take a look:
http://www.youtube.com/watch?v=2OGHdii_42s
This is the code in case you want to see:
https://github.com/mescalito/MySql-NodeJS-Heroku
createConnections accepts config as well as connectionString.
export function createConnection(connectionUri: string | ConnectionConfig): Connection;
So below solution would work.
var connection = mysql.createConnection('mysql://b32fa719432e40:87de815a#us-cdbr-east-04.cleardb.com/heroku_28437b49d76cc53?reconnect=true);
or you can set the connection url in environment variable DATABASE_URL
var connection = mysql.createConnection(process.env.DATABASE_URL);
One should need to use pool connection for database connection to handle better mysql conncurrent request as follows
var pool = mysql.createPool({
connectionLimit : 100,
host : 'us-cdbr-iron-east-05.cleardb.net',
user : 'b5837b0f1d3d06',
password : '9d9ae3d5',
database : 'heroku_db89e2842543609',
debug : 'false'
});
It just because pool maintain the connection.release() on its own , so you do not have to bother where you should need to release the connection.
On the other hand you should need to provide user, password and database name as i mentioned in my code.
When u get provisioned from heroku then you get a cleardb database name.
on clicking clear db database button you will find username and password.
and to get database url you have to run the following commands in terminal as follows;
heroku login
heroku app -all (it shows the app list name )
heroku config --app appname (it will provide the database url).
Rest you would need to worry, just add all dependency into your package.json before pushing to heroku master.
After then you will have no problem on deploying nodejs application to heroku server.
Looking at the log timestamps, it seems like your connections are timing out. Either create a wrapper for 'getConnection' that checks the connection health and re-establishes if necessary, or try to use the mysql Connection Pool feature, which can do that for you.