where do i close the mysql connection?
I need to run queries in sequence. I am writing code that looks like this at present:
var sqlFindMobile = "select * from user_mobiles where mobile=?";
var sqlNewUser = "insert into users (password) values (?)";
//var sqlUserId = "select last_insert_id() as user_id";
var sqlNewMobile = "insert into user_mobiles (user_id, mobile) values (?,?)";
connection.connect(function(err){});
var query = connection.query(sqlFindMobile, [req.body.mobile], function(err, results) {
if(err) throw err;
console.log("mobile query");
if(results.length==0) {
var query = connection.query(sqlNewUser, [req.body.password], function(err, results) {
if(err) throw err;
console.log("added user");
var user_id = results.insertId;
var query = connection.query(sqlNewMobile, [user_id, req.body.mobile], function(err, results) {
if(err) throw err;
console.log("added mobile");
//connection.end();
});
});
}
});
//connection.end();
(I am a beginner with node, npm-express and npm-mysql. I have tried searching SO for "express mysql cannot enqueue" to find related questions and have not found them.)
I fixed this problem use this method:
connection.end() in your connection.query function
The fixed code is here
If you're using the node-mysql module by felixge then you can call connection.end() at any point after you've made all of the connection.query() calls, since it will wait for all of the queries to finish before it terminates the connection.
See the example here for more information.
If you're wanting to run lots of queries in series, you should look into the async module, it's great for dealing with a series of asynchronous functions (i.e. those that have a callback).
Maybe the problem is that the mySQL query is executed after the connection is already closed, due to the asynchronous nature of Node. Try using this code to call connection.end() right before the thread exits:
function exitHandler(options, err) {
connection.end();
if (options.cleanup)
console.log('clean');
if (err)
console.log(err.stack);
if (options.exit)
process.exit();
}
//do something when app is closing
process.on('exit', exitHandler.bind(null, {cleanup: true}));
Code adapted from #Emil Condrea, doing a cleanup action just before node.js exits
In my case connection.end was being called in a spot that was hard to notice, so an errant call to connection.end could be the problem with this error
Related
My Node app is freezing when running a function I've written. The code will stop executing at a specific line on the 11th loop, with no error message. No further requests can be made to the app, and it must be restarted. The data is changing every time it's run, and it is always the 11th iteration.
The function it is calling is relatively complex, so I can't put the code here. My question is how do you approach diagnosing a piece of code when you see no failure notices? I assume the problem is mysql-related, as the last thing logged in the simplified version of the code below is 'Org request start 11' - no error message ever follows. But I don't understand how the code can stop executing here without entering the callback.
Example code is here:
async function processDataset(datasets){
for (let i = 0; i < datasets.length; i++) {
const dataset= datasets[i];
const orgData = await getOrgData(dataset.id);
const parsedDataset = await processDataset(dataset, orgData);
}
}
function getOrgData(datasetId) {
return new Promise(function (resolve, reject) {
console.log("Org request start", datasetId);
connection.getConnection(function (err, connection) {
console.log("got connection");
if (err) {
console.log(err);
reject(err);
}
connection.query("select * from orgs;", function (error, rows, fields) {
console.log("query returned");
connection.release();
if (error) {
console.log(error);
reject(error);
}
resolve(rows);
});
});
});
}
Running Node v12 / Express, mysql 5.5
So I found the problem was that elsewhere I had a query running that was missing a line to release the mysql connection in the second function called. Adding this resolved the bug. I don't understand why this doesn't trigger an error, or how I would have gone about diagnosing this with a more systematic approach, so would like to hear any comments.
Would like to know the best practice in writing mysql queries in nodejs with connection pool. Found some related threads. But none of them answered the exact question. So starting a new one.
var mysql = require('mysql');
var pool = mysql.createPool(...);
pool.getConnection(function(err, connection) {
// insert into first table
connection.query( 'Insert into table values(....)', function(err, rows) {
//get the auto incremented value from first insert and use it in the second insert
connection.query("insert into table2 values(rows.insertID,..)",function(err,rows){
//release the connection to pool after performing all the inserts
connection.release();
});
});
}); `
Doubts :
Is this a correct method to write these queries
While using pool is there a need to use end() . What I understood is that when we use the release() function the connection will go back to the pool and we can reuse it. So will there be a need to use end() any where.
Your code looks fine (IE opening a pool, multiple queries while grabbing the insertID of the previous query). With a pool, connection.release() will 'close' the session and return to the pool. connection.end() is not needed.
Additional:
IMO, connection pools are used if you have operations that would result in blocking other users (mostly large reads, or large writes). Since you are inserting data (assuming this would be relatively fast), you could try using Transactions.
Transactions provide methods to fail safe your queries (increase integrity of your database). If you use a transaction, you can check if EVERYTHING was called without errors before committing to the DB (you have to option to rollback changes).
var mysql = require('mysql');
var connection = mysql.createConnection({...});
// Open it up
connection.connect();
connection.beginTransaction(function(err){
if(err) { throw err; }
// Use connection.escape() to sanitize user input
var insertQuery = 'Insert into table values(....)';
connection.query(insertQuery, function(err, result) {
if (err) {
return connection.rollback(function() {
throw err;
});
}
//get the auto incremented value from first insert and use it in the second insert
var rowId = result.insertId;
var insertQuery2 = "insert into table2 values(" + rowId + ",..)";
connection.query(insertQuery2, function(err, result){
if(err) {
return connection.rollback(function(){
throw err;
});
}
connection.commit(function(err){
if (err) {
return connection.rollback(function() {
throw err;
});
}
// Close it up
connection.end();
});
});
});
});
I am trying to build an api using sailsjs that calls stored procedures of a MYSQL database. I decided to decouple the query by adding it to a service so that others functions might be able to call them and use them. Below is what I came up with.
under /api/controller/MySqlController
getAllUsers: function (req, res) {
MySqlSpService.spGetAllUsers(function(err, result){
if(err) return res.serverError(err);
return res.ok(result[1]);
});
},
under /api/services/MYSQLService
var MySqlSpService= {
spGetAllUsers: function(callback) {
Database.query('call userDb.sp_get_all_users(#message, #mystatus)', function (err, results) {
callback(err, results);
}); // end query
}
module.exports = MySqlSpService;
When I hit the api the data is displayed exactly how I thought it would be. But the problem is that when I try to call the spGetAllUsers service and assign to a variable, I get a undefined value.
Like this:
var users = MySqlSpService.spGetAllUsers(function(err, result){
if(err) return res.serverError(err);
return result[1];
});
I believe the problem is with the callbacks but I am not sure how to retrieve the data from the query. I have searched for an answer but I can't seem to find the right search terms that match my problem. Any help would be greatly appreciated Thanks in advance.
Indeed, your problem is about callback and asynchronous code.
The MySqlSpService.spGetAllUsers() function does not return anything, there is no return statement in this method. But it executes a callback function where you have the opportunity to execute code that depends on the SQL query result.
You have to write your code like this because the database query is executed asynchronously.
console.log('This is executed first');
MySqlSpService.spGetAllUsers(function(err, result){
console.log('This is executed when the result of the database query is received');
if(err) return res.serverError(err);
users = result[1];
/**
* Here you can access to the "users" data
* and execute code depending on it
*/
});
console.log('This is executed in second and it is not possible to know the result of the query yet');
Tools like async can help you to organize your asynchronous code. By default, async is available globally in sails.js.
So I am making my first attempt with Node and I can't really wrap my head around how to work with the MySQL connection. The script is somewhat simplified like this
var mysql = require('mysql');
var connection = mysql.createConnection({
host : '192.168.40.1',
user : 'user',
password : 'password',
database : 'database'
});
function DoSomething(connection, item, callback) {
connection.query(
'SELECT COUNT(*) AS count FROM another_table WHERE field=?',
item.field,
function (err, results) {
if (err) throw err;
if (results.length > 0 && results[0].count >= 1) {
callback(err, connection, item, 'Found something')
}
});
}
function DoSomethingElse(connection, item, callback) {
// Similar to DoSomething()
}
function StoreResult(err, connection, item, reason) {
if (err) throw err;
connection.query(
'INSERT INTO result (item_id, reason) VALUES (?, ?)',
[item.id, reason],
function (err, results) {
if (err) throw err;
});
}
connection.query('SELECT * FROM table WHERE deleted=?', [0], function (err, results)
{
if (err) throw err;
results.forEach(function (item, index) {
DoSomething(connection, item, StoreResult);
DoSomethingElse(connection, item, StoreResult);
});
});
connection.end();
What I am having trouble with (as far as I can tell) is that since DoSomething() it seems that connection.end() is called before all of the DoSomething()'s have finished causing errors that queries can't be performed when the connection is closed.
I tried playing around with the async library, but I haven't gotten anywhere so far. Anyone with some good advice on how to do this?
The problem with your code is that you're closing the connection synchronously while an asynchronous request is still being handled. You should call connection.end() only after all query callbacks have been called.
Since you are doing multiple queries, this means using some way to wait for all their results. The simplest way is to nest every next call into the callback of the previous one, but that way leads to the pyramid of doom. There are a number of popular libraries that solve this, but my own preference is for async.
Using async I would rewrite your code as follows:
async.waterfall([function(next) {
connection.query('SELECT * FROM table WHERE deleted=?', [0], next); // note the callback
},
function(results, next) {
// asynchronously handle each results. If they should be in order, use forEachSeries
async.forEach(results, function(item, next) {
// handle in parallel
async.parallel([function(done) {
DoSomething(connection, item, function(err, connection, item, reason) {
// This is a hack, StoreResult should have a callback which is called
// after it's done, because the callback is now being called too soon
StoreResult(err, connection, item, reason);
callback(err);
});
}, function(done) {
DoSomethingElse(connection, item, function(err, connection, item, reason) {
// This is a hack, StoreResult should have a callback which is called
// after it's done, because the callback is now being called too soon
StoreResult(err, connection, item, reason);
callback(err);
}], function(err) {
// this callback is called with an error as soon as it occurs
// or after all callbacks are called without error
next(err);
});
}, function(err) {
// idem
next(err);
});
}], function(err, results) {
// idem
// Do whatever you want to do with the final error here
connection.end();
});
This also allows you to solve a possible issue with the order of your queries in the forEach: They are started in order, but are not guaranteed to finish in order due to their asynchronous nature.
Close your connection after you have done everything you want in the script.
When programming in asynchronous language, keep in mind that the real ending point of your script is the last asynchronous callback, instead of the last line like other scripts (e.g. PHP).
Note that you don't want to simply ignore the connection.end(); as the underlying MySQL driver will keep the connection alive and your script will stuck in the last line forever until you kill it.
This is the modified version of your code.
connection.query('SELECT * FROM table WHERE deleted=?', [0], function (err, results)
{
if (err) throw err;
results.forEach(function (item, index) {
DoSomething(connection, item, StoreResult);
DoSomethingElse(connection, item, StoreResult);
});
// End your connection here
connection.end();
});
All examples i've seen were doing a query first and then send to client some info.
If I do a query FIRST and then use results in functions it works:
client.query(
('SELECT * FROM '+TABLES_USERS),
function(err, results, fields) {
var Users = (results);
io.sockets.on('connection', function (socket) {
socket.on('event1', function (data) {
var requser = Users[data];
socket.emit('event2', requser);
});
});
client.end();
});
But now i need to do a query on client's request.
I tried something like this but query doesn't work:
io.sockets.on('connection', function (socket) {
socket.on('event1', function (data) {
console.log('query required'); /*works*/
client.query(
('SELECT * FROM '+TABLES_USERS+' WHERE id_user ='+data),
function(err, results, fields) {
if (err) {throw err; /*doesn't work*/
console.log('error'); /*doesn't work*/ }
console.log('query is done'); /*doesn't work too. so i think query just doesn't work cuz there are no error and no results*/
socket.emit('event2', results);
client.end();
});
});
});
There are some things you are not doing ok (in my opinion) in the example above:
1) You don't ask for login after the client is connected to Socket.IO, instead you check to see if his session contains data that can verify is he's connected.
You should read the article about Express, Socket.IO and sessions., since it explains everything in detail (if you are using Express)
2) I think MongoDB, CouchDB and possibly other databases are better suited for realtime applications than MySQL.
I've solved the problem by using node-mysql-libmysqlclient instead of node-mysql. But if someone knows a way to do a query AFTER client's request in node-mysql, please tell me))