I am using node.js mysql driver. But when the server under high load, i guess mysql is doing queue for queries. How can I prevent this? I want to do query instantly.
How can I resolve this? My queries are laggy.
My code:
var mysql = require('mysql');
var connection = mysql.createConnection({
host : 'ip',
user : 'db',
password : 'pass'
});
connection.connect(function(err) {
if (err) {
console.error('error connecting: ' + err.stack);
return;
}
console.log('connected as id ' + connection.threadId);
});
connection.query('USE db');
connection.query({ sql:"select token,COUNT(token) as sayim from tokens where username=? or anon=?", timeout: 10000},[data,data],function(err,info) {
if (info[0].sayim != 0) {
callback(info[0].token);
}else{
callback("0");
}
});
Ps: The server is not returning any error for this. But when I do a query, server is responding after approximately 10 seconds. If server is not under the high load it is responding instantly.
You could use PoolConnection instead.
var connection = mysql.createPool({
host : 'ip',
user : 'db',
password : 'pass',
database : 'db'
});
Another thing that comes to mind: In your example, you use asynchronous functions one after the other which should cause you some troubles as well.
Related
I have a question, Before I change my code, I got error [error: connection lost: the server closed the connection.], possibly because its idle for sometime.
This is my old code.
const dbConn = mysql.createConnection({
host : 'localhost',
user : 'root',
password : '',
database : 'test'
});
dbConn.connect(function(err) {
if(err) throw err;
console.log("Database Connected!");
})
module.exports = dbConn;
After searching for a while, most of it recommend using createPool intead of createConnection, so I change my code to this
const dbConn = mysql.createPool({
host : 'localhost',
user : 'root',
password : '',
database : 'test'
});
module.exports = dbConn;
Then my question is, do I have to release the connection everytime we complete a query? This is how I do query.
dbConn.query("SELECT * FROM spesialisasi s ORDER BY s.nama_spesialisasi ASC ",
function(err, res) {
if(err) {
console.log("error: ", err);
result(err, null);
} else {
result(null, res);
}
}
)
From my knowledge, pool.query() will automatically release the connection when the query completes. Here's the section on connection pooling from the MySQL NPM docs
This happens pretty frequently (once a week for about 30-40 minutes), where all of a sudden my database mentions max connections when I try to connect via heidisql, and any apis calls respond with the following error:
Cannot read property 'release' of undefined
I am calling .release() after every query in mysql. Is there something I am missing, am I suppose to call .end as well? I am using nodejs with mysql.
Here is the way I wrap every query and the pool code:
var mysql = require('mysql');
var mysql_pool = mysql.createPool({
connectionLimit: config.mysql.limit,
host: config.mysql.host,
user: config.mysql.user,
password: config.mysql.pw,
database: config.mysql.db //,
// debug: true
});
var qSelect = "SELECT id FROM Users";
var qValues = [];
var qCall = mysql.format(qSelect, qValues);
mysql_pool.getConnection(function(err_pool, connection) {
if (err_pool) {
connection.release();
console.log(' Error getting mysql_pool connection: ' + err_pool);
throw err_pool;
}
connection.query(qCall, function(err, userFound, fields) {
connection.release();
if (err) {
console.log("get user : " + err);
} else {
//some code here
}
});
Can someone please advise, appreciate it.
You should remove first connection.release() used in if loop
if (err_pool) {
console.log(' Error getting mysql_pool connection: ' + err_pool);
throw err_pool;
}
How do I check if a MySQL database is ready for some queries from a Node MySQL Connection Pool?
I have a Docker environment consisting of thee containers:
container 1: web server
container 2: api
container 3: database
The database container runs a MySQL database. The api container connects to that database. All three containers are started at the same time. The web server container is up after 0,5s. The api container is up after 2s. The database server is up after 20s.
Currently, the api tries to access the tables of the database before the database is up and running. This leads to errors like connection refused. The following code segment always ends up with the message "Error querying database!" when the MySQL database is not yet up:
const sql: string = 'SELECT * FROM sometable;';
MySQL.createPool({
connectionLimit: 10,
acquireTimeout: 30000,
waitForConnections: true,
database: 'mydatabase',
host: 'localhost',
multipleStatements: true,
password: 'mypassword',
user: 'root',
}).query(sql, (err, result) => {
if (result) {
console.log('Successfully queried database.');
} else {
console.log('Error querying database!');
}
});
Versions in use:
OS: Ubuntu 19.10
Node: v13.6.0
MySQL (Node API): "#types/mysql": "2.15.8" and "mysql": "2.17.1"
MySQL (Docker Database): mysql:5.7.28
TypeScript: 3.7.4
I would like to check (and wait) the database readiness out of the api, possibly using the Connection Pool I use for queries. Is that possible?
Retry to connect with setTimeout():
(answer in Javascript rather than typescript)
'use strict';
const dbpool = require('mysql').createPool({
connectionLimit: 10,
acquireTimeout: 30000,
waitForConnections: true,
database: 'mydatabase',
host: 'localhost',
multipleStatements: true,
password: 'mypassword',
user: 'root',
});
const sql = 'SELECT * FROM sometable;';
const attemptConnection = () =>
dbpool.getConnection((err, connection) => {
if (err) {
console.log('error connecting. retrying in 1 sec');
setTimeout(attemptConnection, 1000);
} else {
connection.query(sql, (errQuery, results) => {
connection.release();
if (errQuery) {
console.log('Error querying database!');
} else {
console.log('Successfully queried database.');
}
});
}
});
attemptConnection();
Here is my test run:
$ sudo service mysql stop; node test.js & sudo service mysql start
[1] 24737
error connecting. retrying in 1 sec
error connecting. retrying in 1 sec
$ Successfully queried database.
FYI, The program never ends because it needs dbpool.end();
Your API should try to connect to the database with a timeout and a certain threshold of connection attempts. However, there are readily available solutions for this scenario.
Try using wait-for-mysql module.
waitForMy = require 'wait-for-mysql'
config =
username: user
password: pass
quiet: true
query: 'SELECT 1'
waitForMy.wait(config)
Here you have a variation but with no need to mysql pooling. I'm using this on my server and it does work:
const mysql = require('mysql')
var db // global, to use later to db.query
var dbInfo = {
host : 'example.org',
database : 'some_database',
user : 'bob',
password : 'secret'
}
function connectToDb(callback) {
const attemptConnection = () => {
console.log('Attempting to connect to db')
dbInfo.connectTimeout = 2000 // same as setTimeout to avoid server overload
db = mysql.createConnection(dbInfo)
db.connect(function (err) {
if (err) {
console.log('Error connecting to database, try again in 1 sec...')
db.destroy() // end immediately failed connection before creating new one
setTimeout(attemptConnection, 2000)
} else {
callback()
}
})
}
attemptConnection()
}
// now you simply call it with normal callback
connectToDb( () => {
console.log('Connection successfully')
// do some queries
db.query(..)
})
Having an odd issue with node mysql. I am running a mysql container (docker) on my local machine with a standard setup. I am able to get past mysql.createConnection, but my code always fails at connection.connect. MySQL Error logs on the container show no attempted/failed connections, and I can connect via Workbench. Has anyone had this experience trying to connect to MySQL via node.
router.get('/page', function(req,res){
const msg = 'My Query from config, tested and confirmed on Workbench';
var connection = mysql.createConnection({
host : '127.0.0.1',
user : 'user',
password : 'pw',
database : 'db',
port : 3306,
debug : true
});
console.log('Connect to server', connection.state); //Doesn't get past this guy
connection.connect(function(err) {
if (err) {
console.log('error connecting: ' + err.stack);
return err;
}
else {
console.log('connected as id ' + connection.threadId);
}
});
middleware.logger.debug('Make Query');
connection.query(msg, function(err, rows, fields) {
if (!err) {
console.log('The solution is: ', rows);
res.json('We got some rows!');
}
else {
console.log('Error while performing Query.', err);
res.json(err);
}
});
console.log('End Connection');
connection.end();
});
Also, should note: I get the same error whether my docker container is running or not.
EDIT: Okay, want to amend my problem, this is actually an issue with express router. Expanded code above.
I am using node-mysql for the first time, and I have a program with no errors, no warnings, but is not working properly... Here is the code for the program, very simple:
var mysql = require('mysql');
var connection = mysql.createConnection({
host : 'localhost',
user : 'root',
password : '',
database: 'nodetest',
port: 8080
});
connection.connect();
var usr = "BLASHDASD"
var userId = usr;
var sql = 'UPDATE userlist SET user1= ' + connection.escape(userId) + ' WHERE id=1 ';
console.log(sql);
connection.query('sql', function(err, rows, fields) {
console.log(err);
console.log("BLAHSDBKASD");
});
connection.end();
And here is the console output:
C:\wamp\www\nodePHP-master\js>node nodeTest.js
UPDATE userlist SET user1= 'BLASHDASD' WHERE id=1
But nothing is happening in my MySQL table... I even copied and pasted the UPDATE line above and just ran it as SQL code and it worked great... Need some ideas of what is going on. Thanks a bunch
EDIT:
Answered my own question... was listening on wrong port, so connection was failing. Here is updated code for those interested/search in the future:
//TEST
var mysql = require('mysql');
var connection = mysql.createConnection({
host : 'localhost',
user : 'root',
password : '',
database: 'nodetest',
port: 3306,
});
connection.connect(function(err){
if(err) {
console.log(err)
} else {
console.log("connected");
}
});
var usr = "BLASHDASD"
var userId = usr;
var sql = 'UPDATE userlist SET user1= ' + connection.escape(userId) + ' WHERE id=1 ';
console.log(sql);
connection.query(sql, function(err, rows, fields) {
console.log(err);
});
connection.end();
You are having problems with node's asynchronous nature, a very common issue when coming to Node. You also had a small but significant error in your code (you have 'sql' as a quoted string), but here is something structurally similar that should point you in the right direction.
var mysql = require('mysql');
var connection = mysql.createConnection({
host : 'locahost',
user : 'foo',
password : 'bar',
database : 'test'
});
// the callback inside connect is called when the connection is good
connection.connect(function(err){
var sql = "select 'Joe' as name from dual";
connection.query(sql, function(err, rows, fields) {
if (err) return console.log(err);
// you need to end your connection inside here.
connection.end();
console.log(rows[0].name);
});
});
You will likely start wondering about ways to avoid all these callbacks. You may wish to look at my answer to this question for a more extended mysql example as well as an alternative implementation which offers an alternative to callback-mania.