intermittent 500 error coming from node.js websocket server - mysql

I have a websocket node.js app (game server) that runs a multiplayer html5 game.
The game has a website also. The game server and the website are on the same Apache VPS.
The game server uses mysql to store and retrieve data using mysql pooling from the node.js mysql package.
It works fine 99% of the time, but intermittently, at a random point, it will all of a sudden stop being able to get a mysql connection.
When this happens the website stops working and shows a 500 http error. I believe that this is whats causing the problem in the game server. Because of the 500 http error, mysql can no longer be connected to and thus pool.getConnection no longer works in the game server.
I find it strange that even though Apache is throwing up a 500 error, the game server can still be accessed successfully through a websocket as usual. The only thing that appears to have stopped working inside the game server is mysql. The game client connects to the game server via websocket and the functions work correctly, except for being able to connect to mysql.
If I ctrl+c the game server to stop the node.js app (game server) then the 500 error goes away. The website instantly serves up again, and then if I restarting the game server, mysql is now working again.
Obviously something in the game server is causing this to happen. So far i cannot find what it is. I am stuck now, i've spent a full week trying everything i could think of to debug this.
After running debug mode on mysql, im seeing this;
<-- ErrorPacket
ErrorPacket {
fieldCount: 255,
errno: 1203,
sqlStateMarker: '#',
sqlState: '42000',
message: 'User (i've hidden this) already has more than
\'max_user_connections\'
active connections' }
But I have it set to 100000 connections. No way is there that many being used. Every time I finish with a connection I use connection.release() to put it back into the pool. What do I need to do to fix this?
Please, any suggestion you have to debug this is greatly appreciated!
Thanks in advance.
here is the way i'm using mysql in the game server
const mysql = require('mysql');
const pool = mysql.createPool({
connectionLimit : 100000,
host : '***********',
user : '***********',
password : "'***********",
database : '***********',
debug : true
});
pool.getConnection(function(err,connection){
if (err) {
console.log(err);
return false;
}
connection.query("select * from aTable ",function(err,rows){
if(err) {
console.log(err);
connection.release();
return false;
}
// dos stuff here
connection.release();
})
})
1 thing i am wondering, if there is an error in the top error catch block
here ->
pool.getConnection(function(err,connection){
if (err) {
console.log(err);
return false;
}
Then the connection is not released right? So that would keep a connection alive and this is whats causing the problem? over time an error happens here and there, after a random amount of time, enough of these happen and that's what is causing it? it's like a build up of open connections???

This was the mistake i was making;
pool.getConnection(function(err,connection){
if (err) {
console.log(err);
return false;
}
connection.query("select * from aTable ",function(err,rows){
if(err) {
console.log(err);
connection.release();
return false;
}
if(somethingrelevant){
// do stuff
connection.release();
}
})
})
And that meant that if somethingrelevant didn't happen, then the connection would stay open.
My pool would continue to open new connections but they weren't always being put back.

Related

MySQL NodeJS - How do I verify that my connection pool is actually online?

So I have a simple NodeJS bot that makes use of MySQL to store data.
I initialize mysql by using the createPool function.
I find that this works fine, even if my server is dead. I can run it and no errors are fired, but when I run any code that uses a query on the server, then it will fire errors if the server is off.
I want to simplify this by simply forcing a message / error if the bot is started and the server is not online.
This is the code that I have setup so far. Ignore the dotenv stuff, its just to keep my login / database info out of the code.
const mysql = require("mysql");
const path = require('path');
const envconfpath = path.join(__dirname, './.env');
require('dotenv').config({ path: envconfpath });
const connectionPool = mysql.createPool({
connectionLimit : 999,
host: process.env.HOSTNAME, database: process.env.DATABASE,
user: process.env.USERNAME, password: process.env.PASSWORD
});
// This is the code that does not function.
connectionPool.getConnection((err, con) => {
if (err)
console.warn(`Databased failed to start with error: ${err}`);
if(con.state === 'disconnected')
console.log(`Database connection failed. Check that its running and that your configuration is correct`);
else
console.log('Database connection successful');
});
Now, I do want to clear up any possible misunderstanding, I can use my database fine, I simply want to fire an error on initial run if the database's server is not online.
The bottom of that code that gets the connection and runs checks seems to always fire an error. Ive tried various methods for checking whether the server was online, but the code above was made by using examples from the mysql / nodejs documentation
This is the error that it fires, which makes no sense. If the pool is valid (which it is) the getConnection function (which is used on the module's documentation) should return a connection just fine. It says that the state of the connection is invalid, yet I can query the connection fine in other parts of the code outside of the small part I pasted in here.
As for the ACCESS DENIED part? No clue. I run the database both remotely and locally, Ive verified that both worked, and during testing, I dont use a password. This is a completely open database running locally on my system, so I dont see why it would have access issues.
Databased failed to start with error: Error: ER_ACCESS_DENIED_ERROR: Access denied for user 'USER'#'HOST' (using password: NO)
----- UNCAUGHT EXCEPTION -----
TypeError: Cannot read properties of undefined (reading 'state')

ETIMEDOUT on subsequent database connections under load

I've been having a problem for several weeks now about a ETIMEDOUT error when doing subsequent database calls under load and it has left me completely stumped.
For a heads up, I have an Azure App service where I'm running my node js backend. It's running on a S2 plan. I also have a flexible MySQL server on azure too. I don't remember the plan's name, but it's a mid priced one that can allow up to 2700 connections. We are hosting an API that can see thousands of requests in a day and will make a few database calls per request.
The server and database calls all run normally when the server isn't under load, but the moment the server goes under load, even if it's a little such as 100 requests all at the same second, the database calls start to fail with ETIMEDOUT. Here's how I have the database calls setup.
First I grab a connection to be used. My connection is setup as
getConnection(bMultipleStatement) {
return mysql.createConnection({
host: process.env.MYSQL_HOST,
user: process.env.MYSQL_USER,
password: process.env.MYSQL_PASSWORD,
database: process.env.MYSQL_DATABASE,
charset: 'utf8mb4',
multipleStatements: bMultipleStatement
});
}
conn = base.getConnection(false);
Once I get a connection from the database, I perform basic SELECTS and UPDATES as such
conn.query(sql, [variables], function (error, results) {
if (error){
if (conn) { conn.end(); }
// log error
}
else {
if (conn) { conn.end(); }
if (results.length > 0) {
//continue
}
else {
//log not found
}
}
});
The first 20-30 requests will be able to successfully run this query, but everything else after it will just generate a ETIMEDOUT. I have checked my azure metrics and can see that the amount of connections and CPU % are both low very low. Around 5% CPU and 50 active connections.
This problem has left me stumped. Most everything I've searched for this issue only involves being completely unable to connect rather than it working fine until it goes under load. Has anybody experienced a similar issue and know how to solve it? The only thing that I was able to find to help was that it doesn't run into this issue locally, only when it's on azure, but this is because my local machine has much better hardware than the azure server. Upgrading is a last resort option of ours because that's just putting a bandaid over the problem.

move express js app from localhost to web server

I was developing a CRUD web app through the use of express js, node js and a local SQL database (MySQL) locally, but now it's time to move to the web and I chose an apache web server that supports MySQL.
I already have the database loaded on the Apache web server.
I have already modified the credentials for the connection to the web server (I know that they are correct because I am connected via MySQL local server to it via a tunnel), through something like that:
app.use(
connection(mysql,{
host: '127.127.127.127', //'web hostname'
user: 'root',
port: '80',
password : 'password',
database:'db_name'
},'pool') //or single
);
But when I'm going to query on it I get an error on the connection.query() command:
... throw err; // Rethrow non-MySQL errors
^
TypeError: Cannot read property 'query' of undefined ...
for example in this part of the code:
exports.view1 = function(req, res){
req.getConnection(function(err,connection){
var query = connection.query('SELECT * FROM table',function(err,rows)
{
if(err){
console.log("Error Selecting : %s ",err );
}else{
res.render('view',{page_title:"Table view",data:rows});
}
});
});
};
1) I'm sure that this error is due to the fact that the connection variable is empty, so it is not possible to perform the query, every help is welcome.
2) I have not found much on the net, but is it possible to continue to use express js even on an Apache web server? Do I have to change something in the formulation of database queries?
[SOLVED] Maybe it can help someone in my same situation. I had to change servers online, going from a Mysql server to a node js, in order to already have the basic functionality of node for the management of my app. However I had to install the mysql-server package, but the connection to my app express js, once moved from local to the server, with the right credential changes, worked properly.

Node.js MySQL connection management with HTTP Server

I do understand that one should be closing mysql connections on existing node process. But with long live connections like Http servers things are little different and i seems to be confusing myself.
With node-mysql (npm package) i will be doing connection.end(); in order to end connection with mysql.
Now the questions is should i be doing it every time before sending the final response to http client or how it should work ?
For Example
// assuming express app
app.get("/users",function(req,res){
connection.query("SELECT * FROM users",function(err,rows){
connection.end();
if(error){
res.send(error);
}else{
res.send(rows);
}
})
})
Personally in a huge project it is hard to maintain connection.end(); statements for every Http request. So i am interested in knowing what are the best ways to manage mysql connection in NodeJs HTTP Servers.

Node.js, ORM2 and Mysql. Connection are not closing

I am using node.js, express, ORM2 and Mysql. Everytime a page loads a new mysql connection is opened. The issue here is that the connection doesn't close, it stays open. So, each requests result a new "sleep" status connection in my mysql "show processlist" command.
Thanks,
Radu
Actually because i am new at node i did not realize that my application never ends the execution and i have to use singleton method for the mysql connection.
Using something like:
if (connection) return cb(null, connection);
orm.connect(settings.database, function (err, db) {
if (err) return cb(err);
connection = db;
db.settings.set('instance.returnAllErrors', true);
setup(db, cb);
});
This will keep just one mysql connection open.