I am using node-mysql in my project. I have a scenario where the execution of the second query will be based on the result returned from the first. For example, first query, select * from table where user=x; then based on the existence I will run another query. I am using pool.getConnection(). May I know whether I should create two connections or use one connection?
Option a)
pool.getConnection(function(err, connection) {
// Use the connection
connection.query( 'SELECT something FROM sometable', function(err, rows) {
//store the result here as a var a;
connection.release();
});
});
//check var a and start another connection to run the second query?
Option b)
pool.getConnection(function(err, connection) {
// Use the connection
connection.query( 'SELECT something FROM sometable', function(err, rows) {
//...based on the result,
connection.query(second query)
connection.release();
});
});
Thanks
Hammer
Both options are using a pool, so you're already using multiple connections. So either option doesn't really matter, the only difference might be that the first option may result in the second query not being executed right away if the entire pool is in use after the first query is done.
Related
I have used one signle connection object of MYSQL in node JS to serve for multiple users.
I mean to say that MySQL connection will be created upon starting the script & it will remain same until the life of the node script/server.
Practically, this is possible to do & i have done the same. Please take a look at below code of NodeJS/MySQL script.
#################################
var http = require('http');
var mysql = require('mysql');
var con = mysql.createConnection({
host: "192.168.1.105",
user: "root",
password: "XXXXXX",
database: "mydb"
});
con.connect(function(err) {
if (err) {
console.error('error: ' + err.message);
process.exit(1);
}
http.createServer(function (req, res) {
continueExecution(req,res);
}).listen(8082);
});
async function continueExecution(req,res){
res.write('calledddd\n');
for (let step = 0; step < 50; step++) {
// Runs 5 times, with values of step 0 through 4.
var bar = `Company Inc ${step}`;
var sql = `INSERT INTO customers (name, address) VALUES ('${bar}', 'Highway 37')`;
res.write(sql + "\n");
con.query(sql, function (err, result) {
if (err) throw err;
res.write("1 record inserted\n");
});
}
res.write('reached\n');
for (let ste = 0; ste < 50; ste++) {
res.write('started Update\n');
var bar = `Company Inc ${ste}`;
var sql = `UPDATE customers SET name = 'UPDATE RECORD' WHERE name = '${bar}'`;
con.query(sql, function (err, result) {
if (err) throw err;
res.write(result.affectedRows + " record(s) updated\n");
if(ste == 50) {
res.writeHead(200, {'Content-Type': 'text/html\n'});
res.write('Databse connected\n');
res.end();
}
});
}
}
#################################
I have several questions in my mind as i am technical expert. But i didn't find any resources over my questions. Please help me on this
Q1. Are there any type of consequences of using one single MySQL connection to provide response to multiple users?
Q2. Let's take an example.
100 users wants to access table name "users_data" at the same time. 25 users are updating their records on the same table with unique primary key. 50 users are selecting their records. another 25 users deleting their records.
All these operations are being done at the same time via parallel Node Script calls from remote device.
To complete all these MySQL transactions, system is using only 1 database connection.
What will happen in this case?
To answer your questions, one of the consequences of using a unique connection is that it can lead to slower request execution.
In fact, even if node will make the requests asynchronously, your database will execute all those requests synchronously, so one after the other in the order they came. As node makes the requests asynchronously, the order in which they are executed by your database is not granted, and the issue you are referencing to might happen.
One easy way to avoid this is to use a connection pool which will create a given number of connection, using the same db user. Here are some links that might help you with this :
using a connection pool with node.js
connect a mysql database with node.js
I've been developing an "Employee leave management" web app project for our internal use using node.js with express and ejs template. Now, my employer wants me to make the app accessible through internet and I'm worried about SQL injection.
Let's say I have a button like this in html:
Edit
This will GET from index.js file:
const { edit } = require("./request");
app.get("/edit/:ReqID", edit);
This will then go to module edit in request.js file:
module.exports = {
edit: (req, res) => {
let ReqID= req.params.ReqID;
let squery = `SELECT * FROM table1 WHERE ReqID="${ReqID}";
SELECT * FROM table2 WHERE ReqID="${ReqID}";`;
db.query(squery, function (err, result) {
if (err) {
return res.status(500).send(err);
}
res.render("edit.ejs", {
srecords1: result[0],
srecords2: result[1]
})
})
}
}
There might be two or more queries in there and I'm using mysql driver for node.js with multipleStatements: true and I'm aware of warning "Support for multiple statements is disabled for security reasons (it allows for SQL injection attacks if values are not properly escaped)." This will return something like http://localhost:port/edit/reqid on the browser address box. I saw a video from youtube that says SQL Injection can be done through the browser's address box like http://localhost:port/edit/reqid;";SELECT * FROM users; so I did that and for sure I can see that syntax being send to the server. So I follow the suggestion in the video to do a placeholder like this:
module.exports = {
edit: (req, res) => {
let ReqID= req.params.ReqID;
let squery = `SELECT * FROM table1 WHERE ReqID= ?;
SELECT * FROM table2 WHERE ReqID= ?;`;
db.query(squery, [ReqID, ReqID], function (err, result) {
if (err) {
return res.status(500).send(err);
}
res.render("edit.ejs", {
srecords1: result[0],
srecords2: result[1]
})
})
}
}
Then I try the extreme http://localhost:port/edit/reqid;";DELETE FROM users; and http://localhost:port/edit/reqid;";DROP TABLE users; separately and it works! First it deletes data from users tble and for sure the second drop table command also worked. After the first attempt, I refresh the browser with the same sql injection syntax and I've got this message:
{"code":"ER_BAD_TABLE_ERROR","errno":1051,"sqlMessage":"Unknown table 'users'","sqlState":"42S02","index":1,"sql":"SELECT * FROM table1 WHERE ReqID= "ReqID;";drop table users;";SELECT * FROM table1 WHERE ReqID= "ReqID;";drop table users;";"}
So, the table users clearly have been dropped from the database.
Update:
I did further testing based on the information I gained from this answer and I did something like this:
module.exports = {
edit: (req, res) => {
let ReqID= req.params.ReqID;
db.query(`SELECT * FROM table1 WHERE ReqID= ?; SELECT * FROM table2 WHERE ReqID= ?;` , [ReqID, ReqID], function (err, result) {
if (err) {
return res.status(500).send(err);
}
res.render("edit.ejs", {
srecords1: result[0],
srecords2: result[1]
})
})
}
}
Then I re-test with multiple variation of http://localhost:port/edit/reqid;";DROP TABLE users; (double quote in between)
http://localhost:port/edit/reqid;';DROP TABLE users; (single quote in between) etc. and it doesn't seem to be dropping the table anymore. However, I still see the statement being sent to the server so I'm still wary of the DROP syntax being effective somehow.
Update 2:
Note: Fortunately, the deployment has been delayed and I have more time to sort out the issue.
After researching for a while, taking the comments into consideration and testing multiple method, I came up with this structure:
function(req, res) {
let dcode = [req.body.dcode];
let query1 =`SELECT col1, col2 FROM table1 WHERE DCode=?`;
db.query(query1, dcode, function(err, result_1) {
if (err) {
return res.status(500).send(err);
}
let query2 =`SELECT col1, col2 FROM table2 WHERE DCode=?`;
db.query(query2, dcode, function(err, result_2) {
if (err) {
return res.status(500).send(err);
}
res.render("login.ejs", {
result1: result_1,
result2: result_2
});
});
});
}
Which is simple enough and no major change to my current codes. Would this be sufficient to prevent SQL injection in node.js?
Allowing multi-statement strings, itself, invites SQL injection. So, avoid it.
Plan A:
Consider ending an array (perhaps in JSON) to the server; let it then execute each statement, and return an array of resultsets.
But it would be simpler to simply issue the statements one at a time.
(If the client and server are far apart, the one-statement-at-a-time method may cause a noticeable latency.)
Plan B:
Build suitable Stored procedures for any multi-statement needs. This, where practical, avoids multi-statement calls. And avoids latency issues (usually).
Here are a few suggestions that might help:
Never use template strings like this: Select * from table where id = ${value}. SQL injections will happen - 100%!. Instead you should use build in driver defense mechanism. Like this: query('Select * from table where id = ?', [value]). This should prevent SQL injection.
Use single statements per query. If you need to do multiple operations in one request to database - consider creating stored procedure. Stored procedures also have build in security mechanism.
Consider using query builder or ORM. They also have additional layer of security on top of build in driver one.
You could also explicitly escape SQL string with help of 3rd party library.
I am running a query which gives me confusing results. I have a node.js server which takes some user input from a form on the client side and uses socket.io to get the input on the server. here is the function which runs after receiving user input
databaseCheck(data);
function databaseCheck(userInput){
var mysql = require('mysql');
var connection = mysql.createConnection({
host : '12.34.56.78',
user : 'user',
password : 'password',
database : 'db'
});
connection.connect();
var query = connection.query('SELECT * from `table` WHERE `email` = userInput', function(err, rows, fields) {
if (!err) console.log(rows[0].username);
else console.log("connection failure");
});
connection.end();
}
So when ever I use this code, it keeps printing "connection failure" to the screen. It doesn't happen when I replace userInput with the "example#email.com" so I'm guessing there is some problem with using the variable userInput in the query. Can someone tell me what is wrong with my code?
Not only do you need to pass the userInput by appending it to the string, you need to escape it so that the query recognizes it as a string:
connection.connect();
var query = 'SELECT * from `table` WHERE `email` = ' + JSON.stringify(userInput);
console.log(query);
connection.query(query, function(err, rows, fields) {
if (!err) console.log(rows[0].username);
else console.log(err.name, err.message);
});
connection.end();
It also helps to make the error message more informative by displaying the actual error instead of a generic message.
Lastly, put connection.end(); inside the callback. According to what you said, it appears to work like you had it but it's generally a bad idea to end a connection before an asynchronous process using the connection has called back.
Ignore my last comment, it appears I was wrong in this particular case. According to the repository documentation, it says:
Closing the connection is done using end() which makes sure all remaining queries are executed before sending a quit packet to the mysql server.
Try this for testing and resolution. Printing to the log will let you see what you are putting in the query.
var querystring = "SELECT * from table WHERE email LIKE " +
userInput;
console.log(querystring);
var query = connection.query(querystring, function(err, rows, fields) {...
The situation is: In one http GET request I need to select from one table the information I need and send to the client, and at the same time I need to retrieve the user IP and insert into a database. I'm using Node.js for this experiment.
The thing is: Is there a way to make the two actions together? Or do I have to connect and make two separate queries? Is there a way to render the page and do the other INSERT action in the background? What is the fastest option?
app.get('/', function({
connect.query("SELECT column1, column2 FROM table;", function(err, ...
render("index", ...);
});
connect.query("INSERT INTO table2 SET ip=11111111;");
});
The procedure approach suggested by #skv is nice but you have to wait for the write before doing the read and eventually returning a result to the user.
I would argue for another approach.
Queue the ip-address and a timestamp internally in something like an array or list.
Do the read from the database and return a result to the user
Create a background job that will nibble of the internal array and do the inserts
This has several benefits
The user gets a result faster
The writes can be done later if the system is being called in bursts
The writes can be done in batches of tens or hundreds of inserts reducing the time it takes to write one row.
You can make a stored procedure do this
Basically these are two different operations, but doing it in stored procedures might give you the assurance that it will surely happen, you can pass the IP address as the parameter into the stored procedure, this will also avoid any worries of performance in the code for you as the db takes care of insert, please remember that any select that does not insert into a table or a variable will produce a result set for you to use, hope this helps.
DELIMITER $
CREATE PROCEDURE AddIPandReturnInfo
(
#IPAddress varchar(20)
)
BEGIN
INSERT INTO Yourtable (IPAddress);
SELECT * FROM Tablename;
END $
DELIMITER ;
Well, I assume you're using this module https://github.com/felixge/node-mysql
The MySQL protocol is sequential, then, to execute paralell queries against mysql, you need multiple connections. You can use a Pool to manage the connections.(builtin in the module)
Example:
var mysql = require('mysql');
var pool = mysql.createPool({
host: 'example.org',
user: 'bob',
password: 'secret',
connectionLimit: 5 // maximum number of connections to create at once **10 by default**
});
app.get('/', function (req, res) {
// get a connection from the pool //async
pool.getConnection(function (err, connection) {
// Use the connection
connection.query('SELECT something FROM table1', function (err, rows) {
// Do somethig whith the mysql_response and end the client_response
res.render("index", {...
});
connection.release();
// Don't use the connection here, it has been closed.
});
});
//async
var userIp = req.connection.remoteAddress || req.headers['x-forwarded-for'] || null;
if (userIp) {
// get a connection from the pool again
pool.getConnection(function (err, connection) {
// Use the connection
connection.query('INSERT INTO table2 SET ip=?', [userIp], function (err, rows) {
// And done with the insert.
connection.release(); // Conn Close.
});
});
}
});
I'm using the node-mysql driver with connection pooling.
Releasing the connection back into the pool when there's only one query, is easy:
pool.getConnection(function(err, connection) {
if (err) {
throw err;
}
query = "SELECT * FROM user WHERE id = ?";
connection.query(query, [id], function(err, users) {
connection.release();
if (err) {
throw err;
}
// ...
});
});
What if I need to use the connection a second time? I'd have to move the release() down a few lines. But what happens if the error is thrown? Is the connection never returned to the pool?
Do I have to use some control flow lib to have a "finally" moment in which I could release it?
Any better ideas?
One way this could be handled is promises. Since you're building a pool, you can construct your requests with something like q (or native promises, coming soon):
// assuming you have your local getConnection above or imported
exports.getConnection = function(queryParams) {
var d = q.defer();
local.getConnection(function(err, conn) {
if(err) d.reject(err);
d.resolve(conn);
});
});
So, wrap few of your other calls into promises like that and then just compose your query:
db.getConnection()
.then(function(conn){
makeRequest()
.then(...)
...
.catch(function(err){
// at the end you release the conn
});
Does it seem like something you're asking for?
When you perform a MySQL query, for that time it locks the database until query completes. After the successfull completion of query, it releases that database lock.
Same case here: connection.release(); simply releases the DB connection, nothing else.
You should use separate connections for that situation. That's what the connection pool is for. That way one doesn't have to wait for the other one to finish before it can start. I only use the same connection if one query cannot start until the other query has finished.