First and foremost, I'm using this module: https://github.com/felixge/node-mysql
The issue I have is I need to use db.end() inside each db.query function.
For example:
db.query('SELECT query here ', function(err, results){
//What if I want to run another query here after getting data? I do this:
db.query('SELECT query here ', function(err, results){
});
db.end();
});
If I were to remove that db.end(); at the end of the first query call, I would get these that start piling up in my process listing window.
Which is totally understandable because I am not ending the connection. My confusion is, I do NOT need to add a db.end() below to my second query function... and even if I do not, it doesn't add sleep processes.. why? Does the outermost db.end() end all queries that are within its scope?
Related
I have started a project using Node.js and Express to connect a MySQL DB to my app, but in some cases, I need multiple SQL Statements to fetch data for the dropdown menus as they come from dimensional tables.
The query works and returns all the data I need but one thing in the documentation called my attention and I'm not sure I fully understand what is meant: (it allows for SQL injection attacks if values are not properly escaped)
https://www.npmjs.com/package/mysql#multiple-statement-queries
What do they mean by "properly escaped"? Could someone please give me an example?
The query I'm running is very similar to this:
let SQL = 'SELECT * FROM dim_val1; SELECT * FROM dim_val2; SELECT * FROM dim_val3;'
connection.query(SQL, function (error, results) {
if (error) throw error;
// `results` is an array with one element for every statement in the query:
console.log(results[0]); // [{1: 1}]
console.log(results[1]); // [{2: 2}]
console.log(results[2]); // [{2: 2}]
});
Is it risky? Is there any workaround to fetch data from multiple tables?
Thanks!
I need every day to get a list of all Customers past due and insert them into a new table, then i need to move the dates with one month, so they will be processed next month again.
Currently i am running 2 SQL Queries in series
insert into history select * from customers where nextdate<CURDATE()
update customers set nextdate=calculation() where nextdate<CURDATE()
But sometimes customers are updated, but not inserted into history.
In think that the update begin to run before mysql finished the select.
I am using Node js, he current method i am using for serialization, is that i am running the update in the callback from the insert, but probably the insert is called back before the code actually runs.
It doesn't happen every time but i think that making a Stored procedure with both in it can help, anyone had some experience with it?
There are two issues here. The first is that your SELECT and UPDATE don't wait for each because SELECT does not lock the table. You can force locked reading by using:
INSERT INTO history SELECT * FROM customers WHERE nextdate<CURDATE() FOR UPDATE;
See https://dev.mysql.com/doc/refman/5.7/en/innodb-locking-reads.html for more information on Locking Reads.
The second is that you probably also want to learn about transactions, which let you issue a batch of queries such that either all of them get committed, or none of them do.
In the case of your code, you want to add the start and end commands for issueing a transaction:
START TRANSACTION;
INSERT INTO history SELECT * FROM customers WHERE nextdate<CURDATE() FOR UPDATE;
UPDATE customers SET nextdate=calculation() WHERE nextdate<CURDATE();
COMMIT;
See https://dev.mysql.com/doc/refman/5.7/en/commit.html for all the details on this mechanism, which anything from MySQL to Postgres to SQLite supports.
you can use promise provided by nodejs to make them in series or you can use Async series
Async Series
async.series([
function(callback) {
//run your First SQL Query based on response send success or failure callback
callback(null, 'one');
},
function(callback) {
//run your Second SQL Query based on response send success or failure callback
callback(null, 'two');
}
]
// your call call back should close connection if needed.
or you can go with first option
Using Promise
(I will recommend to use this one)
var conn = db.config(mysql);
run_query(conn,[ 'insert into history select * from customers where nextdate<CURDATE()' ]).then(function(result){
console.log(result); // result of 1st query
return run_query(conn,[ 'update customers set nextdate=calculation() where nextdate<CURDATE()']);
}).then(function(result){
console.log(result.something); // result of 2nd query
conn.end();
}).catch(function(err){
// will run if any error occur
console.log('there was an error', err);
});
Context:
I'm building a web application that calls data from a large db (several millions of rows for table); sometimes a user can change his mind and call for new data before the query on the db has been completed.
Technical question:
I tried to kill the query in this cases using:
app.get("/data", function(req, res) {
if (req.query.killQuery == "true") {
con.query("KILL \"" + threadId + "\"", function(err) {
if (err) throw err;
console.log("I have interrupted the executing query for a new request");
giveData(req, res); //The function that will execute a new query
});
return;
}
giveData(req, res); //The function that will execute a new query
});
Now I have several doubts about this code:
I had to use a second connection to kill the thread of the first, since the first was unable to perform new queries before the first was completed. Is this a Node.js behaviour or is it the right way to do this kind of things?
The KILL thread_id statement closes the whole connection instead of stopping the single query. Again, is it Node.js behaviour, or is it MySQL itself? Should I really disconnect and reconnect to stop a query and start with an other?
If you have a modern version of MySQL, you can use KILL QUERY <threadId> instead which will only kill the currently executing query on that connection but leave the connection intact.
We are using mysql module for node and i was just wondering if this approach is good or does it have any bad effects on our application, consider this situation
dbPool.getConnection(function(err, db) {
if(err) return err;
db.query()
Here i am calling the dbPool object and requesting a connection from the pool then using it. However i found another implementation (which is the one i am asking about) which uses the dbPool object directly like:
dbPool.query('select * from test where id = 1' function(err, rows) {})
so i was wondering what does the second implementation does exactly, does it automatically return a free connection and use it ? can explain what is happening exactly in the second case and if it has any effect + or - on my application ? Thank you.
So this is so what called callback chaining. In NodeJS you have a lot of asynchronous calls going around. But sometimes you want to do something when the connection is done with MySQL. That's why the getConnection functionality has a callBack feature.
dbPool.getConnection(function(err, db) {
if(err) return err;
db.query()
Is equal to this:
dbPool.query('select * from test where id = 1' function(err, rows) {})
dbPool.query() will wait for the connection to be open, you don't have to put all your queries inside the getConnection to make it work. This is why it also has a callBack feature.
Tell me if I'm wrong. I hope this solves your question.
This question is mainly about the best practice of writing queries in nodejs. We had referred several tutorials, but were not able to reach a conclusion.
We have a node js API layer which is mainly used for reading and writing to database. Here is a sample code:
pool.query("update node SET changed = " + params.updationTime + " where nid = " + params.nid);
pool.query("update node_revision SET timestamp = " + params.updationTime +" where nid = " + params.nid);
pool.end();
Is this a correct way of writing code or should we write the sql queries in async format itself.
If your pool configuration allows more than one connection then likely both queries are executed in parallel. Type of call itself does not matter. This example takes 2 seconds to finish:
connection.query('select sleep(1)');
connection.query('select sleep(1)', function() { console.log('done!') });
As well as this one:
connection.query('select sleep(1)', function() {
connection.query('select sleep(1)', function() {
console.log('done!')
});
});
because mysql protocol itself is "sequential" (that is, client is allowed to send next query only after result of previous is fully received). Most async clients hide this limitation by queueing commands internally. In case of two connections, queries actually go in parallel:
connection1.query('select sleep(1)', function() { console.log('done1') });
connection2.query('select sleep(1)', function() { console.log('done2') });
"done1" and "done2" are both going to appear on screen in approximately 1 second
pool.query is a shorlcut for pool.getConnection() + connection.query() + connection.release() - see readme
When writing SQL queries in NodeJS, I cannot promote Knex.js enough!
Programatic way to build dynamic queries. (writing dynamic raw SQL strings is a very manual process)
Connection pools.
Transaction support.
String escaping.
And on and on.
For your specific question, you just make the queries and execute them (using callbacks or Promises), the Knex connection pool will handle all the pooling, and generally things will just work for you.
You'll like it, give it a try : )
I suggest you to use sails.js (http://sailsjs.org/#/) framework, which uses Waterline Query Language(http://sailsjs.org/#/documentation/concepts/ORM/Querylanguage.html) to retrieve data from mySQL/mongodb/Redis database.