I have started a project using Node.js and Express to connect a MySQL DB to my app, but in some cases, I need multiple SQL Statements to fetch data for the dropdown menus as they come from dimensional tables.
The query works and returns all the data I need but one thing in the documentation called my attention and I'm not sure I fully understand what is meant: (it allows for SQL injection attacks if values are not properly escaped)
https://www.npmjs.com/package/mysql#multiple-statement-queries
What do they mean by "properly escaped"? Could someone please give me an example?
The query I'm running is very similar to this:
let SQL = 'SELECT * FROM dim_val1; SELECT * FROM dim_val2; SELECT * FROM dim_val3;'
connection.query(SQL, function (error, results) {
if (error) throw error;
// `results` is an array with one element for every statement in the query:
console.log(results[0]); // [{1: 1}]
console.log(results[1]); // [{2: 2}]
console.log(results[2]); // [{2: 2}]
});
Is it risky? Is there any workaround to fetch data from multiple tables?
Thanks!
Related
I need every day to get a list of all Customers past due and insert them into a new table, then i need to move the dates with one month, so they will be processed next month again.
Currently i am running 2 SQL Queries in series
insert into history select * from customers where nextdate<CURDATE()
update customers set nextdate=calculation() where nextdate<CURDATE()
But sometimes customers are updated, but not inserted into history.
In think that the update begin to run before mysql finished the select.
I am using Node js, he current method i am using for serialization, is that i am running the update in the callback from the insert, but probably the insert is called back before the code actually runs.
It doesn't happen every time but i think that making a Stored procedure with both in it can help, anyone had some experience with it?
There are two issues here. The first is that your SELECT and UPDATE don't wait for each because SELECT does not lock the table. You can force locked reading by using:
INSERT INTO history SELECT * FROM customers WHERE nextdate<CURDATE() FOR UPDATE;
See https://dev.mysql.com/doc/refman/5.7/en/innodb-locking-reads.html for more information on Locking Reads.
The second is that you probably also want to learn about transactions, which let you issue a batch of queries such that either all of them get committed, or none of them do.
In the case of your code, you want to add the start and end commands for issueing a transaction:
START TRANSACTION;
INSERT INTO history SELECT * FROM customers WHERE nextdate<CURDATE() FOR UPDATE;
UPDATE customers SET nextdate=calculation() WHERE nextdate<CURDATE();
COMMIT;
See https://dev.mysql.com/doc/refman/5.7/en/commit.html for all the details on this mechanism, which anything from MySQL to Postgres to SQLite supports.
you can use promise provided by nodejs to make them in series or you can use Async series
Async Series
async.series([
function(callback) {
//run your First SQL Query based on response send success or failure callback
callback(null, 'one');
},
function(callback) {
//run your Second SQL Query based on response send success or failure callback
callback(null, 'two');
}
]
// your call call back should close connection if needed.
or you can go with first option
Using Promise
(I will recommend to use this one)
var conn = db.config(mysql);
run_query(conn,[ 'insert into history select * from customers where nextdate<CURDATE()' ]).then(function(result){
console.log(result); // result of 1st query
return run_query(conn,[ 'update customers set nextdate=calculation() where nextdate<CURDATE()']);
}).then(function(result){
console.log(result.something); // result of 2nd query
conn.end();
}).catch(function(err){
// will run if any error occur
console.log('there was an error', err);
});
First and foremost, I'm using this module: https://github.com/felixge/node-mysql
The issue I have is I need to use db.end() inside each db.query function.
For example:
db.query('SELECT query here ', function(err, results){
//What if I want to run another query here after getting data? I do this:
db.query('SELECT query here ', function(err, results){
});
db.end();
});
If I were to remove that db.end(); at the end of the first query call, I would get these that start piling up in my process listing window.
Which is totally understandable because I am not ending the connection. My confusion is, I do NOT need to add a db.end() below to my second query function... and even if I do not, it doesn't add sleep processes.. why? Does the outermost db.end() end all queries that are within its scope?
This question is mainly about the best practice of writing queries in nodejs. We had referred several tutorials, but were not able to reach a conclusion.
We have a node js API layer which is mainly used for reading and writing to database. Here is a sample code:
pool.query("update node SET changed = " + params.updationTime + " where nid = " + params.nid);
pool.query("update node_revision SET timestamp = " + params.updationTime +" where nid = " + params.nid);
pool.end();
Is this a correct way of writing code or should we write the sql queries in async format itself.
If your pool configuration allows more than one connection then likely both queries are executed in parallel. Type of call itself does not matter. This example takes 2 seconds to finish:
connection.query('select sleep(1)');
connection.query('select sleep(1)', function() { console.log('done!') });
As well as this one:
connection.query('select sleep(1)', function() {
connection.query('select sleep(1)', function() {
console.log('done!')
});
});
because mysql protocol itself is "sequential" (that is, client is allowed to send next query only after result of previous is fully received). Most async clients hide this limitation by queueing commands internally. In case of two connections, queries actually go in parallel:
connection1.query('select sleep(1)', function() { console.log('done1') });
connection2.query('select sleep(1)', function() { console.log('done2') });
"done1" and "done2" are both going to appear on screen in approximately 1 second
pool.query is a shorlcut for pool.getConnection() + connection.query() + connection.release() - see readme
When writing SQL queries in NodeJS, I cannot promote Knex.js enough!
Programatic way to build dynamic queries. (writing dynamic raw SQL strings is a very manual process)
Connection pools.
Transaction support.
String escaping.
And on and on.
For your specific question, you just make the queries and execute them (using callbacks or Promises), the Knex connection pool will handle all the pooling, and generally things will just work for you.
You'll like it, give it a try : )
I suggest you to use sails.js (http://sailsjs.org/#/) framework, which uses Waterline Query Language(http://sailsjs.org/#/documentation/concepts/ORM/Querylanguage.html) to retrieve data from mySQL/mongodb/Redis database.
I am currently using juggling db with NodeJS for ORM. I now need to do some reporting, which involves joining data from several tables using arbitrary SQL with SUM and GROUP BY too. How would I do this using the jugglingdb framework, and get a list of objects containing data from several columns.
You can access to the query function from the mysql package using the adapter client:
var Schema = require('jugglingdb').Schema;
var schema = new Schema('mysql', {
// your config
});
schema.client.query('your very wild query', function(err, data) {
// data will be an Array of Objects if no error
});
Actually this is a direct call to the query function of node-mysql package
I am writing some non-web app helper, and came across a need for a synchronous query call.
Basically, within a loop I need to check the database to see if the value exists. If it doesn't then insert the value. Currently, with node-mysql I can only get it to work with a callback. Because of that, node.js treats the call as asynchronous and keeps processing my request before the query is finished. This is a big issue because in the end it could be inserting duplicates because they were in the queue.
Ideal Solution - doesn't work. Results is actually the object of client, and I can't find the actual results within. However this does make it synchronous.
results = client.query('SELECT COUNT(md5) as md5Count FROM table WHERE md5 = "' + md5 + '"')
The following does not work. Node.js treats it as asynchronous, and outerResult is still the object of client.
outerResult = client.query('SELECT COUNT(md5) as md5Count FROM board WHERE md5 = "' + md5 + '"', function selectCb(err, results, fields) {console.log(results);});
Any help is appreciated.
Basically, within a loop I need to check the database to see if the value exists. If it doesn't then insert the value.
This is a problem best served with SQL. You don't solve this problem by talking to the database repeatedly, you solve this problem by having SQL only insert where the index value doesn't already exist.
INSERT INTO mytable ( name, address )
SELECT #name, #address FROM DUAL
WHERE NOT EXISTS (SELECT * FROM mytable WHERE name = #name, address = #address)
This is a super simplified example, and not the most optimized. You can do the same thing here with sets of data, instead of record by record, if you like.
Basically, within a loop I need to
check the database to see if the value
exists. If it doesn't then insert the
value. Currently, with node-mysql I
can only get it to work with a
callback. Because of that, node.js
treats the call as asynchronous and
keeps processing my request before the
query is finished. This is a big issue
because in the end it could be
inserting duplicates because they were
in the queue.
There is an asynchronous solution, there always is.
basically your worried that duplicate entries could be entered.
I presume you have an array of data to loop through. Your problem is solved with _.uniq or some other filter solution.
So you simply call _.uniq(md5s).forEach(function() { })