Executing multiple queries in Nodejs route - mysql

I have Node.js post route in which I am executing several queries in sequence with async/await.
router.post("/foo", async (req, res) => {
const qr1 = "str1";
await db.query(qr1); // Works fine
const qr2 = "str2";
await db.query(qr2); // Works fine
const qr3 = "str3";
await db.query(qr3, async (err, result) => {
if (err) {
console.log(err);
}
if (result.length > 0) {
// Required data is received – That part works
// do something and continue to step 4
}
});
// step 4 – Never gets here
});
All queries which perform table manipulation i.e. delete, insert etc work fine. Once I get to a select query I receive the required data but i need to continue to the next step which doesn't happen.
I know I can accomplish what I need by nesting step 4 within step 3, but I would like to know if there is a different way to do that.

The issue you have is that comment // step 4 is executed immediately whereas the async(err code is executed asynchronously when query 3 executes. Here is your code corrected to achieve your result:
router.post("/foo", async (req, res) => {
try {
const qr1 = "str1";
await db.query(qr1); // Works fine
const qr2 = "str2";
await db.query(qr2); // Works fine
const qr3 = "str3";
const q3Result = await db.query(qr3);
console.log('step 4');
// step 4 – Should work fine
}
catch(err) {
console.log(err);
}
});

Related

How to implement a map nested MySQL query in Node that populates a property of each element

I've searched this so many ways and can't seem to get a solution that works for me - perhaps because I'm new to JS and the callback "hell" as I've seen it described.
I first run a query that returns a list of results ("FailTypes"), then I want to iterate over each item in the list and add a property ("FailTypeAreaScores") which itself requires another query. I figured the map function would be the best to do this.
Here's the calling function, 1st query:
static async getFailTypes(eq, callback) {
const sql = 'CALL getFailTypes(?)';
db.query(sql, eq, async (err, rows) => {
let result = Object.values(JSON.parse(JSON.stringify(rows)));
let fts = await Promise.all(result[0].map(async ft => {
const newFt = await getFailTypeAreaScores(ft, (err, data) => {return data})
return newFt }));
if (err){
console.log(err.message);
} else {
callback(null, fts);
};
});
};
Here's the nested query:
async function getFailTypeAreaScores(ft, callback){
const sql = 'CALL getFailTypeAreaScores(?);';
db.query(sql, ft.ID, async(err, rows) =>{
let result = Object.values(JSON.parse(JSON.stringify(rows)));
if (err){
console.log(err.message);
} else {
ft.ftas = result[0];
callback(null, ft);
}
});
};
I've tried various methods of promises and callbacks and nothing seems to work. Right now, I'm getting the data back as expected at the {return data} and if I replace that with a {console.log(data)} I can see the new ft with the added property in the console.
However, it doesn't seem to get out of that function and back up to the map to replace the original ft.
So, my resulting callback(null, fts) just returns a list of null.

Nodejs MySQL keeps returning the results

I am having this issue where the result of the MySQL query in NodeJS keeps returning the results in the console and I am wondering why is this happenning?
Here is what I have done:
Server.js
app.get("/api/listproduct", (req, res) => {
db.query("SELECT * FROM products" , (err, result) => {
if (err) {
console.log(err);
} else {
console.log(result)
res.send(result);
}
}
)
})
ShowProduct.js
useEffect(async () => {
const result = await axios.get('http://localhost:3000/api/listproduct');
console.log(result.data)
setProducts(result.data);
});
As you can see that the result are sort of looping to the console as shown here where it was supposed to just return only one set rather than many of the same sets of results.
What am I missing here and how to solve this? Many thanks in advance and greatly appreciate any helps. Thanks
This has nothing to do with the Nodejs/MySQL backend, but your frontend React code.
You don't have a dependency array in your useEffect, so it's called every time the component is rendered. Since it calls setState, it causes a new render, and effectively an infinite loop of renders. If you don't have dependencies for your effect, add an empty array to make the effect get called only once.
useEffect(async () => {
const result = await axios.get("http://localhost:3000/api/listproduct");
setProducts(result.data);
}, []); // <- that empty array

MySQL querying in NodeJS

I'm trying to write an app that will check whether or not an webpage has changed.
I'm using NodeJs mysql because I'm familiar with them.
So at the moment, I have the problem that my query is too 'slow', so my function will not return true, even though it should. Can I force my app to wait for the query somehow?
Edit:
checkEntry(webpage, callback) {
var oldPage;
this.mysql.query('SELECT OldWebpagecol FROM Web_Scraping.OldWebpage WHERE idOldWebpage = 15', (err, rows) => {
if (err) console.log(err);
oldPage = rows[0].OldWebpagecol;
if (webpage === oldPage) {
return true;
}
return false;
})
}
You can use async/await when you make a request.
For example,
(async () => {
let response = await fetch(‘/api/users’);
})();

discord.js/node.js make code wait until sql query returns result

I am working on a discord.js bot, and I'm storing a bunch of information on various servers in a database. The problem is, that the code doesn't wait for the database to return the results. In the current situation, I'm trying to check if the server specific prefix checks out.
I tried using async and await at various places, but those didn't work. If I could, I'd rather not use .then(), because I don't really want to put all the commands inside a .then().
const { Client, Attachment, RichEmbed } = require('discord.js');
const client = new Client();
const mysql = require("mysql");
const config = require("./config.json")
var con = mysql.createConnection({
host: 'localhost',
user: 'root',
password: '',
database: 'botdb'
})
client.on("ready", () => {
console.log("I'm ready")
})
client.on("message", message => {
if (message.author.bot) return;
if (message.channel.type === 'dm') return;
let msg = message.content.split(" ");
let command = msg[0];
let prefix;
con.query(`SELECT * FROM serversettings WHERE ServerID = ${message.guild.id}`, (err, rows) => {
if (err) throw err;
prefix = rows[0].Prefix;
console.log(prefix)
})
console.log(`Prefix: ${prefix}, Command: ${command}`)
if (command === `${prefix}examplecommand`) {
//Do something
}
//Other code that uses prefix and command
}
It should log the prefix first, and then the Prefix: ${prefix}, Command: ${command} part, but it does it the other way around, so the examplecommand doesn't work.
Your result is caused by the fact that what's outside your query callback is executed immediately after the call. Keep in mind the mysql module is callback-based.
Possible Solutions
Place the code inside the callback so it's executed when the query is completed.
Wrap the query in a promise and await it.
function getGuild(guildID) {
return new Promise((resolve, reject) => {
con.query(`SELECT * FROM serversettings WHERE ServerID = '${guildID}', (err, rows) => {
if (err) return reject(err);
resolve(rows);
});
});
}
const [guild] = await getGuild(message.guild.id) // destructuring 'rows' array
.catch(console.error);
console.log(guild.prefix);
Use a Promise-based version of a MySQL wrapper, like promise-mysql. You could use it the same way as the code above, without worrying about coding your own Promises.
const [guild] = await con.query(`SELECT * FROM serversettings WHERE serverID = '${message.guild.id}'`)
.catch(console.error);
console.log(guild.prefix);

Why is my AWS Lambda node.js mysql query not returning?

I'm trying to write some external data into some local tables. We'll be looping through an array, writing most of the data in each array element to the main table and the rest to related tables, replacing all the data each time.
I've stripped the code down to the bare bones to show the problem I'm having. The DELETE runs fine, but the INSERT runs only once, and doesn't even return.
I have a screenshot of the output at https://imgur.com/a/zA6Hz8g .
In it, you can see that the code for the DELETE runs fine (ComQueryPacket sent, OkPacket returned) but when it gets to the INSERT, the ComQueryPacket is sent but nothing is returned. And then the code just falls through.
This results in the first row writing successfully, but no subsequent rows get written.
I've tried changing the connection to use pools, but that didn't help either.
Any ideas?
var mysql = require('mysql');
var promise = require('promise');
const con = mysql.createConnection({
<connectionInfo>,
debug: true
});
function connectToDB() {
return new promise((resolve, reject) => {
console.log("IN connectToDB");
con.connect(function(err) {
if (err) {
console.log("ERROR: Could not connect -- " + err);
reject;
}
console.log("Connected!");
resolve();
});
});
}
function deleteExistingMainRow() {
return new promise((resolve, reject) => {
var query = "DELETE FROM my_table";
con.query(query, [],
function(err, result) {
if (err) {
console.log("ERROR in deleteExistingMainRow: " + err);
reject;
}
else {
console.log("DEBUG: Successful delete of main row");
resolve();
}
});
});
}
function writeMainRow(data_row) {
return new promise((resolve, reject) => {
console.log("IN writeMainRow");
var query = 'INSERT INTO my_table SET id = ?';
con.query(query, [data_row.id],
function(err, result) {
console.log("YES we tried to query");
if (err) {
console.log("ERROR in writeMainRow: " + err);
reject(err);
}
else {
console.log("DEBUG: Successful write of main row");
resolve();
}
});
});
}
exports.handler = function(event, context) {
connectToDB().then(function(script) {
deleteExistingMainRow().then(function(script) {
var data = [{ "id": 1 }, { "id": 2 }, { "id": 3 }];
data.forEach(data_row => {
writeMainRow(data_row).then(function(script) {
console.log("DEBUG: Main row written in forEach");
},
function(err) {
if (err) { console.log("ERR"); } process.exit(0);
}());
});
console.log("DEBUG: Hey we're exiting now");
con.commit;
con.end(function(err) {
console.log("Error on con end: " + err);
});
context.done(null, "DONE");
process.exit(0);
});
});
};
Just a few moths ago AWS made Node.js v 8.10 runtime available in lambda.
Which means, you can use async/await and Promises. So, we can rearrange code to something like this:
exports.handler = async (event, context) => {
const dbConnection = await connectToDB();
await deleteExistingMainRow();
const data = [{ "id": 1 }, { "id": 2 }, { "id": 3 }];
// use here for...of loop to keep working with async/await behaviour
for(const data_row of data){
await writeMainRow(data_row);
}
}
Also, you can rewrite your code to use native Promises or async/await functions.
And of course, cover logic on try/catch block, I've skipped them for simplicity.
The reason why your code is not behaving as you expect is because of the asynchronous nature of NodeJS.
Your for_each loop spawns several threads that are going to INSERT the data in your database.
As soon as these threads are started, the rest of the code will execute, starting with console.log("DEBUG: Hey we're exiting now");
So the commit happens before all the INSERT calls are done and, more importantly, you're calling Process.exit() in your code. This terminates the runtime, even before the INSERT can finish.
Call callback() instead as per https://docs.aws.amazon.com/lambda/latest/dg/nodejs-prog-model-handler.html
Handling your multiple asynchronous writes can be done differently. First, as grynets commented before me, I would strongly suggest to rewrite your code using async/await to make the call easier to read.
Then, you have to understand that each call to writeMainRow will return its own Promise and your code must wait for ALL promises to complete before to commit() and to callback()
Promise.all(...) will do that for you. See the doc at https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Promise/all
try using
INSERT INTO table_name(id) VALUES (?);
I know both your query and the above query works the same. Just give it a Try.
And just make sure your for loop is working properly sending values to the writeMainRow(function). It wouldnt show an error you if pass an empty value and make sure you are not passing the same values in the for loop. And i think you have to pass writeMainRow(data_row.id) rather than writeMainRow(data_row).
Hope this helps.
And one more suggestion if you are updating multiple rows there are options in mysql node library like transactions. Using those functions will be more effective and you can roll back the result if you face error. Other option is to write procedures, in which case you mysql server will bear the computation.