I'm pretty new to nodejs and I'm having some difficulties to understand how to use the mysql connection object.
My problem is not in the code but in the design pattern.
lets say I have a user module
module.exports = function(){
return{
id: "",
load: function(id){
var sql = 'SELECT * from users where id = '+ DB.escape(id);
console.log(1);
DB.query(sql, function (err, rows) {
this.id = rows[0].id; // not working
console.log(rows[0].id); // prints the id 4
console.log(2);
});
console.log(3);
}
}
}
from outside the module i run the next code
var user = require('../modules/user');
var selected_user = user();
console.log("entering users me route");
selected_user.load(4);
console.log("user id is " + selected_user.id); //This does not print the id 4
when I run the code, the console logs 1, then 3, and then 2.
This is due to the asynchronous flow of node js.
But if I'm building a website, and I need the query to end in order to populate my user object before I send the HTML to the browser???
What's the right way to do it ?
Also when I try to populate the id property of user in the id i receive from the DB it does not work.
Any ideas?
Thanks
There are several ways to do this. I would go with Promises.
Suppose you have an asynchronous function "getUsers".
It looks like this:
function getUsers() {
longQuery(function(err, result){
// What to do with result?
});
You need to rewrite it to be able to use the result.
Let's try:
function getUsers() {
return new Promise(function(resolve, reject) {
longQuery(function(err, result){
if(err) reject(err)
else resolve(result)
});
});
Now this function returns a promise. What do we do with that promise?
function handleRequest(req, res) {
getUsers().then(function(result) {
// Do stuff with result
res.send(myProcessedData);
}).catch(function(err) {console.log(err)};
}
This could also have been done with callbacks, passing the response object as a parameter to the query function, and many other ways, but I think promises are a very elegant way for handling this.
this.id = rows[0].id; // not working
The above line is not working because you are setting it to this.id from inside a callback function. When you are inside a callback function this does not mean the this in the main object.
For more discussion about this: see How to access the correct `this` context inside a callback?
To tackle the asynchronous nature of javascript you can either use promise like the answer from matanso or you can pass a callback function to your load method. So your load: function(id) method will be load: function(id, callbackFunction) and call the callback function when you get all the data that you need.
Related
I have an ordinary
var express = require('express')
Node express www page, using session, pug, etc as usual. My db calls
var db = require('./scripts/myHappyMysqlScript')
I'm naturally using mysql, so in the db script
var mysql = require('mysql')
So for example
app.get('/catPhotos', (req, response) => {
response.render('catPhotos.pug');
})
Say a page has a table showing something from the petNames database,
app.get('/pets', function(req, res, next) {
db.allPetNames(function(err, petsList) {
res.render('pets.pug',
{
'petsList': petsList,
'pretty' : true
})
})
all good so far.
But here's a case with three tables on the pug page, and three different database calls:
db.cats(function(err, c) {
db.dogs(function(err, d) {
db.budgies(function(err, b) {
res.render('bigScreen.pug',
{
'cats' : c,
'k9s': d,
'budgies': b,
'pretty' : true
})
})
})
})
I just nest them like that.
This does seem to work perfectly.
It correctly waits sequentially. Errors fall through and are handled properly, and so on.
But is there a better syntax, better way?
What's the Node Way for real™ Node, not-Swift, programmers?!
Perhaps given that I'm using the mysql library, if that's relevant.
Note, one better way overall is to use something like Ajax to just stream in each "part" of the web page. Indeed I do that all the time. What I'm asking here, assuming at res.render I indeed want to return all that info at once, is there something better than nesting like that? Cheers
You can get rid of nested database calls by using promises.
Since you mentioned that you are using mysql library for interacting with the database, unfortunately, this library doesn't provide a promise-based API. So to get rid of nested database calls in your code, you need to create a promise-based wrapper around the callback version of database calls.
For a general overview of what promises are and how they work, see the following links:
MDN - Promise.
MDN - Using Promises
Following is an example of how you can create a promise-based wrapper and then use that wrapper to get rid of nested database calls.
This promise-based wrapper is just a function that returns a promise. It creates a promise instance, wraps the underlying database call, and eventually when the database call returns the data, it notifies your code.
function getCats() {
return new Promise((resolve, reject) => {
// make the database call
db.cats((error, cats) => {
// in case of an error, reject the promise by
// calling "reject" function
// Also pass the "error" object to the "reject" function
// as an argument to get access to the error message
// in the code that calls this "getCats" function
if (error) {
reject(error);
return;
}
// if there was no error, call the "resolve" function
// to resolve the promise. Promise will be resolved
// in case of a successful database call
// Also pass the data to the "resolve" function
// to access this data in the code that calls this
// "getCats" function
resolve(cats);
});
});
}
Now in your route handler function, instead of calling db.cats(...), call this getCats wrapper function.
There are two ways you can call the function that returns a promise:
Promise-chaining (For details, visit the links mentioned above)
async-await syntax (Recommended)
The following code example uses async-await syntax. For this, first, mark the route handler function as async by using the async keyword before the function keyword. By doing this, we can use the await keyword inside this route handler function.
app.get('/pets', async function(req, res, next) {
try {
const cats = await getCats();
// similar wrappers for other database calls
const dogs = await getDogs();
const budgies = await getBudgies();
// render the pub template, passing in the data
// fetched from the database
...
catch (error) {
// catch block will be invoked if the promise returned by
// the promise-based wrapper function is rejected
// handle the error appropriately
}
});
The above code example only shows how to wrap the db.cats(...) database call in a promise-based wrapper and use that wrapper to get the data from the database. Similarly, you can create wrappers for db.dogs(...) and db.budgies(...) calls.
Instead of creating a separate promise-based wrapper for each database call, ideally, you should create a re-usable promise-based wrapper function that takes in a function to call and wraps that function call in a promise just like shown in the above code example, i.e. getCats function.
Parallel Database calls
One important thing to note in the above code is the route handler function
const cats = await getCats();
const dogs = await getDogs();
const budgies = await getBudgies();
is that this will lead to sequential database calls which may or may not be what you want.
If these database calls do not depend on each other, then you can call the promise-based wrappers in parallel using Promise.all() method.
The following code example shows how you can call your promise-based wrapper functions in parallel using Promise.all().
app.get('/pets', async function(req, res, next) {
try {
// "petsData" will be an array that will contain all the data from
// three database calls.
const petsData = await Promise.all([getCats(), getDogs(), getBudgies()]);
// render the pub template, passing in the data
// fetched from the database
...
catch (error) {
...
}
});
I hope this is enough to help you get rid of the nested database calls in your current code and start using promises in your code.
If you're trying to use MySQL with Nodejs, the module you should be looking for is mysql2 rather than mysql.
mysql2 provides a promise based approach and is a much refined version of mysql module for nodejs.
For example, for executing a query,
in mysql
con.query(sql_query, (err, rows, field)=>{ //some code here }
in mysql2, you can use the async approach as well as promise approach. Also, prepared statements in mysql2 are more easier than mysql.
//async approach
class A {
static async fn(sql, params){
const [data] = await con.execute(sql, [params]);
return data;
}
}
//promise approach remains same as **mysql** itself.
Here's the documentation for
mysql2 & more docs
If your database calls returned promises instead of using callbacks, you could:
const cats = await db.cats();
const dogs = await db.dogs();
const budgies = await db.budgies();
res.render('bigScreen.pug', {
cats : cats,
k9s: dogs,
budgies: budgies,
pretty : true
});
// Or request them all in parallel instead of waiting for each to finish
const [
cats,
dogs,
budgies
] = Promise.all([
dg.cats(),
dg.dogs(),
db.budgies()
]);
Simply convert the mysql functions into promises using the nodejs standard lib util.promisify
example:
const { promisify } = require('util');
const catsPromise = promisify(db.cats);
const dogsPromise = promisify(db.dogs);
const budgiesPromise = promisify(db.budgies);
async function routeHandler() {
let err = null;
try {
const cats = await catsPromise();
const dogs = await dogsPromise();
const budgies = await budgiesPromise();
} catch(error) {
err = error;
}
if (err) {
console.log(err);
// you should res.end() or res.render(someErrorPage) here
// failure to do so will leave the request open
} else {
res.render('bigScreen.pug', {
'cats' : cats,
'k9s': dogs,
'budgies': budgies,
'pretty' : true
});
}
}
Promise.all() method seems a more famous and cleaner way to make multiple calls in parallel like your use case.
But there is one more alternate way. : Multiple statement queries
To use this feature you have to enable it for your connection:
var connection = mysql.createConnection({multipleStatements: true});
Once enabled, you can execute multiple statement queries like any other query:
db.query('SELECT cats; SELECT dogs', function (error, results, fields) {
if (error) throw error;
// `results` is an array with one element for every statement in the query:
console.log(results[0]); // [{cat1,cat2}]
console.log(results[1]); // [{dog1,dog2}]
});
It is technically more efficient as requires less back and forth with MySQL connection.
(However, this feature is disabled by default as it allows for SQL injection attacks if values are not properly escaped). To use this feature you have to enable it for your connection.)
What I am trying to do is create a chrome extension that creates new, nested, bookmark folders, using promises.
The function to do this is chrome.bookmarks.create(). However I cannot just
loop this function, because chrome.bookmarks.create is asynchronous. I need to wait until the folder is created, and get its new ID, before going on to its children.
Promises seem to be the way to go. Unfortunately I cannot find a minimal working example using an asynchronous call with its own callback like chrome.bookmarks.create.
I have read some tutorials 1, 2, 3, 4. I have searched stackOverflow but all the questions do not seem to be about plain vanilla promises with the chrome extension library.
I do not want to use a plugin or library: no node.js or jquery or Q or whatever.
I have tried following the examples in the tutorials but many things do not make sense. For example, the tutorial states:
The promise constructor takes one argument—a callback with two
parameters: resolve and reject.
But then I see examples like this:
const wait = ms => new Promise(resolve => setTimeout(resolve, ms));
How this works is a mystery to me.
Also, how can you call resolve() when its never been defined? No example in the tutorials seem to match real life code. Another example is:
function isUserTooYoung(id) {
return openDatabase() // returns a promise
.then(function(col) {return find(col, {'id': id});})
How do I pass in col, or get any results!
So if anyone can give me a minimal working example of promises with an asynchronous function with its own callback, it would be greatly appreciated.
SO wants code, so here is my non-working attempt:
//loop through all
function createBookmarks(nodes, parentid){
var jlen = nodes.length;
var i;
var node;
for(var i = 0; i < nodes.length; i++){
var node = nodes[i];
createBookmark(node, parentid);
}
}
//singular create
function createBookmark(node, parentid){
var bookmark = {
parentId : parentid,
index : node['index'],
title : node['title'],
url : node['url']
}
var callback = function(result){
console.log("creation callback happened.");
return result.id; //pass ID to the callback, too
}
var promise = new Promise(function(resolve, reject) {
var newid = chrome.bookmarks.create(bookmark, callback)
if (newid){
console.log("Creating children with new id: " + newid);
resolve( createBookmarks(bookmark.children, newid));
}
});
}
//allnodes already exists
createBookmarks(allnodes[0],"0");
Just doesn't work. The result from the callback is always undefined, which it should be, and I do not see how a promise object changes anything. I am equally mystified when I try to use promise.then().
var newid = promise.then( //wait for a response?
function(result){
return chrome.bookmarks.create(bookmark, callback);
}
).catch(function(error){
console.log("error " + error);
});
if (node.children) createBookmarks(node.children, newid);
Again, newid is always undefined, because of course bookmarks.create() is asynchronous.
Thank you for any help you can offer.
Honestly, you should just use the web extension polyfill. Manually promisifying the chrome APIs is a waste of time and error prone.
If you're absolutely insistent, this is an example of how you'd promisify chrome.bookmarks.create. For other chrome.* APIs, you also have to reject the callback's error argument.
function createBookmark(bookmark) {
return new Promise(function(resolve, reject) {
try {
chrome.bookmarks.create(bookmark, function (result) {
if (chrome.runtime.lastError) reject(chrome.runtime.lastError)
else resolve(result)
})
} catch (error) {
reject(error)
}
})
}
createBookmark({})
.then(function (result) {
console.log(result)
}).catch(function (error) {
console.log(error)
})
To create multiple bookmarks, you could then:
function createBookmarks(bookmarks) {
return Promise.all(
bookmarks.map(function (bookmark) {
return createBookmark(bookmark)
})
)
}
createBookmarks([{}, {}, {}, {}])
.catch(function (error) {
console.log(error)
})
Take the advantage of the convention that the callback function always be the last argument, I use a simple helper function to promisify the chrome API:
function toPromise(api) {
return (...args) => {
return new Promise((resolve) => {
api(...args, resolve);
});
};
}
and use it like:
toPromise(chrome.bookmarks.create)(bookmark).then(...);
In my use case, it just works most of the time.
I have the following code structure in exports.js
module.exports = {
getData:function(param1,param2,callback){
sql.query('SELECT users FROM table',function(error,result){
callback(null,result[0]);
});
}
}
And I called it from main file app.js file like
var common = require('./exports');
console.log(common.getData(null,null));
I got the following error
TypeError: callback is not a function
However I found a similar question here. But didn't fixed the problem. Any help would be appreciated..!
if you want print results
var common = require('./exports');
common.getData(null, null, function(err, result){
console.log(result)l
})
getData() accepts 3 arguments
The last of which should be a function which you have not provided in your example. Add a callback function in your call like so:
common.getData(null,null, function(err, result) {
console.log(result)
});
The sql.query function uses the callback function you pass in as the third parameter and thats why your getting the error to say it's missing.
So I'm selecting Activities from the mongodb and populating User for each.
var query = Activity.find(query).populate("user");
return query.sort({created:"desc"}).exec(function(err, activities) {
debugger;
if (!err) {
return res.json(activities);
} else {
res.status(400).json(err);
}
});
As you can see I have a debugger; breakpoint is there, When I'm pring activities it prints an array of activities with the user object populated.
Also when I'm calling something like activities[0].toJSON() I get everything good!
But the response comes back with the user property empty !
I looked into the source of express.response.json(OBJ) and saw this line:
var body = JSON.stringify(val, replacer, spaces);
val is my activities
When calling JSON.stringify(activities) it will create a json with an empty user field.. any suggestions ?
Try the lean option. That gives back plain JS objects with no mongoose weirdness. Also, your error handling seems a little awkward, can be simplified.
var query = Activity.find(query).populate("user");
query.sort({created:"desc"}).lean().exec(function(err, activities) {
if (err) return res.status(400).json(err);
res.json(activities);
});
I would go even further, not hard-coding error sending in routes but simply passing along via if (err) return next(err) to error-handling middleware defined elsewhere in your app. You can still set the status, then use detection in your middleware, something like this:
app.use(function(err, req, res, next){
err.status = err.status || 500;
res.status(err.status).json(err);
});
I'm very new to nodejs and have a question.
Trying to create a function that will call the value of any field where I mention its ID from a table:
function getUserInfo (userID, dynamicField) {
var query = connection.query('SELECT '+dynamicField+' from users WHERE userID = '+connection.escape(userID));
query.on('result', function(row) {
return(row.dynamicField);
});
};
console.log(getUserInfo(8, userEmail)) //this should get me the userEmail value of the user with userID=8
However, I get "undefined". If I use console.log rather than return, it logs the value but this has no use as a function to be used inside other functions to get a value.
I will be glad if I can get help for modifying the function.
This is a common mistake amongst async/nodejs beginners. You have essentially wrapped an async function inside a sync function which breaks down the nature of node's event loop. The return expression needs to be replaced with a callback. See below:
// Method
function getUserInfo (userID, dynamicField, callback) {
var query = connection.query('SELECT '+dynamicField+' from users WHERE userID = '+connection.escape(userID));
query.on('result', function(row) {
callback(null, row.dynamicField);
});
};
// Implementation
getUserInfo(8, userEmail, function(err, result){
console.log(err || result);
});
By convention, in Nodejs we always pass an error object first in the callback. In this case since there is no error to capture, we pass null in its place.