In Laravel, you can do something like
$object = Cache->remember(key, duration, function() {
$result = mysql_fetch_something();// retrieve from MySQL here
return $result;
});
where basically, Laravel checks the cache first if it exists there, and if not, it allows you to retrieve the value from the database and automatically put it in cache while also returning it. Is there a similar construct in node; that is, a 1 stop cache check, db failover mechanism?
In node there is no special command for this but you can build it yourself.
Just check with the redis command EXISTS whether the key is in redis and if not just check mysql and store it.
You can do some thing like this. in cache.js
var isCacheAvailable = true;
exports.init = function () {
var server = config.get('Cache.server');
var port = config.get('Cache.port');
client = redis.createClient(port,server);
// handle redis connection temporarily going down without app crashing
client.on("error", function (err) {
logger.error("Error connecting to redis server " + server + ":" + port, err);
isCacheAvailable = false;
});
}
exports.isCacheAvailable = function(){
return isCacheAvailable;
}
Check the isCacheAvailable() function where you intend to use cache.
if(cache.isCacheAvailable()) {
// use cache to fetch data
} else {
// fallback to mysql db
}
Hope this helps.
Related
There is a field called counter in a model and whenever if I call a custom remote method like
Request URL
http://loopback:3000/api/models/increment_counter
Request body
EMPTY
Response
{
"counter" : [Value after Increment]
}
Currently to increment First i have to get the counter value from db and increment it one and update the couter value, This involves two queries, Is it possible to do it in a single NodeAPI call like the below mysql query.I am currently using Mysql as DB.
mysql_query("
UPDATE model
SET counter = counter + 1
WHERE model_id = '1'
");
Thank you
Given the MySQL syntax you want, you seem to need an atomic counter.
database transactions
With the MySQL connector, you can use database transactions. It is supported by the MySQL connector. It's a bit overkill for just atomic counters but it will get the job done.
Here is an example, inside a custom remote method
MyModel.someRemoteMethodForIncrementing = function(callback) {
// Start the transaction
MyModel.beginTransaction({
isolationLevel: MyModel.Transaction.READ_COMMITTED
}, function(err, tx) {
// Retrieve the current counter value as part of the transaction
MyModel.findById(id, {
transaction: tx
}, function(err, data) {
if (err) {
console.log(err);
return tx.rollback(function(err) {
callback(err);
});
}
// Increment the counter as part of the transaction
var inc = data.counter + 1;
MyModel.updateAttributes({
counter: inc
}, {
transaction: tx
}, function(err, newData) {
if (err) {
console.log(err);
return tx.rollback(function(err) {
callback(err);
});
}
// Commit the transaction to make it happen
tx.commit(function(err) {
if (err) return callback(err);
// Counter should have been incremented
callback();
});
});
});
});
};
I haven't tested it but it should work, let me know
extented operators
In the future, you will be able to use the $inc (increment) extended operator but so far it's only supported by MongoDB connector, not MySQL.
Just for reference, here is the syntax (works only with MongoDB)
PUT api/Model/:id?filter={"$inc":{"name":propertyToIncrement, "count":incrementAmount}}
There is an ongoing PR that I am trying to get landed to get MySQL support, but there are many things to be done before it can get merged.
Yes you can do it in a single loopback remote method call. Assume you are sending an Id in your request
yourModel.remoteMethod = function(data, cb){
yourModel.findById(data.id, function(err, object){
if(err){
cb(err, null);
}
object.counter += 1;
object.save(function(saveErr, afterSaveObj){
cb(saveErr, afterSaveObj.counter);
})
});
}
here cb is a callback which loopback passes to your remoteMethod.
Print haphazardly and not wait for MySQL send a reply.
var mysql=require('mysql').createConnection({
host:'localhost',user:'root',password:'',database:'dbname',port:3306
});
mysql.connect(function(error){if(error){throw error;}else{console.log('MYSQL SERVER [OK]');}});
console.log('// INI //');
var sql='SELECT * FROM sys_users';
mysql.query(sql,function(error,rows){
if(!error){
for(i=0;i<rows.length;i++){
if(i<rows.length-1)
console.log(rows[i].username);
else console.log(rows[i].username);
}
}else console.log('Error');
});
console.log('// END //');
mysql.end();
would have to print:
MYSQL SERVER [OK]
// INI //
List item
// END //
but nevertheless printed:
// INI //
// END //
MYSQL SERVER [OK]
List item
That's the nature of Node.js - callbacks and async code processing.
If you want to run come code after the database response will be returned, place that code inside the callback:
mysql.query(sql,function(error,rows){
if (!error){
for(i=0;i<rows.length;i++){
if(i<rows.length-1)
console.log(rows[i].username);
else
console.log(rows[i].username);
}
} else {
console.log('Error');
}
/* Code to run. Move it to first `if`, if you want to run it only, when no error will occur. */
console.log('//END //');
});
mysql.end();
i have simple web sockets html5 , when the server is up every thing is working fine
the problem is when i close the server ( for testing )
im getting :
WebSocket connection to 'ws://127.0.0.1:7777/api' failed: Error in connection establishment: net::ERR_CONNECTION_REFUSED
which i unable to catch its never jumps to onerror or onclose in case of this error
init: function () {
this.m_wsiSendBinary = new WebSocket("ws://127.0.0.1:7681/wsapi");
this.m_wsiSendBinary.onopen = function(evt) {
cc.log("Send Binary WS was opened.");
};
this.m_wsiSendBinary.onmessage = (function(evt) {
this.handleServerResponse(yStr);
this.m_wsiSendBinary.onerror = function(evt) {
};
this.m_wsiSendBinary.onclose = function(evt) {
cc.log("m_wsiSendBinary websocket instance closed.");
self.m_wsiSendBinary = null;
};
}).bind(this);
I do not have full answer, however I dealt with similar issue and have a partial and not so elegant solution (but may help someone). Unfortunately without the elimination of the error message.
Two business requirements:
BR1 - Handle state in initialization when the server is not available.
BR2 - Handle state when the server stops.
Solution for BR1
var global_connection_openned=null;//Here you have the result
init: function () {
global_connection_openned=false;
this.m_wsiSendBinary = new WebSocket("ws://127.0.0.1:7681/wsapi");
this.m_wsiSendBinary.onopen = function(evt)
{
global_connection_openned=true;
};
Solution for BR2 (assumes the BR1)
//somewhere in your project called by setInterval(..) which will detect the connection is lost (and tries to reestablish/reopen the connetion.
{
if (this.m_wsiSendBinary==null || this.m_wsiSendBinary.readyState==3)
this.init();
if (!global_connection_openned)
this.m_wsiSendBinary=null;
}
Anyway, I would be really curious if there is solid and proper solution of this use case.
I'm working on creating a user registration system for a website that I am working on but I am running into a few issues.
I'm trying to stay away from having to nest callbacks because it gets kind of messy, What I need help with is finding if there is a way to create synchronous queries with node-mysql
Here's what I'm trying to achieve.
connection.query("select 1 as email from users where email = " + connection.escape(email), function(err, rows, fields) {
if(err) {
var error = {
error_message: err.code,
error_number: err.errno
};
return res.send(error);
}
if(rows.length > 0) {
var error = {
message: 'Email Address is Taken',
code: 2
};
return res.send(error);
}
});
connection.query("insert into users (email, password) values ("+connection.escape(email)+", "+connection.escape(hash)+")", function(err, rows, fields) {
if(err) {
var error = {
error_message: err.code,
error_number: err.errno
};
return res.send(error);
}
});
My goal is to have the first query run and if that returns a row then to not execute the second query but if the first query returns 0 rows then continue and run the second query.
I know I can nest the second query inside the first query and put if in an else but that's what I don't want to do because while I have those two queries I also have it set u to use bcrypt to encrypt the password which would have to be nested as well.
Is there a way to write it so that I don't need to nest the two queries or is nesting them going to be my only option?
You could simply use a module for node that provide synchronous functions.
Here you'll find a module that provide sync/async functions to deal with mysql.
https://github.com/Will-I4M/node-mysql-libmysqlclient
Here is how you could use it in order to execute a synchronous query :
var config = require("./config.json") ;
var mysql = require('mysql-libmysqlclient') ;
var client = mysql.createConnectionSync(config.host, config.user, config.password, config.database) ;
var query = "SELECT * FROM Users ;" ;
var handle = client.querySync(query) ;
var results = handle.fetchAllSync() ;
console.log(JSON.stringify(results)) ;
As jfriend00 said above, if you're going to develop in node.js, then you MUST become comfortable with writing async code.
"chained promises" is probably your best bet:
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Promise/then
http://html5hive.org/node-js-quickies-working-with-mysql/
ADDENDUM:
This tutorial illustrates promise chaining with node.js SQL queries. It also discusses how you can use Q and/or Step to simplify your code:
http://code.tutsplus.com/tutorials/managing-the-asynchronous-nature-of-nodejs--net-36183
There could be conditions when you need sync queries (or at least for readability or simplicity). I do not agree with that everything have to be done in the async way at node.js.
I have tested a lot of available solutions and ended up with the "sync-mysql" module (https://github.com/ForbesLindesay/sync-mysql).
Easy to install and use, but not that good in performance (especially if you have to do a lot of sub-queries).
People talk about chained promises here, but give no example code. Here's what we did in a training session today to run a sequence of SQL statements synchronously using promises (credits to trainer and trainees), no additional libraries required:
let mysql = require("mysql");
let conn = mysql.createConnection({host: "localhost", user: "app",
password: "*******", database: "people"});
//returns a promise that resolves to a result set on success
function execSql(statement, values) {
let p = new Promise(function (res, rej) {
conn.query(statement, values, function (err, result) {
if (err) rej(err);
else res(result);
});
});
return p;
}
function insertUserAndFriend(user, friend) {
execSql("INSERT INTO usr (nam) VALUES (?);",[user])
.then(function (result) {
console.log("Inserted " + user);
return execSql("SELECT id, nam from usr where nam = ?;", [user]);
})
.then((result) => {
let { id, nam } = result[0];
console.log("Result: " + id + " " + nam);
return execSql("INSERT INTO friend (usr,nam) VALUES (?,?);",
[id, friend]);
})
.then((result) => {
console.log("Inserted " + friend);
})
.catch((err) => {
console.log("Error: " + err);
})
.finally(function (res) {
conn.end();
});
}
conn.connect(function (err) {
if (err) throw err;
insertUserAndFriend("Bonnie", "Clyde");
});
For reference, here is the create.sql of the toy database:
DROP TABLE IF EXISTS friend;
DROP TABLE IF EXISTS usr;
CREATE TABLE usr (
id INT unsigned NOT NULL AUTO_INCREMENT,
nam VARCHAR(50) UNIQUE NOT NULL,
PRIMARY KEY (id)
);
CREATE TABLE friend (
usr INT unsigned NOT NULL,
FOREIGN KEY (usr) REFERENCES usr (id),
nam VARCHAR(50) UNIQUE NOT NULL
);
For most things I code in node.js, I like asynchronous code. However, I completely understand that asynchronous code is extremely and dangerously incompatible with the need to write and maintain business logic. I've used a variety of alternative methods. The modules to make things synchronous still leave you with data scoping issues that complicate things. Promises worked best for me. Using that approach, I found myself practically writing an interpreter for a new language on top of JavaScript. I may seem absurd but the most practical and safest method for me ended up being to use the shelljs module and the mysql shell client. It's not great execution performance but it makes for much better developer performance and keeps business logic clear and orderly, as is crucial for business logic. Here's snippet of code to give an example of some of what I created:
var shell = require('shelljs');
module.exports = {
user: '',
password: '',
runSql: function (sql) {
var command = "echo '" + sql.replace(/'/g, "'\\''") + "' | mysql -u" + this.user.replace(/'/g, "'\\''") + " -p'" + this.password.replace(/'/g, "'\\''") + "'";
var raw = shell.exec(command, {silent: true}).stdout;
//console.log( 'BASH -> MySQL YIELD: "' + raw + '"' );
if (raw.substr(0, 5) === 'ERROR') {
console.log('ERROR Resulting from: ' + sql + '\n' + raw);
return [];
}
var rows = raw.split('\n');
var names = [];
for (var r = 0; r < rows.length; r += 1) {
columns = rows[r].split('\t');
// Capture column names
if (r === 0) {
names = columns;
continue;
}
// Reformat row into named valued
var fields = {};
for (var c = 0; c < columns.length; c += 1) {
fields[names[c]] = columns[c];
}
rows[r] = fields;
}
// Eliminate extraneous first and last rows
rows.splice(0, 1);
rows.splice(rows.length - 1, 1);
return rows;
},
}
Symplest solution I could find is the sync-sql module.
Install the required modules
npm install sync-sql
npm install sync-mysql
Sample index.js
const Mysql = require('sync-mysql')
const connection = new Mysql({
host:'localhost',
user:'root',
password:'password',
database:'demo'
})
var result = connection.query('SELECT NOW()')
console.log(result)
https://www.geeksforgeeks.org/how-to-run-synchronous-queries-using-sync-sql-module-in-node-js/
I know I am late to this party but I feel I can help people like me that needed a way to use MySQL in a synchronous way.
Answer is here.
Oh and I had to add a pool.end(); after my query code to close the connection and stop the infinite wait loop. See here.
We have a node.js app that uses node_msyql, a great little library for accessing MySQL databases.
Unfortunately, if our connection is not used for maybe 8-10 hours, the next time we try to run a query, we get a "No database selected" error back from the server. We need to add a "USE db" somewhere, but I can't figure out where.
Now, it makes sense to me that connections would go stale, and it seems as though node_mysql is refreshing those stale connections, but there doesn't seem to be a way to make sure that the right db is connected. I was looking for a .connected() callback or event or something that would let me make sure the correct DB was alway USE'd, but no luck so far.
Any suggestions how to do this?
Ys, client tries to reconnect. You can try to query 'use db' on reconnect using code like this:
client._connection.on('connect', function() { client.query('use db'); })
This is where reconnection happen in the node-mysql ('end' handler):
https://github.com/felixge/node-mysql/blob/master/lib/mysql/client.js
var connection = self._connection = new Stream(),
parser = self._parser = new Parser();
connection
.on('error', function(err) {
var connectionError = err.code && err.code.match(/ECONNREFUSED|ENOTFOUND/);
if (connectionError) {
if (cb) {
cb(err);
return;
}
}
self.emit('error', err);
})
.on('data', function(b) {
parser.write(b);
})
.on('end', function() {
if (self.ending) {
self.connected = false;
self.ending = false;
return;
}
if (!self.connected) {
return;
}
self.connected = false;
self._prequeue(connect);
});
connection.connect(self.port, self.host);
because of node-mysql update following code maybe work:
client._socket.on('connect', function() {
console.log("use ",DB_NAME);
client.query('use '+DB_NAME);
});
When using the node-mysql-promise package, you may want to use this code to do the same:
dbConn.pool.on('connection', function() {
dbConn.pool.query("USE myDBname");
} );