I am working on angular app and php as backend that will do the data processing to MySQL. recently i found out node-mysql plugin for Nodejs that will communicate to MySQL through JS.
After the documentations provided there, i have a question which i want to ask here and need some enlightenment from all of you guys.
According to documentation, we will have to declare the connection in JS file as
var mysql = require('mysql');
var connection = mysql.createConnection({
host : 'localhost',
user : 'me',
password : 'secret',
database : 'my_db'
});
connection.connect();
connection.query('SELECT 1 + 1 AS solution', function(err, rows, fields) {
if (err) throw err;
console.log('The solution is: ', rows[0].solution);
});
connection.end();
By providing all the sensitive database login data in JS, will it be a big hole for security issue there? if Yes, how can prevent it?
And the query will be done as well through the JS file
var mysql = require('mysql');
var pool = mysql.createPool(...);
pool.getConnection(function(err, connection) {
// Use the connection
connection.query( 'SELECT something FROM sometable', function(err, rows) {
// And done with the connection.
connection.release();
// Don't use the connection here, it has been returned to the pool.
});
});
Is that mean that the attacker will easily to find out what is the query that we use to query to database? Unlike server side language like PHP where we just call the php file together with the params.
Is it safe to use this driver on Nodejs?
Sorry for being newbie with this such questions.
Node JS is server side too. Node JS using javascript for coding, not mean it will expose to your clients browser. It just seen on server side and stand as backend who give response to client browser request.
For simple explanation, just imagine Node JS as PHP server but in Javascript language and don't need apache server. Of course they have different behavior and many different feature. You better read some tutorial about how Node JS work first and try your self before read advance tutorial.
Related
i have three different Files.
app.js / DataBase.js and the client.html.
In the app.js file i start my server with node.js - express.js and open a websocket. (see below) In Database.js i create a connection with my mysql Database.
I was able to send the data from the database to my client page like this:
db = require('./mysql');
io.on('connection', function (WebSocket) {
WebSocket.on('new_data', function (name) {
WebSocket.emit('data_from_the_database', db.DataexportFromDataBase);
});
});
But now: The user should send the app.js File the varaible "name" which should used in the query from the dataexportFromDataBase.
How i can send the variable to the dataBase file? with socket.io?
i tried:
Include the dataBaseConnection in the app.js. There was no problem and i can handle that, but i want three different files. (Server / Database with the querys / Client). I guess my problem ist here:
WebSocket.emit('data_from_the_database', 'db.DataExportFromDataBase');
Need help :)
I am using express 4.x, and the latest MySQL package for node.
The pattern for a PHP application (which I am most familiar with) is to have some sort of database connection common file that gets included and the connection is automatically closed upon the completion of the script. When implementing it in an express app, it might look something like this:
// includes and such
// ...
var db = require('./lib/db');
app.use(db({
host: 'localhost',
user: 'root',
pass: '',
dbname: 'testdb'
}));
app.get('/', function (req, res) {
req.db.query('SELECT * FROM users', function (err, users) {
res.render('home', {
users: users
});
});
});
Excuse the lack of error handling, this is a primitive example. In any case, my db() function returns middleware that will connect to the database and store the connection object req.db, effectively giving a new object to each request. There are a few problems with this method:
This does not scale at all; database connections (which are expensive) are going to scale linearly with fairly inexpensive requests.
Database connections are not closed automatically and will kill the application if an uncaught error trickles up. You have to either catch it and reconnection (feels like an antipattern) or write more middleware that EVERYTHING must call pior to output to ensure the connection is closed (anti-DRY, arguably)
The next pattern I've seen is to simply open one connection as the app starts.
var mysql = require('mysql');
var connection = mysql.createConnection(config);
connection.on('connect', function () {
// start app.js here
});
Problems with this:
Still does not scale. One connection will easily get clogged with more than just 10-20 requests on my production boxes (1gb-2gb RAM, 3.0ghz quad CPU).
Connections will still timeout after a while, I have to provide an error handler to catch it and reconnection - very kludgy.
My question is, what kind of approach should be taken with handing database connections in an express app? It needs to scale (not infinitely, just within reason), I should not have to manually close in the route/include extra middleware for every path, and I (preferably) to not want to catch timeout errors and reopen them.
Since, you're talk about MySQL in NodeJS, I have to point you to KnexJS! You'll find writing queries is much more fun. The other thing they use is connection pooling, which should solve your problem. It's using a little package called generic-pool-redux which manages things like DB connections.
The idea is you have one place your express app access the DB through code. That code, as it turns out, is using a connection pool to share the load among connections. I initialize mine something like this:
var Knex = require('knex');
Knex.knex = Knex({...}); //set options for DB
In other files
var knex = require('knex').knex;
Now all files that could access the DB are using the same connection pool (set up once at start).
I'm sure there are other connection pool packages out there for Node and MySQL, but I personally recommend KnexJS if you're doing any dynamic or complex SQL queries. Good luck!
What is considered best practice for handling and managing connections when building an API or web application with Node.js that depends on MySQL (or in my case, MariaDB)?
Per the documentation for node-mysql, there seem to be two methods to use:
var connection = mysql.createConnection({...});
app.get("/", function(req, res) {
connection.query("SELECT * FROM ....", function(error, result) {
res.json(result);
});
});
-- or --
var pool = mysql.createPool({...});
app.get("/", function(req, res) {
pool.getConnection(error, connection) {
if (error) {
console.log("Error getting new connection from pool");
} else {
connection.query("SELECT * FROM ....", function(error, result) {
connection.release();
res.json(result);
});
}
});
});
To me, it makes the most sense to use the second option, as it should use as many connections as are needed, as opposed to relying on a single connection. However, I have experienced problems using a pool with multiple routes, i.e each route gets a new connection from the pool, executes a query, and releases it back into the pool. Each time I get a connection from a pool, use it, and release it, it seems there is still a process in MySQL waiting for another request. Eventually, these processes build up in MySQL (visible by running SHOW PROCESSLIST) and the application is no longer able to retrieve a connection from the pool.
I have resorted to using the first method because it works and my application doesn't crash, but it doesn't seem like a robust solution. However, node-mariasql looks promising, but I can't tell if that will be any better than what I am currently using.
My question is: what is the best way to handle/structure MySQL connections when building an API or web application that relies heavily on SQL queries on almost every request?
Changing connection.release() to connection.destory() solved my issue. I'm not sure what the former is supposed to do, but the latter behaves as expected and actually removes the connection. This means once a connection is done being used, it kills the MySQL process and creates another when needed. This also means that many queries can hit the API simultaneously, and slow queries will not block new ones.
Better late then never.
connection.destroy() would mean that on each impression you are making a new connection to mySQL, instead of just grabbing an idle connection and querying on that which would have less overhead. Basically you are not using the pool anymore.
Its possible your mySQL user had a limited number of connections to mysql, or that the number of queries you were making to sql were slower then the number of impressions coming into your server.
You can try tweaking the connectionLimit parameter to something higher, so your server can handle more connections simultaneously.
var pool = mysql.createPool({
connectionLimit : 10,
host : 'example.org',
user : 'bob',
password : 'secret'
});
I have a login system with my NodeJS using mysql-node.
The problem i have how ever is how to keep the user logged in, if they refresh the page they have to login again, because i do not know how to store the session.
My login system is like this:
socket.on('login', function(data,callBack){
var username = sanitize(data['login']).escape(),
pass = sanitize(data['password']).escape();
var query = connection.query('SELECT uid FROM users WHERE name = ? AND pass = ?', [username,pass],
function(err,results){
if(err){
console.log('Oh No! '+err);
} else if(results.length == 1){
//some how set a session here
} else if(!results.length) {
console.log('No rows found!');
}
});
});
I'm having difficulty understanding how i set up a session for each client that connects. Is this possible with NodeJS ?
Reading that they assign express to var app but if i already have this : var app = http.createServer( ... how can i also assign express to it :S bit confusing
You need to understand the difference between a express' server and a native NodeJS' server, here my link comparaison nodejs server vs express server
So you can do:
var app = express();
var server = http.createServer(app);
This enable you to have still the low level functionnaly with NodeJS.
So, if you don't want to use existing modules or framework, you can build your own session manager:
using cookie
using IP/UA
using socket
The best way would be first to implement it with socket, for example:
server.on('connection', function (socket) {
socket.id = id;
});
or
server.on('request', function (req, res) {
req.connection.id = id; // The socket can also be accessed at request.connection.
});
So, you just need to implement a middleware who check the id.
If you want to prevent from session prediction, session sidejacking, etc. you need to combine cookies, ip, socket, and your ideas to make it more secure for your app.
Once you've done your session manager, you can choose where to store the sessions, in a simple object, in redis, in mongodb, in mysql ... (express use MemoryStore by default, but maybe not now)
I don't have an idea if nodejs has core feature of saving sessions. you need to use a database along with it. using Express will help you to utilized a database to persist user sessions. You better study and use it
http://expressjs.com/
http://blog.modulus.io/nodejs-and-express-sessions
I don't think there is any session mechanism within Nodejs' core. However, they are plenty of libraries that would allow you to do it. The first that comes to mind is Connect's session, which is a middleware for Nodejs. Have a look at this question for more details on how to use it.
Have a look at this tutorial from dailyjs which tries to include Express's session into a notepad webapp. The source code is available here. (Note that Express' session is based on Connect's, and is practically the same).
EDIT: Here is a more complete example for Node authentication, using mongoose. They do however show their schemas, so I assume you can easily do the transition to MySQL.
I'm aware of the popularity of a module like node-mysql for connecting to a database from an application, but I can't find any info on the connecting process without using a module like this.
Obviously I could go fishing around the modules themselves for the answer, but is there really no user-case for simple connections with simple queries without module dependency and bloated functionality?
I find it strange given the very simple I/O of a process like MySQL.
This has less to do with node.js and more to do with knowing how to implement the MySql client/server protocol. You simply need to create a tcp connection to the server and send the correct format and sequence of data per the protocol. node-mysql has done the difficult part: abstracting the protocol into something much easier to use.
This is subjective, but looking at the example in https://github.com/felixge/node-mysql
for me looks like simple connection and simple Query
var mysql = require('mysql');
var connection = mysql.createConnection({
host : 'localhost',
user : 'me',
password : 'secret',
});
connection.connect();
connection.query('SELECT 1 + 1 AS solution', function(err, rows, fields) {
if (err) throw err;
console.log('The solution is: ', rows[0].solution);
});
connection.end();
If you have a look to the source code you'll see what it takes to implement the mysql client protocol, I would say that is not that simple
https://github.com/felixge/node-mysql/blob/master/lib/Connection.js
https://github.com/felixge/node-mysql/tree/master/lib/protocol
But again this is something subjective,IMHO I don't think that there is a simpler way to query MySql.