In an app with socket.io and node-mysql module, what's the best way (most optimized and well designed) to use mysql connections when an event is triggered?
create one connection and pass it to the callback
create a connection every time the callback is called
Let's write some code to better explain
Case 1
var connection = mysql.createConnection({
host : 'example.org',
user : 'bob',
password : 'secret',
});
io.sockets.on('connection', function cb1(){
socket.on('event', function cb2(){
connection.connect(function cb3(){
connection.query();
});
});
});
Case 2
io.sockets.on('connection', function cb1(){
socket.on('event', function cb2(){
var connection = mysql.createConnection({
host : 'example.org',
user : 'bob',
password : 'secret',
});
connection.connect(function cb3(){
connection.query();
});
});
});
My server listen for several socket events and I'm experiencing a lot of timeout errors! I think it could be related to the use of the connections.
Any suggestions?
I think the best would be to have a connection opened all the time and run just query inside socket.io event. Like this:
var connection = mysql.createConnection({
host : 'example.org',
user : 'bob',
password : 'secret',
});
connection.connect(function cb3(){
io.sockets.on('connection', function cb1(){
socket.on('event', function cb2(){
connection.query();
});
});
});
I would look at connection pooling, which is implemented by node-mysql. This has the advantage of being able to use multiple connections to your database, while keeping the total number of connections limited to prevent flooding the database server.
Another possibility could be node-mysql-queues, which can be used to prevent possible issues with multiple queries run through a single connection.
Related
This question already has answers here:
MySQL 8.0 - Client does not support authentication protocol requested by server; consider upgrading MySQL client
(32 answers)
Closed 3 years ago.
I am trying to connect node js to MySQL database which is up and running according to the MySQL Workbench. However the usual create connection code template isn't working.
Any ideas as to why this is?
I have tried putting a function and error around the connection.connect() part to see if it actually connects. But the terminal window comes up with the same error nonetheless:
Marcs-MacBook-Pro:npm-global marcwatts$ node index.js
Error while performing Query.
Do we think that it is connecting ok or does it not connect in the first place?
var mysql = require('mysql');
var connection = mysql.createConnection({
host : 'localhost',
user : 'root',
password : '<hidden from stack overflow>',
database : 'songdata'
});
connection.connect();
connection.query('SELECT * from songdata2', function(err, rows, fields) {
if (!err)
console.log('The solution is: ', rows);
else
console.log('Error while performing Query.');
});
connection.end();
I just want to connect so I can learn how to write the code which takes each song and places its lyrics inside the Watson Tone analyzer and outputs the results in a new table in MySQL. Any advice on this part would be much appreciated too as I am new to web apps!
The recommended way to establish a connection is this:
var mysql = require('mysql');
var connection = mysql.createConnection({
host : 'example.org',
user : 'bob',
password : 'secret'
});
connection.connect(function(err) {
if (err) {
console.error('error connecting: ' + err.stack);
return;
}
console.log('connected as id ' + connection.threadId);
});
I have two node microservices talking to one common mysql database. Both microservices have this below code to create a connection pool with a connectionlimit of 10 as follows:
// Initializing pool
var pool = mysql.createPool({
connectionLimit: 10,
host: 'localhost',
port: '3306',
user: 'root',
password: 'root'
});
function addConnection(req, res) {
pool.getConnection(function (err, connection) {
if (err) {
connection.release();
res.json({ "code": 500, "status": "Error" });
return;
}
connection.query("select * from user", function (err, rows) {
connection.release();
if (!err) {
res.json(rows);
}
});
connection.on('error', function (err) {
res.json({ "code": 500, "status": "Error" });
return;
});
});
}
For mysql database I have the max_connections set to 200(SHOW VARIABLES LIKE 'max_connections'; returns 200).
With the pool connectionLimit set to 10 for each of the microservice, in which cases or scenarios will the number of connections for any of the microservice will go above 10?
i.e. When and how the node services would be able to maintain more connections then expected?
If I have 3 instances running of same microservice then how does the pool connectionLimit works? What would be the limit of connections for each instance of microservice?
In one of the microservice say I have two apis which does database transactions and both connects to the database(getting connection) through two different functions having
same implementation of mysql.createPool({}) as above. What would happen if both apis are called concurrently and the number of requests made for each of them per second is 100 or more?
Will the number of connections made available be 10 or 20(since there are two mysql pools created with a connectionLimit of 10 each)?
Ideally it would not; but it can go above 10; if suppose some connections become stale i.e. they are closed from client end but are still open on Server end.
If you have multiple instances of same micro-service deployed in multiple VM or docker containers; they are like independent services.. and they have no connection among each other.. Hence, each of them will create its own 10 connection.
Firstly if u set connection pool limit as 10; that does NOT mean that during first moment 10 connections would be created.. While creating a pool; you also specify initial connection parameter suppose 5.. so, when service starts only 5 connections would be created.. and more created only when needed.. with UPPER Limit set as defined by parameter max_connections. Coming back to your question; well if you have NOT implemented synchronization etc. properly then yes it is possible that both pools will initialize their INITIAL_CONNECTIONS..
Thanks in advance for your help!
I am making a node/express app and I want to use a mysql database. But I can't seem to connect to the database.
I know I'm supposed to use the node-mysql module (https://github.com/felixge/node-mysql), but I must be doing it wrong.
I'm completely new to this. Do I have to create a database and then create a table? Is there a way to create a database elsewhere so it doesn't get erased every time I restart the app?
Here's my code. Can someone answer the questions above and tell me what I'm doing wrong below? Thanks!
var express = require('express'),
routes = require('./routes'),
user = require('./routes/user'),
http = require('http'),
io = require('socket.io'),
path = require('path');
var app = express();
var server = http.createServer(app);
sio = io.listen(server);
app.configure(function(){
app.set('port', process.env.PORT || 3000);
app.set('views', __dirname + '/views');
app.set('view engine', 'jade');
app.use(express.favicon());
app.use(express.logger('dev'));
app.use(express.bodyParser());
app.use(express.methodOverride());
app.use(app.router);
app.use(require('stylus').middleware(__dirname + '/public'));
app.use(express.static(path.join(__dirname, 'public')));
});
app.configure('development', function(){
app.use(express.errorHandler());
});
server.listen(app.get('port'), function(){
console.log("Express server listening on port " + app.get('port'));
});
var mysql = require('mysql');
var connection = mysql.createConnection({
user : 'me',
password : 'secret',
host : 'localhost',
port : app.get('port')
});
connection.connect();
connection.query('CREATE TABLE tesTable (integer int, textfield VARCHAR(100), PRIMARY KEY(integer))',
function(err, result){
if(err) {
console.log(err);
} else {
console.log("Table testTable Created");
}
});
By the way, in package.json, I listed
"mysql": "2.0.0-rc2"
as a dependency and did
'npm install'
You can just create your tables in the database as you normally do and it will be persistent enough not to be deleted anytime you restart your app. You can connect to your database by using the following code if you want:
var mysql = require('mysql');
app.use( connection(mysql, {
host: 'myhost',
user: 'user_name',
password: 'password',
port: 3306, //port mysql
database: 'database_name',
multipleStatements: 'true' //false by default
}, 'pool'));
req.getConnection(function(err, connection) {
connection.query("SELECT * FROM `table_name`;",function (error,row){
if(!error){
//do something.....
}
else console.log("Error : "+err);
});
//do something else...
});
My first suggestion, besides the question about any errors, is that you should try this
var mysql = require('mysql');
var connection = mysql.createConnection({
user : 'me',
password : 'secret',
host : 'localhost',
port : your_MySQL_port
});
The port: app.get('port') in your given example returns your http server port, but not the port of your MySQL server.
Check https://github.com/felixge/node-mysql#connection-options at 'port'.
To get your MySQL port to insert in your_MySQL_port on Linux or Mac OS, just open a terminal an type:
ps ax | grep mysqld
as result you will see something like --port=1234 in the generated output. In this case 1234 is your_MySQL_port.
In this exmaple your code should look like:
var mysql = require('mysql');
var connection = mysql.createConnection({
user : 'me',
password : 'secret',
host : 'localhost',
port : 1234
});
You might want to read a few tutorials on MySQL in general before jumping into it with Node, but in case you feel adventurous, here's a very brief tutorial on how I'd go about setting up MySQL on my machine.
I am assuming that you have access to the MySQL shell. If you're using a frontend the process may be different.
After installing MySQL and configuring an account, you'd want to log into the MySQL shell with the command
mysql -u me -p
followed by the password for the MySQL user when prompted.
Next, you'll want to create a database. You'd do this with the command
create database mydatabase;
then
use mydatabase;
Next you'll want to create a table. Run the CREATE TABLE query in the shell to set up a table. You could do this in Node, but then you'd be running the command needlessly every time you started the app.
Now you should be able to connect using node-mysql
var connection = mysql.createConnection({
user : 'me',
password : 'secret',
host : 'localhost',
database : 'mydatabase',
port : 3306 // or whatever your mysql port is
});
I was facing same issue so how i solve it...
Mysql Part
1-I have already installed wamp and can access to phpmyadmin.
2-Using phpmyadmin i have created a database and a table say it users(please insert 2,3 rows data using insert menu option) look like
check
> http://prntscr.com/72gsxc
3-Make sure wamp is running and mysql service is also running.
4-Click wamp and mouseover MYSQL and then click my.ini and search there port by default it is 3306.
Node Part
var pool = mysql.createPool({
host : 'localhost',
user : 'root',
password : '',
database : 'cricstream_2',
port : 3306
});
var readTable = 'SELECT * FROM countries';
pool.getConnection(function(error, connection){
connection.query(readTable, function(err, rows){
if(err) throw err;
else {
console.log(rows);
}
});
});
Do I have to create a database and then create a table?
Yes, you should create the database and the table ahead of time, that is correct.
Is there a way to create a database elsewhere so it doesn't get erased every time I restart the app?
Yes, you typically install MySQL server on a computer, this installation manages a run of databases, which you can remove/backup/duplicate etc, and create new databases. Your NodeJS app would just be making a connection to your MySQL server installation.
I'm trying to figure out what the best time to actually initialize connections for mysql in node is.
Am I supposed to create a pool of connections and then set them to some global so that all my models have access to the pool? Or am I supposed to initialize connections whenever I'm doing queries?(Seems bad).
I'm sure there's some "proper" way to do it, but I'm not really certain what the best way is.
If you are going to pool connections, then don't initialize connections right when they're needed. When not using a pool, you can just store connection information when your application is starting up, and use it when you need it:
var mysql = require('mysql');
var connection = mysql.createConnection({
host: 'localhost',
user: 'me',
password: 'secret'
});
Then for single use cases:
connection.connect();
connection.query('SELECT 1 + 1 AS solution', function(err, rows, fields) {
// we are done with the connection
connection.end();
if (err) throw err;
console.log('The solution is: ', rows[0].solution);
});
If you're pooling, you should just create a connection pool when your application is starting and fetch connections when you need them. You should not make more than one pool.
var mysql = require('mysql');
var pool = mysql.createPool({
host: 'example.org',
user: 'bob',
password: 'secret'
});
Then when you need a connection, you'd do something like this:
pool.getConnection(function(err, connection) {
connection.query( 'SELECT something FROM sometable', function(err, rows) {
// we are done using the connection, return it to the pool
connection.release();
// the connection is in the pool, don't use it here
});
});
After more research, think I've figured out the right way.
1) Create a connection pool on app start
2) Include that file in your models.
3) Get a connection from the pool.
In the interest of keeping your code cleaner, I think you can also just invoke the pool object directly, according to the manual at https://github.com/felixge/node-mysql. This should abstract the logic for getting and releasing connections from the pool.
EG:
var result = yield pool.query("SELECT * FROM users");
(I'm using co-mysql with support for generators, but syntactically it should be the same w/out the callbacks)
I'm just getting into coding server side javascript and have been reading tutorials on socket.io and node.js, but I haven't come across anything demonstrating how to use node.js to access a mysql database.
Say for instance I want to create a method that listens to a table in my database at
mysql.something.com (with database: database, username: username, etc), how would
I get socket.io with node.js to connect to that database and listen for new input to that table and then subsequently return that input?
I'm wondering if anyone could give me a specific example that uses a publish subscribe model.
Thanks for the help.
You have to poll mysql database for changes at regular interval and when detect a change emit a socket.io event. Here's a pseudo code
var mysql = require('mysql');
var connect = mysql.createConnection({
host: 'localhost'
, database: 'your_database'
, username: 'user'
, password: 'password'});
var initial_result;
// check for changes after 1 second
setTimeout(function(){
connect.query('select * from your_table', function(err, result) {
if(err) { throw new Error('Failed');}
initial_result = initial_result || result;
if(Changed(initial_result, result)) { socket.emit('changed', result); }
});
function Changed(pre, now) {
// return true if pre != now
}
}, 1000);