I am having an issue with the session variable not setting. I am assuming the problem has something to do with the functions being asynchronous(async/await).
How do I make sure that session is set before jwt.sign and committing (how should I use async await or promises here)?
connection.getConnection(function(err, conn) {
try {
conn.beginTransaction(function(err) {
if (err) {
throw err
}
conn.query(`INSERT INTO USER SET username = ?, email = ?; SELECT LAST_INSERT_ID() AS id;`, [username, hashedEmail], (error, results) => {
if(error){
throw error;
} else {
// this doesn't set
req.session.verify = results[0].id;
jwt.sign(
{},
process.env.gmail_secret,
{
expiresIn: '1d',
},
(err, emailToken) => {
//...
}
)
await conn.commit(async function(error) {
if (error) {
throw err;
}
await conn.destroy();
req.flash("flash", "You have successfully created an account. The last step is to confirm your email so we know you are legit =].");
return res.redirect('/register');
});
}
})
})
} catch {
return conn.rollback(function() {
conn.destroy();
req.flash("flash", 'A server error has occurred.');
res.redirect('/register');
});
}
});
Related
connection.query(`START TRANSACTION;`, async function (err) {
if (err) {
req.flash("flash", "Something went wrong while deleting. Try again.");
return res.redirect("back");
} else {
await connection.query(
`INSERT INTO... ; SELECT LAST_INSERT_ID();`,
async (error, results1) => {
if (error) {
await connection.query(`ROLLBACK;`, function (err) {
req.flash("flash", "There was an error while posting.");
return res.redirect("/post");
});
} else {
var post_id = await results1[0].insertId;
await connection.query(
`INSERT INTO...`,
[],
async (error, results) => {
if (error) {
await connection.query(`ROLLBACK;`, function (err) {
req.flash("flash", "There was an error while posting.");
return res.redirect("/post");
});
} else {
await connection.query(
`INSERT INTO...`,
async (error, results) => {
if (error) {
await connection.query(`ROLLBACK;`, function (err) {
req.flash("flash", "There was an error while posting.");
return res.redirect("/post");
});
}
await connection.query(`ROLLBACK;`, function(err){
req.flash("flash", "Went through...");
return res.redirect("back");
})
...
});
First I start the transaction. Then I have the code in the middle to insert data into 3 tables. I wanted to test the rollback which is supposed to undo everything after START TRANSACTION but it didn't do anything. What did I do wrong?
(The code works by itself so please ignore if I missplaced any brackets).
The problem was that I was using a pool directly. You need to get a new connection and then use that to make all the queries and transactions. That is also the reason beginTransaction wasn't working for me before. What I should have done is:
connection.getConnection(function(err, con) {
con.query...
con.beginTransaction()...
con.query..
con.rollback()...
})
You can check the multiple-transaction-manager library to manage your transaction.
https://www.npmjs.com/package/#multiple-transaction-manager/mysql
Example;
// init manager & context
const txnMngr: MultiTxnMngr = new MultiTxnMngr();
const mysqlContext = new MysqlDBContext(txnMngr, pool);
// Add first step
mysqlContext.addTask("DELETE FROM test_table");
// Add second step
mysqlContext.addTask("INSERT INTO test_table(id, name) VALUES (:id, :name)", { "id": 1, "name": "Dave" });
// Add third step
mysqlContext.addTask("INSERT INTO test_table(id, name) VALUES (:id, :name)", { "id": 2, "name": "Kevin" });
// jest test
await expect(txnMngr.exec()).resolves.not.toBeNull();
I am using Nodejs MySQL and tried to create database level transaction so that I can execute a bunch of statements in a batch and rollback if there is an error in any step. I tried to follow this tutorial.
My database module is:
let mysql = require('mysql')
let keys = require('../config/keys')
let util = require('util')
let pool = mysql.createPool({
connectionLimit: 20,
host: keys.connection.host,
user: keys.connection.user,
password: keys.connection.password,
database: keys.connection.database,
dateStrings: true
// debug:true //Set this to true for verbose debugging. Leaving this to default for now cause it is creating too many messages at my console
})
pool.getConnection((err, connection) => {
if (err) {
if (err.code === 'PROTOCOL_CONNECTION_LOST') {
console.error('Database connection was closed.')
}
if (err.code === 'ER_CON_COUNT_ERROR') {
console.error('Database has too many connections.')
}
if (err.code === 'ECONNREFUSED') {
console.error('Database connection was refused.')
}
}
if (connection) connection.release()
return
})
pool.query = util.promisify(pool.query)
const connection = () => {
return new Promise((resolve, reject) => {
pool.getConnection((err, connection) => {
if (err) reject(err);
console.log("MySQL pool connected: threadId " + connection.threadId);
const query = (sql, binding) => {
return new Promise((resolve, reject) => {
connection.query(sql, binding, (err, result) => {
if (err) reject(err);
resolve(result);
});
});
};
const release = () => {
return new Promise((resolve, reject) => {
if (err) reject(err);
console.log("MySQL pool released: threadId " + connection.threadId);
resolve(connection.release());
});
};
resolve({
query,
release
});
});
});
};
// const query = (sql, binding) => {
// return new Promise((resolve, reject) => {
// pool.query(sql, binding, (err, result, fields) => {
// if (err) reject(err);
// resolve(result);
// });
// });
// };
module.exports = {
pool,
connection
}
In my route, I am trying to use the connection which should allow transaction:
const mysql = require('../../middleware/database')
async function buildCoreSchemas(){
const connection = await mysql.connection();
try{
await connection.query("START TRANSACTION");
await connection.query(`CREATE TABLE adjustreason (
AdjustID int NOT NULL AUTO_INCREMENT,
AdjustReason varchar(100) NOT NULL,
PRIMARY KEY (AdjustID)
)`)
await connection.query(`insert into adjustreason(AdjustReason) values('sdsds')`)
await connection.query(`insert into adjustreason(FAKECOLUMN) values('sdsds')`)
await connection.query("COMMIT");
}
catch(err){
await connection.query("ROLLBACK");
console.log(err)
return false
}
finally {
await connection.release();
}
As you can see I my second insert statement is wrong as there is no column name called FAKE COLUMN. So, the error gets caught and I get the error message in my console:
Unknown column 'FAKECOLUMN' in 'field list
But when I go and look at my database the transaction is not rollbacked because I can see that the first record is still there. What am I doing wrong?
Ciao, try to modify code in this way:
connection.beginTransaction(function(err) {
if (err) { throw err; }
connection.query(`CREATE TABLE adjustreason (
AdjustID int NOT NULL AUTO_INCREMENT,
AdjustReason varchar(100) NOT NULL,
PRIMARY KEY (AdjustID)
)`, function (error, results, fields) {
if (error) {
return connection.rollback(function() {
throw error;
});
}
connection.query(`insert into adjustreason(AdjustReason) values('sdsds')`, function
(error, results, fields) {
if (error) {
return connection.rollback(function() {
throw error;
});
}
connection.query(`insert into adjustreason(FAKECOLUMN) values('sdsds')`, function
(error, results, fields) {
if (error) {
return connection.rollback(function() {
throw error;
});
}
connection.commit(function(err) {
if (err) {
return connection.rollback(function() {
throw err;
});
}
console.log('success!');
});
});
});
});
});
so you call connection.query inside connection.beginTransaction and if one of those query fails, you call connection.rollback. Otherwise connection.commit
I have a console program where the user scans in serial numbers, and those serial numbers get added to a database.
const mysql = require('mysql2');
const read = require('readline-sync');
const conn = new mysql.createConnection(config);
conn.connect(
function(err){
if(err){
throw err;
}
else{
console.log("Connection Established");
while(1){
var sn = read.question('Scan in serial number: ');
conn.query('INSERT INTO test (serial) VALUES (?);',
[sn], function(err, results, fields){
if (err){
throw err;
}
else{
console.log("Added stuff");
}
});
}
}
}
);
When the code runs it successfully connects to the database but queries the database. It continually prompts for user input.
Alternatively, I tried storing serial numbers in an array and then loops through it adding each element, like this.
const mysql = require('mysql2');
const read = require('readline-sync');
var array = [];
var sn = " ";
while (1) {
sn = read.question('Scan in serial number, or enter "done" if finished scanning');
if (sn == "done") {
break;
}
array.push(sn);
}
conn.connect(
function (err) {
if (err) {
throw err;
}
else {
console.log("Connection Established");
array.forEach(function (sn) {
conn.query('INSERT INTO test (serial) VALUES (?);',
[sn], function (err, results, fields) {
if (err) {
throw err;
}
else {
console.log("Added stuff");
}
});
});
}
}
);
In this case, it works inconsistently. Sometimes it works fine, and other times it fails to connect and throws a timeout error. Is there a better way to accomplish this and/or am I doing something wrong?
var promises = []
function dbOp(value) {
return new Promise(function(resolve, reject) {
conn.query('INSERT INTO test (serial) VALUES (?);',
[value], function (err, results, fields) {
if (err) {
return reject(err)
}
else {
console.log("Added stuff");
resolve(results)
}
}
}
conn.connect(function(err){
if(err){
throw err;
}
else{
for (i = 0; i < array.length; ++i) {
promises.push(dbOp(array[i]));
}
}
});
Promise.all(promises)
.then((results) => {
console.log("done", results);
})
.catch((e) => {
console.log(e)
});
This might be caused by short idle timeout setting in your mysql server. client.connect() is pretty much a no-op in mysql2, it connects immediately when you call mysql.createConnection(). You can change order to establish connection only after all data is collected:
const mysql = require('mysql2');
const read = require('readline-sync');
var array = [];
var sn = ' ';
while (1) {
sn = read.question('Scan in serial number, or enter "done" if finished scanning');
if (sn == 'done') {
const conn = mysql.createConnection(config);
array.forEach(function(sn) {
conn.query('INSERT INTO test (serial) VALUES (?);', [sn], function(err, results, fields) {
if (err) {
throw err;
} else {
console.log('Added stuff');
}
});
});
}
array.push(sn);
}
Im using express-validator to check if the req.body entered is valid and to check if there is duplicate email in the MySQL database
Here is my code:
router.post(
"/signup",
[
body("uemail","email is not valid")
.isEmail()
.normalizeEmail()
.custom(async (email, {req} )=>{
const queryString = "SELECT uid FROM EarlyUsers WHERE `uemail` = ?";
return await connection.query(queryString, [email], (err, rows, fields) => {
if (err) {
console.log(err)
}else {
if (rows.length != 0) {
return false
} else {
return true
}
}
});
})
,
body("uname").isLength({ min: 5 })
],
authControllers.signUp
);
I dont know why this custom validator does not work.
I've tried to throw new Error instead of return false, but it just crash the whole thing . I really need help with this
For it to work correctly instead of returning false you reject the Promise.
if (rows.length != 0) {
return Promise.reject("user already exists.");
}
I have achieved this way it might be helpful for others, I'm using sequelize :)
const User = require("../../models/User");
body('email', 'Invalid email').exists().isEmail().trim().escape().custom(userEmail=> {
return new Promise((resolve, reject) => {
User.findOne({ where: { email: userEmail } })
.then(emailExist => {
if(emailExist !== null){
reject(new Error('Email already exists.'))
}else{
resolve(true)
}
})
})
}),
I found this solution to check that the email is not duplicate:
router.post('/register',
body('email').isEmail().normalizeEmail().withMessage('The email format is not correct.').custom((email) => {
const queryString = `SELECT * FROM users WHERE user_email = "${email}"`;
return getFinalEmail(queryString).then(user => {
console.log(user);
if (user) {
return Promise.reject('E-mail already in use');
}
});
}),
// -- other validations
// .....
(req, res) => {
/* your code for this route */
}); // end of router('/register')
function getFinalEmail(param) {
return new Promise(function(resolve, reject) {
getEmailData(param, function(result) {
console.log(result);
resolve(result);
});
});
}
function getEmailData(query, callback) {
database.query(query, function(error, data){
if(data.length > 0) {
return callback(true);
} else {
return callback(false);
}
});
}
In the above code users is the name of my table and user_email is the column that email data of users are stored.
Am using node-mysql to add records to a database but am facing a challenge when the records to be inserted are an array of objects and I need the operation to be a transaction. I have simplified my problem by creating a test project to better explain my problem.
Lets say I have to tables users and orders and the data to be inserted looks like this
var user = {
name: "Dennis Wanyonyi",
email: "example#email.com"
};
var orders = [{
order_date: new Date(),
price: 14.99
}, {
order_date: new Date(),
price: 39.99
}];
I want to first insert a user to the database and use the insertId to add the each of the orders for that user. Am using a transaction since in case of an error, I want to rollback the whole process. Here is how I try to insert all the records using node-mysql transactions.
connection.beginTransaction(function(err) {
if (err) { throw err; }
connection.query('INSERT INTO users SET ?', user, function(err, result) {
if (err) {
return connection.rollback(function() {
throw err;
});
}
for (var i = 0; i < orders.length; i++) {
orders[i].user_id = result.insertId;
connection.query('INSERT INTO orders SET ?', orders[i], function(err, result2) {
if (err) {
return connection.rollback(function() {
throw err;
});
}
connection.commit(function(err) {
if (err) {
return connection.rollback(function() {
throw err;
});
}
console.log('success!');
});
});
}
});
});
However I have a problem iterating over the array of orders without having to call connection.commit multiple times within the for loop
I would suggest to construct a simple string for multiple row insert query for orders table in the for loop first and then execute it outside the for loop. Use the for loop to only construct the string. So you can rollback the query whenever you want or on error. By multiple insert query string i mean as follows:
INSERT INTO your_table_name
(column1,column2,column3)
VALUES
(1,2,3),
(4,5,6),
(7,8,9);
You can use Promise.all functionality of Bluebird for this.
var promiseArray = dataArray.map(function(data){
return new BluebirdPromise(function(resolve, reject){
connection.insertData(function(error, response){
if(error) reject(error);
else resolve(response);
}); //This is obviously a mock
});
});
And after this:
BluebirdPromise.all(promiseArray).then(function(result){
//result will be the array of "response"s from resolve(response);
database.commit();
});
This way, you can work all the inserts asyncronously and then use database.commit() only once.
Some kind of task in Node.js are Asynchronous( like I/O , DB and etc..), and there is a lots of LIBS that help to handle it.
but if you want don't use any lib,for iterating an array in JS and use it in an asynchronous functionality its better to implement it as recursive function.
connection.beginTransaction(function(err) {
if (err) {
throw err;
}
connection.query('INSERT INTO users SET ?', user, function(err, result) {
if (err) {
return connection.rollback(function() {
throw err;
});
}
// console.log(result.insertId) --> do any thing if need with inserted ID
var insertOrder = function(nextId) {
console.log(nextId);
if ((orders.length - 1) < nextId) {
connection.commit(function(err) {
if (err) {
return connection.rollback(function() {
throw err;
})
}
console.log(" ok");
});
} else {
console.log(orders[nextId]);
connection.query('INSERT INTO orders SET ?', orders[nextId], function(err, result2) {
if (err) {
return connection.rollback(function() {
throw err;
});
}
insertOrder(nextId + 1);
});
}
}
insertOrder(0);
});
});
as you can see I rewrite your for loop as a recursive function inside.
I would use the async.each to do the iteration and to fire all the queries in parallel. If some of the queries will fail, the asyncCallback will be called with an error and the program will stop processing the queries. This will indicate that we should stop executing queries and rollback. If there is no error we can call the commit.
I' ve decoupled the code a bit more and split it into functions:
function rollback(connection, err) {
connection.rollback(function () {
throw err;
});
}
function commit(connection) {
connection.commit(function (err) {
if (err) {
rollback(connection, err);
}
console.log('success!');
});
}
function insertUser(user, callback) {
connection.query('INSERT INTO users SET ?', user, function (err, result) {
return callback(err, result);
});
}
function insertOrders(orders, userId, callback) {
async.each(orders, function (order, asyncCallback) {
order.user_id = userId;
connection.query('INSERT INTO orders SET ?', order, function (err, data) {
return asyncCallback(err, data);
});
}, function (err) {
if (err) {
// One of the iterations above produced an error.
// All processing will stop and we have to rollback.
return callback(err);
}
// Return without errors
return callback();
});
}
connection.beginTransaction(function (err) {
if (err) {
throw err;
}
insertUser(user, function (err, result) {
if (err) {
rollback(connection, err);
}
insertOrders(orders, result.insertId, function (err, data) {
if (err) {
rollback(connection, err);
} else {
commit(connection);
}
});
});
});
you need to use async library for these kind of operation.
connection.beginTransaction(function(err) {
if (err) { throw err; }
async.waterfall([
function(cb){
createUser(userDetail, function(err, data){
if(err) return cb(err);
cb(null, data.userId);
});
},
function(userid,cb){
createOrderForUser(userid,orders, function() {
if(err) return cb(err);
cb(null);
});
}
], function(err){
if (err)
retrun connection.rollback(function() {
throw err;
});
connection.commit(function(err) {
if (err) {
return connection.rollback(function() {
throw err;
});
}
console.log('success!');
});
});
});
var createUser = function(userdetail, cb){
//-- Creation of Orders
};
var createOrderForUser = function (userId, orders, cb) {
async.each(orders, function(order, callback){
//-- create orders for users
},function(err){
// doing err checking.
cb();
});
};
See if you can write a Stored Procedure to encapsulate the queries, and have START TRANSACTION ... COMMIT in the SP.
The tricky part comes with needing to pass a list of things into the SP, since there is no "array" mechanism. One way to achieve this is to have a commalist (or use some other delimiter), then use a loop to pick apart the list.
currentLogs = [
{ socket_id: 'Server', message: 'Socketio online', data: 'Port 3333', logged: '2014-05-14 14:41:11' },
{ socket_id: 'Server', message: 'Waiting for Pi to connect...', data: 'Port: 8082', logged: '2014-05-14 14:41:11' }
];
console.warn(currentLogs.map(logs=>[ logs.socket_id , logs.message , logs.data , logs.logged ]));