I am having problems with transactions using node.js and mysql. The problem is that my transactions do not run in isolation, even if I set the isolation level to 'serializable'.
I set up the following minimal example to exemplify my problem. I am using a single table with two collumns (id, val):
CREATE TABLE `A` (
`id` int(11) unsigned NOT NULL AUTO_INCREMENT,
`val` int(11) DEFAULT NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
INSERT INTO `A` (`id`, `val`) VALUES (1,0);
Note that I am unsing InnoDB table type because it supports transactions.
My node.js program just reads the value from the single row in table A, and increments it using an update statement. The select statement uses the FOR UPDATE modifier to obtain a lock on the row. The two sql statement are wrapped into a transaction, and I execute 10 of these transactions in a for loop:
var orm = require("orm");
orm.connect("mysql://root#localhost/test", function (err, db) {
db.driver.execQuery("SET GLOBAL tx_isolation='SERIALIZABLE';", function (err, data) {
for (var i = 0; i < 10; i++) {
db.driver.execQuery("START TRANSACTION;", function (err, data) {
db.driver.execQuery("SELECT * FROM A FOR UPDATE;", function (err, data) {
var value = data[0].val
console.log('reading value: ',value)
db.driver.execQuery("UPDATE A SET val="+value+"+1 WHERE id=1", function (err, data) {
console.log('writing value: ', value+1)
db.driver.execQuery("COMMIT;", function (err, data) {})
})
})
})
}
})
})
I would expect that after running this code, the value stored in the table A is incremented by 10. However, it is just incremented by 1.
To understand what's going on I added printouts to the code. I would expect to see printouts
reading value 0
writing value 1
reading value 1
writing value 2
...
However I get the printouts
reading value 0
reading value 0
...
writing value 1
writing value 1
...
One way to fix the problem is to establish a new db connection for each transaction, but I'd prefer not to do that for performance reasons.
Can someone explain what is going on, and how I can change the above into a working example for transactions in node.js?
Ok, we found a solution using the node-mysql-transaction package (see code below).
var mysql = require('mysql');
var transaction = require('node-mysql-transaction');
var trCon = transaction({
connection: [mysql.createConnection,{
host : 'localhost',
database : 'test',
user : 'root',
password : 'root',
}],
dynamicConnection: 32,
idleConnectionCutoffTime: 1000,
timeout:600
});
for(var i=0;i<10;i++) {
trCon.set(function(err, safeCon){
safeCon.query('SELECT * FROM A FOR UPDATE;',[],function(err,result){
if (err) safeCon.rollback()
var val = result[0].val
val += 1
safeCon.query('UPDATE A SET val='+val,[],function(err,result){
if (err) safeCon.rollback(err)
else safeCon.commit();
})
})
})
}
Related
I am using Node.js. I use mysql and bluebird packages.
const pool = mysql.createPool({ ... });
const query = (stmt, params) => {
return Promise.promisify(pool.query, { context: pool })(stmt, params);
};
const params = { ... };
const stmt = 'insert into table set ? on duplicate key update ?';
return query(stmt, [params, params])
.then(results => {
// I want to know actually what is done, insert or update
});
There should be a key affectedRows from the return object. From the reference, affectedRows will be 1 if it is inserted, and 0 or 2 if it is updated.
return query(stmt, [params, params])
.then(results => {
// I want to know actually what is done, insert or update
if (results.affectedRows === 1) {
// inserted
} else {
// updated
}
});
For INSERT ... ON DUPLICATE KEY UPDATE statements, the affected-rows value per row is 1 if the row is inserted as a new row, 2 if an existing row is updated, and 0 if an existing row is set to its current values. If you specify the CLIENT_FOUND_ROWS flag, the affected-rows value is 1 (not 0) if an existing row is set to its current values.
Reference: https://dev.mysql.com/doc/refman/8.0/en/mysql-affected-rows.html
While I'm not as savvy with this bit of node api, the basics are the same:
query(stmt, [params.col1, params.col2])
.then(results => {
Console.log("Affected rows: " + results.affectedRows);
});
Now, the real problem is that MySQL is a fan of essentially returning garbage diagnostic information. If you modify more than 1 row, you'll have no idea what occurred on each one, thanks to this:
With ON DUPLICATE KEY UPDATE, the affected-rows value per row is 1 if the row is inserted as a new row, 2 if an existing row is updated, and 0 if an existing row is set to its current values.
If you can afford it, do these statements one at a time, and check the affected row count. Otherwise, I'm digging through some MySQL internal functions but I'm not seeing much.
As a side note, you're overreaching with your wildcards there. Instead, use that space to update/insert the columns you want, and parameterize the input values:
-- Please don't name it 'table'
INSERT INTO my_table (column1, column2)
VALUES (?, ?)
ON DUPLICATE KEY UPDATE column1 = VALUES(column1), column2 = VALUES(column2)
I insert a relation by the following code:
db.Where(exercise).FirstOrCreate(&exercise).Model(&User{ID: userID}).Association("Exercises").Append(&exercise)
Corresponding SQL printed by debug console to the code is:
INSERT INTO `user_exercise` (`user_id`,`exercise_id`) SELECT 1,1 FROM DUAL WHERE NOT EXISTS (SELECT * FROM `user_exercise` WHERE `user_id` = 1 AND `exercise_id` = 1)
I want know if there are new record created in user_exercise, but due to the generated SQL, if a relation in user_exercise already exists, it won't insert and produce no error.
Go-Gorm's Association object doesn't have a RowsAffected attribute, so I can't get RowsAffected from the query to confirm if a new record is created.
Though I can get RowsAffected from the first db object, like
db.Where(exercise).FirstOrCreate(&exercise).Model(&User{ID: userID}).Association("Exercises").Append(&exercise)
if db.RowsAffected == 1 {
// do something
}
I wonder since the db is shared by all queries, if another query executed at the same time and affected rows > 0, is it safe to get RowsAffected from the global db object?
Assuming that the user_execise table has an unique constraint (user_id, exercise_id) the insert should return an error if you try to do it to an already created record. (Exactly what you want)
So just do something like this...
db.Where(exercise).FirstOrCreate(&exercise)
ue := struct {
UserID uint
ExerciseID uint
}{
UserID: userID,
ExerciseID exercise.ID
}
if err := db.Table("user_exercise").Create(&ue).Error; err != nil {
// will enter here if it wasn't created
}
If it doesn't returns an Error means that a new record was created
I have a small DB where i insert new data to each column at a different time. Because I'm only inserting one new value, the values for other columns will become NULL. But if thats the case, i want to keep the old value.
My DB looks like this:
One solution would be using coalesce() i guess, but i'm updating each column dynamically, and so the other column names are unknown.
function database_call(request) {
database.query(request, function (err, result) {
if (err) {
console.log(err);
}
});
}
subscribedItem.on("changed", function (dataValue) {
let databaseAttribute = subscribedItem.itemToMonitor.nodeId.value;
let databaseValue = dataValue.value.value;
databaseAttribute = databaseAttribute.substring(databaseAttribute.indexOf('.')+1)
databaseAttribute = databaseAttribute.replace(".", '');
databaseAttribute = databaseAttribute.replace(/"/g, '');
database_call("INSERT INTO Prozessdaten ("+databaseAttribute+") VALUES ("+databaseValue+")");
});
I've found this that implements a 'vertical' coalesce.
You should first do a query like this, using SUBSTRING_INDEX and GROUP_CONCAT to obtain the latest not-null value available in the database for each column.
SELECT
SUBSTRING_INDEX(GROUP_CONCAT(05Hz ORDER BY ID DESC SEPARATOR '##INDEX##'), '##INDEX##', 1) AS 05Hz,
SUBSTRING_INDEX(GROUP_CONCAT(5Hz ORDER BY updated_at DESC SEPARATOR '##INDEX##'), '##INDEX##', 1) AS 5Hz
FROM
table
LIMIT 1
After that, update the single value you really need to update and perform an insert specifying all the values for every column.
Trying to insert data into mysql with 'INSERT INTO users SET ?', I get 500 error in front end. data is reaching the server but unable to insert into database. Tried even 'INSERT INTO users VALUES ?' but still fails. i'm able to retrieve the data from the database('SELECT * FROM users') but not inserting it.
router.post('/userstuff', function(req, res, next) {
var eachuser = {
name: req.body.name,
age: req.body.age
};
connection.query('INSERT INTO users SET ?',eachuser, function(err, rows){
if(err) console.log('Error selecting: %s ', err);
console.log(rows)
});
});
edit: added the schema:
use 02sqldb;
CREATE TABLE IF NOT EXISTS users (
id INT PRIMARY KEY AUTO_INCREMENT,
name VARCHAR(30),
age int(2)
);
And what if you try to first do a dummy query. I don't know for sure you can simply put "?" to insert a value without giving a column like
'INSERT INTO table_name (column1,column2,column3,...)
VALUES ('?','?','?',...)'
If it still isn't working you know it's something else.
I have 2 arrays :
columns = ['column1', 'column2'];
data = ['data1', 'data2'];
I'd like to update the table using a prepared query:
conn.query('UPDATE table SET ?? = ? WHERE id = ?', [columns, data, id],
function(err, info){
Excepted sql query :
UPDATE table SET column1 = 'data1', column2 = 'data2' WHERE id = 10
But I get something that looks like :
UPDATE table SET 'column1', 'column2' = 'data1', 'data2' WHERE id = 10
This feature works well for select or insert but it seems not working for update queries. Any thoughts on how I can get this work ?
From node-mysql docs, about escaping query values, we have this:
Arrays are turned into list, e.g. ['a', 'b'] turns into 'a', 'b'
, so it won't work the way you expect.
But, in the docs we also have this:
Objects are turned into key = 'val' pairs. Nested objects are cast to strings.
with an example:
var post = {id: 1, title: 'Hello MySQL'};
var query = connection.query('INSERT INTO posts SET ?', post, function(err, result) {
// Neat!
});
console.log(query.sql); // INSERT INTO posts SET `id` = 1, `title` = 'Hello MySQL'
So, in order to do what you want, the best option IMO, is to convert your arrays into an object like:
{
column1: 'data1',
column2: 'data2'
}
Just to clarify, since after Googling for ages I didn't find an exact example to show what I was looking for. Here is the code which hopefully is what Bobby Shark found out, since I don't think 'SET ??' works.
To UPDATE multiple columns (but not necessarily all) for one row, without typing up every column in the query, but passing an object of {column_name1: 'new_foo', column_name2: 'new_bar', column_name2: 'new_baz'}
(using body-parser to get object of form data)
var blog = req.body.blog; // object as described above
var sql = "UPDATE blog SET ? WHERE id = ?";
connection.query(sql, [blog, req.params.id], function(err, updatedBlog){
if(err){
console.log(err);
} else {
res.redirect("/blogs/" + req.params.id);
}
});
This post by Bala Clark helped (though we've read it in the docs 10 times!). Hope this helps to see example code of object with multiple column updates (name/value pairs). Again found no specific examples in many sites.