I am using knex 0.19.4 in node js 10.x. I have 2 SQL statements - Insert and Update which has to happen as a transaction.
// Using Var, so that below code has access to this variable
var sqlStageInsert = kx('stage').insert({
officeid: 'OFF000',
taskid: 'T002',
});
var sqlTaskPcUpdate = kx('task')
.update({ pc: 100})
.where('task.taskno', taskno)
.limit(1);
1st Try - 2nd SQL sqlTaskPcUpdate Not getting Executed
const sqlUpdateInsert = kx.transaction(function (trx) {
sqlStageInsert.transacting(trx)
.then(function () {
console.log(sqlTaskPcUpdate.toString()); // This is outputing correct SQL
return sqlTaskPcUpdate.transacting(trx);
})
.then(trx.commit)
.catch(trx.rollback);
});
await sqlUpdateInsert;
2nd Try - Getting error Transaction query already complete. This is based on Commit/rollback a knex transaction using async/await
await kx.transaction(async (trx) => {
try {
await sqlStageInsert.transacting(trx);
await sqlTaskPcUpdate.transacting(trx);
trx.commit();
} catch (error) {
trx.rollback();
throw error;
}
});
I would suggest you to try inserting the data in stage table first and then retrieve a common value which belongs to both the table for applying in where class of updating task table(Assuming both the table contain any one common column having same data).
Please note that as per knexjs.org website, knex.transaction() uses returning statement with respect to PostgreSQL to insert/ update more than one table to maintain consistency, however MySQL won't support transaction due to which I'm using return values in below code.
FYR, http://knexjs.org/#Transactions
Please refer below code snippet for your reference :
db.transaction(trx => {
trx.insert({
officeid: 'OFF000',
taskid: 'T002'
})
.into('stage')
.then(() => {
return trx('stage').where({taskid}).then(resp => resp[0].taskno)
})
.then(stageTaskno => {
return trx('task')
.update({pc: 100})
.where({stageTaskno})
.limit(1)
.then(resp => {
resp.json(resp)
})
})
.then(trx.commit)
.catch(trx.rollback)
});
hope this is helpful, cheers!
Related
I am writing unit tests for my code and wish to use transactions to prevent any stray data between tests.
The code uses Sequelize ORM for all interactions with the database. Since changing the actual code is not an option, I would be using cls-hooked to maintain transaction context instead of passing transaction to all the queries. There is a problem, however. On reading the official documentation and trying to go about it, the above approach seems to only work for managed transactions.
So far, the test code looks somewhat like:
test("Test decription", async () => {
try {
await sequelize.transaction(async (t) => {
//Actual test code
});
} catch (error) {
//Do nothing if query rolled back
}
});
What I intend to achieve (for obvious reasons):
let t;
beforeEach(async () => {
t = await sequelize.transaction();
});
test("Test decription", async () => {
//Actual test code
});
afterEach(async () => {
await t.rollback();
});
Is this possible? If yes, any help in implementing this would be appreciated.
I'm having the same problem -- after much Googling, I found this closed issue where they indicate that unmanaged transactions aren't supported by design 🥲
It's true that Sequelize doesn't automatically pass transactions to queries when you're using unmanaged transactions. But you can manually set the transaction property on the CLS namespace, just like Sequelize does on a managed transaction:
https://github.com/sequelize/sequelize/blob/v6.9.0/lib/transaction.js#L135
namespace.run(() => {
namespace.set('transaction', transaction);
/* run code that does DB operations */
});
This is tricky for tests because describe() calls can be nested and each can have their own beforeAll()/afterAll() and beforeEach()/afterEach() hooks. To do this right, each before hook needs to set up a nested transaction and the corresponding after hook should roll it back. In addition, the test case itself needs to run in a nested transaction so that its DB operations don't leak into other tests.
For anyone from the future:
I was facing the above problem and found a way to fix it with a helper function and cls-hooked.
const transaction = async (namespace: string, fn: (transaction: Transaction) => unknown) => {
const nameSpace = createNamespace(namespace);
db.Sequelize.useCLS(nameSpace);
const sequelize = db.sequelize;
const promise = sequelize.transaction(async (transaction: Transaction) => {
try {
await fn(transaction);
} catch (error) {
console.error(error);
throw error;
}
throw new TransactionError();
});
await expect(promise).rejects.toThrow(TransactionError);
destroyNamespace(namespace);
};
What the above code does is creating a cls namespace and transaction that will be discarded after the test run, the TransactionError is thrown to ensure the entire transaction is always rolled back on each test run.
Usage on tests would be:
describe('Transaction', () => {
test('Test', async () => {
await transaction('test', async () => {
// test logic here
});
});
});
I have read in the mysql 8 docs that a collection.add operation returns the id of the document added i the result. But I have seen no example of how to get that returned id.
I tried the following. The document is inserted but no clue about the returned result
mysqlx
.getSession(mysqlxOptions)
.then(function (session) {
var db = session.getSchema('oa4');
// Use the collection 'my_collection'
var myColl = db.getCollection('order_items');
return myColl;
})
.then(function (myColl) {
// Insert documents
return Promise
.all([
myColl.add(req.body).execute()
])
})
.then(function(result){
console.log(result);
res.send(result);
})
.catch(function (err) {
// Handle error
console.log(err);
});
What is the right way to get the result and pass it on?
The execute() method returns a Promise that resolves to a Result instance, which in turn provides a getGeneratedIds() method that in the case of Collection.add() contains an list of the _ids that have been auto-generated by the server for any inserted document that does not have one already.
Promise.all([myColl.add(req.body).execute()])
.then(function (results) {
console.log(results[0].getGeneratedIds()[0]);
})
In this case, and assuming req.body is itself a single document, if it contains an _id property, its value will be effectively used as the identifier, and the server will not auto-generate one, and as such, it will not be available in the list returned by getGeneratedIds().
Disclaimer: I'm the lead maintainer of the X DevAPI Node.js connector for MySQL.
I have an array of (few thousands of) data object that I need to insert or update depending on a condition. A simple upsert() method is implemented in my model class.
Implementation
csv.upsert = async function (values, condition) {
let obj = await csv.findOne({ where: condition });
if (obj) {
// update
console.log("existing record updated")
return await obj.update(values);
} else {
// insert
console.log("new record inserted")
return await csv.create(values);
}
}
This upsert method is then used, where I loop through the array of objects to insert or update them in db.
Usage
try {
await models.sequelize.authenticate();
await models.sequelize.sync();
let dataToBeInserted = getArrayOfDatatoInsert();
dataToBeInserted.map(async function (data) {
let condition = {
'categorygroup': data.categorygroup,
'category': data.category,
'country': data.country,
'city': data.city
};
await csvModel.upsert(data, condition);
})
// await restofthestuff();
} catch (error) {
console.log("error", error);
}
For test I took a dataset where all of my data needs to be updated.
When I run this method:
I can see in the (along with sequelize log turned on) log that "existing record updated" message is printed for each and every record that exists which is desired output. Only the last few (30) data gets updated in the db. Where as it works for csv.create(values)
~ How can I update all the records and obviously not just the last 30 data, any help's appreciated. ~
EDIT: Apparently I got this to work by using csv.update(values, {where: condition}) instead of using obj.update(values).
New question: I didn't look further into the sequelize's update method but is this a bug or am I doing something wrong here?
As detailed in the commented code below, your log is after your return and so will never be executed.
Also you were not using await in an async function, so either don't make it async or use await.
csv.upsert = async function (values, condition) {
const obj = await csv.findOne({ where: condition });
// you can await here, no need for then as you are in an async function
if (obj) { // update
// you need to log before your return condition
console.log("existing record updated")
return obj.update(values);
// since we return here, the rest of the code will not be executed
// you can skip the use of else
}
// we are in this part of the code only if object is falsy
// insert
console.log("new record inserted")
return csv.create(values);
}
You could also use Promise.all to ensure all the upsert are done:
await Promise.all(dataToBeInserted.map(async function (data) {
let condition = {
'categorygroup': data.categorygroup,
'category': data.category,
'country': data.country,
'city': data.city
};
await csvModel.upsert(data, condition);
}))
this will also ensure that if an error occur it gets catched by your try / catch
Maybe that will help you find what's causing the unexpected behaviours.
If I have a view in my (mvc project) contains data from many tables in the database, what is the best way to fetch them without getting into the nested tree of doom
Model1.findAll().then(model1Data => {
Model2.findAll().then(model2Data => {
Model3.findAll().then(model3Data => {
Modeln.findAll().then(modelnData => {
res.render('view', {
model1Data: model1Data,
model2Data: model2Data,
model3Data: model3Data,
modelnData: modelnData
});
})
})
})
})
Note: the above query has no where clauses, joins, or any other conditions
Here you can use 2 ways either Promise.all() or async/await :
Promise.all() :
const promises = [
Model1.findAll(),
Model2.findAll(),
Model3.findAll(),
Modeln.findAll()
]
Promise.all(promises).then((data) => {
res.render('view', data );
});
Async/await :
let model1Data = await Model1.findAll();
let model2Data = await Model2.findAll();
let model3Data = await Model3.findAll();
let modelnData = await Modeln.findAll();
res.render('view', {
model1Data: model1Data,
model2Data: model2Data,
model3Data: model3Data,
modelnData: modelnData
});
NOTE :
I would suggest to use Promise.all() if the queries are not dependent
on each other , as it will start execution and don't wait for the
first one to complete as it does in async/await.
For More Deatil : DO READ
I'm using knex in loopback for DB operation with mysql.
My task is to update the 2 table by using the transaction.
When I enter new entry in one tabe, i want to use id of that entry for 2nd query operation.
But when transaction throw the error it not rolling back the data/ removing the first table entry if second table entry throws error. but in my case transaction always do commit not rollback i put my example code in below:
addTest : (data) => {
return new promise(function(resolve, reject) {
knex.transaction(function(t) {
return knex('foo')
.transacting(t)
.insert({
foo_id: data.foo_id ? data.foo_id : null,
foo_name: data.foo_name ? data.foo_name : null,
date_entered : new Date()
})
.then(function() {
return knex('bar')
.transacting(t)
.insert({
bar_id: data.bar_id ? data.bar_id : null,
bar_name : data.bar_name ? data.bar_name : null
})
})
.then(t.commit)
.catch(function(e) {
t.rollback();
throw e;
})
})
.then(function() {
// it worked
// resolve('sucess');
console.log('success');
})
.catch(function(e) {
// it failed
console.log('error'+e);
});
});
}
please, provide me suitable suggestion.
thank you
You can avoid having to call t.commit or t.rollback youself. See the docs.
Make your code inside the transaction function something like this
return t.insert({}).into('foo').returning('id')
.then( function(idArray) {
return t.insert({fooId: idArray[0]}).into('bar')
})
That lets knex handle the commiting and rolling back itself based on the result result of that promise. Also, note how I got the inserted fooId and applied it to the bar object for insertion. That was kind of mentioned in the question.