I have a base controller for generic insert/update operations across the whole API, using only a table dictionary so we can use the same function to insert data into many tables.
The problem is there is a table that uses a correlative number generated via trigger, and when sequelize returns the inserted value, it includes the new ID but the correlative field returns empty, and I need it to show it on the interface.
I've thought of just querying the new field again to the API, or querying it on the same save function again when it includes these certain tables names, but is there a way to tell sequelize to "wait" for this new generated value and then return the data alright? Just like getting the new ID
Or maybe this needs to be fixed on the database? I don't have much experience in that field, but we are using MySQL if that helps.
function Init(models, dictionary) {
this.post = (req, res, next) => {
const { obj } = req.body;
const model = models[dictionary[obj._type]];
//Just stripping fields starting with "_"
const objClear = {};
for (const attr in obj) {
if (attr.charAt(0) !== '_') {
objClear[attr] = obj[attr];
}
}
//Saving
model.create(objClear).then(
(objSaved) => {
const data = {
obj: objSaved.get({ plain: true }),
action: 'inserted',
};
//I guess I could query the new row here again
res.json(data);
},
).catch(next);
};
}
module.exports = {
Init,
};
The response looks like:
{"obj":{"TOTAL":"0","ID":14,...,"TRANSACTION_NO":""},"action":"inserted"}
Where TRANSACTION_NO is the field generated with a trigger.
AFAIK, you have to query the new row unless you use Postgres (in which case you might try the Model.create option called "options.returning")
Two quick tests that did NOT solve the problem:
an afterCreate hook - the model still shows fields created by a trigger as null.
a model having a default value from a DB function - the model shows the function call,
not the result of the function (which does make it to the DB field).
Hope someone else has a solution!
Following is the code:
Accounts.findOrCreate({
where: {
userName: request.payload.userName
},
attributes: { exclude: ['password','sessionToken'] },
defaults: request.payload
}).spread(function (account, created) {
if (created) {
var account = account.get({
plain: true
});
console.log(account); // has the password and sessionToken fields
return reply(account).code(201);
} else {
return reply("user name already exists").code(422);
}
});
I noticed that sequelize first fires a select query in which the password field is not present, then it fires an insert statement in which the password field is present, and that needs to be there.
I would just like the password and sessionToken not be present in the resulting account object. I could of course delete those properties from the object but I am looking for a more straightforward way.
It seems like you need to delete those fields manually. According to the source code, findOrCreate method first fires the findOne function and then it goes with create if instance was not found. The create method does not accept attributes parameter. In such a case all fields will be returned.
Good solution would be to create instance method in the Accounts model in order to return an instance with only the desired attributes.
{
instanceMethods: {
toJson: function() {
let account = {
id: this.get('id'),
userName: this.get('userName')
// and other fields you want to include
};
return account;
}
}
}
Then you could simply use the toJson method when returning raw representation of object:
Accounts.findOrCreate({ where: { userName: 'username' } }).spread((account, created) => {
return account ? account.toJson() : null;
});
As mentioned by piotrbienias you can follow his way otherwise just delete the unwanted elements like this:
Accounts.findOrCreate({
where: {
userName: request.payload.userName
},
defaults: request.payload
}).spread(function (account, created) {
if (created) {
var account = account.get({
plain: true
});
delete account.password;
delete account.sessionToken;
console.log(account); // now you don't have the password and sessionToken fields
return reply(account).code(201);
} else {
return reply("user name already exists").code(422);
}
});
I have a webserver with JSON data in it. This is what my data looks like
[
{
iduser: 1,
username: "joe",
password: "****"
},
{
iduser: 2,
username: "gina",
password: "****"
}
]
In my app I take some user input and wish to compare it to the username and password field. Here is where I check the data
.service('LoginService', function ($q, $http) {
return {
loginUser: function (name, pw) {
var deferred = $q.defer();
var promise = deferred.promise;
var user_data = $http.get("http://<my ip address>:<port>/login");
user_data.then(function ($scope, result) {
$scope.user = result.data;
})
for (var x in $scope.user) {
if (name == x.username && pw == x.password) {
deferred.resolve('Welcome ' + name + '!');
} else {
deferred.reject('Wrong credentials.');
}
}
promise.success = function (fn) {
promise.then(fn);
return promise;
}
promise.error = function (fn) {
promise.then(null, fn);
return promise;
}
return promise;
}
}
})
I am still learning angularJS and I know this is not a secure way to check the data I just want this loop to work.
My understanding of what I have here is that $scope.user holds my JSON data. Then the data is cycled through with the for loop and the user input name is compared to the field username of each iteration. But this is not the case as I am getting a fail every time.
I'm almost certain its a syntax error, but I don't know JavaScript or AngularJS well enough to find the problem. Any help is really appreciated, Thanks.
Edit 1
After what Nujabes said I made some changes since I don't need $scope.
//previous code the same
user_data.then(function (result) {
var user = result.data;
})
for (var x in user) {
if (name == x.username && pw == x.password) {
//prior code the same
I don't think var can hold the data and thats why I'm still getting errors. I think it should be in an array.
I think your syntax error is that you omit $scope.
You should inject $scope service to this line:
.service('LoginService',function($q,$http,$scope){ ...
});
And this code :
user_data.then(function ($scope, result) {
$scope.user = result.data;
});
Omit the $scope.
->
user_data.then(function (result) {
$scope.user = result.data;
});
like this.
Give it a try.
I hope it work.
(However, why do you want to use $scope service in your 'service'?
I think, defining local value and returning some method is a better choice.
and you use the $scope service in your 'controller'.)
$scope.user you are trying to loop through is array right ?
using (for/in) will store the key in the variable x which is in your case the index of each element (0,1,2,..) , to loop through arrays use (for/of) like this :
for (var value of array)
this will give you the values ...
Im using express, body-parser and moongose to build a RESTful web service with Node.js. Im getting json data in the body of a POST request, that function looks like this:
router.route('/match')
// create a match (accessed at POST http://localhost:3000/api/match)
.post(function(req, res) {
if (req._body == true && req.is('application/json') == 'application/json' ) {
var match = new Match(); // create a new instance of the match model
match.name = req.body.name; // set the match name and so on...
match.host = req.body.host;
match.clients = req.body.clients;
match.status = req.body.status;
// save the match and check for errors
match.save(function(err) {
if (err) {
//res.send(err);
res.json({ status: 'ERROR' });
} else {
res.json({ status: 'OK', Match_ID: match._id });
}
});
} else {
res.json({ status: 'ERROR', msg: 'not application/json type'});
}
});
The model Im using for storing a match in the database looks like this:
// app/models/match.js
var mongoose = require('mongoose');
var Schema = mongoose.Schema;
var MatchSchema = new Schema({
name: String,
host: String,
clients: { type: [String]},
date: { type: Date, default: Date.now },
status: { type: String, default: 'started' }
});
module.exports = mongoose.model('Match', MatchSchema);
But how do I validate that the json data in the body of the POST request has the key/value fields I want? For clarification, I dont want to insert data in the database that is incomplete. If I test to skip a key/value pair in the json data I get a missing field in the database and when I tries to read req.body.MISSING_FIELD parameter in my code I get undefined. All fields except date in the model is required.
Im using json strings like this to add matches in the database
{"name": "SOCCER", "host": "HOST_ID1", "clients": ["CLIENT_ID1", "CLIENT_ID2"], "status": "started"}
I use a very simple function that takes an array of keys, then loops through it and ensures that req.body[key] is not a falsy value. It is trivial to modify it to accommodate only undefined values however.
In app.js
app.use( (req, res, next ) => {
req.require = ( keys = [] ) => {
keys.forEach( key => {
// NOTE: This will throw an error for ALL falsy values.
// if this is not the desired outcome, use
// if( typeof req.body[key] === "undefined" )
if( !req.body[key] )
throw new Error( "Missing required fields" );
})
}
})
in your route handler
try{
// Immediately throws an error if the provided keys are not in req.body
req.require( [ "requiredKey1", "requiredKey2" ] );
// Other code, typically async/await for simplicity
} catch( e ){
//Handle errors, or
next( e ); // Use the error-handling middleware defined in app.js
}
This only checks to ensure that the body contains the specified keys. IT won't validate the data sent in any meaningful way. This is fine for my use case because if the data is totally borked then I'll just handle the error in the catch block and throw an HTTP error code back at the client. (consider sending a meaningful payload as well)
If you want to validate the data in a more sophisticated way, (like, say, ensuring that an email is the correct format, etc) you might want to look into a validation middle-ware, like https://express-validator.github.io/docs/
I'm using Sequelize in my Nodejs project and I found a problem that I'm having a hard time to solve.
Basically I have a cron that gets an array of objects from a server than inserts it on my database as a object ( for this case, cartoons ). But if I already have one of the objects, I have to update it.
Basically I have a array of objects and a could use the BulkCreate() method. But as the Cron starts again, it doesn't solve it so I was needing some sort of update with an upsert true flag. And the main issue: I must have a callback that fires just once after all these creates or updates. Does anyone have an idea of how can I do that? Iterate over an array of object.. creating or updating it and then getting a single callback after?
Thanks for the attention
From the docs, you don't need to query where to perform the update once you have the object. Also, the use of promise should simplify callbacks:
Implementation
function upsert(values, condition) {
return Model
.findOne({ where: condition })
.then(function(obj) {
// update
if(obj)
return obj.update(values);
// insert
return Model.create(values);
})
}
Usage
upsert({ first_name: 'Taku' }, { id: 1234 }).then(function(result){
res.status(200).send({success: true});
});
Note
This operation is not atomic.
Creates 2 network calls.
which means it is advisable to re-think the approach and probably just update values in one network call and either:
Look at the value returned (i.e. rows_affected) and decide what to do.
Return success if update operation succeeds. This is because whether the resource exists is not within this service's responsibility.
You can use upsert
It's way easier.
Implementation details:
MySQL - Implemented as a single query INSERT values ON DUPLICATE KEY UPDATE values
PostgreSQL - Implemented as a temporary function with exception handling: INSERT EXCEPTION WHEN unique_constraint UPDATE
SQLite - Implemented as two queries INSERT; UPDATE. This means that the update is executed regardless of whether the row already
existed or not
MSSQL - Implemented as a single query using MERGE and WHEN (NOT) MATCHED THEN Note that SQLite returns undefined for created, no
matter if the row was created or updated. This is because SQLite
always runs INSERT OR IGNORE + UPDATE, in a single query, so there
is no way to know whether the row was inserted or not.
Update 07/2019 now with async/await
async function updateOrCreate (model, where, newItem) {
// First try to find the record
const foundItem = await model.findOne({where});
if (!foundItem) {
// Item not found, create a new one
const item = await model.create(newItem)
return {item, created: true};
}
// Found an item, update it
const item = await model.update(newItem, {where});
return {item, created: false};
}
I liked the idea of Ataik, but made it a little shorter:
function updateOrCreate (model, where, newItem) {
// First try to find the record
return model
.findOne({where: where})
.then(function (foundItem) {
if (!foundItem) {
// Item not found, create a new one
return model
.create(newItem)
.then(function (item) { return {item: item, created: true}; })
}
// Found an item, update it
return model
.update(newItem, {where: where})
.then(function (item) { return {item: item, created: false} }) ;
}
}
Usage:
updateOrCreate(models.NewsItem, {slug: 'sometitle1'}, {title: 'Hello World'})
.then(function(result) {
result.item; // the model
result.created; // bool, if a new item was created.
});
Optional: add error handling here, but I strongly recommend to chain all promises of one request and have one error handler at the end.
updateOrCreate(models.NewsItem, {slug: 'sometitle1'}, {title: 'Hello World'})
.then(..)
.catch(function(err){});
This might be an old question, but this is what I did:
var updateOrCreate = function (model, where, newItem, onCreate, onUpdate, onError) {
// First try to find the record
model.findOne({where: where}).then(function (foundItem) {
if (!foundItem) {
// Item not found, create a new one
model.create(newItem)
.then(onCreate)
.catch(onError);
} else {
// Found an item, update it
model.update(newItem, {where: where})
.then(onUpdate)
.catch(onError);
;
}
}).catch(onError);
}
updateOrCreate(
models.NewsItem, {title: 'sometitle1'}, {title: 'sometitle'},
function () {
console.log('created');
},
function () {
console.log('updated');
},
console.log);
User.upsert({ a: 'a', b: 'b', username: 'john' })
It will try to find record by hash in 1st param to update it, if it will not find it - then new record will be created
Here is example of usage in sequelize tests
it('works with upsert on id', function() {
return this.User.upsert({ id: 42, username: 'john' }).then(created => {
if (dialect === 'sqlite') {
expect(created).to.be.undefined;
} else {
expect(created).to.be.ok;
}
this.clock.tick(1000);
return this.User.upsert({ id: 42, username: 'doe' });
}).then(created => {
if (dialect === 'sqlite') {
expect(created).to.be.undefined;
} else {
expect(created).not.to.be.ok;
}
return this.User.findByPk(42);
}).then(user => {
expect(user.createdAt).to.be.ok;
expect(user.username).to.equal('doe');
expect(user.updatedAt).to.be.afterTime(user.createdAt);
});
});
Sound likes you want to wrap your Sequelize calls inside of an async.each.
This can be done with the custom event emitter.
Assuming your data is in a variable called data.
new Sequelize.Utils.CustomEventEmitter(function(emitter) {
if(data.id){
Model.update(data, {id: data.id })
.success(function(){
emitter.emit('success', data.id );
}).error(function(error){
emitter.emit('error', error );
});
} else {
Model.build(data).save().success(function(d){
emitter.emit('success', d.id );
}).error(function(error){
emitter.emit('error', error );
});
}
}).success(function(data_id){
// Your callback stuff here
}).error(function(error){
// error stuff here
}).run(); // kick off the queries
you can use findOrCreate and then update methods in sequelize. here is a sample with async.js
async.auto({
getInstance : function(cb) {
Model.findOrCreate({
attribute : value,
...
}).complete(function(err, result) {
if (err) {
cb(null, false);
} else {
cb(null, result);
}
});
},
updateInstance : ['getInstance', function(cb, result) {
if (!result || !result.getInstance) {
cb(null, false);
} else {
result.getInstance.updateAttributes({
attribute : value,
...
}, ['attribute', ...]).complete(function(err, result) {
if (err) {
cb(null, false);
} else {
cb(null, result);
}
});
}
}]
}, function(err, allResults) {
if (err || !allResults || !allResults.updateInstance) {
// job not done
} else {
// job done
});
});
Here is a simple example that either updates deviceID -> pushToken mapping or creates it:
var Promise = require('promise');
var PushToken = require("../models").PushToken;
var createOrUpdatePushToken = function (deviceID, pushToken) {
return new Promise(function (fulfill, reject) {
PushToken
.findOrCreate({
where: {
deviceID: deviceID
}, defaults: {
pushToken: pushToken
}
})
.spread(function (foundOrCreatedPushToken, created) {
if (created) {
fulfill(foundOrCreatedPushToken);
} else {
foundOrCreatedPushToken
.update({
pushToken: pushToken
})
.then(function (updatedPushToken) {
fulfill(updatedPushToken);
})
.catch(function (err) {
reject(err);
});
}
});
});
};
2022 update:
You can use the upsert function:
https://sequelize.org/api/v6/class/src/model.js~model#static-method-upsert
Insert or update a single row. An update will be executed if a row which matches the supplied values on either the primary key or a unique key is found. Note that the unique index must be defined in your sequelize model and not just in the table. Otherwise you may experience a unique constraint violation, because sequelize fails to identify the row that should be updated.
Implementation details:
MySQL - Implemented with ON DUPLICATE KEY UPDATE`
PostgreSQL - Implemented with ON CONFLICT DO UPDATE. If update data contains PK field, then PK is selected as the default conflict key.
Otherwise first unique constraint/index will be selected, which can satisfy conflict key requirements.
SQLite - Implemented with ON CONFLICT DO UPDATE
MSSQL - Implemented as a single query using MERGE and WHEN (NOT) MATCHED THEN
Note that Postgres/SQLite returns null for created, no matter if the row was created or updated