Related
I have a service function to get users from my MySQL database using Sequelize ORM and i would to like to append fullName to the Users.
const filterUsers = async ([...users]:IUsers[]) => {
let searchAttributes = {};
if (users) { searchAttributes = { [Op.or]: users }; }
const filteredUsers = await User.findAll({
raw: true,
nest: true,
where: { ...searchAttributes },
include: [{
model: Club,
as: 'homeClub',
}, {
model: Club,
as: 'awayClub',
}] });
return filteredUsers as unknown as IUsersWithTeams[];
};
FilteredUsers response:
filteredUsers =
[
{
id: 1
name: 'John',
LastName: 'Mayer',
homeClub: 'Barcelona',
awayClub: 'Real Madrid',
},
{
id: 2,
name: 'Adam',
LastName: 'Smith',
homeClub: 'PSG',
awayClub: 'Milan',
},
]
What i would like to receive:
const expectedUserResponse = [
{
id: 1
name: 'John',
LastName: 'Mayer',
FullName: 'John Mayer',
homeClub: 'Barcelona',
awayClub: 'Real Madrid',
},
{
id: 2,
name: 'Adam',
LastName: 'Smith',
FullName: 'Adam Smith',
homeClub: 'PSG',
awayClub: 'Milan',
},
]
How can i do that ? Thank you very much if you can help me ;)
You need to use a virtual field, see documentation and my other answer to the similar question. It seems the official documentation has the example that perfectly fits your request:
const { DataTypes } = require('#sequelize/core');
const User = sequelize.define('user', {
firstName: DataTypes.TEXT,
lastName: DataTypes.TEXT,
fullName: {
type: DataTypes.VIRTUAL,
get() {
return `${this.firstName} ${this.lastName}`;
},
set(value) {
throw new Error('Do not try to set the `fullName` value!');
}
}
});
I have the following tables:
Article - User - Tag - Followers - Suscribes
Article belongs to User (fk: userId in Article table)
Article can have many tag. Here is the generated tagarticle table:
Here is followers table:
And the Suscribes table:
A user can follow many users and suscribe to a country(payId), a tag or an article(for notifications).
How to query all articles of followed users and suscribed country or tag for a specific user?
I assume that you ask about Sequelize way of doing the query.
I am not sure if I understand your question correctly. You are looking for two queries:
Query all articles of followed users,
Query subscribed country/tag/article of specific user,
Let me start with the associations made between the models.
// in User model definition
User.belongsToMany(User, { as: 'Followers', through: 'Followers', foreignKey: 'userId', otherKey: 'followId' });
User.hasMany(Subscribe, { foreignKey: 'userId' });
User.hasMany(Article, { foreignKey: 'userId' });
With use of above association we are now able to query all articles of followed users
models.User.findByPrimary(1, {
include: [
{
model: models.User,
as: 'Followers',
include: [ models.Article ]
}
]
}).then(function(user){
// here you have user with his followers and their articles
});
Above query would generate result similar to
{
id: 1,
Followers: [
{
id: 4,
Articles: [
{
id: 1,
title: 'article title' // some example field of Article model
}
]
}
]
}
If you want to query country/tag/article subscribed by specific user, you would have to make another associations in Subscribe model
// in Subscribe model definition
Subscribe.belongsTo(Tag, { foreignKey: 'tagId' });
Subscribe.belongsTo(Article, { foreignKey: 'articleId' });
Subscribe.belongsTo(Country, { foreignKey: 'payId' });
Now we have all the associations required to perform the second query you asked for
models.User.findByPrimary(1, {
include: [
{
model: models.Subscribe,
include: [ models.Tag, models.Country, models.Article ]
}
]
}).then(function(user){
// here you get user with his subscriptions
});
In this example you get user with all his subscriptions accessed via user.Subscribes, which will have nested attributes Tag, Country and Article. If user subscribed to Tag, both Country and Article would be NULL in this case.
Minimal runnable example of getting articles by followed users with assertions
https://stackoverflow.com/a/42634024/895245 was correct, here is a runnable version of it, also covering some other related functionality like limit and ordering. Further examples of interest at: How to implement many to many association in sequelize tested on:
npm install sequelize#6.5.1 sqlite3#5.0.2
Source:
#!/usr/bin/env node
const assert = require('assert');
const path = require('path');
const { Sequelize, DataTypes } = require('sequelize');
const sequelize = new Sequelize({
dialect: 'sqlite',
storage: 'db.sqlite3',
});
(async () => {
// Create the tables.
const User = sequelize.define('User', {
name: { type: DataTypes.STRING },
}, {});
const Post = sequelize.define('Post', {
body: { type: DataTypes.STRING },
}, {});
User.belongsToMany(User, {through: 'UserFollowUser', as: 'Follows'});
User.hasMany(Post);
Post.belongsTo(User);
await sequelize.sync({force: true});
// Create data.
const users = await User.bulkCreate([
{name: 'user0'},
{name: 'user1'},
{name: 'user2'},
{name: 'user3'},
])
const posts = await Post.bulkCreate([
{body: 'body00', UserId: users[0].id},
{body: 'body11', UserId: users[0].id},
{body: 'body10', UserId: users[1].id},
{body: 'body11', UserId: users[1].id},
{body: 'body20', UserId: users[2].id},
{body: 'body21', UserId: users[2].id},
{body: 'body30', UserId: users[3].id},
{body: 'body31', UserId: users[3].id},
])
await users[0].addFollows([users[1], users[2]])
// Get all posts by authors that user0 follows.
// The posts are placed inside their respetive authors under .Posts
// so we loop to gather all of them.
{
const user0Follows = (await User.findByPk(users[0].id, {
include: [
{
model: User,
as: 'Follows',
include: [
{
model: Post,
}
],
},
],
})).Follows
const postsFound = []
for (const followedUser of user0Follows) {
postsFound.push(...followedUser.Posts)
}
postsFound.sort((x, y) => { return x.body < y.body ? -1 : x.body > y.body ? 1 : 0 })
assert(postsFound[0].body === 'body10')
assert(postsFound[1].body === 'body11')
assert(postsFound[2].body === 'body20')
assert(postsFound[3].body === 'body21')
assert(postsFound.length === 4)
}
// With ordering, offset and limit.
// The posts are placed inside their respetive authors under .Posts
// The only difference is that posts that we didn't select got removed.
{
const user0Follows = (await User.findByPk(users[0].id, {
offset: 1,
limit: 2,
// TODO why is this needed? It does try to make a subquery otherwise, and then it doesn't work.
// https://selleo.com/til/posts/ddesmudzmi-offset-pagination-with-subquery-in-sequelize-
subQuery: false,
include: [
{
model: User,
as: 'Follows',
include: [
{
model: Post,
}
],
},
],
})).Follows
assert(user0Follows[0].name === 'user1')
assert(user0Follows[1].name === 'user2')
assert(user0Follows.length === 2)
const postsFound = []
for (const followedUser of user0Follows) {
postsFound.push(...followedUser.Posts)
}
postsFound.sort((x, y) => { return x.body < y.body ? -1 : x.body > y.body ? 1 : 0 })
// Note that what happens is that some of the
assert(postsFound[0].body === 'body11')
assert(postsFound[1].body === 'body20')
assert(postsFound.length === 2)
// Same as above, but now with DESC ordering.
{
const user0Follows = (await User.findByPk(users[0].id, {
order: [[
{model: User, as: 'Follows'},
Post,
'body',
'DESC'
]],
offset: 1,
limit: 2,
subQuery: false,
include: [
{
model: User,
as: 'Follows',
include: [
{
model: Post,
}
],
},
],
})).Follows
// Note how user ordering is also reversed from an ASC.
// it likely takes the use that has the first post.
assert(user0Follows[0].name === 'user2')
assert(user0Follows[1].name === 'user1')
assert(user0Follows.length === 2)
const postsFound = []
for (const followedUser of user0Follows) {
postsFound.push(...followedUser.Posts)
}
// In this very specific data case, this would not be needed.
// because user2 has the second post body and user1 has the first
// alphabetically.
postsFound.sort((x, y) => { return x.body < y.body ? 1 : x.body > y.body ? -1 : 0 })
// Note that what happens is that some of the
assert(postsFound[0].body === 'body20')
assert(postsFound[1].body === 'body11')
assert(postsFound.length === 2)
}
// Here user2 would have no post hits due to the limit,
// so it is entirely pruned from the user list as desired.
// Otherwise we would fetch a lot of unwanted user data
// in a large database.
const user0FollowsLimit2 = (await User.findByPk(users[0].id, {
limit: 2,
subQuery: false,
include: [
{
model: User,
as: 'Follows',
include: [ { model: Post } ],
},
],
})).Follows
assert(user0FollowsLimit2[0].name === 'user1')
assert(user0FollowsLimit2.length === 1)
// Case in which our post-sorting is needed.
// TODO: possible to get sequelize to do this for us by returning
// a flat array directly?
// It's not big deal since the LIMITed result should be small,
// but feels wasteful.
// https://stackoverflow.com/questions/41502699/return-flat-object-from-sequelize-with-association
// https://github.com/sequelize/sequelize/issues/4419
{
await Post.truncate({restartIdentity: true})
const posts = await Post.bulkCreate([
{body: 'body0', UserId: users[0].id},
{body: 'body1', UserId: users[1].id},
{body: 'body2', UserId: users[2].id},
{body: 'body3', UserId: users[3].id},
{body: 'body4', UserId: users[0].id},
{body: 'body5', UserId: users[1].id},
{body: 'body6', UserId: users[2].id},
{body: 'body7', UserId: users[3].id},
])
const user0Follows = (await User.findByPk(users[0].id, {
order: [[
{model: User, as: 'Follows'},
Post,
'body',
'DESC'
]],
subQuery: false,
include: [
{
model: User,
as: 'Follows',
include: [
{
model: Post,
}
],
},
],
})).Follows
assert(user0Follows[0].name === 'user2')
assert(user0Follows[1].name === 'user1')
assert(user0Follows.length === 2)
const postsFound = []
for (const followedUser of user0Follows) {
postsFound.push(...followedUser.Posts)
}
// We need this here, otherwise we would get all user2 posts first:
// body6, body2, body5, body1
postsFound.sort((x, y) => { return x.body < y.body ? 1 : x.body > y.body ? -1 : 0 })
assert(postsFound[0].body === 'body6')
assert(postsFound[1].body === 'body5')
assert(postsFound[2].body === 'body2')
assert(postsFound[3].body === 'body1')
assert(postsFound.length === 4)
}
}
await sequelize.close();
})();
Super many to many to do the "posts by followed users" query without post processing
Super many to many means explicitly setting belongsTo/hasMany between each model and the through table, in addition to the belongsToMany of each model.
This is the only way I found to nicely make the "posts by followed users" query without post processing.
const assert = require('assert');
const path = require('path');
const { Sequelize, DataTypes, Op } = require('sequelize');
const sequelize = new Sequelize({
dialect: 'sqlite',
storage: 'tmp.' + path.basename(__filename) + '.sqlite',
define: {
timestamps: false
},
});
(async () => {
// Create the tables.
const User = sequelize.define('User', {
name: { type: DataTypes.STRING },
});
const Post = sequelize.define('Post', {
body: { type: DataTypes.STRING },
});
const UserFollowUser = sequelize.define('UserFollowUser', {
UserId: {
type: DataTypes.INTEGER,
references: {
model: User,
key: 'id'
}
},
FollowId: {
type: DataTypes.INTEGER,
references: {
model: User,
key: 'id'
}
},
}
);
// Super many to many.
User.belongsToMany(User, {through: UserFollowUser, as: 'Follows'});
UserFollowUser.belongsTo(User)
User.hasMany(UserFollowUser)
User.hasMany(Post);
Post.belongsTo(User);
await sequelize.sync({force: true});
// Create data.
const users = await User.bulkCreate([
{name: 'user0'},
{name: 'user1'},
{name: 'user2'},
{name: 'user3'},
])
const posts = await Post.bulkCreate([
{body: 'body0', UserId: users[0].id},
{body: 'body1', UserId: users[1].id},
{body: 'body2', UserId: users[2].id},
{body: 'body3', UserId: users[3].id},
{body: 'body4', UserId: users[0].id},
{body: 'body5', UserId: users[1].id},
{body: 'body6', UserId: users[2].id},
{body: 'body7', UserId: users[3].id},
])
await users[0].addFollows([users[1], users[2]])
// Get all the posts by authors that user0 follows.
// without any post process sorting. We only managed to to this
// with a super many to many, because that allows us to specify
// a reversed order in the through table with `on`, since we need to
// match with `FollowId` and not `UserId`.
{
const postsFound = await Post.findAll({
order: [[
'body',
'DESC'
]],
include: [
{
model: User,
attributes: [],
required: true,
include: [
{
model: UserFollowUser,
on: {
FollowId: {[Op.col]: 'User.id' },
},
attributes: [],
where: {UserId: users[0].id},
}
],
},
],
})
assert.strictEqual(postsFound[0].body, 'body6')
assert.strictEqual(postsFound[1].body, 'body5')
assert.strictEqual(postsFound[2].body, 'body2')
assert.strictEqual(postsFound[3].body, 'body1')
assert.strictEqual(postsFound.length, 4)
}
await sequelize.close();
})();
I'm making a small website with i18n. When starting I used local json files, but after switching to mongodb I experience an error I don't understand. A good explanation is highly appreciated.
The error I get is this:
TypeError: Method Uint8Array.length called on incompatible receiver [object Object]
I structure i18n data with "da" and "en" properties. I then use a method for filtering relevant language.
Here are examples of my data, both json and corresponding mongoose result. Both copied from terminal (printed with console.log):
json
[ { lang: { da: 'Dansk', en: 'Danish' }, rating: 5 },
{ lang: { da: 'Engelsk', en: 'English' }, rating: 5 },
{ lang: { da: 'Tysk', en: 'German' }, rating: 5 } ]
mongoose
[ { _id: 57e2561369e4bc0a8ca6c630,
lang: { da: 'Dansk', en: 'Danish' },
rating: 5,
id: '57e2561369e4bc0a8ca6c630' },
{ _id: 57e2561369e4bc0a8ca6c631,
lang: { da: 'Engelsk', en: 'English' },
rating: 5,
id: '57e2561369e4bc0a8ca6c631' },
{ _id: 57e2561369e4bc0a8ca6c632,
lang: { da: 'Tysk', en: 'German' },
rating: 5,
id: '57e2561369e4bc0a8ca6c632' } ]
filterLanguage method
var traverse = require('traverse');
var filterLanguage = function(language, obj) {
return traverse(obj).map(function (item) {
if (this.key === language) {
this.parent.update(item);
}
});
};
So filterLanguage('da', languages); should return something like:
[ { lang: 'Dansk', rating: 5 },
{ lang: 'Engelsk', rating: 5 },
{ lang: 'Tysk', rating: 5 } ]
filterLanguage() works on a local, valid JSON file but not on the Mongoose result set...
I tried JSON.stringify(obj). I also tried setting toObject() method in Mongoose models but no luck.
What am I doing wrong and how can I fix it?
--------- EDIT ---------
Model
var mongoose = require('mongoose');
var languageSchema = mongoose.Schema({
local: {
lang: { da: String, en: String },
rating: Number
}
});
module.exports = mongoose.model('Language', languageSchema);
Query
language.find({}, function(err, results) {
var obj = filterLanguage(lang, results.languages);
console.log(obj);
});
I tried setting toObject on the schema like this:
languageSchema.set('toObject', { virtuals: true });
The error you get is related to traverse not being able to handle ObjectId instances in your results array.
Since it looks like you don't use _id or id, the easiest way to fix this is to exclude those properties from the result documents:
language.find({}, '-_id -id', function(err, results) { ... })
(-id is probably superfluous, because it's a virtual that depends on _id)
Im using 'json-csv' library to create a csv from a users arrays with nested objects and arrays.
var users = [
{
subscriptions: [
{
package : {
name: 'Grammar'
},
state: 'EXPIRED',
timerange: {
period : 5550
},
transaction:{
amount: 10000
}
},
{
package : {
name: 'GK'
},
state: 'ACTIVE',
timerange: {
period : 30
},
transaction:{
amount: 10340
}
},
],
account:{
balance: 200
},
name: "Johhy Moe",
email: null,
user_id: "123456789",
username: null,
user_type: "facebook",
id: 3,
createdAt: "2016-07-11T08:02:40.000Z",
updatedAt: "2016-07-11T08:02:40.000Z",
},
{
subscriptions: [
{
package : {
name: 'GK'
},
state: 'EXPIRED',
timerange: {
period : 42
},
transaction:{
amount: 5252
}
},
{
package : {
name: 'MATH'
},
state: 'ACTIVE',
timerange: {
period : 25
},
transaction:{
amount: 200
}
}
],
account:{
balance: 1500
},
name: "John Doe",
email: null,
user_id: "123456789",
username: null,
user_type: "facebook",
id: 7,
createdAt: "2016-07-29T06:44:18.000Z",
updatedAt: "2016-07-29T06:44:18.000Z"
},
]
Now i want the generated csv to be like this
USERID,NAME,FBID,ACCOUNT,SUBSCRIPTION,PRICE,STATE,TIMEPERIOD
3,Johhy Moe,123456789,200,Grammar,10000,EXPIRED,5550
3,Johhy Moe,123456789,200,GK,10340,ACTIVE,30
7,John Doe,123456789,1500,GK,5252,EXPIRED,30
7,John Doe,123456789,1500,MATH,200,ACTIVE,25
As you see if there are two objects inside subscription array for each user, i want to repeat that user again but with different subscription data.
I've thought of using the library because my users array can go up to thousands of users with hundreds of subscription.
And i'm at a loss to what i should do.
my Code:
var options= {
fields : [
{
name : 'id',
label : 'USERID'
},
{
name : 'name',
label : 'Name'
},
{
name : 'user_id',
label : 'FBID'
},
{
name : 'account.balance',
label : 'ACCOUNT'
},
{
name: '',
label: 'Subscription'
}
]
}
var source = es.readArray(users)
source
.pipe(jsoncsv.csv(options))
.pipe(res)
I dont want to use a library also. So if someone could provide me with a resource to make my own csv file with strings and also using streams , that would be great. Thanks!!
This will solve your problem. Now you just have to change console.log to fs and write to your file.
var json2csv = function (json, listKeys) {
var str = "";
var prefix = "";
for (var i = 0; i < listKeys.length; i++) {
str += prefix + json[listKeys[i]];
prefix = ",";
}
return str;
};
var async = require('async');
var csvData = ['USERID,NAME,FBID,ACCOUNT,SUBSCRIPTION,PRICE,STATE,TIMEPERIOD'];
async.each(users, function (user, callback) {
var csvRow1 = {
USERID: user.id,
NAME: user.name,
FBID: user.user_id,
ACCOUNT: user.account.balance
};
async.each(user.subscriptions, function (subscription, callback) {
var csvRow2 = JSON.parse(JSON.stringify(csvRow1));
csvRow2.SUBSCRIPTION = subscription.package.name;
csvRow2.PRICE = subscription.transaction.amount;
csvRow2.STATE = subscription.state;
csvRow2.TIMEPERIOD = subscription.timerange.period;
csvData.push(json2csv(csvRow2, ['USERID', 'NAME', 'FBID', 'ACCOUNT', 'SUBSCRIPTION', 'PRICE', 'STATE', 'TIMEPERIOD']));
callback(null);
}, function (err) {
callback(err);
});
}, function (err) {
if (err) {
// return err;
} else {
// return csvData;
}
});
I have the following Json -
Property {
id: 122334,
source:
[ id : 123,
address:
{
city: "little rock",
state: "Arkansas",
country: "USA"
},
unit:
{
id: 222,
name: "The wall",
count: 2
}, ]
[ {id: 8889,
address:
{
city: "milka",
state: "Arkansas",
country: "USA"
},
unit:
{
id: 555,
name: "The watt",
count: 3
},
},
]
}
I am parsing it the following way -
string data = client.DownloadString(URL);
JToken token = JObject.Parse(data);
if (!string.IsNullOrEmpty(Convert.ToString(token["property"].Children())))
{
token["property"].Children().ToList().ForEach(child =>
{
string GetID = Convert.ToString(child["source"]["unit"]["id"]);
if (GetID == id)
{
//move rest of the code here
}
else
{
}
});
}
But I get the execption - cannot access child value on newtonsoft.json.linq.jproperty.
at the line -
string GetID = Convert.ToString(child["source"]["unit"]["id"]);
What am I doing wrong?
enter code here