How to manage visibility in multiple Observable calls? - json

I developped an Angular2 service to retrieve a list a categories from a backend server and count how many 'links' exist per category.
Once I have the number of links for each category, I add a property to the Json object to 'store' the value.
Here is the code:
nbLinks = '';
...
getCategories() {
return this.category.find({where: {clientId: this.userApi.getCurrentId()}}).map((data) => {
this.categoriesList = data;
for (var i = 0; i < this.categoriesList.length; i++) {
var obj = this.categoriesList[i].id;
this.category.countLinks(obj).subscribe((linksCount) => {
this.nbLinks = linksCount;
}, err => {
console.log(err);
});
}
return data;
},
err => {
console.log(err);
}
);
I am getting the categories in a json object with the correct 'where' clause.
I am looping on the Json to 'count' the number of link in this category.
My problem is that outside the for loop (getting out) the variable i is bigger than my Json length so the app is crashing.
My second problem is that I do not have the visiblity of this.nbLinks outside the for ... loop.
Thanks an Regards

I'm not sure I understand your code, but two things stand out:
1) It looks like you're mixing synchronous and asynchronous code. It cannot work.
Sync code: the for loop. Async code: the observable.
Instead, could you refactor your code to ONLY work with observables and chain all the operations? You can wrap any piece of data in an observable with Observable.from() or Observable.of().
For instance:
getCategories() {
const categories = this.category.find({where: {clientId: this.userApi.getCurrentId()}});
return Observable.from(categories)
.map(category => countLinksInCategory(category));
}
If countLinksInCategory() is an async operation, then have that function return an Observable, and use .mergeMap() instead of .map() in the code above.
2) Try avoiding setting an outside variable from within your observable
// This part is not ideal
Obs.subscribe(linksCount => {
this.nbLinks = linksCount;
});
I would suggest renaming getCategories() to getNumLinks() to reflect the role of the function. The only job of the Observable inside this function is to produce a value. Then, the consumer of the Observable can use that value (i.e. assign it, display it...).
In terms of code:
getNumLinks(): Observable<number> {
// Here, count the number of links - See example code above.
// Eventually, return an observable wrapping the final value.
}
Then, elsewhere in your code:
// This is where you assign the value returned by the Observable.
// Note that we are OUTSIDE the Observable now.
getNumLinks().subscribe(numLinks => this.nbLinks = numLinks);

Related

Angular doesn't merge/concat/extend my json object

I have a problem concating 2 json objects together. Basicly my app is doing a get on my rest server every second and i'm only sending the newest data back so as angular is refreshing the whole object i found on google that i can concat the 2 jsons together (old and new) so i can keep everything. But the problem is that none of the concat/merge/extend functions work and i don't know what i'm missing.
data: any = null;
constructor(private _http: Http) {
setInterval(() => this.getLogs(), 1000)
}
public getLogs() {
return this._http.get('http://localhost')
.map((res: Response) => res)
.subscribe(data => {
if data._body != ''{
//this.data = data.json()
if this.data == null
this.data = data.json();
else
extend(this.data,data.json()); // PROBLEM HERE
}
console.log(this.data);
});
}
So far i tried this.data.concat(data.json()); if i try extend(this.data, data.json()) or merge(this.data, data.json()); I get errors saying that it's not defined. The concat function doesn't do anything. Doesn't trigger errors neither concat so i don't know what it is doing.
I'm logging the object everytme and i can see the object always stays at the first ever response i get (meaning it only does the if this.data == null).
https://www.w3schools.com/jsref/jsref_concat_array.asp states
The concat() method is used to join two or more arrays.
This method does not change the existing arrays, but returns a new
array, containing the values of the joined arrays.
So you need to concat the two arrays into the data variable
data: any = null;
constructor(private _http: Http) {
setInterval(() => this.getLogs(), 1000)
}
public getLogs() {
return this._http.get('http://localhost')
.map((res: Response) => res)
.subscribe(data => {
if data._body != ''{
//this.data = data.json()
if this.data == null
this.data = data.json();
else
this.data = this.data.concat(data.json());
}
console.log(this.data);
});
}
You can use spread operator to generate new object:
this.data = {...this.data, ...data.json()};
What this does is create a new object and then first migrates all the fields and values from this.data and then same thing from data.json() while overriding any existing fields that were already in this.data.
Not sure where you're getting extend from. That's not a function.
You can't concat two objects together. You're calling res.json(), so the return is no longer JSON. Even if you were, you can't just concat JSON strings together and expect the result to be valid.
You'd want to merge the objects together, which can be done with Object.assign(this.data, data.json() or a spread: this.data = {...this.data, ...data.json()}.
On top of that, you'd want to try/catch your JSON parsing before assigning. Plus, your map function is doing literally nothing. You can parse it there instead.
You can also streamline this by just initializing data to an empty object.
public data: any = {}
public getLogs() {
return this._http.get('http://localhost')
.map(res => res.json())
.filter(res => !!res) // ensure data exists
.subscribe(data => {
Object.assign(this.data, data);
});
}
Having said that, making a REST call every second seems like an egregious waste of resources and will put strain on Angular's change detection, with performance degrading as data increases. If the objects don't need to be merged, i.e. each call is segmented data, consider pushing new data to an array instead of an object. Plus, you might want to consider doing something a little more sane, like implementing an event stream like SSE (server sent events) on the backend.

getBulkProperties very slow

Following the advice given on one of my other questions (Get a list of all objects from loaded model), I added this code on my application to retrieve all properties from all objects:
function onGeometryLoadedEvent(viewer) {
var dbIds = Object.keys(
viewer.model.getData().instanceTree.nodeAccess.dbIdToIndex
);
this.viewer.model.getBulkProperties(
dbIds,
{
propFilter: false,
ignoreHidden: true
},
(objects) => console.log(objects),
() => console.log('error')
)
}
It correctly shows all the objects. The problem is it takes A LOT of time to complete (+1 minute), even for a model with just 10,000 objects.
Is this normal?
I really need the list of objects with their categories, I have to sort them after getting them to present a list of all the available categories and properties to the user.
I know I can use the Model Derivatives API, but I'd like to avoid it if possible.
Note that when you list all elements on the model it will include those that are not visible, like categories. If you need only elements on the model, then you need the leaf of the model. This article described it and the source code is below:
function getAllLeafComponents(viewer, callback) {
var cbCount = 0; // count pending callbacks
var components = []; // store the results
var tree; // the instance tree
function getLeafComponentsRec(parent) {
cbCount++;
if (tree.getChildCount(parent) != 0) {
tree.enumNodeChildren(parent, function (children) {
getLeafComponentsRec(children);
}, false);
} else {
components.push(parent);
}
if (--cbCount == 0) callback(components);
}
viewer.getObjectTree(function (objectTree) {
tree = objectTree;
var allLeafComponents = getLeafComponentsRec(tree.getRootId());
});
}
Now with the .getBulkProperties you can specify which property you want, so something like Category, right? So specify that, as shown at this article and used with the above .getAllLeafComponent function:
getAllLeafComponents(viewer, function (dbIds) {
viewer.model.getBulkProperties(dbIds, ['Category'],
function(elements){
// ToDo here
// this will only include leaf with Category property
})
})
And I would expect this to be faster than your original code, as I'm filtering elements and properties.

ReactJS Fixed-Data-Table and Async JSON for DataListStore

I am trying to learn ReactJS with ES6 along with setting up an instance of Fixed-Data-Table. I'm using the ObjectDataExample example from the github repo, but instead of the faker() values fed to the DataListStore, I want to use a DataListStore that gets its cache from a remote JSON resource. This is how I have defined my DataListStore:
class MyDataListStore {
constructor(/* url string */ url) {
this.url = url || 'http://localhost:8080/default-json';
this._cache = [];
this.pageSize = 1;
this.size = 0;
this.getRemoteData(url);
}
getRemoteData() {
/**
* Fetch remote JSON to be used in the store.
*/
var that = this;
fetch(this.url).then(function(response) {
return response.json();
}).then(function(j) {
console.log(j);
//this.pageSize = j["pages"];
that.size = j["total"];
that._cache = j["table"];
if (that._cache) {
// do something here?
}
});
}
getObjectAt(/*number*/ index) /*?object*/ {
if (index < 0 || index > this.size){
return undefined;
}
if (this._cache[index] === undefined) {
//this._cache[index] = this.createFakeRowObjectData(index);
}
return this._cache[index];
}
getSize() {
return this.size;
}
}
module.exports = MyDataListStore;
As you can see I'm following the FakeObjectDataListStore provided with the example from fixed-data-table more or less. The JSON is fetched properly, the _cache is populated with an array of objects, and when you output getSize once getRemoteData has executed, you do get the size of the _cache. However, I haven't figured out how my fixed-data-table Table component should be updated once the data has been fetched. Currently the Table is rendered but is simple blank with no rows.
class ObjectDataExample extends React.Component {
constructor(props) {
super(props);
this.state = {
dataList: new MyDataListStore()
};
}
render() {
var {dataList} = this.state;
return <Table
rowHeight={70} rowsCount={dataList.getSize()} width={1170} height={500} headerHeight={30}>
<Column
header={<Cell>ID</Cell>}
cell={<TextCell data={dataList} col="id" />}
width={50}
fixed={true}
/>
<Column
header={<Cell>Email</Cell>}
cell={<TextCell data={dataList} col="email" />}
width={300}
fixed={true}
/>
</Table>
}
}
module.exports = ObjectDataExample;
I think the main issue is that I don't have any code meant to populate the table once MyDataListStore is populated with the data from the async call. However, I can't find any help from the examples given in the Fixed-Data-Table github repo or the docs. Any idea how to get this done? I assume I need to set up some sort of event listener, but I'm not sure where/how to do this, as I'm still new to both ReactJS and Fixed-Data-Table.
Edit: I should also add that when the page loads, I get the following error:
Uncaught TypeError: Cannot read property 'id' of undefined
once I set the initial this.size to more than 0. So of course the table doesn't have the available data when it's first loading.
Edit 2: After looking into this further, it looks like if I run the fetch in componentDidMount of my ObjectDataExample and use this.setState(); to reset the dataList object, then I get the table updated. However, this looks a little messy and I'd assume there's a better way to do this directly from my MyDataListStore object.
Thanks,
One design issue with the current implementation of MyDataListStore is that it does not provide a way to notify the caller when the data has been loaded.
One possible way you might do this is to implement some sort of factory function (in the example below, I'm pretending that one exists called MyDataListStore.of) that returns a Promise that eventually resolves the MyDataListStore instance once the data loads:
// In the ObjectData component constructor, we call the MyDataListStore
// factory function and once it resolves, we assign it to our
// state. This will cause our component to re-render.
constructor() {
MyDataListStore.of(myDataListStoreUrl).then(store => {
this.setState({ dataList: store });
});
}
Now, once the data in the data list store resolves, our template (specified in your render function) will render correctly.
The DataListStore.of function we used earlier might look something like this:
class MyDataListStore {
static of(url) {
const dataListStore = new MyDataListStore(url);
return dataListStore.getRemoteData().then(() => return dataListStore);
}
/* ... other MyDataListStore properties/methods ... */
}
And finally we need to update the getRemoteData to return a promise. This is what will allow any clients of our MyDataListStore class to be notified that the data has loaded:
getRemoteData() {
/**
* Fetch remote JSON to be used in the store.
*/
var that = this;
// Return the chained promise! This promise will resolve
// after our last callback is called.
return fetch(this.url).then(function(response) {
return response.json();
}).then(function(j) {
console.log(j);
//this.pageSize = j["pages"];
that.size = j["total"];
that._cache = j["table"];
if (that._cache) {
// do something here?
}
});
}

What's the Reactive way to collapse elements into an array?

Take the following TypeScript/Angular 2 code sample:
query(): Rx.Observable<any> {
return Observable.create((o) => {
var refinedPosts = new Array<RefinedPost>();
var observable = this.server.get('http://localhost/rawData.json').toRx().concatMap(
result =>
result.json().posts
)
.map((post: any) => {
// Assume I want to convert the raw JSON data into a nice class object with
// methods, etc.
var refinedPost = new RefinedPost();
refinedPost.Message = post.Message.toLowerCase();
refinedPosts.push(refinedPost);
})
.subscribeOnCompleted(() => {
o.onNext(refinedPosts);
})
});
}
Written out, the database is returning JSON. I want to iterate over the raw JSON and create a custom object, eventually returning to subscribers an Array<RefinedPost>.
The code works and the final subscribers get what they need, but I can't help but feel like I didn't do it the "Reactive Way". I cheated and used an external accumulator to gather up the elements in the Array, which seems to defeat the purpose of using streams.
So, the question is, is there a better, more concise, reactive way to write this code?
Answering my own question.
query(): Rx.Observable<any> {
return this.server.get('http://localhost/rawData.json').toRx().concatMap(
result =>
result.json().posts
)
.map((post: any) => {
var refinedPost = new RefinedPost();
refinedPost.Message = post.Message.toLowerCase();
return refinedPost;
}).toArray();
}
This removes the internal accumulator and the wrapped Observable. toArray() took the sequence of items and brought them together into an array.

Convert Mongoose docs to json

I returned mongoose docs as json in this way:
UserModel.find({}, function (err, users) {
return res.end(JSON.stringify(users));
}
However, user.__proto__ was also returned. How can I return without it? I tried this but not worked:
UserModel.find({}, function (err, users) {
return res.end(users.toJSON()); // has no method 'toJSON'
}
You may also try mongoosejs's lean() :
UserModel.find().lean().exec(function (err, users) {
return res.end(JSON.stringify(users));
});
Late answer but you can also try this when defining your schema.
/**
* toJSON implementation
*/
schema.options.toJSON = {
transform: function(doc, ret, options) {
ret.id = ret._id;
delete ret._id;
delete ret.__v;
return ret;
}
};
Note that ret is the JSON'ed object, and it's not an instance of the mongoose model. You'll operate on it right on object hashes, without getters/setters.
And then:
Model
.findById(modelId)
.exec(function (dbErr, modelDoc){
if(dbErr) return handleErr(dbErr);
return res.send(modelDoc.toJSON(), 200);
});
Edit: Feb 2015
Because I didn't provide a solution to the missing toJSON (or toObject) method(s) I will explain the difference between my usage example and OP's usage example.
OP:
UserModel
.find({}) // will get all users
.exec(function(err, users) {
// supposing that we don't have an error
// and we had users in our collection,
// the users variable here is an array
// of mongoose instances;
// wrong usage (from OP's example)
// return res.end(users.toJSON()); // has no method toJSON
// correct usage
// to apply the toJSON transformation on instances, you have to
// iterate through the users array
var transformedUsers = users.map(function(user) {
return user.toJSON();
});
// finish the request
res.end(transformedUsers);
});
My Example:
UserModel
.findById(someId) // will get a single user
.exec(function(err, user) {
// handle the error, if any
if(err) return handleError(err);
if(null !== user) {
// user might be null if no user matched
// the given id (someId)
// the toJSON method is available here,
// since the user variable here is a
// mongoose model instance
return res.end(user.toJSON());
}
});
First of all, try toObject() instead of toJSON() maybe?
Secondly, you'll need to call it on the actual documents and not the array, so maybe try something more annoying like this:
var flatUsers = users.map(function() {
return user.toObject();
})
return res.end(JSON.stringify(flatUsers));
It's a guess, but I hope it helps
model.find({Branch:branch},function (err, docs){
if (err) res.send(err)
res.send(JSON.parse(JSON.stringify(docs)))
});
I found out I made a mistake. There's no need to call toObject() or toJSON() at all. The __proto__ in the question came from jquery, not mongoose. Here's my test:
UserModel.find({}, function (err, users) {
console.log(users.save); // { [Function] numAsyncPres: 0 }
var json = JSON.stringify(users);
users = users.map(function (user) {
return user.toObject();
}
console.log(user.save); // undefined
console.log(json == JSON.stringify(users)); // true
}
doc.toObject() removes doc.prototype from a doc. But it makes no difference in JSON.stringify(doc). And it's not needed in this case.
Maybe a bit astray to the answer, but if anyone who is looking to do the other way around, you can use Model.hydrate() (since mongoose v4) to convert a javascript object (JSON) to a mongoose document.
An useful case would be when you using Model.aggregate(...). Because it is actually returning plain JS object, so you may want to convert it into a mongoose document in order to get access to Model.method (e.g. your virtual property defined in the schema).
PS. I thought it should have a thread running like "Convert json to Mongoose docs", but actually not, and since I've found out the answer, so I think it is not good to do self-post-and-self-answer.
You can use res.json() to jsonify any object.
lean() will remove all the empty fields in the mongoose query.
UserModel.find().lean().exec(function (err, users) {
return res.json(users);
}
It worked for me:
Products.find({}).then(a => console.log(a.map(p => p.toJSON())))
also if you want use getters, you should add its option also (on defining schema):
new mongoose.Schema({...}, {toJSON: {getters: true}})
Try this options:
UserModel.find({}, function (err, users) {
//i got into errors using so i changed to res.send()
return res.send( JSON.parse(JSON.stringify(users)) );
//Or
//return JSON.parse(JSON.stringify(users));
}
Was kinda laughing at how cumbersome this was for a second, given that this must be extremely common.
Did not bother digging in the docs and hacked this together instead.
const data = await this.model.logs.find({ "case_id": { $regex: /./, $options: 'i' }})
let res = data.map(e=>e._doc)
res.forEach(element => {
//del unwanted data
delete element._id
delete element.__v
});
return res
First i get all docs which have any value at all for the case_id field(just get all docs in collection)
Then get the actual data from the mongoose document via array.map
Remove unwanted props on object by mutating i directly