I would like to build a tree structre from a plain json array.
The regular depth is approx. 6/7 (max 10) and has about 5,000 records.
My input json looks like this
[3,"01","GruppenAnfangHook",1,0,1,0,"Installationsmaterial",1.0,"",null,null,0.0,-1.0,null,803.0300,803.0300,0.00000,1,1]
[5,"01.001","JumboAnfangHook",3,0,3,0,"MBS Wandler 1.000",6.0,"St",null,null,0.0,-6.0,0.0000,336.7800,56.1300,0.00000,2,2],
[38,"","ArtikelHook",3,5,3,0,"ASK 61.4 1000/5A 5VA Kl.1 Preis lt. Hr. K am 16.05.17",6.0,"stk",6.0,6.0,0.0,-6.0,null,21.5000,21.5000,0.00000,3,3]
But I need it structured with childrens like that
{"0":34,1":"02.003",2":"JumboBegin","3":26,"4":0, "5":26,"6":0, "children":[
{ "0":36,"1":"", "2":"Article","3":26,"4":34,"5":26,6:"0", 7: "Artikel"},
{ "0":35,"1":"", "2":"JumboEnd",3":26,"4":34, "5":26, 6:"0",7:"Stunde"}
]}
My best approach so far was to build the child-structure with the following JS function in the frontend
function nest(data, parentId = 0) {
return data.reduce((r, e) => {
let obj = Object.assign({}, e)
if (parentId == e[4]) {
let children = nest(data, e[0])
if (children.length) obj.children = children
r.push(obj)
}
return r;
}, [])}
It works well and fast (< 1s) with a small (<500) amount of records but my browser begins to freeze at 2,000 and above.
My thought was it is too much data and so I tried to solve it in the CFML backend.
Due to I'm new with recursion, Ben Nadels Blog helped me alot, so I used his post about recursion and created a working example with sample data.
q = queryNew("id,grpCol,jumCol,leiCol,name,typ,order");
The grpCol is level 0, up to 5 groups can be placed in each other, in those groups can be placed two kinds of containers (jumCol and leiCol), they can be placed in each other to, but not in themselfs.
But now I am failing to convert it to a array of structures with child members. The structure of the HTML tree generated as output in the example is exactly what I want for my frontend JSON.
Because of the recursion I don't get, how to store it in an array outside of the function.
My goal is a final return as serzializeJson(array).
Related
I'm doing a SQL query in Node-Red to output a load of time/value data. This data is then passed to a web page for display in a graph.
Previously I've used php to do the SQL query, which I'm trying to replace. However SQL queries in php are delivered in a different format.
With Node-Red, I get:
[
{
"Watts": 1018,
"Time": 1453825454
},
{
"Watts": 1018,
"Time": 1453825448
},
{
"Watts": 1010,
"Time": 1453825442
}]
With PHP, I get:
[
[1453819620000,962],
[1453819614000,950],
[1453819608000,967],
[1453819602000,947]
]
I think I'm getting an array from php and an array of JSON objects from Node-Red. How do I convert the Node-Red object to be output from Node-Red in the same format as the PHP is? (Ie: I want to handle the processing at the server, rather than the client.)
A function node can be used to generate something in the same format.
var array = msg.payload;
var phpFormat = "[";
for (var i=0; i<array.length; i++) {
phpFormat += "[" +
// time format differ, NodeJS is in seconds
// php is in milliseconds
(array[i].Time * 1000 ) +
"," +
array[i].Watts + "],";
}
//take the last "," off
phpFormat = phpFormat.substring(0,phpFormat.lenght - 1);
phpFormat += "]";
msg.payload = phpFormat;
return msg;
I've had a bit of help from a chap at work and here is what he's come up with, modified for node-red by me:
var outputArray = [];
for(var i in msg.payload){
var entryData = [msg.payload[i]['Time']];
for(var attr in msg.payload[i]) {
if(attr!='Time') {
entryData.push(msg.payload[i][attr])}
};
outputArray.push(entryData); }
var returnMsg={"payload":outputArray};
return returnMsg;
I know, I know, this question is over 2 years old... however, for the next 500 people seeking an answer to a similar problem, I'd like to highlight the new JSONata expression feature built-in to the change node. Using this simple expression:
payload.[Time, Watts]
transforms your JS objects into the requested output of an array of arrays. In fact, much of my old repetitive looping through arrays has been replaced with some simpler (to me) expressions like this.
The magic of the lambda syntax evaluator is documented on the JSONata site. There you will also find the online exerciser where you can build an expression against your own data and immediately see the resulting structure.
Note: in order to use a jsonata expression in your change node, be sure to select the J: pulldown next to the input field (not the {} JSON option)... two totally different things!
I am building a CSV file parser through node and Angular . so basically a user upload a csv file , on my server side which is node the csv file is traversed and parsed using node-csv
. This works fine and it returns me an array of object based on csv file given as input , Now on angular end I need to display two table one is csv file data itself and another is cross tabulation analysis. I am facing problem while rendering data, so for a table like
I am getting parse responce as
For cross tabulation we need data in a tabular form as
I have a object array which I need to manipulate in best possible way so as to make easily render on html page . I am not getting a way how to do calculation on data I get so as to store cross tabulation result .Any idea on how should I approach .
data json is :
[{"Sample #":"1","Gender":"Female","Handedness;":"Right-handed;"},{"Sample #":"2","Gender":"Male","Handedness;":"Left-handed;"},{"Sample #":"3","Gender":"Female","Handedness;":"Right-handed;"},{"Sample #":"4","Gender":"Male","Handedness;":"Right-handed;"},{"Sample #":"5","Gender":"Male","Handedness;":"Left-handed;"},{"Sample #":"6","Gender":"Male","Handedness;":"Right-handed;"},{"Sample #":"7","Gender":"Female","Handedness;":"Right-handed;"},{"Sample #":"8","Gender":"Female","Handedness;":"Left-handed;"},{"Sample #":"9","Gender":"Male","Handedness;":"Right-handed;"},{"Sample #":";"}
There are many ways you can do this and since you have not been very specific on the usage, I will go with the simplest one.
Assuming you have an object structure such as this:
[
{gender: 'female', handdness: 'lefthanded', id: 1},
{gender: 'male', handdness: 'lefthanded', id: 2},
{gender: 'female', handdness: 'righthanded', id: 3},
{gender: 'female', handdness: 'lefthanded', id: 4},
{gender: 'female', handdness: 'righthanded', id: 5}
]
and in your controller you have exposed this with something like:
$scope.members = [the above array of objects];
and you want to display the total of female members of this object, you could filter this in your html
{{(members | filter:{gender:'female'}).length}}
Now, if you are going to make this a table it will obviously make some ugly and unreadable html so especially if you are going to repeat using this, it would be a good case for making a directive and repeat it anywhere, with the prerequisite of providing a scope object named tabData (or whatever you wish) in your parent scope
.directive('tabbed', function () {
return {
restrict: 'E',
template: '<table><tr><td>{{(tabData | filter:{gender:"female"}).length}}</td></tr><td>{{(tabData | filter:{handedness:"lefthanded"}).length}}</td></table>'
}
});
You would use this in your html like so:
<tabbed></tabbed>
And there are ofcourse many ways to improve this as you wish.
This is more of a general data structure/JS question than Angular related.
Functional helpers from Lo-dash come in very handy here:
_(data) // Create a chainable object from the data to execute functions with
.groupBy('Gender') // Group the data by its `Gender` attribute
// map these groups, using `mapValues` so the named `Gender` keys persist
.mapValues(function(gender) {
// Create named count objects for all handednesses
var counts = _.countBy(gender, 'Handedness');
// Calculate the total of all handednesses by summing
// all the values of this named object
counts.Total = _(counts)
.values()
.reduce(function(sum, num) { return sum + num });
// Return this named count object -- this is what each gender will map to
return counts;
}).value(); // get the value of the chain
No need to worry about for-loops or anything of the sort, and this code also works without any changes for more than two genders (even for more than two handednesses - think of the aliens and the ambidextrous). If you aren't sure exactly what's happening, it should be easy enough to pick apart the single steps and their result values of this code example.
Calculating the total row for all genders will work in a similar manner.
I am using Jackson to parse JSON from a json inputStream which looks like following:
[
[ 36,
100,
"The 3n + 1 problem",
56717,
0,
1000000000,
0,
6316,
0,
0,
88834,
0,
45930,
0,
46527,
5209,
200860,
3597,
149256,
3000,
1
],
[
........
],
[
........
],
.....// and almost 5000 arrays like above
]
This is the original feed link: http://uhunt.felix-halim.net/api/p
I want to parse it and keep only the first 4 elements of every array and skip other 18 elements.
36
100
The 3n + 1 problem
56717
Code structure I have tried so far:
while (jsonParser.nextToken() != JsonToken.END_ARRAY) {
jsonParser.nextToken(); // '['
while (jsonParser.nextToken() != JsonToken.END_ARRAY) {
// I tried many approaches here but not found appropriate one
}
}
As this feed is pretty big, I need to do this efficiently with less overhead and memory.
Also there are three models to procress JSON: Streaming API, Data Binding and Tree Model. Which one is appropriate for my purpose?
How can I parse this json efficiently with Jackson? How can I skip those 18 elements and jump to next array for better performance?
Edit: (Solution)
Jackson and GSon both works in almost in the same mechanism (incremental mode, since content is read and written incrementally), I am switching to GSON as it has a function skipValue() (pretty appropriate with name). Although Jackson's nextToken() will work like skipValue(), GSON seems more flexible to me. Thanks #Kowser bro for his recommendation, I came to know about GSON before but somehow ignored it. This is my working code:
reader.beginArray();
while (reader.hasNext()) {
reader.beginArray();
int a = reader.nextInt();
int b = reader.nextInt();
String c = reader.nextString();
int d = reader.nextInt();
System.out.println(a + " " + b + " " + c + " " + d);
while (reader.hasNext())
reader.skipValue();
reader.endArray();
}
reader.endArray();
reader.close();
This is for Jackson
Follow this tutorial.
Judicious use of jasonParser.nextToken() should help you.
while (jasonParser.nextToken() != JsonToken.END_ARRAY) { // might be JsonToken.START_ARRAY?
The pseudo-code is
find next array
read values
skip other values
skip next end token
This is for gson.
Take a look at this tutorial. Consider following second example from the tutorial.
Judicious use of reader.begin* reader.end* and reader.skipValue should do the job for you.
And here is the documentation for JsonReader
I have a jquery third party application that has nested lists that serialize to an output like below. The list will always only have 2 levels, but I am having trouble trying to figure out how to parse it. I am using coldFusion.
The List Looks like (line breaks added for visualization, they cannot be used as a delimiter):
[{"id":1},
{"id":197,"children":[{"id":198},{"id":199},{"id":200}]},
{"id":2,"children":[{"id":3},{"id":4},{"id":143},{"id":6},{"id":5},{"id":7},{"id":8},{"id":9},{"id":10},{"id":11},{"id":12}]},
{"id":15,"children":[{"id":17},{"id":190},{"id":191},{"id":131},{"id":16},{"id":142},{"id":124}]},
{"id":114}]
I want to loop through each id and convert into a parentid and childid like so:
id:1 parentid: 10000 childid: 10000
id:197 parentid: 10001 childid: 10000 (new parent)
id:198 parentid: 10001 childid: 10001 (first child)
id:199 parentid: 10001 childid: 10002 (second child)
id:200 parentid: 10001 childid: 10003 (third child)
id:2 parentid: 10002 childid: 10000 (new parent)
... and so on
Your help is appreciated.
Edit: Code is below for what I am trying to do
<script type="text/javascript">
$(document).ready(
function()
{
var updateOutput = function(e)
{
var list = e.length ? e : $(e.target),
output = list.data('output');
if (window.JSON) {
output.val(window.JSON.stringify(list.nestable('serialize')));//, null, 2));
} else {
output.val('JSON browser support required for this demo.');
}
};
//this is where i need help
var postOutline = function(output){
$.post("something.cfc", {
method: 'getoutline',
output: output
});
};
// activate Nestable for list 1
$('#nestable3').nestable({
group: 1
})
// .on('change', updateOutput);
.on('change', postOutline);
// output initial serialised data
updateOutput($('#nestable3').data('output', $('#nestable-output')));
}
);
</script>
You just need to use deserializeJson(). You don't need to parse it by hand. From the docs:
Description Converts a JSON (JavaScript Object Notation) string data
representation into CFML data, such as a CFML structure or array.
From there you simply need to use your usual CFML to process it however you like.
Finding the children for each ID is easy as it's in the data structure.
Unfortunately the data structure is not ideal for extracting parent information.
The most expedient way I can think of is to use structFindValue() to find all the occurrences of the current ID, and then loop over that finding the entry for which the match has a descendant children". Then traverse across to the ID, which will be the parent ID of those children (if that makes sense).
(the above initial suggestion won't work, as structFindValue() doesn't give enough information).
You're gonna need to brute-force this, doing something like this:
array = deserializeJson(json); // json is your data from the client
for (childElement in array){
childId = childElement.id;
for (parentElement in array){
if (structKeyExists(parentElement, "children")){ // not all of them have children
if (arrayFind(parentElement.children, {id=childId})){
writeOutput("Parent of #childId# is #parentElement.id#<br>");
}
}
}
}
Obviously that's not a precise solution for what you need, but it shows the technique for looking-up the parent. Someone else might be able to come up with a less ham-fisted way of doing it.
I've got two model/proxy/stores I'm concerned with Questions and Choices. Both get data from a REST server as JSON. My process currently goes like this:
// load numQuestions records from store.Questions
var qs = Ext.getStore('Question');
//... loadmask, etc.
qs.load({
scope : this,
params : {
limit : numQuestions
},
callback : function() {
this.createQuestionCards(numQuestions);
}
});
Once I have the Questions, I loop through and fetch the Choices that are relevant to each Question like:
for ( i = 0; i < numQuestions; i++) {
// ... misc ...
Assessor.questionChoices[i] = qs.getAt(i).choices();
// ...misc...
},
This works well, except that it makes an XMLHTTPRequest for every loop iteration. With minimum response times in the 0.15 sec area, that is fine for N < ~40. Once the numbers get to 200, which should be a common use case, the delay is nasty.
How do I get ExtJS to "batch" the requests and send them after the loop body? For example:
var choiceBatch = qs.createBatch();
for ( i = 0; i < numQuestions; i++) {
// ... misc ...
Assessor.questionChoices[i] = choiceBatch.getAt(i).choices();
// ...misc...
};
choiceBatch.execute();
The Ext.data.proxy.Rest has a config option batchActions and since it's basically an AjaxProxy with different methods it will probably work in the same way as the AjaxProxy.
Since I am not getting clear answer about restful batch with multipart...
testing on my own with batchActions=true in Ext.data.proxy.Rest v4.2.1 result that batch is only within the same store and HTTP method. (batchActions default to false for the REST)
That means if there is 200 post & 1 delete and you call store.sync(), it will batch into 2 request, the POST request body will be wrapped with an array of records instead of single record.
I am looking for if it can batch all stores with all GET, POST, PUT and DELETE by using multipart/mixed but the result is negative. (check out OData Batch Processing)
Regarding the OP, what you looking for is the model associations. Once you create Questions and Choices Ext model and let the server respond with nested json data (So the Questions contain the child Choices embedded in a request) Ext will create question record along with question.choices() child store automatically.