Angularjs: Filtering JSON - json

The structure of the json response I have to work with is giving me all sorts of trouble.
I would like to filter children elements based on the entityType field.
For example I want only element of Folder type.
JSON:
{
"id" : "df1d2550-1442-41b8-9588-785e229c5728",
"path" : "",
"name" : "",
"entityType" : "Folder",
"children" : {
"child1" : {
"id" : "02427ae5-364e-47d0-8998-0876c596d586",
"name" : "child1",
"entityType" : "Book"
},
"child2" : {
"id" : "2bcef8b3-d3a3-410e-a481-a69ec7dce24d",
"name" : "child2",
"entityType" : "Letter"
},
"child3" : {
"id" : "12ee8334-d596-4b59-a09d-c286117f3966",
"name" : "child2",
"entityType" : "Book"
},
"Art" : {
"id" : "e3f2980e-433c-4eaa-b444-ed9702949ffc",
"name" : "Art",
"entityType" : "Folder"
},
"English" : {
"id" : "8fe0f14a-6f76-41aa-9ab3-3e63cd5a900b",
"name" : "English",
"entityType" : "Folder"
}
},
"properties" : { },
"ancestors" : [ ]
};
The JS code:
$scope.isFolder = function(item) {
return item.entityType === "Folder";
};
HTML
<div ng-repeat="item in library.children | filter:isFolder">
<pre>{{item.name}} - {{item.entityType}}</pre>
</div>
This code will display all the children when I only want 2.
Any idea what I am doing wrong ?
Plunker
Thanks

The reason your filter doesn't work is that Filter works on arrays but you have an object literal.
So you can either convert your object literal into an array like:
{
"id" : "df1d2550-1442-41b8-9588-785e229c5728",
"path" : "",
"name" : "",
"entityType" : "Folder",
"children" : [
{
"id" : "02427ae5-364e-47d0-8998-0876c596d586",
"name" : "child1",
"entityType" : "Book"
},
{
"id" : "2bcef8b3-d3a3-410e-a481-a69ec7dce24d",
"name" : "child2",
"entityType" : "Letter"
},
{
"id" : "12ee8334-d596-4b59-a09d-c286117f3966",
"name" : "child2",
"entityType" : "Book"
},
{
"id" : "e3f2980e-433c-4eaa-b444-ed9702949ffc",
"name" : "Art",
"entityType" : "Folder"
},
{
"id" : "8fe0f14a-6f76-41aa-9ab3-3e63cd5a900b",
"name" : "English",
"entityType" : "Folder"
}
],
"properties" : { },
"ancestors" : [ ]
};
You can use below code to convert children's properties to an array:
var arr = [];
for (var key in library.children) {
if (obj.hasOwnProperty(key)) {
obj[key]["key"] = key
arr.push(obj[key]);
}
};
library.children = arr;
Or create your own filter than takes in the object literal as callum linington do.

I went through the documentation. Although it is not as explicit as you would expect, looks like the filter function will only work on Array elements and not objects.
In your example, library.children is an object, not an array. Hence, the filter function is never called (which you can check by adding a console statement inside the function).
The specific part in the documentation which mentions that the filter function is called on Array can be found here. You will find it in the "Argument" section, "expression" row - look for the function description. It says that the function is called on Arrays and not Objects, which is your scenario.

So you can easily solve this by writing your own filter here is your adapted plnkr
Simple filter:
app.filter('byKey', function () {
return function (array, key, value) {
var take = [];
angular.forEach(array, function (e) {
if (e[key] === value){
take.push(e);
}
});
return take;
}
});
Simple usage:
<div ng-repeat="item in library.children | byKey: 'entityType' : 'Folder'">
<pre>{{item.name}} - {{item.entityType}}</pre>
</div>
The breakdown:
return function (input, arg1.....argn) {
}
The above return function in the filter is what drives the filter, the input is what you specify on the left side of the | (pipe), in your case: item in library.children.
It's the right side that confuses people. The first string should be the filter name, in my case byKey as that is what I have named it. Each time you use : you are allowing yourself to pass an arg.
So, I have specified this:
return function (array, key, value) {}
The first argument - array, is specified by the left of the |. The remaining args are specified each time you use :, in your case
byKey : 'entityType' : 'Folder'
byKey = filter, 'entityType' = key, 'Folder' = the value of the key

Related

How to retrieve all key-value pairs avoiding key duplication from JSON in Groovy script

I am totally new to groovy script and would like some help to solve this out. I have a JSON response I want to manipulate and get desired parameters back by avoiding duplication. The Json response does not have indexes like 0,1,2.. that I can iterate through.
Here is the response that I want to work with:
{
"AuthenticateV2" : {
"displayName" : "Verification of authentication",
"description" : "notification ",
"smsTemplate" : "authentication.v2.0_sms",
"emailHeaderTemplate" : "v2.0_header",
"emailBodyTemplate" : "html",
"parameters" : {
"displayName" : "USER_DISPLAY_NAME",
"actionTokenURL" : "VERIFICATION_LINK",
"customToken" : "VERIFICATION_CODE"
},
"supportedPlans" : [
"connectGo"
]
},
"PasswordRecovery" : {
"displayName" : "Verification of password recovery",
"description" : "notification",
"smsTemplate" : "recovery.v1.0_sms",
"emailHeaderTemplate" : "recovery.v1.0_header",
"emailBodyTemplate" : "recovery.v1.0_body_html",
"parameters" : {
"displayName" : "USER_DISPLAY_NAME",
"actionTokenURL" : "VERIFICATION_LINK",
"customToken" : "VERIFICATION_CODE",
"adminInitiated" : false,
"authnId" : "AUTHENTICATION_IDENTIFIER",
"authnType" : "EMAIL",
"user" : {
"displayName" : "USER_DISPLAY_NAME"
}
},
"supportedPlans" : [
"connectGo"
]
},
"PasswordReset" : {
"displayName" : "password reset",
"description" : "notification",
"smsTemplate" : "recovery.v1.0_sms",
"emailHeaderTemplate" : "recovery.v1.0_header",
"emailBodyTemplate" : "html",
"parameters" : {
"displayName" : "USER_DISPLAY_NAME",
"user" : {
"displayName" : "USER_DISPLAY_NAME"
}
}
The expected output that I want to have:
{
"displayName" : "USER_DISPLAY_NAME",
"actionTokenURL" : "VERIFICATION_LINK",
"customToken" : "VERIFICATION_CODE",
"customToken" : "VERIFICATION_CODE",
"adminInitiated" : false,
"authnId" : "AUTHENTICATION_IDENTIFIER",
"authnType" : "EMAIL"
}
I need to retrieve all fields under parameters tag and also want to avoid duplication
You should first get familiar with parsing and producing JSON in Groovy.
Then, assuming the provided response is a valid JSON (it's not - there are 2 closing curlies (}) missing at the end) to get all the parameters keys merged into one JSON we have to convert the JSON string into a Map object first using JsonSlurper:
def validJsonResponse = '<your valid JSON string>'
Map parsedResponse = new JsonSlurper().parseText(validJsonResponse) as Map
Now, when we have a parsedResponse map we can iterate over all the root items in the response and transform them into the desired form (which is all the unique parameters keys) using Map::collectEntries method:
Map uniqueParameters = parsedResponse.collectEntries { it.value['parameters'] }
Finally, we can convert the uniqueParameters result back into a pretty printed JSON string using JsonOuput:
println JsonOutput.prettyPrint(JsonOutput.toJson(uniqueParameters))
After applying all the above we'll get the output
{
"displayName": "USER_DISPLAY_NAME",
"actionTokenURL": "VERIFICATION_LINK",
"customToken": "VERIFICATION_CODE",
"adminInitiated": false,
"authnId": "AUTHENTICATION_IDENTIFIER",
"authnType": "EMAIL",
"user": {
"displayName": "USER_DISPLAY_NAME"
}
}
If you want to get rid of user entry from the final output just remove it from the resulting uniqueParameters map (uniqueParameters.remove('user')) before converting it back to JSON string.

Splitting Json to multiple jsons in NIFI

I have the below json file which I want to split in NIFI
Input:
[ {
"id" : 123,
"ticket_id" : 345,
"events" : [ {
"id" : 3322,
"type" : "xyz"
}, {
"id" : 6675,
"type" : "abc",
"value" : "sample value",
"field_name" : "subject"
}, {
"id" : 9988,
"type" : "abc",
"value" : [ "text_file", "json_file" ],
"field_name" : "tags"
}]
}]
and my output should be 3 different jsons like below:
{
"id" : 123,
"ticket_id" : 345,
"events.id" :3322,
"events.type":xyz
}
{
"id" : 123,
"ticket_id" : 345,
"events.id" :6675,
"events.type":"abc",
"events.value": "sample value"
"events.field_name":"subject"
}
{
"id" : 123,
"ticket_id" : 345,
"events.id" :9988,
"events.type":"abc",
"events.value": "[ "text_file", "json_file" ]"
"events.field_name":"tags"
}
I want to know can we do it using splitjson? I mean can splitjson split the json based on the array of json objects present inside the json?
Please let me know if there is a way to achieve this.
If you want 3 different flow files, each containing one JSON object from the array, you should be able to do it with SplitJson using a JSONPath of $ and/or $.*
Using reduce function:
function split(json) {
return json.reduce((acc, item) => {
const events = item.events.map((evt) => {
const obj = {id: item.id, ticket_id: item.ticket_id};
for (const k in evt) {
obj[`events.${k}`] = evt[k];
}
return obj;
});
return [...acc, ...events];
}, []);
}
const input = [{"id":123,"ticket_id":345,"events":[{"id":3322,"type":"xyz"},{"id":6675,"type":"abc","value":"sample value","field_name":"subject"},{"id":9988,"type":"abc","value":["text_file","json_file"],"field_name":"tags"}]}];
const res = split(input);
console.log(res);

jq : mapping array in object to another object while keeping parent keys and adding new ones

I would like to map the following structure
{
"id" : "OUTER_ID",
"name" : "OUTER_NAME"
"items" : [
{
"id" : "INNER_ID_1",
"name" : "INNER_NAME_1",
},
{
"id" : "INNER_ID_2",
"name" : "INNER_NAME_2",
}
]
}
into this
{
"payload": [
{
"key" : "INNER_NAME_1_KEY",
"data" : {
"id" : "OUTER_ID",
"name" : "OUTER_NAME",
"items" : [
{
"id" : "INNER_ID_1",
"name" : "INNER_NAME_1"
}
]
}
},
{
"key" : "INNER_NAME_2_KEY",
"data" : {
"id" : "OUTER_ID",
"name" : "OUTER_NAME",
"items" : [
{
"id" : "INNER_ID_2",
"name" : "INNER_NAME_2"
}
]
}
}
]
}
So, for each item in the initial items array, I want to create an entry in the output's payload, i.e I want to map items[i] to payload[i].data.items while also creating the payload, key and data keys in the output, and setting payload[i].data.id and payload[i].data.name to the input's outer id and name.
Can this be done with jq?
Sure, you can use the following filter :
.id as $id | .name as $name | {payload : [ .items[] | {key:.id, data:{id:$id, name: $name, items:[.]}} ] }
You can try it here.

NiFi: Extract Content of FlowFile and Add that Content to the Attributes

I am generating random data from the following JSON/AVRO schema:
{
"type" : "record",
"namespace" : "test",
"name" : "metro_data",
"fields": [
{
"name" : "PersonID",
"type" : "int"
},
{
"name" : "TripStartStation",
"type" : {
"type" : "enum",
"name" : "StartStation",
"symbols" : ["WIEHLE_RESTON_EAST", "SPRING_HILL", "GREENSBORO"]
}
},
{
"name" : "TripEndStation",
"type" : {
"type" : "enum",
"name" : "EndStation",
"symbols" : ["WIEHLE_RESTON_EAST", "SPRING_HILL", "GREENSBORO""]
}
}
]
}
The above schema generates this, for example:
[ {
"PersonID" : -1089196095,
"TripStartStation" : "WIEHLE_RESTON_EAST",
"TripEndStation" : "SPRING_HILL"
}
I want to take the PersonID number of the schema, and add it to the Attributes. Eg, the blank in this photo needs to pull the actual PersonID number generated from the flow:
I have tried to use EvaluateJSONPath with the following configuration, and that's how I end up with the empty string set under PersonalID:
Is my next processor UpdateAttribute? Not sure how to pull that content. Thanks!
You are having array of json message(s)(ex: [...]) and You need to split the array of json into individual flowfiles using SplitJson processor with split expression as $.*
Then use EvaluateJsonProcessor to extract PersonID value as a attribute.
Flow:
--> SplitJson --> EvaluateJsonPath--> other processors
For more details refer to this link regards to the same issue.

mongo db remove json objects

I have a mongo json object as follows
{
"_id" : new BinData(3, "RDHABb22XESWvP83FplqJw=="),
"name" : "NEW NODE",
"host" : null,
"aet" : null,
"studies" : ["1.3.12.2.1107.5.99.3.30000008061114424970500000589"],
"testcases" : [new BinData(3, "Zhl+zIXomkqAd8NIkRiTjQ==")],
"sendentries" : [{
"_id" : "1.3.12.2.1107.5.99.3.30000008061114424970500000589",
"Index" : 0,
"Type" : "Study"
}, {
"_id" : "cc7e1966-e885-4a9a-8077-c3489118938d",
"Index" : 1,
"Type" : "TestCase"
}]
}
The fields "Studies" and "TestCases" are now obsolete and I am now storing that information in a new field called SendEntries. I would like to get rid of the Studies and TestCases from the old entries and unmap those fields going forward. I want to know how I can update my current collections to get rid of the Studies and TestCases fields.
I'm just few weeks into Mongo.
You can use the $unset operator with update.
db.collection.update({},
{ $unset: {
"studies": "",
"testcases": ""
},
{ "upsert": false, "muti": true }
)
And that will remove all of the fields from all of your documents in your collection
Use $unset, there's a manual page e.g.
db.yourCollection.update( { },
{ $unset: {
Studies: "",
testcases: ""
}
},
{ multi: true }
)