I'm trying to transform the following JSON
{
"data": {
"keyvalues": [
{
"key": "location",
"value": "sydney, au"
},
{
"key": "weather",
"value": "sunny"
}
]
},
"food": {
"name": "AllFoods",
"date": "2018-03-08T09:35:17-03:00",
"count": 2,
"food": [
{
"name": "chocolate",
"date": "2018-03-08T12:59:58-03:00",
"rating": "10",
"data": null
},
{
"name": "hot dog",
"date": "2018-03-08T09:35:17-03:00",
"rating": "7",
"data": {
"keyvalues": [
{
"key": "topping",
"value": "mustard"
},
{
"key": "BUN type",
"value": "toasted"
},
{
"key": "servings",
"value": "2"
}
]
}
}
]
}
}
Into, something simpler like this, using JOLT (in NIFI). Bringing the first top-level food attributes (name, date, count) into the header and then pulling the nested food array up, and then flattening out the food.data.keyvalues into a dict/hashmap.
{
"header": {
"location": "sydney, au",
"weather": "sunny",
"date": "2018-03-08",
"count": 2
},
"foods": [
{
"name": "chocolate",
"date": "2018-03-08T12:59:58-03:00",
"rating": "10"
},
{
"name": "hot dog",
"date": "2018-03-08T09:35:17-03:00",
"rating": "7",
"topping": "mustard",
"bun_type": "toasted",
"servings": "2"
}
]
}
I've got the first data part working, but I'm not sure how to handle the nested food element. The top level food info needs to move into the header section, and the second level food array, needs to flatten out the data.keyvalues.
Current spec... (only handles the top data.keyvalues)
[
{
"operation": "shift",
"spec": {
"data": {
"keyvalues": {
"*": { "#value": "#key" }
}
}
}
}
]
Spec
[
{
"operation": "shift",
"spec": {
"data": {
"keyvalues": {
"*": {
"value": "header.#(1,key)"
}
}
},
"food": {
"date": "header.date",
"count": "header.count",
"food": {
"*": {
"name": "foods[&1].name",
"date": "foods[&1].date",
"rating": "foods[&1].rating",
"data": {
"keyvalues": {
"*": {
"value": "foods[&4].#(1,key)"
}
}
}
}
}
}
}
}
]
Related
I am trying to write Jolt spec for the following input. I need to populate the primaryEmail field based on the condition if primary field is true in the emails array
[
{
"uid": "1234mark",
"name": "mark",
"userName": "markw",
"displayName": "Mark W",
"emails": [
{
"primary": false,
"value": "mark#gmail.com"
},
{
"primary": true,
"value": "mark#hotmail.com"
}
]
},
{
"uid": "9876steve",
"name": "steve",
"userName": "stevew",
"displayName": "Steve W",
"emails": [
{
"primary": false,
"value": "steve#gmail.com"
},
{
"primary": true,
"value": "steve#hotmail.com"
}
]
}
]
The desired output is
[
{
"user": {
"externalId": "1234mark",
"name": "mark",
"userName": "markw",
"displayName": "Mark W",
"primaryEmail": "mark#hotmail.com"
}
},
{
"user": {
"externalId": "9876steve",
"name": "steve",
"userName": "stevew",
"displayName": "Steve W",
"primaryEmail": "steve#hotmail.com"
}
}
]
But I get the following incorrect output since I am not able to populate the primaryEmail field conditionally properly.
[
{
"user": {
"externalId": "1234mark",
"name": "mark",
"userName": "markw",
"displayName": "Mark W"
}
},
{
"user": {
"externalId": "9876steve",
"name": "steve",
"userName": "stevew",
"displayName": "Steve W"
}
}
]
The spec I have created is the following
[
{
"operation": "shift",
"spec": {
"*": {
"uid": "[&1].user.externalId",
"name": "[&1].user.name",
"userName": "[&1].user.userName",
"displayName": "[&1].user.displayName",
"title": "[&1].user.title",
"emails": {
"*": {
"primary": {
"true": {
"#(2,value)": "primaryEmail"
}
}
}
}
}
}
}
]
Could someone please help with this query. Thanks.
What you need is to go 5 levels the three up from the innermost object while adding an extra node called user such as
[
{
"operation": "shift",
"spec": {
"*": {
"uid": "[&1].user.externalId",
"*": "[&1].user.&", // the attributes except for "uid" and "emails" array
"emails": {
"*": {
"primary": {
"true": {
"#(2,value)": "[&5].user.&2Email" // replicate literal "primary" by using &2
}
}
}
}
}
}
}
]
the demo on the site http://jolt-demo.appspot.com/ is
My input JSON is like
{
"common": {
"name": "abc"
},
"details": {
"id": 4,
"node": [
{
"name": "node1",
"array2": []
},
{
"name": "node2",
"array2": [
{
"name": "node2_a2_1"
}
]
},
{
"name": "node3",
"array2": [
{
"name": "node3_a2_1"
},
{
"name": "node3_a2_2"
},
{
"name": "node3_a2_3"
}
]
}
]
}
}
What I want is for each leaf node in array2 (e.g. {"name": "node3_a2_1"}) I will traverse towards the root and add all common items and create an array of JSON objects without any nested object. So, the output I want is like
[
{
"common_name": "abc",
"id": 4,
"node_name": "node2",
"name": "node2_a2_1"
},
{
"common_name": "abc",
"id": 4,
"node_name": "node3",
"name": "node3_a2_1"
},
{
"common_name": "abc",
"id": 4,
"node_name": "node3",
"name": "node3_a2_2"
},
{
"common_name": "abc",
"id": 4,
"node_name": "node3",
"name": "node3_a2_3"
}
]
Could you please suggest how can I do that?
You can walk through the array2 while picking values of each element from their original location in the JSON value.
For example; traverse } four times to grab the value of id by using #(4,id), and use #(3,name)[&1] as the common distinguishing identifier for each attribute such as
[
{
"operation": "shift",
"spec": {
"details": {
"node": {
"*": {
"array2": {
"*": {
"#(5,common.name)": "#(3,name)[&1].common_name",
"#(4,id)": "#(3,name)[&1].id",
"#(2,name)": "#(3,name)[&1].node_name",
"name": "#(3,name)[&1].&"
}
}
}
}
}
}
},
{
"operation": "shift",
"spec": {
"*": {
"*": ""
}
}
}
]
I am trying to transform the following object into an array, the object is formatted already as required just needs to be output as an array containing that object.
[
{
"PLCTime": 1643804542000,
"LevelID": "53.99.2",
"Data1Type": "Axis1 Dist",
"Data1": 1,
"Data2Type": "Axis2 Dist",
"Data2": 2,
"Data3Type": "Axis3 Dist",
"Data3": 3,
"Data4Type": "Axis4 Dist",
"Data4": 4,
"Data5Type": "Axis5 Dist",
"Data5": 5.5,
"Data6Type": "Axis6 Dist",
"Data6": 6
}
]
I would like the output to be an array containing the one object. Currently my spec is:
[
{
"operation": "shift",
"spec": {
"*": {
"Data*Type": {
"#(0)": "name"
},
"Data*": {
"#(0)": "value"
},
"*": "&"
}
}
},
{
"operation": "shift",
"spec": {
"LevelID": "assetId",
"PLCTime": "dataPoints[].timestamp",
"name": {
"*": {
"#": "dataPoints[0].measures[&].&2",
"#(3,value[&])": "dataPoints[0].measures[&].value"
}
}
}
}
]
Which gets me the following, but you can see the result is not an array.
{
"assetId": "53.99.2",
"dataPoints": [
{
"timestamp": 1643804542000,
"measures": [
{
"name": "Axis1 Dist",
"value": 1
},
{
"name": "Axis2 Dist",
"value": 2
},
{
"name": "Axis3 Dist",
"value": 3
},
{
"name": "Axis4 Dist",
"value": 4
},
{
"name": "Axis5 Dist",
"value": 5.5
},
{
"name": "Axis6 Dist",
"value": 6
}
]
}
]
}
The output I am trying for is :
[
{
"assetId": "53.99.2",
"dataPoints": [
{
"timestamp": 1643804542000,
"measures": [
{
"name": "Axis1 Dist",
"value": 1
},
{
"name": "Axis2 Dist",
"value": 2
},
{
"name": "Axis3 Dist",
"value": 3
},
{
"name": "Axis4 Dist",
"value": 4
},
{
"name": "Axis5 Dist",
"value": 5.5
},
{
"name": "Axis6 Dist",
"value": 6
}
]
}
]
}
]
e.g which needs to be enclosed in a set of []
You can
prefix the identifiers on the right-hand side with [0]. for the
second tranformation spec in order to nest the whole content within
the square brackets such as
{
"operation": "shift",
"spec": {
"LevelID": "[0].assetId",
"PLCTime": "[0].dataPoints[].timestamp",
"name": {
"*": {
"#": "[0].dataPoints[0].measures[&].&2",
"#(3,value[&])": "[0].dataPoints[0].measures[&].value"
}
}
}
}
or
add the following spec to the current ones, alternatively
{
"operation": "shift",
"spec": {
"#": "[]"
}
}
I am trying to iterate lists inside a list and form a single list with multiple objects. Iterating lists, I am able to achieve. But applying tags before the list iterating to each object is not happening if there are more objects in single list.
My input request is like below:
[
{
"success": [
{
"id": "4",
"Offers": [
{
"name": "Optional",
"type": {
"id": "1",
"name": "Optional"
},
"productOfferings": [
{
"id": "3",
"name": "Test1"
}
]
},
{
"name": "Default",
"type": {
"id": "2",
"name": "Default"
},
"productOfferings": [
{
"id": "1",
"name": "Test2"
},
{
"id": "2",
"name": "Test3"
}
]
}
]
}
]
}
]
My spec is like below:
[
{
"operation": "shift",
"spec": {
"*": {
"success": {
"*": {
"Offers": {
"*": {
"name": "[&1].[&3].typeName",
"type": {
"id": "[&2].[&4].typeNameId",
"name": "[&2].[&4].typeNameValue"
},
"productOfferings": {
"*": {
"id": "[&3].[&1].id",
"name": "[&3].[&1].name"
}
}
}
}
}
}
}
}
},
{
"operation": "shift",
"spec": {
"*": {
"*": "[]"
}
}
}
]
Output Received from spec:
[
{
"typeName": "Optional",
"typeNameId": "1",
"typeNameValue": "Optional",
"id": "3",
"name": "Test1"
},
{
"typeName": "Default",
"typeNameId": "2",
"typeNameValue": "Default",
"id": "1",
"name": "Test2"
},
{
"id": "2",
"name": "Test3"
}
]
But Expected output is like below:
[
{
"typeName": "Optional",
"typeNameId": "1",
"typeNameValue": "Optional",
"id": "3",
"name": "Test1"
},
{
"typeName": "Default",
"typeNameId": "2",
"typeNameValue": "Default",
"id": "1",
"name": "Test2"
},
{
"typeName": "Default",
"typeNameId": "2",
"typeNameValue": "Default",
"id": "2",
"name": "Test3"
}
]
If there are more objects inside productOfferings object, I am not able to add typeName,typeNameId, typeNameValue to the actual object. Please help to fix this issue.
You seem just needing to collect all into the productOfferings array while prepending each key by the common identifier [&3].[&1] such as
[
{
"operation": "shift",
"spec": {
"*": {
"success": {
"*": {
"Offers": {
"*": {
"productOfferings": {
"*": {
"#(2,name)": "[&3].[&1].typeName",
"#(2,type.id)": "[&3].[&1].typeNameId",
"#(2,type.name)": "[&3].[&1].typeNameValue",
"*": "[&3].[&1].&"
}
}
}
}
}
}
}
}
},
{
"operation": "shift",
"spec": {
"*": {
"*": "[]"
}
}
}
]
I'm looking for a way to copy or move a key from nested object to the top level
Input:
{
"id": "123",
"name": "foo",
"details": {
"orderNumber": "456789",
"addr": "N st 124",
"date": "2021-01-01"
}
}
desired output:
{
"id": "123",
"name": "foo",
"orderNumber": "456789",
"details": {
"orderNumber": "456789",
"addr": "N st 124",
"date": "2021-01-01"
}
}
or ideally
{
"id": "123",
"name": "foo",
"orderNumber": "456789",
"details": {
"addr": "N st 124",
"date": "2021-01-01"
}
}
the closest I could get is below transformation, but it converts object to value array
[
{
"operation": "shift",
"spec": {
"id": "id",
"name": "name",
"details": {
"orderNumber": "orderNumber",
"*": "details"
}
}
}
]
You're so close to the result, just a slight change(adding an ampersand) is needed such as
[
{
"operation": "shift",
"spec": {
"id": "id",
"name": "name",
"details": {
"orderNumber": "orderNumber",
"*": "&1.&"
}
}
}
]
in this case the keys keeps on appearing.