My input JSON is like
{
"common": {
"name": "abc"
},
"details": {
"id": 4,
"node": [
{
"name": "node1",
"array2": []
},
{
"name": "node2",
"array2": [
{
"name": "node2_a2_1"
}
]
},
{
"name": "node3",
"array2": [
{
"name": "node3_a2_1"
},
{
"name": "node3_a2_2"
},
{
"name": "node3_a2_3"
}
]
}
]
}
}
What I want is for each leaf node in array2 (e.g. {"name": "node3_a2_1"}) I will traverse towards the root and add all common items and create an array of JSON objects without any nested object. So, the output I want is like
[
{
"common_name": "abc",
"id": 4,
"node_name": "node2",
"name": "node2_a2_1"
},
{
"common_name": "abc",
"id": 4,
"node_name": "node3",
"name": "node3_a2_1"
},
{
"common_name": "abc",
"id": 4,
"node_name": "node3",
"name": "node3_a2_2"
},
{
"common_name": "abc",
"id": 4,
"node_name": "node3",
"name": "node3_a2_3"
}
]
Could you please suggest how can I do that?
You can walk through the array2 while picking values of each element from their original location in the JSON value.
For example; traverse } four times to grab the value of id by using #(4,id), and use #(3,name)[&1] as the common distinguishing identifier for each attribute such as
[
{
"operation": "shift",
"spec": {
"details": {
"node": {
"*": {
"array2": {
"*": {
"#(5,common.name)": "#(3,name)[&1].common_name",
"#(4,id)": "#(3,name)[&1].id",
"#(2,name)": "#(3,name)[&1].node_name",
"name": "#(3,name)[&1].&"
}
}
}
}
}
}
},
{
"operation": "shift",
"spec": {
"*": {
"*": ""
}
}
}
]
Related
This is an input that needs to be transformed using Jolt Transformation to obtain the expected output.
I am attempting to create a jolt transformation for the below input:
{
"flights": [
{
"id": "123",
"route": "BJS-SIN"
},
{
"id": "456",
"route": "SIN-PEK"
},
{
"id": "789",
"route": "SIN-BJS"
}
],
"prices": [
{
"id": "abc",
"amount": 560
},
{
"id": "def",
"amount": 780
}
],
"solutions": [
{
"price-ref": "abc",
"flights-ref": [
"123",
"456"
]
},
{
"price-ref": "def",
"flights-ref": [
"123",
"789"
]
}
]
}
Desired output would like :
{
"solutions": [
{
"flights": [
{
"id": "123",
"route": "BJS-SIN"
},
{
"id": "456",
"route": "SIN-PEK"
}
],
"price": {
"id": "abc",
"amount": 560
}
},
{
"flights": [
{
"id": "123",
"route": "BJS-SIN"
},
{
"id": "789",
"route": "SIN-BJS"
}
],
"price": {
"id": "def",
"amount": 780
}
}
]
}
As the output, all the data should be constructed refer to the solutions node. How about the mapping expression in JOLT?
all the values from the source json are generated randomly and the keys names are fixed.
I had tried many times but could not find the right spec, so please help.
You can use this spec:
[
{
"operation": "shift",
"spec": {
"*": "&",
"flights|prices": {
"*": {
"*": "&2.#(1,id).&"
}
}
}
},
{
"operation": "shift",
"spec": {
"solutions": {
"*": {
"flights*": {
"*": {
"*": {
"#(6,flights.&)": "&5[&4].flights[&2]"
}
}
},
"price*": {
"*": {
"#(5,prices.&)": "&4[&3].price"
}
}
}
}
}
}
]
I have following json data . I want to transform following data by jolt nifi processor into result data
{
"data": [
{
"source": "Environment Sensors",
"alert_count": "2",
"category": "envs",
"alert_array": {
"alerts": [
{
"name": "neeraj",
"id": "123"
},
{
"name": "arun",
"id": "897"
}
]
}
}
]
}
result data
{
"alert": "2",
"subcategory": "envs",
"alert_array": [
{
"category": "Environment Sensors",
"newName": "neeraj",
"newID": "123"
},
{
"category": "Environment Sensors",
"newName": "arun",
"newID": "897"
}
]
}
Here value of category is the value of source
You can use the following shift transformation spec
[
{
"operation": "shift",
"spec": {
"data": {
"*": {
"alert_count": "alert",
"category": "sub&",
"alert_array": {
"*": {
"*": {
"#(3,category)": "&3[&1].category",
"name": "&3[&1].newName",
"id": "&3[&1].newID"
}
}
}
}
}
}
}
]
first reach the elements of the innermost object, and determine the value of the category element by going 3 levels up along with the others which are alreay located there.
i want to use JOLT Transform to convert an dictionary to an JSON Object. Here below i demonstrate it with an example below.
There is in the root an Array which contains the results from different computers. "Computer1", "Computer2", and more.
This structure should be stay. I removed the 2nd and more array elemt which will reoccur in the same way with {...}.
Given Object:
[
{
"name": "Computer1",
"events": {
"counts": [
{
"countType": "CRITICAL",
"count": 5
},
{
"countType": "HIGH",
"count": 12
},
{
"countType": "LOW",
"count": 40
}
]
},
"processes": {
"counts": [
{
"countType": "CRITICAL",
"count": 0
},
{
"countType": "HIGH",
"count": 2
},
{
"countType": "LOW",
"count": 80
}
]
}
},
{
"name": "Computer2",
"events": {...},...
}
]
Desired Output:
[
{
"name": "Computer1",
"events": {
"CRITICAL": 5,
"HIGH": 12,
"LOW": 40
},
"processes": {
"CRITICAL": 0,
"HIGH": 2,
"LOW": 80
}
}
, {
"name": "Computer2",
"events": {...},
...
}
]
Please help to identify the right JOLT spec.
Thanks in advance
Marcus
You can use such a shift transformation specs
[
{
"operation": "shift",
"spec": {
"*": {
"name": "&1.&",
"events|processes": { // pipe represents "OR" logic
"counts": {
"*": {
"#count": "&4.&3.#countType" // "&4" and "&3" represent going four and three level up the tree to grab the indices of the outermost level list and the key name of the objects("events" and "processes") respectively
}
}
}
}
}
},
{
"operation": "shift",
"spec": {
"*": ""
}
}
]
I am trying to iterate lists inside a list and form a single list with multiple objects. Iterating lists, I am able to achieve. But applying tags before the list iterating to each object is not happening if there are more objects in single list.
My input request is like below:
[
{
"success": [
{
"id": "4",
"Offers": [
{
"name": "Optional",
"type": {
"id": "1",
"name": "Optional"
},
"productOfferings": [
{
"id": "3",
"name": "Test1"
}
]
},
{
"name": "Default",
"type": {
"id": "2",
"name": "Default"
},
"productOfferings": [
{
"id": "1",
"name": "Test2"
},
{
"id": "2",
"name": "Test3"
}
]
}
]
}
]
}
]
My spec is like below:
[
{
"operation": "shift",
"spec": {
"*": {
"success": {
"*": {
"Offers": {
"*": {
"name": "[&1].[&3].typeName",
"type": {
"id": "[&2].[&4].typeNameId",
"name": "[&2].[&4].typeNameValue"
},
"productOfferings": {
"*": {
"id": "[&3].[&1].id",
"name": "[&3].[&1].name"
}
}
}
}
}
}
}
}
},
{
"operation": "shift",
"spec": {
"*": {
"*": "[]"
}
}
}
]
Output Received from spec:
[
{
"typeName": "Optional",
"typeNameId": "1",
"typeNameValue": "Optional",
"id": "3",
"name": "Test1"
},
{
"typeName": "Default",
"typeNameId": "2",
"typeNameValue": "Default",
"id": "1",
"name": "Test2"
},
{
"id": "2",
"name": "Test3"
}
]
But Expected output is like below:
[
{
"typeName": "Optional",
"typeNameId": "1",
"typeNameValue": "Optional",
"id": "3",
"name": "Test1"
},
{
"typeName": "Default",
"typeNameId": "2",
"typeNameValue": "Default",
"id": "1",
"name": "Test2"
},
{
"typeName": "Default",
"typeNameId": "2",
"typeNameValue": "Default",
"id": "2",
"name": "Test3"
}
]
If there are more objects inside productOfferings object, I am not able to add typeName,typeNameId, typeNameValue to the actual object. Please help to fix this issue.
You seem just needing to collect all into the productOfferings array while prepending each key by the common identifier [&3].[&1] such as
[
{
"operation": "shift",
"spec": {
"*": {
"success": {
"*": {
"Offers": {
"*": {
"productOfferings": {
"*": {
"#(2,name)": "[&3].[&1].typeName",
"#(2,type.id)": "[&3].[&1].typeNameId",
"#(2,type.name)": "[&3].[&1].typeNameValue",
"*": "[&3].[&1].&"
}
}
}
}
}
}
}
}
},
{
"operation": "shift",
"spec": {
"*": {
"*": "[]"
}
}
}
]
I'm trying to transform the following JSON
{
"data": {
"keyvalues": [
{
"key": "location",
"value": "sydney, au"
},
{
"key": "weather",
"value": "sunny"
}
]
},
"food": {
"name": "AllFoods",
"date": "2018-03-08T09:35:17-03:00",
"count": 2,
"food": [
{
"name": "chocolate",
"date": "2018-03-08T12:59:58-03:00",
"rating": "10",
"data": null
},
{
"name": "hot dog",
"date": "2018-03-08T09:35:17-03:00",
"rating": "7",
"data": {
"keyvalues": [
{
"key": "topping",
"value": "mustard"
},
{
"key": "BUN type",
"value": "toasted"
},
{
"key": "servings",
"value": "2"
}
]
}
}
]
}
}
Into, something simpler like this, using JOLT (in NIFI). Bringing the first top-level food attributes (name, date, count) into the header and then pulling the nested food array up, and then flattening out the food.data.keyvalues into a dict/hashmap.
{
"header": {
"location": "sydney, au",
"weather": "sunny",
"date": "2018-03-08",
"count": 2
},
"foods": [
{
"name": "chocolate",
"date": "2018-03-08T12:59:58-03:00",
"rating": "10"
},
{
"name": "hot dog",
"date": "2018-03-08T09:35:17-03:00",
"rating": "7",
"topping": "mustard",
"bun_type": "toasted",
"servings": "2"
}
]
}
I've got the first data part working, but I'm not sure how to handle the nested food element. The top level food info needs to move into the header section, and the second level food array, needs to flatten out the data.keyvalues.
Current spec... (only handles the top data.keyvalues)
[
{
"operation": "shift",
"spec": {
"data": {
"keyvalues": {
"*": { "#value": "#key" }
}
}
}
}
]
Spec
[
{
"operation": "shift",
"spec": {
"data": {
"keyvalues": {
"*": {
"value": "header.#(1,key)"
}
}
},
"food": {
"date": "header.date",
"count": "header.count",
"food": {
"*": {
"name": "foods[&1].name",
"date": "foods[&1].date",
"rating": "foods[&1].rating",
"data": {
"keyvalues": {
"*": {
"value": "foods[&4].#(1,key)"
}
}
}
}
}
}
}
}
]