I've spent quite a lot of time figuring it out but I'm stuck, I have a nested JSON and I want to enrich the values of "attr" with those matching the keys of "codes", thanks in advance.
My Input JSON:
{
"items": {
"a1b2xxxx": {
"name": "item 1",
"attr": [
"A",
"B",
"C"
]
},
"c2b2cxxxx": {
"name": "item 2",
"attr": [
"D",
"E",
"F"
]
}
},
"codes": {
"A": {
"color": "green"
},
"B": {
"size": "M"
},
"C": {
"sku": "NS"
},
"D": {
"stock": 2
},
"E": {
"some_key": "some_value"
},
"F": {
"foo": "bar"
}
}
}
My Desired Output JSON:
{
"items": {
"a1b2xxxx": {
"name": "item 1",
"attr": {
"A": {
"color": "green"
},
"B": {
"size": "M"
},
"C": {
"sku": "NS"
}
}
},
"c2b2xxxx": {
"name": "item 2",
"attr": {
"D": {
"stock": 2
},
"E": {
"some_key": "some_value"
},
"F": {
"foo": "bar"
}
}
}
},
"codes": {
"A": {
"color": "green"
},
"B": {
"size": "M"
},
"C": {
"sku": "NS"
},
"D": {
"stock": 2
},
"E": {
"some_key": "some_value"
},
"F": {
"foo": "bar"
}
}
}
My approach is following:
Using cardinality operation convert attr to an array of objects
Then maybe I can map values from codes using modify-default-beta
But I am stuck at step 1. Here is my transformer:
[
{
"operation": "cardinality",
"spec": {
"items": {
"*": {
"attr": "ONE"
}
}
}
}
]
You can use a single shift transformation such as
[
{
"operation": "shift",
"spec": {
"items": {
"*": {
"name": "&2.&1.&",
"attr": {
"*": { // the level of the indexes of "attr" array, eg. 0,1,2
"*": { // the level of the components of "attr" array, eg. A,B,C,D,E,F
"#(5,codes.&)": "&5.&4.&" // going four levels up the tree to get the object labels of "a1b2xxxx" and "c2b2cxxxx"
}
}
}
}
},
"*": "&" // objects(attributes or array) other than "items", eg. "codes"
}
}
]
the demo on the site http://jolt-demo.appspot.com/ is
Another spec can be like this:
You can use values of array with # and use them as key #(0) like "#": "&4.&3.&2.#(0).color"
[
{
"operation": "shift",
"spec": {
"*": "&",
"items": {
"*": {
"*": "&2.&1.&",
"attr": {
"*": {
"#": "&4.&3.&2.#(0).color"
}
}
}
}
}
}
]
Related
I am trying to convert the input JSON:
for example , incoming json data structure is
{
"12346565": [
{
"type": "13",
"value": "3",
"score": "5"
},
{
"type": "45",
"value": "1",
"score": "5"
}
],
"12346777": [
{
"type": "41",
"value": "10",
"score": "3"
}
]
}
The expected output is:
{
"12346565": [
{
"model": "13",
"v": "3"
},
{
"model": "45",
"v": "1"
}
],
"12346777": [
{
"model": "41",
"v": "10"
}
]
}
change:
type -> model;
value -> v
remove:
score
Any ideas?
You can use modify and remove transformation specs consecutively such as
[
{
// generate new pairs with values copied from the former attributes
"operation": "modify-overwrite-beta",
"spec": {
"*": {
"*": {
"model": "#(1,type)",
"v": "#(1,value)"
}
}
}
},
{
// get rid of undesired attributes
"operation": "remove",
"spec": {
"*": {
"*": {
"type": "",
"value": "",
"score": ""
}
}
}
}
]
the demo on the site http://jolt-demo.appspot.com/ is
My input JSON is like
{
"common": {
"name": "abc"
},
"details": {
"id": 4,
"node": [
{
"name": "node1",
"array2": []
},
{
"name": "node2",
"array2": [
{
"name": "node2_a2_1"
}
]
},
{
"name": "node3",
"array2": [
{
"name": "node3_a2_1"
},
{
"name": "node3_a2_2"
},
{
"name": "node3_a2_3"
}
]
}
]
}
}
What I want is for each leaf node in array2 (e.g. {"name": "node3_a2_1"}) I will traverse towards the root and add all common items and create an array of JSON objects without any nested object. So, the output I want is like
[
{
"common_name": "abc",
"id": 4,
"node_name": "node2",
"name": "node2_a2_1"
},
{
"common_name": "abc",
"id": 4,
"node_name": "node3",
"name": "node3_a2_1"
},
{
"common_name": "abc",
"id": 4,
"node_name": "node3",
"name": "node3_a2_2"
},
{
"common_name": "abc",
"id": 4,
"node_name": "node3",
"name": "node3_a2_3"
}
]
Could you please suggest how can I do that?
You can walk through the array2 while picking values of each element from their original location in the JSON value.
For example; traverse } four times to grab the value of id by using #(4,id), and use #(3,name)[&1] as the common distinguishing identifier for each attribute such as
[
{
"operation": "shift",
"spec": {
"details": {
"node": {
"*": {
"array2": {
"*": {
"#(5,common.name)": "#(3,name)[&1].common_name",
"#(4,id)": "#(3,name)[&1].id",
"#(2,name)": "#(3,name)[&1].node_name",
"name": "#(3,name)[&1].&"
}
}
}
}
}
}
},
{
"operation": "shift",
"spec": {
"*": {
"*": ""
}
}
}
]
I am trying to iterate lists inside a list and form a single list with multiple objects. Iterating lists, I am able to achieve. But applying tags before the list iterating to each object is not happening if there are more objects in single list.
My input request is like below:
[
{
"success": [
{
"id": "4",
"Offers": [
{
"name": "Optional",
"type": {
"id": "1",
"name": "Optional"
},
"productOfferings": [
{
"id": "3",
"name": "Test1"
}
]
},
{
"name": "Default",
"type": {
"id": "2",
"name": "Default"
},
"productOfferings": [
{
"id": "1",
"name": "Test2"
},
{
"id": "2",
"name": "Test3"
}
]
}
]
}
]
}
]
My spec is like below:
[
{
"operation": "shift",
"spec": {
"*": {
"success": {
"*": {
"Offers": {
"*": {
"name": "[&1].[&3].typeName",
"type": {
"id": "[&2].[&4].typeNameId",
"name": "[&2].[&4].typeNameValue"
},
"productOfferings": {
"*": {
"id": "[&3].[&1].id",
"name": "[&3].[&1].name"
}
}
}
}
}
}
}
}
},
{
"operation": "shift",
"spec": {
"*": {
"*": "[]"
}
}
}
]
Output Received from spec:
[
{
"typeName": "Optional",
"typeNameId": "1",
"typeNameValue": "Optional",
"id": "3",
"name": "Test1"
},
{
"typeName": "Default",
"typeNameId": "2",
"typeNameValue": "Default",
"id": "1",
"name": "Test2"
},
{
"id": "2",
"name": "Test3"
}
]
But Expected output is like below:
[
{
"typeName": "Optional",
"typeNameId": "1",
"typeNameValue": "Optional",
"id": "3",
"name": "Test1"
},
{
"typeName": "Default",
"typeNameId": "2",
"typeNameValue": "Default",
"id": "1",
"name": "Test2"
},
{
"typeName": "Default",
"typeNameId": "2",
"typeNameValue": "Default",
"id": "2",
"name": "Test3"
}
]
If there are more objects inside productOfferings object, I am not able to add typeName,typeNameId, typeNameValue to the actual object. Please help to fix this issue.
You seem just needing to collect all into the productOfferings array while prepending each key by the common identifier [&3].[&1] such as
[
{
"operation": "shift",
"spec": {
"*": {
"success": {
"*": {
"Offers": {
"*": {
"productOfferings": {
"*": {
"#(2,name)": "[&3].[&1].typeName",
"#(2,type.id)": "[&3].[&1].typeNameId",
"#(2,type.name)": "[&3].[&1].typeNameValue",
"*": "[&3].[&1].&"
}
}
}
}
}
}
}
}
},
{
"operation": "shift",
"spec": {
"*": {
"*": "[]"
}
}
}
]
Iam new to jolt. The transformation Iam trying to achieve is really simple.I have a json as shown below
{
"Object": [
{
"date": "01-01-2021",
"Sold": {
"ItemType": "New",
"ItemID": 1,
"Description": "desc 1"
}
},
{
"date": "01-01-2021",
"Sold": {
"ItemType": "New 2",
"ItemID": 2,
"Description": "desc 2"
}
}
]
}
My expected output is
[{
"ItemType": "New",
"ItemID": 1,
}
,{
"ItemType": "New 2",
"ItemID": 2,
}]
But when I use this transform
[
{
"operation": "shift",
"spec": {
"Object": {
"*": {
"Sold": {
"ItemType": "ItemType"
}
}
}
}
}
]
The output I obtain is
{
"ItemType" : [ "New", "New 2" ]
}
What is the mistake Iam making?
You should read about applying JOLT on JSON arrays, here for example.
You need to reference the index array using [&2], which means look up the tree three levels: zero (Object), then one (Sold), then two (ItemType/ItemID) and use that as an index array in the output.
[
{
"operation": "shift",
"spec": {
"Object": {
"*": {
"Sold": {
"ItemType": "[&2].ItemType",
"ItemID": "[&2].ItemID"
}
}
}
}
}
]
Output:
[ {
"ItemType" : "New",
"ItemID" : 1
}, {
"ItemType" : "New 2",
"ItemID" : 2
} ]
I'm trying to transform the following JSON
{
"data": {
"keyvalues": [
{
"key": "location",
"value": "sydney, au"
},
{
"key": "weather",
"value": "sunny"
}
]
},
"food": {
"name": "AllFoods",
"date": "2018-03-08T09:35:17-03:00",
"count": 2,
"food": [
{
"name": "chocolate",
"date": "2018-03-08T12:59:58-03:00",
"rating": "10",
"data": null
},
{
"name": "hot dog",
"date": "2018-03-08T09:35:17-03:00",
"rating": "7",
"data": {
"keyvalues": [
{
"key": "topping",
"value": "mustard"
},
{
"key": "BUN type",
"value": "toasted"
},
{
"key": "servings",
"value": "2"
}
]
}
}
]
}
}
Into, something simpler like this, using JOLT (in NIFI). Bringing the first top-level food attributes (name, date, count) into the header and then pulling the nested food array up, and then flattening out the food.data.keyvalues into a dict/hashmap.
{
"header": {
"location": "sydney, au",
"weather": "sunny",
"date": "2018-03-08",
"count": 2
},
"foods": [
{
"name": "chocolate",
"date": "2018-03-08T12:59:58-03:00",
"rating": "10"
},
{
"name": "hot dog",
"date": "2018-03-08T09:35:17-03:00",
"rating": "7",
"topping": "mustard",
"bun_type": "toasted",
"servings": "2"
}
]
}
I've got the first data part working, but I'm not sure how to handle the nested food element. The top level food info needs to move into the header section, and the second level food array, needs to flatten out the data.keyvalues.
Current spec... (only handles the top data.keyvalues)
[
{
"operation": "shift",
"spec": {
"data": {
"keyvalues": {
"*": { "#value": "#key" }
}
}
}
}
]
Spec
[
{
"operation": "shift",
"spec": {
"data": {
"keyvalues": {
"*": {
"value": "header.#(1,key)"
}
}
},
"food": {
"date": "header.date",
"count": "header.count",
"food": {
"*": {
"name": "foods[&1].name",
"date": "foods[&1].date",
"rating": "foods[&1].rating",
"data": {
"keyvalues": {
"*": {
"value": "foods[&4].#(1,key)"
}
}
}
}
}
}
}
}
]