Iam new to jolt. The transformation Iam trying to achieve is really simple.I have a json as shown below
{
"Object": [
{
"date": "01-01-2021",
"Sold": {
"ItemType": "New",
"ItemID": 1,
"Description": "desc 1"
}
},
{
"date": "01-01-2021",
"Sold": {
"ItemType": "New 2",
"ItemID": 2,
"Description": "desc 2"
}
}
]
}
My expected output is
[{
"ItemType": "New",
"ItemID": 1,
}
,{
"ItemType": "New 2",
"ItemID": 2,
}]
But when I use this transform
[
{
"operation": "shift",
"spec": {
"Object": {
"*": {
"Sold": {
"ItemType": "ItemType"
}
}
}
}
}
]
The output I obtain is
{
"ItemType" : [ "New", "New 2" ]
}
What is the mistake Iam making?
You should read about applying JOLT on JSON arrays, here for example.
You need to reference the index array using [&2], which means look up the tree three levels: zero (Object), then one (Sold), then two (ItemType/ItemID) and use that as an index array in the output.
[
{
"operation": "shift",
"spec": {
"Object": {
"*": {
"Sold": {
"ItemType": "[&2].ItemType",
"ItemID": "[&2].ItemID"
}
}
}
}
}
]
Output:
[ {
"ItemType" : "New",
"ItemID" : 1
}, {
"ItemType" : "New 2",
"ItemID" : 2
} ]
Related
This is an input that needs to be transformed using Jolt Transformation to obtain the expected output.
I am attempting to create a jolt transformation for the below input:
{
"flights": [
{
"id": "123",
"route": "BJS-SIN"
},
{
"id": "456",
"route": "SIN-PEK"
},
{
"id": "789",
"route": "SIN-BJS"
}
],
"prices": [
{
"id": "abc",
"amount": 560
},
{
"id": "def",
"amount": 780
}
],
"solutions": [
{
"price-ref": "abc",
"flights-ref": [
"123",
"456"
]
},
{
"price-ref": "def",
"flights-ref": [
"123",
"789"
]
}
]
}
Desired output would like :
{
"solutions": [
{
"flights": [
{
"id": "123",
"route": "BJS-SIN"
},
{
"id": "456",
"route": "SIN-PEK"
}
],
"price": {
"id": "abc",
"amount": 560
}
},
{
"flights": [
{
"id": "123",
"route": "BJS-SIN"
},
{
"id": "789",
"route": "SIN-BJS"
}
],
"price": {
"id": "def",
"amount": 780
}
}
]
}
As the output, all the data should be constructed refer to the solutions node. How about the mapping expression in JOLT?
all the values from the source json are generated randomly and the keys names are fixed.
I had tried many times but could not find the right spec, so please help.
You can use this spec:
[
{
"operation": "shift",
"spec": {
"*": "&",
"flights|prices": {
"*": {
"*": "&2.#(1,id).&"
}
}
}
},
{
"operation": "shift",
"spec": {
"solutions": {
"*": {
"flights*": {
"*": {
"*": {
"#(6,flights.&)": "&5[&4].flights[&2]"
}
}
},
"price*": {
"*": {
"#(5,prices.&)": "&4[&3].price"
}
}
}
}
}
}
]
I've spent quite a lot of time figuring it out but I'm stuck, I have a nested JSON and I want to enrich the values of "attr" with those matching the keys of "codes", thanks in advance.
My Input JSON:
{
"items": {
"a1b2xxxx": {
"name": "item 1",
"attr": [
"A",
"B",
"C"
]
},
"c2b2cxxxx": {
"name": "item 2",
"attr": [
"D",
"E",
"F"
]
}
},
"codes": {
"A": {
"color": "green"
},
"B": {
"size": "M"
},
"C": {
"sku": "NS"
},
"D": {
"stock": 2
},
"E": {
"some_key": "some_value"
},
"F": {
"foo": "bar"
}
}
}
My Desired Output JSON:
{
"items": {
"a1b2xxxx": {
"name": "item 1",
"attr": {
"A": {
"color": "green"
},
"B": {
"size": "M"
},
"C": {
"sku": "NS"
}
}
},
"c2b2xxxx": {
"name": "item 2",
"attr": {
"D": {
"stock": 2
},
"E": {
"some_key": "some_value"
},
"F": {
"foo": "bar"
}
}
}
},
"codes": {
"A": {
"color": "green"
},
"B": {
"size": "M"
},
"C": {
"sku": "NS"
},
"D": {
"stock": 2
},
"E": {
"some_key": "some_value"
},
"F": {
"foo": "bar"
}
}
}
My approach is following:
Using cardinality operation convert attr to an array of objects
Then maybe I can map values from codes using modify-default-beta
But I am stuck at step 1. Here is my transformer:
[
{
"operation": "cardinality",
"spec": {
"items": {
"*": {
"attr": "ONE"
}
}
}
}
]
You can use a single shift transformation such as
[
{
"operation": "shift",
"spec": {
"items": {
"*": {
"name": "&2.&1.&",
"attr": {
"*": { // the level of the indexes of "attr" array, eg. 0,1,2
"*": { // the level of the components of "attr" array, eg. A,B,C,D,E,F
"#(5,codes.&)": "&5.&4.&" // going four levels up the tree to get the object labels of "a1b2xxxx" and "c2b2cxxxx"
}
}
}
}
},
"*": "&" // objects(attributes or array) other than "items", eg. "codes"
}
}
]
the demo on the site http://jolt-demo.appspot.com/ is
Another spec can be like this:
You can use values of array with # and use them as key #(0) like "#": "&4.&3.&2.#(0).color"
[
{
"operation": "shift",
"spec": {
"*": "&",
"items": {
"*": {
"*": "&2.&1.&",
"attr": {
"*": {
"#": "&4.&3.&2.#(0).color"
}
}
}
}
}
}
]
I have following json data . I want to transform following data by jolt nifi processor into result data
{
"data": [
{
"source": "Environment Sensors",
"alert_count": "2",
"category": "envs",
"alert_array": {
"alerts": [
{
"name": "neeraj",
"id": "123"
},
{
"name": "arun",
"id": "897"
}
]
}
}
]
}
result data
{
"alert": "2",
"subcategory": "envs",
"alert_array": [
{
"category": "Environment Sensors",
"newName": "neeraj",
"newID": "123"
},
{
"category": "Environment Sensors",
"newName": "arun",
"newID": "897"
}
]
}
Here value of category is the value of source
You can use the following shift transformation spec
[
{
"operation": "shift",
"spec": {
"data": {
"*": {
"alert_count": "alert",
"category": "sub&",
"alert_array": {
"*": {
"*": {
"#(3,category)": "&3[&1].category",
"name": "&3[&1].newName",
"id": "&3[&1].newID"
}
}
}
}
}
}
}
]
first reach the elements of the innermost object, and determine the value of the category element by going 3 levels up along with the others which are alreay located there.
My input JSON is like
{
"common": {
"name": "abc"
},
"details": {
"id": 4,
"node": [
{
"name": "node1",
"array2": []
},
{
"name": "node2",
"array2": [
{
"name": "node2_a2_1"
}
]
},
{
"name": "node3",
"array2": [
{
"name": "node3_a2_1"
},
{
"name": "node3_a2_2"
},
{
"name": "node3_a2_3"
}
]
}
]
}
}
What I want is for each leaf node in array2 (e.g. {"name": "node3_a2_1"}) I will traverse towards the root and add all common items and create an array of JSON objects without any nested object. So, the output I want is like
[
{
"common_name": "abc",
"id": 4,
"node_name": "node2",
"name": "node2_a2_1"
},
{
"common_name": "abc",
"id": 4,
"node_name": "node3",
"name": "node3_a2_1"
},
{
"common_name": "abc",
"id": 4,
"node_name": "node3",
"name": "node3_a2_2"
},
{
"common_name": "abc",
"id": 4,
"node_name": "node3",
"name": "node3_a2_3"
}
]
Could you please suggest how can I do that?
You can walk through the array2 while picking values of each element from their original location in the JSON value.
For example; traverse } four times to grab the value of id by using #(4,id), and use #(3,name)[&1] as the common distinguishing identifier for each attribute such as
[
{
"operation": "shift",
"spec": {
"details": {
"node": {
"*": {
"array2": {
"*": {
"#(5,common.name)": "#(3,name)[&1].common_name",
"#(4,id)": "#(3,name)[&1].id",
"#(2,name)": "#(3,name)[&1].node_name",
"name": "#(3,name)[&1].&"
}
}
}
}
}
}
},
{
"operation": "shift",
"spec": {
"*": {
"*": ""
}
}
}
]
I'm trying to transform the following JSON
{
"data": {
"keyvalues": [
{
"key": "location",
"value": "sydney, au"
},
{
"key": "weather",
"value": "sunny"
}
]
},
"food": {
"name": "AllFoods",
"date": "2018-03-08T09:35:17-03:00",
"count": 2,
"food": [
{
"name": "chocolate",
"date": "2018-03-08T12:59:58-03:00",
"rating": "10",
"data": null
},
{
"name": "hot dog",
"date": "2018-03-08T09:35:17-03:00",
"rating": "7",
"data": {
"keyvalues": [
{
"key": "topping",
"value": "mustard"
},
{
"key": "BUN type",
"value": "toasted"
},
{
"key": "servings",
"value": "2"
}
]
}
}
]
}
}
Into, something simpler like this, using JOLT (in NIFI). Bringing the first top-level food attributes (name, date, count) into the header and then pulling the nested food array up, and then flattening out the food.data.keyvalues into a dict/hashmap.
{
"header": {
"location": "sydney, au",
"weather": "sunny",
"date": "2018-03-08",
"count": 2
},
"foods": [
{
"name": "chocolate",
"date": "2018-03-08T12:59:58-03:00",
"rating": "10"
},
{
"name": "hot dog",
"date": "2018-03-08T09:35:17-03:00",
"rating": "7",
"topping": "mustard",
"bun_type": "toasted",
"servings": "2"
}
]
}
I've got the first data part working, but I'm not sure how to handle the nested food element. The top level food info needs to move into the header section, and the second level food array, needs to flatten out the data.keyvalues.
Current spec... (only handles the top data.keyvalues)
[
{
"operation": "shift",
"spec": {
"data": {
"keyvalues": {
"*": { "#value": "#key" }
}
}
}
}
]
Spec
[
{
"operation": "shift",
"spec": {
"data": {
"keyvalues": {
"*": {
"value": "header.#(1,key)"
}
}
},
"food": {
"date": "header.date",
"count": "header.count",
"food": {
"*": {
"name": "foods[&1].name",
"date": "foods[&1].date",
"rating": "foods[&1].rating",
"data": {
"keyvalues": {
"*": {
"value": "foods[&4].#(1,key)"
}
}
}
}
}
}
}
}
]