A JOLT transformation question about mapping reference - json

This is an input that needs to be transformed using Jolt Transformation to obtain the expected output.
I am attempting to create a jolt transformation for the below input:
{
"flights": [
{
"id": "123",
"route": "BJS-SIN"
},
{
"id": "456",
"route": "SIN-PEK"
},
{
"id": "789",
"route": "SIN-BJS"
}
],
"prices": [
{
"id": "abc",
"amount": 560
},
{
"id": "def",
"amount": 780
}
],
"solutions": [
{
"price-ref": "abc",
"flights-ref": [
"123",
"456"
]
},
{
"price-ref": "def",
"flights-ref": [
"123",
"789"
]
}
]
}
Desired output would like :
{
"solutions": [
{
"flights": [
{
"id": "123",
"route": "BJS-SIN"
},
{
"id": "456",
"route": "SIN-PEK"
}
],
"price": {
"id": "abc",
"amount": 560
}
},
{
"flights": [
{
"id": "123",
"route": "BJS-SIN"
},
{
"id": "789",
"route": "SIN-BJS"
}
],
"price": {
"id": "def",
"amount": 780
}
}
]
}
As the output, all the data should be constructed refer to the solutions node. How about the mapping expression in JOLT?
all the values from the source json are generated randomly and the keys names are fixed.
I had tried many times but could not find the right spec, so please help.

You can use this spec:
[
{
"operation": "shift",
"spec": {
"*": "&",
"flights|prices": {
"*": {
"*": "&2.#(1,id).&"
}
}
}
},
{
"operation": "shift",
"spec": {
"solutions": {
"*": {
"flights*": {
"*": {
"*": {
"#(6,flights.&)": "&5[&4].flights[&2]"
}
}
},
"price*": {
"*": {
"#(5,prices.&)": "&4[&3].price"
}
}
}
}
}
}
]

Related

Jolt Transform to append extracted data

I'm new to Jolt Transform and have some EDI data that I want to transform. I want to keep the original data and append a few extracted elements in order to make it easier for a downstream process to get to. Here's a sample:
Input
{
"id": "2000",
"segments": [
{
"REF02": "xxxxxxxxxxxx",
"REF01": "0F",
"id": "REF"
},
{
"REF02": "yyyyyyyyyyyy",
"REF01": "1L",
"id": "REF"
}
],
"loops": [
{
"id": "2100",
"segments": []
},
{
"id": "2100",
"segments": [],
"loops": [
{
"id": "2300",
"segments": [
{
"DTP01": "348",
"DTP03": "20220101",
"DTP02": "D8",
"id": "DTP"
},
{
"DTP01": "349",
"DTP03": "20221231",
"DTP02": "D8",
"id": "DTP"
}
]
}
]
}
]
}
Desired Output
{
"ext": {
"subscriber": "xxxxxxxxxxxx",
"start": "20220101"
},
"id": "2000",
"segments": [
{
"REF02": "xxxxxxxxxxxx",
"REF01": "0F",
"id": "REF"
},
{
"REF02": "yyyyyyyyyyyy",
"REF01": "1L",
"id": "REF"
}
],
"loops": [
{
"id": "2100",
"segments": []
},
{
"id": "2100",
"segments": [],
"loops": [
{
"id": "2300",
"segments": [
{
"DTP01": "348",
"DTP03": "20220101",
"DTP02": "D8",
"id": "DTP"
},
{
"DTP01": "349",
"DTP03": "20221231",
"DTP02": "D8",
"id": "DTP"
}
]
}
]
}
]
}
That is, I'd like to append the ext element - copying values from the original.
In xpath terms, the extracted values' paths look something like this:
subscriber :
/segments[../id eq "2000"]/..[REF01 eq "0F"]/REF02/text()
start :
/loops[../id eq "2000"]/../segments[../id eq "2100"])[1]/..
[id eq "NM1" and NM101 eq "IL"][DTP01 eq "348"]/DTP03/text()
Thanks for the help.
I ended up with this solution. If there's any feedback on streamlining, making more elegant, let me know.
[
{
"operation": "default",
"spec": {
"dummy": {}
}
},
{
"operation": "shift",
"spec": {
"dummy": {
"#2,segments": {
"*": {
"REF01": {
"0F": {
"#2,REF02": "ext.subscriber"
}
}
}
},
"#2,loops": {
"*": {
"id": {
"2100": {
"#2,loops": {
"*": {
"id": {
"2300": {
"#2,segments": {
"*": {
"DTP01": {
"348": {
"#2,DTP03": "ext.start"
}
}
}
}
}
}
}
}
}
}
}
}
},
"id": "id",
"segments": "segments",
"loops": "loops"
}
}
]
You can replicate the whole content by use of "#": "" within a shift transformation while adding less curly braces to nest the inner stuff, at the same time determining the paths to search to be used in the next spec such as
[
{
"operation": "shift",
"spec": {
"#": "",
"segments": {
"*": {//indexes of the array
"#REF02": "ext.su.#REF01"
}
},
"loops": {
"*": {//indexes of the array
"#loops": {
"*": {//indexes of the array
"#segments": {
"*": {//indexes of the array
"#DTP03": "ext.st.#5,id.#3,id.#DTP01"
}
}
}
}
}
}
}
},
{
"operation": "shift",
"spec": {
"ext": {
"#su.0F": "&1.subscriber",
"#st.2100.2300.348": "&1.start"
},
"*": "&" //replicate the other attributes/arrays/objects
}
}
]

Create array of simpler JSON objects from a nested JSON using jolt

My input JSON is like
{
"common": {
"name": "abc"
},
"details": {
"id": 4,
"node": [
{
"name": "node1",
"array2": []
},
{
"name": "node2",
"array2": [
{
"name": "node2_a2_1"
}
]
},
{
"name": "node3",
"array2": [
{
"name": "node3_a2_1"
},
{
"name": "node3_a2_2"
},
{
"name": "node3_a2_3"
}
]
}
]
}
}
What I want is for each leaf node in array2 (e.g. {"name": "node3_a2_1"}) I will traverse towards the root and add all common items and create an array of JSON objects without any nested object. So, the output I want is like
[
{
"common_name": "abc",
"id": 4,
"node_name": "node2",
"name": "node2_a2_1"
},
{
"common_name": "abc",
"id": 4,
"node_name": "node3",
"name": "node3_a2_1"
},
{
"common_name": "abc",
"id": 4,
"node_name": "node3",
"name": "node3_a2_2"
},
{
"common_name": "abc",
"id": 4,
"node_name": "node3",
"name": "node3_a2_3"
}
]
Could you please suggest how can I do that?
You can walk through the array2 while picking values of each element from their original location in the JSON value.
For example; traverse } four times to grab the value of id by using #(4,id), and use #(3,name)[&1] as the common distinguishing identifier for each attribute such as
[
{
"operation": "shift",
"spec": {
"details": {
"node": {
"*": {
"array2": {
"*": {
"#(5,common.name)": "#(3,name)[&1].common_name",
"#(4,id)": "#(3,name)[&1].id",
"#(2,name)": "#(3,name)[&1].node_name",
"name": "#(3,name)[&1].&"
}
}
}
}
}
}
},
{
"operation": "shift",
"spec": {
"*": {
"*": ""
}
}
}
]

Jolt unnest list to top level

I have a JSON:
[
{
"id": 1015380,
"type": "campaign",
"stats": [
{
"message_sends_by_any_user": 13,
"ctr": "0.094",
}
]
},
{
"id": 1015695,
"type": "campaign",
"stats": [
{
"message_sends_by_any_user": 7,
"ctr": "0.091",
}
]
}
]
I want to "unnested" stats list to top level. JOLT config:
[
{
"operation": "shift",
"spec": {
"*": {
"id": "[&1].id",
"type": "[&1].type",
"stats": {
"*": "[&2].&"
}
}
}
}
]
Expected:
[
{
"id": 1015380,
"type": "campaign",
"message_sends_by_any_user": 13,
"ctr": "0.094",
},
{
"id": 1015695,
"type": "campaign",
"message_sends_by_any_user": 7,
"ctr": "0.091"
}
]
But actual output contains array indexes like this:
[
{
"id" : 1015380,
"type" : "campaign",
"0" : {
"message_sends_by_any_user" : 13,
"ctr" : "0.094"
}, ...
How to avoid these indexes?
You can prefer walking through the indexes of the stats list by taking id and type attributes inside such as
[
{
"operation": "shift",
"spec": {
"*": {
"stats": {
"*": {
"#(2,id)": "[&3].id",
"#(2,type)": "[&3].type",
"*": "[&3].&"
}
}
}
}
}
]
the demo on the site http://jolt-demo.appspot.com is

Jolt Conversion - iterate list within a list and form single list

I am trying to iterate lists inside a list and form a single list with multiple objects. Iterating lists, I am able to achieve. But applying tags before the list iterating to each object is not happening if there are more objects in single list.
My input request is like below:
[
{
"success": [
{
"id": "4",
"Offers": [
{
"name": "Optional",
"type": {
"id": "1",
"name": "Optional"
},
"productOfferings": [
{
"id": "3",
"name": "Test1"
}
]
},
{
"name": "Default",
"type": {
"id": "2",
"name": "Default"
},
"productOfferings": [
{
"id": "1",
"name": "Test2"
},
{
"id": "2",
"name": "Test3"
}
]
}
]
}
]
}
]
My spec is like below:
[
{
"operation": "shift",
"spec": {
"*": {
"success": {
"*": {
"Offers": {
"*": {
"name": "[&1].[&3].typeName",
"type": {
"id": "[&2].[&4].typeNameId",
"name": "[&2].[&4].typeNameValue"
},
"productOfferings": {
"*": {
"id": "[&3].[&1].id",
"name": "[&3].[&1].name"
}
}
}
}
}
}
}
}
},
{
"operation": "shift",
"spec": {
"*": {
"*": "[]"
}
}
}
]
Output Received from spec:
[
{
"typeName": "Optional",
"typeNameId": "1",
"typeNameValue": "Optional",
"id": "3",
"name": "Test1"
},
{
"typeName": "Default",
"typeNameId": "2",
"typeNameValue": "Default",
"id": "1",
"name": "Test2"
},
{
"id": "2",
"name": "Test3"
}
]
But Expected output is like below:
[
{
"typeName": "Optional",
"typeNameId": "1",
"typeNameValue": "Optional",
"id": "3",
"name": "Test1"
},
{
"typeName": "Default",
"typeNameId": "2",
"typeNameValue": "Default",
"id": "1",
"name": "Test2"
},
{
"typeName": "Default",
"typeNameId": "2",
"typeNameValue": "Default",
"id": "2",
"name": "Test3"
}
]
If there are more objects inside productOfferings object, I am not able to add typeName,typeNameId, typeNameValue to the actual object. Please help to fix this issue.
You seem just needing to collect all into the productOfferings array while prepending each key by the common identifier [&3].[&1] such as
[
{
"operation": "shift",
"spec": {
"*": {
"success": {
"*": {
"Offers": {
"*": {
"productOfferings": {
"*": {
"#(2,name)": "[&3].[&1].typeName",
"#(2,type.id)": "[&3].[&1].typeNameId",
"#(2,type.name)": "[&3].[&1].typeNameValue",
"*": "[&3].[&1].&"
}
}
}
}
}
}
}
}
},
{
"operation": "shift",
"spec": {
"*": {
"*": "[]"
}
}
}
]

Jolt transformation array data

I want to transform a JSON using JOLT like this:
Input: {
"array": [
"1","2","3","4"
],
"array2": [
{
"something": "123",
"something1": "Plane"
},
{
"something3": "567",
"something4": "Car"
}
]
}
Into the following format, as you can see from output I need data from both arrays to fit exact param names, not empty one like in the first or existed param names like in the second one.
Output: {
"one_array": [
{
"code": "1",
"description": "",
},
{
"code": "2",
"description": "",
},
{
"code": "3",
"description": "",
},
{
"code": "4",
"description": "",
}
], "other_array": [
{
"id": "123",
"type": "Plane"
},
{
"id": "567",
"type": "Car"
}
]
}
Some clarifications are really appreciated
You can achieve this using 2 shift operations and the default operation as below.
[
{
"operation": "shift",
"spec": {
"array": {
"*": {
"#": "one_array[&].id"
}
},
"array2": {
"*": {
"*": {
"#": "tmp_array[&2]"
}
}
}
}
},
{
"operation": "shift",
"spec": {
"one_array": "one_array",
"tmp_array": {
"*": {
"0": "other_array[&1].id",
"1": "other_array[&1].type"
}
}
}
},
{
"operation": "default",
"spec": {
"one_array[]": {
"*": {
"description": ""
}
}
}
}
]