Jolt Transform to append extracted data - json

I'm new to Jolt Transform and have some EDI data that I want to transform. I want to keep the original data and append a few extracted elements in order to make it easier for a downstream process to get to. Here's a sample:
Input
{
"id": "2000",
"segments": [
{
"REF02": "xxxxxxxxxxxx",
"REF01": "0F",
"id": "REF"
},
{
"REF02": "yyyyyyyyyyyy",
"REF01": "1L",
"id": "REF"
}
],
"loops": [
{
"id": "2100",
"segments": []
},
{
"id": "2100",
"segments": [],
"loops": [
{
"id": "2300",
"segments": [
{
"DTP01": "348",
"DTP03": "20220101",
"DTP02": "D8",
"id": "DTP"
},
{
"DTP01": "349",
"DTP03": "20221231",
"DTP02": "D8",
"id": "DTP"
}
]
}
]
}
]
}
Desired Output
{
"ext": {
"subscriber": "xxxxxxxxxxxx",
"start": "20220101"
},
"id": "2000",
"segments": [
{
"REF02": "xxxxxxxxxxxx",
"REF01": "0F",
"id": "REF"
},
{
"REF02": "yyyyyyyyyyyy",
"REF01": "1L",
"id": "REF"
}
],
"loops": [
{
"id": "2100",
"segments": []
},
{
"id": "2100",
"segments": [],
"loops": [
{
"id": "2300",
"segments": [
{
"DTP01": "348",
"DTP03": "20220101",
"DTP02": "D8",
"id": "DTP"
},
{
"DTP01": "349",
"DTP03": "20221231",
"DTP02": "D8",
"id": "DTP"
}
]
}
]
}
]
}
That is, I'd like to append the ext element - copying values from the original.
In xpath terms, the extracted values' paths look something like this:
subscriber :
/segments[../id eq "2000"]/..[REF01 eq "0F"]/REF02/text()
start :
/loops[../id eq "2000"]/../segments[../id eq "2100"])[1]/..
[id eq "NM1" and NM101 eq "IL"][DTP01 eq "348"]/DTP03/text()
Thanks for the help.

I ended up with this solution. If there's any feedback on streamlining, making more elegant, let me know.
[
{
"operation": "default",
"spec": {
"dummy": {}
}
},
{
"operation": "shift",
"spec": {
"dummy": {
"#2,segments": {
"*": {
"REF01": {
"0F": {
"#2,REF02": "ext.subscriber"
}
}
}
},
"#2,loops": {
"*": {
"id": {
"2100": {
"#2,loops": {
"*": {
"id": {
"2300": {
"#2,segments": {
"*": {
"DTP01": {
"348": {
"#2,DTP03": "ext.start"
}
}
}
}
}
}
}
}
}
}
}
}
},
"id": "id",
"segments": "segments",
"loops": "loops"
}
}
]

You can replicate the whole content by use of "#": "" within a shift transformation while adding less curly braces to nest the inner stuff, at the same time determining the paths to search to be used in the next spec such as
[
{
"operation": "shift",
"spec": {
"#": "",
"segments": {
"*": {//indexes of the array
"#REF02": "ext.su.#REF01"
}
},
"loops": {
"*": {//indexes of the array
"#loops": {
"*": {//indexes of the array
"#segments": {
"*": {//indexes of the array
"#DTP03": "ext.st.#5,id.#3,id.#DTP01"
}
}
}
}
}
}
}
},
{
"operation": "shift",
"spec": {
"ext": {
"#su.0F": "&1.subscriber",
"#st.2100.2300.348": "&1.start"
},
"*": "&" //replicate the other attributes/arrays/objects
}
}
]

Related

JOLT Keep structure after array shift

I'm trying to construct a JOLT transformation such that it will change a parameter value if the paramter name matches in a given array.
Example Input:
{
"component": {
"parameters": [
{
"parameter": {
"name": "var_name",
"value": "val"
}
},
{
"parameter": {
"name": "1",
"value": "2"
}
}
]
},
"additional": "onemore"
}
Desired Output:
{
"component": {
"parameters": [
{
"parameter": {
"name": "var_name",
"value": "new_val"
}
},
{
"parameter": {
"name": "1",
"value": "2"
}
}
]
},
"additional": "onemore"
}
My Current JOLT transform:
[
{
"operation": "shift",
"spec": {
"component": {
"parameters": {
"*": {
"parameter": {
"name": {
"var_name": {
"#new_val": "&6.&5[&4].&3.value"
}
}
}
}
}
}
}
}
]
The problem with my JOLT transform is that it deletes the rest of the Json, whereas I'd like to mantain it unchanged if there's no match
I tried looking for a solution, but the closest I got was this one, which allowed me to make the current transform, but I don't understand how to fix it properly.
EDIT:
I actually missed part of the input when writing the example, but thanks to #Barbaros Özhan answer I was able to complete my transformation.
For future rference if anybody else needs it my original input had additonal fields like:
{
"component": {
"parameters": [
{
"option": "true",
"parameter": {
"name": "var_name",
"description": "desc",
"value": "val"
}
},
{
"option": "false",
"parameter": {
"name": "1",
"description": "desc",
"value": "2"
}
}
],
"other value": "baz"
},
"additional": "onemore"
}
And the final transformation I used was slightly tweaked to account for those like so:
[
{
"operation": "shift",
"spec": {
"component": {
"parameters": {
"*": {
"parameter": {
"name": {
"#": "&5.&4[&3].&2.&1",
"var_name": {
"#new_val": "&6.&5[&4].&3.value"
},
"*": {
"#(2,value)": "&6.&5[&4].&3.value"
}
},
"value": "&6.&5[&4].&3.value",
"*": "&4.&3[&2].&1.&"
},
"*": "&3.&2[&1].&"
}
},
"*": "&1.&"
},
"*": "&"
}
}
]
You can use this shift transformation spec
[
{
"operation": "shift",
"spec": {
"component": {
"parameters": {
"*": {
"parameter": {
"name": {
"#": "&5.&4[&3].&2.&1", // # matches the value taken from one level up, eg. replicating "name"
"var_name": {
"#new_val": "&6.&5[&4].&3.value"
},
"*": {
"#(2,value)": "&6.&5[&4].&3.value"
}
}
}
}
}
},
"*": "&" // the "else" case(the attributes other than "component")
}
}
]

Jolt transformation on basis of another conditional value in the json

I am trying to use jolt to transform a json payload and my input json looks like this
{
"root": {
"entities": [
{
"fields": [
{
"name": "ContactType",
"value": "Company"
},
{
"name": "HandlerName",
"value": "XYZ"
},
{
"name": "Reference",
"value": "123-4443"
},
{
"entities": [
{
"parent": {
"fieldName": "DefendantAddress"
},
"fields": [
{
"name": "Address",
"value": "1662369113138 Somewhere Street"
},
{
"name": "PostalCode",
"value": "XXXXX"
}
]
}
],
"name": "DefendantAddress"
}
]
}
]
}
}
I am looking to transform the Address data based on value in ContactType field.
If contact type is Company, the output should be
{
"HandlerName": "XYZ",
"reference": "123-4443",
"companyDetails": {
"address": "1662369113138 Somewhere Street",
"postalCode": "XXXXX"
}
}
If contact type is Person, then output should be
{
"HandlerName": "XYZ",
"reference": "123-4443",
"personDetails": {
"address": "1662369113138 Somewhere Street",
"postalCode": "XXXXX"
}
}
I am able to check using my jolt spec if the contact type is Company or Person but struggling to navigate through the hierarchy to reach the child entities. This is my jolt spec. I am looking for implementation is ???? area or if another solution can be provided for the problem.
[
{
"operation": "shift",
"spec": {
"root": {
"entities": {
"*": {
"fields": {
"*": {
"name": {
"HandlerName": {
"#(2,value)": "handlerName"
},
"Reference": {
"#(2,value)": "reference"
},
"ContactType": {
"#(2,value)": {
"Company": {
?????
}
}
}
}
}
}
}
}
}
}
}
]
You can start by diving deep into the innermost objects in order to match the value and name attributes at this level( "#(0,value)": "personDetails.#(0,name)" ), and then accumulate the subelements from the outer object after going some(6) levels up the tree to reach the level of it. Use conditional within the upcoming shift transformation spec such as
[
{
"operation": "shift",
"spec": {
"root": {
"entities": {
"*": {
"fields": {
"*": {
"entities": {
"*": {
"fields": {
"*": {
"#(6,fields)": { //traverse "{" characters 6 times to reach the level of outer "fields" tag
"*": {
"#(0,value)": "common.#(0,name)"
}
},
"#(0,value)": "p.#(0,name)"
}
}
}
}
}
}
}
}
}
}
},
{
"operation": "cardinality",
"spec": {
"common": {
"*": "ONE"
}
}
},
{
"operation": "shift",
"spec": {
"common": {
"ContactType": {
"Company|Person": { // "|" is "OR" operator
"#(2,HandlerName)": "HandlerName",
"#(2,Reference)": "reference",
"#(3,p)": "personDetails"
}
}
}
}
}
]
the demo on the site https://jolt-demo.appspot.com/ is :

Jolt transform array to dissipate an outer attribute to each sub-object of an array

I want to apply a Jolt Transformation, but this is still cloudy on my side:
This is my input data:
{
"results": 1,
"data": [
{
"detail": [
{
"num": "140"
},
{
"num": "158"
},
{
"num": "180"
},
{
"num": "183"
},
{
"num": "213"
}
],
"code": "01007"
}
],
"response_code": 200
}
My desired output:
[
{
"code": "01007",
"num": "140"
},
{
"code": "01007",
"num": "158"
},
....
{
"code": "01007",
"num": "213"
}
]
And my trial for the JOLT specification so far, I do not understand how to add a custom field to all elements of the list:
[
{
"operation": "shift",
"spec": {
"data": {
"*": {
"detail": {
"*": {
"code": "[&1].code",
"#": "[&1]"
}
}
}
}
}
}
]
You can use the shift transformation like this
[
{
"operation": "shift",
"spec": {
"data": {
"*": {
"detail": {
"*": {
"#(2,code)": "[&1].code", // to go 2 levels up to reach the level of the attribute "code"
"*": "[&1].&" // to get the values of the existing attributes within the current object without object wrapper
}
}
}
}
}
}
]
applying just some slight changes.
the demo on the site http://jolt-demo.appspot.com/ is

JSON transformation using JOLT for Multiple arrays

I am new new to JSON. I have Input JSON which includes two arrays. 1st one (main results) contains two vendor and other one (BankDetailSet ->results) has vendor bank details (Key and Account ).i-e vendor can have multiple accounts. In put each vendor has two accounts. I am trying with Jolt, in output, I am able to get Vendor info and bank (bsb and account_number). The issue is corresponding vendor bank (bsb and account_number ) have been populated with wrong details.
In Input
1st vendor (Chigo PvtLimited") has BankKey (9877988787 & 89797879798)
while second one (UFCDD Pvt Limited) has BankKey (652588887 & 294454545)
In Out Put 1st vendor (Chigo PvtLimited") has bsb (9877988787 & 652588887)
while second one (UFCDD Pvt Limited) has bsb (89797879798 & 294454545)
The question is why Bankey values have been interchanged in output (bsb). I have pasted below the current and expected output
The Input is :
{
"d": {
"results": [
{
"__metadata": {
"type": "Core vendor.Vendor"
},
"VendorNumber": "7779898",
"VendorName": "Chigo PvtLimited",
"BankDetailSet": {
"results": [
{
"__metadata": {
"type": "UFCDR Pvt Limited"
},
"BankKey": "9877988787",
"BankAccount": "987788798778879"
},
{
"__metadata": {
"type": "UFCDR Pvt Limited"
},
"BankKey": "89797879798",
"BankAccount": "564654456456465"
}
]
}
},
{
"__metadata": {
"type": "Alpha vendor.Vendor"
},
"VendorNumber": "987545",
"VendorName": "UFCDD Pvt Limited",
"BankDetailSet": {
"results": [
{
"__metadata": {
"type": "UFCDD.BankDetail"
},
"BankKey": "652588887",
"BankAccount": "66887454"
},
{
"__metadata": {
"type": "UFCDR Pvt Limited"
},
"BankKey": "294454545",
"BankAccount": "4578777"
}
]
}
}
]
}
}
the Spec I've tried :
[
{
"operation": "shift",
"spec": {
"d": {
"results": {
"*": {
"VendorNumber": "vendors.[&1].name",
"VendorName": "vendors.[&1].VendorName",
"BankDetailSet": {
"results": {
"*": {
"BankKey": "vendors[&1].Bank[&4].bsb",
"BankAccount": "vendors[&1].Bank[&4].account_number"
}
}
}
}
}
}
}
}
]
The Current Output
{
"vendors": [{
"name": "7779898",
"VendorName": "Chigo PvtLimited",
"Bank": [{
"bsb": "9877988787",
"account_number": "987788798778879"
}, {
"bsb": "652588887",
"account_number": "66887454"
}]
}, {
"Bank": [{
"bsb": "89797879798",
"account_number": "564654456456465"
}, {
"bsb": "294454545",
"account_number": "4578777"
}],
"name": "987545",
"VendorName": "UFCDD Pvt Limited"
}]
}
and the Expected Output
{
"vendors": [
{
"name": "7779898",
"VendorName": "Chigo PvtLimited",
"Bank": [
{
"bsb": "9877988787",
"account_number": "987788798778879"
},
{
"bsb": "89797879798",
"account_number": "564654456456465"
}
]
},
{
"name": "987545",
"VendorName": "UFCDD Pvt Limited",
"Bank": [
{
"bsb": "652588887",
"account_number": "66887454"
},
{
"bsb": "294454545",
"account_number": "4578777"
}
]
}
]
}
You can collect the elements "VendorNumber", "VendorName" and the object with key name "BankDetailSet" under common object notation, then set the relative positioning wildcards such as [&1], [&4] to meet at the same level of indexes of the outermost "results" array such as
[
{
"operation": "shift",
"spec": {
"d": {
"results": {
"*": {
"VendorNu*": "vendors[&1].name",
"VendorNa*": "vendors[&1].&",
"Bank*": {
"results": {
"*": {
"Bank*": "vendors[&1].Bank[&4].&(0,1)"
}
}
}
}
}
}
}
}
]
the demo on the site http://jolt-demo.appspot.com/ is
Edit : What you need within the last comments is just reverse of the previous one, eg. use
"Bank*": "vendors[&4].Bank[&1].&(0,1)"
instead of
"Bank*": "vendors[&1].Bank[&4].&(0,1)"
or literally use (as in your case)
"BankKey": "vendors[&4].Bank[&1].bsb",
"BankAccount": "vendors[&4].Bank[&1].account_number"
such as
[
{
"operation": "shift",
"spec": {
"d": {
"results": {
"*": {
"VendorNu*": "vendors[&1].name",
"VendorNa*": "vendors[&1].&",
"Bank*": {
"results": {
"*": {
"BankKey": "vendors[&4].Bank[&1].bsb",
"BankAccount": "vendors[&4].Bank[&1].account_number"
}
}
}
}
}
}
}
}
]
the demo is
# Barbaros Özhan, Thank you so much, below one is producing the required result..
[
{
"operation": "shift",
"spec": {
"d": {
"results": {
"*": {
"VendorNu*": "vendors[&1].name",
"VendorNa*": "vendors[&1].&",
"Bank*": {
"results": {
"*": {
"BankKey": "vendors[&4].Bank[&1].bsb",
"BankAccount": "vendors[&4].Bank[&1].account_number"
}
}
}
}
}
}
}
}
]

Jolt transformation array data

I want to transform a JSON using JOLT like this:
Input: {
"array": [
"1","2","3","4"
],
"array2": [
{
"something": "123",
"something1": "Plane"
},
{
"something3": "567",
"something4": "Car"
}
]
}
Into the following format, as you can see from output I need data from both arrays to fit exact param names, not empty one like in the first or existed param names like in the second one.
Output: {
"one_array": [
{
"code": "1",
"description": "",
},
{
"code": "2",
"description": "",
},
{
"code": "3",
"description": "",
},
{
"code": "4",
"description": "",
}
], "other_array": [
{
"id": "123",
"type": "Plane"
},
{
"id": "567",
"type": "Car"
}
]
}
Some clarifications are really appreciated
You can achieve this using 2 shift operations and the default operation as below.
[
{
"operation": "shift",
"spec": {
"array": {
"*": {
"#": "one_array[&].id"
}
},
"array2": {
"*": {
"*": {
"#": "tmp_array[&2]"
}
}
}
}
},
{
"operation": "shift",
"spec": {
"one_array": "one_array",
"tmp_array": {
"*": {
"0": "other_array[&1].id",
"1": "other_array[&1].type"
}
}
}
},
{
"operation": "default",
"spec": {
"one_array[]": {
"*": {
"description": ""
}
}
}
}
]