JSON transformation using JOLT for Multiple arrays - json

I am new new to JSON. I have Input JSON which includes two arrays. 1st one (main results) contains two vendor and other one (BankDetailSet ->results) has vendor bank details (Key and Account ).i-e vendor can have multiple accounts. In put each vendor has two accounts. I am trying with Jolt, in output, I am able to get Vendor info and bank (bsb and account_number). The issue is corresponding vendor bank (bsb and account_number ) have been populated with wrong details.
In Input
1st vendor (Chigo PvtLimited") has BankKey (9877988787 & 89797879798)
while second one (UFCDD Pvt Limited) has BankKey (652588887 & 294454545)
In Out Put 1st vendor (Chigo PvtLimited") has bsb (9877988787 & 652588887)
while second one (UFCDD Pvt Limited) has bsb (89797879798 & 294454545)
The question is why Bankey values have been interchanged in output (bsb). I have pasted below the current and expected output
The Input is :
{
"d": {
"results": [
{
"__metadata": {
"type": "Core vendor.Vendor"
},
"VendorNumber": "7779898",
"VendorName": "Chigo PvtLimited",
"BankDetailSet": {
"results": [
{
"__metadata": {
"type": "UFCDR Pvt Limited"
},
"BankKey": "9877988787",
"BankAccount": "987788798778879"
},
{
"__metadata": {
"type": "UFCDR Pvt Limited"
},
"BankKey": "89797879798",
"BankAccount": "564654456456465"
}
]
}
},
{
"__metadata": {
"type": "Alpha vendor.Vendor"
},
"VendorNumber": "987545",
"VendorName": "UFCDD Pvt Limited",
"BankDetailSet": {
"results": [
{
"__metadata": {
"type": "UFCDD.BankDetail"
},
"BankKey": "652588887",
"BankAccount": "66887454"
},
{
"__metadata": {
"type": "UFCDR Pvt Limited"
},
"BankKey": "294454545",
"BankAccount": "4578777"
}
]
}
}
]
}
}
the Spec I've tried :
[
{
"operation": "shift",
"spec": {
"d": {
"results": {
"*": {
"VendorNumber": "vendors.[&1].name",
"VendorName": "vendors.[&1].VendorName",
"BankDetailSet": {
"results": {
"*": {
"BankKey": "vendors[&1].Bank[&4].bsb",
"BankAccount": "vendors[&1].Bank[&4].account_number"
}
}
}
}
}
}
}
}
]
The Current Output
{
"vendors": [{
"name": "7779898",
"VendorName": "Chigo PvtLimited",
"Bank": [{
"bsb": "9877988787",
"account_number": "987788798778879"
}, {
"bsb": "652588887",
"account_number": "66887454"
}]
}, {
"Bank": [{
"bsb": "89797879798",
"account_number": "564654456456465"
}, {
"bsb": "294454545",
"account_number": "4578777"
}],
"name": "987545",
"VendorName": "UFCDD Pvt Limited"
}]
}
and the Expected Output
{
"vendors": [
{
"name": "7779898",
"VendorName": "Chigo PvtLimited",
"Bank": [
{
"bsb": "9877988787",
"account_number": "987788798778879"
},
{
"bsb": "89797879798",
"account_number": "564654456456465"
}
]
},
{
"name": "987545",
"VendorName": "UFCDD Pvt Limited",
"Bank": [
{
"bsb": "652588887",
"account_number": "66887454"
},
{
"bsb": "294454545",
"account_number": "4578777"
}
]
}
]
}

You can collect the elements "VendorNumber", "VendorName" and the object with key name "BankDetailSet" under common object notation, then set the relative positioning wildcards such as [&1], [&4] to meet at the same level of indexes of the outermost "results" array such as
[
{
"operation": "shift",
"spec": {
"d": {
"results": {
"*": {
"VendorNu*": "vendors[&1].name",
"VendorNa*": "vendors[&1].&",
"Bank*": {
"results": {
"*": {
"Bank*": "vendors[&1].Bank[&4].&(0,1)"
}
}
}
}
}
}
}
}
]
the demo on the site http://jolt-demo.appspot.com/ is
Edit : What you need within the last comments is just reverse of the previous one, eg. use
"Bank*": "vendors[&4].Bank[&1].&(0,1)"
instead of
"Bank*": "vendors[&1].Bank[&4].&(0,1)"
or literally use (as in your case)
"BankKey": "vendors[&4].Bank[&1].bsb",
"BankAccount": "vendors[&4].Bank[&1].account_number"
such as
[
{
"operation": "shift",
"spec": {
"d": {
"results": {
"*": {
"VendorNu*": "vendors[&1].name",
"VendorNa*": "vendors[&1].&",
"Bank*": {
"results": {
"*": {
"BankKey": "vendors[&4].Bank[&1].bsb",
"BankAccount": "vendors[&4].Bank[&1].account_number"
}
}
}
}
}
}
}
}
]
the demo is

# Barbaros Özhan, Thank you so much, below one is producing the required result..
[
{
"operation": "shift",
"spec": {
"d": {
"results": {
"*": {
"VendorNu*": "vendors[&1].name",
"VendorNa*": "vendors[&1].&",
"Bank*": {
"results": {
"*": {
"BankKey": "vendors[&4].Bank[&1].bsb",
"BankAccount": "vendors[&4].Bank[&1].account_number"
}
}
}
}
}
}
}
}
]

Related

Jolt Transform to append extracted data

I'm new to Jolt Transform and have some EDI data that I want to transform. I want to keep the original data and append a few extracted elements in order to make it easier for a downstream process to get to. Here's a sample:
Input
{
"id": "2000",
"segments": [
{
"REF02": "xxxxxxxxxxxx",
"REF01": "0F",
"id": "REF"
},
{
"REF02": "yyyyyyyyyyyy",
"REF01": "1L",
"id": "REF"
}
],
"loops": [
{
"id": "2100",
"segments": []
},
{
"id": "2100",
"segments": [],
"loops": [
{
"id": "2300",
"segments": [
{
"DTP01": "348",
"DTP03": "20220101",
"DTP02": "D8",
"id": "DTP"
},
{
"DTP01": "349",
"DTP03": "20221231",
"DTP02": "D8",
"id": "DTP"
}
]
}
]
}
]
}
Desired Output
{
"ext": {
"subscriber": "xxxxxxxxxxxx",
"start": "20220101"
},
"id": "2000",
"segments": [
{
"REF02": "xxxxxxxxxxxx",
"REF01": "0F",
"id": "REF"
},
{
"REF02": "yyyyyyyyyyyy",
"REF01": "1L",
"id": "REF"
}
],
"loops": [
{
"id": "2100",
"segments": []
},
{
"id": "2100",
"segments": [],
"loops": [
{
"id": "2300",
"segments": [
{
"DTP01": "348",
"DTP03": "20220101",
"DTP02": "D8",
"id": "DTP"
},
{
"DTP01": "349",
"DTP03": "20221231",
"DTP02": "D8",
"id": "DTP"
}
]
}
]
}
]
}
That is, I'd like to append the ext element - copying values from the original.
In xpath terms, the extracted values' paths look something like this:
subscriber :
/segments[../id eq "2000"]/..[REF01 eq "0F"]/REF02/text()
start :
/loops[../id eq "2000"]/../segments[../id eq "2100"])[1]/..
[id eq "NM1" and NM101 eq "IL"][DTP01 eq "348"]/DTP03/text()
Thanks for the help.
I ended up with this solution. If there's any feedback on streamlining, making more elegant, let me know.
[
{
"operation": "default",
"spec": {
"dummy": {}
}
},
{
"operation": "shift",
"spec": {
"dummy": {
"#2,segments": {
"*": {
"REF01": {
"0F": {
"#2,REF02": "ext.subscriber"
}
}
}
},
"#2,loops": {
"*": {
"id": {
"2100": {
"#2,loops": {
"*": {
"id": {
"2300": {
"#2,segments": {
"*": {
"DTP01": {
"348": {
"#2,DTP03": "ext.start"
}
}
}
}
}
}
}
}
}
}
}
}
},
"id": "id",
"segments": "segments",
"loops": "loops"
}
}
]
You can replicate the whole content by use of "#": "" within a shift transformation while adding less curly braces to nest the inner stuff, at the same time determining the paths to search to be used in the next spec such as
[
{
"operation": "shift",
"spec": {
"#": "",
"segments": {
"*": {//indexes of the array
"#REF02": "ext.su.#REF01"
}
},
"loops": {
"*": {//indexes of the array
"#loops": {
"*": {//indexes of the array
"#segments": {
"*": {//indexes of the array
"#DTP03": "ext.st.#5,id.#3,id.#DTP01"
}
}
}
}
}
}
}
},
{
"operation": "shift",
"spec": {
"ext": {
"#su.0F": "&1.subscriber",
"#st.2100.2300.348": "&1.start"
},
"*": "&" //replicate the other attributes/arrays/objects
}
}
]

Jolt spec to transform fields in selective objects

For my input json, I need to write a Jolt spec that can modify manager attribute by joining the value & position fields in only the user type object and not the other object type account.
The input could be a list of such objects.
Input:
[
{
"user": {
"userName": "Mike",
"manager": {
"value": "Harvey",
"position": "Director"
}
},
"account": {
"userLogin": "Mike1987",
"status": true
}
},
{
"user": {
"userName": "Alex",
"manager": {
"value": "Daniel",
"position": "President"
}
},
"account": {
"userLogin": "Alex12a",
"status": true
}
}
]
Desired output:
[
{
"user": {
"userName": "Mike",
"managerRef": {
"info": "Harvey_Director"
}
},
"account": {
"userLogin": "Mike1987",
"status": true
}
},
{
"user": {
"userName": "Alex",
"managerRef": {
"info": "Daniel_President"
}
},
"account": {
"userLogin": "Alex12a",
"status": true
}
}
]
Could someone please help in creating the spec for this transformation. Thanks.
You can use a modify transformation spec which contains a concat function such as
[
{
"operation": "modify-overwrite-beta",
"spec": {
"*": {
"user": {
"manager": {
"info": "=concat(#(1,value),'_',#(1,position))"
}
}
}
}
},
{
// get rid of existing attributes within the object
"operation": "remove",
"spec": {
"*": {
"u*": {
"m*": {
"p*|v*": ""
}
}
}
}
}
]
the demo on the site http://jolt-demo.appspot.com/ is
An alternative method which includes a shift transformation as desired, based on the comment would be
[
{
"operation": "shift",
"spec": {
"*": {
"user": {
"*": "[&2].&1.&",
"manager": {
"*": "[&3].&2.&1.info"
}
},
"*": "[&1].&"
}
}
},
{
"operation": "modify-overwrite-beta",
"spec": {
"*": {
"user": {
"manager": {
"info": "=join('_',#(1,&))"
}
}
}
}
}
]
the demo for this case would be :

Combine shifted values and default values into a same sub object using Jolt

Input:
{
"banking_account": {
"accounts": [
{
"accountId": "id1"
},
{
"accountId": "id2"
}
]
}
}
Expected output:
{
"Data": {
"Accounts": [
{
"Account": {
"Attribute1": "default",
"Identification": "id1"
}
},
{
"Account": {
"Attribute1": "default",
"Identification": "id2"
}
}
]
}
}
My current spec:
[
{
"operation": "shift",
"spec": {
"banking_account": {
"accounts": {
"*": {
"accountId": "Data.Accounts.[&1].Account.Identification"
}
}
}
}
},
{
"operation": "default",
"spec": {
"Attribute": "default"
}
},
{
"operation": "shift",
"spec": {
"Data": {
"Accounts": {
"*": {
"*": "Data.Accounts.[&1].&",
"#(3,Attribute)": "Data.Accounts.[&1].Account.Attribute1"
}
}
}
}
}
]
Current output:
{
"Data": {
"Accounts": [
{
"Account": [
{
"Attribute1": "default"
},
{
"Identification": "id1"
}
]
},
{
"Account": [
{
"Attribute1": "default"
},
{
"Identification": "id2"
}
]
}
]
}
}
It seems instead of inserting the key-value pair into the existing "Account" sub-object, it's making "Account" as a list, with one sub-object containing "Identification", and another object containing "Attribute1".
Could you help me understand why this happens, and how could I configure to avoid this?
Thanks a lot for your help in advance!
Just prefixing the desired fixed value(default) with a # symbol would handle along with using a single shift transformation spec as in the following
[
{
"operation": "shift",
"spec": {
"banking_account": {
"a*s": {
"*": {
"#default": "Data.A&(2,1)s[&1].A&(2,1).Attribute1", // "&(2,1)" represents going tree two levels up and grabbing the second piece(with index 1 which's after 0) from the literal(`accounts`) splitted by asterisk symbols
"accountId": "Data.A&(2,1)s[&1].A&(2,1).Identification"
}
}
}
}
}
]
the demo on the site http://jolt-demo.appspot.com/ is

Jolt transformation on basis of another conditional value in the json

I am trying to use jolt to transform a json payload and my input json looks like this
{
"root": {
"entities": [
{
"fields": [
{
"name": "ContactType",
"value": "Company"
},
{
"name": "HandlerName",
"value": "XYZ"
},
{
"name": "Reference",
"value": "123-4443"
},
{
"entities": [
{
"parent": {
"fieldName": "DefendantAddress"
},
"fields": [
{
"name": "Address",
"value": "1662369113138 Somewhere Street"
},
{
"name": "PostalCode",
"value": "XXXXX"
}
]
}
],
"name": "DefendantAddress"
}
]
}
]
}
}
I am looking to transform the Address data based on value in ContactType field.
If contact type is Company, the output should be
{
"HandlerName": "XYZ",
"reference": "123-4443",
"companyDetails": {
"address": "1662369113138 Somewhere Street",
"postalCode": "XXXXX"
}
}
If contact type is Person, then output should be
{
"HandlerName": "XYZ",
"reference": "123-4443",
"personDetails": {
"address": "1662369113138 Somewhere Street",
"postalCode": "XXXXX"
}
}
I am able to check using my jolt spec if the contact type is Company or Person but struggling to navigate through the hierarchy to reach the child entities. This is my jolt spec. I am looking for implementation is ???? area or if another solution can be provided for the problem.
[
{
"operation": "shift",
"spec": {
"root": {
"entities": {
"*": {
"fields": {
"*": {
"name": {
"HandlerName": {
"#(2,value)": "handlerName"
},
"Reference": {
"#(2,value)": "reference"
},
"ContactType": {
"#(2,value)": {
"Company": {
?????
}
}
}
}
}
}
}
}
}
}
}
]
You can start by diving deep into the innermost objects in order to match the value and name attributes at this level( "#(0,value)": "personDetails.#(0,name)" ), and then accumulate the subelements from the outer object after going some(6) levels up the tree to reach the level of it. Use conditional within the upcoming shift transformation spec such as
[
{
"operation": "shift",
"spec": {
"root": {
"entities": {
"*": {
"fields": {
"*": {
"entities": {
"*": {
"fields": {
"*": {
"#(6,fields)": { //traverse "{" characters 6 times to reach the level of outer "fields" tag
"*": {
"#(0,value)": "common.#(0,name)"
}
},
"#(0,value)": "p.#(0,name)"
}
}
}
}
}
}
}
}
}
}
},
{
"operation": "cardinality",
"spec": {
"common": {
"*": "ONE"
}
}
},
{
"operation": "shift",
"spec": {
"common": {
"ContactType": {
"Company|Person": { // "|" is "OR" operator
"#(2,HandlerName)": "HandlerName",
"#(2,Reference)": "reference",
"#(3,p)": "personDetails"
}
}
}
}
}
]
the demo on the site https://jolt-demo.appspot.com/ is :

Jolt transformation array data

I want to transform a JSON using JOLT like this:
Input: {
"array": [
"1","2","3","4"
],
"array2": [
{
"something": "123",
"something1": "Plane"
},
{
"something3": "567",
"something4": "Car"
}
]
}
Into the following format, as you can see from output I need data from both arrays to fit exact param names, not empty one like in the first or existed param names like in the second one.
Output: {
"one_array": [
{
"code": "1",
"description": "",
},
{
"code": "2",
"description": "",
},
{
"code": "3",
"description": "",
},
{
"code": "4",
"description": "",
}
], "other_array": [
{
"id": "123",
"type": "Plane"
},
{
"id": "567",
"type": "Car"
}
]
}
Some clarifications are really appreciated
You can achieve this using 2 shift operations and the default operation as below.
[
{
"operation": "shift",
"spec": {
"array": {
"*": {
"#": "one_array[&].id"
}
},
"array2": {
"*": {
"*": {
"#": "tmp_array[&2]"
}
}
}
}
},
{
"operation": "shift",
"spec": {
"one_array": "one_array",
"tmp_array": {
"*": {
"0": "other_array[&1].id",
"1": "other_array[&1].type"
}
}
}
},
{
"operation": "default",
"spec": {
"one_array[]": {
"*": {
"description": ""
}
}
}
}
]