Combine two values to an object with a JOLT transformation - json

I am pretty new to Jolt. The source format was generated from some software and does not have a good structure, so I would like to transform this. Each text ("First text" and "Second text") has an associated link ("First text" -> "Link of first text"; "Second text" -> "Link of second text"). How would you do such a transformation? Any help please?
Sample Input:
{
"flow": [
"^First text",
{
"*": ".^.c-0"
},
"^Second text",
{
"*": ".^.c-1"
},
{
"c-0": [
{
"->": "Link of first text"
}
],
"c-1": [
{
"->": "Link of second text"
}
]
}
]
}
Expected Output:
{
"flow": {
"links": [
{
"message": "^First text",
"link": "Link of first text"
},
{
"message": "^Second text",
"link": "Link of second text"
}
]
}
}

What you need is to use shift transformation.
One option is to create individual arrays(lists) within the first,
and then dissipate them to separate objects within the second transformation such as
[
{
"operation": "shift",
"spec": {
"flow": {
"*": {
"*text": { // filter out by the key names ending with "text"
"$": "&3.message" // go 3 levels up to grab the key name "flow"
},
"c*": { // filter out by the key names starting with "c"
"*": {
"*": "&4.link" // go 4 levels up to grab the key name "flow"
}
}
}
}
}
},
{
"operation": "shift",
"spec": {
"*": { // stands for "flow"
"*": {
"*": "&2.links[&].&1" // &2 -> go 2 levels up to grab the key name "flow"
// &1 -> go 1 level up to grab the key names of the arrays
}
}
}
}
]
the demo on the site http://jolt-demo.appspot.com/ is

Related

How does the jolt wildcards work for RHS?

I'm trying to understand the working of '#' wild card.
I have used the '#' in the spec while preparing the productList and it works and I got the output as per my expectation. But I'm not sure about the working of it.
Can anyone please help me to understand the working of it?
Here is the input JSON
{
"orders": [
{
"order_parts": [
{
"id": "0001",
"items": [
{
"id": "00101",
"goodIdentificationList": [
{
"goodIdentificationTypeId": "UPCA",
"idValue": "42684666380437"
},
{
"idValue": "V-ASHBY",
"goodIdentificationTypeId": "SHOPIFY_PROD_SKU"
}
],
"productName": "BLACK / 6 / 809"
},
{
"id": "00102",
"goodIdentificationList": [
{
"goodIdentificationTypeId": "SHOPIFY_PROD_ID",
"idValue": "42684666380437"
},
{
"idValue": "V-ASHBY",
"goodIdentificationTypeId": "UPCA"
}
],
"productName": "BLACK / 6 / 809"
}
]
},
{
"id": "0002",
"items": [
{
"id": "00103",
"goodIdentificationList": [
{
"goodIdentificationTypeId": "SHOPIFY_PROD_ID",
"idValue": "42684666380437"
},
{
"idValue": "V-ASHBY",
"goodIdentificationTypeId": "UPCA"
}
],
"productName": "BLACK / 6 / 809"
}
]
}
]
}
]
}
Expected by the below spec:-
Check every map of goodIdentificationList, get the idValue where goodIdentificationTypeId - UPCA, and put in the productList as gtin.
Get the id from the items list and put it in the productList as an itemId.
Get the productName from the items list and put it in the productList as a name.
Jolt Spec is like below
[
{
"operation": "shift",
"spec": {
"orders": {
"*": {
"order_parts": {
"*": {
"items": {
"*": {
"goodIdentificationList": {
"*": {
"goodIdentificationTypeId": {
"UPCA": {
"#(2,idValue)": "[&5].productList.[#8].gtin"
}
}
}
},
"id": "[&1].productList.[#4].itemId",
"productName": "[&1].productList.[#4].name"
}
}
}
}
}
}
}
},
{
"operation": "shift",
"spec": {
"*": {
"productList": {
"*": "productList.[]"
}
}
}
}
]
By the above spec, I'm able to prepare the productList as I was expecting.
But want to understand the working of '#' here.
Output JSON
{
"productList" : [ {
"itemId" : "00101",
"name" : "BLACK / 6 / 809"
}, {
"itemId" : "00103",
"name" : "BLACK / 6 / 809"
}, {
"itemId" : "00102",
"name" : "BLACK / 6 / 809"
} ]
}
Any help will be appreciated.
Thanks!
You principally need this spec
[
{
"operation": "shift",
"spec": {
"order*": {
"*": {
"order*": {
"*": {
"items": {
"*": {
"id": "&3.&1.&",
"prod*": "&3.&1.&"
}
}
}
}
}
}
}
},
{
"operation": "shift",
"spec": {
"*": {
"*": "productList[]"
}
}
}
]
in which, the combination of the indexes of "items" array(&1) vs. the indexes of "order_parts" array(&3) handles separation of three individual objects, and then then tiding up innermost part while tagging the desired value(productList), and combining those newly generated objects will spontaneously form an array.
Replacement of
"id": "&3.&1.&",
"prod*": "&3.&1.&"
with
"id": "&3[&1].&",
"prod*": "&3[&1].&"
or
"id": "&3[#2].&",
"prod*": "&3[#2].&"
would handle exactly the same for this current case. But if there was no array the identifiers [&1] or [#2] would produce square brackets. The difference for right-hand-size usage of them is [&1] will traverse {, while [#2] will traverse both { and the current : characters in order to reach the target within the current tree, and [&1] will use indexes 0,1,2, which starts from zero, everytime and this will generate some null components for generated arrays for some cases, while [#2] won't. Eg. On the RHS of the spec, # is only valid in the the context of an array, like "[#2]".What "[#2]" means is, go up the three levels(including the colon at the current level) and ask that node how many matches it has had, and then use that as an index in the arrays.

JOLT shift transformation to filter values in array

I want to use a JOLT transformation to do two things:
filter the elements in the array called myarray so that only elements remain which have a "v_518" attribute
filter out all attributes of the remaining elements except for "v_518" and "lfdn"
Input:
{
"isError": false,
"isValid": true,
"myarray": [
{
"p_0001": "1",
"p_0002": "1",
"p_0003": "1",
"p_0004": "1",
"v_518": "0,214506186",
"lfdn": 89709
},
{
"p_0001": "2",
"p_0002": "1",
"p_0003": "1",
"v_518": "0,3823236",
"lfdn": 89710
},
{
"p_0001": "3",
"p_0002": "1",
"p_0003": "1",
"lfdn": 89711
}
],
"errorMessage": null,
"exceptionMessage": null,
"innerExceptionMessage": null
}
Desired output:
{
"isError": false,
"isValid": true,
"myarray": [
{
"v_518": "0,214506186",
"lfdn": 89709
},
{
"v_518": "0,3823236",
"lfdn": 89710
}
],
"errorMessage": null,
"exceptionMessage": null,
"innerExceptionMessage": null
}
What I tried so far, but doesn't work as intended:
[
{
"operation": "shift",
"spec": {
"isError": "isError",
"isValid": "isValid",
"myarray": {
// loop thru all the elements in value array
"*": {
"v_518": {
// if the value "v_518" exists
// grab the whole object and write it out to
// a v_518_array array.
"#(1,v_518)": "v_518_array",
"#(1,lfdn)": "v_518_array"
}
}
},
"errorMessage": "errorMessage",
"exceptionMessage": "exceptionMessage",
"innerExceptionMessage": "innerExceptionMessage"
}
}
]
I tried working with the examples in http://jolt-demo.appspot.com/#andrewkcarter2 but I couldn't figure out how to do it.
I was able to solve my issue. This answer was the hint I needed to get the ball rolling: https://stackoverflow.com/a/38154541/1561441
The key is referencing the array you are currently transforming via "value" = "array[&1].value".
In my mind I spent way too much time on this issue. Does anyone know of a good documentation for the Jolt syntax? I couldn't find a satisfactory one by googling myself.
[
{
"operation": "shift",
"spec": {
"isError": "isError",
"isValid": "isValid",
"myarray": {
// loop thru all the elements in value array
"*": {
"v_518": {
// if the value "v_518" exists
// grab the whole object and write it out to
// a v_518_array array.
"#1": "v_518_array"
}
}
},
"errorMessage": "errorMessage",
"exceptionMessage": "exceptionMessage",
"innerExceptionMessage": "innerExceptionMessage"
}
},
{
"operation": "shift",
//Transform array: https://stackoverflow.com/questions/37865871/how-do-i-transform-an-array-using-jolt
"spec": {
"v_518_array": {
// loop thru all the elements in value array
"*": {
"v_518": "v_518_array[&1].v_518",
"lfdn": "v_518_array[&1].lfdn"
}
}
}
}
]
Here's a slightly better solution:
[
{
"operation": "shift",
"spec": {
"isError": "isError",
"isValid": "isValid",
"myarray": {
// loop thru all the elements in value array
"*": {
"v_518": {
// if the value "v_518" exists
// grab the whole object and write it out to
// a v_518_array array.
"#1": "v_518_array"
}
}
},
"errorMessage": "errorMessage",
"exceptionMessage": "exceptionMessage",
"innerExceptionMessage": "innerExceptionMessage"
}
},
{
"operation": "shift",
//Transform array: https://stackoverflow.com/questions/37865871/how-do-i-transform-an-array-using-jolt
"spec": {
"v_518_array": {
// loop thru all the elements in value array
"*": {
"v_518": "&2[&1].v_518", //notice the generic shorthand here
"lfdn": "&2[&1].lfdn"
}
}
}
}
]
Still there's a briefer, and more dynamic method to figure out your issue such as
[
{
"operation": "shift",
"spec": {
"myarray": {
"*": {
"v_518": {
"#(1,&)": "&3.[&2].&",
"#(1,lfdn)": "&3.[&2].lfdn"
}
}
},
"*": "&"
}
}
]

unable to convert json list to objects using jolt

I need to use jolt transform to do the below JSON transformation.
need to create new columns from the list from reeval column where sometimes we only one value and some times we get multiple values my input data :-
example 1:
{
"id":"1",
"reeval":["one","two"]
}
example 2:
{
"id":"2",
"reeval":["one","two","three"]
}
example 3:
{
"id":"3",
"reeval":["one"]
}
I have written jolt expresson as below
[
{
"operation": "shift",
"spec": {
"id": "id",
"reeval": {
"*": "&"
}
}
}
]
with above jolt expression is working fine but unable to add column name
output for above jolt is as below
example 1:
{
"id" : "1",
"0" : "one",
"1" : "two"
}
example 2:
{
"id" : "2",
"0" : "one",
"1" : "two",
"2" : "three"
}
here i am unable to change the names of the columns as i need to change colunms as below
my expected output after jolt transformation should be like
example 1:
{
"id":"1",
"reeval":"one",
"reeval1":"two"
}
example 2:
{
"id":"2",
"reeval":"one",
"reeval1":"two",
"reeval2":"three"
}
example 3:
{
"id":"3",
"reeval":"one"
}
Prepending &1 to the current ampersand would suffice in order to go one level up the tree, and to grab the key name in the first shift transformation, and then apply another to rename only the key with index zero such as
[
{
"operation": "shift",
"spec": {
"id": "id",
"reeval": {
"*": "&1&"
}
}
},
{
"operation": "shift",
"spec": {
"reeval0": "reeval",
"*": "&"
}
}
]

How can I combine two arrays to create a key value pair with Jolt?

I've already created a spec to convert my JSON input
{
"rows": [
{
"row": [
"row1",
"row2",
"row3"
],
"header": [
"header1",
"header2",
"header3"
]
},
{
"row": [
"row4",
"row5",
"row6"
],
"header": [
"header4",
"header5",
"header6"
]
}
]
}
to convert to key-value pairs as following object result :
{
"header1" : "row1",
"header2" : "row2",
"header3" : "row3",
"header4" : "row4",
"header5" : "row5",
"header6" : "row6"
}
Is this possible to do using Jolt?
Is there a copy/paste error in your input? Judging by your desired output, the second object's header array should be ["header4", "header5", "header6"]. If that's the case, this spec should work:
[
{
"operation": "shift",
"spec": {
"rows": {
"*": {
"header": {
"*": {
"*": {
"#(3,row[#2])": "&"
}
}
}
}
}
}
}
]
One option is to use the following shift transformation spec :
[
{
"operation": "shift",
"spec": {
"*s": { // rows array
"*": {
"&(1,1)": { // row array
"*": {
"#": "#(3,header[&1])"
}
}
}
}
}
}
]
where
"*s": { stands for rows
"&(1,1)": { -> not immediate(zeroth) level but one more level up by using &(1, and grab the value there the first asterisk exists by &(..,1)
"#": "#(3,header[&1])" -> 3 levels needed as stated at the right hand side traverse the colon
as well in order to reach the level of &(1,1) which is used to
represent the "row" array along with &1 representation to go one level up the tree to reach the indexes of the array "row" while matching with the values of "row" through use of # on the left hand side
the demo on the site http://jolt-demo.appspot.com/ is :

How can I use "not equal" condition while filtering array using JOLT specification

I want to filter JSON array using JOLT transformation, where condition is negative. In the below example I want only records where URL value is not equal to Not Available.
{
"Photos": [
{
"Id": "327703",
"Caption": "TEST>> photo 1",
"Url": "Not Available."
},
{
"Id": "327704",
"Caption": "TEST>> photo 2",
"Url": "http://bob.com/0001/327704/photo.jpg"
},
{
"Id": "327705",
"Caption": "TEST>> photo 3",
"Url": "http://bob.com/0001/327705/photo.jpg"
}
]
}
Take a look on very similar question Removing Elements from array based on a condition. Based on it you can solve it as below:
[
{
"operation": "shift",
"spec": {
"Photos": {
// loop thru all the photos
"*": {
// for each URL
"Url": {
// For "Not Available." do nothing.
"Not Available.": null,
// In other case pass thru
"*": {
"#2": "Photos[]"
}
}
}
}
}
}
]
Generally when you want to negate filter you do a filter and as transformation pass null which skips item.