Jolt transform json - how to add default fields - json

I have below input JSON:
{
"id": "2ef8a2ee-054f-4b43-956a-8aa4f51a41d5",
"type": "VOICE",
"tags": [
{
"id": "some id 1",
"description": "some description 1"
},
{
"id": "some id 2",
"description": "some description 2"
}
],
"transcription": {
"key1": "val1",
"key2": "val2"
}
}
But, output JSON should look like similarly, and add only default values:
{
"id": "2ef8a2ee-054f-4b43-956a-8aa4f51a41d5",
"created": "2019-06-18T18:12:37",
"firstName": "Khusan",
"lastName": "Sharipov",
"status": "OPEN"
"type": "VOICE",
"tags": [
{
"id": "some id 1",
"description": "some description 1"
},
{
"id": "some id 2",
"description": "some description 2"
}
],
"transcription": {
"key1": "val1",
"key2": "val2"
}
}
This is my JOLT spec:
[
{
"operation": "shift",
"spec": {
}
},
{
"operation": "shift",
"spec": {
"*": {
"": "TRASH",
"*": {
"$": "&2"
}
}
}
},
{
"operation": "remove",
"spec": {
"TRASH": ""
}
},
{
"operation": "default",
"spec": {
"firstName": "Khusan",
"lastName": "Sharipov",
"status": "OPEN"
}
}
]
I should edit JOLT spec but I do not understand how ( default fields first name, last name and status work. created can be added as "created": "#(3,ninjaed-in-time)"

If you want to add only new fields you need to use only default operation like below:
[
{
"operation": "default",
"spec": {
"firstName": "Khusan",
"lastName": "Sharipov",
"status": "OPEN"
}
}
]

Related

JOLT spec to populate data based on tag presence

I have requirement where input will have one or two arrays based on scenarios:
Input 1:
{
"name": "xyz",
"age": "30",
"addressA": [
{
"id": "111111",
"details": ""
},
{
"id": "111112",
"details": ""
}
],
"addressB": [
{
"id": "222222",
"details": ""
},
{
"id": "222223",
"details": ""
}
]
}
Input 2:
{
"name": "xyz",
"age": "30",
"addressB": [
{
"id": "222222",
"details": ""
},
{
"id": "222223",
"details": ""
}
]
}
So when 'addressA' is present in input json it should use that to give output if its no present then it should use 'addressB' to populate the same
Output expected:
For input 1:
[
{
"id": "111111",
"details": ""
},
{
"id": "111112",
"details": ""
}
]
For Input 2:
[
{
"id": "222222",
"details": ""
},
{
"id": "222223",
"details": ""
}
]
Any help would be appreciated.
I see option in JOLT to have if-else based on string value, but could not find anything to check just if exists
Thanks
Mahendra
You can determine the arrays A and B within the first shift transformation spec, and then use conditional within the second one such as
[
{
"operation": "shift",
"spec": {
"address*": {
"*": "&(1,1)" // in order to determine the labels of the arrays as "A" and "B"
}
}
},
{
"operation": "shift",
"spec": {
"A": {
"#": ""
},
"*": "&"
}
},
{
"operation": "shift",
"spec": {
"*": ""
}
}
]

Applying cardinality for multiple columns in Jolt

I am trying to apply Jolt for below data
input:
[
{
"id": "500",
"code": "abc",
"date": "2020-10-10",
"category": 1,
"amount": 100,
"result": 0
},
{
"id": "500",
"code": "abc",
"date": "2020-10-10",
"category": 2,
"amount": 200,
"result": 1
}
]
jolt used:
[
{
"operation": "shift",
"spec": {
"*": {
"id": "#(1,id).id",
"code": "#(1,id).code",
"date": "#(1,id).group1.date",
"category": "#(1,id).group1.group2[&1].category"
}
}
},
{
"operation": "cardinality",
"spec": {
"*": {
"id": "ONE"
}
}
},
{
"operation": "shift",
"spec": {
"*": ""
}
}
]
current output:
{
"id": "500",
"code": [
"abc",
"abc"
],
"group1": {
"date": [
"2020-10-10",
"2020-10-10"
],
"group2": [
{
"category": 1
},
{
"category": 2
}
]
}
}
expected:
{
"id": "500",
"code": "abc",
"group1": {
"date": "2020-10-10",
"group2": [
{
"category": 1
},
{
"category": 2
}
]
}
}
If i keep column of code & date in cardinality, it's fine. But in my use case, there are multiple such columns to be added. Are there any better ways to handle this scenario?
You should add each added node and use "*" wildcard to represent the rest of the attributes within the cardinality transformation such as
{
"operation": "cardinality",
"spec": {
"*": {
"*": "ONE",
"group1": {
"*": "ONE",
"group2": "MANY"
}
}
}
}
where "group2": "MANY" will make group2 to be excepted for extracting only the first element of the respective list.
the demo on the site http://jolt-demo.appspot.com/ :

Jolt Conversion - iterate list within a list and form single list

I am trying to iterate lists inside a list and form a single list with multiple objects. Iterating lists, I am able to achieve. But applying tags before the list iterating to each object is not happening if there are more objects in single list.
My input request is like below:
[
{
"success": [
{
"id": "4",
"Offers": [
{
"name": "Optional",
"type": {
"id": "1",
"name": "Optional"
},
"productOfferings": [
{
"id": "3",
"name": "Test1"
}
]
},
{
"name": "Default",
"type": {
"id": "2",
"name": "Default"
},
"productOfferings": [
{
"id": "1",
"name": "Test2"
},
{
"id": "2",
"name": "Test3"
}
]
}
]
}
]
}
]
My spec is like below:
[
{
"operation": "shift",
"spec": {
"*": {
"success": {
"*": {
"Offers": {
"*": {
"name": "[&1].[&3].typeName",
"type": {
"id": "[&2].[&4].typeNameId",
"name": "[&2].[&4].typeNameValue"
},
"productOfferings": {
"*": {
"id": "[&3].[&1].id",
"name": "[&3].[&1].name"
}
}
}
}
}
}
}
}
},
{
"operation": "shift",
"spec": {
"*": {
"*": "[]"
}
}
}
]
Output Received from spec:
[
{
"typeName": "Optional",
"typeNameId": "1",
"typeNameValue": "Optional",
"id": "3",
"name": "Test1"
},
{
"typeName": "Default",
"typeNameId": "2",
"typeNameValue": "Default",
"id": "1",
"name": "Test2"
},
{
"id": "2",
"name": "Test3"
}
]
But Expected output is like below:
[
{
"typeName": "Optional",
"typeNameId": "1",
"typeNameValue": "Optional",
"id": "3",
"name": "Test1"
},
{
"typeName": "Default",
"typeNameId": "2",
"typeNameValue": "Default",
"id": "1",
"name": "Test2"
},
{
"typeName": "Default",
"typeNameId": "2",
"typeNameValue": "Default",
"id": "2",
"name": "Test3"
}
]
If there are more objects inside productOfferings object, I am not able to add typeName,typeNameId, typeNameValue to the actual object. Please help to fix this issue.
You seem just needing to collect all into the productOfferings array while prepending each key by the common identifier [&3].[&1] such as
[
{
"operation": "shift",
"spec": {
"*": {
"success": {
"*": {
"Offers": {
"*": {
"productOfferings": {
"*": {
"#(2,name)": "[&3].[&1].typeName",
"#(2,type.id)": "[&3].[&1].typeNameId",
"#(2,type.name)": "[&3].[&1].typeNameValue",
"*": "[&3].[&1].&"
}
}
}
}
}
}
}
}
},
{
"operation": "shift",
"spec": {
"*": {
"*": "[]"
}
}
}
]

jolt - copy or move a key from nested object to the top level

I'm looking for a way to copy or move a key from nested object to the top level
Input:
{
"id": "123",
"name": "foo",
"details": {
"orderNumber": "456789",
"addr": "N st 124",
"date": "2021-01-01"
}
}
desired output:
{
"id": "123",
"name": "foo",
"orderNumber": "456789",
"details": {
"orderNumber": "456789",
"addr": "N st 124",
"date": "2021-01-01"
}
}
or ideally
{
"id": "123",
"name": "foo",
"orderNumber": "456789",
"details": {
"addr": "N st 124",
"date": "2021-01-01"
}
}
the closest I could get is below transformation, but it converts object to value array
[
{
"operation": "shift",
"spec": {
"id": "id",
"name": "name",
"details": {
"orderNumber": "orderNumber",
"*": "details"
}
}
}
]
You're so close to the result, just a slight change(adding an ampersand) is needed such as
[
{
"operation": "shift",
"spec": {
"id": "id",
"name": "name",
"details": {
"orderNumber": "orderNumber",
"*": "&1.&"
}
}
}
]
in this case the keys keeps on appearing.

Jolt transformation array data

I want to transform a JSON using JOLT like this:
Input: {
"array": [
"1","2","3","4"
],
"array2": [
{
"something": "123",
"something1": "Plane"
},
{
"something3": "567",
"something4": "Car"
}
]
}
Into the following format, as you can see from output I need data from both arrays to fit exact param names, not empty one like in the first or existed param names like in the second one.
Output: {
"one_array": [
{
"code": "1",
"description": "",
},
{
"code": "2",
"description": "",
},
{
"code": "3",
"description": "",
},
{
"code": "4",
"description": "",
}
], "other_array": [
{
"id": "123",
"type": "Plane"
},
{
"id": "567",
"type": "Car"
}
]
}
Some clarifications are really appreciated
You can achieve this using 2 shift operations and the default operation as below.
[
{
"operation": "shift",
"spec": {
"array": {
"*": {
"#": "one_array[&].id"
}
},
"array2": {
"*": {
"*": {
"#": "tmp_array[&2]"
}
}
}
}
},
{
"operation": "shift",
"spec": {
"one_array": "one_array",
"tmp_array": {
"*": {
"0": "other_array[&1].id",
"1": "other_array[&1].type"
}
}
}
},
{
"operation": "default",
"spec": {
"one_array[]": {
"*": {
"description": ""
}
}
}
}
]