I am trying to transform a JSON using JOLT. This JSON consists of nested arrays and I am not able to transform it correctly. Can someone please help. Thanks.
{
"root": [
{
"id": "1234",
"password": "key1234",
"devices": [
{
"details": {
"deviceType": "tv-iot",
"deviceId": "tv-iot-111"
}
},
{
"details": {
"deviceType": "machine-iot",
"deviceId": "machine-iot-999"
}
}
]
},
{
"id": "6789",
"password": "key6789",
"devices": [
{
"details": {
"deviceType": "phone-iot",
"deviceId": "phone-iot-111"
}
},
{
"details": {
"deviceType": "mobile-iot",
"deviceId": "mobile-iot-999"
}
}
]
}
]
}
This is the spec that I have written.
[
{
"operation": "shift",
"spec": {
"root": {
"*": {
"id": "[&1].userid",
"password": "[&1].pwd",
"devices": {
"*": {
"details": {
"deviceType": "[&2].deviceCategory",
"deviceId": "[&2].deviceUniqueValue"
}
}
}
}
}
}
}
]
The expected JSON that I am looking for is:
[
{
"userid": "1234",
"pwd": "key1234",
"devices": [
{
"details": {
"deviceCategory": "tv-iot",
"deviceUniqueValue": "tv-iot-111"
}
},
{
"details": {
"deviceCategory": "machine-iot",
"deviceUniqueValue": "machine-iot-999"
}
}
]
},
{
"userid": "6789",
"pwd": "key6789",
"devices": [
{
"details": {
"deviceCategory": "phone-iot",
"deviceUniqueValue": "phone-iot-111"
}
},
{
"details": {
"deviceCategory": "mobile-iot",
"deviceUniqueValue": "mobile-iot-999"
}
}
]
}
]
However, I get this wrong output. Somehow, my nested objects are getting transformed into list.
[
{
"userid" : "1234",
"pwd" : "key1234",
"deviceCategory" : [ "tv-iot", "phone-iot" ],
"deviceUniqueValue" : [ "tv-iot-111", "phone-iot-111" ]
},
{
"deviceCategory" : [ "machine-iot", "mobile-iot" ],
"deviceUniqueValue" : [ "machine-iot-999", "mobile-iot-999" ],
"userid" : "6789",
"pwd" : "key6789"
}
]
I am unable to figure out what is wrong. Can someone please help?
UPDATE(Solution): Was able to come up with a shorter spec that works as well !
[
{
"operation": "shift",
"spec": {
"root": {
"*": {
"id": "[&1].userId",
"password": "[&1].pwd",
"*": "[&1].&"
}
}
}
},
{
"operation": "shift",
"spec": {
"*": {
"devices": {
"*": {
"details": {
"deviceType": "[&4].&3.[&2].&1.deviceCategory",
"deviceId": "[&4].&3.[&2].&1.deviceUniqueVal"
}
}
},
"*": "[&1].&"
}
}
}
]
You can start by deep diving into the innermost object while partitioning the sub-objects by id values through a shift transformation such as
[
{
"operation": "shift",
"spec": {
"root": {
"*": {
"devices": {
"*": {
"details": {
"*": {
"#(4,id)": "#(5,id).userid",
"#(4,password)": "#(5,id).pwd",
"#": "#(5,id).devicedetails[&3].&2.&1"
}
}
}
}
}
}
}
},
{
// get rid of top level object names
"operation": "shift",
"spec": {
"*": ""
}
},
{
// get rid of repeating components of each arrays respectively
"operation": "cardinality",
"spec": {
"*": {
"us*": "ONE",
"pwd": "ONE"
}
}
},
{
// determine new key names for attributes respectively
"operation": "modify-overwrite-beta",
"spec": {
"*": {
"*": {
"*": {
"*": {
"deviceCategory": "=(#(1,deviceType))",
"deviceUniqueValue": "=(#(1,deviceId))"
}
}
}
}
}
},
{
// get rid of extra elements generated
"operation": "remove",
"spec": {
"*": {
"*": {
"*": {
"*": {
"deviceType": "",
"deviceId": ""
}
}
}
}
}
}
]
Related
Hi guys,
I'm working on some complex filter which is configured to work for two different JSONs, the difference is that one of them has one more node. Everything works fine except in one corner case and I have difficulties with keeping the JSONs structure.
Here is my case:
JSON 1:
{
"header": {
"something": "something"
},
"payload": {
"decisions": [
{
"sequenceId": "1",
"decision": "serviceToDelete",
"something": "something"
}
],
"elements": [
{
"serviceName": "serviceToDelete",
"something": "something"
}
]
},
"moreStuff": []
}
JSON 2:
{
"message": {
"header": {
"something": "something"
},
"payload": {
"decisions": [
{
"sequenceId": "1",
"decision": "serviceToDelete",
"something": "something"
}
],
"elements": [
{
"serviceName": "serviceToDelete",
"something": "something"
}
]
},
"someStuff": {
"otherStuff": "stuff"
}
},
"moreStuff": []
}
FILTER
[
{
"operation": "shift",
"spec": {
"*": "&",
"message": {
"*": "&1.&",
"payload": {
"*": "&",
"decisions": {
"*": {
"decision": {
"serviceToDelete": null,
"*": {
"#2": "&6.&4"
}
}
}
},
"elements": {
"*": {
"serviceName": {
"serviceToDelete": null,
"*": {
"#2": "&6.&4"
}
}
}
}
},
"workflow": {
"*\\].*": {
"serviceToDelete": null,
"*": {
"#1": "&3.&(2,1).&2"
}
}
}
}
}
},
{
"operation": "shift",
"spec": {
"*": "&",
"workflow": {
"*": {
"*ServiceId": {
"#1": "&3"
}
}
}
}
},
{
"operation": "shift",
"spec": {
"*": "&",
"w*": {
"*": {
"*": {
"#": "message.&3.&"
}
}
}
}
},
{
"operation": "shift",
"spec": {
"*": "&",
"message": {
"*": "&1.&",
"decisions": "&1.payload.decisions",
"elements": "&1.payload.elements"
}
}
},
{
"operation": "shift",
"spec": {
"*": "&",
"payload": {
"*": "&",
"decisions": {
"*": {
"decision": {
"serviceToDelete": null,
"*": {
"#2": "decisions"
}
}
}
},
"elements": {
"*": {
"serviceName": {
"serviceToDelete": null,
"*": {
"#2": "elements"
}
}
}
}
}
}
},
{
"operation": "shift",
"spec": {
"*": "&",
"decisions": "payload.decisions",
"elements": "payload.elements"
}
}
]
OUTPUT 1
{
"header" : {
"something" : "something"
},
"moreStuff" : [ ]
}
OUTPUT 2
{
"message" : {
"header" : {
"something" : "something"
},
"someStuff" : {
"otherStuff" : "stuff"
}
},
"moreStuff" : [ ]
}
As you see here after filter the this node is missing:
"payload": {
"decisions": [],
"elements": []
}
I want to keep the JSON structure. So, this node will be there and just be empty. Like this:
EXPECTED 1:
{
"header": {
"something": "something"
},
"payload": {
"decisions": [],
"elements": []
},
"moreStuff": []
}
EXPECTED 2:
{
"message": {
"header": {
"something": "something"
},
"payload": {
"decisions": [],
"elements": []
},
"someStuff": {
"otherStuff": "stuff"
}
},
"moreStuff": []
}
Keep the "payload" node empty also with "decisions" and "elements".
What you need is to add a default transformation spec along with a new shift spec such as
{
"operation": "default",
"spec": {
"*": {
"payload": {
"decisions": [],
"elements": []
}
}
}
},
{
"operation": "shift",
"spec": {
"*": "&",
"header": {
"*": "&1.&",
"payload": "&"
}
}
}
to the bottom of the current spec, just before the closing square bracket.
I have two following json input and i want to convert it into expected output by jolt
input - 1
{
"alert_array": {
"alerts": [
{
"id": "1234",
"data": {
"parameter": [
{
"key": "level",
"value": "93.5"
}
]
}
}
]
}
}
input - 2
{
"alert_array": {
"alerts": [
{
"id": "1234",
"data": {}
}
]
}
}
expected output - 1
{
"alertArray": [
{
"id": "1234",
"properties": [
{
"key": "level",
"value": "93.5"
}
]
}
]
}
expected output - 2
{
"alertArray": [
{
"id": "1234",
"properties": []
}
]
}
In input-1 data contain some parameter but in input-2 data object is blank
You can use default transformation along with shift transformations
in order to add "properties": [] for the empty case of "data" object for the both of the input
JSON values such as
[
{
"operation": "shift",
"spec": {
"*": {
"alerts": {
"*": {
"id": "alertArray.&",
"*": {
"*": {
"*": {
"id": "alertArray.properties[&1].key",
"*": "alertArray.properties[&1].&"
}
}
}
}
}
}
}
},
{
"operation": "default",
"spec": {
"*": {
"properties": []
}
}
},
{
"operation": "shift",
"spec": {
"*": "&[]"
}
}
]
the demo for the first case :
the demo for the second case :
If there was no such need(to fill up for the "data"), then the following single spec would be sufficient :
[
{
"operation": "shift",
"spec": {
"*": {
"alerts": {
"*": {
"id": "alertArray[&1].&",
"*": {
"*": {
"*": {
"id": "alertArray[&4].properties[&1].key",
"*": "alertArray[&4].properties[&1].&"
}
}
}
}
}
}
}
}
]
I have a problem, I don't understand how to update an attribute with a condition in jolt. For example, I have an Object with an inner array of Items. I need to update an Item attribute if another Item attribute equals to something and to return the Object.
Input:
{
"object": {
"id": "3cf1543e-be4d-11eb-84c0-87ba01ce01e0",
"a": "abc",
"del_sign": false,
"items": [
{
"id": "111",
"del_sign": false
},
{
"id": "222",
"del_sign": false
},
{
"id": "333",
"del_sign": false
}
],
"b": [],
"c": []
}
}
I need:
{
"object": {
"id": "3cf1543e-be4d-11eb-84c0-87ba01ce01e0",
"a": "abc",
"del_sign": false,
"items": [
{
"id": "111",
// here changes to true
"del_sign": true
},
{
"id": "222",
"del_sign": false
},
{
"id": "333",
"del_sign": false
}
],
"b": [],
"c": []
}
}
My current jolt spec:
[
{
"operation": "shift",
"spec": {
"object": {
"items": {
"*": {
"id": {
"111": {
"#2": {
"#true": "del_sign",
"$1": "&3"
}
}
}
}
}
}
}
}
]
You can use two step of shift transformations.
Determine the related del_sign value within a conditional as having two arrays for id and del_sign within the same object(items), and properly format them within the second step such as
[
{
"operation": "shift",
"spec": {
"*": {
"*": "&",
"items": {
"*": {
"id": {
"111": {
"#(2,&1)": "&4.&3.id",
"#true": "&4.&3.del_sign"
},
"*": {
"#1": "&4.&3.&2",
"#(2,del_sign)": "&4.&3.del_sign"
}
}
}
}
}
}
},
{
"operation": "shift",
"spec": {
"*": "&",
"items": {
"*": {
"*": "&2.[#2].&"
}
}
}
}
]
Edit : If there are more attributes other than the current ones(id,del_sign), then prefer using the following code in order not to individually repeat the each key such as
[
{
"operation": "shift",
"spec": {
"*": {
"*": "&",
"items": {
"*": {
"id": {
"111": {
"#2": {
"#true": "&5.&2.del_sign",
"*": "&5.&1.&"
}
},
"*": {
"#2": "&4.&3"
}
}
}
}
}
}
},
{
"operation": "cardinality",
"spec": {
"items": {
"*": {
"del_sign": "ONE"
}
}
}
},
{
"operation": "shift",
"spec": {
"*": "&",
"items": {
"*": {
"*": "&2.[#2].&"
}
}
}
}
]
I need to split the data list into two separate idExists and idNotExists lists, based on whether id exists or not.
Can someone help me with Jolt Specification to achieve the below results?
Input:
{
"data": [
{
"name": "a",
"id": "100"
},
{
"name": "b",
"id": "101"
},
{
"name": "c"
}
]
}
Desired Output:
{
"IdExists": [
{
"name": "a",
"id": "100"
},
{
"name": "b",
"id": "101"
}
],
"IdNotExists": [
{
"name": "c"
}
]
}
I have tried with the below spec but it is not working as expected
[
{
"operation": "modify-default-beta",
"spec": {
"data": {
"": {
"id": "NotExist"
}
}
}
},
{
"operation": "shift",
"spec": {
"data": {
"": {
"id": {
"10*": {
"#2": "IdExists[]"
},
"*": {
"#2": "IdNotExists[]"
}
}
}
}
}
}
]
I had some modification on your spec, added * as it was missing in the selectors.
Modified shifting condition to NotExist and finally shift only name to the IdNotExists array.
[
{
"operation": "modify-default-beta",
"spec": {
"data": {
"*": {
"id": "NotExist"
}
}
}
},
{
"operation": "shift",
"spec": {
"data": {
"*": {
"id": {
"NotExist": {
"#(2,name)": "IdNotExists[].name"
},
"*": {
"#2": "IdExists[]"
}
}
}
}
}
}
]
I'm wondering if it's possible to sort or bring the min value in case of an array of json. I read something about this issue but found nothing.
This is the Input:
{
"intData": [
{
"DATE": "2018",
"NOME": "raf"
},
{
"DATE": "2001",
"NOME": "fabio"
},
{
"DATE": "2002",
"NOME": "fabiola"
}
]
}
I would:
{
"intData": [
{
"DATE": "2001",
"NOME": "fabio"
},
{
"DATE": "2002",
"NOME": "fabiola"
},
{
"DATE": "2018",
"NOME": "raf"
}
]
}
or
{
"DATE": "2001",
"NOME": "fabio"
}
Is it possible?
Ordered Results
The steps are as follows:
Create object with structure: $.DATE.NOME.#
Sort it
Turn it back into an array
[
{
"operation": "shift",
"spec": {
"intData": {
"*": {
"#": "#(1,DATE).#(1,NOME)"
}
}
}
},
{
"operation": "sort"
},
{
"operation": "shift",
"spec": {
"*": {
"*": {
"#": "intData.[]"
}
}
}
}
]
First result
The steps are as follows:
Create object with structure: $.DATE.NOME.#
Sort it
Turn it back into an array
Get first result
[
{
"operation": "shift",
"spec": {
"intData": {
"*": {
"#": "#(1,DATE).#(1,NOME)"
}
}
}
},
{
"operation": "sort"
},
{
"operation": "shift",
"spec": {
"*": {
"*": {
"#": "[]"
}
}
}
},
{
"operation": "shift",
"spec": {
"0": {
"#": ""
}
}
}
]