Jolt Transformation - Move object to array - json

I'm coming to the conclusion that Jolt is beyond me.
With this input data:-
{
"cluster_id": "1",
"data": {
"id": 1,
"types": [
{
"incident_id": 10,
"incident_ref": "AAA",
"incident_code": "123",
"incident_date": "2010-11-15T00:01:00Z"
},
{
"incident_id": 20,
"incident_ref": "BBB",
"incident_code": "456",
"incident_date": "2020-11-15T00:01:00Z"
}
]
}
}
Spec:-
[
{
"operation": "shift",
"spec": {
"cluster_id": "id",
"data": {
"types": {
"*": {
"incident_id": "incidents",
"incident_ref": "incidents"
}
}
}
}
}
]
Gives:-
{
"id" : "1",
"incidents" : [ 10, "AAA", 20, "BBB" ]
}
How would I get the result of:-
{
"id" : "1",
"incidents" : [
{"id": 10, "ref": "AAA", "code": "123", date: "2010-11-15T00:01:00Z"},
{"id": 20, "ref": "BBB", "code": "456", date: "2020-11-15T00:01:00Z"},
]
}
Tried a bunch of permutations but getting nowhere!


You can use this spec:
[
{
"operation": "shift",
"spec": {
"*": "id",
"data": {
"types": {
"*": {
"incident_*": "incidents[&1].&(0,1)"
}
}
}
}
}
]
To prevent using incident text in the spec, you can use the below spec:
[
{
"operation": "shift",
"spec": {
"*": "id",
"data": {
"types": {
"*": {
"*_*": "&(0,1)[&1].&(0,2)"
}
}
}
}
}
]


You can use two level of shift transformations by extracting substring from tag names such as
[
{
"operation": "shift",
"spec": {
"*": "id",
"data": {
"types": {
"*": "incidents"
}
}
}
},
{
"operation": "shift",
"spec": {
"id": "&",
"*": {
"*": {
"*_*": "&2[&1].&(0,2)"
}
}
}
}
]

Related

JOLT apply different transformations based on success of the call

I'm calling a rest web service and I receive the following responses depending if the call was successful or not.
Input 1:
{
"success": true,
"data": {
"series": "XX/32/V32/LM",
"number": 242,
"end_date": "31/12/2023",
"premium": "2309.68",
"premium_net": "2286.58",
"total_premium": 2494.46,
"commission": 1,
"installments": [
{
"number": 1,
"due_date": "30/12/2022",
"value": "2494.46",
"currency": "RON"
}
],
"reference_premium": 1061,
"direct_settlement_cover": 1,
"direct_settlement": 184.78,
"direct_settlement_net": 182.93,
"bm": "B0",
"exclusion_countries": [
"BY",
"IL",
"IR",
"MA",
"RUS",
"TN",
"UA"
]
},
"message": "Polița a fost emisă cu succes"
}
Input 2:
{
"success": false,
"message": "Eroare validare date",
"data": {
"pay_document": [
"Trebuie să menționați modalitatea de plată"
]
}
}
And I wrote the following operations that work individually
Success spec:
[
{
"operation": "shift",
"spec": {
"message": "body.info_message",
"data": {
"#1": "body.good_id",
"series": "body.policy_series",
"number": "body.policy_number",
"end_date": "body.policy_end_date",
"direct_settlement_cover": {
"1": {
"#Decontare directa": "clauses[0].clause_name",
"#1": "clauses[0].clause_id"
}
},
"direct_settlement": "clauses[0].premium",
"installments": {
"*": {
"number": "body.installments[&1].number",
"value": "body.installments[&1].amount",
"currency": "body.installments[&1].currency",
"due_date": "body.installments[&1].due_date"
}
}
}
}
},
{
"operation": "modify-overwrite-beta",
"spec": {
"body": {
"good_id": "=toInteger"
}
}
}
]
Error spec:
[
{
"operation": "shift",
"spec": {
"message": "body.info_message",
"data": {
"*": {
"*": "body.error_message"
}
}
}
},
{
"operation": "modify-overwrite-beta",
"spec": {
"*": {
"error_message": "=join(' | ',#(1,&))"
}
}
}
]
Now I'm trying to combine those two and display the error messages if success is false or return the model to be saved. I tried to place success inside my transformation but it keeps returning null. what is the best practice in this regard...

You can use this spec:
I'm just adding the first shift operator and changing the data key with true and false in your code. So the final code is like the below:
[
{
"operation": "shift",
"spec": {
"*": "&",
"data": "#(1,success)"
}
},
{
"operation": "shift",
"spec": {
"message": "body.info_&",
"true": {
"#1": "body.good_id",
"series": "body.policy_&",
"number": "body.policy_&",
"end_date": "body.policy_&",
"direct_settlement_cover": {
"1": {
"#Decontare directa": "clauses[0].clause_name",
"#1": "clauses[0].clause_id"
}
},
"direct_settlement": "clauses[0].premium",
"installments": {
"*": {
"*": "body.&2[&1].&",
"value": "body.&2[&1].amount"
}
}
},
"false": {
"*": {
"*": "body.error_message"
}
}
}
},
{
"operation": "modify-overwrite-beta",
"spec": {
"*": {
"good_id": "=toInteger",
"error_message": "=join(' | ',#(1,&))"
}
}
}
]

JOLT shift through properties with different names

I have a JSON:
{
"relations": {
"advertiser_id": {
"9968": {
"name": "Advance/Unicredit",
"id": 9968
},
"10103": {
"name": "Advance/ ORIMI",
"id": 10103
}
},
"campaign_id": {
"256292": {
"name": "Interests_Aidata",
"id": 256292,
"advertiser_id": 9968
},
"257717": {
"name": "G_14.04",
"id": 257717,
"advertiser_id": 10103
}
}
}
}
I thought that it's an easy shift operation, but I'm stuck because of all values inside random property names like "9968": I don't understand how to move through json with these different propertie names.
Expected Output:
[
{
"name": "Interests_Aidata",
"id": 256292,
"advertiser_id": 9968
},
{
"name": "G_14.04",
"id": 257717,
"advertiser_id": 10103
}
]
UPDATE
Is it possible to add top-level (under relations) advertiser_id or campaign_id as additional propety like in an example?
[
{
"name": "Interests_Aidata",
"id": 256292,
"advertiser_id": 9968,
"entity_type": "campaign_id"
},
{
"name": "G_14.04",
"id": 257717,
"advertiser_id": 10103,
"entity_type": "campaign_id"
}
]

If your aim is to accumulate each array of objects under those properties(advertiser_id and campaign_id), then use the following spec
[
{
"operation": "shift",
"spec": {
"*": {
"*": {
"*": {
"#": "&2"
}
}
}
}
}
]
If you're interested in only the stuff under campaign_id as lately edited, then use
[
{
"operation": "shift",
"spec": {
"*": {
"campaign_id": {
"*": {
"#": ""
}
}
}
}
}
]
The following spec can be used the desired result determined by the last update
[
{
"operation": "shift",
"spec": {
"*": {
"campaign_id": {
"*": {
"*": "[#2].&",
"$1": "[#2].entity_type"
}
}
}
}
}
]

Jolt unnest list to top level

I have a JSON:
[
{
"id": 1015380,
"type": "campaign",
"stats": [
{
"message_sends_by_any_user": 13,
"ctr": "0.094",
}
]
},
{
"id": 1015695,
"type": "campaign",
"stats": [
{
"message_sends_by_any_user": 7,
"ctr": "0.091",
}
]
}
]
I want to "unnested" stats list to top level. JOLT config:
[
{
"operation": "shift",
"spec": {
"*": {
"id": "[&1].id",
"type": "[&1].type",
"stats": {
"*": "[&2].&"
}
}
}
}
]
Expected:
[
{
"id": 1015380,
"type": "campaign",
"message_sends_by_any_user": 13,
"ctr": "0.094",
},
{
"id": 1015695,
"type": "campaign",
"message_sends_by_any_user": 7,
"ctr": "0.091"
}
]
But actual output contains array indexes like this:
[
{
"id" : 1015380,
"type" : "campaign",
"0" : {
"message_sends_by_any_user" : 13,
"ctr" : "0.094"
}, ...
How to avoid these indexes?

You can prefer walking through the indexes of the stats list by taking id and type attributes inside such as
[
{
"operation": "shift",
"spec": {
"*": {
"stats": {
"*": {
"#(2,id)": "[&3].id",
"#(2,type)": "[&3].type",
"*": "[&3].&"
}
}
}
}
}
]
the demo on the site http://jolt-demo.appspot.com is

Jolt transformation array data

I want to transform a JSON using JOLT like this:
Input: {
"array": [
"1","2","3","4"
],
"array2": [
{
"something": "123",
"something1": "Plane"
},
{
"something3": "567",
"something4": "Car"
}
]
}
Into the following format, as you can see from output I need data from both arrays to fit exact param names, not empty one like in the first or existed param names like in the second one.
Output: {
"one_array": [
{
"code": "1",
"description": "",
},
{
"code": "2",
"description": "",
},
{
"code": "3",
"description": "",
},
{
"code": "4",
"description": "",
}
], "other_array": [
{
"id": "123",
"type": "Plane"
},
{
"id": "567",
"type": "Car"
}
]
}
Some clarifications are really appreciated

You can achieve this using 2 shift operations and the default operation as below.
[
{
"operation": "shift",
"spec": {
"array": {
"*": {
"#": "one_array[&].id"
}
},
"array2": {
"*": {
"*": {
"#": "tmp_array[&2]"
}
}
}
}
},
{
"operation": "shift",
"spec": {
"one_array": "one_array",
"tmp_array": {
"*": {
"0": "other_array[&1].id",
"1": "other_array[&1].type"
}
}
}
},
{
"operation": "default",
"spec": {
"one_array[]": {
"*": {
"description": ""
}
}
}
}
]

JOLT transformation for JSON originally from Protobuf format

I have json converted from protobuf protocol with following format:
{
"id": "6aa0734f-6d6a-4b95-8a2b-2dde346f9df7",
"measurements": [
{
"ts": "1590331111510000000",
"values": [
{},
{
"name": 1,
"value": -1.8093990087509155
},
{
"name": 2,
"value": 0.35456427931785583
}
],
"parameters": [
"Stat",
"VoltageAnglePhaseB"
]
}
]
}
expected output is:
{
"id" : "6aa0734f-6d6a-4b95-8a2b-2dde346f9df7",
"ts" : "1590331111510000000",
"Stat":-1.8093990087509155,
"VoltageAnglePhaseB":0.35456427931785583
}
I've started my Jolt spec like this:
[
{
"operation": "shift",
"spec": {
"id": "id",
"measurements": {
"*": {
"ts": "ts",
"parameters": {
"*": ""
},
"*": ""
}
}
}
}
]
But facing the problem to extract name and value from the JSON.
Does anyone have an idea?

It can be done with 2 shift operations. First, shift the values to the valuesArr, same way shift the parameters to the parametersArr. Then match the values with the index.
[
{
"operation": "shift",
"spec": {
"id": "id",
"measurements": {
"*": {
"ts": "ts",
"values": {
"*": {
"value": "valuesArr"
}
},
"parameters": "parametersArr"
}
}
}
},
{
"operation": "shift",
"spec": {
"id": "id",
"ts": "ts",
"valuesArr": {
"*": "#(2,parametersArr[&])"
}
}
}
]