How to transform all the keys in this list of jsons to lowercase
[
{
"KEY01": "1111",
"KEY02": "VALUE01",
"KEY03": true
},
{
"KEY01": "2222",
"KEY02": "VALUE02",
"KEY03": true
}
]
desired result
[
{
"key01": "1111",
"key02": "VALUE01",
"key03": true
},
{
"key01": "2222",
"key02": "VALUE02",
"key03": true
}
]
I used this jolt transformation formula and it did only generate an object with values only being transformed to lowercase
[
{
"operation": "modify-overwrite-beta",
"spec": {
"*": {
"*": "=toLower"
}
}
}
]
Result
[{
"KEY01": "1111",
"KEY02": "value01",
"KEY03": true
}, {
"KEY01": "2222",
"KEY02": "value02",
"KEY03": true
}]
This produces the desired result:
[
{
"operation": "shift",
"spec": {
"*": {
"*": {
"$": "[&2].&1.key",
"#": "[&2].&1.value"
}
}
}
},
{
"operation": "modify-overwrite-beta",
"spec": {
"*": {
"*": {
"key": "=toLower"
}
}
}
},
{
"operation": "shift",
"spec": {
"*": {
"*": {
"value": "[&2].#(1,key)"
}
}
}
}
]
Produces the following:
[
{
"key01" : "1111",
"key02" : "VALUE01",
"key03" : true
},
{
"key01" : "2222",
"key02" : "VALUE02",
"key03" : true
}
]
Related
I'm coming to the conclusion that Jolt is beyond me.
With this input data:-
{
"cluster_id": "1",
"data": {
"id": 1,
"types": [
{
"incident_id": 10,
"incident_ref": "AAA",
"incident_code": "123",
"incident_date": "2010-11-15T00:01:00Z"
},
{
"incident_id": 20,
"incident_ref": "BBB",
"incident_code": "456",
"incident_date": "2020-11-15T00:01:00Z"
}
]
}
}
Spec:-
[
{
"operation": "shift",
"spec": {
"cluster_id": "id",
"data": {
"types": {
"*": {
"incident_id": "incidents",
"incident_ref": "incidents"
}
}
}
}
}
]
Gives:-
{
"id" : "1",
"incidents" : [ 10, "AAA", 20, "BBB" ]
}
How would I get the result of:-
{
"id" : "1",
"incidents" : [
{"id": 10, "ref": "AAA", "code": "123", date: "2010-11-15T00:01:00Z"},
{"id": 20, "ref": "BBB", "code": "456", date: "2020-11-15T00:01:00Z"},
]
}
Tried a bunch of permutations but getting nowhere!
You can use this spec:
[
{
"operation": "shift",
"spec": {
"*": "id",
"data": {
"types": {
"*": {
"incident_*": "incidents[&1].&(0,1)"
}
}
}
}
}
]
To prevent using incident text in the spec, you can use the below spec:
[
{
"operation": "shift",
"spec": {
"*": "id",
"data": {
"types": {
"*": {
"*_*": "&(0,1)[&1].&(0,2)"
}
}
}
}
}
]
You can use two level of shift transformations by extracting substring from tag names such as
[
{
"operation": "shift",
"spec": {
"*": "id",
"data": {
"types": {
"*": "incidents"
}
}
}
},
{
"operation": "shift",
"spec": {
"id": "&",
"*": {
"*": {
"*_*": "&2[&1].&(0,2)"
}
}
}
}
]
I am trying to do a JOLT shift transformation of an inputted JSON-list
Below my input:
[
{
"number": 1001,
"description": "KA01"
},
{
"number": 1002,
"description": "KA02"
}
]
And I want to create this output:
{
"actions" : [
{
"_type": "SetFieldValue",
"fieldName": "UUID",
"value": "uuid"
},
{ "_type": "InsertRow" },
{
"_type": "SetFieldValue",
"fieldName": "number",
"value": "1001"
},
{
"_type": "SetFieldValue",
"fieldName": "description",
"value": "KA01"
},
{ "_type": "InsertRow" },
{
"_type": "SetFieldValue",
"fieldName": "number",
"value": "1002"
},
{
"_type": "SetFieldValue",
"fieldName": "description",
"value": "KA02"
},
{
"_type": "SetFieldValue",
"fieldName": "start"
}
]
}
I haven´t done much with JOLT transformation and need help in this case.
First of all you can divide the JSON value into parts by key names and values, and then add _type attributes as being default such as
[
{
"operation": "shift",
"spec": {
"*": {
"*": {
"$": "&1.[&2].fieldName",
"#": "&1.[&2].value"
}
},
"#UUID": "x[#2].fieldName",
"#uuid": "x[#2].value",
"#start": "y[#2].fieldName"
}
},
{
"operation": "shift",
"spec": {
"*": {
"*": ""
}
}
},
{
"operation": "default",
"spec": {
"*": {
"_type": "SetFieldValue"
}
}
},
{
"operation": "shift",
"spec": {
"*": {
"#": ""
},
"#InsertRow": "_type"
}
},
{
"operation": "sort",
"spec": {}
}
]
I am trying to convert below JSON into name value pair :
{
"Size": "2",
"done": "true",
"records": [
{
"Id": "a7g6s0000004GZuAAM",
"NN": "00096411.0",
"Name": "ISOLIN TRADE & INVEST",
"RecordType": {
"attributes": {
"type": "TestType"
},
"Name": "Term"
}
},
{
"Id": "a7g6s0000004GZzAAM",
"Number": "00096412.0",
"Name": "ISOLIN"
}
]
}
Expecting output JSON :
{
"Size" : "2",
"done" : "true",
"Items" : [ {
"Fields" : [ {
"Name" : "Id",
"Value" : "a7g6s0000004GZuAAM"
}, {
"Name" : "NN",
"Value" : "00096411.0"
}, {
"Name" : "Name",
"Value" : "ISOLIN TRADE & INVEST"
}, {
"Name" : "RecordType_Name",
"Value" : "Term"
} ]
}, {
"Fields" : [ {
"Name" : "Id",
"Value" : "a7g6s0000004GZzAAM"
}, {
"Name" : "Number",
"Value" : "00096412.0"
}, {
"Name" : "Name",
"Value" : "ISOLIN"
} ]
} ]
}
I am using below jolt spec but transformation of RecordType element is not as expected :
Jolt Spec :
[
{
"operation": "remove",
"spec": {
"records": {
"*": {
"attributes": " "
}
}
}
},
{
"operation": "shift",
"spec": {
"*": "&",
"records": {
"*": {
"*": {
"$": "Items.&2.[#2].Name",
"#": "Items.&2.[#2].Value"
}
}
}
}
},
{
"operation": "shift",
"spec": {
"*": "&",
"Items": {
"*": {
"*": "Items.[#2].Fields[]"
}
}
}
}
]
How can I transform this into required format ?
I think remove transformation is not needed, and better to index each Field.
The trick for the solution would be distinguish RecordType attribute from the others by use of
"RecordType": {"Name": "Field&2.&1\\_&"}
such as
[
{
"operation": "shift",
"spec": {
"*": "&",
"records": {
"*": {
"*": "Field&1.&",
"RecordType": {
"Name": "Field&2.&1\\_&"
}
}
}
}
},
{
"operation": "shift",
"spec": {
"*": "&",
"Field*": {
"*": {
"$": "Items[0].&2.[#2].Name",
"#": "Items[0].&2.[#2].Value"
}
}
}
}
]
Trying to make a Jolt script that will put in a single number line, then a array of strings in one single array and a tag on the end of that array. These are the values that I have been working with.
JSON INPUT
[
{
"foo": "111",
"bar": "222",
"sun": "333",
"ListofStrings": [
"Dog",
"Train"
],
"ID": "BLAH"
},
{
"foo": "999",
"bar": "222",
"sun": "777",
"ListofStrings": [
"CAT",
"PLANE"
],
"ID": "HAHA"
}
]
JOLT SPEC This is what I have been working with that prints out the ListofStrings but this is the one that works stably.
[
{
"operation": "shift",
"spec": {
"*": {
"foo": "input[].number",
"bar": "input[].number",
"sun": "input[].number",
"ListofStrings": "input[].List"
}
}
},
{
"operation": "default",
"spec": {
"app_id": "test",
"input": {
"*": {
"app_id": "test"
}
}
}
}
]
CURRENT OUTPUT
{
"input" : [ {
"number" : "111"
}, {
"number" : "222"
}, {
"number" : "333"
}, {
"List" : [ "Dog", "Train" ]
}, {
"number" : "999"
}, {
"number" : "222"
}, {
"number" : "777"
}, {
"List" : [ "Cat", "Car" ]
} ],
"app_id" : "test"
}
DESIRED OUTPUT
{
"input" : [ {
"number" : "111"
"List" : [ "Dog", "Train" ]
"ID": "BLAH_foo"
}, {
"number" : "222"
"List" : [ "Dog", "Train" ]
"ID": "BLAH_bar"
}, {
"number" : "333"
"List" : [ "Dog", "Train" ]
"ID": "BLAH_sun"
}, {
"number" : "999"
"List" : [ "Cat", "Car" ]
"ID": "HAHA_foo"
}, {
"number" : "222"
"List" : [ "Cat", "Car" ]
"ID": "HAHA_bar"
}, {
"number" : "777"
"List" : [ "Cat", "Car" ]
"ID": "HAHA_sun"
} ],
"app_id" : "test"
}
Check this spec
[
//Converting list to Map
{
"operation": "shift",
"spec": {
"*": {
"ListofStrings": null,
"*": {
"#": "#1.number",
"#(1,ListofStrings)": "#1.list"
}
}
}
},
//Shift the number and list to the input array
{
"operation": "shift",
"spec": {
"*": {
"$": "input[#2].number",
"#(0,list)": "input[#2].List"
}
}
}, {
"operation": "default",
"spec": {
"app_id": "test",
"input": {
"*": {
"app_id": "test"
}
}
}
}
]
Edit 1
Add the ID node to the map using first shift operation "ID": null, and "#(1,ID)": "#1.ID". Then shift the ID node to the input array in the second shift operation "#(0,ID)": "input[#2].ID".
[
//Converting list to Map
{
"operation": "shift",
"spec": {
"*": {
"ListofStrings": null,
"ID": null,
"*": {
"#": "#1.number",
"#(1,ListofStrings)": "#1.list",
"#(1,ID)": "#1.ID"
}
}
}
},
//Shift the number and list to the input array
{
"operation": "shift",
"spec": {
"*": {
"$": "input[#2].number",
"#(0,list)": "input[#2].List",
"#(0,ID)": "input[#2].ID"
}
}
}, {
"operation": "default",
"spec": {
"app_id": "test",
"input": {
"*": {
"app_id": "test"
}
}
}
}
]
I have json converted from protobuf protocol with following format:
{
"id": "6aa0734f-6d6a-4b95-8a2b-2dde346f9df7",
"measurements": [
{
"ts": "1590331111510000000",
"values": [
{},
{
"name": 1,
"value": -1.8093990087509155
},
{
"name": 2,
"value": 0.35456427931785583
}
],
"parameters": [
"Stat",
"VoltageAnglePhaseB"
]
}
]
}
expected output is:
{
"id" : "6aa0734f-6d6a-4b95-8a2b-2dde346f9df7",
"ts" : "1590331111510000000",
"Stat":-1.8093990087509155,
"VoltageAnglePhaseB":0.35456427931785583
}
I've started my Jolt spec like this:
[
{
"operation": "shift",
"spec": {
"id": "id",
"measurements": {
"*": {
"ts": "ts",
"parameters": {
"*": ""
},
"*": ""
}
}
}
}
]
But facing the problem to extract name and value from the JSON.
Does anyone have an idea?
It can be done with 2 shift operations. First, shift the values to the valuesArr, same way shift the parameters to the parametersArr. Then match the values with the index.
[
{
"operation": "shift",
"spec": {
"id": "id",
"measurements": {
"*": {
"ts": "ts",
"values": {
"*": {
"value": "valuesArr"
}
},
"parameters": "parametersArr"
}
}
}
},
{
"operation": "shift",
"spec": {
"id": "id",
"ts": "ts",
"valuesArr": {
"*": "#(2,parametersArr[&])"
}
}
}
]