I need a help in Jolt transform.
I have array, that consist from many json objects. (https://jolt-demo.appspot.com/ )
[
{
"table_name": "table_vd",
"scratch_name": "l1_sample_1_1",
"scratch_order": 1
},
{
"table_name": "table_vd",
"scratch_name": "l1_sample_1_1",
"scratch_order": 34
},
{
"table_name": "table_vd",
"scratch_name": "l1_sample_2",
"scratch_order": 3
}
]
What transformation of Jolt needs to be done to get the following result?
{
"table_name" : "table_vd",
"data" : [ {
"scratch_name" : "l1_sample_1_1",
"scratch_order" : 1
}, {
"scratch_name" : "l1_sample_1_1",
"scratch_order" : 34
}, {
"scratch_name" : "l1_sample_2",
"scratch_order" : 3
} ]
}
Now i have next Jolt construction:
[
{
"operation": "shift",
"spec": {
"*": {
"table_name": "table_name",
"*": {
"#0": "data[0].&1"
}
}
}
}
]
But the result below is not correct:
{
"table_name" : [ "table_vd", "table_vd", "table_vd" ],
"data" : [ {
"scratch_name" : [ "l1_sample_1_1", "l1_sample_1_1", "l1_sample_2" ],
"scratch_order" : [ 1, 34, 3 ]
} ]
}
You started well by using conditional logic through seperating "table_name" and others("*"). Rearrange "*"(the rest) as "*": "data[&1].&", and then pick the first element of the values of "table_name" array through use of cardinality spec such as
[
{
"operation": "shift",
"spec": {
"*": {
"table_name": "&",
"*": "data[&1].&"
}
}
},
{
"operation": "cardinality",
"spec": {
"table_name": "ONE"
}
}
]
Related
I have an array of JSONs as listed below:
[
{
"id": 1
},
{
"id": 2
},
{
"id": 3
}
]
The following are the objectives:
(1) Modify the list above into:
{
"data": [
{
"id": 1
},
{
"id": 2
},
{
"id": 3
}
],
"idList": [ 1, 2, 3 ]
}
(2) Calculate the Minimum and Maximum of "idList" to finally obtain:
{
"data": [
{
"id": 1
},
{
"id": 2
},
{
"id": 3
}
],
"minID": 1,
"maxID": 3,
}
I think (2) is straightforward after getting (1), as I can simply use:
min(#(1,idList))
I have a problem in converting the original input into (1), here's my attempt:
[
{
"operation": "shift",
"spec": {
"*": "data"
}
},
{
"operation": "shift",
"spec": {
"data": { "*": { "id": "idList" } }
}
}
]
which yields:
{
"idList" : [ 1, 2, 5 ]
}
Can anyone help on this?
Also, am a newbie to this Jolt Transform technique, can anyone suggest a good source for mastering this ? (like a book)
You can consecutively use a shift, and modify transformation specs such as
[
{
// for the first objective
"operation": "shift",
"spec": {
"*": {
"#": "data",
"*": "idList"
}
}
},
{
// for the second one
"operation": "modify-overwrite-beta",
"spec": {
"minID": "=min(#(1,idList))",
"maxID": "=max(#(1,idList))"
}
}
]
the demo on the site http://jolt-demo.appspot.com/ is
The Jolt transform should modify the input as shown, I need to push the data aggregated like that in to the DB based on the Key value as identifier.
Input Json :
[
{
"Key": "991641500~2167767723",
"itemNm": "2067875000"
},
{
"Key": "991641500~2167767724",
"itemNm": "2067875085"
},
{
"Key": "991641500~2167767723",
"itemNm": "2067875063"
},
{
"Key": "991641500~2167767724",
"itemNm": "2067875004"
}
]
Output JSON:
The output JSON should be merged as follows, I need to have the key and the contents of those as elements of a JSon Array.
[
{
"Key": "991641500~2167767723",
"Items": [
{
"itemNm": "2067875004"
},
{
"itemNm": "2067875085"
}
]
},
{
"Key": "991641500~2167767724",
"Items": [
{
"itemNm": "2067875000"
},
{
"itemNm": "2067875063"
}
]
}
]
You can use the following transformation spec :
[
{
// group attributes under common objects by their Key(#(1,Key))
"operation": "shift",
"spec": {
"*": {
"Key": "#(1,Key).&",
"itemNm": "#(1,Key).items[&1].&"
}
}
},
{
// get rid of object labels
"operation": "shift",
"spec": {
"*": ""
}
},
{
// get rid of redundant null components of the arrays
"operation": "modify-overwrite-beta",
"spec": {
"*": "=recursivelySquashNulls"
}
},
{
// pick only single one from identical components of each "Key" array
"operation": "cardinality",
"spec": {
"*": {
"Key": "ONE"
}
}
}
]
the demo on the site http://jolt-demo.appspot.com/ is :
I am trying to convert below JSON into name value pair :
{
"Size": "2",
"done": "true",
"records": [
{
"Id": "a7g6s0000004GZuAAM",
"NN": "00096411.0",
"Name": "ISOLIN TRADE & INVEST",
"RecordType": {
"attributes": {
"type": "TestType"
},
"Name": "Term"
}
},
{
"Id": "a7g6s0000004GZzAAM",
"Number": "00096412.0",
"Name": "ISOLIN"
}
]
}
Expecting output JSON :
{
"Size" : "2",
"done" : "true",
"Items" : [ {
"Fields" : [ {
"Name" : "Id",
"Value" : "a7g6s0000004GZuAAM"
}, {
"Name" : "NN",
"Value" : "00096411.0"
}, {
"Name" : "Name",
"Value" : "ISOLIN TRADE & INVEST"
}, {
"Name" : "RecordType_Name",
"Value" : "Term"
} ]
}, {
"Fields" : [ {
"Name" : "Id",
"Value" : "a7g6s0000004GZzAAM"
}, {
"Name" : "Number",
"Value" : "00096412.0"
}, {
"Name" : "Name",
"Value" : "ISOLIN"
} ]
} ]
}
I am using below jolt spec but transformation of RecordType element is not as expected :
Jolt Spec :
[
{
"operation": "remove",
"spec": {
"records": {
"*": {
"attributes": " "
}
}
}
},
{
"operation": "shift",
"spec": {
"*": "&",
"records": {
"*": {
"*": {
"$": "Items.&2.[#2].Name",
"#": "Items.&2.[#2].Value"
}
}
}
}
},
{
"operation": "shift",
"spec": {
"*": "&",
"Items": {
"*": {
"*": "Items.[#2].Fields[]"
}
}
}
}
]
How can I transform this into required format ?
I think remove transformation is not needed, and better to index each Field.
The trick for the solution would be distinguish RecordType attribute from the others by use of
"RecordType": {"Name": "Field&2.&1\\_&"}
such as
[
{
"operation": "shift",
"spec": {
"*": "&",
"records": {
"*": {
"*": "Field&1.&",
"RecordType": {
"Name": "Field&2.&1\\_&"
}
}
}
}
},
{
"operation": "shift",
"spec": {
"*": "&",
"Field*": {
"*": {
"$": "Items[0].&2.[#2].Name",
"#": "Items[0].&2.[#2].Value"
}
}
}
}
]
I'm so close with this but afraid I do need a hand with a jolt transformation. I've done most of the work but can't get the last minute transformation working.
Here is my data:
{
"totalElements": 168,
"columns": {
"dimension": {
"id": "variables/daterangehour",
"type": "time"
},
"columnIds": [
"1"
]
},
"rows": [
{
"itemId": "119050300",
"value": "00:00 2019-06-03",
"data": [
120
]
},
{
"itemId": "119050805",
"value": "05:00 2019-06-08",
"data": [
98
]
},
{
"itemId": "119050923",
"value": "23:00 2019-06-09",
"data": [
172
]
}
]
}
}
This is my Jolt:
[{
"operation": "shift",
"spec": {
"rows": {
"*": {
"value": "[&1].date",
"data": "[&1].data"
}
}
}
}
]
It gives me this result:
[ {
"date" : "00:00 2019-06-03",
"data" : [ 120 ]
}, {
"date" : "22:00 2019-06-09",
"data" : [ 307 ]
}, {
"date" : "23:00 2019-06-09",
"data" : [ 172 ]
} ]
This causes my system issues, I actual need the data field like this:
[ {
"date" : "00:00 2019-06-03",
"data" : "120"
}, {
"date" : "05:00 2019-06-08",
"data" : "98"
} ]
How do I pluck the item out of the array / square brackets? It will only ever be one item in there.
You should go deeper and take value. Is it clear for you?
[
{
"operation": "shift",
"spec": {
"rows": {
"*": {
"value": "[&1].date",
"data": {
"*": "[&2].data"
}
}
}
}
}
]
And if you need Strings add this:
{
"operation": "modify-overwrite-beta",
"spec": {
"*": {
"data": "=toString"
}
}
}
I need to perform a Jolt transformation on the below example json:
Input
[
{
"DOCUMENT_HEADER_ID": "29120",
"Descrizione": "GAS",
"PrezzoTotale": "8.51"
},
{
"DOCUMENT_HEADER_ID": "29120",
"Descrizione": "IMPOSTE",
"PrezzoTotale": "7.25"
},
...
]
I need to trasform to
{
"DOCUMENT_HEADER_ID" : "29120",
"invoice" : [
{
"DOCUMENT_HEADER_ID" : "29120",
"Descrizione" : "GAS",
"PrezzoTotale" : "8.51"
}, {
"DOCUMENT_HEADER_ID" : "29120",
"Descrizione" : "IMPOSTE",
"PrezzoTotale" : "7.25"
},
...
]
}
Using this spec
[
{
"operation": "shift",
"spec": {
"#": "invoice"
}
}
]
I have this output:
{
"invoice" : [ {
"DOCUMENT_HEADER_ID" : "29120",
"Descrizione" : "GAS",
"PrezzoTotale" : "8.51"
}, {
"DOCUMENT_HEADER_ID" : "29120",
"Descrizione" : "IMPOSTE",
"PrezzoTotale" : "7.25"
},
...
]
}
but I can not figure out how to extract the value of DOCUMENT_HEADER_ID from the first element of original array
Can do it in a single shift.
[
{
"operation": "shift",
"spec": {
"#": "invoice",
"0": {
"DOCUMENT_HEADER_ID": "DOCUMENT_HEADER_ID"
}
}
}
]
I found the correct spec
[
{
"operation": "shift",
"spec": {
"#": "invoice"
}
},
{
"operation": "shift",
"spec": {
"invoice": {
"0": {
"DOCUMENT_HEADER_ID": "DOCUMENT_HEADER_ID"
},
"#": "invoice"
}
}
}
]
If having to retrieve single element, data like :
"array_field": ["array_single_element"]
can use:
{
"operation": "modify-overwrite-beta",
"spec": {
"array_field": "=concat(#(1,array_field))",
"*": "=concat(#(1,&))"
}
}