The Jolt transform should modify the input as shown, I need to push the data aggregated like that in to the DB based on the Key value as identifier.
Input Json :
[
{
"Key": "991641500~2167767723",
"itemNm": "2067875000"
},
{
"Key": "991641500~2167767724",
"itemNm": "2067875085"
},
{
"Key": "991641500~2167767723",
"itemNm": "2067875063"
},
{
"Key": "991641500~2167767724",
"itemNm": "2067875004"
}
]
Output JSON:
The output JSON should be merged as follows, I need to have the key and the contents of those as elements of a JSon Array.
[
{
"Key": "991641500~2167767723",
"Items": [
{
"itemNm": "2067875004"
},
{
"itemNm": "2067875085"
}
]
},
{
"Key": "991641500~2167767724",
"Items": [
{
"itemNm": "2067875000"
},
{
"itemNm": "2067875063"
}
]
}
]
You can use the following transformation spec :
[
{
// group attributes under common objects by their Key(#(1,Key))
"operation": "shift",
"spec": {
"*": {
"Key": "#(1,Key).&",
"itemNm": "#(1,Key).items[&1].&"
}
}
},
{
// get rid of object labels
"operation": "shift",
"spec": {
"*": ""
}
},
{
// get rid of redundant null components of the arrays
"operation": "modify-overwrite-beta",
"spec": {
"*": "=recursivelySquashNulls"
}
},
{
// pick only single one from identical components of each "Key" array
"operation": "cardinality",
"spec": {
"*": {
"Key": "ONE"
}
}
}
]
the demo on the site http://jolt-demo.appspot.com/ is :
Related
I am using JOLTTransformJson Processor in Nifi.
My input is:
[
{
"col_name": "time",
"data_type": "timestamp",
"is_nullable": true
},
{
"col_name": "otherData",
"data_type": "string",
"is_nullable": false
}
]
I am using the below spec:
[
{
"operation": "shift",
"spec": {
"*": {
"col_name": "name",
"data_type": "type[0]",
"is_nullable": {
"true": "type[1]",
"false": "type[1]"
}
}
}
},
{
"operation": "default",
"spec": {
"*": {
"type[1]": "notnull"
}
}
}
]
Expected Output is :
{
"type": "record",
"name": "table_name",
"fields": [
{
"name": "time",
"type": [
"timestamp",
"null"
]
},
{
"name": "otherData",
"type": [
"string",
"notnull"
]
}
]
}
But getting the below one as the current result by combining all values in array like:
{
"name": [
"time",
"otherData"
],
"type": [
[
"timestamp",
"int"
],
null
]
}
Can someone please help what am I missing.
You can use the following first shift transformation spec through walking by the objects of the array in order to be able to repeatedly apply the techniques :
[
{
"operation": "shift",
"spec": {
"#record": "type", // fixed values formed by using # wildcards left-hand-side
"#table_name": "name",
"*": {
"col_*": "fields[#2].&(0,1)", // replicate the 1st replacement of the asterisk from the current(0th) level
"is_nullable": { // conditional logic starts here
"#(1,data_type)": "fields[#3].type",
"true": {
"#null": "fields[#4].type"
},
"false": {
"#notnull": "fields[#4].type"
}
}
}
}
},
{ //to sort the attributes within the fields array
"operation": "sort",
"spec": {
"fields": ""
}
},
{ //to sort whole result
"operation": "shift",
"spec": {
"type": "&",
"name": "&",
"fields": "&"
}
}
]
the second and third transformation are just added to sort the attributes/arrays as desired
I need to merge array data from two different arrays from the Input JSON and make a flat list of items in the Output JSON. The first array contains the key required for the output and the second array contains the value.
So far, all my attempts at a spec haven't even come close, which is why I haven't listed one. Please see the Input and Desired Output below. Thanks for any help!!
Input JSON :
{
"data": [
{
"2": {
"value": "DC1"
},
"3": {
"value": 5
}
},
{
"2": {
"value": "DC2"
},
"3": {
"value": 10
}
}
],
"fields": [
{
"id": 2,
"label": "DataCenter",
"type": "text"
},
{
"id": 3,
"label": "CCount",
"type": "numeric"
}
],
"metadata": {
"numFields": 2,
"numRecords": 2,
"skip": 0,
"totalRecords": 2
}
}
Desired Output JSON:
[
{
"DataCenter": "DC1",
"CCount": "5"
},
{
"DataCenter": "DC2",
"CCount": "10"
}
]
You can use this spec:
[
{
"operation": "shift",
"spec": {
"data": "&",
"fields": {
"*": {
"label": "fields.#(1,id)"
}
}
}
},
{
"operation": "shift",
"spec": {
"data": {
"*": {
"*": {
"*": "[&2].#(4,fields.&1)"
}
}
}
}
}
]
In your desired output CCount value is a string, You can add the below spec to change an integer to a string.
,
{
"operation": "modify-overwrite-beta",
"spec": {
"*": {
"CCount": "=toString"
}
}
}
You might reciprocally match id value of fields array vs. the object keys of the data array within a common object within the first shift transformation spec in order to prepare for the second one in which we tile the attributes into their individual objects such as
[
{
"operation": "shift",
"spec": {
"*": { // represents the node for both "data" and "fields" arrays
"*": {
"*": {
"value": "&1.&",
"#1,label": "#2,id.&"
}
}
}
}
},
{
"operation": "shift",
"spec": {
"*": {
"value": {
"*": "[&].#2,label"
}
}
}
}
]
the demo on the site http://jolt-demo.appspot.com/ is
I am trying to convert the below JSON payload into a JSON that has the field name as the value of the field:
The jolt file that I have is working if field values are different. But if field values are the same then it is giving an array in the response.
Can you please help to provide jolt specifications for this?
Input JSON Payload:
{
"action": {
"Success": true,
"records": [
{
"Id": "Test_abc",
"SubscriptionID": "ID_1"
},
{
"Id": "Test_abc",
"SubscriptionID": "ID_2"
},
{
"Id": "Test_xyz",
"SubscriptionID": "ID_3"
}
],
"type": "update"
}
}
Expected output:
{
"action": {
"Success": true,
"records": {
"Test_abc": {
"SubscriptionID": "ID_1"
},
"Test_abc": {
"SubscriptionID": "ID_2"
},
"Test_xyz": {
"SubscriptionID": "ID_3"
}
},
"type": "update"
}
}
Solution not found yet.
You can use the following shift transformation spec
[
{
"operation": "shift",
"spec": {
"*": {
"*": "&1.&", // &1 replicates the literal "action"
"records": {
"*": {
"S*": "&3.&2.#(1,Id).&" // &3 replicates the literal "action" (by going three level up the tree), &2 for "records", & replicates the current level attribute
}
}
}
}
}
]
presumingly converting one of the Id value from Test_abc to another one such as Test_def, since the desired output is wrong as a JSON value(There cannot be more than one object with the same tag at the same level)
Your output is wrong. You can't have multiple objects with the same keys.
If You want to support all objects of records array, You should bring them into an array like this spec:
[
{
"operation": "shift",
"spec": {
"*": {
"*": "&1.&",
"records": {
"*": {
"S*ID": "&3.&2[].#(1,Id).&"
}
}
}
}
}
]
But, if you want to have an object as your records in the output, You should remove the duplicate ID like this spec:
[
{
"operation": "shift",
"spec": {
"*": {
"*": "&1.&",
"records": {
"*": {
"S*ID": "&3.&2.#(1,Id).&"
}
}
}
}
},
{
"operation": "cardinality",
"spec": {
"*": {
"records": {
"*": {
"*": "ONE"
}
}
}
}
}
]
I have this JSON:
{
"mapTrue": "true",
"code": [
"X",
"Y",
"Z"
],
"expired": [
"true",
"false",
"true"
]
}
I want to use the "mapTrue" value in order to filter the "code" array.
If "mapTrue": "true", I'll take the 0 and 2 values in "expired" array, therefore I need to output
"code": ["X", "Z"].
Using the same logic, for "mapTrue: "false" I'll return "code": ["Y"].
This specification returns the right "code" array but it doesn't use the "mapTrue" value.
[
{
"operation": "shift",
"spec": {
"expired": {
"*": {
"true": {
"#(3,code[&1])": "code"
}
}
}
}
}
]
That's where my problem is.
This works:
[
{
"operation": "shift",
"spec": {
"expired": "#(1,mapTrue)",
"code": "code"
}
},
{
"operation": "shift",
"spec": {
"*": {
"*": {
"&1": {
"#(3,code[&1])": "code[]"
}
}
}
}
}
]
An alternative option would be using the following specs
[
{
// separate the values for "true", "false" while match "code" with the value of "mapTrue"
"operation": "shift",
"spec": {
"c*": {
"*": "#(2,expired[&])", // walk through the indexes(0,1,2) of "expired" array while matching with the components of "code"
"#(1,mapTrue)": "&1"
}
}
},
{
"operation": "shift",
"spec": {
"c*": {
"*": {
"#(2,&)": "&2" // grab the value going tree up two levels by using "#(2,&)", and copy the main object's label "code" by grabbing it after going tree two levels again
}
}
}
}
]
the demo on the site http://jolt-demo.appspot.com/ is
The requirement is to flatten the hierarchical list into single list as shown in output with key's value as KEY and value's value as VALUE in final list. (sample output added).
INPUT:
{
"optionList": [{
"key": "General",
"values": [{
"key": "A",
"value": ["a"]
},
{
"key": "B",
"value": ["b"]
},
{
"key": "C",
"value": ["c"]
}]
}]
}
Required Output :
{
"A":"a",
"B":"b",
"C":"c"
}
Below transformation should work for you:
[
{
"operation": "shift",
"spec": {
"optionList": {
"*": {
"values": {
"*": {
"value": "#(1,key)"
}
}
}
}
}
}, {
"operation": "cardinality",
"spec": {
"*": {
"#": "ONE"
}
}
}
]
See also:
Transpose data in an Array.
CardinalityTransform