I need to extract values from primary_keys array and join them into one string. But I'm having troubles with values extraction.
Simplified input json:
{
"primary_keys": [
"ITEM",
"LOC",
"COMP_ID"
],
"ITEM": "ID158",
"LOC": 41,
"COMP_ID": "BPF",
"VALUE": 0.78
}
Expected output:
{
"PK": "ID158|41|BPF",
"ITEM": "ID158",
"LOC": 41,
"COMP_ID": "BPF",
"VALUE": 0.78
}
Content of primary_keys array may differ from one flowfile to another. I appreciate any input. thanks!
You can use modify-overwrite-beta transformation along with join function after deriving an array(PK) composed of the respective values for the each members of primary_keys array through use of shift transformations such as
[
{
"operation": "shift",
"spec": {
"*": "&",
"primary_keys": {
"*": {
"*": { "$": "PK.#(4,&)" }
}
}
}
},
{
"operation": "shift",
"spec": {
"*": "&",
"PK": {
"*": {
"$": "&2"
}
}
}
},
{
"operation": "modify-overwrite-beta",
"spec": {
"PK": "=join('|',#(1,&))"
}
}
]
Related
I need to merge array data from two different arrays from the Input JSON and make a flat list of items in the Output JSON. The first array contains the key required for the output and the second array contains the value.
So far, all my attempts at a spec haven't even come close, which is why I haven't listed one. Please see the Input and Desired Output below. Thanks for any help!!
Input JSON :
{
"data": [
{
"2": {
"value": "DC1"
},
"3": {
"value": 5
}
},
{
"2": {
"value": "DC2"
},
"3": {
"value": 10
}
}
],
"fields": [
{
"id": 2,
"label": "DataCenter",
"type": "text"
},
{
"id": 3,
"label": "CCount",
"type": "numeric"
}
],
"metadata": {
"numFields": 2,
"numRecords": 2,
"skip": 0,
"totalRecords": 2
}
}
Desired Output JSON:
[
{
"DataCenter": "DC1",
"CCount": "5"
},
{
"DataCenter": "DC2",
"CCount": "10"
}
]
You can use this spec:
[
{
"operation": "shift",
"spec": {
"data": "&",
"fields": {
"*": {
"label": "fields.#(1,id)"
}
}
}
},
{
"operation": "shift",
"spec": {
"data": {
"*": {
"*": {
"*": "[&2].#(4,fields.&1)"
}
}
}
}
}
]
In your desired output CCount value is a string, You can add the below spec to change an integer to a string.
,
{
"operation": "modify-overwrite-beta",
"spec": {
"*": {
"CCount": "=toString"
}
}
}
You might reciprocally match id value of fields array vs. the object keys of the data array within a common object within the first shift transformation spec in order to prepare for the second one in which we tile the attributes into their individual objects such as
[
{
"operation": "shift",
"spec": {
"*": { // represents the node for both "data" and "fields" arrays
"*": {
"*": {
"value": "&1.&",
"#1,label": "#2,id.&"
}
}
}
}
},
{
"operation": "shift",
"spec": {
"*": {
"value": {
"*": "[&].#2,label"
}
}
}
}
]
the demo on the site http://jolt-demo.appspot.com/ is
pls help me out wit this jolt specification. pls help me
Notes :
Resourcename is the last element of ResourceId which will be a new attribute that we need to add to the expected output
Tags field needs to be copied and splited as mentioned in the expected output.
Input :
[
{
"ResourceId": "/subscriptions/bb842437aa4/resourceGroups/ECHLABHENKEL/providers/Microsoft.Compute/virtualMachines/pmoapps",
"Tags": "Name\": \"PMOapplication\",\"Owner\": \"Breil sathish"
},
{
"ResourceId": "/subscriptions/bb842437aa4/resourceGroups/HCLTECHLABHENKEL/providers/Microsoft.Compute/virtualMachines/pmoapps",
"Tags": "Name\": \"PMOapplication\",\"Owner\": \"Breil sathish1"
}
]
Expected Output :
[
{
"ResourceId": "/subscriptions/bb842437aa4/resourceGroups/ECHLABHENKEL/providers/Microsoft.Compute/virtualMachines/pmoapps",
"Tags": "Name\": \"PMOapplication\",\"Owner\": \"Breil sathish",
"Resourcename": "pmoapps",
"Name": "PMOapplication",
"Owner": "Breil sathish"
},
{
"ResourceId": "/subscriptions/bb842437aa4/resourceGroups/HCLTECHLABHENKEL/providers/Microsoft.Compute/virtualMachines/pmoapps",
"Tags": "Name\": \"PMOapplication\",\"Owner\": \"Breil sathish1",
"Resourcename": "pmoapps",
"Name": "PMOapplication",
"Owner": "Breil sathish1"
}
]
Thanks,
N Sathish
You can use the following transformation spec as reading the explanations stated at the beginning of each of them such as
[
{ // Convert the value of the attributes to individual arrays with the identical names
"operation": "modify-overwrite-beta",
"spec": {
"*": {
"ResourceName": "=split('/', #(1,ResourceId))",
"tag": "=split(',', #(1,Tags))"
}
}
},
{ // Generate tag0 and tag1 attributes by splitting members of the "tag" array
"operation": "shift",
"spec": {
"*": {
"*": "&1.&",
"tag": {
"*": {
"#": "&3.&2&1"
}
}
}
}
},
{ // Split related strings by colon characters for "tag" array while deriving the last element of "ResourceName" array
"operation": "modify-overwrite-beta",
"spec": {
"*": {
"R*": "=lastElement(#(1,&))",
"tag*": "=split(': ', #(1,&))"
}
}
},
{ // Match components of "tag" array component 1 against component 2
"operation": "shift",
"spec": {
"*": {
"*": "&1.&",
"tag*": {
"#1,&[1]": "&2.&1.#(2,&[0])"
}
}
}
},
{ // Split the values by \" character combination
"operation": "modify-overwrite-beta",
"spec": {
"*": {
"tag*": {
"*": "=split('\"', #(1,&))"
}
}
}
},
{ // Prune undesired values for right-hand-side
"operation": "modify-overwrite-beta",
"spec": {
"*": {
"tag*": {
"*": "=join('', #(1,&))"
}
}
}
},
{ // Prune undesired values for left-hand-side(keys)
"operation": "shift",
"spec": {
"*": {
"*": "[&1].&",
"tag*": {
"\"*\"": "[&2].&(0,1)",
"*\"": "[&2].&(0,1)"
}
}
}
}
]
I need to flatten a db using nifi. I read in a table based in a PK ID. For each row the nifi content is shown as a MapRecord. I need to pull each field value out of the MapRecord and make it a json property.
Input:
{
"maxResults": 150,
"total": 89,
"issues": "MapRecord[{issueId=1, firstName=Jack, lastName=Smith}]",
"address": "MapRecord[{addressId=1, street=Mockingbird Lane, town=Timbuktoo}]"
}
Notice in the MapRecord nothing is in quotes. I don't know why this is like this. It is obviously not JSON.
I want the result to look like:
Expected output:
{
"maxResults": 150,
"total": 89,
"firstName": "Jack",
"lastName": "Smith",
"street": "Mockingbird Lane",
"town": "Timbuktoo"
}
Does anyone know how to do this using a JOLT transform?
Thanks.
You can manage it to solve by applying too many transformation specs. In the first step need to extract the key-value pairs. In order to perform this split function within modify-overwrite-beta might be used. At the later steps, need to keep the existing pairs(maxResults&total) and properly combine with the extracted ones using some trick methods such as
[
{
"operation": "modify-overwrite-beta",
"spec": {
"i": "=split('}',#(1,issues))",
"a": "=split('}',#(1,address))"
}
},
{
"operation": "modify-overwrite-beta",
"spec": {
"i": "=split(', ',#(1,i))",
"a": "=split(', ',#(1,a))"
}
},
{
"operation": "shift",
"spec": {
"maxResults": "&",
"total": "&",
"i|a": {
"0": { "1|2": { "#": "theRest" } }
}
}
},
{
"operation": "shift",
"spec": {
"*": "arr0.&",
"theRest": { "*": "&" }
}
},
{
"operation": "modify-overwrite-beta",
"spec": {
"*": "=split('=',#(1,&))"
}
},
{
"operation": "shift",
"spec": {
"arr0": {
"*": {
"#": "arr[1]",
"$": "arr[0]"
}
},
"*": {
"*": {
"#": "arr.[&1]"
}
}
}
},
{
"operation": "shift",
"spec": {
"*": "&",
"arr": {
"0": null,
"*": {
"*": "[&1].#(2,[0].[&])"
}
}
}
},
{
"operation": "shift",
"spec": {
"0": null,
"*": ""
}
}
]
I have a Json with this structure:
{"code":"0000",
"usercode":"sample",
"specifications":{
"c":"d","e":"f"
}}
I need to build a jolt to convert the json to this form:
{"code":"0000",
"usercode":"sample",
"specifications":[
{"key":"c",
"value":"d"},
{"key":"e",
"value":"f"}
]}
I tried this, but is my first jolt.
[
{
"operation": "shift",
"spec": {
"code": "code",
"usercode": "usercode",
"specifications": {
"*": {
"key": "#c",
"value": "#d"
}
}
}
}
]
First check this example: https://jolt-demo.appspot.com/#mapToList to understand whats going on :)
This spec will do the trick:
[
{
"operation": "shift",
"spec": {
"code": "&",
"usercode": "&",
"specifications": {
"*": {
"$": "&2[#2].key",
"#": "&2[#2].value"
}
}
}
}
]
I was trying to have a key value pair mapped to an array, distinguishing each value as a type using jolt transform spec
Input json
{
"testurl": "someurl",
"website": "someurl2"
}
tried this spec
[
{
"operation": "shift",
"spec": {
"testurl": "Urls[].testurl",
"website": "Urls[].website"
}
},
{
"operation": "shift",
"spec": {
"Urls": {
"*": {
"$": "Urls[&1].val"
}
}
}
}
]
Expected result is like this
{
"Urls": [{
"url": "someurl",
"val": "testurl"
}, {
"url": "someurl2",
"val": "website"
}]
}
Yes you can turn a set of Json key,value pair into an array.
It requires 2 shifts to be safe.
1st shift isolates all the properties you want to turn into an array.
2nd shift is able to use a "*" to then match all those items and place them an array.
Spec
[
{
"operation": "shift",
"spec": {
"testurl": "temp.testurl",
"website": "temp.website"
}
},
{
"operation": "shift",
"spec": {
"temp": {
"*": {
"$": "Urls[#2].val",
"#": "Urls[#2].url"
}
}
}
}
]