How do I define the following so that it removes all 'familyName'
{
"operation": "modify-overwrite-beta",
"spec": {
"*": "=removeRecursive('familyName')"
}
}
Hi You can use remove operation to remove instead of modify-overwrite-beta ?
{
"operation": "remove",
"spec": {
"familyName": ""
}
}
Related
I have this collection of driver details and I want to use jolt to copy the properties to concatenate first name and last name within a single property called name
{
"drivers_details": [
{
"first_name": "Doru",
"last_name": "Petre",
"cnp": "1641201390687",
"id_series": "RK",
"id_number": "123456"
},
{
"first_name": "GIGI",
"last_name": "FANE",
"cnp": "16412013906871",
"id_series": "RK",
"id_number": "1234567"
}
]
}
I am using this spec for the shift transformation spec
[
{
"operation": "shift",
"spec": {
"drivers_details": {
"*": {
"cnp": "body.drivers[&1].tin",
"id_series": "body.drivers[&1].id_series",
"id_number": "body.drivers[&1].id_number"
}
}
}
}
]
and this one for modify-overwrite, but it doesn't work how I was expecting
{
"operation": "modify-overwrite-beta",
"spec": {
"drivers_details": {
"*": {
"name": "=concat(#(2,first_name),' ',#(2,last_name))"
}
}
}
}
What is the correct way of doing this?
You can use modify transformation as in the following case, and get rid of names other than newly formed attribute name such as
[
{
"operation": "modify-overwrite-beta",
"spec": {
"drivers_details": {
"*": {
"name": "=concat(#(1,first_name),' ',#(1,last_name))"
}
}
}
},
{
"operation": "remove",
"spec": {
"drivers_details": {
"*": {
"*name": ""
}
}
}
},
{ // this last spec stands only for ordering of the attributes if the ordering and putting the "name" at the top matters.
// So, you might get rid of the "remove" transformation above if this is used. Otherwise keep it.
"operation": "shift",
"spec": {
"drivers_details": {
"*": {
"name": "&2[&1].&",
"cnp": "&2[&1].&",
"id*": "&2[&1].&"
}
}
}
}
]
where those attributes(first/last_name) stays right-hand-side at the current level, so just only need to traverse 1 colon(:) which appears as #(1,..)
You can use this spec without the remove operation:
[
{
"operation": "modify-overwrite-beta",
"spec": {
"*": {
"*": {
"name": "=concat(#(1,first_&),' ',#(1,last_&))"
}
}
}
},
{
"operation": "shift",
"spec": {
"*": {
"*": {
"name": "&2[&1].&",
"cnp": "&2[&1].&",
"id_*": "&2[&1].&"
}
}
}
}
]
i have JSON value as below :
{
"table": "table_name",
"op_type": "U",
"before": {
"AAAA": "1-1111",
"BBBB": "2022-08-31 03:57:01"
},
"after": {
"AAAA": "1-1111",
"BBBB": "2022-08-31 10:10:34",
"DDDD": "2023-08-31 23:59:59"
}
}
I want to add column_names field like this :
,"changed_columns": "AAAA,BBBB,DDDD"
is there a way to do this?
You can use the following specs in which the main idea is to arrange the attributes so as to generate an array with unique elements within the an array by using successive shift transformation, then combine them within a modify transformation such as
[
{
// combine common key names for each respective values for the attributes
"operation": "shift",
"spec": {
"before|after": {
"*": {
"$": "&"
}
}
}
},
{
// construct an array from those newly formed keys
"operation": "shift",
"spec": {
"*": {
"$": "changed_columns"
}
}
},
{
// make them comma-separated
"operation": "modify-overwrite-beta",
"spec": {
"*": "=join(',',#(1,&))"
}
}
]
the demo on the site http://jolt-demo.appspot.com/ is
Edit : If your aim is to keep newly generated attribute along with the existing ones, then you can prefer using the following spec
[
{
"operation": "shift",
"spec": {
"*": "&", //else case
"before|after": {
"*": {
"$": "cc.&",
"#": "&2.&"
}
}
}
},
{
"operation": "shift",
"spec": {
"cc": {
"*": {
"$": "changed_columns"
}
},
"*": "&" //else case
}
},
{
"operation": "modify-overwrite-beta",
"spec": {
"changed_columns": "=join(',',#(1,&))"
}
}
]
I have following json input . The Input contain many special character I want to remove all the special character form the input data
Expected Input
{
"A": "pwnbfd%2hdj&mdnb",
"B": "my name is param (India) ",
"C": "#pqwe",
"D": "jfdk#djnsn(america) djhfb "
}
Expected Output
{
"A": "pwnbfd2hdjmdnb",
"B": "my name is param India",
"C": "pqwe",
"D": "jfdkdjnsnamerica djhfb"
}
I need the above changes by using jolt transform json processor in apache nifi. There could be many other keywords in json payload.
I need to remove all the special characters from the input json so please help!!!!!
I don't know(furthermore don't think) whether there's a straightforward method to remove all of them, but you can individually remove each desired character within a modify transformation by using successive split and join functions like this :
[
{
"operation": "modify-overwrite-beta",
"spec": {
"*": "=split('%',#(1,&))"
}
},
{
"operation": "modify-overwrite-beta",
"spec": {
"*": "=join('',#(1,&))"
}
},
{
"operation": "modify-overwrite-beta",
"spec": {
"*": "=split('&',#(1,&))"
}
},
{
"operation": "modify-overwrite-beta",
"spec": {
"*": "=join('',#(1,&))"
}
},
{
"operation": "modify-overwrite-beta",
"spec": {
"*": "=split('#',#(1,&))"
}
},
{
"operation": "modify-overwrite-beta",
"spec": {
"*": "=join('',#(1,&))"
}
},
{
"operation": "modify-overwrite-beta",
"spec": {
"*": "=split('\\)',#(1,&))"
}
},
{
"operation": "modify-overwrite-beta",
"spec": {
"*": "=join('',#(1,&))"
}
},
{
"operation": "modify-overwrite-beta",
"spec": {
"*": "=split('\\(',#(1,&))"
}
},
{
"operation": "modify-overwrite-beta",
"spec": {
"*": "=join('',#(1,&))"
}
}
]
I try to truncate strings in an array with JOLT. The character "^" should be removed from any string that contains it in the first position.
Sample Input:
{
"scores": [
"^aaaa",
"^bbbb",
"cccc",
"^dddd",
"eeee"
]
}
Expected Output:
{
"scores" : [ "aaaa", "bbbb", "cccc", "dddd", "eeee" ]
}
My Spec:
[
{
"operation": "modify-overwrite-beta",
"spec": {
"splittedScores": "=split('\\^',#(1,scores))",
"scores": "=join('',#(1,splittedScores))" // <--- Not working
}
}
]
Seems you'd need one extra shift transformation sandwiched between modify transformations to tame the subarrays such as
[
{
"operation": "modify-overwrite-beta",
"spec": {
"*": "=split('\\^',#(1,&))"
}
},
{
"operation": "shift",
"spec": {
"*": {
"*": "&"
}
}
},
{
"operation": "modify-overwrite-beta",
"spec": {
"*": "=join('',#(1,&))"
}
},
{
"operation": "shift",
"spec": {
"*": {
"#": "scores"
}
}
}
]
the demo on the site http://jolt-demo.appspot.com/ is
or only one shift transformation will be sufficient to use(a straightforward option) such as
[
{
"operation": "shift",
"spec": {
"*": {
"*": {
"^*": {
"$(0,1)": "&3"
},
"*": {
"$": "&3"
}
}
}
}
}
]
splitting by "$(0,1)" wildcard for the object under "^*" node. &3 represents going three levels up to grab the name of the key scores .
the demo on the site http://jolt-demo.appspot.com/ is
If ^ is being removed in the first position really matters, then the second method; if all ^ characters from any strings should be removed, then the first method should be used.
In Nifi, I have the following JSON:
{
"list1":[1,2,3],
"list2":[4,5]
}
I want the output to be:
{
"Final_List":[1,2,3,4,5]
}
How should I do this?
You can add a JoltTransformJSON processor with a shift type transformation such as
[
{
"operation": "shift",
"spec": {
"*": {
"*": {
"#": "Final_List"
}
}
}
}
]
Edit for special case :
If it was the case that those arrays have some common integer elements, then the above solution would yield repeating elements within the Final_List, but the following make that result having unique elements for this case too :
[
{
"operation": "shift",
"spec": {
"*": {
"*": {
"*": "Final_List.&"
}
}
}
},
{
"operation": "shift",
"spec": {
"Final_List": {
"*": {
"$": "&2"
}
}
}
},
{
"operation": "modify-overwrite-beta",
"spec": {
"*": "=toInteger(#(1,Final_List))"
}
}
]