I want to change the name of fields in an array by Jolt. Please explain how it works with your answer.
Input:
{
"referenceName": "***",
"name": "***"
}
Desired output:
{
"ReferenceName": "***",
"ColumnName": "***"
}
You can use this simple spec:
[
{
"operation": "shift",
"spec": {
"referenceName": "ReferenceName",
"name": "ColumnName"
}
}
]
Or you can have simpler:
[
{
"operation": "shift",
"spec": {
"r*": "R&(0,1)",
"n*": "ColumnN&(0,1)"
}
}
]
When you want to change some names in your JSON, You can use the shift operation like the above spec:
r*: match all keys that started with the r like referenceName and * match everything after r that equals eferenceName. You can write R&(0,1) that &(0,1) value is equal to first *.
n*: match all keys that started with the n like name and * match everything after n that is equal to ame. You can write ColumnN&(0,1) that &(0,1) value is equal to first *.
Note: & means the key's value, But when you use the &(0,1), you are saying that in the 0 level or current level please get the first * value. Suppose you had a second * in this level, You should get it like this: &(0,2)
Related
Value input
"ItemInternalId" : [ "01", "02", "011055000000345" ]
Value output expected
"ItemInternalId" : [ "01", "16", "011055000000345" ]
Jolt only has two functions, "lastElement" and "firstElement", so I couldn't find anything on the internet that modified the second value
No need to use a modify transformation which has those functions, but a shift transformation spec along with index of the respective component such as
[
{
"operation": "shift",
"spec": {
"*": { //stands for "ItemInternalId"
"1": { // the level of the second component of the array in which index starts from zero
"#16": "&2" // the fixed value might be assigned by prepending the value with "#" on the left-hand-side, &2 represents going the tree two levels up to grab the tag of the array
},
"*": "&1"//else case. The &1 on the right-hand-side represents going the tree one level up to grab the tag of the array
}
}
}
]
assuming your input is
{
"ItemInternalId": [
"01",
"02",
"011055000000345"
]
}
the demo on the site http://jolt-demo.appspot.com/ is
Input json :
{
"userid" : "31b25f023c58c969388991a6b9b4030000000000",
"username_id": "54fca0dd914ae593ef65988b4a3e93cccc590000000000"
}
There is 10 0's ahead of each value. The overall length of string can vary, but it will be appended by 10 0's always. I would like to remove it.
Thus expected output:
{
"userid" : "31b25f023c58c969388991a6b9b403",
"username_id": "54fca0dd914ae593ef65988b4a3e93cccc59"
}
Hence i am looking for the right way using jolt transform to perform this truncation.
You can start with a modify transformation in which the substring function is applied for each value of the attributes after determining their ten reduced length in order to extract the value except for the last ten characters, as it's mentioned ten zeroes are always right-padded, such as
[
{
"operation": "modify-overwrite-beta",
"spec": {
"userid_len": "=size(#(1,userid))",
"userid_len_dif": "=intSum(-10,#(1,userid_len))",
"userid": "=substring(#(1,&),0,#(1,userid_len_dif))",
"username_id_len": "=size(#(1,username_id))",
"username_id_len_dif": "=intSum(-10,#(1,username_id_len))",
"username_id": "=substring(#(1,&),0,#(1,username_id_len_dif))"
}
},
{
"operation": "shift",
"spec": {
"use*id": "&"
}
}
]
the demo on the site http://jolt-demo.appspot.com/ is
Input:
{
"Remarks":"COMMENTS:(87) Test Comments"
}
Expecting Output is below:
{
"Remarks" : "C(87): TestComments"
"Id" : 87
}
I want to replace COMMENTS:(87) string with C(87): and need to get 87 in brackets and print the same in separate attribute "Id".
Can anyone help on this ?
You can use split function along with modify transformation in order to split the string by opening and closing parentheses, and then combine the desired substrings by using concat function. At the last step get rid of the auxiliary elements through use of remove transformation spec such as
[
{
"operation": "modify-overwrite-beta",
"spec": {
"str0": "=split('\\(', #(1,Remarks))",
"str1": "=split('\\)', #(1,str0))",
"str2": "#(1,str1[1])",
"Remarks": "=concat('C(',#(1,str2[0]),'):',#(1,str2[1]))",
"Id": "#(1,str2[0])"
}
},
{
"operation": "remove",
"spec": {
"str*": ""
}
}
]
this is a simple question,I am new to nifi and jolt. I just wanted to know,how to travserse the jolt spec while using wild card characters.For example,this is an example in jolt demo site,
input is
{
"data": {
"1234": {
"clientId": "12",
"hidden": true
},
"1235": {
"clientId": "35",
"hidden": false
}
}
}
Spec is
[
{
"operation": "shift",
"spec": {
"data": {
"*": {
"hidden": {
"true": {
// if hidden is true, then write the value disabled to the RHS output path
// Also #(3,clientId) means lookup the tree 3 levels, to the "1234" or "1235" level,
// and then come back down down the tree and grabe the value of "clientId"
"#disabled": "clients.#(3,clientId)"
},
"false": {
"#enabled": "clients.#(3,clientId)"
}
}
}
}
}
}
]
and output is
{
"clients" : {
"12" : "disabled",
"35" : "enabled"
}
}
How did we get the above output? like what #(3,clientsid).As far as I understand, it goes 3 levels up.But 3 levels with respect to what..the spec or the input? Either way,how to move 3 levels up,can you please define which are the levels here?
Thanks in advance
Just count the number of each opening curly-braces({) or colons(:) in the backward direction. Whenever they're not independent such as :{, then count this as only one in order to reach to the target key which is "*" wildcard just under "data" key in this case, and for #(3,clientId); first level is crossing the colon(:) next to "#disabled" or "#enabled", second level is crossing opening curly-braces next to those boolean keys for each, and then the third level is crossing opening curly-brace just after "hidden" key to reach the indexes the object with the "data" key.
I'm a newbie on Apache Nifi and have the following Problem: I would like to transform a json file as follows:
From:
{
"Property1": "x1",
"Property2": "Tag_**2ABC**",
"Property3": "x3",
"Property4": "x4"
}
to:
{
"**2ABC**_Property1": "x1",
"**2ABC**_Property3": "x3",
"**2ABC**_Property4": "x4"
},
it means: taking the value from a certain Attribute to update all other attributes.
I could find examples using JoltTransformer-Processor that works well when the update is only adding a string. But not for my case
What I've done so far: I have set each Attribute using evaluateJSONPath processor. But I just tried quite a lot of possibilities to use the update Attribute processor to do it without success. All my possible tests looked like (within UpdateAttribute):
Property1 --> ${'Property2':substring(4,6)}"_"${'Property1'}
Using Jolt:
[
{"operation": "modify-overwrite-beta",
"spec": {
"Property1": "${'Property2':substring(4,6)}_${'Property1'}"
}
}
]
Which point am I missing here? Thanks in advance!
I don't know about Nifi, but here is how you can do it in Jolt.
Spec
[
{
"operation": "shift",
"spec": {
// match Property2
"Property2": {
"Tag_*": { // capture the nasty "**2ABC**" part to reference later
// go back up the tree to the root
"#2": {
// match and ignore Property2
"Property2": null,
//
// match Property* and use it and the captured
// "prefix" to create the output key
// &(2,1) references the Tag_*, and pull off the "**2ABC**" part
"Property*": "&(2,1)_&"
}
}
}
}
}
]