Input:
{
"Remarks":"COMMENTS:(87) Test Comments"
}
Expecting Output is below:
{
"Remarks" : "C(87): TestComments"
"Id" : 87
}
I want to replace COMMENTS:(87) string with C(87): and need to get 87 in brackets and print the same in separate attribute "Id".
Can anyone help on this ?
You can use split function along with modify transformation in order to split the string by opening and closing parentheses, and then combine the desired substrings by using concat function. At the last step get rid of the auxiliary elements through use of remove transformation spec such as
[
{
"operation": "modify-overwrite-beta",
"spec": {
"str0": "=split('\\(', #(1,Remarks))",
"str1": "=split('\\)', #(1,str0))",
"str2": "#(1,str1[1])",
"Remarks": "=concat('C(',#(1,str2[0]),'):',#(1,str2[1]))",
"Id": "#(1,str2[0])"
}
},
{
"operation": "remove",
"spec": {
"str*": ""
}
}
]
Related
I need convert the date in NiFi:
2022-11-22 00:00:00
To:
2022-11-22T00:00:00.000Z (ISO 8601)
can someone help me convert this date?
You can use the following spec within a JoltTransformJSON processor presuming that you need a conversion for the current value of an attribute (namely dt nested within a simple JSON object),
2022-11-22 00:00:00 extracted from the variable $now
[
{
// reformat by adding the milliseconds option
"operation": "default",
"spec": {
"dt": "${now():format('yyyy-MM-ddHH:mm:ss.SSS')}"
}
},
{
// split and recombine the pieces of the attribute's value
"operation": "modify-overwrite-beta",
"spec": {
"date": "=substring(#(1,dt),0,10)",
"time": "=substring(#(1,dt),10,22)",
"dt":"=concat(#(1,date),'T',#(1,time),'Z')"
}
},
{
// pick only theoriginal tag name
"operation": "shift",
"spec": {
"dt": "&"
}
}
]
If you (or can) have this info as an attribute, you can use NiFi's expression language to transform it.
For instance, with something like this: ${my_attribute:toDate("yyyy-MM-dd HH:mm:ss", "Europe/Paris"):format("yyyy-MM-dd'T'HH:mm:ss'Z'")}
Input json :
{
"userid" : "31b25f023c58c969388991a6b9b4030000000000",
"username_id": "54fca0dd914ae593ef65988b4a3e93cccc590000000000"
}
There is 10 0's ahead of each value. The overall length of string can vary, but it will be appended by 10 0's always. I would like to remove it.
Thus expected output:
{
"userid" : "31b25f023c58c969388991a6b9b403",
"username_id": "54fca0dd914ae593ef65988b4a3e93cccc59"
}
Hence i am looking for the right way using jolt transform to perform this truncation.
You can start with a modify transformation in which the substring function is applied for each value of the attributes after determining their ten reduced length in order to extract the value except for the last ten characters, as it's mentioned ten zeroes are always right-padded, such as
[
{
"operation": "modify-overwrite-beta",
"spec": {
"userid_len": "=size(#(1,userid))",
"userid_len_dif": "=intSum(-10,#(1,userid_len))",
"userid": "=substring(#(1,&),0,#(1,userid_len_dif))",
"username_id_len": "=size(#(1,username_id))",
"username_id_len_dif": "=intSum(-10,#(1,username_id_len))",
"username_id": "=substring(#(1,&),0,#(1,username_id_len_dif))"
}
},
{
"operation": "shift",
"spec": {
"use*id": "&"
}
}
]
the demo on the site http://jolt-demo.appspot.com/ is
I'm a newbie on Apache Nifi and have the following Problem: I would like to transform a json file as follows:
From:
{
"Property1": "x1",
"Property2": "Tag_**2ABC**",
"Property3": "x3",
"Property4": "x4"
}
to:
{
"**2ABC**_Property1": "x1",
"**2ABC**_Property3": "x3",
"**2ABC**_Property4": "x4"
},
it means: taking the value from a certain Attribute to update all other attributes.
I could find examples using JoltTransformer-Processor that works well when the update is only adding a string. But not for my case
What I've done so far: I have set each Attribute using evaluateJSONPath processor. But I just tried quite a lot of possibilities to use the update Attribute processor to do it without success. All my possible tests looked like (within UpdateAttribute):
Property1 --> ${'Property2':substring(4,6)}"_"${'Property1'}
Using Jolt:
[
{"operation": "modify-overwrite-beta",
"spec": {
"Property1": "${'Property2':substring(4,6)}_${'Property1'}"
}
}
]
Which point am I missing here? Thanks in advance!
I don't know about Nifi, but here is how you can do it in Jolt.
Spec
[
{
"operation": "shift",
"spec": {
// match Property2
"Property2": {
"Tag_*": { // capture the nasty "**2ABC**" part to reference later
// go back up the tree to the root
"#2": {
// match and ignore Property2
"Property2": null,
//
// match Property* and use it and the captured
// "prefix" to create the output key
// &(2,1) references the Tag_*, and pull off the "**2ABC**" part
"Property*": "&(2,1)_&"
}
}
}
}
}
]
I have a JSON input that I am transform using JOLT shift. My problem is I want to use the value of an input key, as a new key in the output data, and in parallel add another value into that new outputted key. Here is my input:
"Description": {
"Name": "John",
"KeyNameId": "John123",
"Description": "John's description"
}
And I want my output to be:
"Description": {
"John123": "John's description"
}
Anyway to do this without using two shift operations?
Or with two shifts if one isn't possible?
Yes, it can be done in a single shift using the "#(Number,words)" operator.
Input - slightly modified for clarity
{
"Top": {
"Name": "John",
"KeyNameId": "John123",
"Description": "John's description"
}
}
Spec
[
{
"operation": "shift",
"spec": {
"Top": {
// match the key "Description" and copy it's value to the Output.
// The Output path being defined by #(1,KeyNameId), which means
// go back up the tree 2 levels (0,1) and lookup the value of
// "KeyNameId"
"Description": "#(1,KeyNameId)"
}
}
}
]
To be more precise,
[
{
"operation": "shift",
"spec": {
"Description": {
"#Description": "Description.#KeyNameId"
}
}
}
]
Please help me in writing the jolt spec to get the expected output with special character "#" as shown below. When I place "#FirstName":"Names.FirstName" in the spec it is not working. Please help in resolving this.
JSON Input:
{
"FirstName": "First"
}
Jolt Spec:
[
{
"operation": "shift",
"spec": {
"FirstName": "Names.FirstName"
}
}
]
Expected Output:
{
"Names" : {
"#FirstName" : "First"
}
}
Updated : As of Jolt version 0.0.21, the "shift" transform now allows special characters to escaped.
This Spec will allow "#FirstName" to be a key in the output.
[
{
"operation": "shift",
"spec": {
"FirstName": "Names.\\#FirstName"
}
}
]