Is it possible to match an entire path in Jolt shiftr? - json

Suppose I want to transform the following
Original
{
"data": {
"a": {
"b": {
"c": {
"value": 1
}
}
}
}
}
For simplicity, say I just want to change value to newValue
Result
{
"data" : {
"a" : {
"b" : {
"c" : {
"newValue" : 1
}
}
}
}
}
I could do that with the following Jolt spec:
Spec
[
{
"operation": "shift",
"spec": {
"data": {
"a": {
"b": {
"c": {
"value": "&4.&3.&2.&1.newValue"
}
}
}
}
}
}
]
But I feel like there should be a less verbose syntax... perhaps something like the following (which does not work):
Desired Syntax... or something like it
[
{
"operation": "shift",
"spec": {
"data.a.b.c.value": "data.a.b.c.newValue" // Even nicer to use & somehow
}
}
]
Is there any Jolt shiftr functionality that I'm missing that would make this nicer?

There is not an existing transform that does that.
Couple of issues:
1) the transform you desire is more of a "nudge" than a "shift". In that you want to change a single value in the Json tree and leave the rest of the tree alone. That doesn't exist in Jolt.
"shift" makes a new output map, and "copies" data from the input to the output.
I tried making shift write to the same input Json tree structure, but it is "dangerous" as it can easily run into ConcurrentModification exceptions.
2) The transform style you have laid out is nice for simple things, but I don't know how it would scale to all the complex stuff wildcard logic (*, [], &).

It is the answer
{
"operation": "modify-overwrite-beta",
"spec": {
"jobInput": {
"ai": {
"common": {
"seed": "#(4,ai.model_structure.seed)"
}
}
}
}
}
#(4,ai.model_structure.seed)
upper sentence means backward 4step on json tree and use seed with this path(ai.model_structure.seed).

Related

Nifi - ignore(or remove) the first digit of JSON

I have a JSON with a variable that I need to ignore the first digit.
{
"destinatarioDTO" : {
"cnpj" : "01377071000170"
}
}
I got the variable with the EvaluateJsonPath.
I need to transform the result 01377071000170 into 1377071000170 (remove the first digit).
You can add a JoltTransformJSON processor along with the following spec
[
{
"operation": "modify-overwrite-beta",
"spec": {
"destinatarioDTO": {
"len_cnpj": "=size(#(1,cnpj))",
"cnpj": "=substring(#(1,cnpj),1,#(1,len_cnpj))" // extract the value starting from the second character(one with the index 1) till the end of the string
}
}
},
{
"operation": "remove",
"spec": {
"destinatarioDTO": {
"len_cnpj": "" // get rid of newly generated, auxiliary attribute
}
}
}
]

JSON attribute value split by space and put them into new attributes using Jolt transform Apache nifi

I have json object as following,
{
"sensorId":2,
"dataValue":26.7,
"dateTime":"2020:12:29 14:20:31"
}
I want to convert it to the following,
{
"sensorId":2,
"dataValue":26.7,
"date":"2020:12:29",
"time":"14:20:31"
}
Using Apache nifi Jolt transform
you can split with space ("* *") and assign splitted parts.
[
{
"operation": "shift",
"spec": {
"sensorId": "sensorId",
"dataValue": "dataValue",
"dateTime": {
"* *": {
"$(0,1)": "date",
"$(0,2)": "time"
}
}
}
}
]

JSON to JSON using JOLT Transformation

I am new to JOLT and got stuck up this requirement, i saw some examples online but in my requirement i needed to add element in a new structure. I hope anyone will be able to understand what i am trying to say
Input JSON
[
{
"ROWSET": {
"ROW": {
"CLTCORP": "1000", //This is CorpId
"CTLITEM": "5000", //This is CorpItemCd
"WHID": "17", //This is WarehouseId
"CTLFAC": "AAHC", //This is FacilityName
"CORP": "001" //This is CorpItem
}
}
}
]
This is expected JSON
{
"SupplyItemData": {
"CorpId": 1000,
"CorpItemCd": 5000
"Warehouse": [{
"WarehouseId": 17,
"FacilityName": "AAHC"
}]
"CorpItem": 001
}
}
Any help or suggestion is appreciated.
I followed few links Transform JSON-JSON JOLT but could not relate exaclty to my use case
You can use the shift operator to do this. First use the * operator to interate through the root level array. Then inside that, simply map the fields to new field names as follows.
[
{
"operation": "shift",
"spec": {
"*": {
"ROWSET": {
"ROW": {
"CLTCORP": "SupplyItemData.CorpId",
"CTLITEM": "SupplyItemData.CorpItemCd",
"WHID": "SupplyItemData.Warehouse.[0].WarehouseId",
"CTLFAC": "SupplyItemData.Warehouse.[0].FacilityName",
"CORP": "SupplyItemData.CorpItem"
}
}
}
}
}
]

Break down JSON properties to array of objects

I am trying to transform a simple JSon object into an array of objects with keys and values broken out, but I'm not sure how to quite get there.
I have tried this a number of ways but the closest I got was to create an object with two arrays, instead of an array with multiple objects with two properties each:
EDIT: I am trying to write a spec which would take any object, not this specific object. I do not know what the incoming object will be other than it will have simple properties (values will not be arrays or other objects).
Sample Input:
{
"property": "someValue",
"propertyName" : "anotherValue"
}
Expected Output:
{
"split_attributes": [
{
"key" : "property",
"value": "someValue"
},
{
"key" : "propertyName",
"value" : "anotherValue"
}
]
}
My spec so far:
{
"operation": "shift",
"spec": {
"*": {
"$": "split_attributes[#0].key",
"#": "split_attributes[#0].value"
}
}
}
Produces
{
"split_attributes" : [
{
"key" : [ "property", "propertyName" ],
"value" : [ "someValue", "anotherValue"]
}
]
}
SOLUTION
I was pretty close, and after looking at the tests, the solution was obvious (it's identical to one of the tests)
{
"operation": "shift",
"spec": {
"*": {
"$": "split_attributes[#2].key",
"#": "split_attributes[#2].value"
}
}
}
From what it seems, I was creating an array but I was looking at the wrong level for an index to the new array. I'm still fuzzy on the whole # level (for example where in the "tree" (and of which object) is #0, #1 and #2 actually looking).

JOLT transformation for nested JSON?

I have a JSON that looks like this :
{
"Level1": {
"Level2": {
"val1": "Test",
"val2": "Val"
}
}
}
When I apply the followng Jolt shift transformation to this :
[{
"operation": "shift",
"spec": {
"Level1": {
"Level2": {
"val1": "val001",
"val2": "val002"
}
}
}
}]
I get the folliwng result :
{
"val001": "Test",
"val002": "Val"
}
Why cant I see the Level1, Level2 in the output? Please an someone help, I want to see that in the output too similar to whats the input.
The values in the shift spec usually refer to the location of the key in the output, so you'd need to include Level1 and Level2 in the values:
[{
"operation": "shift",
"spec": {
"Level1": {
"Level2": {
"val1": "Level1.Level2.val001",
"val2": "Level1.Level2.val002"
}
}
}
}]
If Level1 and/or Level2 can be arbitrary, you can use the # operator to "go back up the tree" and get the values (see the Shiftr javadoc for examples).