Value input
"ItemInternalId" : [ "01", "02", "011055000000345" ]
Value output expected
"ItemInternalId" : [ "01", "16", "011055000000345" ]
Jolt only has two functions, "lastElement" and "firstElement", so I couldn't find anything on the internet that modified the second value
No need to use a modify transformation which has those functions, but a shift transformation spec along with index of the respective component such as
[
{
"operation": "shift",
"spec": {
"*": { //stands for "ItemInternalId"
"1": { // the level of the second component of the array in which index starts from zero
"#16": "&2" // the fixed value might be assigned by prepending the value with "#" on the left-hand-side, &2 represents going the tree two levels up to grab the tag of the array
},
"*": "&1"//else case. The &1 on the right-hand-side represents going the tree one level up to grab the tag of the array
}
}
}
]
assuming your input is
{
"ItemInternalId": [
"01",
"02",
"011055000000345"
]
}
the demo on the site http://jolt-demo.appspot.com/ is
Related
I want to change the name of fields in an array by Jolt. Please explain how it works with your answer.
Input:
{
"referenceName": "***",
"name": "***"
}
Desired output:
{
"ReferenceName": "***",
"ColumnName": "***"
}
You can use this simple spec:
[
{
"operation": "shift",
"spec": {
"referenceName": "ReferenceName",
"name": "ColumnName"
}
}
]
Or you can have simpler:
[
{
"operation": "shift",
"spec": {
"r*": "R&(0,1)",
"n*": "ColumnN&(0,1)"
}
}
]
When you want to change some names in your JSON, You can use the shift operation like the above spec:
r*: match all keys that started with the r like referenceName and * match everything after r that equals eferenceName. You can write R&(0,1) that &(0,1) value is equal to first *.
n*: match all keys that started with the n like name and * match everything after n that is equal to ame. You can write ColumnN&(0,1) that &(0,1) value is equal to first *.
Note: & means the key's value, But when you use the &(0,1), you are saying that in the 0 level or current level please get the first * value. Suppose you had a second * in this level, You should get it like this: &(0,2)
this is a simple question,I am new to nifi and jolt. I just wanted to know,how to travserse the jolt spec while using wild card characters.For example,this is an example in jolt demo site,
input is
{
"data": {
"1234": {
"clientId": "12",
"hidden": true
},
"1235": {
"clientId": "35",
"hidden": false
}
}
}
Spec is
[
{
"operation": "shift",
"spec": {
"data": {
"*": {
"hidden": {
"true": {
// if hidden is true, then write the value disabled to the RHS output path
// Also #(3,clientId) means lookup the tree 3 levels, to the "1234" or "1235" level,
// and then come back down down the tree and grabe the value of "clientId"
"#disabled": "clients.#(3,clientId)"
},
"false": {
"#enabled": "clients.#(3,clientId)"
}
}
}
}
}
}
]
and output is
{
"clients" : {
"12" : "disabled",
"35" : "enabled"
}
}
How did we get the above output? like what #(3,clientsid).As far as I understand, it goes 3 levels up.But 3 levels with respect to what..the spec or the input? Either way,how to move 3 levels up,can you please define which are the levels here?
Thanks in advance
Just count the number of each opening curly-braces({) or colons(:) in the backward direction. Whenever they're not independent such as :{, then count this as only one in order to reach to the target key which is "*" wildcard just under "data" key in this case, and for #(3,clientId); first level is crossing the colon(:) next to "#disabled" or "#enabled", second level is crossing opening curly-braces next to those boolean keys for each, and then the third level is crossing opening curly-brace just after "hidden" key to reach the indexes the object with the "data" key.
Can anyone please help me a JOLT spec to convert my matrix type json to denormalized json. Please find the below my input json and my expected josn output.
Input Json:
[
{
"attributes": [
{
"name": "brand",
"value": "Patriot Lighting"
},
{
"name": "color",
"value": "Chrome"
},
{
"name": "price",
"value": "49.97 USD"
}
]
},
{
"attributes": [
{
"name": "brand",
"value": "Masterforce"
},
{
"name": "color",
"value": "Green"
},
{
"name": "price",
"value": "99.0 USD"
}
]
}
]
Expected Json output:
[
{
"brand": "Patriot Lighting",
"color": "Chrome",
"price": "49.97 USD"
},
{
"brand": "Masterforce",
"color": "Green",
"price": "99.0 USD"
}
]
I was trying to build JOLT spec to convert this json. But challenge is the json which I have multiple tables with "attributes" tag.
Thanks in advance!
JOLT is not easy to use but I get a lot out of some other StackOverflow questions floating around and I just started reading up on the source code comments
[
{
"operation": "shift",
"spec": {
// for each element in the array
"*": {
"attributes": {
// for each element in the attribute
"*": {
// grab the value
// - put it in an array
// - but it must be indexed by the "positions" found four steps back
// - put the value in a key
// - that is determined by moving one step back and looking at member name
"value": "[#4].#(1,name)"
}
}
}
}
}
]
This is seems very obscure at first glance but I hope the comments explain everything.
Please go read on JOLT transformation to copy single value along an array
Also this is almost mandatory for JOLT beginners https://docs.google.com/presentation/d/1sAiuiFC4Lzz4-064sg1p8EQt2ev0o442MfEbvrpD1ls/edit#slide=id.g9a487080_011
If you need another example, I just answered a question here Nifi JOLT: flat JSON object to a list of JSON object
And, probably your best friend can be found at https://jolt-demo.appspot.com
I came across this question regarding replacing a non-existing field with another field's value. It explains the usage of modify-default-beta operation. Example mentioned in the post:
Spec
[
{
"operation": "modify-default-beta",
"spec": {
"shipping_address": {
"address": "#(2,payment_address.address)"
}
}
}
]
Input A, where there is not shipping address
{
"payment_address": {
"address": "some address"
},
"shipping_address": {}
}
Produces output A, where the billing address is copied over
{
"payment_address" : {
"address" : "some address"
},
"shipping_address" : {
"address" : "some address"
}
}
What is the meaning of the "2" in #(2,payment_address.address). I tried this example here and it works even if I replace the "2" with a "3".
The Jolt operations do a parallel tree walk of the input JSON and the Spec. It starts at the root the Spec & input JSON and then does a depth first traversal.
While it is is doing it's depth first traversal it maintains a "stack" of the data / nodes that it has matched.
Thus in this spec, when you "match" down to "address"
"operation": "modify-default-beta",
"spec": {
"shipping_address": {
"address": "#(2,payment_address.address)"
}
}
The stack looks like :
Stack "pointer" Matched value Pointer to Input
0 "address" Value of Address : String if it exists
1 "shipping_address" Value of "shipping_address" : Map if it exists
2 "_root_" A made up entry to point to Input Json (Map or List)
3 "_root_" Another "root" that points to a Map
WorkAround to deal with top level List JSON input
So the "2" gets you back up to the "top level" of the input Json, so that you can "navigate" down "payment_address.address".
"3" works because you are now in the wrapper that makes it so that all "top level" input to the Transforms is a Map, to workaround the fact that a top level list / "[]" if valid JSON. This wrapper is special cased to have the same reference as the "2".
"4" does not not exist in the stack, and so it doesn't do anything.
I have a JSON input that I am transform using JOLT shift. My problem is I want to use the value of an input key, as a new key in the output data, and in parallel add another value into that new outputted key. Here is my input:
"Description": {
"Name": "John",
"KeyNameId": "John123",
"Description": "John's description"
}
And I want my output to be:
"Description": {
"John123": "John's description"
}
Anyway to do this without using two shift operations?
Or with two shifts if one isn't possible?
Yes, it can be done in a single shift using the "#(Number,words)" operator.
Input - slightly modified for clarity
{
"Top": {
"Name": "John",
"KeyNameId": "John123",
"Description": "John's description"
}
}
Spec
[
{
"operation": "shift",
"spec": {
"Top": {
// match the key "Description" and copy it's value to the Output.
// The Output path being defined by #(1,KeyNameId), which means
// go back up the tree 2 levels (0,1) and lookup the value of
// "KeyNameId"
"Description": "#(1,KeyNameId)"
}
}
}
]
To be more precise,
[
{
"operation": "shift",
"spec": {
"Description": {
"#Description": "Description.#KeyNameId"
}
}
}
]