I want to transform my json into another json with jolt. I have the following JSON :
{
"centre": [
{
"name": "A",
"accesses": [
{
"name": "a",
"totalInputs": 140,
"totalOutputs": 77
},
{
"name": "b",
"totalInputs": 1374,
"totalOutputs": 1068
}
]
},
{
"name": "B",
"accesses": [
{
"name": "c",
"totalInputs": 610,
"totalOutputs": 511
}
]
}
]
}
and I want to extract information from diferent levels of the tree and form a list of jsons. This is my expected output:
[
{
"center": "A",
"accesses": "a",
"totalInputs": 140,
"totalOutputs": 77
},
{
"center": "A",
"accesses": "b",
"totalInputs": 1374,
"totalOutputs": 1068
},
{
"center": "B",
"accesses": "c",
"totalInputs": 610,
"totalOutputs": 511
}
]
You can separate the objects by center and accesses through use of &3 and &1 substitutions respectively while walking by the indices of the accesses array within a shift transformation such as
[
{
"operation": "shift",
"spec": {
"centre": {
"*": {
"accesses": {
"*": {
"#(2,name)": "&3[&1].&4", // go the tree up two levels and grab the "name"'s value while separating the objects by the indices of ***center*** and ***accesses*** arrays
"n*": "&3[&1].&2", // filter "name" in this object by using "n*"(eg. a key starting with "n") since only name's key needs to be replaced, and go two levels up to grab the literal "accesses"
"*": "&3[&1].&" // the rest of the attributes within the current object
}
}
}
}
}
},
{
"operation": "shift",
"spec": {
"*": {
"*": "" // get rid of the key names
}
}
}
]
the demo on the site http://jolt-demo.appspot.com/ is
Related
I'm using jolt and I have an input object where I would like to take the keys out of one property and insert them into each object of an array in another property dynamically:
My input:
{
"data": {
"NAN_KEY": 1,
"TEMP": 3
},
"attributes": [
{
"name": "attribute1",
"value": 3
},
{
"name": "attribute2",
"value": 2
}
]
}
The result I'm aiming for:
"attributes": [
{
"name": "attribute1",
"value": 3,
"NAN_KEY": 1,
"TEMP": 3
},
{
"name": "attribute2",
"value": 2,
"NAN_KEY": 1,
"TEMP": 3
}
]
This question was posted previously in this thread
But after using the solution, I realized I needed it to dynamically add the entire object instead of hardcoding the fields
Any help is appreciated!
Yes, it's possible to make it more dynamic such as
[
{
"operation": "shift",
"spec": {
"attributes": {
"*": {
"#2,data": { "*": "[&1].&" }, // go two levels up the tree to grab the values of the "data" array
"*": "[&1].&"
}
}
}
}
]
the demo on the site http://jolt-demo.appspot.com/ is
I am working on a Jolt transforms processor in Apache Nifi, I am facing some issues, please help me out.
Input:
{
"resourceid": "d6315d4d7f0c",
"timestamp": [
166406,
166404,
166504
],
"Key": [
"mem",
"net",
"diskspace"
],
"data": [
89,
90,
91
]
}
Expected output:
[
{
"resourceid": "d6315d4d7f0c",
"timestamp": 166406,
"Key": "mem",
"data": 89
},
{
"resourceid": "d6315d4d7f0c",
"timestamp": 166404,
"Key": "net",
"data": 90
},
{
"resourceid": "d6315d4d7f0c",
"timestamp": 166504,
"Key": "diskspace",
"data": 91
}
]
You can loop through the indexes of one of the arrays(in this case I've chosen timestamp) within a single shift transformation such as
[
{
"operation": "shift",
"spec": {
"timestamp": {
"*": {
"#(2,resourceid)": "[&].resourceid",
"#": "[&].&2", // "[&]" represents indexes 0,1,2... nested within array to construct a structure of type array
"#(2,Key[&])": "[&].Key", // go two levels up the three in order to reach the level of "Key", and grab its value
"#(2,data[&])": "[&].data"
}
}
}
}
]
the demo on the site http://jolt-demo.appspot.com/ is
I have the following input JSON
[
{
"name": "Project1",
"Addresses": [
[
"B24",
"Niyam Street",
"67897",
"New York"
],
[
"A14",
"Prinston Str",
"London"
]
]
},
{
"name": "Project2",
"Addresses": [
[
"123",
"Portland Street",
"234"
],
[
"Lalbag",
"Kolaba"
],
[
"8th Avenue",
"3rd Signal"
]
]
}
]
I would like to transform it into like the below:
[
{
"name": "Project1",
"Addresses": [
"B24 Niyam Street 67897 New York",
"A14 Prinston Str London"
]
},
{
"name": "Project2",
"Addresses": [
"123 Portland Street 234",
"Lalbag Kolaba",
"8th Avenue 3rd Signal"
]
}
]
The Addresses attribute value is a two-dimensional array of dynamic size.
Could you please help me with a valid jolt spec or some hints to achieve it? I am lost. Thank you so much.
You can use combination of shift and modify transformations such as
[
{
"operation": "shift",
"spec": {
"*": {
"*": "&", // the attributes other than "Addresses"
"Addresses": {
"*": "&1.&2.&" // determine the objects with indexed key names(0,1,2...) where &2 represents going two level up the tree to get the outermost indices of the objects of the main array, & for indices of the "Addresses" array, and &1 to keep the key name "Addresses" to transfer to the upcoming specs
}
}
}
},
{
"operation": "modify-overwrite-beta",
"spec": {
"*": {
"*": {
"*": "=join(' ',#(1,&))" // concatenate all components of each array respectively at once
}
}
}
},
{
// combine the attributes back again
"operation": "shift",
"spec": {
"*": {
"*": "[&].&1"
},
"A*": {
"*": {
"*": "[&1].&2"
}
}
}
}
]
the demo on the site http://jolt-demo.appspot.com/ is
i want to use JOLT Transform to convert an dictionary to an JSON Object. Here below i demonstrate it with an example below.
There is in the root an Array which contains the results from different computers. "Computer1", "Computer2", and more.
This structure should be stay. I removed the 2nd and more array elemt which will reoccur in the same way with {...}.
Given Object:
[
{
"name": "Computer1",
"events": {
"counts": [
{
"countType": "CRITICAL",
"count": 5
},
{
"countType": "HIGH",
"count": 12
},
{
"countType": "LOW",
"count": 40
}
]
},
"processes": {
"counts": [
{
"countType": "CRITICAL",
"count": 0
},
{
"countType": "HIGH",
"count": 2
},
{
"countType": "LOW",
"count": 80
}
]
}
},
{
"name": "Computer2",
"events": {...},...
}
]
Desired Output:
[
{
"name": "Computer1",
"events": {
"CRITICAL": 5,
"HIGH": 12,
"LOW": 40
},
"processes": {
"CRITICAL": 0,
"HIGH": 2,
"LOW": 80
}
}
, {
"name": "Computer2",
"events": {...},
...
}
]
Please help to identify the right JOLT spec.
Thanks in advance
Marcus
You can use such a shift transformation specs
[
{
"operation": "shift",
"spec": {
"*": {
"name": "&1.&",
"events|processes": { // pipe represents "OR" logic
"counts": {
"*": {
"#count": "&4.&3.#countType" // "&4" and "&3" represent going four and three level up the tree to grab the indices of the outermost level list and the key name of the objects("events" and "processes") respectively
}
}
}
}
}
},
{
"operation": "shift",
"spec": {
"*": ""
}
}
]
I have a working transformation, but it is using hard-coded property name references. I've been trying to figure out how to define a transformation that is generic but I haven't found anything that works.
My input data looks like this:
{
"data": [
{
"A": 1,
"B": "testing",
"C": 100
},
{
"A": 2,
"B": "testing",
"C": 200
}
],
"newVals": {
"B": "new val",
"C": 7
}
}
I want to apply the values in the newVals object and apply them to the objects in the array so that the resulting data looks like this:
[
{
"A": 1,
"B": "new val",
"C": 7
},
{
"A": 2,
"B": "new val",
"C": 7
}
]
The transformation I have is working, but requires the property names to be hard-coded. I am trying to define a transformation that would allow any matching property name to be properly modified. So that if another data set had an property "D", with a new value in the newVals object it would be applied.
My hard-coded transformation looks like this:
[
{
"operation": "modify-default-beta",
"spec": {
"data": {
"*": {
"new_B": "#(3,newVals.B)",
"new_C": "#(3,newVals.C)"
}
}
}
},
{
"operation": "shift",
"spec": {
"data": {
"*": {
"#(0,A)": "[#2].A",
"#(0,new_B)": "[#2].B",
"#(0,new_C)": "[#2].C"
}
}
}
}
]
Is there a way to write that so that in addition to the example data above it can transform this:
{
"data": [
{
"A": 1,
"E": "testing",
"F": 100
}
],
"newVals": {
"E": "new val",
"F": 7
}
}
For the first desired result;
Convert modify-default-beta to modify-overwrite-beta spec, this case no need need to rename the keys, eg. new_ prefixes are not needed. Meanwhile, no need to explicitly write each element, rather replace them with "*": "#(3,newVals.&)". And then just pick the objects of the data array within the shift spec such as
[
{
"operation": "modify-overwrite-beta",
"spec": {
"data": {
"*": {
"*": "#(3,newVals.&)"
}
}
}
},
{
"operation": "shift",
"spec": {
"data": {
"#": ""
}
}
}
]
For the second desired result;
Using just one step of shift spec would be sufficient such as
[
{
"operation": "shift",
"spec": {
"data": {
"0": {
"A": "&2.[#2].&",
"B": "&2.[#2].E",
"C": "&2.[#2].F"
}
},
"*": "&"
}
}
]
where "*": "&" is used to represent the array other than data