I am trying to convert below JSON into name value pair :
{
"Size": "2",
"done": "true",
"records": [
{
"Id": "a7g6s0000004GZuAAM",
"NN": "00096411.0",
"Name": "ISOLIN TRADE & INVEST",
"RecordType": {
"attributes": {
"type": "TestType"
},
"Name": "Term"
}
},
{
"Id": "a7g6s0000004GZzAAM",
"Number": "00096412.0",
"Name": "ISOLIN"
}
]
}
Expecting output JSON :
{
"Size" : "2",
"done" : "true",
"Items" : [ {
"Fields" : [ {
"Name" : "Id",
"Value" : "a7g6s0000004GZuAAM"
}, {
"Name" : "NN",
"Value" : "00096411.0"
}, {
"Name" : "Name",
"Value" : "ISOLIN TRADE & INVEST"
}, {
"Name" : "RecordType_Name",
"Value" : "Term"
} ]
}, {
"Fields" : [ {
"Name" : "Id",
"Value" : "a7g6s0000004GZzAAM"
}, {
"Name" : "Number",
"Value" : "00096412.0"
}, {
"Name" : "Name",
"Value" : "ISOLIN"
} ]
} ]
}
I am using below jolt spec but transformation of RecordType element is not as expected :
Jolt Spec :
[
{
"operation": "remove",
"spec": {
"records": {
"*": {
"attributes": " "
}
}
}
},
{
"operation": "shift",
"spec": {
"*": "&",
"records": {
"*": {
"*": {
"$": "Items.&2.[#2].Name",
"#": "Items.&2.[#2].Value"
}
}
}
}
},
{
"operation": "shift",
"spec": {
"*": "&",
"Items": {
"*": {
"*": "Items.[#2].Fields[]"
}
}
}
}
]
How can I transform this into required format ?
I think remove transformation is not needed, and better to index each Field.
The trick for the solution would be distinguish RecordType attribute from the others by use of
"RecordType": {"Name": "Field&2.&1\\_&"}
such as
[
{
"operation": "shift",
"spec": {
"*": "&",
"records": {
"*": {
"*": "Field&1.&",
"RecordType": {
"Name": "Field&2.&1\\_&"
}
}
}
}
},
{
"operation": "shift",
"spec": {
"*": "&",
"Field*": {
"*": {
"$": "Items[0].&2.[#2].Name",
"#": "Items[0].&2.[#2].Value"
}
}
}
}
]
Related
I'm coming to the conclusion that Jolt is beyond me.
With this input data:-
{
"cluster_id": "1",
"data": {
"id": 1,
"types": [
{
"incident_id": 10,
"incident_ref": "AAA",
"incident_code": "123",
"incident_date": "2010-11-15T00:01:00Z"
},
{
"incident_id": 20,
"incident_ref": "BBB",
"incident_code": "456",
"incident_date": "2020-11-15T00:01:00Z"
}
]
}
}
Spec:-
[
{
"operation": "shift",
"spec": {
"cluster_id": "id",
"data": {
"types": {
"*": {
"incident_id": "incidents",
"incident_ref": "incidents"
}
}
}
}
}
]
Gives:-
{
"id" : "1",
"incidents" : [ 10, "AAA", 20, "BBB" ]
}
How would I get the result of:-
{
"id" : "1",
"incidents" : [
{"id": 10, "ref": "AAA", "code": "123", date: "2010-11-15T00:01:00Z"},
{"id": 20, "ref": "BBB", "code": "456", date: "2020-11-15T00:01:00Z"},
]
}
Tried a bunch of permutations but getting nowhere!
You can use this spec:
[
{
"operation": "shift",
"spec": {
"*": "id",
"data": {
"types": {
"*": {
"incident_*": "incidents[&1].&(0,1)"
}
}
}
}
}
]
To prevent using incident text in the spec, you can use the below spec:
[
{
"operation": "shift",
"spec": {
"*": "id",
"data": {
"types": {
"*": {
"*_*": "&(0,1)[&1].&(0,2)"
}
}
}
}
}
]
You can use two level of shift transformations by extracting substring from tag names such as
[
{
"operation": "shift",
"spec": {
"*": "id",
"data": {
"types": {
"*": "incidents"
}
}
}
},
{
"operation": "shift",
"spec": {
"id": "&",
"*": {
"*": {
"*_*": "&2[&1].&(0,2)"
}
}
}
}
]
I'm trying to understand how to remove the key and object layers for the below. Any help would be greatly appreciated. Trying to use a JOLT processor in NIFI to address this data change.
INPUT:
[ {
"rbr" : {
"fetchTime" : "2020-07-06T23:46:23.677Z",
"customMetadata" : {
"x" : "1",
"o2" : {
"x2" : "y"
}
}
},
"xyz": {
"fetchTime" : "2020-07-06T23:46:23.677Z",
"customMetadata" : {
"x" : "1",
"o2" : {
"x2" : "y"
}
}
}
}
]
desired output:
[
{
"fetchTime" : "2020-07-06T23:46:23.677Z",
"customMetadata" : {
"x" : "1",
"o2" : {
"x2" : "y"
}
},
"type": "rbr"
},
{
"fetchTime" : "2020-07-06T23:46:23.677Z",
"customMetadata" : {
"x" : "1",
"o2" : {
"x2" : "y"
}
},
"type": "xyz"
}
]
Can be done with simple shift operation
[
{
"operation": "shift",
"spec": {
"*": {
"*": {
"fetchTime": "[#2].fetchTime",
"customMetadata": "[#2].customMetadata",
"$": "[#2].type"
}
}
}
}
]
Edit 1
Can shift all the values from current level using "#": "[#3].&",
[
{
"operation": "shift",
"spec": {
"*": {
"*": {
"*": {
"#": "[#3].&",
"$1": "[#3].type"
}
}
}
}
},
{
"operation": "modify-overwrite-beta",
"spec": {
"*": {
"type": "=firstElement(#(1,type))"
}
}
}
]
Iterate over the array, and for each obj create newObj like so:
objKey=Object.keys(obj)[0];
newObj=obj[objKey];
newObj.type=objKey;
and push into new array.
Trying to make a Jolt script that will put in a single number line, then a array of strings in one single array and a tag on the end of that array. These are the values that I have been working with.
JSON INPUT
[
{
"foo": "111",
"bar": "222",
"sun": "333",
"ListofStrings": [
"Dog",
"Train"
],
"ID": "BLAH"
},
{
"foo": "999",
"bar": "222",
"sun": "777",
"ListofStrings": [
"CAT",
"PLANE"
],
"ID": "HAHA"
}
]
JOLT SPEC This is what I have been working with that prints out the ListofStrings but this is the one that works stably.
[
{
"operation": "shift",
"spec": {
"*": {
"foo": "input[].number",
"bar": "input[].number",
"sun": "input[].number",
"ListofStrings": "input[].List"
}
}
},
{
"operation": "default",
"spec": {
"app_id": "test",
"input": {
"*": {
"app_id": "test"
}
}
}
}
]
CURRENT OUTPUT
{
"input" : [ {
"number" : "111"
}, {
"number" : "222"
}, {
"number" : "333"
}, {
"List" : [ "Dog", "Train" ]
}, {
"number" : "999"
}, {
"number" : "222"
}, {
"number" : "777"
}, {
"List" : [ "Cat", "Car" ]
} ],
"app_id" : "test"
}
DESIRED OUTPUT
{
"input" : [ {
"number" : "111"
"List" : [ "Dog", "Train" ]
"ID": "BLAH_foo"
}, {
"number" : "222"
"List" : [ "Dog", "Train" ]
"ID": "BLAH_bar"
}, {
"number" : "333"
"List" : [ "Dog", "Train" ]
"ID": "BLAH_sun"
}, {
"number" : "999"
"List" : [ "Cat", "Car" ]
"ID": "HAHA_foo"
}, {
"number" : "222"
"List" : [ "Cat", "Car" ]
"ID": "HAHA_bar"
}, {
"number" : "777"
"List" : [ "Cat", "Car" ]
"ID": "HAHA_sun"
} ],
"app_id" : "test"
}
Check this spec
[
//Converting list to Map
{
"operation": "shift",
"spec": {
"*": {
"ListofStrings": null,
"*": {
"#": "#1.number",
"#(1,ListofStrings)": "#1.list"
}
}
}
},
//Shift the number and list to the input array
{
"operation": "shift",
"spec": {
"*": {
"$": "input[#2].number",
"#(0,list)": "input[#2].List"
}
}
}, {
"operation": "default",
"spec": {
"app_id": "test",
"input": {
"*": {
"app_id": "test"
}
}
}
}
]
Edit 1
Add the ID node to the map using first shift operation "ID": null, and "#(1,ID)": "#1.ID". Then shift the ID node to the input array in the second shift operation "#(0,ID)": "input[#2].ID".
[
//Converting list to Map
{
"operation": "shift",
"spec": {
"*": {
"ListofStrings": null,
"ID": null,
"*": {
"#": "#1.number",
"#(1,ListofStrings)": "#1.list",
"#(1,ID)": "#1.ID"
}
}
}
},
//Shift the number and list to the input array
{
"operation": "shift",
"spec": {
"*": {
"$": "input[#2].number",
"#(0,list)": "input[#2].List",
"#(0,ID)": "input[#2].ID"
}
}
}, {
"operation": "default",
"spec": {
"app_id": "test",
"input": {
"*": {
"app_id": "test"
}
}
}
}
]
I need to perform a Jolt transformation on the below example json:
Input
[
{
"DOCUMENT_HEADER_ID": "29120",
"Descrizione": "GAS",
"PrezzoTotale": "8.51"
},
{
"DOCUMENT_HEADER_ID": "29120",
"Descrizione": "IMPOSTE",
"PrezzoTotale": "7.25"
},
...
]
I need to trasform to
{
"DOCUMENT_HEADER_ID" : "29120",
"invoice" : [
{
"DOCUMENT_HEADER_ID" : "29120",
"Descrizione" : "GAS",
"PrezzoTotale" : "8.51"
}, {
"DOCUMENT_HEADER_ID" : "29120",
"Descrizione" : "IMPOSTE",
"PrezzoTotale" : "7.25"
},
...
]
}
Using this spec
[
{
"operation": "shift",
"spec": {
"#": "invoice"
}
}
]
I have this output:
{
"invoice" : [ {
"DOCUMENT_HEADER_ID" : "29120",
"Descrizione" : "GAS",
"PrezzoTotale" : "8.51"
}, {
"DOCUMENT_HEADER_ID" : "29120",
"Descrizione" : "IMPOSTE",
"PrezzoTotale" : "7.25"
},
...
]
}
but I can not figure out how to extract the value of DOCUMENT_HEADER_ID from the first element of original array
Can do it in a single shift.
[
{
"operation": "shift",
"spec": {
"#": "invoice",
"0": {
"DOCUMENT_HEADER_ID": "DOCUMENT_HEADER_ID"
}
}
}
]
I found the correct spec
[
{
"operation": "shift",
"spec": {
"#": "invoice"
}
},
{
"operation": "shift",
"spec": {
"invoice": {
"0": {
"DOCUMENT_HEADER_ID": "DOCUMENT_HEADER_ID"
},
"#": "invoice"
}
}
}
]
If having to retrieve single element, data like :
"array_field": ["array_single_element"]
can use:
{
"operation": "modify-overwrite-beta",
"spec": {
"array_field": "=concat(#(1,array_field))",
"*": "=concat(#(1,&))"
}
}
I want to transform a JSON using JOLT, in the following way.:
Input of json file
{
"Document_No": "ANG4",
"LineNo10": {
"Type": 1,
"Customer_No_": "1"
},
"LineNo11": {
"Type": 2,
"Customer_No_": "2"
},
"LineNo12": {
"Type": 3,
"Customer_No_": "3"
}
}
Desired output of json file
[
{
"Document_No":"ANG4"
"Type" : 1,
"Customer_No_" : "1",
"Line_No" : "10"
},
{
"Document_No":"ANG4"
"Type" : 2,
"Customer_No_" : "2",
"Line_No" : "11"
},
{
"Document_No":"ANG4"
"Type" : 3,
"Customer_No_" : "3",
"Line_No" : "12"
}
]
I don't lack a lot, but i don't know what transformations add to get the desired output
my Jolt specificaton is below
[
{
"operation": "shift",
"spec": {
"LineNo*": {
"#": "&",
"$": "&.Line_No"
}
}
},
{
"operation": "shift",
"spec": {
"*": {
"#": "[]"
}
}
}
]
My output json file :
[ {
"Type" : 1,
"Customer_No_" : "1",
"Line_No" : "LineNo10"
}, {
"Type" : 2,
"Customer_No_" : "2",
"Line_No" : "LineNo11"
}, {
"Type" : 3,
"Customer_No_" : "3",
"Line_No" : "LineNo12"
} ]
Could anyone help me with this?
Spec
[
{
"operation": "shift",
"spec": {
"LineNo*": {
// copy Document_No down into the "LineNo" maps
"#(1,Document_No)": "&1.Document_No",
"$": "&1.Line_No",
"*": "&1.&"
}
}
},
{
// accumulate all the "built" LineNo's into an array
"operation": "shift",
"spec": {
"*": {
"#": "[]"
}
}
}
]