Would like to flatten JUST the properties nested JSON, but still work for all objects in the input array
Having trouble putting all three together in one spec (type field, geo field, properties field). I wrote specs to do each one individually, but when I combine the specs to be of use together in one object, it produces the wrong output - the array of objects really messes it up.
Thanks!
Input:
[
{
"type": "Feature",
"geometry": {
"type": "MultiLineString",
"coordinates": [
[
[
-11.11,
11.11
]
]
]
},
"properties": {
"tester_email": "tester123#123.com",
"phase_test": "Test 1"
}
},
{
"type": "Feature22222",
"geometry": {
"type": "MultiLineString",
"coordinates": [
[
[
-11.11,
11.11
]
]
]
},
"properties": {
"tester_email": "tester123#123.com",
"phase_test": "Test 1"
}
}
]
JOLT Spec:
[{
"operation": "shift",
"spec": {
"*": {
"type": "[&1].type",
"geometry": "[&1].geometry",
"properties": "[&1]"
}
}
}
]
Current Output:
[ [ {
"type" : "Feature",
"geometry" : {
"type" : "MultiLineString",
"coordinates" : [ [ [ -11.11, 11.11 ] ] ]
}
}, {
"tester_email" : "tester123#123.com",
"phase_test" : "Test 1"
} ], [ {
"type" : "Feature22222",
"geometry" : {
"type" : "MultiLineString",
"coordinates" : [ [ [ -11.11, 11.11 ] ] ]
}
}, {
"tester_email" : "tester123#123.com",
"phase_test" : "Test 1"
} ] ]
Desired Output:
[ {
"type" : "Feature",
"geometry" : {
"type" : "MultiLineString",
"coordinates" : [ [ [ -11.11, 11.11 ] ] ]
},
"tester_email" : "tester123#123.com",
"phase_test" : "Test 1"
},{
"type" : "Feature22222",
"geometry" : {
"type" : "MultiLineString",
"coordinates" : [ [ [ -11.11, 11.11 ] ] ]
},
"tester_email" : "tester123#123.com",
"phase_test" : "Test 1"
} ]
This worked:
[
{
"operation": "remove",
"spec": {
"*": {
"properties": {
"type": ""
}
}
}
},
{
"operation": "shift",
"spec": {
"*": {
"*": "[&1].&",
"properties": {
"*": "[&2].&"
}
}
}
}
]
Related
I have a JSON input :
{
"Name": "travel.0.0",
"Documents": [
{
"Id": 6732210,
"Name": "Passport",
"Fields": [
{
"Name": "First Name",
"Type": "text"
}
],
"Variants": [
{
"Name": "diplomatic_passport",
"TrainingSamples": [
{
"Name": "First Name",
"Value": "VGhlIFBPU1QsIFBVVCBhbmQgUEFUQ0ggcmVxdWVzdHMgY2FuIGhhdmUgdGhlIHJlcXVlc3QgYm9keSAocGF5bG9hZCksIHN1Y2ggYXMgSlNPTiBvciBYTUwgZGF0YS4gSW4gU3dhZ2dlciB0ZXJtcywgdGhlIHJlcXVlc3QgYm9keSBpcyBjYWxsZWQgYSBib2R5IHBhcmFtZXRlci4gVGhlcmUgY2FuIGJlIG9ubHkgb25lIGJvZHkgcGFyYW1ldGVyLCBhbHRob3VnaCB0aGUgb3BlcmF0aW9uIG1heSBoYXZlIG90aGVyIHBhcmFtZXRlcnMgKHBhdGgsIHF1ZXJ5LCBoZWFkZXIpLg==",
"BoundingBoxes": [
{
"Name": "string",
"Dimensions": [
{
"PageIndex": 0,
"Left": 0,
"Top": 0,
"Right": 0,
"Bottom": 0
}
]
}
]
}
]
}
]
}
]
}
My JOLT Spec is :
[
{
"operation": "shift",
"spec": {
"Name": "ProjectName",
"Documents": {
"*": {
"Name": {
"#": ["DocumentDefinitions[#].Name", "DocumentDefinitions[#].SectionDefinitions[#].Name"]
},
"Variants": {
"*": {
"Name": "DocumentDefinitions[#].SectionDefinitions[#].VarientNames[&1]",
"TrainingSamples": {
"0": {
"Name": "DocumentDefinitions[#].SectionDefinitions[#].ImageSources[#].ImageName.",
"Value": "DocumentDefinitions[#].SectionDefinitions[#].ImageSources[#].ImageValue."
}
}
}
},
"Fields": {
"*": {
"Name": "DocumentDefinitions[#].SectionDefinitions[#].Fields[&1].Name.",
"Type": "DocumentDefinitions[#].SectionDefinitions[#].Fields[&1].Type."
}
}
}
}
}
}
]
My Output is :
{
"ProjectName" : "travel.0.0",
"DocumentDefinitions" : [ {
"Name" : "Passport",
"SectionDefinitions" : [ {
"Name" : "Passport",
"VarientNames" : [ "diplomatic_passport" ],
"ImageSources" : [ {
"ImageName" : "First Name",
"ImageValue" : "VGhlIFBPU1QsIFBVVCBhbmQgUEFUQ0ggcmVxdWVzdHMgY2FuIGhhdmUgdGhlIHJlcXVlc3QgYm9keSAocGF5bG9hZCksIHN1Y2ggYXMgSlNPTiBvciBYTUwgZGF0YS4gSW4gU3dhZ2dlciB0ZXJtcywgdGhlIHJlcXVlc3QgYm9keSBpcyBjYWxsZWQgYSBib2R5IHBhcmFtZXRlci4gVGhlcmUgY2FuIGJlIG9ubHkgb25lIGJvZHkgcGFyYW1ldGVyLCBhbHRob3VnaCB0aGUgb3BlcmF0aW9uIG1heSBoYXZlIG90aGVyIHBhcmFtZXRlcnMgKHBhdGgsIHF1ZXJ5LCBoZWFkZXIpLg=="
} ],
"Fields" : [ {
"Name" : "First Name",
"Type" : "text"
} ]
} ]
} ]
}
This output looks fine , until i have only one JSON object inside my input array of documents . However , when i put another object , it looks somewhat like this which is not desired . It is clubbing all values of both objects together as below :
{
"ProjectName" : "travel.0.0",
"DocumentDefinitions" : [ {
"Name" : [ "Passport", "Passport" ],
"SectionDefinitions" : [ {
"Name" : [ "Passport", "Passport" ],
"VarientNames" : [ [ "diplomatic_passport", "diplomatic_passport" ] ],
"ImageSources" : [ {
"ImageName" : [ "First Name", "First Name" ],
"ImageValue" : [ "VGhlIFBPU1QsIFBVVCBhbmQgUEFUQ0ggcmVxdWVzdHMgY2FuIGhhdmUgdGhlIHJlcXVlc3QgYm9keSAocGF5bG9hZCksIHN1Y2ggYXMgSlNPTiBvciBYTUwgZGF0YS4gSW4gU3dhZ2dlciB0ZXJtcywgdGhlIHJlcXVlc3QgYm9keSBpcyBjYWxsZWQgYSBib2R5IHBhcmFtZXRlci4gVGhlcmUgY2FuIGJlIG9ubHkgb25lIGJvZHkgcGFyYW1ldGVyLCBhbHRob3VnaCB0aGUgb3BlcmF0aW9uIG1heSBoYXZlIG90aGVyIHBhcmFtZXRlcnMgKHBhdGgsIHF1ZXJ5LCBoZWFkZXIpLg==", "VGhlIFBPU1QsIFBVVCBhbmQgUEFUQ0ggcmVxdWVzdHMgY2FuIGhhdmUgdGhlIHJlcXVlc3QgYm9keSAocGF5bG9hZCksIHN1Y2ggYXMgSlNPTiBvciBYTUwgZGF0YS4gSW4gU3dhZ2dlciB0ZXJtcywgdGhlIHJlcXVlc3QgYm9keSBpcyBjYWxsZWQgYSBib2R5IHBhcmFtZXRlci4gVGhlcmUgY2FuIGJlIG9ubHkgb25lIGJvZHkgcGFyYW1ldGVyLCBhbHRob3VnaCB0aGUgb3BlcmF0aW9uIG1heSBoYXZlIG90aGVyIHBhcmFtZXRlcnMgKHBhdGgsIHF1ZXJ5LCBoZWFkZXIpLg==" ]
} ],
"Fields" : [ {
"Name" : [ "First Name", "First Name" ],
"Type" : [ "text", "text" ]
} ]
} ]
} ]
}
Please suggest what changes should i do in my JOLT spec , such that each transformed object in the array , get listed as a seprate object , and not get clubbed .
Check this spec,
[
{
"operation": "shift",
"spec": {
"Name": "ProjectName",
"Documents": {
"*": {
"Name": "DocumentDefinitions[&1].Name",
"Variants": {
"*": {
"Name": "DocumentDefinitions[&3].SectionDefinitions[&1].VarientNames[]",
"TrainingSamples": {
"*": {
"Name": "DocumentDefinitions[&5].SectionDefinitions[&1].ImageSources[&1].ImageName",
"Value": "DocumentDefinitions[&5].SectionDefinitions[&1].ImageSources[&1].ImageValue"
}
}
}
},
"Fields": {
"*": {
"Name": "DocumentDefinitions[&3].SectionDefinitions[&3].Fields[&1].Name",
"Type": "DocumentDefinitions[&3].SectionDefinitions[&3].Fields[&1].Type"
}
}
}
}
}
}
]
Trying to make a Jolt script that will put in a single number line, then a array of strings in one single array and a tag on the end of that array. These are the values that I have been working with.
JSON INPUT
[
{
"foo": "111",
"bar": "222",
"sun": "333",
"ListofStrings": [
"Dog",
"Train"
],
"ID": "BLAH"
},
{
"foo": "999",
"bar": "222",
"sun": "777",
"ListofStrings": [
"CAT",
"PLANE"
],
"ID": "HAHA"
}
]
JOLT SPEC This is what I have been working with that prints out the ListofStrings but this is the one that works stably.
[
{
"operation": "shift",
"spec": {
"*": {
"foo": "input[].number",
"bar": "input[].number",
"sun": "input[].number",
"ListofStrings": "input[].List"
}
}
},
{
"operation": "default",
"spec": {
"app_id": "test",
"input": {
"*": {
"app_id": "test"
}
}
}
}
]
CURRENT OUTPUT
{
"input" : [ {
"number" : "111"
}, {
"number" : "222"
}, {
"number" : "333"
}, {
"List" : [ "Dog", "Train" ]
}, {
"number" : "999"
}, {
"number" : "222"
}, {
"number" : "777"
}, {
"List" : [ "Cat", "Car" ]
} ],
"app_id" : "test"
}
DESIRED OUTPUT
{
"input" : [ {
"number" : "111"
"List" : [ "Dog", "Train" ]
"ID": "BLAH_foo"
}, {
"number" : "222"
"List" : [ "Dog", "Train" ]
"ID": "BLAH_bar"
}, {
"number" : "333"
"List" : [ "Dog", "Train" ]
"ID": "BLAH_sun"
}, {
"number" : "999"
"List" : [ "Cat", "Car" ]
"ID": "HAHA_foo"
}, {
"number" : "222"
"List" : [ "Cat", "Car" ]
"ID": "HAHA_bar"
}, {
"number" : "777"
"List" : [ "Cat", "Car" ]
"ID": "HAHA_sun"
} ],
"app_id" : "test"
}
Check this spec
[
//Converting list to Map
{
"operation": "shift",
"spec": {
"*": {
"ListofStrings": null,
"*": {
"#": "#1.number",
"#(1,ListofStrings)": "#1.list"
}
}
}
},
//Shift the number and list to the input array
{
"operation": "shift",
"spec": {
"*": {
"$": "input[#2].number",
"#(0,list)": "input[#2].List"
}
}
}, {
"operation": "default",
"spec": {
"app_id": "test",
"input": {
"*": {
"app_id": "test"
}
}
}
}
]
Edit 1
Add the ID node to the map using first shift operation "ID": null, and "#(1,ID)": "#1.ID". Then shift the ID node to the input array in the second shift operation "#(0,ID)": "input[#2].ID".
[
//Converting list to Map
{
"operation": "shift",
"spec": {
"*": {
"ListofStrings": null,
"ID": null,
"*": {
"#": "#1.number",
"#(1,ListofStrings)": "#1.list",
"#(1,ID)": "#1.ID"
}
}
}
},
//Shift the number and list to the input array
{
"operation": "shift",
"spec": {
"*": {
"$": "input[#2].number",
"#(0,list)": "input[#2].List",
"#(0,ID)": "input[#2].ID"
}
}
}, {
"operation": "default",
"spec": {
"app_id": "test",
"input": {
"*": {
"app_id": "test"
}
}
}
}
]
I'm so close with this but afraid I do need a hand with a jolt transformation. I've done most of the work but can't get the last minute transformation working.
Here is my data:
{
"totalElements": 168,
"columns": {
"dimension": {
"id": "variables/daterangehour",
"type": "time"
},
"columnIds": [
"1"
]
},
"rows": [
{
"itemId": "119050300",
"value": "00:00 2019-06-03",
"data": [
120
]
},
{
"itemId": "119050805",
"value": "05:00 2019-06-08",
"data": [
98
]
},
{
"itemId": "119050923",
"value": "23:00 2019-06-09",
"data": [
172
]
}
]
}
}
This is my Jolt:
[{
"operation": "shift",
"spec": {
"rows": {
"*": {
"value": "[&1].date",
"data": "[&1].data"
}
}
}
}
]
It gives me this result:
[ {
"date" : "00:00 2019-06-03",
"data" : [ 120 ]
}, {
"date" : "22:00 2019-06-09",
"data" : [ 307 ]
}, {
"date" : "23:00 2019-06-09",
"data" : [ 172 ]
} ]
This causes my system issues, I actual need the data field like this:
[ {
"date" : "00:00 2019-06-03",
"data" : "120"
}, {
"date" : "05:00 2019-06-08",
"data" : "98"
} ]
How do I pluck the item out of the array / square brackets? It will only ever be one item in there.
You should go deeper and take value. Is it clear for you?
[
{
"operation": "shift",
"spec": {
"rows": {
"*": {
"value": "[&1].date",
"data": {
"*": "[&2].data"
}
}
}
}
}
]
And if you need Strings add this:
{
"operation": "modify-overwrite-beta",
"spec": {
"*": {
"data": "=toString"
}
}
}
I want this:
{
"DATA": [
{
"SPEED" : "hi",
"ANGLE" : "180",
"GRIT" : "2000"
},
{
"SPEED" : "mid",
"ANGLE" : "180",
"GRIT" : "2000"
},
{
"SPEED" : "lo",
"ANGLE" : "180",
"GRIT" : "2000"
}
]
}
To become this:
{
"SPEED" : "hi",
"ANGLE" : "180",
"GRIT" : "2000"
},
{
"SPEED" : "mid",
"ANGLE" : "180",
"GRIT" : "2000"
},
{
"SPEED" : "lo",
"ANGLE" : "180",
"GRIT" : "2000"
}
But with this:
[
{
"operation": "shift",
"spec": {
"DATA": {
"*": {
"*": "&"
}
}
}
}
]
I get this:
{
"SPEED" : [ "hi", "mid", "lo" ],
"ANGLE" : [ "180", "180", "180" ],
"GRIT" : [ "2000", "2000", "2000" ]
}
I'm very new to JOLT and since I can't find any documentation, I come to you for help.
Stackoverflow won't let me post this question since it contains mostly code, but that lies in the nature of this question, namely describing what input I have, what I want it to be and what I've tried.
EDIT:
#Pratik Jaiswal
Thanks for your answer, but it looks like I can do the same with
[
{
"operation": "shift",
"spec": {
"DATA": {
"*": {
"#": ""
}
}
}
}
]
This doesn't remove the outer brackets, though. Is that even possible with JOLT? Or do I just have to get rid of them in another way?
[
{
"operation": "shift",
"spec": {
"DATA": {
"*": {
"SPEED": "[&1].SPEED",
"ANGLE": "[&1].ANGLE",
"GRIT": "[&1].GRIT"
}
}
}
}
]
Test the spec here http://jolt-demo.appspot.com/#inception
I am trying to do a number of shifts to copy some data into multiple arrays and I am having trouble getting the data to correct array.
This is what I have
{
"input": {
"rating_date": "2018-08-06",
"Rates": {
"name": "Generic ratings Card",
"version": "v1",
"value": "2600.00",
"name1": [
{
"hits": "321",
"genre": "fiction",
"ratingName": "name1"
},
{
"hits": "654",
"genre": "fiction",
"ratingName": "name1"
}
],
"name2": [
{
"hits": "123",
"genre": "nonfiction",
"ratingName": "name2"
},
{
"hits": "456",
"genre": "fiction",
"ratingName": "name2"
}
]
}
},
"spec": {
"operation": "shift",
"spec": {
"rating_date": "rating_by_date[#0].date",
"Rates": {
"name*": {
"$": "rating_by_date[#2].ratecards[#1].type",
"*": {
"#": "rating_by_date[#3].ratecards[#2].rates[#0].&"
}
}
}
}
},
"expected": {
"rating_by_date": [
{
"date": "2018-08-06",
"ratecards": [
{
"type": "name1",
"rates": [
{
"hits": "321",
"genre": "fiction",
"ratingName": "name1"
},
{
"hits": "654",
"genre": "fiction",
"ratingName": "name1"
}
]
},
{
"type": "name2",
"rates": [
{
"hits": "123",
"genre": "nonfiction",
"ratingName": "name2"
},
{
"hits": "456",
"genre": "fiction",
"ratingName": "name2"
}
]
}
]
}
]
}
}
The idea is to take the objects in the rates array and transform them (in a later shift) into Key-Value pairs, which I already know how to do.
Now, the spec I have is not shifting the data into appropriate arrays and I end up with something like this:
{
"rating_by_date" : [ {
"date" : "2018-08-06",
"ratecards" : [ {
"type" : "name1",
"rates" : [ {
"0" : {
"hits" : "321",
"genre" : "fiction",
"ratingName" : "name1"
}
} ]
}, {
"rates" : [ {
"1" : {
"hits" : "654",
"genre" : "fiction",
"ratingName" : "name1"
}
} ]
} ]
}, {
"ratecards" : [ {
"type" : "name2",
"rates" : [ {
"0" : {
"hits" : "123",
"genre" : "nonfiction",
"ratingName" : "name2"
}
} ]
}, {
"rates" : [ {
"1" : {
"hits" : "456",
"genre" : "fiction",
"ratingName" : "name2"
}
} ]
} ]
} ]
}
Instead of creating new objects in the same array, it is creating new arrays altogether. I am clearly not understanding exactly how to reference array structures on the RHS, I'd appreciate a clarification.
I managed to get it to work, but I'm not sure why it would not work in one shift step. Here is what I had to do to make it work:
[
{
"operation": "shift",
"spec": {
"rating_date": "rating_by_date[0].date",
"Rates": {
"name*": {
"$": "rating_by_date[0].ratecards.&.type",
"#": "rating_by_date[0].ratecards.&.rates"
}
}
}
},
{
"operation": "shift",
"spec": {
"rating_by_date": {
"*": {
"ratecards": {
"*": {
"#": "rating_by_date[&3].ratecards2[]"
}
},
"*": "rating_by_date[&1].&"
}
}
}
}
]