I have an SAP Idoc with additional customer specific fields as JSON
{
"E1EDL20": [
{
"VBELN": "1234567890",
"VSTEL": "ZO01",
"LGNUM": "123",
"NTGEW": "100.660",
"ANZPK": "00002",
"/XYZ/XYZ1234_E1EDL20": {
"ACTION": "POSTED",
"CONSIGNMENT": "12345678",
"FREIGHT_ORDER_EXTERNAL": "S123456789",
"IN_YARD": "X",
"IN_YARD_DATE": "20201123",
"IN_YARD_TIME": "100923"
},
"E1ADRM1": [
{
"PARTNER_Q": "LF",
"PARTNER_ID": "0000100000",
"/XYZ/XYZ1234_E1ADRM1": {
"SUPPLIER": "X"
}
},
{
"PARTNER_Q": "OSP",
"PARTNER_ID": "ZO01",
"/XYZ/XYZ1234_E1ADRM1": {
"SUPPLIER": "X"
}
}
]
}
]
}
I want to extract the first field of an array and put it to the top level.
This is the expected output:
{
"deliveryInternal" : "1234567890",
"receivingPoint" : "ZO01",
"warehouseNumber" : "123",
"netWeight" : "100.66",
"numberOfPackages" : "00002",
"type": "LF",
"action" : "POSTED",
"consignment" : "12345678",
"freightOrderExternal" : "S123456789",
"inYard" : "X",
"inYardDate" : "00002"
}
Does anyone have an idea how the problem can be solved ?
You can use shift operation with index-based keys as below
[
{
"operation": "shift",
"spec": {
"E1EDL20": {
"0": {
"VBELN": "deliveryInternal",
"VSTEL": "receivingPoint",
"/XYZ/XYZ1234_E1EDL20": {
"ACTION": "action"
},
"E1ADRM1": {
"0": {
"PARTNER_Q": "type"
}
}
}
}
}
}
]
Related
I have an SAP Idoc with additional customer specific fields as JSON
{
"E1EDL20": [
{
"VBELN": "1234567890",
"VSTEL": "ZO01",
"LGNUM": "123",
"NTGEW": "100.660",
"ANZPK": "00002",
"/XYZ/XYZ1234_E1EDL20": {
"ACTION": "POSTED",
"CONSIGNMENT": "12345678",
"FREIGHT_ORDER_EXTERNAL": "S123456789",
"IN_YARD": "X",
"IN_YARD_DATE": "20201123",
"IN_YARD_TIME": "100923"
},
"E1ADRM1": [
{
"PARTNER_Q": "LF",
"PARTNER_ID": "0000100000",
"/XYZ/XYZ1234_E1ADRM1": {
"SUPPLIER": "X"
}
},
{
"PARTNER_Q": "OSP",
"PARTNER_ID": "ZO01",
"/XYZ/XYZ1234_E1ADRM1": {
"SUPPLIER": "X"
}
}
]
}
]
}
Then I tried to remove the root array, rename the fields and do a type conversation of Strings to Integer, Double and Boolean and concatination of a date and time field to a timestamp. Here my Spec.
[
// Our E1EDL20 array has only one entry -> remove array
{
"operation": "cardinality",
"spec": {
"E1EDL20": "ONE"
}
},
// Move all contained fields to root level and properly rename them
{
"operation": "shift",
"spec": {
"E1EDL20": {
"VBELN": "deliveryInternal",
"VSTEL": "receivingPoint",
"LGNUM": "warehouseNumber",
"NTGEW": "netWeight",
"ANZPK": "numberOfPackages",
"/XYZ/XYZ1234_E1EDL20": {
"ACTION": "action",
"CONSIGNMENT": "consignment",
"FREIGHT_ORDER_EXTERNAL": "freightOrderExternal",
// Rename Content of the field to Boolean String
"IN_YARD": {
"X": { "#true": "inYard" },
"*": { "#false": "inYard" }
},
"IN_YARD_DATE": "inYardDateTmp",
"IN_YARD_TIME": "inYardTimeTmp"
},
// Suppliers
"E1ADRM1": {
"*": {
"PARTNER_Q": "supplier[&1].type",
"PARTNER_ID": "supplier[&1].number",
"/XYZ/XYZ1234_E1ADRM1": {
"SUPPLIER": {
"X": { "#true": "supplier[&2].supplier" },
"*": { "#false": "supplier[&2].supplier" }
}
}
}
}
}
}
},
{
"operation": "modify-overwrite-beta",
"spec": {
// Type conversions, String to Double or Integer
"netWeight": ["=toDouble", null],
"numberOfPackages": ["=toInteger", null],
// Type conversions, String to Boolean
"inYard": ["=toBoolean"],
"supplier": {
"*": {
"belomSupplier": ["=toBoolean"]
}
}
}
},
{
// Create Timestamp for inYardDate
"operation": "modify-default-beta",
"spec": {
"temp_DateYear": "=substring(#(1,inYardDateTmp),0,4)",
"temp_DateMonth": "=substring(#(1,inYardDateTmp),4,6)",
"temp_DateDay": "=substring(#(1,inYardDateTmp),6,8)",
"temp_TimeHours": "=substring(#(1,inYardTimeTmp),0,2)",
"temp_TimeMinutes": "=substring(#(1,inYardTimeTmp),2,4)",
"temp_TimeSeconds": "=substring(#(1,inYardTimeTmp),4,6)",
// Prior to this line, only temporary fields were created from substrings as preparation for the timestamp concatenateion
"inYardDate": "=concat(#(1,temp_DateYear),'-',#(1,temp_DateMonth),'-',#(1,temp_DateDay),'T',#(1,temp_TimeHours),':',#(1,temp_TimeMinutes),':',#(1,temp_prodTimeSeconds),'Z')"
}
},
{
// Remove temporary substring fields
"operation": "remove",
"spec": {
"temp_DateYear": "",
"temp_DateMonth": "",
"temp_DateDay": "",
"temp_TimeHours": "",
"temp_TimeMinutes": "",
"temp_TimeSeconds": "",
"inYardDateTmp": "",
"inYardTimeTmp": ""
}
}
]
I expected after the JOLT transforamtion an the following output.
{
"deliveryInternal" : "1234567890",
"receivingPoint" : "ZO01",
"warehouseNumber" : "123",
"netWeight" : 100.66,
"numberOfPackages" : 2,
"action" : "POSTED",
"consignment" : "12345678",
"freightOrderExternal" : "S123456789",
"inYard" : true,
"inYardDate" : "2020-11-23T10:09:Z",
"supplier" : [ {
"type" : "LF",
"number" : "0000100000",
"supplier" : true
}, {
"type" : "OSP",
"number" : "ZO01",
"supplier" : true
} ]
}
But i getting the following output.
{
"deliveryInternal" : "1234567890",
"receivingPoint" : "ZO01",
"warehouseNumber" : "123",
"netWeight" : 100.66,
"numberOfPackages" : 2,
"action" : "POSTED",
"consignment" : "12345678",
"freightOrderExternal" : "S123456789",
"inYard" : true,
"supplier" : [ {
"type" : "LF",
"number" : "0000100000"
}, {
"type" : "OSP",
"number" : "ZO01"
} ],
"inYardDate" : "2020-11-23T10:09:Z"
}
Does anyone have an idea how the problem can be solved ?
Are you just trying to add supplier: true to your suppliers? I couldn't see any other differences than that. If that's the case you can add this to the end of your chain spec:
{
// Add supplier = true to supplier elements
"operation": "shift",
"spec": {
"supplier": {
"*": {
"#true": "supplier[&1].supplier",
"*": "supplier[&1].&"
}
},
"*": "&"
}
}
I have a JSON input :
{
"Name": "travel.0.0",
"Documents": [
{
"Id": 6732210,
"Name": "Passport",
"Fields": [
{
"Name": "First Name",
"Type": "text"
}
],
"Variants": [
{
"Name": "diplomatic_passport",
"TrainingSamples": [
{
"Name": "First Name",
"Value": "VGhlIFBPU1QsIFBVVCBhbmQgUEFUQ0ggcmVxdWVzdHMgY2FuIGhhdmUgdGhlIHJlcXVlc3QgYm9keSAocGF5bG9hZCksIHN1Y2ggYXMgSlNPTiBvciBYTUwgZGF0YS4gSW4gU3dhZ2dlciB0ZXJtcywgdGhlIHJlcXVlc3QgYm9keSBpcyBjYWxsZWQgYSBib2R5IHBhcmFtZXRlci4gVGhlcmUgY2FuIGJlIG9ubHkgb25lIGJvZHkgcGFyYW1ldGVyLCBhbHRob3VnaCB0aGUgb3BlcmF0aW9uIG1heSBoYXZlIG90aGVyIHBhcmFtZXRlcnMgKHBhdGgsIHF1ZXJ5LCBoZWFkZXIpLg==",
"BoundingBoxes": [
{
"Name": "string",
"Dimensions": [
{
"PageIndex": 0,
"Left": 0,
"Top": 0,
"Right": 0,
"Bottom": 0
}
]
}
]
}
]
}
]
}
]
}
My JOLT Spec is :
[
{
"operation": "shift",
"spec": {
"Name": "ProjectName",
"Documents": {
"*": {
"Name": {
"#": ["DocumentDefinitions[#].Name", "DocumentDefinitions[#].SectionDefinitions[#].Name"]
},
"Variants": {
"*": {
"Name": "DocumentDefinitions[#].SectionDefinitions[#].VarientNames[&1]",
"TrainingSamples": {
"0": {
"Name": "DocumentDefinitions[#].SectionDefinitions[#].ImageSources[#].ImageName.",
"Value": "DocumentDefinitions[#].SectionDefinitions[#].ImageSources[#].ImageValue."
}
}
}
},
"Fields": {
"*": {
"Name": "DocumentDefinitions[#].SectionDefinitions[#].Fields[&1].Name.",
"Type": "DocumentDefinitions[#].SectionDefinitions[#].Fields[&1].Type."
}
}
}
}
}
}
]
My Output is :
{
"ProjectName" : "travel.0.0",
"DocumentDefinitions" : [ {
"Name" : "Passport",
"SectionDefinitions" : [ {
"Name" : "Passport",
"VarientNames" : [ "diplomatic_passport" ],
"ImageSources" : [ {
"ImageName" : "First Name",
"ImageValue" : "VGhlIFBPU1QsIFBVVCBhbmQgUEFUQ0ggcmVxdWVzdHMgY2FuIGhhdmUgdGhlIHJlcXVlc3QgYm9keSAocGF5bG9hZCksIHN1Y2ggYXMgSlNPTiBvciBYTUwgZGF0YS4gSW4gU3dhZ2dlciB0ZXJtcywgdGhlIHJlcXVlc3QgYm9keSBpcyBjYWxsZWQgYSBib2R5IHBhcmFtZXRlci4gVGhlcmUgY2FuIGJlIG9ubHkgb25lIGJvZHkgcGFyYW1ldGVyLCBhbHRob3VnaCB0aGUgb3BlcmF0aW9uIG1heSBoYXZlIG90aGVyIHBhcmFtZXRlcnMgKHBhdGgsIHF1ZXJ5LCBoZWFkZXIpLg=="
} ],
"Fields" : [ {
"Name" : "First Name",
"Type" : "text"
} ]
} ]
} ]
}
This output looks fine , until i have only one JSON object inside my input array of documents . However , when i put another object , it looks somewhat like this which is not desired . It is clubbing all values of both objects together as below :
{
"ProjectName" : "travel.0.0",
"DocumentDefinitions" : [ {
"Name" : [ "Passport", "Passport" ],
"SectionDefinitions" : [ {
"Name" : [ "Passport", "Passport" ],
"VarientNames" : [ [ "diplomatic_passport", "diplomatic_passport" ] ],
"ImageSources" : [ {
"ImageName" : [ "First Name", "First Name" ],
"ImageValue" : [ "VGhlIFBPU1QsIFBVVCBhbmQgUEFUQ0ggcmVxdWVzdHMgY2FuIGhhdmUgdGhlIHJlcXVlc3QgYm9keSAocGF5bG9hZCksIHN1Y2ggYXMgSlNPTiBvciBYTUwgZGF0YS4gSW4gU3dhZ2dlciB0ZXJtcywgdGhlIHJlcXVlc3QgYm9keSBpcyBjYWxsZWQgYSBib2R5IHBhcmFtZXRlci4gVGhlcmUgY2FuIGJlIG9ubHkgb25lIGJvZHkgcGFyYW1ldGVyLCBhbHRob3VnaCB0aGUgb3BlcmF0aW9uIG1heSBoYXZlIG90aGVyIHBhcmFtZXRlcnMgKHBhdGgsIHF1ZXJ5LCBoZWFkZXIpLg==", "VGhlIFBPU1QsIFBVVCBhbmQgUEFUQ0ggcmVxdWVzdHMgY2FuIGhhdmUgdGhlIHJlcXVlc3QgYm9keSAocGF5bG9hZCksIHN1Y2ggYXMgSlNPTiBvciBYTUwgZGF0YS4gSW4gU3dhZ2dlciB0ZXJtcywgdGhlIHJlcXVlc3QgYm9keSBpcyBjYWxsZWQgYSBib2R5IHBhcmFtZXRlci4gVGhlcmUgY2FuIGJlIG9ubHkgb25lIGJvZHkgcGFyYW1ldGVyLCBhbHRob3VnaCB0aGUgb3BlcmF0aW9uIG1heSBoYXZlIG90aGVyIHBhcmFtZXRlcnMgKHBhdGgsIHF1ZXJ5LCBoZWFkZXIpLg==" ]
} ],
"Fields" : [ {
"Name" : [ "First Name", "First Name" ],
"Type" : [ "text", "text" ]
} ]
} ]
} ]
}
Please suggest what changes should i do in my JOLT spec , such that each transformed object in the array , get listed as a seprate object , and not get clubbed .
Check this spec,
[
{
"operation": "shift",
"spec": {
"Name": "ProjectName",
"Documents": {
"*": {
"Name": "DocumentDefinitions[&1].Name",
"Variants": {
"*": {
"Name": "DocumentDefinitions[&3].SectionDefinitions[&1].VarientNames[]",
"TrainingSamples": {
"*": {
"Name": "DocumentDefinitions[&5].SectionDefinitions[&1].ImageSources[&1].ImageName",
"Value": "DocumentDefinitions[&5].SectionDefinitions[&1].ImageSources[&1].ImageValue"
}
}
}
},
"Fields": {
"*": {
"Name": "DocumentDefinitions[&3].SectionDefinitions[&3].Fields[&1].Name",
"Type": "DocumentDefinitions[&3].SectionDefinitions[&3].Fields[&1].Type"
}
}
}
}
}
}
]
I want this:
{
"DATA": [
{
"SPEED" : "hi",
"ANGLE" : "180",
"GRIT" : "2000"
},
{
"SPEED" : "mid",
"ANGLE" : "180",
"GRIT" : "2000"
},
{
"SPEED" : "lo",
"ANGLE" : "180",
"GRIT" : "2000"
}
]
}
To become this:
{
"SPEED" : "hi",
"ANGLE" : "180",
"GRIT" : "2000"
},
{
"SPEED" : "mid",
"ANGLE" : "180",
"GRIT" : "2000"
},
{
"SPEED" : "lo",
"ANGLE" : "180",
"GRIT" : "2000"
}
But with this:
[
{
"operation": "shift",
"spec": {
"DATA": {
"*": {
"*": "&"
}
}
}
}
]
I get this:
{
"SPEED" : [ "hi", "mid", "lo" ],
"ANGLE" : [ "180", "180", "180" ],
"GRIT" : [ "2000", "2000", "2000" ]
}
I'm very new to JOLT and since I can't find any documentation, I come to you for help.
Stackoverflow won't let me post this question since it contains mostly code, but that lies in the nature of this question, namely describing what input I have, what I want it to be and what I've tried.
EDIT:
#Pratik Jaiswal
Thanks for your answer, but it looks like I can do the same with
[
{
"operation": "shift",
"spec": {
"DATA": {
"*": {
"#": ""
}
}
}
}
]
This doesn't remove the outer brackets, though. Is that even possible with JOLT? Or do I just have to get rid of them in another way?
[
{
"operation": "shift",
"spec": {
"DATA": {
"*": {
"SPEED": "[&1].SPEED",
"ANGLE": "[&1].ANGLE",
"GRIT": "[&1].GRIT"
}
}
}
}
]
Test the spec here http://jolt-demo.appspot.com/#inception
I am trying to do a number of shifts to copy some data into multiple arrays and I am having trouble getting the data to correct array.
This is what I have
{
"input": {
"rating_date": "2018-08-06",
"Rates": {
"name": "Generic ratings Card",
"version": "v1",
"value": "2600.00",
"name1": [
{
"hits": "321",
"genre": "fiction",
"ratingName": "name1"
},
{
"hits": "654",
"genre": "fiction",
"ratingName": "name1"
}
],
"name2": [
{
"hits": "123",
"genre": "nonfiction",
"ratingName": "name2"
},
{
"hits": "456",
"genre": "fiction",
"ratingName": "name2"
}
]
}
},
"spec": {
"operation": "shift",
"spec": {
"rating_date": "rating_by_date[#0].date",
"Rates": {
"name*": {
"$": "rating_by_date[#2].ratecards[#1].type",
"*": {
"#": "rating_by_date[#3].ratecards[#2].rates[#0].&"
}
}
}
}
},
"expected": {
"rating_by_date": [
{
"date": "2018-08-06",
"ratecards": [
{
"type": "name1",
"rates": [
{
"hits": "321",
"genre": "fiction",
"ratingName": "name1"
},
{
"hits": "654",
"genre": "fiction",
"ratingName": "name1"
}
]
},
{
"type": "name2",
"rates": [
{
"hits": "123",
"genre": "nonfiction",
"ratingName": "name2"
},
{
"hits": "456",
"genre": "fiction",
"ratingName": "name2"
}
]
}
]
}
]
}
}
The idea is to take the objects in the rates array and transform them (in a later shift) into Key-Value pairs, which I already know how to do.
Now, the spec I have is not shifting the data into appropriate arrays and I end up with something like this:
{
"rating_by_date" : [ {
"date" : "2018-08-06",
"ratecards" : [ {
"type" : "name1",
"rates" : [ {
"0" : {
"hits" : "321",
"genre" : "fiction",
"ratingName" : "name1"
}
} ]
}, {
"rates" : [ {
"1" : {
"hits" : "654",
"genre" : "fiction",
"ratingName" : "name1"
}
} ]
} ]
}, {
"ratecards" : [ {
"type" : "name2",
"rates" : [ {
"0" : {
"hits" : "123",
"genre" : "nonfiction",
"ratingName" : "name2"
}
} ]
}, {
"rates" : [ {
"1" : {
"hits" : "456",
"genre" : "fiction",
"ratingName" : "name2"
}
} ]
} ]
} ]
}
Instead of creating new objects in the same array, it is creating new arrays altogether. I am clearly not understanding exactly how to reference array structures on the RHS, I'd appreciate a clarification.
I managed to get it to work, but I'm not sure why it would not work in one shift step. Here is what I had to do to make it work:
[
{
"operation": "shift",
"spec": {
"rating_date": "rating_by_date[0].date",
"Rates": {
"name*": {
"$": "rating_by_date[0].ratecards.&.type",
"#": "rating_by_date[0].ratecards.&.rates"
}
}
}
},
{
"operation": "shift",
"spec": {
"rating_by_date": {
"*": {
"ratecards": {
"*": {
"#": "rating_by_date[&3].ratecards2[]"
}
},
"*": "rating_by_date[&1].&"
}
}
}
}
]
I am trying to transform json to complex array. The inner array of INVLOC is not generating the way I want.
My input json is :
{
"valid": "true",
"message": "",
"data": {
"ARINVT01": [
{
"firstName": "andrew",
"lastname": "Gilly",
"INVLOC": {
"data": [
{
"mmm": "MAIN",
"nnn": "0.000"
},
{
"mmm": "A1",
"nnn": "0.000"
}
]
}
},
{
"firstName": "Jack",
"lastname": "watson",
"INVLOC": {
"data": [
{
"mmm": "MAIN",
"nnn": "0.000"
}
]
}
}
]
}
}
Expected output :
[
{
"FNAME": "andrew",
"LNAME": "Gilly",
"INVBALANCES": [
{
"user": "MAIN",
"CURBAL": "0.000"
},
{
"user": "A1",
"CURBAL": "0.000"
}
]
},
{
"FNAME": "Jack",
"LNAME": "watson",
"INVBALANCES": [
{
"user": "A1",
"CURBAL": "0.000"
}
]
}
]
My spec :
[
{
"operation": "shift",
"spec": {
"data": {
"ARINVT01": {
"*": {
"firstName": "[&1].FNAME",
"lastname": "[&1].LNAME",
"INVLOC": {
"data": {
"*": {
"mmm": "[&1].INVBALANCES.[#1].user",
"nnn": "[&1].INVBALANCES.[#1].CURBAL"
}
}
}
}
}
}
}
}
]
Getting output :
[ {
"FNAME" : "andrew",
"LNAME" : "Gilly",
"INVBALANCES" : [ {
"user" : "MAIN"
}, {
"CURBAL" : "0.000"
} ]
}, {
"INVBALANCES" : [ {
"user" : "A1"
}, {
"CURBAL" : "0.000"
} ],
"FNAME" : "Jack",
"LNAME" : "watson"
} ]
Can someone Help me with my spec?
And also if I can get some sort of tutorial in understanding the array transformation of JOLT.
Spec
[
{
"operation": "shift",
"spec": {
"data": {
"ARINVT01": {
"*": { // array index of ARINVT01
//
// &0 is "firstName"
// &1 is the current array index of ARINVT01
"firstName": "[&1].FNAME",
"lastname": "[&1].LNAME",
"INVLOC": {
"data": {
"*": { // array index of data array
//
// &0 is "mmm"
// &1 is the current array index of the data array
// &2 is "data"
// &3 is "INVLOC"
// &4 is the current array index of ARINVT01
// basically the "&" "number" logic is
// how many levels to walk back up the document
// to lookup the currently matched value
"mmm": "[&4].INVBALANCES.[&1].user",
"nnn": "[&4].INVBALANCES.[&1].CURBAL"
}
}
}
}
}
}
}
}
]