I'm trying to flatten a JSON file for SQL injestion. However, one of the levels is a date and therefore will not match my database (unless I create millions of fields). I need help please.
Original file:
[
{
"Transactions": {
"Sales": {
"2023-03-31": {
"Item": "Monitor",
"Manufacturer": "BenQ",
"cost": "214.12",
"currency": "Sterling"
},
"2023-03-30": {
"Item": "Keyboard",
"Manufacturer": "Dell",
"cost": "14",
"currency": "Sterling"
},
"2023-03-28": {
"Item": "Laptop",
"Manufacturer": "Acer",
"cost": "840",
"currency": "Sterling"
}
}
}
}
]
What I would like it to look like:
[
{
"Sale-1-item": "Monitor",
"Sale-1-Manufacturer": "BenQ",
"Sale-1-cost": "214.12",
"Sale-1-currency": "Sterling"
"Sale-2-Item": "Keyboard",
"Sale-2-Manufacturer": "Dell",
"Sale-2-cost": "14",
"Sale-2-currency": "Sterling"
"Sale-3-Item": "Laptop",
"Sale-3-Manufacturer": "Acer",
"Sale-3-cost": "840",
"Sale-3-currency": "Sterling"
}
]
Initially I tried using "Item" : "Item-&1" but this created chaos with items such as "Item-2023-03-01" which would need a column in my database for every day of the year! I then tried following the advice in https://github.com/bazaarvoice/jolt/issues/638 but because the next level up is a wildcard it ends up failing as I can't seem to work out how to use the index in the next level down. Any help very gratefully received. Thank you :)
You can use these successive shift transformations
[
{
// convert each innermost attribute to independent arrays
"operation": "shift",
"spec": {
"*": {
"*": {
"*": {
"*": {
"*": "&"
}
}
}
}
}
},
{
// make each indices object key names
"operation": "shift",
"spec": {
"*": {
"*": {
"#": "&.&2"
}
}
}
},
{
// prefix the keys with those indices along with your literal "Sale"
"operation": "shift",
"spec": {
"*": {
"*": "Sale-&1-&"
}
}
}
]
the demo on the site http://jolt-demo.appspot.com/ is
Related
I need to merge array data from two different arrays from the Input JSON and make a flat list of items in the Output JSON. The first array contains the key required for the output and the second array contains the value.
So far, all my attempts at a spec haven't even come close, which is why I haven't listed one. Please see the Input and Desired Output below. Thanks for any help!!
Input JSON :
{
"data": [
{
"2": {
"value": "DC1"
},
"3": {
"value": 5
}
},
{
"2": {
"value": "DC2"
},
"3": {
"value": 10
}
}
],
"fields": [
{
"id": 2,
"label": "DataCenter",
"type": "text"
},
{
"id": 3,
"label": "CCount",
"type": "numeric"
}
],
"metadata": {
"numFields": 2,
"numRecords": 2,
"skip": 0,
"totalRecords": 2
}
}
Desired Output JSON:
[
{
"DataCenter": "DC1",
"CCount": "5"
},
{
"DataCenter": "DC2",
"CCount": "10"
}
]
You can use this spec:
[
{
"operation": "shift",
"spec": {
"data": "&",
"fields": {
"*": {
"label": "fields.#(1,id)"
}
}
}
},
{
"operation": "shift",
"spec": {
"data": {
"*": {
"*": {
"*": "[&2].#(4,fields.&1)"
}
}
}
}
}
]
In your desired output CCount value is a string, You can add the below spec to change an integer to a string.
,
{
"operation": "modify-overwrite-beta",
"spec": {
"*": {
"CCount": "=toString"
}
}
}
You might reciprocally match id value of fields array vs. the object keys of the data array within a common object within the first shift transformation spec in order to prepare for the second one in which we tile the attributes into their individual objects such as
[
{
"operation": "shift",
"spec": {
"*": { // represents the node for both "data" and "fields" arrays
"*": {
"*": {
"value": "&1.&",
"#1,label": "#2,id.&"
}
}
}
}
},
{
"operation": "shift",
"spec": {
"*": {
"value": {
"*": "[&].#2,label"
}
}
}
}
]
the demo on the site http://jolt-demo.appspot.com/ is
I have following JSON data . I want to transform following data by Jolt nifi processor into result data
{
"alert_array": {
"alerts": [
{
"alertId": "alt001",
"severity": "High",
"alert_type": "Alert"
},
{
"alertId": "alt002",
"severity": "High",
"alert_type": "Alert"
}
]
}
}
result data
{
"alert_array": [
{
"Id": "alt001",
"speed": "High",
"type": "Alert"
},
{
"Id": "alt002",
"speed": "High",
"type": "Alert"
}
]
}
please help to write spac of jolt transformation
If you wouldn't rename the attributes, then using
[
{
"operation": "shift",
"spec": {
"*": {
"*": {
"*": "&2[&]" // &2 represents going two levels up to get the literal "alert_array"
}
}
}
}
]
would suffice.
the demo on the site http://jolt-demo.appspot.com/ is
But in your case, you need to qualify each one individually such as
[
{
"operation": "shift",
"spec": {
"*": {
"*": {
"*": {
"alert*": "&3[&1].&(0,1)",
"severity": "&3[&1].speed",
"alert_*": "&3[&1].&(0,1)"
}
}
}
}
}
]
the demo on the site http://jolt-demo.appspot.com/ is
Problems
Hi, I'm using Jolt to transform e-commerce products data. For e.g:
[
{
"objectID": 1,
"string_facet": [
{
"facet_name": "eco_collection",
"facet_value": "No"
},
{
"facet_name": "performance_fabric",
"facet_value": "No"
}
]
},
{
"objectID": 2,
"string_facet": [
{
"facet_name": "activity",
"facet_value": [
"Hiking"
]
}
]
}
]
And my desired output is:
[
{
"objectID": 1,
"string_facet": {
"eco_collection": "No",
"performance_fabric": "No"
}
},
{
"objectID": 2,
"string_facet": {
"activity": [
"Hiking"
]
}
}
]
What I have done so far
I have tried this spec, but it isn't what I need:
[
{
"operation": "shift",
"spec": {
"*": {
"*": "[&1].&",
"string_facet": {
"*": {
"#facet_value": "[&1].string_facet.#facet_name"
}
}
}
}
}
]
I'm looking for an explanation in addition to solution.
Any help would be much appreciated!
You principally need "#facet_value": "#facet_name" match, start with it, then add
&2 to represent going two levels up and grabbing the key name's value(string_facet)
&3 to represent going three levels up and grabbing the indices of the main array to accumulate each attributes nested within individual object of their own.
Then, add an extra shift transformation spec to get rid of the integer key names such as
[
{
"operation": "shift",
"spec": {
"*": {
"*": "&1.&",
"string_facet": {
"*": {
"#facet_value": "&3.&2.#facet_name"
}
}
}
}
},
{
"operation": "shift",
"spec": {
"*": ""
}
}
]
the demo on the site http://jolt-demo.appspot.com/ is
Jolt is new to me and I have been struggling with this issue till the point where I created this post.
I want to turn this:
{
"Properties": [{
"Id": "property1",
"Values": ["randomValue1", "randomValue2"]
}, {
"Id": "property2",
"Values": "randomValue3"
}, {
"Id": "property3",
"Values": "randomValue4"
}]
}
into this
{
"Properties": [{
"Id": "property1",
"Values": "randomValue1"
},{
"Id": "property1",
"Values": "randomValue2"
}, {
"Id": "property2",
"Values": "randomValue3"
}, {
"Id": "property3",
"Values": "randomValue4"
}]
}
The values for each property can be 1 value or an array of an unknown amount of values.
I changed the following json into what is seen in the first json already:
{
"Properties": {
"property1": ["randomValue1", "randomValue1"],
"property2": ["randomValue3"],
"property3": ["randomValue4"]
}
}
Spec:
[{
"operation": "shift",
"spec": {
"Properties": {
"*": {
"*": "Properties[#2].Values",
"$": "Properties[#2].Id"
}
}
}
}]
Property names on RHS are generic and the number of values for a property can differ as well.
Thank you in advanced for taking the time to assist me.
check if this helps:
[
{
"operation": "cardinality",
"spec": {
"Properties": {
"*": {
// normalize values to always be a list
"Values": "MANY"
}
}
}
},
{
"operation": "shift",
"spec": {
"Properties": {
"*": {
"Values": {
"*": {
// create arrays with values and ids
"#": "Values",
"#(2,Id)": "Id"
}
}
}
}
}
},
{
"operation": "shift",
"spec": {
"Values": {
"*": {
// create the final list joining ids and values at the indexes you want
"#": "Properties[&1].Values",
"#(2,Id[#1])": "Properties[&1].Id"
}
}
}
}
]
I was trying to have a key value pair mapped to an array, distinguishing each value as a type using jolt transform spec
Input json
{
"testurl": "someurl",
"website": "someurl2"
}
tried this spec
[
{
"operation": "shift",
"spec": {
"testurl": "Urls[].testurl",
"website": "Urls[].website"
}
},
{
"operation": "shift",
"spec": {
"Urls": {
"*": {
"$": "Urls[&1].val"
}
}
}
}
]
Expected result is like this
{
"Urls": [{
"url": "someurl",
"val": "testurl"
}, {
"url": "someurl2",
"val": "website"
}]
}
Yes you can turn a set of Json key,value pair into an array.
It requires 2 shifts to be safe.
1st shift isolates all the properties you want to turn into an array.
2nd shift is able to use a "*" to then match all those items and place them an array.
Spec
[
{
"operation": "shift",
"spec": {
"testurl": "temp.testurl",
"website": "temp.website"
}
},
{
"operation": "shift",
"spec": {
"temp": {
"*": {
"$": "Urls[#2].val",
"#": "Urls[#2].url"
}
}
}
}
]