this is my input json
{
"Name": "Back-test job",
"market": "2",
"TimeZone": "257",
"company": "1"
}
and I want to add the whole json as an object like this:
{
"Name": "Back-test job",
"market": "2",
"TimeZone": "257",
"company": "1",
"wholeJson":{
"Name": "Back-test job",
"market": "2",
"TimeZone": "257",
"company": "1"}
}
do you have any idea for spec?
Just call the base json value twice through use of &1, which refers one-level up as having a nested structure "*": {...} within the specification of shift operation, in one of them prepend with the key wholeJson and also apply a sort specification against getting unordered result such as
[
{
"operation": "shift",
"spec": {
"*": {
"#": "&1",
"#(1,&)": "wholeJson.&1"
}
}
},
{
"operation": "sort",
"spec": {
"*": ""
}
}
]
Related
I need some help with this jolt transformation.
Input:
{
"product": "monitor",
"ID": "222",
"price": "300"
}
expected output:
{
"product": "monitor",
"ID": "222",
"price": "300",
"keys": ["product","ID","price"]
}
You can use $ wildcard along with # wildcard within a shift transformation such as
[
{
"operation": "shift",
"spec": {
"*": {
"#": "&",
"$": "keys"
}
}
}
]
the demo on the site http://jolt-demo.appspot.com/ is
Any help is greatly Appreciated.
I Have input JSON that can have Phone in array or it can be blank or it can be missing.
[
{
"Name": "abc",
"Phone": [
{
"office-1": "123",
"home-1": "989"
},
{
"office-1": "456",
"home-1": "999"
}
],
"Email": "abc#123.com"
},
{
"Name": "efg",
"Phone": [],
"Email": "efg#123.com"
},
{
"Name": "xyz",
"Email": "xyz#123.com"
}
]
My Jolt is already able to convert the Phone number array, but it is not working if the label Phone is missing in JSON input.
Expected output:
[
{
"Name": "abc",
"office-1": "123",
"home-1": "989",
"Email": "abc#123.com"
},
{
"Name": "abc",
"office-1": "456",
"home-1": "999",
"Email": "abc#123.com"
},
{
"Name": "efg",
"Email": "efg#123.com"
},
{
"Name": "xyz",
"Email": "xyz#123.com"
}
]
Please help
You can walk through the objects after separating the stuff under Phone node and the others such as
[
{
"operation": "shift",
"spec": {
"*": {
"Phone": {
"*": {
"*": "&3.&1.&",
"#(2,Name)": "&3.&1.Name",
"#(2,Email)": "&3.&1.Email"
}
},
"*": "&1.&1.&" // "else" case
}
}
},
{
// get rid of object labels
"operation": "shift",
"spec": {
"*": {
"*": ""
}
}
},
{
// get rid of duplicated values of some attributes
"operation": "cardinality",
"spec": {
"*": {
"*": "ONE"
}
}
}
]
the demo on the site http://jolt-demo.appspot.com/ is
new sample of data where I have kept non array data in between array data
I'm trying to flatten a JSON file for SQL injestion. However, one of the levels is a date and therefore will not match my database (unless I create millions of fields). I need help please.
Original file:
[
{
"Transactions": {
"Sales": {
"2023-03-31": {
"Item": "Monitor",
"Manufacturer": "BenQ",
"cost": "214.12",
"currency": "Sterling"
},
"2023-03-30": {
"Item": "Keyboard",
"Manufacturer": "Dell",
"cost": "14",
"currency": "Sterling"
},
"2023-03-28": {
"Item": "Laptop",
"Manufacturer": "Acer",
"cost": "840",
"currency": "Sterling"
}
}
}
}
]
What I would like it to look like:
[
{
"Sale-1-item": "Monitor",
"Sale-1-Manufacturer": "BenQ",
"Sale-1-cost": "214.12",
"Sale-1-currency": "Sterling"
"Sale-2-Item": "Keyboard",
"Sale-2-Manufacturer": "Dell",
"Sale-2-cost": "14",
"Sale-2-currency": "Sterling"
"Sale-3-Item": "Laptop",
"Sale-3-Manufacturer": "Acer",
"Sale-3-cost": "840",
"Sale-3-currency": "Sterling"
}
]
Initially I tried using "Item" : "Item-&1" but this created chaos with items such as "Item-2023-03-01" which would need a column in my database for every day of the year! I then tried following the advice in https://github.com/bazaarvoice/jolt/issues/638 but because the next level up is a wildcard it ends up failing as I can't seem to work out how to use the index in the next level down. Any help very gratefully received. Thank you :)
You can use these successive shift transformations
[
{
// convert each innermost attribute to independent arrays
"operation": "shift",
"spec": {
"*": {
"*": {
"*": {
"*": {
"*": "&"
}
}
}
}
}
},
{
// make each indices object key names
"operation": "shift",
"spec": {
"*": {
"*": {
"#": "&.&2"
}
}
}
},
{
// prefix the keys with those indices along with your literal "Sale"
"operation": "shift",
"spec": {
"*": {
"*": "Sale-&1-&"
}
}
}
]
the demo on the site http://jolt-demo.appspot.com/ is
I have the below JSON input, and I want to remove the value tag and set the Title value in TypeOfSTA.
I want to use JOLT transformation to convert in output JSON.
Below is my request JSON and output JSON.
Request JSON:
{
"value": [
{
"TypeOfSTA": {
"Title": "ashish"
},
"EndDate": "2007-05-30T18:30:00Z",
"IGACode": "175",
"StartDate": "2006-05-31T18:30:00Z",
"STATenure": "12",
"TenureUnit": "Month",
"TotalAmount": "5000",
"EmpEmailID": "employee175#test.com"
},
{
"TypeOfSTA": {
"Title": "Rajan"
},
"EndDate": "2007-05-30T18:30:00Z",
"IGACode": "175",
"StartDate": "2006-05-31T18:30:00Z",
"STATenure": "12",
"TenureUnit": "Month",
"TotalAmount": "5000",
"EmpEmailID": "employee175#test.com"
}
]
}
Please help me to transform using JOLT transformation.
output JSON:
[
{
"TypeOfSTA": "ashish"
"EndDate": "2007-05-30T18:30:00Z",
"IGACode": "175",
"StartDate": "2006-05-31T18:30:00Z",
"STATenure": "12",
"TenureUnit": "Month",
"TotalAmount": "5000",
"EmpEmailID": "employee175#test.com"
},
{
"TypeOfSTA": "Rajan",
"EndDate": "2007-05-30T18:30:00Z",
"IGACode": "175",
"StartDate": "2006-05-31T18:30:00Z",
"STATenure": "12",
"TenureUnit": "Month",
"TotalAmount": "5000",
"EmpEmailID": "employee175#test.com"
}
]
You can use the shift tranformation spec below
[
{
"operation": "shift",
"spec": {
"*": {
"*": {
"TypeOfSTA": { "*": "&1" },
"*": "&"
}
}
}
}
]
The asterisks denote the name of the outermost array(value), indexes of it, the index of TypeOfSTA object, and the rest of the keys existing within the array respectively.
The &1 notation is used to go up and grab the value of the respective object's key.
The result screen from Jolt Transform Demo :
The result screen from Apache NiFi's JoltTransformJSON Processor :
EDIT : There was a single element for the value array within the previous case, in order to avoid the issues related to having multiple or single elements, eg. as a generic case just prepend each ampersand notation by [&2] and [&1] respectively such as
[
{
"operation": "shift",
"spec": {
"*": {
"*": {
"TypeOfSTA": { "*": "[&2].&1" },
"*": "[&1].&"
}
}
}
}
]
I am quite new to JOLT and I need to transform my JSON files to the desired schema. This is my input
[
{
"PK": 12345,
"FULL_NAME":"Amit Prakash",
"BIRTHDATE":"1987-05-25",
"SEX":"M",
"EMAIL": "amprak#mail.com",
"PHONE": "809386731",
"TS":"2015-11-19 14:36:34.0"
},
{
"PK": 12654,
"FULL_NAME": "Rohit Dhand",
"BIRTHDATE":"1979-02-01",
"SEX":"M",
"EMAIL": "rodha#mail.com",
"PHONE": "937013861",
"TS":"2015-11-20 11:03:02.6"
},
...
]
and this is my desired output:
{
"records": [
{
"attribs": [{
"type": "customer",
"reference": "CUST"
}],
"name": "Amit Prakash",
"personal_email": "amprak#mail.com",
"mobile": "809386731",
"id": 12345
},
{
"attribs": [{
"type": "customer",
"reference": "CUST"
}],
"name": "Rohit Dhand",
"personal_email": "rodha#mail.com",
"mobile": "937013861",
"id": 12654
},
...
]
}
So far, I have only managed up to this point:
[
{
"operation": "remove",
"spec": {
"*": {
"BIRTHDATE": "",
"SEX": "",
"TS": ""
}
}
},
{
"operation": "shift",
"spec": {
"*": "records"
}
}
]
But I can't go on from here. I don't know how to rename keys in the output.
Also, what's the alternative to remove operation? remove operation is good if you have fewer keys to exclude than to include, but how about the reverse (few keys to include, more than to exclude within a JSON object)?
Spec
[
{
"operation": "shift",
"spec": {
"*": {
"PK": "records[&1].id",
"PHONE": "records[&1].mobile",
"EMAIL": "records[&1].personal_email",
"FULL_NAME": "records[&1].name"
}
}
},
{
"operation": "default",
"spec": {
"records[]": {
"*": {
"attribs[]": {
"0": {
"type": "customer",
"reference": "CUST"
}
}
}
}
}
}
]
Shift makes copy of your data, whereas the other operations do not. Thus, one way to remove stuff is to just not have it copied across in your shift spec.
Generally the remove is used to get rid of things that would "mess up" the shift.