I try append a new field in a JSON structure,i need it in a determined position, before the array but the SPEC locate the field in the end.
This is my JSON original and the code of transformation written in JOLT TRANSFORMATION web:
INPUT:
{
"Id": ">COS",
"equipment": "ALA",
"elementId": "M15463",
"zone": "AMBA",
"hub": "AVA",
"terminalServer": "XS0156",
"Area": "null",
"timestamp": "1576155950000",
"collectedData": [
{
"name": "llljljiouohh",
"instance": "X1.M1.YE9.ON18",
"value": "5",
"unit": "db"
}
]
}
JSON spec:
[
{
"operation": "default",
"spec": {
"timestamp_dt": "2022-10-14 15:00"
}
}
]
**
result:**
{
"Id": ">COS",
"equipment": "ALA",
"elementId": "M15463",
"zone": "AMBA",
"hub": "AVA",
"terminalServer": "XS0156",
"Area": "null",
"timestamp": "1576155950000",
"collectedData": [
{
"name": "llljljiouohh",
"instance": "X1.M1.YE9.ON18",
"value": "5",
"unit": "db"
}
],
"timestamp_dt": "2022-10-14 15:00"
}
Expected:
{
"Id": ">COS",
"equipment": "ALA",
"elementId": "M15463",
"zone": "AMBA",
"hub": "AVA",
"terminalServer": "XS0156",
"Area": "null",
"timestamp": "1576155950000",
"timestamp_dt": "2022-10-14 15:00",
"collectedData": [
{
"name": "llljljiouohh",
"instance": "X1.M1.YE9.ON18",
"value": "5",
"unit": "db"
}
]
}
Any suggestion please? Thanks!
You can individually write each key-value pairs in the desired order within a shift transformation such as
[
{
"operation": "default",
"spec": {
"timestamp_dt": "2022-10-14 15:00"
}
},
{
"operation": "shift",
"spec": {
"Id": "&",
"equipment": "&",
"elementId": "&",
"zone": "&",
"hub": "&",
"terminalServer": "&",
"Area": "&",
"timestamp": "&",
"timestamp_dt": "&",
"collectedData": "&"
}
}
]
Related
I have an array of objects and I want to split each key from the object into a separate array in JSON.
Input JSON:
[
{
"Company": "Tesla",
"Age": 30.2
},
{
"Company": "Facebook",
"Age": 40.5
},
{
"Company": "Amazon",
"Age": 60
}
]
Expected Output
[
{
"value": "Tesla",
"variable": "Company",
"model": "device"
},
{
"value": "Facebook",
"variable": "Company",
"model": "device"
},
{
"value": "Amazon",
"variable": "Company",
"model": "device"
},
{
"value": 30.2,
"variable": "Age",
"model": "device"
},
{
"value": 40.5,
"variable": "Age",
"model": "device"
},
{
"value": 60,
"variable": "Age",
"model": "device"
}
]
Here the value of the Model is fixed. I have tried with some jolt spec. It's not working.
Any help would be much appreciated!
You can use shift transformation spec such as
[
{
// distinguish the attributes by indexes
"operation": "shift",
"spec": {
"*": {
"*": {
"#": "&1[&2].value", // # represents values of the keys inherited from one upper level, &1 represents keys "Company" or "Age", [&2] represents going tree two levels up to grab the outermost indexes in an array fashion
"$": "&1[&2].variable", // $ represents values of the keys inherited from one upper level
"#device": "&1[&2].model"
}
}
}
},
{
// get rid of object/array labels
"operation": "shift",
"spec": {
"*": {
"*": ""
}
}
}
]
the demo on the site http://jolt-demo.appspot.com/ is
Edit : If the input was
[
{
"Company": "Tesla",
"Age": 30.2,
"Time": "27/10/1992"
},
{
"Company": "Facebook",
"Age": 40.5,
"Time": "27/10/1982"
}
]
as mentioned in the comment, then you could convert the spec to
[
{
"operation": "shift",
"spec": {
"*": {
"Company|Age": {
"#": "&1[&2].value",
"$": "&1[&2].variable",
"#(1,Time)": "&1[&2].DateofBirth",
"#device": "&1[&2].model"
}
}
}
},
{
"operation": "shift",
"spec": {
"*": {
"*": ""
}
}
}
]
in order to get the following desired output :
[
{
"value": "Tesla",
"variable": "Company",
"DateofBirth": "27/10/1992",
"model": "device"
},
{
"value": "Facebook",
"variable": "Company",
"DateofBirth": "27/10/1982",
"model": "device"
},
{
"value": 30.2,
"variable": "Age",
"DateofBirth": "27/10/1992",
"model": "device"
},
{
"value": 40.5,
"variable": "Age",
"DateofBirth": "27/10/1982",
"model": "device"
}
]
I have a requirement where re-arrangement of fields need to be done in a nested JSON which has array objects.
Below is the sample input:
{
"_id": "1234",
"Name": "John",
"City": "New Jersey",
"Order": [
{
"item": "pizza",
"qty": 2
},
{
"item": "coke",
"qty": 2
}
],
"State": "NJ",
"Phone": "67755321",
"Paymenttype": "Card"
}
My Jolt is able to populate all fields, but unable to figure out how to get the 'order data' in same manner in output
My code:
[
{
"operation": "shift",
"spec": {
"_id": "_id",
"Name": "Name",
"State": "State",
"Phone": "Phone"
}
}
]
Expected output:
{
"_id" : "1234",
"Name" : "John",
"State" : "NJ",
"Phone" : "67755321",
"Order": [
{
"item": "pizza",
"qty": 2
},
{
"item": "coke",
"qty": 2
}
],
"Paymenttype": "Card"
}
You can just use ampersand substitutions for the values of the attributes such as
[
{
"operation": "shift",
"spec": {
"_id": "&",
"Name": "&",
"State": "&",
"Phone": "&",
"Order": "&",
"Paymenttype": "&"
}
}
]
the demo on the site https://jolt-demo.appspot.com/ is :
even using "*" wildcards within the keys might be practical(depending on the existing key names) such as
[
{
"operation": "shift",
"spec": {
"_id": "&",
"N*": "&",
"St*": "&",
"Ph*": "&",
"Or*": "&",
"Pa*": "&"
}
}
]
I am new to nifi and I am trying to create a week_start_date and week_number from the date in json format.
I am using jolt transform.
The input is google ads api response.
This is the spec I use:
[
{
"operation": "shift",
"spec": {
"customer_id": {
"*": "[&].customer_id"
},
"customer_name": {
"*": "[&].customer_name"
},
"account_currency_code": {
"*": "[&].account_currency_code"
},
"campaign_id": {
"*": "[&].campaign_id"
},
"campaign_name": {
"*": "[&].campaign_name"
},
"campaign_status": {
"*": "[&].campaign_status"
},
"ad_group_id": {
"*": "[&].ad_group_id"
},
"ad_group_name": {
"*": "[&].ad_group_name"
},
"clicks": {
"*": "[&].clicks"
},
"cost": {
"*": "[&].cost"
},
"impressions": {
"*": "[&].impressions"
},
"device": {
"*": "[&].device"
},
"date": {
"*": "[&].date"
},
"week_number": {
"*": "[&].week_number"
},
"year": {
"*": "[&].year"
},
"keywords": {
"*": "[&].keywords"
},
"keywords_id": {
"*": "[&].keywords_id"
}
}
},
{
"operation": "modify-default-beta",
"spec": {
"date": {
"date": "=intSubtract(#(1,date))"
}
}
}
]
The expected output should be:
[
{
"customer_id": "2538943578",
"customer_name": "test.com",
"account_currency_code": "USD",
"campaign_id": "11137311251",
"campaign_name": "testers",
"campaign_status": "ENABLED",
"ad_group_id": "1111",
"ad_group_name": "tesst- E",
"clicks": "6",
"cost": "26580000",
"impressions": "40",
"device": "DESKTOP",
"date": "2021-12-01",
"week_number": "48",
"week_start_date": "2021-11-29",
"year": 2021,
"keywords": "test",
"keywords_id": "56357925842"
}
]
the output I have:
[
{
"customer_id": "2538943578",
"customer_name": "test.com",
"account_currency_code": "USD",
"campaign_id": "11137311251",
"campaign_name": "testers",
"campaign_status": "ENABLED",
"ad_group_id": "1111",
"ad_group_name": "tesst- E",
"clicks": "6",
"cost": "26580000",
"impressions": "40",
"device": "DESKTOP",
"date": "2021-12-01",
"week_number": "2021-11-29",
"year": 2021,
"keywords": "test",
"keywords_id": "56357925842"
}
]
I am not sure on how to use correctly the modify-default-beta
Also I tried looking at the docs:
https://github.com/bazaarvoice/jolt/tree/master/jolt-core/src/test/resources/json/shiftr
What is the correct way also to understand the structure?
I am facing a problem, transforming flat JSON to the nested JSON using jolt transformation. And I am very new to jolt Transformation. Input and output detail is given below.
My input:
[
{
"policyNo": 1,
"lProdCode": 500,
"name": "Prasad",
"id": "10",
"Age": "56"
},
{
"policyNo": 1,
"lProdCode": 500,
"name": "Mahapatra",
"id": "101",
"Age": "56"
},
{
"policyNo": 2,
"lProdCode": 500,
"name": "Pra",
"id": "109",
"Age": "56"
},
{
"policyNo": 3,
"lProdCode": 400,
"name": "Pra",
"id": "108",
"Age": "56"
},
{
"policyNo": 1,
"lProdCode": 500,
"name": "Pra",
"id": "108",
"Age": "56"
}
]
expected output
[
{
"policyNo": 1,
"lProdCode": 500,
"beneficiaries": [
{
"name": "Prasad",
"id": "10900629001",
"Age": "56"
},
{
"name": "Mahapatra",
"id": "10900629001",
"Age": "56"
},
{
"name": "Pra",
"id": "108",
"Age": "56"
}
]
},
{
"policyNo": 2,
"lProdCode": 500,
"beneficiaries": [
{
"name": "Pra",
"id": "10900629001",
"Age": "56"
}
]
},
{
"policyNo": 3,
"lProdCode": 400,
"beneficiaries": [
{
"name": "Pra",
"id": "108",
"Age": "56"
}
]
}
]
Principally you need to group by policyNo attribute along with generating a new list(beneficiaries) for the attributes other than policyNo&lProdCode. That might be handled within a shift transformation. Then add three more steps to prune the roughnesses stems from the first transformation such as
[
{
"operation": "shift",
"spec": {
"*": {
"policyNo": "#(1,policyNo).&",
"lProdCode": "#(1,policyNo).&",
"*": "#(1,policyNo).beneficiaries[&1].&"
}
}
},
{
"operation": "modify-overwrite-beta",
"spec": {
"*": "=recursivelySquashNulls"
}
},
{
"operation": "cardinality",
"spec": {
"*": {
"policyNo": "ONE",
"lProdCode": "ONE"
}
}
},
{
"operation": "shift",
"spec": {
"*": ""
}
}
]
I'm looking for a way to copy or move a key from nested object to the top level
Input:
{
"id": "123",
"name": "foo",
"details": {
"orderNumber": "456789",
"addr": "N st 124",
"date": "2021-01-01"
}
}
desired output:
{
"id": "123",
"name": "foo",
"orderNumber": "456789",
"details": {
"orderNumber": "456789",
"addr": "N st 124",
"date": "2021-01-01"
}
}
or ideally
{
"id": "123",
"name": "foo",
"orderNumber": "456789",
"details": {
"addr": "N st 124",
"date": "2021-01-01"
}
}
the closest I could get is below transformation, but it converts object to value array
[
{
"operation": "shift",
"spec": {
"id": "id",
"name": "name",
"details": {
"orderNumber": "orderNumber",
"*": "details"
}
}
}
]
You're so close to the result, just a slight change(adding an ampersand) is needed such as
[
{
"operation": "shift",
"spec": {
"id": "id",
"name": "name",
"details": {
"orderNumber": "orderNumber",
"*": "&1.&"
}
}
}
]
in this case the keys keeps on appearing.