How to sum values in json using jq - json

I just started using jq parser as i need to parse some values out of a json file and make few changes and output an updated json file with the changes. Here is one of the samples out of many i'm working on.
input json:
{
"name": ".",
"metadata": {
"path": ".",
"type": "terraform_dir",
"terraformWorkspace": "default"
},
"pastBreakdown": {
"resources": [],
"totalHourlyCost": "0",
"totalMonthlyCost": "0"
},
"breakdown": {
"resources": [
{
"name": "vm01",
"metadata": {},
"hourlyCost": "0.007989726027400584",
"monthlyCost": 3.9660999999999986,
"costComponents": [
{
"name": "vm01-201",
"unit": "years",
"hourlyQuantity": "0.0001141552511416",
"monthlyQuantity": "0.0833333333333333",
"price": "69.99",
"hourlyCost": "0.007989726027400584",
"monthlyCost": "5.832499999999997667"
}
]
},
{
"name": "public_ip.example",
"metadata": {},
"hourlyCost": "0.005",
"monthlyCost": 2.482,
"costComponents": [
{
"name": "IP address (static)",
"unit": "hours",
"hourlyQuantity": "1",
"monthlyQuantity": "730",
"price": "0.005",
"hourlyCost": "0.005",
"monthlyCost": "3.65"
}
]
},
{
"name": "storage_account",
"metadata": {},
"hourlyCost": "0.17575342465753424425",
"monthlyCost": 87.24400000000001,
"costComponents": [
{
"name": "Write",
"unit": "10K operations",
"hourlyQuantity": "0.136986301369863",
"monthlyQuantity": "100",
"price": "0.1",
"hourlyCost": "0.0136986301369863",
"monthlyCost": "10"
},
{
"name": "List",
"unit": "10K operations",
"hourlyQuantity": "0.136986301369863",
"monthlyQuantity": "100",
"price": "0.05",
"hourlyCost": "0.00684931506849315",
"monthlyCost": "5"
},
{
"name": "Read",
"unit": "10K operations",
"hourlyQuantity": "0.0136986301369863",
"monthlyQuantity": "10",
"price": "0.01",
"hourlyCost": "0.000136986301369863",
"monthlyCost": "0.1"
},
{
"name": "All other operations",
"unit": "10K operations",
"hourlyQuantity": "0.136986301369863",
"monthlyQuantity": "100",
"price": "0.004",
"hourlyCost": "0.000547945205479452",
"monthlyCost": "0.4"
},
{
"name": "retrieval",
"unit": "GB",
"hourlyQuantity": "1.3698630136986301",
"monthlyQuantity": "1000",
"price": "0.01",
"hourlyCost": "0.013698630136986301",
"monthlyCost": "10"
},
{
"name": "write",
"unit": "GB",
"hourlyQuantity": "1.3698630136986301",
"monthlyQuantity": "1000",
"price": "0.0025",
"hourlyCost": "0.00342465753424657525",
"monthlyCost": "2.5"
},
{
"name": "index",
"unit": "10K tags",
"hourlyQuantity": "0.0136986301369863",
"monthlyQuantity": "10",
"price": "0.03",
"hourlyCost": "0.000410958904109589",
"monthlyCost": "0.3"
}
]
}
],
"totalMonthlyCost": "sum value"
From the above json i need to sum the monthlyCost of each resources and update total value in totalMonthlyCost
So the input would be "3.9660999999999986 + 2.482 + 87.24400000000001" the sum of these values should be updated in totalMonthlyCost
I have been trying various options but no luck. I could calculate the values within the array
but not sure how to update the value in totalMonthlyCost

In two steps in accordance with the task description:
(.breakdown.resources | map(.monthlyCost) | add) as $sum
| .breakdown.totalMonthlyCost = $sum
This can be written without the intermediate variable:
.breakdown.totalMonthlyCost
= (.breakdown.resources | map(.monthlyCost) | add)
or more DRYly:
.breakdown
|= (.totalMonthlyCost = (.resources | map(.monthlyCost) | add))

Related

Metadata converter?

Im currently working on a generated collection and got a huge metadata file. Its build up like this:
[
{
"name": "amongnfts #1",
"description": "This is one member of the AmongNFT collection. It's unique and only exists once, but may be a bit cooler than the others.",
"image": "/1.png",
"dna": "055b93fd187e52f85e3ea8b0ad660fa51aaea9e0",
"edition": 1,
"date": 1645368881553,
"attributes": [
{
"trait_type": "background",
"value": "blue"
},
{
"trait_type": "body",
"value": "pink"
},
{
"trait_type": "clothing",
"value": "teamred"
},
{
"trait_type": "hat",
"value": "pilot"
},
{
"trait_type": "visor",
"value": "krieghaus"
},
{
"trait_type": "pet",
"value": "minimateorange"
}
],
"compiler": "HashLips Art Engine"
},
{
"name": "amongnfts #2",
"description": "This is one member of the AmongNFT collection. It's unique and only exists once, but may be a bit cooler than the others.",
"image": "/2.png",
"dna": "a05e9acf4665b9e9e5adbcb9dfc33df255e2e58f",
"edition": 2,
"date": 1645368883622,
"attributes": [
{
"trait_type": "background",
"value": "orange"
},
{
"trait_type": "body",
"value": "white"
},
{
"trait_type": "clothing",
"value": "elf"
},
{
"trait_type": "hat",
"value": "wethair"
},
{
"trait_type": "visor",
"value": "none"
},
{
"trait_type": "pet",
"value": "minimatetan"
}
],
"compiler": "HashLips Art Engine"
},
]
(Goes down for 10.000 editions so thats why I dont do it manually.)
But it should look like this:
{
"nft": [
{
"file_path": "/1.png",
"nft_name": "amongnfts #1",
"external_link": "",
"description": "This is one member of the AmongNFT collection. It's unique and only exists once, but may be a bit cooler than the others.",
"collection": "AmongNFTs",
"properties": [
{
"type": "background",
"name": "blue"
},
{
"type": "body",
"name": "pink"
},
{
"type": "clothing",
"name": "teamred"
},
{
"type": "hat",
"name": "pilot"
},
{
"type": "visor",
"name": "krieghaus"
},
{
"type": "pet",
"name": "minimateorange"
},
],
"levels": "",
"stats": "",
"unlockable_content": "",
"explicit_and_sensitive_content": "",
"supply": "1",
"blockchain": "Polygon",
"sale_type": "",
"price": 0.001,
"method": "",
"duration": "",
"specific_buyer": "",
"quantity": ""
}
]
}
Is there an easy way of converting them automatically and if so, how would the code look like?
The metadata.json file is here.

Flatten a JSON document using jq by filtering an array by keys

I have a JSON in the following format:
{
"subFields": [
{
"id": "question_1",
"type": "radioGroup",
"description": "Description1",
"title": "title1",
"subFields": [
{
"type": "radio",
"label": "Yes",
"value": 1
},
{
"type": "radio",
"label": "No",
"value": 0
},
{
"uiComponent": "SmallContent",
"componentProps": {
"text": "* If the answer to the above question is “Yes”, please contact the Support immediately."
}
}
]
},
{
"uiComponent": "Spacer"
},
{
"id": "question_2",
"type": "radioGroup",
"description": "Description2",
"title": "Title2",
"subFields": [
{
"type": "radio",
"label": "Label - Value 1",
"value": 1
},
{
"type": "radio",
"label": "Label - Value 2",
"value": 2
},
{
"type": "radio",
"label": "Label - Value 3",
"value": 3
},
{
"type": "radio",
"label": "Other",
"value": 13,
"subFields": [
{
"id": "question_2a",
"type": "string",
"condition": {
"type": "BinaryExpression",
"operator": "==",
"left": {
"type": "Identifier",
"name": "question_2"
},
"right": {
"type": "Literal",
"value": 13
}
}
}
]
}
]
},
{
"id": "question_2_b",
"style": {
"marginTop": "30px"
},
"type": "radioGroup",
"description": "Description3",
"title": "",
"subFields": [
{
"type": "radio",
"label": "Label - Radio 1",
"value": 1
},
{
"type": "radio",
"label": "Label - Radio 2",
"value": 2
},
{
"type": "radio",
"label": "Label - Radio 3",
"value": 3
}
]
},
{
"uiComponent": "Spacer"
},
{
"id": "question_3",
"type": "radioGroup",
"description": "Description3",
"title": "Title3",
"subFields": [
{
"type": "radio",
"label": "Yes",
"value": 1
},
{
"type": "radio",
"label": "No",
"value": 0
}
]
},
{
"uiComponent": "Spacer"
},
{
"condition": {
"type": "BinaryExpression",
"operator": "==",
"left": {
"type": "Identifier",
"name": "signer_type"
},
"right": {
"type": "Literal",
"value": "entity"
}
},
"subFields": [
{
"uiComponent": "Spacer"
},
{
"id": "question_4",
"type": "radioGroup",
"description": "Description_4",
"title": "Title_4",
"subFields": [
{
"type": "radio",
"label": "Yes",
"value": 1
},
{
"type": "radio",
"label": "No",
"value": 0
}
]
},
{
"uiComponent": "Spacer"
}
],
"uiComponent": "Block"
},
{
"uiComponent": "Spacer"
}
],
"uiComponent": "Container"
}
and I would like to generate the following output:
[
{
"id": "question_1",
"title": "title1",
"description": "Description1",
"type": "radioGroup",
"questions": "radio,Yes,1"
},
{
"id": "question_1",
"title": "title1",
"description": "Description1",
"type": "radioGroup",
"questions": "radio,No,0"
},
{
"id": "question_2",
"title": "Title2",
"description": "Description2",
"type": "radioGroup",
"questions": "radio,Label - Value 1,1"
},
{
"id": "question_2",
"title": "Title2",
"description": "Description2",
"type": "radioGroup",
"questions": "radio,Label - Value 2,2"
},
{
"id": "question_2",
"title": "Title2",
"description": "Description2",
"type": "radioGroup",
"questions": "radio,Label - Value 3,3"
},
{
"id": "question_2_b",
"title": "",
"description": "Description3",
"type": "radioGroup",
"questions": "radio,Label - Value 1,1"
},
{
"id": "question_2_b",
"title": "",
"description": "Description3",
"type": "radioGroup",
"questions": "radio,Label - Value 2,2"
},
{
"id": "question_2_b",
"title": "",
"description": "Description3",
"type": "radioGroup",
"questions": "radio,Label - Value 3,3"
},
{
"id": "question_3",
"title": "Title3",
"description": "Description3",
"type": "radioGroup",
"questions": "radio,Yes,1"
},
{
"id": "question_3",
"title": "Title3",
"description": "Description3",
"type": "radioGroup",
"questions": "radio,No,0"
}
]
or in alternative a reduced version:
[
"question_1",
"title1",
"Description1",
"radioGroup",
"radio,Yes,1",
"radio,No,0"
],
[
"question_2",
"title2",
"Description2",
"radioGroup",
"radio,Label - Value 1,1",
"radio,Label - Value 2,2",
"radio,Label - Value 3,3",
],
[
"question_2_b",
"Description3",
"radioGroup",
"radio,Label - Value 1,1",
"radio,Label - Value 2,2",
"radio,Label - Value 3,3",
],
[
"question_3",
"Title3",
"Description3",
"radioGroup",
"radio,Yes,1",
"radio,No,0"
]
The objective is to get only the objects that contain the id (to remove the {"uiComponent": "Spacer"} objects) and get only the subFields with these tags inside the array:
"subFields": [
{
"type": "xxxx",
"label": "xxxx",
"value": xxxx
},
I was able to flatten the JSON array by using the following JQ pattern:
jq play 1
.subFields[] | select(has("id") and .id != null)| {id: .id, type: .type, description: .description, anwers: .subFields}
and generated this result:
{
"id": "question_1",
"type": "radioGroup",
"description": "Description1",
"anwers": [
{
"type": "radio",
"label": "Yes",
"value": 1
},
{
"type": "radio",
"label": "No",
"value": 0
},
{
"uiComponent": "SmallContent",
"componentProps": {
"text": "* If the answer to the above question is “Yes”, please contact the Support immediately."
}
}
]
}
{
"id": "question_2",
"type": "radioGroup",
"description": "Description2",
"anwers": [
{
"type": "radio",
"label": "Label - Value 1",
"value": 1
},
{
"type": "radio",
"label": "Label - Value 2",
"value": 2
},
{
"type": "radio",
"label": "Label - Value 3",
"value": 3
},
{
"type": "radio",
"label": "Other",
"value": 13,
"subFields": [
{
"id": "question_2a",
"type": "string",
"condition": {
"type": "BinaryExpression",
"operator": "==",
"left": {
"type": "Identifier",
"name": "question_2"
},
"right": {
"type": "Literal",
"value": 13
}
}
}
]
}
]
}
{
"id": "question_2_b",
"type": "radioGroup",
"description": "Description3",
"anwers": [
{
"type": "radio",
"label": "Label - Radio 1",
"value": 1
},
{
"type": "radio",
"label": "Label - Radio 2",
"value": 2
},
{
"type": "radio",
"label": "Label - Radio 3",
"value": 3
}
]
}
{
"id": "question_3",
"type": "radioGroup",
"description": "Description3",
"anwers": [
{
"type": "radio",
"label": "Yes",
"value": 1
},
{
"type": "radio",
"label": "No",
"value": 0
}
]
}
My problem is that I don't know how to remove these sections:
{
"uiComponent": "SmallContent",
"componentProps": {
"text": "* If the answer to the above question is “Yes”, please contact the Support immediately."
}
}
and
"subFields": [
{
"id": "question_2a",
"type": "string",
"condition": {
"type": "BinaryExpression",
"operator": "==",
"left": {
"type": "Identifier",
"name": "question_2"
},
"right": {
"type": "Literal",
"value": 13
}
}
}
]
I played a little bit arround with this jq for the question_3 only:
jq play 2
.subFields[] | {id: .id, title: .title, description: .description, type: .type, subFields: .subFields} | select(has("id") and .id != null) | select(.id=="question_3") | {id: .id, title: .title, description: .description, type: .type, questions: (.subFields[]|join(","))}
and produced this result:
{
"id": "question_3",
"title": "Title3",
"description": "Description3",
"type": "radioGroup",
"questions": "radio,Yes,1"
}
{
"id": "question_3",
"title": "Title3",
"description": "Description3",
"type": "radioGroup",
"questions": "radio,No,0"
}
and also
jq play 3
.subFields[] | {id: .id, title: .title, description: .description, type: .type, subFields: .subFields} | select(has("id") and .id != null) | select(.id=="question_3") | [.id, .title, .description, .type, (.subFields[]|join(","))]
resulting on this:
[
"question_3",
"Title3",
"Description3",
"radioGroup",
"radio,Yes,1",
"radio,No,0"
]
Can you help me improve those JQ pattern I created to get the intended results?
Thanks in advance!
The following produces your first alternative:
.subFields[]
| select(.id?)
| { id, title, description, type} +
(.subFields[]
| select(.type?)
| [.type,.label,.value] | join(",")
| { questions: .} )
Notice the two select() filters.
The key names are specified explicitly here to ensure the ordering you specified is honored.

papal plus malformed_request JSON with shipping address

To show the Paypal plus iFrame (REST API) i make a Request with JSON
{
"intent": "sale",
"experience_profile_id": "XP-XXXX-XXXX-XXX-XXX",
"redirect_urls": {
"return_url": "https://www.XXXXXXX.de/bestellen.php",
"cancel_url": "https://www.XXXXXXX.de/zahlungabbruch.php"
},
"payer": {
"payment_method": "paypal"
},
"transactions": [{
"amount": {
"total": "53.45",
"currency": "EUR",
"details": {
"subtotal": "49.5",
"shipping": "3.95"
}
},
"description": "Tollewolle",
"invoice_number": "",
"item_list": {
"items": [
{
"quantity": "4",
"name": "Fine Kid - 50",
"price": "8.25",
"currency": "EUR",
"sku": "8-50"
},
{
"quantity": "2",
"name": "Fine Kid - 208",
"price": "8.25",
"currency": "EUR",
"sku": "8-208"
}
]
},
"shipping_address": {
"line1": "Rechnungs Str. 41",
"city": "Flensburg",
"postal_code": "24939",
"country_code": "DE"
}
}]
}
Without the shipping_address it works fine.
With the address i get an Error 'MALFORMED_REQUEST'
I believe Shipping address is a child ob lineitems so you need to move it up a level. E.g.
"transactions": [{"amount": {"total": "53.45", "currency":
"EUR","details":{"subtotal": "49.5", "shipping":
"3.95"}},"description": "Tollewolle", "invoice_number": "",
"item_list": {"items": [{"quantity": "4", "name": "Fine Kid - 50",
"price": "8.25", "currency": "EUR", "sku": "8-50"},{"quantity": "2",
"name": "Fine Kid - 208","price": "8.25", "currency": "EUR", "sku":
"8-208"}],"shipping_address": {"line1": "Rechnungs Str. 41","city":
"Flensburg", "postal_code": "24939", "country_code": "DE"}}}]

How to extract specific data from JSON file

I have this database http://ashleyw.co.uk/project/food-nutrient-database, and I would like to extract the group Dairy and Egg Products into a separate Json file.
Also, within the group some of the data is triplicated. For example:
{
"id": 1008,
"description": "Cheese, caraway",
"tags": [],
"manufacturer": "",
"group": "Dairy and Egg Products",
"portions": [
{
"amount": 1,
"unit": "oz",
"grams": 28.35
}
],
"nutrients": [
{
"value": 25.18,
"units": "g",
"description": "Protein",
"group": "Composition"
},
{
"value": 29.2,
"units": "g",
"description": "Total lipid (fat)",
"group": "Composition"
},
{
"value": 3.06,
"units": "g",
"description": "Carbohydrate, by difference",
"group": "Composition"
},
{
"value": 25.18,
"units": "g",
"description": "Protein",
"group": "Composition"
},
{
"value": 29.2,
"units": "g",
"description": "Total lipid (fat)",
"group": "Composition"
},
{
"value": 3.06,
"units": "g",
"description": "Carbohydrate, by difference",
"group": "Composition"
},
{
"value": 25.18,
"units": "g",
"description": "Protein",
"group": "Composition"
},
{
"value": 29.2,
"units": "g",
"description": "Total lipid (fat)",
"group": "Composition"
},
{
"value": 3.06,
"units": "g",
"description": "Carbohydrate, by difference",
"group": "Composition"
}
]
}
Every child of the nutrients node is triplicated. How can the extras be stripped away?
Using Python:
import json
with open("data.json") as f, open("nutrients.json", "w") as g:
d = json.loads(f.read())
json.dump(d["nutrients"], g)

jquery object as parameter to function

This is my jquery response:
[
{ "depot":
{
"id": "D1",
"intersection": {
"first": "Markham",
"second": "Lawrence"
},
"address": {
"number": "25",
"street": "Cougar Court",
"city": "Scarborough",
"province": "ON",
"postal_code": "M1J"
}
},
"vehicle": [
{
"id": "V1",
"depot_id": "D1",
"model": "Ford Focus",
"price": "45",
"km_per_litre": "15",
"cargo_cu_m": "YES",
"category": "Compact car",
"image": "www.coolcarz.com"
}
,
{
"id": "V2",
"depot_id": "D1",
"model": "Honda Civic",
"price": "45",
"km_per_litre": "150",
"cargo_cu_m": "YES",
"category": "Compact car",
"image": "www.coolcarz.com"
}
,
{
"id": "V8",
"depot_id": "D1",
"model": "Pontiac Aztek",
"price": "10",
"km_per_litre": "6",
"cargo_cu_m": "YES",
"category": "SUV",
"image": "www.nocoolcarz.com"
}
,
{
"id": "V12",
"depot_id": "D1",
"model": "Chevrolet Impala",
"price": "45",
"km_per_litre": "12",
"cargo_cu_m": "YES",
"category": "Standard car",
"image": "www.coolcarz.com"
}
,
{
"id": "V29",
"depot_id": "D1",
"model": "Nissan Leaf",
"price": "150",
"km_per_litre": "0",
"cargo_cu_m": "YES",
"category": "Electronic Car",
"image": "www.coolcarz.com"
}
]
}
,
{ "depot":
{
"id": "A1",
"intersection": {
"first": "Markham",
"second": "Lawrence"
},
"address": {
"number": "25",
"street": "Cougar Court",
"city": "Scarborough",
"province": "ON",
"postal_code": "m1J"
}
},
"vehicle": [
]
}
]
What I want to do is that at some point in my code, when I have received this response data, I want to pass , say data[0] or or data[0].vehicle[1] to a function
The way I am doing it now is:
function(data) {
var items = [];
for( i=0; i<data.length; i++){
items.push('<b>' + data[i].depot.intersection.first+"-"+ data[i].depot.intersection.second + " depot has following cars:"+ '</b>');
for( k=0; k<data[i].vehicle.length;k++){
str = '<li> ' + data[i].vehicle[k].category +", $"+ data[i].vehicle[k].price +' a day </li>';
items.push(str);
in effect I am trying to create a hyperlink (with vehicle category as text), and when user clicks on this hyperlink, I want to pass on the vehicle information array to a new function called moreInfo that does its job. Rit now when I do this and click the hyperlink, I see the error
missing ] after element list
timepass([object Object],[object Object]
Any ideas?
Remove 2 square brackets from beginning and end of your json code and use use it as an object not as an array.
What you did was creating an array of objects not JSON object