Reshape a array to object with jq - json

I am learning advance concepts of jq. And I made a tiny json with array with some films of Charles Chaplin...well this array in json:
[
{
"title": "The Great Dictator",
"year": 1940,
"country": "USA",
"genre": "political satire"
},
{
"title": "Modern Times ",
"year": 1936,
"country": "USA",
"genre": "comedy"
},
{
"title": "The Gold Rush",
"year": 1925,
"country": "USA",
"genre": "comedy"
},
{
"title": "The Kid",
"year": 1921,
"country": "USA",
"genre": "drama"
}
]
And I want to convert or reshape into a object with the genres as the keys and the list of the films as array (comedy is only has two element in the array):
{
"comedy": [
{
"title": "Modern Times ",
"year": 1936,
"country": "USA"
},
{
"title": "The Gold Rush",
"year": 1925,
"country": "USA"
}
],
"political satire": [
{
"title": "The Great Dictator",
"year": 1940,
"country": "USA"
}
],
"drama": [
{
"title": "The Kid",
"year": 1921,
"country": "USA"
}
]
}
But I can't do it. I trying the first step to create a object with genre and foo string as var, but it fails: cat c.json | jq '{.[] | (.genre): "foo" ]}'

It can be done in three lines:
[group_by(.genre)[]
| {(.[0].genre): map_values(del(.genre))}]
| add
aggregate_by/3
The relevant generic abstraction here is:
def aggregate_by(s; f; g):
reduce s as $x (null; .[$x|f] += [$x|g]);
This allows the solution to be written directly as:
aggregate_by(.[]; .genre; del(.genre))

I found:
$ cat c.json | jq '
group_by(.genre)
| map({"genre": .[0].genre,
"film": map(. | del(.genre))})
| [ .[] | {(.genre): .film}]
| add'
{
"comedy": [
{
"title": "Modern Times ",
"year": 1936,
"country": "USA"
},
{
"title": "The Gold Rush",
"year": 1925,
"country": "USA"
}
],
"drama": [
{
"title": "The Kid",
"year": 1921,
"country": "USA"
}
],
"political satire": [
{
"title": "The Great Dictator",
"year": 1940,
"country": "USA"
}
]
}
Maybe it is not the best, because I think there are a lot of steps...but it runs.

You can use a modified version of Jeff Mercado's answer on the page you linked to.
jq 'reduce .[] as $i ({}; .[$i.genre] += [$i])'
That groups the objects as you want but leaves the genre key-value pair. You can delete them like so.
jq 'reduce .[] as $i ({}; .[$i.genre] += [$i|del(.genre)])'
Really, this is just a concrete version of peak's "generic abstraction".

Related

JQ merge WooCommerce REST API responce

Im trying to parse data from woocommerce from linux console.
Need to take only shipping info and item names with quantity.
im making curl request to wp-json/wc/v2/orders/ORDER_ID
then
jq '{order_id:.id,ship_info:.shipping,items: (.line_items[] | {name , quantity} ) }'
If order contains two items, jq will return two objects
example:
{
"order_id": 1234,
"ship_info": {
"first_name": "Sam",
"last_name": "Fisher",
"company": "",
"address_1": "24 Mega Drive",
"address_2": "",
"city": "Eglinton",
"state": "WA",
"postcode": "6032",
"country": "AU",
"phone": ""
},
"items": {
"name": "Black T-shirt",
"quantity": 1
}
}
{
"order_id": 1234,
"ship_info": {
"first_name": "Sam",
"last_name": "Fisher",
"company": "",
"address_1": "24 Mega Drive",
"address_2": "",
"city": "Eglinton",
"state": "WA",
"postcode": "6032",
"country": "AU",
"phone": ""
},
"items": {
"name": "White T-shirt",
"quantity": 1
}
}
I want merge items and use item's name as a key and item's qty as a value. Please advice how to get output like this
{
"order_id": 1234,
"ship_info": {
"first_name": "Sam",
"last_name": "Fisher",
"company": "",
"address_1": "24 Mega Drive",
"address_2": "",
"city": "Eglinton",
"state": "WA",
"postcode": "6032",
"country": "AU",
"phone": ""
},
"items": {
"White T-shirt": "1",
"Black T-shirt": "1"
}
}
With your current jq query you are iterating over the items inside a generated object. That's why you receive one object per item. Rather than merging them afterwards, don't separate them in the first place.
If you changed your query from
jq '{
order_id: .id,
ship_info: .shipping,
items: (.line_items[] | {name, quantity})
}'
to just
jq '{
order_id: .id,
ship_info: .shipping,
items: .line_items
}'
you'd probably already see that .line_items is actually an array.
To transform it according to your desired output, change that line to one of the followings. They should all yield the same result.
items: (.line_items | map({(.name): .quantity}) | add)
items: (INDEX(.line_items[]; .name) | map_values(.quantity))
items: (reduce .line_items[] as $i ({}; .[$i.name] = .quantity))

Need to find key-value pair and replace key-value pair in JSON using JQ

I have this JSON
{
"firstName": "Rajesh",
"lastName": "Kumar",
"gender": "man",
"age": 24,
"address": {
"streetAddress": "126 Udhna",
"city": "Surat",
"state": "WB",
"postalCode": "394221"
},
"phoneNumbers": [
{
"type": "home",
"number": "7383627627"
}
]
}
I need to find the value of the "state" key Using JQ and replace the value in JSON. I do not want to fetch it by providing the position of the key, Like
firstName=$(cat sample-json.json | jq -r '.firstName')
My expected output
{
"firstName": "Rajesh",
"lastName": "Kumar",
"gender": "man",
"age": 24,
"address": {
"streetAddress": "126 Udhna",
"city": "Surat",
"state": "Bihar",
"postalCode": "394221"
},
"phoneNumbers": [
{
"type": "home",
"number": "7383627627"
}
]
}
If you're willing to specify .address:
jq '.address.state = "Bihar"' sample-json.json
Otherwise:
jq 'walk(if type == "object" and has("state") then .state = "Bihar" else . end)' sample-json.json
This last will replace all .state values. If you only want to replace the first occurrence:
jq 'first(..|objects|select(has("state"))).state = "Bihar"' sample-json.json
And so on. It would really help all concerned if you could make the requirements clear.

Add key value to parent subelement if child has specific key:value

I'm trying to understand what's the best way to add a json element to child's parent
if that child contains a specific key:value and finally print the entire json using jq
I try to explain better with an example.
The input json is:
{
"family": {
"surname": "Smith"
},
"components": [
{
"name": "John",
"details": {
"hair": "brown",
"eyes": "brown",
"age": "56"
},
"role": "father"
},
{
"name": "Mary",
"details": {
"hair": "blonde",
"eyes": "green",
"age": "45"
},
"role": "mother"
},
{
"name": "George",
"details": {
"hair": "blonde",
"eyes": "brown",
"age": "25"
},
"role": "child"
}
]
}
I want to add:
"description": "5 years less than 30"
at the same level of "details" if "age" is equal to "25" and then print the result:
{
"family": {
"surname": "Smith"
},
"components": [
{
"name": "John",
"details": {
"hair": "brown",
"eyes": "brown",
"age": "56"
},
"role": "father"
},
{
"name": "Mary",
"details": {
"hair": "blonde",
"eyes": "green",
"age": "45"
},
"role": "mother"
},
{
"name": "George",
"details": {
"hair": "blonde",
"eyes": "brown",
"age": "25"
},
"role": "child",
"description": "5 years less than 30"
}
]
}
The only solution I've found was to apply the update but printing only the "components" content;
then I've removed from the JSON and finally inserted the modified "components" content previously saved, in this way:
cat sample.json | jq -c ' .components[] | select(.details.age=="25") |= . + {description: "5 years less than 30" } ' > /tmp/saved-components.tmp
cat sample.json | jq --slurpfile savedcomponents /tmp/saved-components.tmp 'del(.components) | . + { components: [ $savedcomponents ] }'
I don't think it's the best way to solve these kind of problems, so I'd like to know what is
the right "jq approach".
I forgot to say: I prefer to use jq only, not other tools
Than you
Marco
You can select the object matching the condition and append to that object. Something like below. The key is to use += the modification assignment to not lose the other objects
(.components[] | select(.details.age == "25")) += { "description": "5 years less than 30" }
jqplay - Demo
Here's a straightforward ("no magic") and efficient solution:
.components |=
map(if .details.age=="25" then .description = "5 years less than 30" else . end)

Using jq to filter dataset from The Mathematics Genealogy Project

With this Json Data set.
{
"nodes": [
{
"students": [],
"advisors": [
258
],
"name": "Archie Higdon",
"school": "Iowa State University",
"subject": "74—Mechanics of deformable solids",
"thesis": "Stresses in Moderately Thick Rectangular Plates",
"country": "UnitedStates",
"year": 1936,
"id": 2
},
{
"students": [],
"advisors": [
258
],
"name": "Donald Hill Rock",
"school": "Iowa State University",
"subject": null,
"thesis": "Finite Strain Analysis in Elastic Theory",
"country": "UnitedStates",
"year": 1939,
"id": 3
},
{
"students": [],
"advisors": [
258
],
"name": "William B. Stiles",
"school": "Iowa State University",
"subject": null,
"thesis": "Solutions of Clamped Plated Problems by Means of Functions Derived from Membrane Characteristic Functions",
"country": "UnitedStates",
"year": 1945,
"id": 6
}
]
}
I was wondering how I can use jq in order to find people who graduated from Georgia Tech with a non empty list of students? As in, I want to find all students who graduated from Georgia Tech who went on to advise students themselves.
Assuming simplified json like this which has some values matching the conditions you mention
{
"nodes": [
{ "students": [2, 3], "advisors": [], "name": "Person 1", "school": "Georgia Tech", "id": 1 },
{ "students": [], "advisors": [1], "name": "Person 2", "school": "Georgia Tech", "id": 2 },
{ "students": [], "advisors": [1], "name": "Person 3", "school": "Georgia Tech", "id": 3 }
]
}
You can prune nodes to only people at "Georgia Tech" with
.nodes |= map( select( .school == "Georgia Tech" ) )
You can prune nodes to people with non-empty students with
.nodes |= map( select( .students | length > 0 ) )
You can combine the conditions with and
.nodes |= map(
select( (.school == "Georgia Tech") and (.students | length > 0) )
)
Try it online!

Splitting nested arrays as separate entities

I have some JSON data which contains attributes and some array elements. I would like to push a given set of fields into the array elements and then separate the arrays as separate entities.
Source data looks like this
[
{
"phones": [
{
"phone": "555-555-1234",
"type": "home"
},
{
"phone": "555-555-5678",
"type": "mobile"
}
],
"email": [
{
"email": "a#b.com",
"type": "work"
},
{
"email": "x#c.com",
"type": "home"
}
],
"name": "john doe",
"year": "2012",
"city": "cupertino",
"zip": "555004"
},
{
"phones": [
{
"phone": "555-666-1234",
"type": "home"
},
{
"phone": "555-666-5678",
"type": "mobile"
}
],
"email": [
{
"email": "a#b.com",
"type": "work"
},
{
"email": "x#c.com",
"type": "home"
}
],
"name": "jane doe",
"year": "2000",
"city": "los angeles",
"zip": "555004"
}
]
I expect a result like this
{
"person": [
{
"name": "john doe",
"year": "2012",
"city": "cupertino",
"zip": "555004"
},
{
"name": "jane doe",
"year": "2000",
"city": "los angeles",
"zip": "555004"
}
],
"phones": [
{
"name": "john doe",
"year": "2012",
"phone": "555-555-1234",
"type": "home"
},
{
"name": "john doe",
"year": "2012",
"phone": "555-555-5678",
"type": "mobile"
},
{
"name": "jane doe",
"year": "2000",
"phone": "555-666-1234",
"type": "home"
},
{
"name": "jane doe",
"year": "2000",
"phone": "555-666-5678",
"type": "mobile"
}
],
"email": [
{
"name": "john doe",
"year": "2012",
"email": "a#b.com",
"type": "work"
},
{
"name": "john doe",
"year": "2012",
"email": "x#c.com",
"type": "home"
},
{
"name": "jane doe",
"year": "2000",
"email": "a#b.com",
"type": "work"
},
{
"name": "jane doe",
"year": "2000",
"email": "x#c.com",
"type": "home"
}
]
}
I have been able to get the desired result, but I can't make it work in a generic way.
experiment on jqterm
The code below achieves the job, but I would like to pass the array of columns to be injected into the child arrays, the name of the primary result and an array containing the array field names.
["phones", "email"] as $children
| ["name", "year"] as $ids
|{person: map(with_entries(
. as $data | select($children|contains([$data.key])|not)
))}
+ {"phones": split_child($children[0];$ids)}
+ {"email": split_child($children[1];$ids)}
It's a lot more easier to achieve this using multiple reduces, like:
def split_data($parent; $ids; $arr_cols):
($arr_cols | map([.])) as $p
| reduce .[] as $in ({}; .[$parent] += [$in | delpaths($p)]
| (reduce $ids[] as $k ({}; . + {($k): $in[$k]}) as $s
| reduce $arr_cols[] as $k (.; .[$k] += [$in[$k][] + $s])
);
split_data("person"; ["name", "year"]; ["phones", "email"])
Here's a straightforward solution to the generic problem (it uses reduce only once, in a helper function). To understand it, it might be helpful to see it as an abstraction of this concrete solution:
{ person: [.[] | {name, year, city, zip} ]}
+ { phones: [.[] | ({name, year} + .phones[]) ]}
+ { email: [.[] | ({name, year} + .email[]) ]}
Helper function
Let's first define a helper function for constructing an object by selecting a set of keys:
def pick($ary):
. as $in
| reduce $ary[] as $k ({};
. + {($k): $in[$k]});
split_data
Here finally is the function that takes as arguments the $parent, $ids, and columns of interest. The main complication is ensuring that the supplemental keys ("city" and "zip") are dealt with in the proper order.
def split_data($parent; $ids; $arr_cols):
(.[0]|keys_unsorted - $arr_cols - $ids) as $extra
| { ($parent): [.[] | pick($ids + $extra)] }
+ ([$arr_cols[] as $k
| {($k): [.[] | pick($ids) + .[$k][]] }] | add) ;
The invocation:
split_data("person"; ["name", "year"]; ["phones", "email"])
produces the desired result.