Trying to learn how to think in jq script.
Given this data:
{
"characters": [
{ "First": "Fred", "Last": "Weasley" },
{ "First": "George", "Last": "Weasley" },
{ "First": "Hermione", "Last": "Granger" },
{ "First": "Ron", "Last": "Weasley" },
{ "First": "Hagrid" },
{ "First": "Draco", "Last": "Malfoy" },
{ "First": "Molly", "Last": "Weasley" },
{ "First": "Voldemort" },
{ "First": "Lucius", "Last": "Malfoy" }
]
}
Find all characters with the same last name as "Ron". And no, you don't already know his last name.
Find the last name you are looking for and save it in a variable, then update the characters array accordingly:
jq '
(.characters[] | select(.First == "Ron").Last) as $last
| .characters |= map(select(.Last == $last))
'
{
"characters": [
{
"First": "Fred",
"Last": "Weasley"
},
{
"First": "George",
"Last": "Weasley"
},
{
"First": "Ron",
"Last": "Weasley"
},
{
"First": "Molly",
"Last": "Weasley"
}
]
}
Demo
If you want to make the call with a dynamic first name, provide it using a parameter variable:
jq --arg first "Ron" '
(.characters[] | select(.First == $first).Last) as $last
| .characters |= map(select(.Last == $last))
'
You have an array of objects called "characters" each containing a "first" and "last" variable.
Now to get the first one. Go characters[0].first which would return. "Fred"
Or characters.[3].last would return "Weasley"
Note the first entry in array is fetched by index 0.
Make sense?
Related
I am trying to extract values from a json that I obtained using the curl command for api testing. My json looks as below. I need some help extracting the value "20456" from here?
{
"meta": {
"status": "OK",
"timestamp": "2022-09-16T14:45:55.076+0000"
},
"links": {},
"data": {
"id": 24843,
"username": "abcd",
"firstName": "abc",
"lastName": "xyz",
"email": "abc#abc.com",
"phone": "",
"title": "",
"location": "",
"licenseType": "FLOATING",
"active": true,
"uid": "u24843",
"type": "users"
}
}
{
"meta": {
"status": "OK",
"timestamp": "2022-09-16T14:45:55.282+0000",
"pageInfo": {
"startIndex": 0,
"resultCount": 1,
"totalResults": 1
}
},
"links": {
"data.createdBy": {
"type": "users",
"href": "https://abc#abc.com/rest/v1/users/{data.createdBy}"
},
"data.fields.user1": {
"type": "users",
"href": "https://abc#abc.com/rest/v1/users/{data.fields.user1}"
},
"data.modifiedBy": {
"type": "users",
"href": "https://abc#abc.com/rest/v1/users/{data.modifiedBy}"
},
"data.fields.projectManager": {
"type": "users",
"href": "https://abc#abc.com/rest/v1/users/{data.fields.projectManager}"
},
"data.parent": {
"type": "projects",
"href": "https://abc#abc.com/rest/v1/projects/{data.parent}"
}
},
"data": [
{
"id": 20456,
"projectKey": "Stratus",
"parent": 20303,
"isFolder": false,
"createdDate": "2018-03-12T23:46:59.000+0000",
"modifiedDate": "2020-04-28T22:14:35.000+0000",
"createdBy": 18994,
"modifiedBy": 18865,
"fields": {
"projectManager": 18373,
"user1": 18628,
"projectKey": "Stratus",
"text1": "",
"name": "Stratus",
"description": "",
"date2": "2019-03-12",
"date1": "2018-03-12"
},
"type": "projects"
}
]
}
I have tried the following, but end up getting error:
▶ cat jqTrial.txt | jq '.data[].id'
jq: error (at <stdin>:21): Cannot index number with string "id"
20456
Also tried this but I get strings outside the object that I am not sure how to remove:
cat jqTrial.txt | jq '.data[]'
Assuming you want the project id not the user id:
jq '
.data
| if type == "object" then . else .[] end
| select(.type == "projects")
| .id
' file.json
There's probably a better way to write the 2nd expression
Indeed, thanks to #pmf
.data | objects // arrays[] | select(.type == "projects").id
Your input consists of two JSON documents; both have a data field on top level. But while the first one is itself an object which has an .id field, the second one is an array with one object item, which also has an .id field.
To retrieve both, you could use the --slurp (or -s) option which wraps both top-level objects into an array, then you can address them separately by index:
jq --slurp '.[0].data.id, .[1].data[].id' jqTrial.txt
24843
20456
Demo
I am new to jq and I'm trying to use it to search for a value in a json file based on a key that is located deep in the json structure. Here is a sample of my json file:
{
"data": {
"inventory": {
"location": "remote",
"list": {
"content": [
{
"item": {
"name": "minivan"
},
"owner": {
"id": "12345",
"state": "CA"
}
},
{
"item": {
"name": "sedan"
},
"owner": {
"id": "67890",
"state": "AZ"
}
}
]
}
}
}
}
An example of search that I'm trying to do is:
select item.name where owner.id = "67890"
and the expected output would be:
item.name = "sedan"
I'm trying to run the following:
jq '.[] | select .owner.id = "67890" | .item.name' json
and it generates an error:
jq: error: select/0 is not defined at <top-level>, line 1:
.[] | select .owner.id = "67890" | .item.name
jq: 1 compile error
Any pointers on how to do this in jq would be much appreciated!
Thanks!
First, you have to "navigate" to where you want to make the query. This seems to be an array.
.data.inventory.list.content
[
{
"item": {
"name": "minivan"
},
"owner": {
"id": "12345",
"state": "CA"
}
},
{
"item": {
"name": "sedan"
},
"owner": {
"id": "67890",
"state": "AZ"
}
}
]
Demo
Next, let's iterate over that array's items, which gives us a stream of objects.
.[]
{
"item": {
"name": "minivan"
},
"owner": {
"id": "12345",
"state": "CA"
}
}
{
"item": {
"name": "sedan"
},
"owner": {
"id": "67890",
"state": "AZ"
}
}
Demo
From these objects we select those that match your criteria.
select(.owner.id == "67890")
{
"item": {
"name": "sedan"
},
"owner": {
"id": "67890",
"state": "AZ"
}
}
Demo
Finally, we extract the value you're interested in.
.item.name
"sedan"
Demo
Everything combined in a jq call would be:
jq '.data.inventory.list.content[] | select(.owner.id == "67890").item.name'
"sedan"
Demo
This output is still valid JSON document (containing nothing but a JSON string). If you want to process the output as raw text, use the --raw-output (or -r) option:
jq -r '.data.inventory.list.content[] | select(.owner.id == "67890").item.name'
sedan
Demo
Here's a solution that avoids having to "navigate" to the right place, and which is also quite close to your SQL-like query:
..
| objects
| select(.owner and
(.owner|type=="object" and .id == "67890"))
.item.name
or more succinctly:
..|objects|select(.owner.id? == "67890").item.name
Have a JSON as below,
[
{
"first": "foo",
"last": "bar",
"roll": "32",
"subject": "maths"
},
{
"first": "joe",
"last": "mighty",
"roll": "31",
"subject": "english"
},
{
"first": "foo",
"last": "bar",
"roll": "32",
"subject": "english"
},
{
"first": "joe",
"last": "mighty",
"roll": "31",
"subject": "maths"
},
{
"first": "foo",
"last": "bar",
"roll": "32",
"subject": "science"
}
]
unique_by(.first,.last, .roll) will remove other subject entries. I want convert 'subject' to an array with all the values in the source JSON array. How to map using jq for creating nested array of subject as below,
[
{
"first": "foo",
"last": "bar",
"roll": "32",
"subject": ["maths", "english", "science"]
},
{
"first": "joe",
"last": "mighty",
"roll": "31",
"subject": ["english", "maths"]
}
]
JQ doesn't have a nice merge built-in for such tasks yet, but you can achieve this one using group_by.
group_by(.first, .last, .roll) | map(
(.[0] | {first, last, roll}) + {subject: map(.subject)}
)
Online demo
A variation based on Oguz's excellent answer:
group_by(.first, .last, .roll)
| map( map(.subject) as $subject
| .[0]
| {first, last, roll, $subject})
Collect related items (group by first, last and roll) and reduce each collection into a single item:
First we put all subjects into $subject
Take the first item as a "template" but rewrite the .subject key with $subject.
I'm trying to understand what's the best way to add a json element to child's parent
if that child contains a specific key:value and finally print the entire json using jq
I try to explain better with an example.
The input json is:
{
"family": {
"surname": "Smith"
},
"components": [
{
"name": "John",
"details": {
"hair": "brown",
"eyes": "brown",
"age": "56"
},
"role": "father"
},
{
"name": "Mary",
"details": {
"hair": "blonde",
"eyes": "green",
"age": "45"
},
"role": "mother"
},
{
"name": "George",
"details": {
"hair": "blonde",
"eyes": "brown",
"age": "25"
},
"role": "child"
}
]
}
I want to add:
"description": "5 years less than 30"
at the same level of "details" if "age" is equal to "25" and then print the result:
{
"family": {
"surname": "Smith"
},
"components": [
{
"name": "John",
"details": {
"hair": "brown",
"eyes": "brown",
"age": "56"
},
"role": "father"
},
{
"name": "Mary",
"details": {
"hair": "blonde",
"eyes": "green",
"age": "45"
},
"role": "mother"
},
{
"name": "George",
"details": {
"hair": "blonde",
"eyes": "brown",
"age": "25"
},
"role": "child",
"description": "5 years less than 30"
}
]
}
The only solution I've found was to apply the update but printing only the "components" content;
then I've removed from the JSON and finally inserted the modified "components" content previously saved, in this way:
cat sample.json | jq -c ' .components[] | select(.details.age=="25") |= . + {description: "5 years less than 30" } ' > /tmp/saved-components.tmp
cat sample.json | jq --slurpfile savedcomponents /tmp/saved-components.tmp 'del(.components) | . + { components: [ $savedcomponents ] }'
I don't think it's the best way to solve these kind of problems, so I'd like to know what is
the right "jq approach".
I forgot to say: I prefer to use jq only, not other tools
Than you
Marco
You can select the object matching the condition and append to that object. Something like below. The key is to use += the modification assignment to not lose the other objects
(.components[] | select(.details.age == "25")) += { "description": "5 years less than 30" }
jqplay - Demo
Here's a straightforward ("no magic") and efficient solution:
.components |=
map(if .details.age=="25" then .description = "5 years less than 30" else . end)
I want to filter and assign a value from array based on a condition, and use default in case if array does not have the matched object.
Here is a sample object:
{
"array" : [
{
"id": "A",
"conversations": [
{
"conversation": "1",
"type": "good"
},
{
"conversation": "2",
"type": "bad"
}
]
},
{
"id": "B",
"conversations": [
{
"conversation": "3",
"type": "good"
},
{
"conversation": "4",
"type": "bad"
}
]
},
{
"id": "C",
"conversations": [
{
"conversation": "5",
"type": "bad"
},
{
"conversation": "6",
"type": "bad"
}
]
}
]
}
Required output:
{
"id": "A",
"goodConversation": "1"
}
{
"id": "B",
"goodConversation": "3"
},
{
"id": "C",
"goodConversation": null
}
echo of my input:
echo '{"array":[{"id":"A","conversations":[{"conversation":"1","type":"good"},{"conversation":"2","type":"bad"}]},{"id":"B","conversations":[{"conversation":"3","type":"good"},{"conversation":"4","type":"bad"}]},{"id":"C","conversations":[{"conversation":"5","type":"bad"},{"conversation":"6","type":"bad"}]}]}'
I tried running following jq
jq '.array[] | {id, "goodConversation": .conversations[] | select(.type == "good") | .conversation}'
Actual output:
{
"id": "A",
"goodConversation": "1"
}
{
"id": "B",
"goodConversation": "3"
}
since the object with id: "C" does not have any good conversation the whole object gets filtered out. Is there a way to create the output object which contains "C" with null as value?
Clarification:
"conversations" will have at most one good conversation.
I am using jq 1.5
One way to provide a default value is often to use the // "alternative" operator. Building on the foundations you've laid, you could write:
.array[]
| {id,
"goodConversation":
((.conversations[]
| select(.type == "good")
| .conversation) // null) }
If there is more than one "good" conversation, however, this may not be exactly what you want. If it's not, then consider using first, e.g.:
.array[]
| {id,
"goodConversation":
( first(.conversations[]
| select(.type == "good")
| .conversation) // null)}