I have a JSON object that looks something like this:
{
"a": [{
"name": "x",
"group": [{
"name": "tom",
"publish": true
},{
"name": "joe",
"publish": true
}]
}, {
"name": "y",
"group": [{
"name": "tom",
"publish": false
},{
"name": "joe",
"publish": true
}]
}]
}
I want to select all the entries where publish=true and create a simplified JSON array of objects like this:
[
{
"name": "x"
"groupName": "tom"
},
{
"name": "x"
"groupName": "joe"
},
{
"name": "y"
"groupName": "joe"
}
]
I've tried many combinations but the fact that group is an array seems to prevent each from working. Both in this specific case as well as in general, how do you do a deep select without loosing the full hierarchy?
Using <expression> as $varname lets you store a value in a variable before going deeper into the hierarchy.
jq -n '
[inputs[][]
| .name as $group
| .group[]
| select(.publish == true)
| {name, groupName: $group}
]' <input.json
You can use this:
jq '.[]|map(
.name as $n
| .group[]
| select(.publish==true)
| {name:$n,groupname:.name}
)' file.json
A shorter, effective alternative:
.a | map({name, groupname: (.group[] | select(.publish) .name)})
Online demo
Related
I have this json that i am trying to get the just the id out of based on a contains from another value. I am able to jq the contains part but when I add on | .id i cannot get a result
{
"restrictions": [
{
"id": 1,
"database": {
"match": "exact",
"value": "db_contoso"
},
"measurement": {},
"permissions": [
"write"
]
},
{
"id": 2,
"database": {
"match": "exact",
"value": "db2_contoso"
},
"measurement": {},
"permissions": [
"write"
]
}
]
}
When id run
jq -r '.restrictions[] | .database.value | select(contains("conto")?)
I get the values of db_contoso and db2_contoso. but I am trying to pull just the id based on that. When I add | .id to the end of that command I get nothing.
So that would be to do below. Select the whole object matching the condition and get the value of .id
jq '.restrictions[] | select(.database.value | contains("conto")).id'
I have two JSON files.
file1.json:
{
"Fruits": [
{
"name": "Apple",
"something_else": 123,
"id": 652090
},
{
"name": "Orange",
"something_else": 456,
"id": 28748
}
]}
file2.json:
{
"Fruits": [
{
"weight": 5,
"id": 652090
},
{
"weight": 7,
"id": 28748
}
]}
I want to combine objects from both files if they have a common key 'id', but to extract only 'name' property from file1. How do I do that using jq?
This is what I want to get:
{
"Fruits": [
{
"name": "Apple",
"weight": 5,
"id": 652090
},
{
"name": "Orange",
"weight": 7,
"id": 28748
},
]}
Combine Fruits arrays, group it by id, select groups with 2 elements because we want fruits present in both files. For each selected group; add name field from first group element to second, and collect results in an array.
jq -n '[inputs.Fruits[]]
| reduce (group_by(.id)[] | select(length==2)) as $f
([]; . + [$f[1] + ($f[0] | {name})])' file1.json file2.json
Note that the order files are given on the command line is important, the file with names should be given before the other.
Combining objects with same id and extracting a subset of fields is way much easier though:
jq -n '[inputs.Fruits[]]
| group_by(.id)
| map(select(length==2) | add | {name, id, weight})
' file1.json file2.json
There's plenty of ways this could be constructed. Here's another way:
$ jq '.Fruits |= (. + input.Fruits | [group_by(.id)[] | add | {name,weight,id}])' \
file1.json file2.json
{
"Fruits": [
{
"name": "Orange",
"weight": 7,
"id": 28748
},
{
"name": "Apple",
"weight": 5,
"id": 652090
}
]
}
First time user of jq and I'm wanting to filter out objects based on a value within them and I'm struggling to figure it out.
I have a big json file with lots of product data like what's below. I'm wanting to filter out based upon which website_ids they have.
Example Input:
[{
"product_id": "2",
"sku": "PROD2",
"name": "Product Name 2",
"set": "4",
"type": "simple",
"category_ids": {
"item": "15"
},
"website_ids": {
"item": [
"1",
"4"
]}
},{
"product_id": "3",
"sku": "PROD3",
"name": "Product Name 3",
"set": "4",
"type": "simple",
"category_ids": {
"item": "15"
},
"website_ids": {
"item": [
"1",
"2"
]}
}]
Desired output:
[{
"product_id": "2",
"sku": "PROD2",
"name": "Product Name 2",
"set": "4",
"type": "simple",
"category_ids": {
"item": "15"
},
"website_ids": {
"item": [
"1",
"4"
]}
}]
I've tried a few different things but I'm clearly just not getting it.
jq 'map(.website_ids.item[] | contains("4"))'
Gives me:
[
false,
true,
false,
false
]
Which seems to match the website_ids items I want, but I'm not sure how to get the full JSON object from that.
Any help would be super appreciated! Thanks.
EDIT:
I've used this and it works with my example:
map(select(.website_ids.item[] | contains("4")))
What I've realised is that my example and the file I was actually testing on have some differences.
Sometimes a product has this for the website_id items:
"website_ids": {
"item": "2"
}
Which results in the error:
Cannot iterate over string ("2")
Is there a way around this?
All you need to do is add a select call in your map function, like so:
jq 'map(select(.website_ids.item[] | contains("4")))'
After your edit, it's a bit more complicated, but it can be worked around by checking the type of .website_ids.item and then based off of that type, doing a contains check or a simple equality check:
map((select((.website_ids.item | type) == "array") | select(.website_ids.item[] | contains("4"))), (select((.website_ids.item | type) == "string") | select (.website_ids.item == "4")))
Here it is formatted a bit more readable:
map(
(select((.website_ids.item | type) == "array") | select(.website_ids.item[] | contains("4"))),
(select((.website_ids.item | type) == "string") | select (.website_ids.item == "4"))
)
I have two questions:
How can I use jq to search for "name" fields that start with an underscore (like _RDS_PASSWORD) and remove the leading underscore (so it becomes RDS_PASSWORD)
How can I use jq for "name" fields that start with an underscore (like _RDS_PASSWORD) and pass the value of the value cGFzc3dvcmQK to be decoded via base64? (ex: "cGFzc3dvcmQK" | base64 --decode)
Input:
[
{
"name": "RDS_DB_NAME",
"value": "rds_db_name"
},
{
"name": "RDS_HOSTNAME",
"value": "rds_hostname"
},
{
"name": "RDS_PORT",
"value": "1234"
},
{
"name": "RDS_USERNAME",
"value": "rds_username"
},
{
"name": "_RDS_PASSWORD",
"value": "cGFzc3dvcmQK"
}
]
Desired output:
[
{
"name": "RDS_DB_NAME",
"value": "rds_db_name"
},
{
"name": "RDS_HOSTNAME",
"value": "rds_hostname"
},
{
"name": "RDS_PORT",
"value": "1234"
},
{
"name": "RDS_USERNAME",
"value": "rds_username"
},
{
"name": "RDS_PASSWORD",
"value": "password"
}
]
Q1
walk( if type=="object" and has("name") and .name[0:1] == "_"
then .name |= .[1:]
else .
end)
If your jq does not have walk/1 then you can either upgrade to a more recent version of jq than 1.5, or include its def, which can be found at https://github.com/stedolan/jq/blob/master/src/builtin.jq
Q2
.. | objects | select(has("name") and .name[0:1] == "_") | .value
If you are certain that the encoded string was a UTF-8 string, you could use jq's #base64d; otherwise, invoke jq with the -r option and pipe the results to a decoder as you indicated you planned to do.
jq strikes again. Trying to get the value of DATABASES_DEFAULT based on the name in a json file that has a whole lot of names and I'm completely lost.
My file looks like the following (output of an aws ecs describe-task-definition) only much more complex; I've stripped this to the most basic example I can where the structure is still intact.
{
"taskDefinition": {
"status": "bar",
"family": "bar2",
"volumes": [],
"taskDefinitionArn": "bar3",
"containerDefinitions": [
{
"dnsSearchDomains": [],
"environment": [
{
"name": "bar4",
"value": "bar5"
},
{
"name": "bar6",
"value": "bar7"
},
{
"name": "DATABASES_DEFAULT",
"value": "foo"
}
],
"name": "baz",
"links": []
},
{
"dnsSearchDomains": [],
"environment": [
{
"name": "bar4",
"value": "bar5"
},
{
"name": "bar6",
"value": "bar7"
},
{
"name": "DATABASES_DEFAULT",
"value": "foo2"
}
],
"name": "boo",
"links": []
}
],
"revision": 1
}
}
I need the value of DATABASES_DEFAULT where the name is baz. Note that there are a lot of keypairs with name, I'm specifically talking about the one outside of environment.
I've been tinkering with this but only got this far before realizing that I don't understand how to access nested values.
jq '.[] | select(.name==DATABASES_DEFAULT) | .value'
which is returning
jq: error: DATABASES_DEFAULT/0 is not defined at <top-level>, line 1:
.[] | select(.name==DATABASES_DEFAULT) | .value
jq: 1 compile error
Obviously this a) doesn't work, and b) even if it did, it's independant of the name value. My thought was to return all the db defaults and then identify the one with baz, but I don't know if that's the right approach.
I like to think of it as digging down into the structure, so first you open the outer layers:
.taskDefinition.containerDefinitions[]
Now select the one you want:
select(.name =="baz")
Open the inner structure:
.environment[]
Select the desired object:
select(.name == "DATABASES_DEFAULT")
Choose the key you want:
.value
Taken together:
parse.jq
.taskDefinition.containerDefinitions[] |
select(.name =="baz") |
.environment[] |
select(.name == "DATABASES_DEFAULT") |
.value
Run it like this:
<infile jq -f parse.jq
Output:
"foo"
The following seems to work:
.taskDefinition.containerDefinitions[] |
select(
select(
.environment[] | .name == "DATABASES_DEFAULT"
).name == "baz"
)
The output is the object with the name key mapped to "baz".
$ jq '.taskDefinition.containerDefinitions[] | select(select(.environment[]|.name == "DATABASES_DEFAULT").name=="baz")' tmp.json
{
"dnsSearchDomains": [],
"environment": [
{
"name": "bar4",
"value": "bar5"
},
{
"name": "bar6",
"value": "bar7"
},
{
"name": "DATABASES_DEFAULT",
"value": "foo"
}
],
"name": "baz",
"links": []
}