Does anyone know what json-query filter can be used to select Tigger's food in the sample JSON below? The JSON is a simplified stand-in for a massive and relatively complicated AWS blob.
Some background: I was rather pleased to discover that Ansible has a json-query filter. Given that I was trying to select an element from an AWS JSON blob this looked as if it was just what I needed. However I quickly ran into trouble because the AWS objects have tags and I needed to select items by tag.
I tried selector paths equivalent to Foods[Tags[(Key='For') & (Value='Tigger')]] and similar but didn't manage to get it to work. Using a standalone json-query library such as https://www.npmjs.com/package/json-query I can use the parent attribute but that does not appear to be in Ansible, quite apart from being a deviation from the core idea of json-query.
It might be better to sidestep the problem and use a jsonpath selector. jsonpath is similar to json-query and is a translation from xpath.
{ "Foods" :
[ { "Id": 456
, "Tags":
[ {"Key":"For", "Value":"Heffalump"}
, {"Key":"Purpose", "Value":"Food"}
]
}
, { "Id": 678
, "Tags":
[ {"Key":"For", "Value":"Tigger"}
, {"Key":"Purpose", "Value":"Food"}
]
}
, { "Id": 911
, "Tags":
[ {"Key":"For", "Value":"Roo"}
, {"Key":"Purpose", "Value":"Food"}
]
}
]
}
References
json_query in ansible:
http://docs.ansible.com/ansible/playbooks_filters.html#json-query-filter
json-query standalone node: https://www.npmjs.com/package/json-query
jmespath, the library ansible uses: http://jmespath.org/
json-query standalone python: https://pypi.python.org/pypi/jsonquery/ (red herring)
Do you need list of ids? If so, try:
- debug: msg="{{ lookup('file','test.json') | from_json | json_query(query) }}"
vars:
query: "Foods[].{id: Id, for: (Tags[?Key=='For'].Value)[0]} | [?for=='Tigger'].id"
First construct simple objects with necessary fields and then pipe it to a filter.
Related
I have a JSON document holding configurations with macros. I need to expand macros that reference other elements. For simplicity, consider this input document.
[
{
"first": "Tim",
"Full": "{first} {last}",
"last": "Smith"
},
{
"first": "Jane",
"Full": "{first} {last}",
"last": "Doe"
}
]
Performance is not paramount here so I don't mind blindly checking for every element occurring in every other element.
I worked out the following logic as a proof of concept. But can't figure out how to add a nested loop on $lookup to update the other with_entries value.
jq '
.[]
| . as $lookup
| with_entries(
.value=(
.value
| sub("{first}"; $lookup.first)
)
)
'
JQ processes streams of values/documents and it feels like I now need to work with two streams. Which I hoped to accomplish by using $lookup for the back reference. Now I am stuck.
SOLUTION
oguz ismail provided a great solution for the original question (Please give them a +1) that uses map_values(). It is very clear and I wanted to include it here for reference. You will note that it uses a named group (?<found_key>.*?) in the regular expression (see oniguruma's named group)
map(. as $lookup | map_values(gsub("\\{(?<found_key>.*?)\\}"; $lookup[.found_key])))
I asked how to process when there are non-string elements in the structure. Here is an example that includes an array of colors:
[
{
"first": "Tim",
"Full": "{first} {last}",
"last": "Smith",
"colors": ["red", "blue"]
}
]
oguz ismail provided a solution for this structure as well that only attempts to modify elements that are strings:
map(
. as $lookup
| (.[] | strings)
|= gsub("\\{(?<found_key>.*?)\\}"; $lookup[.found_key])
)
You can use gsub with a named capture group for expanding all the macros in a single run without hardcoding their names. And, with_entries doesn't help at all in this case; use map_values instead.
map(. as $lookup | map_values(gsub("\\{(?<found_key>.*?)\\}"; $lookup[.found_key])))
Online demo
Select objects based on value of variable in object using jq
That shows how to return values directly above the selection criteria but how would I get another object that was adjacent to a value above my selection criteria?
Given the data below, what jq invocation would return the French name of planets whose moon(s) have been spoiled? (this is a structural reproduction of the live data with which I am working -- which actually uses the word "value" in this way, so that's not helping)
{"kind":"solarsystem","name":"Sol",
"Planets": [
{ "kind":"habitable",
"names": { "english":"Earth","french":"Terre"},
"satellites" : [
{"name":"The Moon",
"parameters": [
{"name":"diameter", "intValue":"3476"},
{"name":"diameter_units", "value":"km"},
{"name":"unspoiled","value":"no"}]}]},
{"kind":"uninhabitable",
"names": {"english":"Mars","french":"Mars"},
"satellites" : [
{"name":"Phobos",
"parameters": [
{"name":"diameter", "intValue":"2200"},
{"name":"diameter_units", "value":"m"},
{"name":"unspoiled","value":"yes"}]},
{"name":"Deimos",
"parameters": [
{"name":"diameter", "intValue":"1200"},
{"name":"diameter_units", "value":"m"},
{"name":"unspoiled","value":"yes"}]}]}]}
The program below selects planets whose moons have all been spoiled. As each parameter is a name-value pair, we can use from_entries to transform the array of parameters into an object and retrieve the unspoiled status with just .unspoiled, and thus avoid another select to find the parameter we're interested in.
.Planets[] | select(.satellites | all(.parameters | from_entries .unspoiled == "no")) .names.french
If a single spoiled moon is enough, change all to any.
Online demo
And here, also a solution for the same JSON query using an alternative tool (jtc):
In the simplest form, the following will do:
bash $ <file.json jtc -w'[value]:<no>:[-5][names][french]'
"Terre"
However, that solution will return planet's french name for each of the moon, e.g., for spoiled moons it would give this:
bash $ <file.json jtc -w'[value]:<yes>:[-5][names][french]'
"Mars"
"Mars"
bash $
For the case when there're multiple moons but the name is required only once, strengthen the query like this (showcasing here spoiled moons):
bash $ <file.json jtc -w'<satellites>l:[value]:<yes>[-5][names][french]'
"Mars"
bash $
PS. I'm a deveoper of jtc unix JSON processor
PPS. the above disclaimer is required by SO.
Update:
the answer was updated based on discussion in comments with #oguzismail to enhance structural relationship between value and french labels so that other (irrelevant) possible value matches won't trigger false positives.
If, by a chance, the structural relation [-5][names] is not enough, the query then can be ultimately enhanced by inserting <unspoiled>[-1] before [value]... lexeme
Let's say I have this JSON file below:
{
"team": {
"money": 100,
},
"group": {
"money": 200,
"snack": true,
}
}
I want to select the objects which has a "snack" key including its parent. The current command I'm using is:
jq '..|objects|select(has("snack"))' json
This however, does not include the parent, which in this case is "group". How do I select the parent of the selected object as well?
Instead of using .., you could use paths. That is, you'd select the paths that lead to the items of interest, and work from there. So you'd start with:
paths(objects) as $p
| select(getpath($p)|has("snack"))
| $p
For the given input (after having been corrected), this would yield:
["group"]
So you might want to replace the $p in the last line by $p[-1], but it's not altogether clear how useful that would be. More useful would be getpath( $p[:-1] )
Suppose I have the following data:
{
"dashboards": [
{
"name": "first",
"type": "standard"
},
{
"name": "second",
"type": "custom"
}
]
}
(actually there's a lot more data than that, I am just showing what the structure of the data is)
What I am trying to do is get the first 10 dashboards of type standard.
I know I can get all the standard dashboards with:
jq '.dashboards[] | select(.type == "standard")'
But I can't figure out how to slice the resulting array...
If you want the result as an array, you could use map:
.dashboards | map(select(.type=="standard")) | .[0:10]
However, this is inefficient. For efficiency, it would be better to use limit as discussed below.
If you wanted the items as a stream, you could write:
limit(10; .dashboards[] | select(.type=="standard"))
If you want the results as an array, simply wrap the above jq expression in square brackets.
I have a JSON file containing an array of objects (test.json):
[
{
"name": "Test 1",
"id": 1
},
{
"name": "Test 2",
"id": 2
},
{
"name": "Test 3",
"id": 3
}
]
I want to extract all objects, that have a certain ID. I managed to get an object if I want just one specific ID: jq 'map(select(.id == 2 ))' test.json.
Thing is, I have a list of IDs, say 1 and 3. How do I get a list containing only those object? So in this example a list containing the objects with ID 1 and 3?
You can check the example here: https://jqplay.org/s/xQgpA4yJAz
jq 'map(select(.id | contains(1,3)))'
Man, jq is so great
https://github.com/stedolan/jq/wiki/Cookbook#filter-objects-based-on-the-contents-of-a-key
The solution using contains/1 as presented on this page could just as well be written using ==:
map(select(.id == (1,3)))
The main reason for mentioning this is that contains is full of potential surprises. (Consider, for example, what would happen if .id were string-valued.)
Unfortunately, using either == or contains as above is computationally inefficient (it is O(m*n)), though in practice it is quite fast.