I have a JSON document holding configurations with macros. I need to expand macros that reference other elements. For simplicity, consider this input document.
[
{
"first": "Tim",
"Full": "{first} {last}",
"last": "Smith"
},
{
"first": "Jane",
"Full": "{first} {last}",
"last": "Doe"
}
]
Performance is not paramount here so I don't mind blindly checking for every element occurring in every other element.
I worked out the following logic as a proof of concept. But can't figure out how to add a nested loop on $lookup to update the other with_entries value.
jq '
.[]
| . as $lookup
| with_entries(
.value=(
.value
| sub("{first}"; $lookup.first)
)
)
'
JQ processes streams of values/documents and it feels like I now need to work with two streams. Which I hoped to accomplish by using $lookup for the back reference. Now I am stuck.
SOLUTION
oguz ismail provided a great solution for the original question (Please give them a +1) that uses map_values(). It is very clear and I wanted to include it here for reference. You will note that it uses a named group (?<found_key>.*?) in the regular expression (see oniguruma's named group)
map(. as $lookup | map_values(gsub("\\{(?<found_key>.*?)\\}"; $lookup[.found_key])))
I asked how to process when there are non-string elements in the structure. Here is an example that includes an array of colors:
[
{
"first": "Tim",
"Full": "{first} {last}",
"last": "Smith",
"colors": ["red", "blue"]
}
]
oguz ismail provided a solution for this structure as well that only attempts to modify elements that are strings:
map(
. as $lookup
| (.[] | strings)
|= gsub("\\{(?<found_key>.*?)\\}"; $lookup[.found_key])
)
You can use gsub with a named capture group for expanding all the macros in a single run without hardcoding their names. And, with_entries doesn't help at all in this case; use map_values instead.
map(. as $lookup | map_values(gsub("\\{(?<found_key>.*?)\\}"; $lookup[.found_key])))
Online demo
Related
Select objects based on value of variable in object using jq
That shows how to return values directly above the selection criteria but how would I get another object that was adjacent to a value above my selection criteria?
Given the data below, what jq invocation would return the French name of planets whose moon(s) have been spoiled? (this is a structural reproduction of the live data with which I am working -- which actually uses the word "value" in this way, so that's not helping)
{"kind":"solarsystem","name":"Sol",
"Planets": [
{ "kind":"habitable",
"names": { "english":"Earth","french":"Terre"},
"satellites" : [
{"name":"The Moon",
"parameters": [
{"name":"diameter", "intValue":"3476"},
{"name":"diameter_units", "value":"km"},
{"name":"unspoiled","value":"no"}]}]},
{"kind":"uninhabitable",
"names": {"english":"Mars","french":"Mars"},
"satellites" : [
{"name":"Phobos",
"parameters": [
{"name":"diameter", "intValue":"2200"},
{"name":"diameter_units", "value":"m"},
{"name":"unspoiled","value":"yes"}]},
{"name":"Deimos",
"parameters": [
{"name":"diameter", "intValue":"1200"},
{"name":"diameter_units", "value":"m"},
{"name":"unspoiled","value":"yes"}]}]}]}
The program below selects planets whose moons have all been spoiled. As each parameter is a name-value pair, we can use from_entries to transform the array of parameters into an object and retrieve the unspoiled status with just .unspoiled, and thus avoid another select to find the parameter we're interested in.
.Planets[] | select(.satellites | all(.parameters | from_entries .unspoiled == "no")) .names.french
If a single spoiled moon is enough, change all to any.
Online demo
And here, also a solution for the same JSON query using an alternative tool (jtc):
In the simplest form, the following will do:
bash $ <file.json jtc -w'[value]:<no>:[-5][names][french]'
"Terre"
However, that solution will return planet's french name for each of the moon, e.g., for spoiled moons it would give this:
bash $ <file.json jtc -w'[value]:<yes>:[-5][names][french]'
"Mars"
"Mars"
bash $
For the case when there're multiple moons but the name is required only once, strengthen the query like this (showcasing here spoiled moons):
bash $ <file.json jtc -w'<satellites>l:[value]:<yes>[-5][names][french]'
"Mars"
bash $
PS. I'm a deveoper of jtc unix JSON processor
PPS. the above disclaimer is required by SO.
Update:
the answer was updated based on discussion in comments with #oguzismail to enhance structural relationship between value and french labels so that other (irrelevant) possible value matches won't trigger false positives.
If, by a chance, the structural relation [-5][names] is not enough, the query then can be ultimately enhanced by inserting <unspoiled>[-1] before [value]... lexeme
I have input like this:
{ "prop": ["apple", "banana"] }
{ "prop": ["kiwi", "banana"] }
{ "prop": ["cherry", "orange"] }
How do I print objects where prop contains at least one of kiwi and orange?
(The list of interesting values is longer than just 2, so I'd like to leverage the any function somehow.)
I've tried the following
jq 'select(any(.prop[] | contains(["kiwi", "orange"])))' < input.json
and various variants of the above, but can't figure out the right expressions.
The stream-oriented version of the built-in function any can be most easily used if one bears in mind its signature:
def any(generator; condition):
So we are led to:
select( any( .prop[]; . == "kiwi" or . == "orange" ))
or more succinctly:
select( any(.prop[]; IN("kiwi", "orange")))
whitelist
If the values of interest are provided as a JSON array, say $whitelist, you could tweak the above by substituting $whitelist[] for the explicit stream of values:
select( any(.prop[]; IN($whitelist[]) ))
I think you're looking for IN/2. It's implemented using any, but is far easier to grasp.
select(IN(.prop[]; "kiwi", "orange"))
Online demo
Let's say I have this JSON file below:
{
"team": {
"money": 100,
},
"group": {
"money": 200,
"snack": true,
}
}
I want to select the objects which has a "snack" key including its parent. The current command I'm using is:
jq '..|objects|select(has("snack"))' json
This however, does not include the parent, which in this case is "group". How do I select the parent of the selected object as well?
Instead of using .., you could use paths. That is, you'd select the paths that lead to the items of interest, and work from there. So you'd start with:
paths(objects) as $p
| select(getpath($p)|has("snack"))
| $p
For the given input (after having been corrected), this would yield:
["group"]
So you might want to replace the $p in the last line by $p[-1], but it's not altogether clear how useful that would be. More useful would be getpath( $p[:-1] )
Suppose I have the following data:
{
"dashboards": [
{
"name": "first",
"type": "standard"
},
{
"name": "second",
"type": "custom"
}
]
}
(actually there's a lot more data than that, I am just showing what the structure of the data is)
What I am trying to do is get the first 10 dashboards of type standard.
I know I can get all the standard dashboards with:
jq '.dashboards[] | select(.type == "standard")'
But I can't figure out how to slice the resulting array...
If you want the result as an array, you could use map:
.dashboards | map(select(.type=="standard")) | .[0:10]
However, this is inefficient. For efficiency, it would be better to use limit as discussed below.
If you wanted the items as a stream, you could write:
limit(10; .dashboards[] | select(.type=="standard"))
If you want the results as an array, simply wrap the above jq expression in square brackets.
Say I have the following JSON, stored in my variable jsonVariable.
{
"id": 1,
"details": {
"username": "jamesbrown",
"name": "James Brown"
}
}
I parse this JSON with jq using the following:
echo $jsonVariable | jq '.details.name | select(.name == "James Brown")'
This would give me the output
James Brown
But what if I want to get the id of this person as well? Now, I'm aware this is a rough and simple example - the program I'm working with at the moment is 5 or 6 levels deep with many different JQ functions other than select. I need a way to select a parent's field when I am already 5 or 6 layers deep after carrying out various methods of filtering.
Can anyone help? Is there any way of 'going in reverse', back up to the parent? (Not sure if I'm making sense!)
For a more generic approach, save the value of the "parent" element at the detail level you want, then pipe it at the end of your filter:
jq '. as $parent | .details.name | select(. == "James Brown") | $parent'
Of course, for the trivial case you expose, you could omit this entirely:
jq 'select(.details.name == "James Brown")'
Also, consider that if your selecting filters return many matches for a single parent object, you will receive a copy of the parent object for each match. You may wish to make sure your select filters only return one element at the parent level by wrapping all matches below parent level into an array, or to deduplicate the final result with unique.
Give this a shot:
echo $jsonVariable | jq '{Name: .details.name, Id: .Id} | select(.name == "James Brown")'
Rather than querying up to the value you're testing for, query up to the root object that contains the value you're querying on and the values you wish to select.
You need the object that contains both the id and the name.
$ jq --arg name 'James Brown' 'select(.details.name == $name).id' input.json