I would like to search and replace a specific value in a json line-based data using jq and keep the object unmodified if the match is false.
Considering the following input.json:
{"foo":{"bar":"one"}}
{"foo":{"bar":"two"}}
I tried the following statement, but the else clause is ignored, and therefore lines that don't match are lost:
jq -c '. | if (select(.foo.bar == "one")) then .foo.bar |= "ok" else . end' input.json
produces result:
{"foo":{"bar":"ok"}}
The following command produces the right output:
jq -c '(. | select(.foo.bar == "one")).foo.bar = "ok"' input.json
desired output:
{"foo":{"bar":"ok"}}
{"foo":{"bar":"two"}}
Why the first command fails to output the object . in the else clause?
Don't use select:
. | if .foo.bar == "one" then .foo.bar |= "ok" else . end
select is useful for filtering, but it doesn't return a false value if there's nothing to select, it doesn't return anything.
Related
I have a couple of json files and both or one of the keys below, might exist or none exists.
{
"parent1": {
"parent1key": "parent1value"
},
"parent2": {
"parent2key": "parent2value"
}
}
if both the key: pairs always exist, I could use
jq --arg parent1keyarg $MyVAL1 --arg parent2keyarg $MyVAL2 '(..|objects|select(.parent1key ).parent1key ) |= $parent1keyarg | (..|objects|select(.parent2key).parent2key) |= $parent2keyarg ' jsonfile.json
I want to find the parent1key and parent2key and if found one or both, then replace them with a variable that's generated during runtime. If none of the key: values are found just ignore it rather than troughing up an error.
I want to use jq --arg instead of bash if conditions, just to ensure the JSON formatting is accurate ( or jq is interesting to me ).
how could we write conditions in JQ is what I'm unable to figure out from jq manuals.
Appreciate it if anyone could help me with this.
Thanks!
A simple walk/1 expression would be sufficient to check for existence of the key recursively. It can be further improved by making it a function.
def replace(k; v):
walk(if type == "object" and has(k) then .[k] = v else . end);
replace("parent1key"; "foo") | replace("parent2key"; "bar")
The first argument to replace takes the key name for which the value is to be replaced. Simply pipe any numbers of keys and the associated values that you want to change.
From your description, it would seem the following would suffice:
if has("parent1key") then .parent1key = $a
else . end
| if has("parent2key") then .parent2key = $b
else . end
If you need more than that, then I’d use the above with walk:
walk(if type=="object"
then if has("parent1key") then .parent1key = $a
else . end
| if has("parent2key") then .parent2key = $b
else . end
else . end)
( .parent1.parent1key | select( . ) ) |= $parent1keyarg |
( .parent2.parent2key | select( . ) ) |= $parent2keyarg
I have a JSON object that looks like this:
[{"name":"NAME_1"},"NAME_2"]
I would like an output of
["NAME_1", "NAME_2"]
Some of the entries in the array are an object with a key "name" and some are just a string of the name. I am trying to extract an array of the names. Using
jq -cr '.[].name // []'
throws an error as it is trying to index .name of the string object. Is there a way to check if it is a string, and if so just use its value instead of .name?
echo '[{"name":"NAME_1"},"NAME_2"]' \
| jq '[ .[] | if (.|type) == "object" then .name else . end ]'
[
"NAME_1"
"NAME_2"
]
Ref:
https://stedolan.github.io/jq/manual/#ConditionalsandComparisons
https://stedolan.github.io/jq/manual/#type
As #LéaGris comments, a simpler version
jq '[ .[] | .name? // . ]' file
https://stedolan.github.io/jq/manual/#ErrorSuppression/OptionalOperator:%3f
https://stedolan.github.io/jq/manual/#Alternativeoperator://
You can use the type function which returns "object" for objects.
jq '.[] | if type == "object" then .name else . end' file.json
To get the output as array, just wrap the whole expression into [ ... ].
Just use the error suppression operator with ?, map and scalars
jq 'map( .name?, scalars )'
Note that by using scalars, it is assumed that other than objects with name, all others are names of form NAME_*. If there are other strings as well, and you need to exclude some of them you might need to add some additional logic to do that. e.g. using startswith(..) with a string of your choice.
map( .name?, select( scalars | startswith("NAME") ) )
Demo
With your shown samples only, please try following jq code. Using tostream function here to get the required values from requirement.
jq -c '[.[] | tostream | if .[1] != null then .[1] else empty end]' Input_file
I have a small jq use where I want to reverse sort by a tree's size. I know it's possible with du -h and sort -h etc. But I'm trying it out with tree's json output and jq.
It looks like:
$ tree -h -pug --du -nA -J perl5 | \
awk -v RS= '{ gsub(/,[[:space:]]*]/, "\n]", $0) }1' | \
jq '
walk(if type == "object" and has("contents")
then (.contents|sort_by(.size)|reverse)
else . end)'
jq: error (at <stdin>:65): Cannot index array with string "size"
So to unpack this, awk is used because the json output of tree contains extra commas after the last file or directory in a directory's contents array.
Line 65 of the input contains the report which looks like:
{"type":"report","size":1310049,"directories":24,"files":15}
It doesn't contains contents so the if should be avoiding it.
Here's some simpler test cases:
$ echo '
{"a":0, "c":[
{"a":1, "s":3},
{"a":2, "s":4}]}' | jq -c '
walk(if type == "object" and has("c") and (.c|length) > 0
then (.c|sort_by(.s)|reverse)
else . end)'
[{"a":2,"s":4},{"a":1,"s":3}]
$ echo '
{"a":0, "c":[
{"a":1,"s":3, "c":[
{"a":1,"s":5},
{"a":2,"s":6}]},
{"a":2,"s":4}]}' | jq -c '
walk(if type == "object" and has("c") and (.c|length) > 0
then (.c|sort_by(.s)|reverse)
else . end)'
jq: error (at <stdin>:1): Cannot index array with string "s"
I'm not sure I understand this error message and what I'm missing about walk.
The argument to walk in your post is incorrect.
You presumably want to update the value of .contents, i.e.:
walk(if type == "object" and has("contents")
then .contents |= (sort_by(.size)|reverse)
else . end)
(In the latest (post-1.6) version of jq, you can omit the else . clause.)
p.s.
Version 1.8 of tree fixes the trailing-comma issue.
If your tree produces invalid JSON when using the -J option, rather than using a generic text-processing tool such as awk, it might be better to use a tool such as hjson to "sanitize" the pseudo-JSON, e.g.
tree .... | hjson -j | jq ...
p.p.s.
To understand why one gets the error when the incorrect expression is used as the argument of walk, let's consider the simple test case in the OP, but with an extra debug inserted so we can see what's going on:
echo '
{"a":0, "c":[
{"a":1,"s":3, "c":[
{"a":1,"s":5},
{"a":2,"s":6}]
},
{"a":2,"s":4}]
}' | jq -c '
walk(if type == "object" and has("c")
then debug | .c | sort_by(.s) | reverse
else . end)'
(I've left out the length check as it just clutters up the code.)
This produces:
["DEBUG:",{"a":1,"s":3,"c":[{"a":1,"s":5},{"a":2,"s":6}]}]
["DEBUG:",{"a":0,"c":[[{"a":2,"s":6},{"a":1,"s":5}],{"a":2,"s":4}]}]
jq: error (at <stdin>:6): Cannot index array with string "s"
Now we can see the problem: the second DEBUG line shows that .c has become a bit of a jumble: the first item is an array. This is because we have replaced .c with an array. It is for this reason that the attempt to use sort_by(.s) is failing.
To understand this more completely, it would be necessary to check the definition of walk, which is easily done: you could google jq "def walk" or go to the source: builtin.jq
My json
{
"license": " See license.md",
"dependencies": {
"#gx/core": "0.279.0-b1-abc-1234-0716.4567",
"#gx/api": "0.279.0-b1-abc-1234-0716.4567",
"#gx/name": "0.279.0-b1-abc-1234-0716.4567"
}
}
I want to replace "0.279.0-b1-abc-1234-0716.4567" with "0.279.0-b1-abc-1234-0716.9856" in all places.
jq '.dependencies[].["#gx/core"] |= (if . == "0.279.0-b1-abc-1234-0716.4567" then "0.279.0-b1-abc-1234-0716.9856" else . end)' info.json
jq: error: syntax error, unexpected '[', expecting FORMAT or QQSTRING_START (Unix shell quoting issues?) at <top-level>, line 1:
.dependencies[].["#gx/core"] |= (if . == "0.279.0-b1-abc-1234-0716.4567" then "0.279.0-b1-abc-1234-0716.9856" else . end)
jq: 1 compile error
I am looking for something like this
jq '.dependencies[].["#gx/[a-z]*"] |= (if . == "^(\d+\.){2}[0-9]+(-[a-zA-Z0-9]*){4}\.[0-9]*$" then "0.279.0-b1-abc-1234-0716.9856" else . end)' info.json
Using jq, there are many different approaches, with very different semantics, as can be seen from these solutions to the first problem (without regexes):
walk(if . == "0.279.0-b1-abc-1234-0716.4567"
then "0.279.0-b1-abc-1234-0716.9856" else . end)
A more focused approach:
.dependencies |=
map_values(if . == "0.279.0-b1-abc-1234-0716.4567"
then "0.279.0-b1-abc-1234-0716.9856" else . end)
regexes
The above approaches can all be used in the case of regex searches too, e.g. the last case would become:
.dependencies |= with_entries(
if (.key | test("#gx/[a-z]*"))
and (.value | test("^(\\d+\\.){2}[0-9]+(-[a-zA-Z0-9]*){4}\\.[0-9]*$"))
then .value = "0.279.0-b1-abc-1234-0716.9856" else . end)
Note that the regex strings must be JSON strings, and hence the doubling of backslashes.
if without else
If you have a sufficiently recent version of jq, those dangling occurrences of "else ." can be dropped.
if you up to considering a non-jq solution, let me offer here one based on a walk-path unix utility jtc:
bash $ <file.json jtc -w'[dependencies]<0\.279\.0\-b1\-abc\-1234\-0716\.4567>R:' -u'"0.279.0-b1-abc-1234-0716.9856"'
{
"dependencies": {
"#gx/api": "0.279.0-b1-abc-1234-0716.9856",
"#gx/core": "0.279.0-b1-abc-1234-0716.9856",
"#gx/name": "0.279.0-b1-abc-1234-0716.9856"
},
"license": " See license.md"
}
bash $
walk-path (-w):
[dependencies] addresses (from root) the given record
<...>R: - a search lexeme, finds using RE (suffix R) all (quantifier :) entries matching the given reg.expression.
-u will update (replace) all found matches.
-- or --
using your REs, matching both labels and values:
bash $ <file.json jtc -w'[dependencies]<#gx/[a-z]*>L:<^(\d+\.){2}[0-9]+(-[a-zA-Z0-9]*){4}\.[0-9]*$>R' -u'"0.279.0-b1-abc-1234-0716.9856"'
same result
PS> Disclosure: I'm the creator of the jtc tool
I'm looking for ways to select JSON entries based on an array that I provide as a literal:
$ echo '["a","b","c","d"]' | jq '.[] | select (. == ["a","b"][] )'
"a"
"b"
In the code above, all entries are selected that are in the ["a","b"] array. However, I don't understand how the . == ["a","b"][] predicate works in detail and would be grateful for an explanation. The tricky part is the right-hand side of ==.
Related:
jq - How to select objects based on a 'whitelist' of property values
The key to understanding here is that jq is stream-oriented. ["a","b"][] produces a stream, ergo . == ["a","b"][] produces a stream. select selects the items that produce truthy values in that stream.
To gain an understanding of how jq works, it often helps to pull things apart. In the present case, you could begin by trying:
echo '["a","b","c","d"]' | jq '.[] | (. == ["a","b"][])'
debug is also helpful, e.g.
echo '["a","b","c","d"]' | jq '.[] | select(debug == ["a","b"][])'