I'm using jq to parse some JSON. I want to check whether a property exists or not. If it exists I always want to get the first element in the array. Based on this I want to use if then ... else.
My code looks like this:
JSON_INPUT='{"var1":[{"foo":"bar"}],"var2":[{"fooooo":"baaaaar"}]}'
VAR2=$(echo $JSON_INPUT | jq '.["var2"] | .[0]')
if [ -z "${VAR2}" ]
then
echo "does not exist"
# do some stuff...
else
echo "exist"
# do some stuff...
fi
The JSON_INPUT may contain var2 but must not. If it does not exist VAR2 will be null. But I'm not able to check for this null value. Where is the error?
Where is the error?
Replace
[ -z "${VAR2}" ]
with
[ "${VAR2}" = "null" ]
because jq returns string null if var2 is not found in JSON file.
Or use --exit-status:
if echo "$JSON_INPUT" | jq --exit-status '.var2' >/dev/null; then
echo "exists"
else
echo "does not exist"
fi
I want to check whether a property exists or not. If it exists I always want to get the first element in the array ...
The condition can best be expressed as:
if has("var2")
so since you seem to want VAR2 to be null if the condition is not met, you could write:
VAR2=$(jq 'if has("var2") then .["var2"][0] else null end')
If the condition is not met, then we would find:
echo "$VAR2"
null
There is a small problem, here, though: for example, what if the array's first value is null?
So a more discriminating approach would be to write:
VAR2=$(jq 'if has("var2") then .["var2"][0] else empty end')
Even if neither of these approaches is exactly what you want, they illustrate that putting as much of the logic inside the jq program as possible yields a tidy, easy-to-understand, and efficient solution.
Related
I want to use jq to check if a keyword exists as a key in a JSON at any level. Here is what I came up with:
jq -c 'paths | select(.[-1]) as $p| index("headline") //empty' news.json
The output is an array:
[
6,
7,
9
]
I want to map the output array to jq function any() and get an overall result as true if key exists or false if it doesn't.
How can I get that done with jq ?
Provide any with an iterator and a condition
jq 'any(paths; .[-1] == "headline")'
You can also provide the -e (or --exit-status) option to have an exit status of 0 if the result was true, and 1 if the result was false, which can immediately be used for further processing in the shell.
jq -e 'any(paths; .[-1] == "headline")'
You don't need an intermediate array for that.
any(paths[-1]; . == "headline")
Background
I have an object with each value being a nested list of only strings. For each string value within the nested list, look up the string value within the object and add all of its values into the current value.
Here's what I have so far:
#!/bin/bash
in=$(jq -n '{
"bar": [["re", "de"]],
"do": [["bar","baz"]],
"baz": [["re"]],
"re": [["zoo"]]
}')
echo "expected:"
jq -n '{
"bar": [["re", "de"], ["zoo"]],
"do": [["bar","baz"], ["re", "de"], ["re"], ["zoo"]],
"baz": [["re"], ["zoo"]],
"re": [["zoo"]]
}'
echo "actual:"
echo ${in} | jq '. as $origin
| map_values( . +
until(
length == 0;
(. | flatten | map($origin[.]) | map(select( . != [[]] )) | add )
)
)'
Problem:
The output is the exact same as the input $in. If the until() function is removed from the statement, then the output correctly outputs one iteration. Although I want to recursively lookup the output strings within the object and add the lookup value until the lookup value is empty or non-existing.
For example, the key do has a value of [["bar","baz"]]. If we iterate through the values of do we come across baz. The value of baz within the object is [["re"]]. Add baz's value ["re"] to do so that do equals: [["bar","baz"], ["re"]]. Since re IS a key within the object, add the value of ["re"] which is ["zoo"]. Since ["zoo"] is NOT a key within the object finish baz and continue to the next key within the object.
The following solves the problem as originally stated, but the "expected" output as shown does not quite match the stated problem.
echo ${in} | jq -c '
. as $dict
| map_values(reduce (..|strings) as $v (.;
. + $dict[$v] ))
'
produces (after some manual reformatting for clarity):
{"bar":[["re","de"],["zoo"]],
"do":[["bar","baz"],["re","de"],["re"]],
"baz":[["re"],["zoo"]],"re":[["zoo"]]}
If some kind of recursive lookup is needed, then please reformulate the problem statement, being sure to avoid infinite loops.
I would like to search and replace a specific value in a json line-based data using jq and keep the object unmodified if the match is false.
Considering the following input.json:
{"foo":{"bar":"one"}}
{"foo":{"bar":"two"}}
I tried the following statement, but the else clause is ignored, and therefore lines that don't match are lost:
jq -c '. | if (select(.foo.bar == "one")) then .foo.bar |= "ok" else . end' input.json
produces result:
{"foo":{"bar":"ok"}}
The following command produces the right output:
jq -c '(. | select(.foo.bar == "one")).foo.bar = "ok"' input.json
desired output:
{"foo":{"bar":"ok"}}
{"foo":{"bar":"two"}}
Why the first command fails to output the object . in the else clause?
Don't use select:
. | if .foo.bar == "one" then .foo.bar |= "ok" else . end
select is useful for filtering, but it doesn't return a false value if there's nothing to select, it doesn't return anything.
I have a JSON which goes like this:
{
"results":[
{
"uri":"www.xxx.com"
}
]
}
EDIT
When uri is not present, JSON looks like this:
{
"results":[
]
}
In some cases, uri is present and in some cases, it is not.
Now, I want to use jq to return boolean value if uri is present or not.
This is what I wrote so far but despite uri being present, it gives null.
${search_query_response} contains the JSON
file_status=$(jq -r '.uri' <<< ${search_query_response})
Can anyone guide me?
Since you use jq, it means you are working within a shell script context.
If the boolean result is to be handled by the shell script, you can make jq set its EXIT_CODE depending on the JSON request success or failure status, with jq -e
Example shell script using the EXIT_CODE from jq:
if uri=$(jq -je '.results[].uri') <<<"$search_query_response"
then
printf 'Search results contains an URI: %s.\n' "$uri"
else
echo 'No URI in search results.'
fi
See man jq:
-e / --exit-status:
Sets the exit status of jq to 0 if the last output values was neither false nor null, 1 if the last output value was either false or null, or 4 if no valid result was ever produced. Normally jq exits with 2 if there was any usage problem or system error, 3 if there was a jq program compile error, or 0 if the jq program ran.
Another way to set the exit status is with the halt_error builtin function.
The has function does the job:
jq '.results|map(has("uri"))|.[]'
map the has function on .results.
I want to know if a given filter succeeds in pulling data from a JSON data structure. For example:
###### For the user steve...
% Name=steve
% jq -j --arg Name "$Name" '.[]|select(.user == $Name)|.value' <<<'
[
{"user":"steve", "value":false},
{"user":"tom", "value":true},
{"user":"pat", "value":null},
{"user":"jane", "value":""}
]'
false
% echo $?
0
Note: successful results can include boolean values, null, and even the empty string.
###### Now for user not in the JSON data...
% Name=mary
% jq -j --arg Name "$Name" '.[]|select(.user == $Name)|.value' <<<'
[
{"user":"steve", "value":false},
{"user":"tom", "value":true},
{"user":"pat", "value":null},
{"user":"jane", "value":""}
]'
% echo $?
0
If the filter does not pull data from the JSON data structure, I need to know this. I would prefer the filter to return a non-zero return code.
How would I go about determining if a selector successfully pulls data from a JSON data structure vs. fails to pull data?
Important: The above filter is just an example, the solution needs to work for any jq filter.
Note: the evaluation environment is Bash 4.2+.
You can use the -e / --exit-status flag from the jq Manual, which says
Sets the exit status of jq to 0 if the last output values was neither false nor null, 1 if the last output value was either false or null, or 4 if no valid result was ever produced. Normally jq exits with 2 if there was any usage problem or system error, 3 if there was a jq program compile error, or 0 if the jq program ran.
I can demonstrate the usage with a basic filter as below, as your given example is not working for me.
For a successful query,
dudeOnMac:~$ jq -e '.foo?' <<< '{"foo": 42, "bar": "less interesting data"}'
42
dudeOnMac:~$ echo $?
0
For an invalid query, done with a non-existent entity zoo,
dudeOnMac:~$ jq -e '.zoo?' <<< '{"foo": 42, "bar": "less interesting data"}'
null
dudeOnMac:~$ echo $?
1
For an error scenario, returning code 2 which I created by double-quoting the jq input stream.
dudeOnMac:~$ jq -e '.zoo?' <<< "{"foo": 42, "bar": "less interesting data"}"
jq: error: Could not open file interesting: No such file or directory
jq: error: Could not open file data}: No such file or directory
dudeOnMac:~$ echo $?
2
I've found a solution that meets all of my requirements! Please let me know what you think!
The idea is use jq -e "$Filter" as a first-pass check. Then for the return code of 1, do a jq "path($Filter)" check. The latter will only succeed if, in fact, there is a path into the JSON data.
Select.sh
#!/bin/bash
Select()
{
local Name="$1"
local Filter="$2"
local Input="$3"
local Result Status
Result="$(jq -e --arg Name "$Name" "$Filter" <<<"$Input")"
Status=$?
case $Status in
1) jq --arg Name "$Name" "path($Filter)" <<<"$Input" >/dev/null 2>&1
Status=$?
;;
*) ;;
esac
[[ $Status -eq 0 ]] || Result="***ERROR***"
echo "$Status $Result"
}
Filter='.[]|select(.user == $Name)|.value'
Input='[
{"user":"steve", "value":false},
{"user":"tom", "value":true},
{"user":"pat", "value":null},
{"user":"jane", "value":""}
]'
Select steve "$Filter" "$Input"
Select tom "$Filter" "$Input"
Select pat "$Filter" "$Input"
Select jane "$Filter" "$Input"
Select mary "$Filter" "$Input"
And the execution of the above:
% ./Select.sh
0 false
0 true
0 null
0 ""
4 ***ERROR***
I've added an updated solution below
The fundamental problem here is that when try to retrieve a value from an object using the .key or .[key] syntax, jq — by definition — can't distinguish a missing key from a key with a value of null.
You can instead define your own lookup function:
def lookup(k):if has(k) then .[k] else error("invalid key") end;
Then use it like so:
$ jq 'lookup("a")' <<<'{}' ; echo $?
jq: error (at <stdin>:1): invalid key
5
$ jq 'lookup("a")' <<<'{"a":null}' ; echo $?
null
0
If you then use lookup consistently instead of the builtin method, I think that will give you the behaviour you want.
Here's another way to go about it, with less bash and more jq.
#!/bin/bash
lib='def value(f):((f|tojson)//error("no such value"))|fromjson;'
users=( steve tom pat jane mary )
Select () {
local name=$1 filter=$2 input=$3
local -i status=0
result=$( jq --arg name "$name" "${lib}value(${filter})" <<<$input 2>/dev/null )
status=$?
(( status )) && result="***ERROR***"
printf '%s\t%d %s\n' "$name" $status "$result"
}
filter='.[]|select(.user == $name)|.value'
input='[{"user":"steve","value":false},
{"user":"tom","value":true},
{"user":"pat","value":null},
{"user":"jane","value":""}]'
for name in "${users[#]}"
do
Select "$name" "$filter" "$input"
done
This produces the output:
steve 0 false
tom 0 true
pat 0 null
jane 0 ""
mary 5 ***ERROR***
This takes advantage of the fact the absence of input to a filter acts like empty, and empty will trigger the alternative of //, but a string — like "null" or "false" — will not.
It should be noted that value/1 will not work for filters that are simple key/index
lookups on objects/arrays, but neither will your solution. I'm reasonably sure that to
cover all the cases, you'd need something like this (or yours) and something
like get or lookup.
Given that jq is the way it is, and in particular that it is stream-oriented, I'm inclined to think that a better approach would be to define and use one or more filters that make the distinctions you want. Thus rather than writing .a to access the value of a field, you'd write get("a") assuming that get/1 is defined as follows:
def get(f): if has(f) then .[f] else error("\(type) is not defined at \(f)") end;
Now you can easily tell whether or not an object has a key, and you're all set to go. This definition of get can also be used with arrays.