How to use regex to parse this json: { "success" : true }? - json

I am using a shell script to make an API call and I need to verify that the json response is this:
{ "success" : true }
I am able to echo the call response to see that it has that value but I need to validate the response in an if statement so that the script can continue, I have tried to do this a number of ways with no success
Regex - I have used regex to extract values from other json responses, but I have not found a regex pattern that can extract the value of "success" with this json
String Comparison - I thought of simply using this condition to attempt to match the strings:
if [ "$callResponse" = '{ "success" : true }' ]
However I quickly ran into issues with the script reading the json due to its special characters, I tried using sed to add a backslash before each special character but sed could not read the json either
Lastly I tried to pipe the response to python but got the error "ValueError: No JSON object could be decoded" when using this command:
status=${$callResponse | python -c "import sys, json; print(json.load(sys.stdin)['success'])"}
Does anyone know a regex pattern that could find that specific json string? Is there another simple solution to this issue?
(Note that it is not possible to download jq or any other utilities for this PC)

Since the caller knows that the response is { "success" : true }, I can't think of any reason to not use jq in this case. For instance, you can try something like this:
if echo '{ "success" : true }' | jq --exit-status '.success == true' >/dev/null; then
echo "success"
# Do something success is true in the response.
else
echo "not success" response.
# Do something else success is not true or absent in the
fi
If you want to make an API call and get the response, you can easily pass the JSON response directly from wget to jq instead of going the roundabout way of storing it in an intermediate variable by tweaking it like this:
if wget --timeout 10 -O - -q -t 1 https://your.api.com/endpoint | jq --exit-status '.success == true' >/dev/null; then
echo "success"
else
echo "not success"
fi

To match when the value of success is true in a flexible way:
"success"\s*:\s*"?true"?
This will match all of these:
{ "success" : true }
{ "success" : "true" }
{ "success":true}
To be strict and match the above, but not imbalanced quotes like { "success" : "true }, use this:
"success"\s*:\s*("?)true\1

I would highly recommend not doing it that way.
We used to do it this way long ago and got into trouble when the response code is "200 OK" but receiving {"success": false} seemed to contradict each other.
A better approach is to use the response status codes instead.
Simply return 200 OK if success is true otherwise return the appropriate error status code if its not.
https://www.restapitutorial.com/httpstatuscodes.html
EDIT:
Bash script to help:
COOKIE_FILE="cookies.txt"
SERVER_IP="172.1.2.3"
LOGFILE="logs/api-calls.log"
WGETLOGFILE="logs/last-api-call.log"
#Helper function
on_wget_err ( )
{
EXITCODE=${1}
case ${EXITCODE} in
0) RESULT="OK";;
*) cat ${WGETLOGFILE} >> ${LOGFILE};
grep "HTTP/1.1" ${WGETLOGFILE} | gawk '{print substr($0,16)}'
exit 0;;
esac
}
if wget -O - -qT 4 -t 1 ${SERVER_IP} > /dev/null; then
echo "Server is up"
wget -S -O - --load-cookies ${COOKIE_FILE} "http://${SERVER_IP}${SERVER_ADDRESS}/my/api?param=$1" 2> ${WGETLOGFILE}
on_wget_err ${?}
echo "API was successfull"
else
echo "Server or network down"
exit 1;
fi

Your Python attempt was close. Here's a working one:
callResponse='{ "success" : true }'
status=$(echo "$callResponse" |
python -c "import sys, json; print(json.load(sys.stdin)['success'])")
echo "$status"
Or alternatively, rewritten to go straight in an if statement:
callResponse='{ "success" : true }'
if echo "$callResponse" | python -c "import sys, json; sys.exit(0 if json.load(sys.stdin)['success'] else 1)"
then
echo "Success"
fi

Related

How to use jq to give true or false when uri field is present in my output json

I have a JSON which goes like this:
{
"results":[
{
"uri":"www.xxx.com"
}
]
}
EDIT
When uri is not present, JSON looks like this:
{
"results":[
]
}
In some cases, uri is present and in some cases, it is not.
Now, I want to use jq to return boolean value if uri is present or not.
This is what I wrote so far but despite uri being present, it gives null.
${search_query_response} contains the JSON
file_status=$(jq -r '.uri' <<< ${search_query_response})
Can anyone guide me?
Since you use jq, it means you are working within a shell script context.
If the boolean result is to be handled by the shell script, you can make jq set its EXIT_CODE depending on the JSON request success or failure status, with jq -e
Example shell script using the EXIT_CODE from jq:
if uri=$(jq -je '.results[].uri') <<<"$search_query_response"
then
printf 'Search results contains an URI: %s.\n' "$uri"
else
echo 'No URI in search results.'
fi
See man jq:
-e / --exit-status:
Sets the exit status of jq to 0 if the last output values was neither false nor null, 1 if the last output value was either false or null, or 4 if no valid result was ever produced. Normally jq exits with 2 if there was any usage problem or system error, 3 if there was a jq program compile error, or 0 if the jq program ran.
Another way to set the exit status is with the halt_error builtin function.
The has function does the job:
jq '.results|map(has("uri"))|.[]'
map the has function on .results.

How can I prettyprint JSON on the command line, but allow invalid JSON objects to pass though?

I'm currently tailing some logs in bash that are half JSON, half text like below:
{"response":{"message":"asdfasdf"}}
{"log":{"example":"asdfasdf"}}
here is some text
{"another":{"example":"asdfasdf"}}
more text
Each line is either a full valid JSON object or some text that would fail a JSON parser.
I've looked at jq and underscore-cli to see if they have options to return the invalid object in the case of failure, but I'm not seeing any.
I've also tried to use a || operator to cat the piped input, but I'm losing the value somehow. Maybe I should read up on pipes more? Example: getLogs -t | (underscore print || cat)
I think I could write a script that stores the input. Format it, and return the output if successful. If it fails returned the stored value. I feel like there should be a simpler way though. Any thoughts?
You can use this node library
install with
$ npm install -g js-beautify
Here is what I did:
$ js-beautify -r test.js
beautified test.js
I tested it with an incomplete json file and it worked
jq can check for invalid json
#!/bin/bash
while read p; do
if jq -e . >/dev/null 2>&1 <<<"$p"; then
echo $p | jq
else
echo 'Skipping invalid json'
fi
done < /tmp/tst.txt
{
"response": {
"message": "asdfasdf"
}
}
{
"log": {
"example": "asdfasdf"
}
}
Skipping invalid json
{
"another": {
"example": "asdfasdf"
}
}
Skipping invalid json

How to get an element of a JSON object in bash?

I'm performing a curl command using a bash file and the return is a json object. How to get a element of this json object in this bash file?
Put request
https://sms.world-text.com/v2.0/sms/send?id=11111&key=Testkey&srcaddr=DA_Health&dstaddr=000000000000&method=PUT&txt=Message_Text_Text
Response
{"status":"1","error":"1000","desc":"Authorisation Failure"}
to="000000000000"
message="Test_message"
url="https://sms.world-text.com/v2.0/sms/send?id=11111&key=TestKey&srcaddr=SMSMsg&dstaddr=${to}&method=PUT&txt=${message}"
return=$(curl -sm 5 $url --data-urlencode "${message}" -A 'Test')
Finally, the "return" variable has the value below:
{"status":"1","error":"1000","desc":"Authorisation Failure"}
I expect to perform that validation
if [[ "$status" != 0]]; then
&2 echo "$return"
fi
But how can I get the element "status" and his value "1" from $return in the bash file?
it's ideologically wrong to process JSON format with JSON-agnostic tools (like awk, sed, etc). JSON format must be processed with JSON-aware tools.
E.g., if your curl response was a multi-line JSON (which is quite often the case), then most likely the sed based solution would not work right for you.
One of the unix utilities to work with JSON is jtc, with that one your solution would look like this:
status=$(<<<$return jtc -w[status] -qq)
and then you can apply your check:
if [[ "$status" != 0]]; then
>&2 echo "$return"
fi
PS> Disclosure: I'm the creator of the jtc - shell cli tool for JSON operations
You could use sed to filter out the status code from the $result.
status=$(echo "$return" | sed -E 's/\{"status"\s?:\s?"([0-9]+)".*/\1/')
and then can do test:
if [[ "$status" != 0]]; then
>&2 echo "$return"
fi
You can use jq. Here is a comprehensive guide about the tool -> https://stedolan.github.io/jq/tutorial/.
e.g.
echo '{ "foo": 123, "bar": 456 }' | jq '.foo'
This would print 123.
It also works with nested objects.

parsing json to check whether a field is blank in bash

So, lets say, I am trying to write a shell script which does the following:
1) Ping http://localhost:8088/query?key=q1
2) It returns a json response :
{
"status": "success",
"result": []
"query": "q1"
}
or
{
"status": "success",
"result": ["foo"],
"artist_key": "q1"
}
The "result" is either an empty array or filled array..
I am trying to write a shell script which is checking whether "result" is empty list or not?
Something like this would work:
# Assume result is from curl, but could be in a file or whatever
if curl "http://localhost:8088/query?key=q1" | grep -Pq '"result":\s+\[".+"\]'; then
echo "has result"
else
echo "does not have result"
fi
However, I'm assuming these are on separate lines. If not, there are linters for format it.
Edited (based on the jq comment), here's a jq solution as suggested by Adrian Frühwirth:
result=$( curl "http://localhost:8088/query?key=q1" | jq '.result' )
if [ "$result" == "[]" ]; then
echo "does not have result"
else
echo "has result"
fi
I have learned something new today. And as I play around with this more, maybe it's better to do this:
result=$( curl "http://localhost:8088/query?key=q1" | jq '.result | has(0)' )
if [ "$result" == "true" ]; then
echo "has result"
else
echo "does not have result"
fi
See the manual. I wasn't able to get the -e or --exit-status arguments to work.
I'd use a language that can convert JSON to a native data structure:
wget -O- "http://localhost:8088/query?key=q1" |
perl -MJSON -0777 -ne '
$data = decode_json $_;
exit (#{$data->{result}} == 0)
'
That exits with a success status if the result attribute is NOT empty. Encapsulating into a shell function:
check_result() {
wget -O- "http://localhost:8088/query?key=q1" |
perl -MJSON -0777 -ne '$data = decode_json $_; exit (#{$data->{result}} == 0)'
}
if check_result; then
echo result is NOT empty
else
echo result is EMPTY
fi
I like ruby for parsing JSON:
ruby -rjson -e 'data = JSON.parse(STDIN.read); exit (data["result"].length > 0)'
It's interesting that the exit status requires the opposite comparison operator. I guess ruby will convert exit(true) to exit(0), unlike perl which has no true boolean objects only integers.

How to parse json response in the shell script?

I am working with bash shell script. I need to execute an URL using shell script and then parse the json data coming from it.
This is my URL - http://localhost:8080/test_beat and the responses I can get after hitting the URL will be from either these two -
{"error": "error_message"}
{"success": "success_message"}
Below is my shell script which executes the URL using wget.
#!/bin/bash
DATA=$(wget -O - -q -t 1 http://localhost:8080/test_beat)
#grep $DATA for error and success key
Now I am not sure how to parse json response in $DATA and see whether the key is success or error. If the key is success, then I will print a message "success" and print $DATA value and exit out of the shell script with zero status code but if the key is error, then I will print "error" and print $DATA value and exit out of the shell script with non zero status code.
How can I parse json response and extract the key from it in shell script?
I don't want to install any library to do this since my JSON response is fixed and it will always be same as shown above so any simpler way is fine.
Update:-
Below is my final shell script -
#!/bin/bash
DATA=$(wget -O - -q -t 1 http://localhost:8080/tester)
echo $DATA
#grep $DATA for error and success key
IFS=\" read __ KEY __ MESSAGE __ <<< "$DATA"
case "$KEY" in
success)
exit 0
;;
error)
exit 1
;;
esac
Does this looks right?
If you are going to be using any more complicated json from the shell and you can install additional software, jq is going to be your friend.
So, for example, if you want to just extract the error message if present, then you can do this:
$ echo '{"error": "Some Error"}' | jq ".error"
"Some Error"
If you try this on the success case, it will do:
$echo '{"success": "Yay"}' | jq ".error"
null
The main advantage of the tool is simply that it fully understands json. So, no need for concern over corner cases and whatnot.
#!/bin/bash
IFS= read -d '' DATA < temp.txt ## Imitates your DATA=$(wget ...). Just replace it.
while IFS=\" read -ra LINE; do
case "${LINE[1]}" in
error)
# ERROR_MSG=${LINE[3]}
printf -v ERROR_MSG '%b' "${LINE[3]}"
;;
success)
# SUCCESS_MSG=${LINE[3]}
printf -v SUCCESS_MSG '%b' "${LINE[3]}"
;;
esac
done <<< "$DATA"
echo "$ERROR_MSG|$SUCCESS_MSG" ## Shows: error_message|success_message
* %b expands backslash escape sequences in the corresponding argument.
Update as I didn't really get the question at first. It should simply be:
IFS=\" read __ KEY __ MESSAGE __ <<< "$DATA"
[[ $KEY == success ]] ## Gives $? = 0 if true or else 1 if false.
And you can examine it further:
case "$KEY" in
success)
echo "Success message: $MESSAGE"
exit 0
;;
error)
echo "Error message: $MESSAGE"
exit 1
;;
esac
Of course similar obvious tests can be done with it:
if [[ $KEY == success ]]; then
echo "It was successful."
else
echo "It wasn't."
fi
From your last comment it can be simply done as
IFS=\" read __ KEY __ MESSAGE __ <<< "$DATA"
echo "$DATA" ## Your really need to show $DATA and not $MESSAGE right?
[[ $KEY == success ]]
exit ## Exits with code based from current $?. Not necessary if you're on the last line of the script.
You probably already have python installed, which has json parsing in the standard library. Python is not a great language for one-liners in shell scripts, but here is one way to use it:
#!/bin/bash
DATA=$(wget -O - -q -t 1 http://localhost:8080/test_beat)
if python -c '
import json, sys
exit(1 if "error" in json.loads(sys.stdin.read()) else 0)' <<<"$DATA"
then
echo "SUCCESS: $DATA"
else
echo "ERROR: $DATA"
exit 1
fi
Given:
that you don't want to use JSON libraries.
and that the response you're parsing is simple and the only thing you care about is the presence of substring "success", I suggest the following simplification:
#!/bin/bash
wget -O - -q -t 1 http://localhost:8080/tester | grep -F -q '"success"'
exit $?
-F tells grep to search for a fixed (literal) string.
-q tells grep to produce no output and instead only reflect via its exit code whether a match was found or not.
exit $? simply exits with grep's exit code ($? is a special variable that reflects the most recently executed command's exit code).
Note that if you all you care about is whether wget's output contains "success", the above pipeline will do - no need to capture wget's output in an aux. variable.