next to my attempt to parse a JSON response from curl using bash I now decided to give a try with jq.
I have checked the documentation but I could not find a way to iterate trough the elements and "do" something.
Here's an idea on what I am trying to achieve, cast the result from jq into an array, (it doesn't work)
__json=$($omd_response | ~/local-workspace/bash/jq -r '[.]')
for x in "${__json[#]}"
do
echo "-metadata" $x
done
Any other idea is much appreciated.
Thanks
This:
declare -a things
things=($(jq tostring myfile.json) )
for x in "${things[#]}"; do
echo "-metadata" "$x"
done
almost works. It splits things on whitespace.
This works:
declare -a things
OIFS=$IFS
IFS= things=($(jq -r 'tojson|tostring' myfile.json) )
IFS=$OIFS
for x in "${things[#]}"; do
echo "-metadata" "$x"
done
Really, we need a JSON-aware shell... Something like ksh93's compound variables, but JSON-compatible.
Related
I have this function that iterates over the keys of a given JSON using curl and jq as the JSON processor, My code looks like this : (bash)
function getJSONContent {
option=""
for option in "header" "developer" "category" "description"
do
content=$(curl -s $extension_url | jq ".[\"$extension\"][\"$option\"]")
printf "$content\n"
done
}
But the problem is that it curl's 4 time and I haven't found a better solution to this without getting an error.
Is doing this okay? Or is there just a better solution to do this in Bash / Shellscript?
is there just a better solution ... ?
Yes!
You evidently only need one call to curl and one to jq, but at the very
least, you should avoid calling curl more than once.
Avoid constructing the jq command "on the fly". Instead, you can pass in the shell (or environment) variables
on the command line, e.g. using --arg or --argjson
In this specific case, it looks like you can avoid calling jq more than once by simply using jq's ',' operator.
In brief, try something along the following lines:
curl -s "$extension_url" |
jq --arg extension "$extension" '
.[$extension]["header","developer","category","description"]'
I am trying to take the JSON result from a curl and set each result for a particular JSON object to separate variables.
Using the following line in my script to retrieve results:
PROFILE=$(curl --user admin:admin -k -X GET https://192.168.1.1:8000/rest/call/profiles.json | jq '[.profiles[].id]')
with the above line my results might look something like this (but i could have 1 to many lines returned):
[
"myprofile",
"myprofile1",
"myprofile2",
"myprofile3"
]
Next, trying to determine the best route to set each id that is returned to a unique variable to be used later on in the script. .id could return 1 to 30 results so i'm assuming a do while loop and using the split command is in need here?
Any help is much appreciated, thank you in advance!
i'm not entirely sure what you're asking, but maybe this helps:
echo '[ "myprofile", "myprofile1", "myprofile2", "myprofile3" ]' |
grep -o '"[^"]\+"' | tr -d '"' | while read x; do
echo $x
# do your thing
done
output:
myprofile
myprofile1
myprofile2
myprofile3
I'd like a unix one-liner that will output the current execution environment as a JSON structure like: { "env-var" : "env-value", ... etc ... }
This kinda works:
(echo "{"; printenv | sed 's/\"/\\\"/g' | sed -n 's|\(.*\)=\(.*\)|"\1"="\2"|p' | grep -v '^$' | paste -s -d"," -; echo "}")
but has some extra lines and I think won't work if the environment values or variables have '=' or newlines in them.
Would prefer pure bash/sh, but compact python / perl / ruby / etc one-liners would also be appreciated.
Using jq 1.5 (e.g. jq 1.5rc2 -- see http://stedolan.github.io/jq):
$ jq -n env
This works for me:
python -c 'import json, os;print(json.dumps(dict(os.environ)))'
It's pretty simple; the main complication is that os.environ is a dict-like object, but it is not actually a dict, so you have to convert it to a dict before you feed it to the json serializer.
Adding parentheses around the print statement lets it work in both Python 2 and 3, so it should work for the forseeable future on most *nix systems (especially since Python comes by default on any major distro).
#Alexander Trauzzi asked: "Wondering if anyone knows how to do this, but only passing a subset of the current environment's variables?"
I just found the way to do this:
jq -n 'env | {USER, HOME, PS1}'
I have to curl to a site (statuscake.com) that sends multiple items back in a JSON, each line of which contains multiple items. I want to extract from each line two of them, WebsiteName and TestID, so I can check if WebsiteName matches the one I'm interested in, get the TestID out and pass this to a second curl statement to delete the test.
Although it's more complex, the JSON that comes back is essentially of the form
[{"TestID": 123, "WebsiteName": "SomeSite1"}, {"TestID": 1234, "WebsiteName": "SomeSite2"}]
I can't seem to find a magic jq command to do it all in one - if there is one, I'd be really happy to see it.
I've got
cat $data | jq '[.[] | .WebsiteName]'
to get an array of the website names (and a very similar one for the TestIDs, but I think I've done something daft. data is the information coming back from the curl to get the JSON and that's populated OK.
I want to be able to assign these to two arrays, names and ids, then search names for the index of the relevant name, grab the id from ids and pass that to the curl. Unless there's a better way.
Any advice please?
My Xidel can do it all at once by selecting the JSON with a XPath-like query:
E.g. return all ids where the WebsiteName contains "site2" from an array of objects:
xidel /tmp/x.json -e '$json()[contains((.).WebsiteName, "site2")]/TestID'
Or e.g. to download the original JSON and then make the HTTP request with the ids:
xidel http://statuscake.com/your-url... -f '$json()[contains((.).WebsiteName, "site2")]/TestID!x"/your-delete-url{.}..."'
If I'm getting your question right, it sounds like what you want is to, for each element, select those where .WebsiteName == "needle", and then get .TestID from it. You can do just that:
.[] | select(.WebsiteName == "needle") | .TestID
If you want an array as the result, you can wrap the above script in square brackets.
The jq filters startswith and endswith may be of interest to you. If you're going to pass the result back to cURL, you may also be interested in the #sh formatting filter and the -r command-line flag.
Assuming you have a bash 4+ and assuming the json is valid (does not contain newlines in strings, etc.) this works:
$ echo "$data"
[{"TestID": 123, "WebsiteName": "SomeSite1"}, {"TestID": 1234, "WebsiteName":
"SomeSite2"}, {"TestID": 555, "WebsiteName": "foo*ba#r blah[54]quux{4,5,6}"}]
$ declare -A arr
$ while IFS= read -r line; do
eval "$line"
done < <(jq -M -r '.[] | #sh "arr[\(.WebsiteName)]+=\(.TestID)"' <<<"$data")
$ declare -p arr
declare -A arr='(["foo*ba#r blah[54]quux{4,5,6}"]="555" [SomeSite2]="1234" [SomeSite1]="123" )'
Here is a solution using only jq primitives.
.[]
| if .WebsiteName == "SomeSite1" then .TestID else empty end
This is essentially the same as Santiago's answer but if you are new to jq it may be informative because select/1 is defined as
def select(f): if f then . else empty end;
I am trying to create a menu using the select function in bash. I am accessing an API which will return its output in json format. I will then process what the API returns into a select statement where a user can then interact with.
Here is the API call and how I parse the output:
curl -H "Authorization:Bearer $ACCESS_TOKEN" https://api.runscope.com/buckets \
| python -mjson.tool > output.json
This will send the output from the curl through python's json parsing tool and finally into the output.json file.
I then create an array using this json blob. I had to set IFS to \n in order to parse the file properly:
IFS=$'\n'
BUCKETS=("$(jq '.data | .[].name' output.json)")
I then add an exit option to the array so that users have a way to quit the selection menu:
BUCKETS+=("Exit")
Finally, I create the menu:
select BUCKET in $BUCKETS;
do
case $BUCKET in
"Exit")
echo "Exiting..."
break;;
esac
echo "You picked: $BUCKET"
done
Unfortunately, this does not create the exit option. I am able to see a menu consisting of every other option I want, except the exit option. Every option in the menu and in the array has quotes around them. How do I get the Exit option to show up?
$BUCKETS is expanding to the first element of the BUCKETS array.
Which is then being word-split and used as your select entries.
This makes sense since you wrapped the jq subshell in double quotes which prevented word-splitting from happening there (and means the IFS change isn't doing anything I believe).
Iff your entries can contain spaces and you want that assigned to an array properly the way to do that is by reading the output of jq with a while IFS= read -r entry; do loop.
BUCKETS=()
while IFS= read -r entry; do
BUCKETS+=("$entry")
done < <(jq '.data | .[].name' output.json)
Then appending your exit item to the array.
BUCKETS+=(Exit)
and then using
select BUCKET in "${BUCKETS[#]}"; do
(Both select a in l; do or select a in l\ndo there's no need for ; and \n there.)
That all being said unless you need output.json for something else you can avoid that too.
BUCKETS=()
while IFS= read -r entry; do
BUCKETS+=("$entry")
done < <(curl -H "Authorization:Bearer $ACCESS_TOKEN" https://api.runscope.com/buckets | python -mjson.tool | jq '.data | .[].name')
Instead of
jq '.data | .[].name' output.json
try
jq -r '.data | .[].name' output.json
(-r : raw data without quotes)
And the most important part :
select BUCKET in "${BUCKETS[#]}"
^^^
ARRAY syntax