azure cli az group list return data like
[
"demo3",
"demo",
"demo2",
"NetworkWatcherRG"
]
I'd like to foreach it's value on ubuntu bash then printing below result
demo3
demo
demo2
NetworkWatcherRG
What I've tried :
I tried below script
jq -c '.[]' $(az group list) | while read i; do echo $i ;done
but get image's error
Your command expands to this (see it for yourself with set -x):
jq -c '.[]' '[' '"demo3",' '"demo",' '"demo2",' '"NetworkWatcherRG"' ']'
The command substitution is replaced with the command output, but jq doesn't expect a JSON body as a parameter – either files containing JSON, or a stream in standard input. Since all the parameters ([, "demo3" etc.) are not filenames, you see the errors you do.
You could have Bash make it look like it's a file with process substitution:
jq -c '.[]' <(az group list)
or, more portably, use pipes:
az group list | jq -c '.[]'
Notice that quoting wouldn't help here either: if you ran
jq -c '.[] "$(az group list)"
it would expand to
jq -c '.[]' '[
"demo3",
"demo",
"demo2",
"NetworkWatcherRG"
]'
and jq would try to open a file with the name
[
"demo3",
"demo",
"demo2",
"NetworkWatcherRG"
]
which does not exist.
Related
I have a javascript file which prints a JSON array of objects:
// myfile.js output
[
{ "id": 1, "name": "blah blah", ... },
{ "id": 2, "name": "xxx", ... },
...
]
In my bash script, I want to iterate through each object.
I've tried following, but it doesn't work.
#!/bin/bash
output=$(myfile.js)
for row in $(echo ${output} | jq -c '.[]'); do
echo $row
done
You are trying to invoke myfile.js as a command. You need this:
output=$(cat myfile.js)
instead of this:
output=$(myfile.js)
But even then, your current approach isn't going to work well if the data has whitespace in it (which it does, based on the sample you posted). I suggest the following alternative:
jq -c '.[]' < myfile.js |
while read -r row
do
echo "$row"
done
Output:
{"id":1,"name":"blah blah"}
{"id":2,"name":"xxx"}
Edit:
If your data is arising from a previous process invocation, such as mongo in your case, you can pipe it directly to jq (to remain portable), like this:
mongo myfile.js |
jq -c '.[]' |
while read -r row
do
echo "$row"
done
How can I make jq -c '.[]' < (mongo myfile.js) work?
In a bash shell, you would write an expression along the following lines:
while read -r line ; do .... done < <(mongo myfile.js | jq -c .[])
Note that there are two occurrences of "<" in the above expression.
Also, the above assumes mongo is emitting valid JSON. If it emits //-style comments, those would have somehow to be removed.
Comparison with piping into while
If you use the idiom:
... | while read -r line ; do .... done
then the bindings of any variables in .... will be lost.
I have a json file which has the below data
{"Item": {"ID": {"S": "4869949"},"no":{"N": "2"}}}
I need to retrieve the S value from ID and N value from no and put them into a query as parameters.. the query looks like this:
query --table-name validation \
--key-condition-expression "ID = :v1 AND no = :v2" \
--expression-attribute-values '{":v1": {"S": "${ID}"},":v2": {"N": "${no}"}}' \
--region us-east-1
I have tried using jq but couldn't figure out how to read a json file and retrieve the values as parameters. This is the first time I am using jq. Can someone help with it?
The following may not be exactly what you're after but does show how you can avoid calling jq more than once, and how the command can partly be constructed by jq. In other words, assuming the templates for --key-condition-expression and --expression-attribute-values are fixed, you should be able to adapt the following to your needs:
Assuming a bash or bash-like shell, and that data.json holds:
{"Item": {"ID": {"S": "4869949"},"no":{"N": "2"}}}
we could write:
ID=ID
no=no
< data.json jq --arg ID "$ID" --arg no "$no" -cr '
.Item
| .ID.S as $v1
| .no.N as $v2
| $v1, $v2, {($v1): {"S": $ID}, ($v2): {"N": $no}}
' | while read -r v1
do
read -r v2
read -r query
echo query --table-name validation \
--key-condition-expression "\"ID = $v1 AND no = $v2\"" \
--expression-attribute-values "'""$query""'" \
--region us-east-1
done
jq as a template engine
If the template with :v1 and :v2 in the Q are variable, then you could adapt the above by using jq as a template engine, a topic which is covered in the jq Cookbook: https://github.com/stedolan/jq/wiki/Cookbook#using-jq-as-a-template-engine
It's basic jq syntax:
$ ID=$(cat data.json | jq .Item.ID.S -r)
$ no=$(cat data.json | jq .Item.no.N -r)
$ echo $ID $no
4869949 2
I'd suggest you read something like https://shapeshed.com/jq-json/ to get more familiar though.
I'm parsing a JSON response with a tool called jq.
The output from jq will give me a list of full names in my command line.
I have the variable getNames which contains JSON, for example:
{
"count": 49,
"user": [{
"username": "jamesbrown",
"name": "James Brown",
"id": 1
}, {
"username": "matthewthompson",
"name": "Matthew Thompson",
"id": 2
}]
}
I pass this through JQ to filter the json using the following command:
echo $getNames | jq -r .user[].name
Which gives me a list like this:
James Brown
Matthew Thompson
I want to put each one of these entries into a bash array, so I enter the following commands:
declare -a myArray
myArray=( `echo $getNames | jq -r .user[].name` )
However, when I try to print the array using:
printf '%s\n' "${myArray[#]}"
I get the following:
James
Brown
Matthew
Thompson
How do I ensure that a new index is created after a new line and not a space? Why are the names being separated?
Thanks.
A simple script in bash to feed each line of the output into the array myArray.
#!/bin/bash
myArray=()
while IFS= read -r line; do
[[ $line ]] || break # break if line is empty
myArray+=("$line")
done < <(jq -r .user[].name <<< "$getNames")
# To print the array
printf '%s\n' "${myArray[#]}"
Just use mapfile command to read multiple lines into an array like this:
mapfile -t myArray < <(jq -r .user[].name <<< "$getNames")
I want to push items with awk to a myArray.json file like so
[
item1,
item2
]
To add item1 I tried
echo -e "[\n]" > myArray.json # Create an empty JSON array with "\n"
awk -v var="item1" '/\]/ {print var} 1' myArray.json >> myArray-tmp.json
mv myArray-tmp.json myArray.json
If I now comment out echo -e "[\n]" > myArray.json line (or conditionally skip it), set var="item2" and run the script, item2 is added to the array. I want to add a comma after first line, but not second.
As a possibly simpler alternative, you could just use jq. So, if you had the json file, myArray.json
[ "item1" ]
You can add additional elements simply with
jq '. + ["item2"]' myArray.json
which should result in
[
"item1",
"item2"
]
jq is a great tool so I upvoted and accepted answer. Here's my ugly way.
[
item1
,item2
]
I needed to do conditional to skip first two lines after first item has been added. Run this line instead.
awk -v var="item2" '/\]/ {print ","var} 1' myArray.json >> myArray-tmp.json
also, one easy alternative - a walk-path based unix utility jtc to manipulate json.
In jtc there's an option to apply changes right into the source file - -f (without it, the result will only be printed to the console):
bash $ jtc myArray.json
[
"item1"
]
bash $
bash $ jtc -f -i'"item2"' myArray.json
bash $
bash $ jtc myArray.json
[
"item1",
"item2"
]
bash $
Note: inserted element must be a valid JSON - hence double quotes (”item2” - that makes a valid JSON string) and to preserve them (escape shell interpolation) - there are outer single quotes around the argument
PS> Disclosure: I'm the creator of the jtc - shell cli tool for JSON operations
As a newbee to bash and jq, I was trying to download several urls from a json file using jq command in bash scripts.
My items.json file looks like this :
［
{"title" : [bob], "link" :[a.b.c]},
{"title" : [alice], "link" :[d.e.f]},
{"title" : [carol], "link" :[]}
]
what I was initially doing was just filter the non-empty link and put them in an array and then download the array:
#!/bin/bash
lnk=( $(jq -r '.[].link[0] | select (.!=null)' items.json) )
for element in ${lnk[#]}
do
wget $element
done
But the problem of this approach is that all the files downloaded use the link as the file names.
I wish to filter json file but still keeps the title name with the link so that i can rename the file in the wget command. But I dont have any idea on what structure should I use here. So how can i keep the title to in the filter and use it after?
You can use this:
IFS=$'\n' read -d '' -a titles < <(jq -r '.[] | select (.link[0]!=null) | .title[0]' items.json);
IFS=$'\n' read -d '' -a links < <(jq -r '.[] | select (.link[0]!=null) | .link[0]' items.json);
Then you can iterate over arrays "${title[#]}" & ${links[#]}...
for i in ${!titles[#]}; do
wget -O "${titles[i]}" "${links[#]}"
done
EDIT: Easier & safer approach:
jq -r '.[] | select (.link[0]!=null) | #sh "wget -O \(.title[0]) \(.link[0])"' items.json | bash
Here is a bash script demonstrating reading the result of a jq filter into bash variables.
#!/bin/bash
jq -M -r '
.[]
| select(.link[0]!=null)
| .title[0], .link[0]
' items.json | \
while read -r title; read -r url; do
echo "$title: $url" # replace with wget command
done