I am trying to parse a JSON file which looks like this and then trying to store each field in an array, then trying to read it. However, its going in an infinite loop, I think. Anyone knows what I am doing wrong?
#!/bin/bash
test(){
local file="/Users/f.json"
if [ -f "$file" ]; then
echo "present"
else
echo "absent"
fi
#jq . f.json
while read rule; do
local idd
local username
idd=$(jq --raw-output '.id' <<< ${rule})
username=$(jq --raw-output '.username' <<< ${rule})
#username=$(jq --raw-output '.username')
done
for (( i=0; i<${#idd[#]}; i++ )); do
echo "${idd[i]}"
done
}
test
Here is json:
{
"id": 5679162,
"username": "ryderw1"
}
{
"id": 5679163,
"username": "ryderw3"
}
{
"id": 5679164,
"username": "ryderw4"
}
My desired o/p should be:
5679162
5679163
5679164
I suggest this to read output from jq to an array.
mapfile -t idd < <(jq '.id' /Users/f.json)
declare -p idd
Output:
declare -a idd=([0]="5679162" [1]="5679163" [2]="5679164")
Related
Let's say that with a bash script I want to create a file, so to the command to create the file will be something like this:
myscript hostgroup_one 2
hostgroup_one and the number 2 are parameters.
How can I insert the parameters in the lines below and output all the lines as a file?
{
"bool": {
"should": [
{
"match_phrase": {
"hostgroup.keyword": "$1"
}
}
],
"minimum_should_match": $2
}
}
I'd use jq to build the JSON:
jq -n \
--arg hostgroup "$1" \
--argjson minimum "$2" \
'{bool: {should: [{match_phrase: {"hostgroup.keyword": $hostgroup}}], minimum_should_match: $minimum}}'
will produce your desired output.
While jq is a great tool for manipulating json, as glenn jackman recommends, if you don't have it and your sysadmin won't install it (?!?)...
You can use a "here document"
Your myscript could look something like this:
#!/bin/bash
echo "dollar-1 is '${1}' dollar-2 is '${2}' dollar-3 is '${3}'"
cat <<EOF >"${1}"
{
"bool": {
"should": [
{
"match_phrase": {
"hostgroup.keyword": "${2}"
}
}
],
"minimum_should_match": ${3}
}
}
EOF
echo "done"
I've made the output filename the first parameter, and then your two parameters, so it would be run like:
myscript output.json hostgroup_one 2
You don't need to do that, you could use 2 params and redirect output:
myscript hostgroup_one 2 > output.json
(note you don't have to use EOF as your here-document delimiter, I just like it)
Of course you don't need the echo statements, and you should have error checking (does ${#} equal 3 parameters?)
I'm parsing a JSON response with a tool called jq.
The output from jq will give me a list of full names in my command line.
I have the variable getNames which contains JSON, for example:
{
"count": 49,
"user": [{
"username": "jamesbrown",
"name": "James Brown",
"id": 1
}, {
"username": "matthewthompson",
"name": "Matthew Thompson",
"id": 2
}]
}
I pass this through JQ to filter the json using the following command:
echo $getNames | jq -r .user[].name
Which gives me a list like this:
James Brown
Matthew Thompson
I want to put each one of these entries into a bash array, so I enter the following commands:
declare -a myArray
myArray=( `echo $getNames | jq -r .user[].name` )
However, when I try to print the array using:
printf '%s\n' "${myArray[#]}"
I get the following:
James
Brown
Matthew
Thompson
How do I ensure that a new index is created after a new line and not a space? Why are the names being separated?
Thanks.
A simple script in bash to feed each line of the output into the array myArray.
#!/bin/bash
myArray=()
while IFS= read -r line; do
[[ $line ]] || break # break if line is empty
myArray+=("$line")
done < <(jq -r .user[].name <<< "$getNames")
# To print the array
printf '%s\n' "${myArray[#]}"
Just use mapfile command to read multiple lines into an array like this:
mapfile -t myArray < <(jq -r .user[].name <<< "$getNames")
I am trying to use a bash variable to store json.
testConfig=",{
\"Classification\": \"mapred-site\",
\"Properties\": {
\"mapreduce.map.java.opts\": \"-Xmx2270m\",
\"mapreduce.map.memory.mb\": \"9712\"
}
}"
echo $testConfig
Output: ,{
If I give it in a single line it works.
But i would like to store values in my variable in a clean format.
I tried using cat >>ECHO
That didn't work either
Any help is appreciated as to how I can store this in order to get the output in an expected format.
Thanks.
You may use a here doc as described here:
read -r -d '' testConfig <<'EOF'
{
"Classification": "mapred-site",
"Properties": {
"mapreduce.map.java.opts": "-Xmx2270m",
"mapreduce.map.memory.mb": "9712"
}
}
EOF
# Just to demonstrate that it works ...
echo "$testConfig" | jq .
Doing so you can avoid quoting.
You can use read -d
read -d '' testConfig <<EOF
,{ /
"Classification": "mapred-site",
"Properties": {/
"mapreduce.map.java.opts": "-Xmx2270m",
"mapreduce.map.memory.mb": "9712"
}
}
EOF
echo $testConfig;
I got a Json string as:
{
"id": 3397,
"title": "title_1"
}
{
"id": 3396,
"title": "title_2"
}
what I want to do is get every id in a loop,
I use the following code :
for (( i = 0; i < requestCount; i++ )); do
requestId=$(echo $jsonString[$i] | jq '.id')
echo requestId;
done
but it doesn't work, I think the way I use variable is wrong, I can't find anything useful here jq.
Let jq do the iterating. (That is, let jq do the iterating through the input stream of JSON objects.) For example:
$ jq .id <<< "$json" | while read id ; do echo "hello $id"; done
Output:
hello 3397
hello 3396
This way, you don't have to know how many JSON objects are in the input. You might want to use "read -r", or "IFS= read -r".
The alternatives are ugly and inefficient, e.g.:
$ for ((i=0;i<2;i++)) ; do jq -s --argjson i "$i" '.[$i].id' <<< "$json" ; done
I've to call a file and pass a json as parameters in this way
(suppose that my file is called test.sh), from bash I need to do something like this:
./test.sh "[{\"username\":\"user1\",\"password\":\"pwd1\",\"group\":\"usergroup1\"},{\"username\":\"user2\",\"password\":\"pwd2\",\"group\":\"usergroup2\"},{\"username\":\"user3\",\"password\":\"pwd3\",\"group\":\"usergroup3\"}]"
and the content of test.sh is the following
#!/bin/bash
#read the json
system_user="$1"
printf "$system_user"
accounts=($(jq -s ".[]" <<< $system_user))
printf "$accounts"
for account in "${accounts[#]}"
do
printf "\n\n$account\n\n"
done
the output of -> printf "$system_user" is
[{"username":"user1","password":"pwd1","group":"usergroup1"},{"username":"user2","password":"pwd2","group":"usergroup2"},{"username":"user3","password":"pwd3","group":"usergroup3"}]
but the output of -> printf "$accounts" is something like this
[
[
{
"username":
"user1"
etc. etc. one object for each token :-(
and so on, but what I was expecting is an array of three object (like you can test on jqplay.org)
{
"username": "user1",
"password": "pwd1",
"group": "usergroup1"
}
{
"username": "user2",
"password": "pwd2",
"group": "usergroup2"
}
{
"username": "user3",
"password": "pwd3",
"group": "usergroup3"
}
In this way I can make a foreach on ${accounts[#]}
What I'm doing wrong?
Thank you
With the -c option, you can print each JSON object on a single line, making it easier to populate the array you want.
$ readarray -t arr < <(jq -c '.[]' <<< "[{\"username\":\"user1\",\"password\":\"pwd1\",\"group\":\"usergroup1\"},{\"username\":\"user2\",\"password\":\"pwd2\",\"group\":\"usergroup2\"},{\"username\":\"user3\",\"password\":\"pwd3\",\"group\":\"usergroup3\"}]")
$ printf "Object: %s\n" "${arr[#]}"
Object: {"username":"user1","password":"pwd1","group":"usergroup1"}
Object: {"username":"user2","password":"pwd2","group":"usergroup2"}
Object: {"username":"user3","password":"pwd3","group":"usergroup3"}
You are interchanging bash arrays and JSON arrays. When you are creating accounts array, bash splits the elements per each whitespace. That's why you don't get what you expect. You can try the following:
declare -A accounts
while IFS="=" read -r key value
do
accounts[$key]="$value"
done < <(jq -r "to_entries|map(\"\(.key)=\(.value)\")|.[]" <<< $system_user)
for account in "${accounts[#]}"
do
printf "$account\n"
done
(stolen from here: https://stackoverflow.com/a/26717401/328977)
to get the following output:
{"username":"user1","password":"pwd1","group":"usergroup1"}
{"username":"user2","password":"pwd2","group":"usergroup2"}
{"username":"user3","password":"pwd3","group":"usergroup3"}