I pulled some config variables from a json file with jq.
And once modified, i want to write the whole config array (which can contain keys that were not there at first) to the json file.
The "foreach" part seems quite obvious.
But how to express "change keyA by value A or add keyA=>valueA" to the conf file ?
I'm stuck with something like
for key in "${!conf[#]}"; do
value=${conf[$key]}
echo $key $value
jq --arg key $key --arg value $value '.$key = $value' $conf_file > $new_file
done
Thanks
Extended solution with single jq invocation:
Sample conf array:
declare -A conf=([status]="finished" [nextTo]="2018-01-24" [result]=true)
Sample file conf.json:
{
"status": "running",
"minFrom": "2018-01-23",
"maxTo": "2018-01-24",
"nextTo": "2018-01-23",
"nextFrom": "2018-01-22"
}
Processing:
jq --arg data "$(paste -d':' <(printf "%s\n" "${!conf[#]}") <(printf "%s\n" "${conf[#]}"))" \
'. as $conf | map($data | split("\n")[]
| split(":") | {(.[0]) : .[1]})
| add | $conf + .' conf.json > conf.tmp && mv conf.tmp conf.json
The resulting conf.json contents:
{
"status": "finished",
"minFrom": "2018-01-23",
"maxTo": "2018-01-24",
"nextTo": "2018-01-24",
"nextFrom": "2018-01-22",
"result": "true"
}
I finally run into the following solution : not as compact as RomanPerekhrest, but a bit more human-readable. Thanks.
function update_conf() {
echo update_conf
if [ -n "$1" ] && [ -n "$2" ]; then
conf[$1]=$2
fi
for key in "${!conf[#]}"; do
value=${conf[$key]}
jq --arg key "$key" --arg value "$value" '.[$key] = $value' $confFile > $confTempFile && mv $confTempFile $confF$
done
}
Related
I am trying to get certain value in the json with some for each and if condition.
code looks like the following
for i in "$(jq -r '.value[]' test.json)"; do
result=$(echo $i | jq -r .result)
if [ "$result" == "succeeded" ]
then
name=$(echo $i | jq -r .agent.name)
echo "$name"
fi
done
json file - test.json
{
"count":3,
"value":[
{
"location":"CA",
"result":"failed",
"agent":{
"id":97833,
"name":"Brad"
},
"priority":0
},
{
"location":"TX",
"result":"failed",
"agent":{i
"id":15232,
"name":"Tom"
},
"priority":0
},
{
"location":"CO",
"result":"succeeded",
"agent":{
"id":13412,
"name":"John"
},
"priority":0
}
]
}
I am trying to loop through the json file using jq to get the name of the agent if the result was "succeeded". but the result i get from result=$(echo $i | jq -r .result) seems like it's returning string array #("failed","failed","succeeded").
========================update==============================
jq ' # get the name of the agent if the result was "succeeded".
.value[]
| select(.result == "succeeded")
| var1=.agent.name
| echo $var1
' test.json
if I want to save the .agent.name into a variable and use it with a different command, what do I have to do?
There is no need to use a shell loop.
With your test.json,
jq ' # get the name of the agent if the result was "succeeded".
.value[]
| select(.result == "succeeded")
| .agent.name
' test.json
produces:
"John"
I can create a json object with an array using jq like this
echo '{"input":{"names":[]}}' | jq --arg val "alice" '.input.names[0] += $val'| jq --arg val "bob" '.input.names[1] += $val'
which gives
{
"input": {
"names": [
"alice",
"bob"
]
}
}
Now I want to recreate this in a shell script where "names" come from a shell array
#!/bin/sh
names=( alice bob )
ITER=0
for i in "${names[#]}"
do
echo '{"input":{"names":[]}}' | jq --arg val "$i" '.input.names[$ITER] += $val'
echo ${I} ${ITER}
ITER=$(expr $ITER + 1)
done
but I run into
jq: error: $ITER is not defined at <top-level>, line 1:
.input.names[$ITER] += $val
jq: 1 compile error
0
jq: error: $ITER is not defined at <top-level>, line 1:
.input.names[$ITER] += $val
jq: 1 compile error
1
You could get rid of the shell loop and use one jq call:
names=( alice bob )
jq -n --arg names "${names[*]}" '{"input": {"names": ($names / " ") } }'
or
names=( alice bob )
printf '%s\0' "${names[#]}" |
jq -sR '(. / "\u0000") as $names | { "input": { "names": $names } }'
You don't capture the output of an iteration step, so you lose the output from the jq call and start all over again in the next iteration step.
Here's a bash solution which captures the output using $(...) and provides it for the next using <<<. (Note that there are better ways to solve this problem, e.g. without looping but by proving jq all elements to be added at once.)
json='{"input":{"names":[]}}'
for i in alice bob
do json="$(jq --arg val "$i" '.input.names += [$val]' <<< "$json")"
done
echo "$json"
{
"input": {
"names": [
"alice",
"bob"
]
}
}
I'm about to lose my mind working with jq for the first time today. I've tried every ever so dirty way to make this work somehow. Let's get down to business:
I'm trying to further develop the pretty barebones API for Unifi Controllers for some use cases:
https://dl.ui.com/unifi/6.0.41/unifi_sh_api
Here's some mock up data (the real json data is thousands of lines):
{
"meta": {
"rc": "ok"
},
"data": [
{
"_id": "61a0da77f730e404af0edc3c",
"ip": "10.19.31.120",
"mac": "ff:ec:ff:89:ff:58",
"name": "Test-Accesspoint"
}
]
}
Here is my advancedapi.sh
#!/bin/bash
source unifiapi.sh
unifi_login 1>dev/null
if [ "$1" ];
then
command=$1
shift
else
echo "Please enter command"
exit
fi
#jq -r '.data[] | "\(.name),\(.mac)"'
assembleKeys(){
c="0"
seperator=";"
keys=$(for i in "$#"
do
c=$(expr $c + 1)
if [ $c -gt 1 ];
then
echo -n "${seperator}\(.${i})"
else
echo -n "\(.${i})"
fi
done
echo)
}
assembleJQ(){
c=0
jq=$(echo -en 'jq -r -j \x27.data[] | '
for i in "$#"
do
if [ "$c" != "0" ];
then
echo -en ",\x22 \x22,"
fi
c=1
echo -en ".$i"
done
echo -en ",\x22\\\n\x22\x27")
echo "$jq"
}
getDeviceKeys(){
assembleJQ "$#"
#unifi_list_devices | jq -r '.data[]' | jq -r "${keys}"
#export keys=mac
#export second=name
#unifi_list_devices | jq -r -j --arg KEYS "$keys" --arg SECOND "$second" '.data[] | .[$KEYS]," ",.[$SECOND],"\n"'
unifi_list_devices | $jq
#unifi_list_devices | jq -r -j '.data[] | .mac," ",.name,"\n"'
}
"$command" $#
Users should be able to call any function from the API with as many arguments as they want.
So basically this:
#export keys=mac
#export second=name
#unifi_list_devices | jq -r -j --arg KEYS "$keys" --arg SECOND "$second" '.data[] | .[$KEYS]," ",.[$SECOND],"\n"'
which works, but only with a limited number of arguments. I want to pass $# to jq.
I'm trying to get the name and mac-address of a device by calling:
./advancedapi.sh getDeviceKeys mac name
this should return the mac-address and name of the device. But it only gives me this:
jq -r -j '.data[] | .mac," ",.name,"\n"'
jq: error: syntax error, unexpected INVALID_CHARACTER, expecting $end (Unix shell quoting issues?) at <top-level>, line 1:
'.data[]
jq: 1 compile error
The top-most line being the jq command, which works perfectly fine when called manually.
The assembleKeys function was my attempt at generating a string that looks like this:
jq -r '.data[] | "\(.name),\(.mac)"'
Can anyone explain to me, preferably in-depth, how to do this? I'm going insane here!
Thanks!
I want to pass $# to jq.
There are actually many ways to do this, but the simplest might be using the --args command-line option, as illustrated here:
File: check
#/bin/bash
echo "${#}"
jq -n '$ARGS.positional' --args "${#}"
Example run:
./check 1 2 3
1 2 3
[
"1",
"2",
"3"
]
Projection
Here's an example that might be closer to what you're looking for.
Using your sample JSON:
File: check2
#/bin/bash
jq -r '
def project($array):
. as $in | reduce $array[] as $k ([]; . + [$in[$k]]);
$ARGS.positional,
(.data[] | project($ARGS.positional))
| #csv
' input.json --args "${#}"
./check2 name mac
"name","mac"
"Test-Accesspoint","ff:ec:ff:89:ff:58"
There is error in the json (red highlighted commas)(that commas are not needed)
$
$ cat j.json | jq -r '.data[] | "\(.name)█\(.mac)█\(.ip)█\(._id)"'
Test-Accesspoint█ff:ec:ff:89:ff:58█10.19.31.120█61a0da77f730e404af0edc3c
$ cat j.json
{
"meta": {
"rc": "ok"
},
"data": [
{
"_id": "61a0da77f730e404af0edc3c",
"ip": "10.19.31.120",
"mac": "ff:ec:ff:89:ff:58",
"name": "Test-Accesspoint"
}
]
}
$
If the api have given this json you should probably edit it, before piping it into jq [maybe with sed or something like it]
I have a bash associative array containing dynamic data like:
declare -A assoc_array=([cluster_name]="cpod1" [site_name]="ppod1" [alarm_name]="alarm1")
I have to create the JSON data accordingly.
{
"name": "cluster_name",
"value": $assoc_array[cluster_name],
"regex": False
}
{
"name": "site_name",
"value": $assoc_array[site_name],
"regex": False
}
{
"name": "alert_name",
"value": $assoc_array[alert_name],
"regex": False
}
I have used the following code for it:
for i in "${!assoc_array[#]}"; do
echo $i
alarmjsonarray=$(echo ${alarmjsonarray}| jq --arg name "$i" \
--arg value "${alarm_param[$i]}" \
--arg isRegex "False" \
'. + [{("name"):$name,("value"):$value,("isRegex"):$isRegex}]')
done
echo "alarmjsonarray" $alarmjsonarray
I am getting empty string from it. Can you please help me in it?
Assuming the ASCII record separator character \x1e can't occur in your data (and given your assertions that newlines will never be present), one way to handle this would be:
for key in "${!assoc_array[#]}"; do
printf '%s\x1e%s\n' "$key" "${assoc_array[$key]}"
done | jq -Rn '
[
inputs |
split("\u001e") | .[0] as $key | .[1] as $value |
{"name": $key, "value": $value, "regex": false}
]'
...feel free to change \x1e to a different character (like a tab) that can never exist, as appropriate.
#!/bin/bash
declare -A assoc_array=([cluster_name]="cpod1" [site_name]="ppod1" [alarm_name]="alarm1")
for i in "${!assoc_array[#]}"; do
alarmjsonarray=$(echo "${alarmjsonarray:-[]}" |
jq --arg name "$i" \
--arg value "${assoc_array[$i]}" \
--arg isRegex "False" \
'. + [{("name"):$name,("value"):$value,("isRegex"):$isRegex}]')
done
echo "alarmjsonarray" "$alarmjsonarray"
array was supposed to be initialized before using it. I thought in bash, scope is there out of block also.
Its more simpler way to generate json from bash associative array.
Here is my config.json:
{
"env": "dev",
"dev": {
"projects" : {
"prj1": {
"dependencies": {},
"description": ""
}
}
}
}
Here are my bash commands:
PRJNAME='prj1'
echo $PRJNAME
jq --arg v "$PRJNAME" '.dev.projects."$v"' config.json
jq '.dev.projects.prj1' config.json
The output:
prj1
null
{
"dependencies": {},
"description": ""
}
So $PRJNAME is prj1, but the first invocation only outputs null.
Can someone help me?
The jq program .dev.projects."$v" in your example will literally try to find a key named "$v". Try the following instead:
jq --arg v "$PRJNAME" '.dev.projects[$v]' config.json
You can use --argjson too when you make your json.
--arg a v # set variable $a to value <v>;
--argjson a v # set variable $a to JSON value <v>;
As asked in a comment above there's a way to pass multiple argumets.
Maybe there's a more elegant way, but it works.
If you are sure always all keys needed you can use this:
jq --arg key1 $k1 --arg key2 $k2 --arg key3 $k3 --arg key4 $k4 '.[$key1] | .[$key2] | .[$key3] | .[$key4] '
If the key isn't always used you could do it like this:
jq --arg key $k ' if key != "" then .[$key] else . end'
If key sometimes refers to an array:
jq --arg key $k ' if type == "array" then .[$key |tonumber] else .[$key] end'
of course you can combine these!
you can do this:
key="dev.projects.prj1"
filter=".$key"
cat config.json | jq $filter
My example bash command to replace an array in a json file.
Everything is in variables:
a=/opt/terminal/conf/config.json ; \
b='["alchohol"]' ; \
c=terminal.canceledInspections.resultSendReasons ; \
cat $a | jq .$c ; \
cat $a | jq --argjson b $b --arg c $c 'getpath($c / ".") = $b' | sponge $a ; \
cat $a | jq .$c