Adding values to dynamic keys with jq - json

I try to construct a json object with jq. I start with an empty object and want to add keys and values dynamically.
This works but the key is not variable. It's fixed to "foo":
echo '{"foo": ["baz"]}' | jq --arg value "bar" '.foo += [$value]'
output as expected:
{"foo": ["baz", "bar"]}
What I actually want do do is something like this:
echo '{"foo": ["baz"]}' | jq --arg key "foo" --arg value "bar" '.($key) += [$value]'
Unfortunately this does not work. Here is the output:
jq: error: syntax error, unexpected '(' (Unix shell quoting issues?) at <top-level>, line 1:
.($key) += [$value]
jq: error: try .["field"] instead of .field for unusually named fields at <top-level>, line 1:
.($key) += [$value]
jq: 2 compile errors
I couldn't find a solution or figure it out.
I know that this works: jq --null-input --arg key foo '{($key): "bar"}' but it doesn't solve my problem since I want to append values to existing lists as you can see in the examples.

You need to use square parens [..] instead of (..) as reported in the error message. Just do
jq --arg key "foo" --arg value "bar" '.[$key] += [$value]'
This error line is quite verbose to recommend you the right syntax to use. The emphasis with # is mine
jq: error: try .["field"] instead of .field for unusually named fields at <top-level>, line 1
# ^^^^^^^^^^^^^

Related

ANSI color codes with jq

Trying to make jq work with ANSI color codes.
Test cases:
$ echo '{"a":"b","c":"d"}' | jq -r .c
d # Matches my expected output
$ echo '{"a":"b","c":"\033[31md\033[0m"}' | jq -r .c
parse error: Invalid escape at line 1, column 31 # returns err code 4
$ echo '{"a":"b","c":"d"}' | jq -r '"foo"+.c+"bar"'
foodbar # Correct
$ echo '{"a":"b","c":"d"}' | jq -r '"\033[31m"+.c+"\033[0m"'
jq: error: Invalid escape at line 1, column 4 (while parsing '"\0"') at <top-level>, line 1:
"\033[31m"+.c+"\033[0m"
jq: error: Invalid escape at line 1, column 4 (while parsing '"\0"') at <top-level>, line 1:
"\033[31m"+.c+"\033[0m"
jq: 2 compile errors # returns err code 3
$ jq -rn '"\033[31mbar\033[0m"'
jq: error: Invalid escape at line 1, column 4 (while parsing '"\0"') at <top-level>, line 1:
"\033[31mbar\033[0m"
jq: error: Invalid escape at line 1, column 4 (while parsing '"\0"') at <top-level>, line 1:
"\033[31mbar\033[0m"
jq: 2 compile errors # returns err code 4
P.S. in case it matters, I am using the bash shell with version 5.1.16(1)-release on Linux.
Conslusion: ANSI colors do not work with jq, whether in the JSON string or directly concatenating it through the + operator.
Question: how to make ANSI colors work in jq? Any help would be appreciated.
Octal escape sequences are not valid JSON syntax, so you need to encode the ASCII escape character as \u001b rather than \033. Also, to add to the confusion, some versions of echo will attempt to interpret backslash (escape) sequences itself before passing them to jq, so in cases like this it's much safer to use printf '%s\n':
$ printf '%s\n' '{"a":"b","c":"\u001b[31md\u001b[0m"}' | jq -r .c
d
(You can't see it, but that "d" is red in my terminal.)
BTW, an easy way to find things like this out is to get jq to encode them in JSON for you. Here, I'll set the shell variable to the actual string (using bash's $'...' string format, which interprets ANSI-C escape sequences like \033), then use --arg to pass that to jq:
$ seq=$'\033[31md\033[0m'
$ jq -nc --arg seq "$seq" '{"a":"b","c":$seq}'
{"a":"b","c":"\u001b[31md\u001b[0m"}

jq not writing value of variable but the actual text of the variable name

I'm scripting in bash to edit a json template to replace some field values with arguments to my script, and trying to use jq to do the editing. My code is not replacing the --arg with the value of the argument, but the literal text of the argument name.
My template contains:
{
"name":""
}
My jq code:
jq --arg ad "192.168.5.5" -r '.name = "Addr $ad"' address.tmpl
This outputs:
{
"name": "Addr $ad"
}
Or, if I remove the double-quotes
jq --arg ad "192.168.5.5" -r '.name = Addr $ad' address.tmpl
I get
jq: error: syntax error, unexpected '$', expecting $end (Unix shell quoting issues?) at <top-level>, line 1:
.name = Addr $ad
jq: 1 compile error
According to all that I have read, this should work. What am I doing wrong/how do I fix this????
OS = debian 10
In the jq manual, search for "String interpolation"
jq --arg ad "192.168.5.5" -r '.name = "Addr \($ad)"'
jq --arg ad 192.168.5.5 -r '.name = "Addr " + $ad' address.tmpl
$ad will be expanded by the shell if it is not not hard quoted. In jq, you can use string interpolation "Addr \($ad)", or concatenation (as above), which I find slightly more readable.

JQ: add variable property to existing object

I'm trying to add variable properties to some existing json
{
"item1": {
"proerty1": "test"
},
"item2": {}
}
So if I do something like this it works
echo $contents | jq --arg ITEM1 $item1 '.[$ITEM1].property2 = "test2"'
But when I try to add more arguments like this it fails:
echo $contents | jq --arg ITEM1 $item1 --arg PROPERTY2 $property2 --arg VALUE $value '.[$ITEM1].[PROPERTY2] = $VALUE'
The error I get is:
jq: error: syntax error, unexpected '[', expecting FORMAT or QQSTRING_START (Unix shell quoting issues?) at <top-level>, line 1:
.[$ITEM1].[PROPERTY2] = $VALUE
jq: 1 compile error
So I guess += operator wouldn't be the correct way to do this with variables. What would be the correct way to add a propetry where the whole path .item.property and the value itself are variable
The jq filter should be:
.[$ITEM1][$PROPERTY2] = $VALUE
Your query has an extra ..
An alternative:
You could also use setpath/2, e.g.
setpath([$ITEM1,$PROPERTY2]; $VALUE)
Aside
It's usually best to quote your shell variables, e.g.
echo "$contents" ...

jq filter expression in string interpolation

I have been trying to reduce the array to a string to be used in string interpolation.
For example.
input = ["123", "456"]
expected output = array=123,456
Here is my try
$ echo '["123", "456"]' | jq 'array=\(.|join(","))'
jq: error: syntax error, unexpected INVALID_CHARACTER (Unix shell quoting issues?) at <top-level>, line 1:
array=\(.|join(","))
jq: 1 compile error
Using string interpolation \(.), you can do something like below. Your idea is almost right, but interpolation using \(..) needs the filter to present be inside a string with the expression to be used defined inside parens after a backslash
jq --raw-output '"array=\(join(","))"'
echo '["123", "456"]' | jq -r '"array=" + join(",")'

Select and replace field of key referenced by environment variable with special character using jq

Given the json below:
./versions.json
{
"my-app-1": "v1.0.0",
"my-app-2": "v0.9.1",
"my-app-3": "v2.1.7"
}
I want to replace the version of $APP_NAME to the new version $NEW_VERSION. Given APP_NAME=my-app-2 and NEW_VERSION=v1.0.0, I tried the following:
jq '(."$APP_NAME") = "$NEW_VERSION"' ./versions.json > updated_versions.json
which gives:
./updated_versions.json
{
"my-app-1": "v1.0.0",
"my-app-2": "v0.9.1",
"my-app-3": "v2.1.7",
"$APP_NAME": "$NEW_VERSION"
}
this:
jq "(.$APP_NAME) = \"$NEW_VERSION\"" versions.json > updated_versions.json
gives:
jq: error: app/0 is not defined at <top-level>, line 1:
(.my-app-1) = "v1.0.0"
jq: 1 compile error
How can I escape the special character in the environment variable? I have tried setting APP_NAME=my\\-app\\-1 with no luck.
Thanks
Pass the two variables to jq using its --arg option :
jq --arg appName my-app-2 --arg newVersion v1.0.0 '.[$appName]=$newVersion'
In your first try the jq command was enclosed in single-quotes, so it was left to jq to resolve the variables, but it doesn't look for them in the outer shell context.
Your second try was nearly good (but not very good practice) because the variables were expanded by the shell, but my-app-1 contains special characters (the dashes) and needs to be accessed with either ."my-app-1" or .["my-app-1"].