I'm trying to add variable properties to some existing json
{
"item1": {
"proerty1": "test"
},
"item2": {}
}
So if I do something like this it works
echo $contents | jq --arg ITEM1 $item1 '.[$ITEM1].property2 = "test2"'
But when I try to add more arguments like this it fails:
echo $contents | jq --arg ITEM1 $item1 --arg PROPERTY2 $property2 --arg VALUE $value '.[$ITEM1].[PROPERTY2] = $VALUE'
The error I get is:
jq: error: syntax error, unexpected '[', expecting FORMAT or QQSTRING_START (Unix shell quoting issues?) at <top-level>, line 1:
.[$ITEM1].[PROPERTY2] = $VALUE
jq: 1 compile error
So I guess += operator wouldn't be the correct way to do this with variables. What would be the correct way to add a propetry where the whole path .item.property and the value itself are variable
The jq filter should be:
.[$ITEM1][$PROPERTY2] = $VALUE
Your query has an extra ..
An alternative:
You could also use setpath/2, e.g.
setpath([$ITEM1,$PROPERTY2]; $VALUE)
Aside
It's usually best to quote your shell variables, e.g.
echo "$contents" ...
Related
I'm scripting in bash to edit a json template to replace some field values with arguments to my script, and trying to use jq to do the editing. My code is not replacing the --arg with the value of the argument, but the literal text of the argument name.
My template contains:
{
"name":""
}
My jq code:
jq --arg ad "192.168.5.5" -r '.name = "Addr $ad"' address.tmpl
This outputs:
{
"name": "Addr $ad"
}
Or, if I remove the double-quotes
jq --arg ad "192.168.5.5" -r '.name = Addr $ad' address.tmpl
I get
jq: error: syntax error, unexpected '$', expecting $end (Unix shell quoting issues?) at <top-level>, line 1:
.name = Addr $ad
jq: 1 compile error
According to all that I have read, this should work. What am I doing wrong/how do I fix this????
OS = debian 10
In the jq manual, search for "String interpolation"
jq --arg ad "192.168.5.5" -r '.name = "Addr \($ad)"'
jq --arg ad 192.168.5.5 -r '.name = "Addr " + $ad' address.tmpl
$ad will be expanded by the shell if it is not not hard quoted. In jq, you can use string interpolation "Addr \($ad)", or concatenation (as above), which I find slightly more readable.
Am trying to add a json array with just one array element to a json. And it has to be added only if it does not already exist.
Example json is below.
{
"lorem": "2.0",
"ipsum": {
"key1": "value1",
"key2": "value2"
},
"schemes": ["https"],
"dorum" : "value3"
}
Above is the json, where "schemes": ["https"], exists. Am trying to add schemes only if it does not exist using the below code.
scheme=$( cat rendertest.json | jq -r '. "schemes" ')
echo $scheme
schem='["https"]'
echo "Scheme is"
echo $schem
if [ -z $scheme ]
then
echo "all good"
else
jq --argjson argval "$schem" '. += { "schemes" : $schem }' rendertest.json > test.json
fi
I get the below error in a file when the json array element 'schemes' does not exist. It returns a null and errors out. Any idea where am going wrong?
null
Scheme is
["https"]
jq: error: schem/0 is not defined at <top-level>, line 1:
. += { "schemes" : $schem }
jq: 1 compile error
Edit: the question is not about how to pass on bash variables to jq.
Just use an explicit if condition that checks for the attribute schemes in the root level of the JSON structure
schem='["https"]'
After setting the above variable in your shell, run the following filter
jq --argjson argval "$schem" 'if has("schemes")|not then .+= { schemes: $argval } else . end' json
The argument immediately after the --argjson field is the one that needs to be used in the context of jq, but you were trying to use $schem in the context which is incorrect.
You can even go one level further and check even if schemes is present and if it does not contain the value you expect, then make the overwrite. Modify the filter within '..' to
( has("schemes")|not ) or .schemes != $argval )
which can be run as
jq --argjson argval "$schem" 'if ( (has("schemes")|not) or .schemes != $argval) then (.schemes: $argval) else . end'
I have been trying to reduce the array to a string to be used in string interpolation.
For example.
input = ["123", "456"]
expected output = array=123,456
Here is my try
$ echo '["123", "456"]' | jq 'array=\(.|join(","))'
jq: error: syntax error, unexpected INVALID_CHARACTER (Unix shell quoting issues?) at <top-level>, line 1:
array=\(.|join(","))
jq: 1 compile error
Using string interpolation \(.), you can do something like below. Your idea is almost right, but interpolation using \(..) needs the filter to present be inside a string with the expression to be used defined inside parens after a backslash
jq --raw-output '"array=\(join(","))"'
echo '["123", "456"]' | jq -r '"array=" + join(",")'
I try to construct a json object with jq. I start with an empty object and want to add keys and values dynamically.
This works but the key is not variable. It's fixed to "foo":
echo '{"foo": ["baz"]}' | jq --arg value "bar" '.foo += [$value]'
output as expected:
{"foo": ["baz", "bar"]}
What I actually want do do is something like this:
echo '{"foo": ["baz"]}' | jq --arg key "foo" --arg value "bar" '.($key) += [$value]'
Unfortunately this does not work. Here is the output:
jq: error: syntax error, unexpected '(' (Unix shell quoting issues?) at <top-level>, line 1:
.($key) += [$value]
jq: error: try .["field"] instead of .field for unusually named fields at <top-level>, line 1:
.($key) += [$value]
jq: 2 compile errors
I couldn't find a solution or figure it out.
I know that this works: jq --null-input --arg key foo '{($key): "bar"}' but it doesn't solve my problem since I want to append values to existing lists as you can see in the examples.
You need to use square parens [..] instead of (..) as reported in the error message. Just do
jq --arg key "foo" --arg value "bar" '.[$key] += [$value]'
This error line is quite verbose to recommend you the right syntax to use. The emphasis with # is mine
jq: error: try .["field"] instead of .field for unusually named fields at <top-level>, line 1
# ^^^^^^^^^^^^^
I'm trying to extract a series of properties (named in an input file) in jq and getting error when I feed those from bash via a loop:
while read line; do echo $line; cat big.json | jq ".$line"; sleep 1; done < big.properties.service
cfg.keyload.service.count
jq: error: syntax error, unexpected INVALID_CHARACTER, expecting $end (Unix shell quoting issues?) at <top-level>, line 1:
When i try to do it manually it works
$ line=cfg.keyload.service.count
$ echo $line
cfg.keyload.service.count
$ cat big.json | jq ".$line"
1
Is there any way to get it work in loop?
Here is example
cat >big.json <<EOF
{
"cfg": {
"keyload": {
"backend": {
"app": {
"shutdown": {
"timeout": "5s"
},
"jmx": {
"enable": true
}
}
}
}
}
}
EOF
cat >big.properties.service <<EOF
cfg.keyload.backend.app.shutdown.timeout
cfg.keyload.backend.app.jmx.enable
cfg.keyload.backend.app.jmx.nonexistent
cfg.nonexistent
EOF
...output should be:
cfg.keyload.backend.app.shutdown.timeout
"5s"
cfg.keyload.backend.app.jmx.enable
true
cfg.keyload.backend.app.jmx.nonexistent
null
cfg.nonexistent
null
Immediate Issue - Invalid Input
The "invalid character" at hand here is almost certainly a carriage return. Use dos2unix to convert your input file to a proper UNIX text file, and your original code will work (albeit very inefficiently, rereading your whole big.json every time it wants to extract a single property).
Performant Implementation - Loop In JQ, Not Bash
Don't use a bash loop for this at all -- it's much more efficient to have jq do the looping.
Note the sub("\r$"; "") used in this code to remove trailing carriage returns so it can accept input in DOS format.
jq -rR --argfile infile big.json '
sub("\r$"; "") as $keyname
| ($keyname | split(".")) as $pieces
| (reduce $pieces[] as $piece ($infile; .[$piece]?)) as $value
| ($keyname, ($value | tojson))
' <big.properties.service
properly emits as output, when given the inputs in the question:
cfg.keyload.backend.app.shutdown.timeout
"5s"
cfg.keyload.backend.app.jmx.enable
true
cfg.keyload.backend.app.jmx.nonexistent
null
cfg.nonexistent
null
Your properties file is effectively paths in the json that you want to retrieve values from. Convert them to paths that jq recognizes so you can get those values. Just make an array of keys that would need to be traversed. Be sure to read your properties file as raw input (-R) since it's not json, and use raw output (-r) to be able to output the paths as you want.
$ jq --argfile big big.json '
., (split(".") as $p | $big | getpath($p) | tojson)
' -Rr big.properties.service
cfg.keyload.backend.app.shutdown.timeout
"5s"
cfg.keyload.backend.app.jmx.enable
true
cfg.keyload.backend.app.jmx.nonexistent
null
cfg.nonexistent
null