I want to populate json template with value "Hello Jack", but the "Hello" part shall remain inside of template, is there are any way of doing that, I've tried code below but it gives me error:
jq -n --arg person "Jack" '{my_key: "Hello "$person}'
jq: error: syntax error, unexpected '$', expecting '}' (Unix shell quoting issues?) at <top-level>, line 1:
Use string interpolation syntax like so:
jq -n --arg person Jack '{my_key: "Hello \($person)"}'
And to load the template from a file, use the -f switch:
$ cat template.json
{
"my_key": "Hello \($person)"
}
$ jq -n --arg person Jack -f template.json
{
"my_key": "Hello Jack"
}
Related
Trying to make jq work with ANSI color codes.
Test cases:
$ echo '{"a":"b","c":"d"}' | jq -r .c
d # Matches my expected output
$ echo '{"a":"b","c":"\033[31md\033[0m"}' | jq -r .c
parse error: Invalid escape at line 1, column 31 # returns err code 4
$ echo '{"a":"b","c":"d"}' | jq -r '"foo"+.c+"bar"'
foodbar # Correct
$ echo '{"a":"b","c":"d"}' | jq -r '"\033[31m"+.c+"\033[0m"'
jq: error: Invalid escape at line 1, column 4 (while parsing '"\0"') at <top-level>, line 1:
"\033[31m"+.c+"\033[0m"
jq: error: Invalid escape at line 1, column 4 (while parsing '"\0"') at <top-level>, line 1:
"\033[31m"+.c+"\033[0m"
jq: 2 compile errors # returns err code 3
$ jq -rn '"\033[31mbar\033[0m"'
jq: error: Invalid escape at line 1, column 4 (while parsing '"\0"') at <top-level>, line 1:
"\033[31mbar\033[0m"
jq: error: Invalid escape at line 1, column 4 (while parsing '"\0"') at <top-level>, line 1:
"\033[31mbar\033[0m"
jq: 2 compile errors # returns err code 4
P.S. in case it matters, I am using the bash shell with version 5.1.16(1)-release on Linux.
Conslusion: ANSI colors do not work with jq, whether in the JSON string or directly concatenating it through the + operator.
Question: how to make ANSI colors work in jq? Any help would be appreciated.
Octal escape sequences are not valid JSON syntax, so you need to encode the ASCII escape character as \u001b rather than \033. Also, to add to the confusion, some versions of echo will attempt to interpret backslash (escape) sequences itself before passing them to jq, so in cases like this it's much safer to use printf '%s\n':
$ printf '%s\n' '{"a":"b","c":"\u001b[31md\u001b[0m"}' | jq -r .c
d
(You can't see it, but that "d" is red in my terminal.)
BTW, an easy way to find things like this out is to get jq to encode them in JSON for you. Here, I'll set the shell variable to the actual string (using bash's $'...' string format, which interprets ANSI-C escape sequences like \033), then use --arg to pass that to jq:
$ seq=$'\033[31md\033[0m'
$ jq -nc --arg seq "$seq" '{"a":"b","c":$seq}'
{"a":"b","c":"\u001b[31md\u001b[0m"}
I'm scripting in bash to edit a json template to replace some field values with arguments to my script, and trying to use jq to do the editing. My code is not replacing the --arg with the value of the argument, but the literal text of the argument name.
My template contains:
{
"name":""
}
My jq code:
jq --arg ad "192.168.5.5" -r '.name = "Addr $ad"' address.tmpl
This outputs:
{
"name": "Addr $ad"
}
Or, if I remove the double-quotes
jq --arg ad "192.168.5.5" -r '.name = Addr $ad' address.tmpl
I get
jq: error: syntax error, unexpected '$', expecting $end (Unix shell quoting issues?) at <top-level>, line 1:
.name = Addr $ad
jq: 1 compile error
According to all that I have read, this should work. What am I doing wrong/how do I fix this????
OS = debian 10
In the jq manual, search for "String interpolation"
jq --arg ad "192.168.5.5" -r '.name = "Addr \($ad)"'
jq --arg ad 192.168.5.5 -r '.name = "Addr " + $ad' address.tmpl
$ad will be expanded by the shell if it is not not hard quoted. In jq, you can use string interpolation "Addr \($ad)", or concatenation (as above), which I find slightly more readable.
I got a question about shell's jq. So my JSON object is:
{"1543446000": {"name": "John", "company": "foo"}, "1543359600": {"name": "Kate", "company": "bar"}}
The numers 1543446000 and 1543359600 are UNIX timestamps. How can I parse one of the JSON objects by the timestamp with a shell variable?
My shell script so far:
#!/bin/sh
URL="https://pastebin.com/raw/w7awz7kZ"
export DATE=$(date -d "$today 0" +%s)
JSON=$(curl -H "Accept: application/json" $API_URL)
JSON=$(echo $JSON | jq --arg date $DATE '.$date')
echo $JSON
Doesn't seem to work. My intention is to select the inner JSON object described by one of the timestamps, which are basically midnight of today. So I want to select today's data set.
Any suggestions?
Greets,
Innoberger
You need to use the full syntax for key access, as the dollar sign preclude you using the shorter form. The error message should provide this suggestion.
$ jq --arg date 1543359600 '.$date' tmp.json
jq: error: syntax error, unexpected '$' (Unix shell quoting issues?) at <top-level>, line 1:
.$date
jq: error: try .["field"] instead of .field for unusually named fields at <top-level>, line 1:
.$date
jq: 2 compile errors
Note the error message
'try .["field"] instead of .field'.
You won't need the quotes, though, as that would be how you specify a literal key $date.
$ jq --arg date 1543359600 '.[$date]' tmp.json
{
"name": "Kate",
"company": "bar"
}
I am wanting to pipe values into a bash script that will change values in a json file using jq. I've been working on this for a while now and I can't get past the first set of errors.
Here is my simple json file:
{
"0000000": {
"pogo": "AJHVUYKJBOIHKNNLNM"
},
"7000000": {
"pogo": "PPPVUYKJBOIHKNNLNM"
}
}
Here is my script
#!/bin/bash
#-- pass into script
pgid="${1}"
tpogo="${2}"
file="file.json"
#-- tmp files
tmp_dir="$(mktemp -d -t 'text.XXXXX' || mktemp -d 2>/dev/null)"
tmp_input1="${tmp_dir}/temp_input1.txt"
if [ ! -n "$2" ]; then
{ echo "Check your command as you are missing a variable ..."; echo "Example: script.sh "00000001" "jvkkjbkjbd" ..."; }
exit 1;
fi
jq -r ".${pgid}.pogo = \"${tpogo}\"" "$file" > "$tmp_input1"
cat "$tmp_input1"
rm -rf "$tmp_dir"
Here are the errors:
jq: error: Invalid numeric literal at EOF at line 1, column 9 (while parsing '.0000000.') at <top-level>, line 1:
.0000000.pogo = "XXXXXXX"
jq: error: syntax error, unexpected IDENT, expecting $end (Unix shell quoting issues?) at <top-level>, line 1:
.0000000.pogo = "XXXXXXX"
jq: 2 compile errors
I have tried so many variations from stack and most of them look similar to what I am doing now.
As for the immediate issue: .key works with { "foo": "value" }, but .100 doesn't work with { "100": "value" }; the syntax you're relying is sugar, only available for a limited subset of keys. .["100"] would work, but generating that by expanding shell variables into strings parsed as code is fragile (jq isn't a side-effecting language as of current releases, but in languages that do support I/O operations, such substitutions can be leveraged for injection attacks). To do things the Right Way, pass your variables out-of-band from your code, and use them for lookups in a manner that doesn't rely on what they contain.
The jq equivalent to awk's -v var="$value" is --arg var "$value", used thusly:
jq --arg pgid "$pgid" \
--arg tpogo "$tpogo" \
'.[$pgid].pogo = $tpogo'
Testing this with your data:
json='{"0000000":{"pogo":"AJHVUYKJBOIHKNNLNM"},"7000000":{"pogo":"PPPVUYKJBOIHKNNLNM"}}'
pgid="0000000"
tpogo="XXXXXXX"
jq --arg pgid "$pgid" --arg tpogo "$tpogo" \
'.[$pgid].pogo = $tpogo' <<<"$json"
...emits as output:
{
"0000000": {
"pogo": "XXXXXXX"
},
"7000000": {
"pogo": "PPPVUYKJBOIHKNNLNM"
}
}
I have a .jq template that I want to update values under the samples list, formatted as:
{
"test": "abc",
"p": "1",
"v": "1.0.0",
"samples": [
{
"uptime": $uptime,
"curr_connections": $curr_connections,
"listen_disabled_num": $listen_disabled_num,
"conn_yields": $conn_yields,
"cmd_get": $cmd_get,
"cmd_set": $cmd_set,
"bytes_read": $bytes_read,
"bytes_written": $bytes_writtem,
"get_hits": $get_hits,
"rejected_connections": $rejected_connections,
"limit_maxbytes": $limit_maxbytes,
"cmd_flush": $cmd_flush
}
]
}
My shell script to do this is below, I am basically running a command to pull some memcached output stats and want to insert some of the results into the jq template as key/values.
JQ=`cat template.jq`
SAMPLES=(uptime curr_connections listen_disabled_num conn_yields cmd_get cmd_set cmd_flush bytes_read bytes_written get_hits rejected_connections limit_maxbytes)
for metric in ${SAMPLES[*]}
do
KEY=$(echo stats | nc $HOST $PORT | grep $metric | awk '{print $2}')
VALUE=$(echo stats | nc $HOST $PORT | grep $metric | awk '{print $3}')
echo "Using KEY: $KEY with value: $VALUE"
jq -n --argjson $KEY $VALUE -f template.jq
done
Not sure if this is the best way to handle this scenario, but I am getting a ton of errors such as:
jq: error: conn_yields/0 is not defined at <top-level>, line 12:
"conn_yields": $conn_yields,
jq: error: cmd_get/0 is not defined at <top-level>, line 13:
"cmd_get": $cmd_get,
jq: error: cmd_set/0 is not defined at <top-level>, line 14:
"cmd_set": $cmd_set,
If you are going to invoke jq using -f template.jq, then each of the $-variables in template.jq will have to be set separately on the command-line, one by one. In your case, this does not look like a very happy option.
If you are stuck with template.jq as it is, then it will be hard slogging, though there are alternatives besides setting the $-variables on the command line.
Please see https://github.com/stedolan/jq/wiki/Cookbook#using-jq-as-a-template-engine in the jq Cookbook for an alternative to using $-variables. Consider for example the implications of this illustration of "destructuring":
jq -nc '{a:1,b:2} as {a: $a, b:$b} | [$a,$b]'
[1,2]
Another alternative
In your particular case, you could replace all the "$" characters in template.jq with ".", and then pass in a JSON object with the appropriate keys; e.g. change $uptime to .uptime, and then include a key/value pair for uptime.