What is wrong with this bash script [duplicate] - json

This question already has answers here:
Passing bash variable to jq
(10 answers)
Closed 1 year ago.
#!/bin/bash
filename='delete'
while read p; do
jq 'if .tweet | test('\"$p\"'; "i") then . |= . + {vendor: '\"$p\"'} else empty end' sfilter.json
done < $filename
I keep getting this error, when I use variable = "hello world" but not "hello"
Inshort whenever there is a space in letter it causes this error.
But while executing directly on shell there seems to be no error.
jq: error: syntax error, unexpected $end, expecting QQSTRING_TEXT or QQSTRING_INTERP_START or QQSTRING_END (Unix shell quoting issues?) at <top-level>, line 1:
if .tweet | test("sdas
jq: 1 compile error
But when I run the command on shell it works perfectly. Any ideas?
Edit:
Input delete file
sdas adssad

Don't generate a dynamic jq filter using string interpolation. Pass the value via the --arg option.
#!/bin/bash
filename='delete'
while IFS= read -r p; do
jq --arg p "$p" 'if .tweet | test($p; "i") then . |= . + {vendor: $p} else empty end' sfilter.json
done < "$filename"

pass the value of p to your jq program like this:
while read p; do
jq --arg p "$p" 'if .tweet | test($p; "i") then . |= . + {vendor: $p} else empty end' sfilter.json
done < $filename
The shell error occurs because you concatenated the string incorrectly.
Try this:
p=value
JQ_program='if .tweet | test("'"$p"'"; "i") then . |= . + {vendor: "'"$p"'"} else empty end'
echo "$JQ_program"
# result
# if .tweet | test("value"; "i") then . |= . + {vendor: "value"} else empty end

Related

How to make a new array and add json value in bash

for region in $(jq '.data | keys | .[]' <<< "$data"); do
value=$(jq -r ".data[$region]" <<< "$data");
deliveryRegionId=$(jq -r '.deliveryRegionId' <<< "$value");
json_template='{}';
json_data=$(jq --argjson deliveryRegionId "$deliveryRegionId" --arg deliverableDistance 5000 '.deliveryRegionId=$deliveryRegionId | .deliverableDistance=5000' <<<"$json_template"); echo $json_data;
requestArray=$(jq '. += [$json_data]' <<< $requestArray)
done;
As in the code above, I'm going to create a json value called json_data and add it to the array.
What should I do to make this work?
jq: error: $json_data is not defined at <top-level>, line 1:
. += [$json_data]
jq: 1 compile error
this is error
There's no jq variable named $json_data.
There is a shell variable named that, but you can't access another program's variables.
Provide the value via the environment
json_data="$json_data" jq '. += [ env.json_data ]' <<<"$requestArray"
Provide the value via the environment
export json_data
jq '. += [ env.json_data ]' <<<"$requestArray"
Provide the value as an argument
jq --arg json_data "$json_data" '. += [ $json_data ]' <<<"$requestArray"
There's no reason to use jq so many times! Your entire program can be replaced with this:
requestArray="$(
jq '.data | map( { deliveryRegionId, deliverableDistance: 5000 } )' \
<<<"$data"
)"
Demo on jqplay

JQ: add variable property to existing object

I'm trying to add variable properties to some existing json
{
"item1": {
"proerty1": "test"
},
"item2": {}
}
So if I do something like this it works
echo $contents | jq --arg ITEM1 $item1 '.[$ITEM1].property2 = "test2"'
But when I try to add more arguments like this it fails:
echo $contents | jq --arg ITEM1 $item1 --arg PROPERTY2 $property2 --arg VALUE $value '.[$ITEM1].[PROPERTY2] = $VALUE'
The error I get is:
jq: error: syntax error, unexpected '[', expecting FORMAT or QQSTRING_START (Unix shell quoting issues?) at <top-level>, line 1:
.[$ITEM1].[PROPERTY2] = $VALUE
jq: 1 compile error
So I guess += operator wouldn't be the correct way to do this with variables. What would be the correct way to add a propetry where the whole path .item.property and the value itself are variable
The jq filter should be:
.[$ITEM1][$PROPERTY2] = $VALUE
Your query has an extra ..
An alternative:
You could also use setpath/2, e.g.
setpath([$ITEM1,$PROPERTY2]; $VALUE)
Aside
It's usually best to quote your shell variables, e.g.
echo "$contents" ...

Why is this "if A then B end" statement not accepted with jq 1.6?

According to the jq manual (Conditionals and Comparisons > if-then-else):
if A then B end is the same as if A then B else . end. That is, the
else branch is optional, and if absent is the same as ..
The same is substantiated by the accepted answer to this Stack Overflow question: JSON JQ if without else
So why does this if A then B end statement invoke a parse error?
$ jq --version
jq-1.6
$ echo 2 | jq 'if . == 0 then "zero" end'
jq: error: syntax error, unexpected end (Unix shell quoting issues?) at <top-level>, line 1:
if . == 0 then "zero" end
jq: error: Possibly unterminated 'if' statement at <top-level>, line 1:
if . == 0 then "zero" end
jq: 2 compile errors
$ echo 0 | jq 'if . == 0 then "zero" end'
jq: error: syntax error, unexpected end (Unix shell quoting issues?) at <top-level>, line 1:
if . == 0 then "zero" end
jq: error: Possibly unterminated 'if' statement at <top-level>, line 1:
if . == 0 then "zero" end
jq: 2 compile errors
What I understand to be the equivalent if A then B else . end form of the statement seems to work:
$ echo 2 | jq 'if . == 0 then "zero" else . end'
2
$ echo 0 | jq 'if . == 0 then "zero" else . end'
"zero"
I think you're looking at the manual for the development version of jq, rather than the manual for jq version 1.6.

How to use regex to search the key and value and replace value with new value using jq

My json
{
"license": " See license.md",
"dependencies": {
"#gx/core": "0.279.0-b1-abc-1234-0716.4567",
"#gx/api": "0.279.0-b1-abc-1234-0716.4567",
"#gx/name": "0.279.0-b1-abc-1234-0716.4567"
}
}
I want to replace "0.279.0-b1-abc-1234-0716.4567" with "0.279.0-b1-abc-1234-0716.9856" in all places.
jq '.dependencies[].["#gx/core"] |= (if . == "0.279.0-b1-abc-1234-0716.4567" then "0.279.0-b1-abc-1234-0716.9856" else . end)' info.json
jq: error: syntax error, unexpected '[', expecting FORMAT or QQSTRING_START (Unix shell quoting issues?) at <top-level>, line 1:
.dependencies[].["#gx/core"] |= (if . == "0.279.0-b1-abc-1234-0716.4567" then "0.279.0-b1-abc-1234-0716.9856" else . end)
jq: 1 compile error
I am looking for something like this
jq '.dependencies[].["#gx/[a-z]*"] |= (if . == "^(\d+\.){2}[0-9]+(-[a-zA-Z0-9]*){4}\.[0-9]*$" then "0.279.0-b1-abc-1234-0716.9856" else . end)' info.json
Using jq, there are many different approaches, with very different semantics, as can be seen from these solutions to the first problem (without regexes):
walk(if . == "0.279.0-b1-abc-1234-0716.4567"
then "0.279.0-b1-abc-1234-0716.9856" else . end)
A more focused approach:
.dependencies |=
map_values(if . == "0.279.0-b1-abc-1234-0716.4567"
then "0.279.0-b1-abc-1234-0716.9856" else . end)
regexes
The above approaches can all be used in the case of regex searches too, e.g. the last case would become:
.dependencies |= with_entries(
if (.key | test("#gx/[a-z]*"))
and (.value | test("^(\\d+\\.){2}[0-9]+(-[a-zA-Z0-9]*){4}\\.[0-9]*$"))
then .value = "0.279.0-b1-abc-1234-0716.9856" else . end)
Note that the regex strings must be JSON strings, and hence the doubling of backslashes.
if without else
If you have a sufficiently recent version of jq, those dangling occurrences of "else ." can be dropped.
if you up to considering a non-jq solution, let me offer here one based on a walk-path unix utility jtc:
bash $ <file.json jtc -w'[dependencies]<0\.279\.0\-b1\-abc\-1234\-0716\.4567>R:' -u'"0.279.0-b1-abc-1234-0716.9856"'
{
"dependencies": {
"#gx/api": "0.279.0-b1-abc-1234-0716.9856",
"#gx/core": "0.279.0-b1-abc-1234-0716.9856",
"#gx/name": "0.279.0-b1-abc-1234-0716.9856"
},
"license": " See license.md"
}
bash $
walk-path (-w):
[dependencies] addresses (from root) the given record
<...>R: - a search lexeme, finds using RE (suffix R) all (quantifier :) entries matching the given reg.expression.
-u will update (replace) all found matches.
-- or --
using your REs, matching both labels and values:
bash $ <file.json jtc -w'[dependencies]<#gx/[a-z]*>L:<^(\d+\.){2}[0-9]+(-[a-zA-Z0-9]*){4}\.[0-9]*$>R' -u'"0.279.0-b1-abc-1234-0716.9856"'
same result
PS> Disclosure: I'm the creator of the jtc tool

Add an empty line prior to another matched line in jq?

Say I have a raw input like the following:
"```"
"include <stdio.h>"
"..."
"```"
"''some example''"
"*bob"
"**bob"
"*bob"
And I'd like to add a blank line right before the "*bob":
"```"
"include <stdio.h>"
"..."
"```"
"''some example''"
""
"*bob"
"**bob"
"*bob"
Can this be done with jq?
Yes, but to do so efficiently you'd effectively need jq 1.5 or higher:
foreach inputs as $line (0;
if $line == "*bob" then . + 1 else . end;
if . == 1 then "" else empty end,
$line)
Don't forget to use the -n command-line option!
Here is another solution which uses the -s (slurp) option
.[: .[["*bob"]][0]] + ["\n"] + .[.[["*bob"]][0]:] | .[]
that's a little unreadable but we can make it better with a few functions:
def firstbob: .[["*bob"]][0] ;
def beforebob: .[: firstbob ] ;
def afterbob: .[ firstbob :] ;
beforebob + ["\n"] + afterbob
| .[]
if the above filter is in filter.jq and the sample data is in data then
$ jq -Ms -f filter.jq data
produces
"```"
"include <stdio.h>"
"..."
"```"
"''some example''"
"\n"
"*bob"
"**bob"
"*bob"
One issue with this approach is that beforebob and afterbob won't quite work as you probably want if "*bob" is not in the input. The easiest way to address that is with an if guard:
if firstbob then beforebob + ["\n"] + afterbob else . end
| .[]
with that the input will be unaltered if "*bob" is not present.