I have a json object named version6json as follows
{
"20007.098": {
"os_version": "6.9",
"kernel": "2.6.32-696",
"sfdc-release": "2017.08"
},
"200907.09678”: {
"os_version": "6.9",
"kernel": "2.6.32-696",
"sfdc-release": "201.7909"
},
"206727.1078”: {
"os_version": "6.9",
"kernel": "2.6.32-696.10.2.el6.x86_64",
"sfdc-release": "20097.109”
}
}
I want to add one more key value pair. The key is also a variable and the value too. bundle_release="2019.78" and value= {"release":"2018.1006","kernel":"2.6.32-754.3.5.el6.x86_64","os":"6.10","current":true}
Now I want the bundle_release as key and value as its value, So the new entry would be "2018.1006": {"release":"2018.1006","kernel":"2.6.32-754.3.5.el6.x86_64","os":"6.10","current":true}
To achieve this, I am doing the folllowing
echo "$version6json" | jq --arg "$bundle_release" "$value" '. + {$bundle_release: "${value}"}'
Any help will be appriciated.
P.S- The question is edited as suggested by peak
First, when specifying a key name using a variable in the way you are doing, the variable must be parenthesized, so you would have:
{($bundle_release): ...}
Next, jq variables are not the same as shell variables and should be specified without quoting them, and without using bash-isms.
Third, when setting the value of the shell variable named value, you would have to quote the expression appropriately.
Fourth, to simplify things, use --argjson for $value.
Fifth, your sample JSON is not quite right. Once it's fixed, the following will work in a bash or bash-like environment (assuming you're using a version of jq that supports --argjson):
bundle_release="1034,567"
value='{"release":"2018.1006","kernel":"2.6.32-754.3.5.el6.x86_64","os":"6.10","current":true}'
jq --arg b "$bundle_release" --argjson v "$value" '
. + {($b): $v}' <<< "$version6json"
You're not giving the --arg option enough parameters: from the manual:
--arg name value:
This option passes a value to the jq program as a predefined variable. If you run jq with --arg foo bar, then
$foo is available in the program and has the value "bar". Note that value will be treated as a string, so
--arg foo 123 will bind $foo to "123".
Related
I am having trouble accessing bash variable inside 'jq'.
The snippet below shows my bash loop to check for missing keys in a Json file.
#!/bin/sh
for key in "key1" "key2.key3"; do
echo "$key"
if ! cat ${JSON_FILE} | jq --arg KEY "$key" -e '.[$KEY]'; then
missingKeys+=${key}
fi
done
JSON_FILE:
{
"key1": "val1",
"key2": {
"key3": "val3"
}
}
The script works correctly for top level keys such as "key1". But it does not work correctly (returns null) for "key2.key3".
'jq' on the command line does return the correct value
cat input.json | jq '.key2.key3'
"val3"
I followed answers from other posts to come to this solution. However can't seem to figure out why it does not work for nested json keys.
Using --arg prevents your data from being incorrectly parsed as syntax. Usually, a shell variable you're passing into jq contains literal data, so this is the correct thing.
In this case, your variable contains syntax, not literal data: The . isn't part of the string you want to do a lookup by, but is instead an instruction to jq to do two separate lookups one after the other.
So, in this case, you should do the more obvious thing, instead of using --arg:
jq -e ".$KEY"
Firstly, apologies if this has been asked before (though I don't think it has).
I have a json-string that I am receiving as the output of a cURL command in a bash-script.
It looks a little something like this:
{"123456": {"extract": "this is the bit I am looking for"}}
Now, the key "123456" is dynamic, and I actually need it to form the url for the cURL command. Because of this, the string "123456" is stored as a variable called $PAGE_ID.
How can I use jq to access the value corresponding to this key? I have tried many different iterations based on the jq documentation such as:
curl "$URL$PAGE_ID" | jq '.["$PAGE_ID"]'
curl "$URL$PAGE_ID" | jq '.[env.PAGE_ID]'
curl "$URL$PAGE_ID" | jq ".[$PAGE_ID]"
and they are all somehow-problematic (there is no string interpolation in the first one, the second one returns null and the third one technically looks for the numeric value 123456 in the dictionary and not the string-equivalent).
Is there any way to find the value corresponding to a key that is both a numeric string AND stored in a variable?
Hopefully the following script answers the question.
#!/bin/bash
function data {
cat <<EOF
{"123456": {"extract": "this is the bit I am looking for"}}
EOF
}
PAGE_ID=123456
data | jq -c --arg pid "$PAGE_ID" '.[$pid]'
data | jq --arg pid "$PAGE_ID" '.[$pid].extract'
Output
{"extract":"this is the bit I am looking for"}
"this is the bit I am looking for"
One way:
$ jq --arg pageid "$PAGE_ID" 'to_entries[] | select(.key==$pageid) | .value' <<<'{"123456": {"extract": "this is the bit I am looking for"}}'
{
"extract": "this is the bit I am looking for"
}
You can also use double quotes to force numeric value to be a key :
PAGE_ID=123456
jq ".\"$PAGE_ID\"" <<< '{"123456": {"extract": "this is the bit I am looking for"}}'
# {
# "extract": "this is the bit I am looking for"
# }
Given the json below:
./versions.json
{
"my-app-1": "v1.0.0",
"my-app-2": "v0.9.1",
"my-app-3": "v2.1.7"
}
I want to replace the version of $APP_NAME to the new version $NEW_VERSION. Given APP_NAME=my-app-2 and NEW_VERSION=v1.0.0, I tried the following:
jq '(."$APP_NAME") = "$NEW_VERSION"' ./versions.json > updated_versions.json
which gives:
./updated_versions.json
{
"my-app-1": "v1.0.0",
"my-app-2": "v0.9.1",
"my-app-3": "v2.1.7",
"$APP_NAME": "$NEW_VERSION"
}
this:
jq "(.$APP_NAME) = \"$NEW_VERSION\"" versions.json > updated_versions.json
gives:
jq: error: app/0 is not defined at <top-level>, line 1:
(.my-app-1) = "v1.0.0"
jq: 1 compile error
How can I escape the special character in the environment variable? I have tried setting APP_NAME=my\\-app\\-1 with no luck.
Thanks
Pass the two variables to jq using its --arg option :
jq --arg appName my-app-2 --arg newVersion v1.0.0 '.[$appName]=$newVersion'
In your first try the jq command was enclosed in single-quotes, so it was left to jq to resolve the variables, but it doesn't look for them in the outer shell context.
Your second try was nearly good (but not very good practice) because the variables were expanded by the shell, but my-app-1 contains special characters (the dashes) and needs to be accessed with either ."my-app-1" or .["my-app-1"].
Assume the following JSON file
{
"foo": "hello",
"bar": "world"
}
I want to get the foo field from the JSON object in a standalone object, and I do this:
<file jq '{foo}'
{
"foo": "hello"
}
Now the field I actually want is coming from the shell and is given to jq as an argument like this:
<file jq --arg myarg "foo" '{$myarg}'
{
"myarg": "foo"
}
Unfortunately this doesn't give the expected result {"foo":"hello"}.
Any idea why the name of the variable gets into the object?
A workaround to this is to explicitly defined the object:
<file jq '{($myarg):.[$myarg]}'
Fine, but is there a way to use the shortcut syntax as explained in the man page, but with a variable ?
You can use this to select particular fields of an object: if the input is an object with “user”, “title”, “id”, and “content” fields and you just want “user” and “title”, you can write
{user: .user, title: .title}
Because that is so common, there’s a shortcut syntax for it: {user, title}.
If that matters, I'm using jq version 1.5
In short, no. The shortcut syntax can only be used under very special conditions. For example, it cannot be used with key names that are jq keywords.
Alternatives
The method described in the Q is the preferred one, but for the record, here are two alternatives:
jq --arg myarg "foo" '
.[$myarg] as $v | {} | .[$myarg] = $v'
And of course there's the alternative that comes with numerous caveats:
myarg=foo ; jq "{ $myarg }"
I want to modify a JSON file by using the Linux command line.
I tried these steps:
[root#localhost]# INPUT="dsa"
[root#localhost]# echo $INPUT
dsa
[root#localhost]# CONF_FILE=test.json
[root#localhost]# echo $CONF_FILE
test.json
[root#localhost]# cat $CONF_FILE
{
"global" : {
"name" : "asd",
"id" : 1
}
}
[root#localhost]# jq -r '.global.name |= '""$INPUT"" $CONF_FILE > tmp.$$.json && mv tmp.$$.json $CONF_FILE
jq: error: dsa/0 is not defined at <top-level>, line 1:
.global.name |= dsa
jq: 1 compile error
Desired output:
[root#localhost]# cat $CONF_FILE
{ "global" : {
"name" : "dsa",
"id" : 1 } }
Your only problem was that the script passed to jq was quoted incorrectly.
In your particular case, using a single double-quoted string with embedded \-escaped " instances is probably simplest:
jq -r ".global.name = \"$INPUT\"" "$CONF_FILE" > tmp.$$.json && mv tmp.$$.json "$CONF_FILE"
Generally, however, chepner's helpful answer shows a more robust alternative to embedding the shell variable reference directly in the script: Using the --arg option to pass a value as a jq variable allows single-quoting the script, which is preferable, because it avoids confusion over what elements are expanded by the shell up front and obviates the need for escaping $ instances that should be passed through to jq.
Also:
Just = is sufficient to assign the value; while |=, the so-called update operator, works too, it behaves the same as = in this instance, because the RHS is a literal, not an expression referencing the LHS - see the manual.
You should routinely double-quote your shell-variable references and you should avoid use of all-uppercase variable names in order to avoid conflicts with environment variables and special shell variables.
As for why your quoting didn't work:
'.global.name |= '""$INPUT"" is composed of the following tokens:
String literal .global.name |= (due to single-quoting)
String literal "" - i.e., the empty string - the quotes will be removed by the shell before jq sees the script
An unquoted reference to variable $INPUT (which makes its value subject to word-splitting and globbing).
Another instance of literal "".
With your sample value, jq ended up seeing the following string as its script:
.global.name |= dsa
As you can see, the double quotes are missing, causing jq to interpret dsa as a function name rather than a string literal, and since no argument was passed to (non-existent) function dsa, jq's error message referenced it as dsa/0 - a function with no (0) arguments.
It's much simpler and safer to pass the value using the --arg option:
jq -r --arg newname "$INPUT" '.global.name |= $newname' "$CONF_FILE"
This ensures that the exact value of $INPUT is used and quoted as a JSON value.
Using jq with a straight forward filter, should do it for you.
.global.name = "dsa"
i.e.
jq '.global.name = "dsa"' json-file
{
"global": {
"name": "dsa",
"id": 1
}
}
You can play around with your json-filters, here.