Assume the following JSON file
{
"foo": "hello",
"bar": "world"
}
I want to get the foo field from the JSON object in a standalone object, and I do this:
<file jq '{foo}'
{
"foo": "hello"
}
Now the field I actually want is coming from the shell and is given to jq as an argument like this:
<file jq --arg myarg "foo" '{$myarg}'
{
"myarg": "foo"
}
Unfortunately this doesn't give the expected result {"foo":"hello"}.
Any idea why the name of the variable gets into the object?
A workaround to this is to explicitly defined the object:
<file jq '{($myarg):.[$myarg]}'
Fine, but is there a way to use the shortcut syntax as explained in the man page, but with a variable ?
You can use this to select particular fields of an object: if the input is an object with “user”, “title”, “id”, and “content” fields and you just want “user” and “title”, you can write
{user: .user, title: .title}
Because that is so common, there’s a shortcut syntax for it: {user, title}.
If that matters, I'm using jq version 1.5
In short, no. The shortcut syntax can only be used under very special conditions. For example, it cannot be used with key names that are jq keywords.
Alternatives
The method described in the Q is the preferred one, but for the record, here are two alternatives:
jq --arg myarg "foo" '
.[$myarg] as $v | {} | .[$myarg] = $v'
And of course there's the alternative that comes with numerous caveats:
myarg=foo ; jq "{ $myarg }"
Related
I am having trouble accessing bash variable inside 'jq'.
The snippet below shows my bash loop to check for missing keys in a Json file.
#!/bin/sh
for key in "key1" "key2.key3"; do
echo "$key"
if ! cat ${JSON_FILE} | jq --arg KEY "$key" -e '.[$KEY]'; then
missingKeys+=${key}
fi
done
JSON_FILE:
{
"key1": "val1",
"key2": {
"key3": "val3"
}
}
The script works correctly for top level keys such as "key1". But it does not work correctly (returns null) for "key2.key3".
'jq' on the command line does return the correct value
cat input.json | jq '.key2.key3'
"val3"
I followed answers from other posts to come to this solution. However can't seem to figure out why it does not work for nested json keys.
Using --arg prevents your data from being incorrectly parsed as syntax. Usually, a shell variable you're passing into jq contains literal data, so this is the correct thing.
In this case, your variable contains syntax, not literal data: The . isn't part of the string you want to do a lookup by, but is instead an instruction to jq to do two separate lookups one after the other.
So, in this case, you should do the more obvious thing, instead of using --arg:
jq -e ".$KEY"
Firstly, apologies if this has been asked before (though I don't think it has).
I have a json-string that I am receiving as the output of a cURL command in a bash-script.
It looks a little something like this:
{"123456": {"extract": "this is the bit I am looking for"}}
Now, the key "123456" is dynamic, and I actually need it to form the url for the cURL command. Because of this, the string "123456" is stored as a variable called $PAGE_ID.
How can I use jq to access the value corresponding to this key? I have tried many different iterations based on the jq documentation such as:
curl "$URL$PAGE_ID" | jq '.["$PAGE_ID"]'
curl "$URL$PAGE_ID" | jq '.[env.PAGE_ID]'
curl "$URL$PAGE_ID" | jq ".[$PAGE_ID]"
and they are all somehow-problematic (there is no string interpolation in the first one, the second one returns null and the third one technically looks for the numeric value 123456 in the dictionary and not the string-equivalent).
Is there any way to find the value corresponding to a key that is both a numeric string AND stored in a variable?
Hopefully the following script answers the question.
#!/bin/bash
function data {
cat <<EOF
{"123456": {"extract": "this is the bit I am looking for"}}
EOF
}
PAGE_ID=123456
data | jq -c --arg pid "$PAGE_ID" '.[$pid]'
data | jq --arg pid "$PAGE_ID" '.[$pid].extract'
Output
{"extract":"this is the bit I am looking for"}
"this is the bit I am looking for"
One way:
$ jq --arg pageid "$PAGE_ID" 'to_entries[] | select(.key==$pageid) | .value' <<<'{"123456": {"extract": "this is the bit I am looking for"}}'
{
"extract": "this is the bit I am looking for"
}
You can also use double quotes to force numeric value to be a key :
PAGE_ID=123456
jq ".\"$PAGE_ID\"" <<< '{"123456": {"extract": "this is the bit I am looking for"}}'
# {
# "extract": "this is the bit I am looking for"
# }
I would like to search a JSON file for some key or value, and have it print where it was found.
For example, when using jq to print out my Firefox' extensions.json, I get something like this (using "..." here to skip long parts) :
{
"schemaVersion": 31,
"addons": [
{
"id": "wetransfer#extensions.thunderbird.net",
"syncGUID": "{e6369308-1efc-40fd-aa5f-38da7b20df9b}",
"version": "2.0.0",
...
},
{
...
}
]
}
Say I would like to search for "wetransfer#extensions.thunderbird.net", and would like an output which shows me where it was found with something like this:
{ "addons": [ {"id": "wetransfer#extensions.thunderbird.net"} ] }
Is there a way to get that with jq or with some other json tool?
I also tried to simply list the various ids in that file, and hoped that I would get it with jq '.id', but that just returned null, because it apparently needs the full path.
In other words, I'm looking for a command-line json parser which I could use in a way similar to Xpath tools
The path() function comes in handy:
$ jq -c 'path(.. | select(. == "wetransfer#extensions.thunderbird.net"))' input.json
["addons",0,"id"]
The resulting path is interpreted as "In the addons field of the initial object, the first array element's id field matches". You can use it with getpath(), setpath(), delpaths(), etc. to get or manipulate the value it describes.
Using your example with modifications to make it valid JSON:
< input.json jq -c --arg s wetransfer#extensions.thunderbird.net '
paths as $p | select(getpath($p) == $s) | null | setpath($p;$s)'
produces:
{"addons":[{"id":"wetransfer#extensions.thunderbird.net"}]}
Note
If there are N paths to the given value, the above will produce N lines. If you want only the first, you could wrap everything in first(...).
Listing all the "id" values
I also tried to simply list the various ids in that file
Assuming that "id" values of false and null are of no interest, you can print all the "id" values of interest using the jq filter:
.. | .id? // empty
Using jq, is it possible to replace the value of each parameter in the sample JSON with the value of the variable that is the initial value?
In my scenario, Azure DevOps does not carryout any kind of variable substitution on the JSON file, so I need to do it manually. So for example, say $SUBSCRIPTION_ID is set to abc-123, I'd like to use jq to update the JSON file.
I can pull out the values using .parameters[].value, but I can't seem to find a way of setting each individual value.
The main challenge here is that the solution should be reusable, and different JSON files will have different parameters, so I don't think I can use --argjson.
Example
Original JSON
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/parametersTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"subscriptionId": {
"value": "$SUBSCRIPTION_ID"
},
"topicName": {
"value": "$TOPIC_NAME"
}
}
}
Variables
SUBSCRIPTION_ID="abc-123"
TOPIC_NAME="SomeTopic"
Desired JSON
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/parametersTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"subscriptionId": {
"value": "abc-123"
},
"topicName": {
"value": "SomeTopic"
}
}
}
Export those variables so that you can access them from within jq.
export SUBSCRIPTION_ID TOPIC_NAME
jq '.parameters[].value |= (env[.[1:]] // .)' file
//. part is for leaving variables absent in the environment as is, you can drop it if not necessary
Use --argjson; essentially, you are just going to ignore the attempt at parameterizing the JSON and simply replace the values unconditionally.
jq --argjson x "$SUBSCRIPTION_ID" \
--argjson y "$TOPIC_NAME" \
'.parameters.subscriptionId.value = $x; .parameters.topicName.value = $y' \
config.json
Here is a "data-driven" approach based on the contents of the schema and the available environment variables:
export SUBSCRIPTION_ID="abc-123"
export TOPIC_NAME="SomeTopic"
< schema.json jq '.parameters
|= map_values(if .value | (startswith("$") and env[.[1:]])
then .value |= env[.[1:]] else . end)'
Notice that none of the template names appear in the jq program.
If your shell supports it, you could avoid the "export" commands by prefacing the jq command with the variable assignments along the lines of:
SUBSCRIPTION_ID="abc-123" TOPIC_NAME="SomeTopic" jq -f program.jq schema.json
Caveat
Using environment variables to pass in the parameter values may not be such a great idea. Two alternatives would be to provide the name-value pairs in a text file or as a JSON object. See also Using jq as a template engine
I have a json object named version6json as follows
{
"20007.098": {
"os_version": "6.9",
"kernel": "2.6.32-696",
"sfdc-release": "2017.08"
},
"200907.09678”: {
"os_version": "6.9",
"kernel": "2.6.32-696",
"sfdc-release": "201.7909"
},
"206727.1078”: {
"os_version": "6.9",
"kernel": "2.6.32-696.10.2.el6.x86_64",
"sfdc-release": "20097.109”
}
}
I want to add one more key value pair. The key is also a variable and the value too. bundle_release="2019.78" and value= {"release":"2018.1006","kernel":"2.6.32-754.3.5.el6.x86_64","os":"6.10","current":true}
Now I want the bundle_release as key and value as its value, So the new entry would be "2018.1006": {"release":"2018.1006","kernel":"2.6.32-754.3.5.el6.x86_64","os":"6.10","current":true}
To achieve this, I am doing the folllowing
echo "$version6json" | jq --arg "$bundle_release" "$value" '. + {$bundle_release: "${value}"}'
Any help will be appriciated.
P.S- The question is edited as suggested by peak
First, when specifying a key name using a variable in the way you are doing, the variable must be parenthesized, so you would have:
{($bundle_release): ...}
Next, jq variables are not the same as shell variables and should be specified without quoting them, and without using bash-isms.
Third, when setting the value of the shell variable named value, you would have to quote the expression appropriately.
Fourth, to simplify things, use --argjson for $value.
Fifth, your sample JSON is not quite right. Once it's fixed, the following will work in a bash or bash-like environment (assuming you're using a version of jq that supports --argjson):
bundle_release="1034,567"
value='{"release":"2018.1006","kernel":"2.6.32-754.3.5.el6.x86_64","os":"6.10","current":true}'
jq --arg b "$bundle_release" --argjson v "$value" '
. + {($b): $v}' <<< "$version6json"
You're not giving the --arg option enough parameters: from the manual:
--arg name value:
This option passes a value to the jq program as a predefined variable. If you run jq with --arg foo bar, then
$foo is available in the program and has the value "bar". Note that value will be treated as a string, so
--arg foo 123 will bind $foo to "123".