Modify JSON values based on array of keys using jq - json

I need to change the value of a set of keys (defined in a variable) in a JSON object using jq.
As example, I have this JSON object:
{
foo: {
bar: 1,
baz: 2,
qux: 3
}
}
and the following variable:
update_keys = ["bar", "baz"]
I would like to say 'change the value of the keys in update_keys to X'.
The following works:
.foo = (.foo |
to_entries |
map(if .key == "bar" or .key == "baz"
then . + { "value":"X" }
else .
end) |
from_entries)
But instead of if .key == "bar" or .key == "baz" I am looking for a way to say if .key in update_keys, or a similar logic.

Here you go.
Filter
.foo |= with_entries( .value = if ([.key] | inside(["bar", "baz"])) then "X" else .value end )
Input
{
"foo": {
"bar": 1,
"baz": 2,
"qux": 3
}
}
Output
{
"foo": {
"bar": "X",
"baz": "X",
"qux": 3
}
}
Check out the cookbook for more recipies and techniques of jq usage:
https://github.com/stedolan/jq/wiki/Cookbook

Here's a slightly different approach using --argjson to parameterize update_keys, and index/1:
$ cat update.jq
.foo |= with_entries( . as $in
| if $update_keys | index($in.key) then .value = "X" else empty end)
$ update_keys='["bar", "baz"]'
$ jq --argjson update_keys "$update_keys" -f update.jq input.json
Output:
{
"foo": {
"bar": "X",
"baz": "X"
}
}

In this problem since $update_keys is just an array all that is needed is
.foo[ $update_keys[] ] = "X"
e.g. if
["bar","baz"] as $update_keys
| .foo[ $update_keys[] ] = "X"
is in filter.jq and data.json contains the (slighty corrected) data
{
"foo": {
"bar": 1,
"baz": 2,
"qux": 3
}
}
then
jq -M -f filter.jq data.json
produces
{
"foo": {
"bar": "X",
"baz": "X",
"qux": 3
}
}
If you want to pass in the value for update keys instead of defining it in your script you can easily use --argjson as peak's answer shows.

Related

Reduce nested json (PowerDNS stats)

I'm trying to improve on a jq reduce, but finding that some of the returned data is nested and the code I'm using breaks on that.
This is where I've got the jq code from: https://github.com/influxdata/telegraf/tree/master/plugins/inputs/powerdns_recursor
Taking tonumber off I get the following clipped output:
[...]
"x-ourtime8-16": "0",
"zone-disallowed-notify": "0",
"response-by-qtype": [
{
"name": "A",
"value": "8958"
},
{
"name": "NS",
"value": "6"
},
[...]
The original code, with tonumber left in:
curl -s -H 'X-API-Key: <key>' http://127.0.0.1:8082/api/v1/servers/localhost/statistics | jq 'reduce .[] as $item ({}; . + { ($item.name): ($item.value|tonumber)})'
The output I'm after:
[...]
"x-ourtime8-16": 0,
"zone-disallowed-notify": 0,
"response-by-qtype.A": 8958,
"response-by-qtype.NS": 6,
[...]
I've spent some time Googling jq and nested input, but I don't want the index numbers this gave me in the names. I'm hoping a small tweak will do the trick.
To transform this input :
{
"x-ourtime8-16": "0",
"zone-disallowed-notify": "0",
"response-by-qtype": [
{
"name": "A",
"value": "8958"
},
{
"name": "NS",
"value": "6"
}
]
}
You can run :
jq ' to_entries |
map(if (.value | type) == "string"
then .value |= tonumber
else .key as $key | .value[] |
.name |= $key+"."+. |
.value |= tonumber
end
) | from_entries
' input.json
to get :
{
"x-ourtime8-16": 0,
"zone-disallowed-notify": 0,
"response-by-qtype.A": 8958,
"response-by-qtype.NS": 6
}
You can convert numeric strings to numbers using:
if type == "string" then . as $in | try tonumber catch $in else . end
As a post-processing step, you could use walk as a wrapper:
walk(if type == "string" then . as $in | try tonumber catch $in else . end)

JSON/JQ: Merge 2 files on key-value with condition

I have 2 JSON files. I would like to use jq to take the value of "capital" from File 2 and merge it with File 1 for each element where the same "name"-value pair occurs. Otherwise, the element from File 2 should not occur in the output. If there is no "name"-value pair for an element in File 1, it should have empty text for "capital."
File 1:
{
"countries":[
{
"name":"china",
"continent":"asia"
},
{
"name":"france",
"continent":"europe"
}
]
}
File 2:
{
"countries":[
{
"name":"china",
"capital":"beijing"
},
{
"name":"argentina",
"capital":"buenos aires"
}
]
}
Desired result:
{
"countries":[
{
"name":"china",
"continent":"asia",
"capital":"beijing"
},
{
"name":"france",
"continent":"europe",
"capital":""
}
]
}
You could first construct a dictionary from File2, and then perform the update, e.g. like so:
jq --argfile dict File2.json '
($dict.countries | map( {(.name): .capital}) | add) as $capitals
| .countries |= map( .capital = ($capitals[.name] // ""))
' File2.json
From a JSON-esque perspective, it would probably be better to use null for missing values; in that case, you could simplify the above by omitting // "".
Using INDEX/2
If your jq has INDEX/2, then the $capitals dictionary could be constructed using the expression:
INDEX($dict.countries[]; .name) | map_values(.capital)
Using INDEX makes the intention clearer, but if efficiency were a major concern, you'd probably be better off using reduce explicitly:
reduce $dict.countries[] as $c ({}; . + ($c | {(.name): .capital}))
One way:
$ jq --slurpfile file2 file2.json '
{ countries:
[ .countries[] |
. as $curr |
$curr + { capital: (($file2[0].countries[] | select(.name == $curr.name) | .capital) // "") }
]
}' file1.json
{
"countries": [
{
"name": "china",
"continent": "asia",
"capital": "beijing"
},
{
"name": "france",
"continent": "europe",
"capital": ""
}
]
}
An alternative:
$ jq -n '{ countries: ([inputs] | map(.countries) | flatten | group_by(.name) |
map(select(.[] | has("continent")) | add | .capital //= ""))
}' file[12].json

jq: sort object values

I want to sort this data structure by the object keys (easy with -S and sort the object values (the arrays) by the 'foo' property.
I can sort them with
jq -S '
. as $in
| keys[]
| . as $k
| $in[$k] | sort_by(.foo)
' < test.json
... but that loses the keys.
I've tried variations of adding | { "\($k)": . }, but then I end up with a list of objects instead of one object. I also tried variations of adding to $in (same problem) or using $in = $in * { ... }, but that gives me syntax errors.
The one solution I did find was to just have the separate objects and then pipe it into jq -s add, but ... I really wanted it to work the other way. :-)
Test data below:
{
"": [
{ "foo": "d" },
{ "foo": "g" },
{ "foo": "f" }
],
"c": [
{ "foo": "abc" },
{ "foo": "def" }
],
"e": [
{ "foo": "xyz" },
{ "foo": "def" }
],
"ab": [
{ "foo": "def" },
{ "foo": "abc" }
]
}
Maybe this?
jq -S '.[] |= sort_by(.foo)'
Output
{
"": [
{
"foo": "d"
},
{
"foo": "f"
},
{
"foo": "g"
}
],
"ab": [
{
"foo": "abc"
},
{
"foo": "def"
}
],
"c": [
{
"foo": "abc"
},
{
"foo": "def"
}
],
"e": [
{
"foo": "def"
},
{
"foo": "xyz"
}
]
}
#user197693 had a great answer. A suggestion I got in a private message elsewhere was to use
jq -S 'with_entries(.value |= sort_by(.foo))'
If for some reason using the -S command-line option is not a satisfactory option, you can also perform the by-key sort using the to_entries | sort_by(.key) | from_entries idiom. So a complete solution to the problem would be:
.[] |= sort_by(.foo)
| to_entries | sort_by(.key) | from_entries

Modifying nested json conditionally using jq

I would like to modify something like the following JSON:
{
"foobar": {
"a": {
"adkjfe": {
"A": 1,
"foo": "bar"
}
},
"b": {
"ekjaei": {
"A": 2,
"bar": "foo"
}
}
}
}
to add more data say {"baz": ["bing", "bop"]} to the parent of A if A=1. Assuming I don't know the parent key while leaving the rest of the json untouched. I've tried many different things including:
.foobar | .. | .. | .[] | if select(.A==1) then . += {"baz": "bing"} else . end
which gives me an error and only my modified section.
The result, in this case, that I would like to see is:
{
"foobar": {
"a": {
"adkjfe": {
"A": 1,
"foo": "bar",
"baz": ["bing", "bop"]
}
},
"b": {
"ekjaei": {
"A": 2,
"bar": "foo"
}
}
}
}
Select only fields that are of type object and that match your condition (A == 1) :
jq '(.foobar | .. | select(type == "object" and .A == 1)) |= .+ {"baz": ["bing", "bop"]}' test.json
The () around the filter query take care that the whole document will be returned with the updated fields and not just your subdocument
In general it is better to avoid .. if possible, for reasons of efficiency. In the present case, the following will do the job:
(.foobar[][] | select(.A == 1)) |= .+ {"baz":["bing", "bop"]}

How to swap key and value of an object using jq?

Using jq I would like to inverse a json object so that the property becomes the value and the value becomes the key.
Source:
{
"123": "Foobar"
"567": "Poit"
}
Goal:
{
"Foobar": "123"
"Poit": "567"
}
How can I achieve that?
In your particular case:
to_entries | map( {(.value) : .key } ) | add
More robustly:
to_entries | map( {(.value|tostring) : .key } ) | add
Or if you prefer:
with_entries( .key as $k | .key = (.value|tostring) | .value = $k )
Caveat: all these are potentially lossy.
If some keys have equal values then probably you would like to get an array of keys as value:
to_entries
| map( {(.value) : {(.key):null} } )
| reduce .[] as $item ({}; . * $item)
| to_entries
| map({key:.key, value:(.value|keys)})
| from_entries
input:
{
"key1": "val0",
"key2": "val1",
"key3": "val1"
}
output:
{
"val0": ["key1"],
"val1": ["key2", "key3"]
}
I would use an approach similar to #peak's answer but without using the add method
First use to_entries to get an output like this:
-> to_entries
Output:
[
{
"key": "123",
"value": "Foobar"
},
{
"key": "567",
"value": "Poit"
}
]
Then use map to swap the keys with values:
-> to_entries | map({key: .value|tostring, value: .key}) ###**tostring** method converts a number to string since keys can't be numbers
Output:
[
{
"key": "Foobar",
"value": "123"
},
{
"key": "Poit",
"value": "567"
}
]
Finally use from_entries to remove the key/value and return back to the original format:
-> to_entries | map({key: .value|tostring, value: .key}) | from_entries
Output:
{
"Foobar": "123",
"Poit": "567"
}