I would like to make (using jq) from input:
{
key1: value1,
key2: value2,
key3: [
{key4: value3, key5: value5},
{key4: value4, key5: value6},
.
.
.
{key4: valueN, key5: valueM},
.
.
.
]
}
this output:
{
key1: value1,
key2: value2,
value3: value5,
value4: value6,
.
.
.
valueN: valueM,
.
.
.
}
How to do it?
I have tried some ways, but I haven't able to find the proper solution, mainly jq generated separated objects, I would like to have only one output object containing keys from input object.
For the given input, the following should produce the desired output
.key3 as $t | del(.key3) | . + ( $t | map(to_entries | { (.[0].value) : .[1].value } ) | add )
jq-play link
I would use a reduce on .key3's items and append them successively:
reduce .key3[] as {$key4, $key5} (del(.key3); .[$key4] = $key5)
{
"key1": "value1",
"key2": "value2",
"value3": "value5",
"value4": "value6",
"valueN": "valueM"
}
Demo
key3 almost has the format that from_entries uses, tweak it?
.key3 |
map(.key = .key4) |
map(.value = .key5) |
map( del(.key4,.key5) )
Converts key3 array to:
[
{
"key": "value3",
"value": "value5"
},
{
"key": "value4",
"value": "value6"
},
{
"key": "valueN",
"value": "valueM"
}
]
Pipe to from_entries:
( ... | from_entries) as $obj | . + $obj | del(.key3)
Output:
{
"key1": "value1",
"key2": "value2",
"value3": "value5",
"value4": "value6",
"valueN": "valueM"
}
Related
I'm working with a JSON object having the following structure:
{
"key-foo-1.0": [
{
"key1": "foo",
"key2": "bar",
"id": "01"
},
{
"key1": "foo",
"key2": "bar",
"id": "23"
}
],
"key-bar-1.0": [
{
"key1": "foo",
"key2": "bar",
"id": "45"
},
{
"key1": "foo",
"key2": "bar",
"id": "67"
}
],
"key-baz-1.0": [
{
"key1": "foo",
"key2": "bar",
"id": "89"
}
]
}
I want to get all the id values where the "parent" key name matches the pattern .*foo.* or .*bar.*.
So in my example something like this:
cat json | jq <some filter>
01
23
45
67
Based on https://unix.stackexchange.com/questions/443884/match-keys-with-regex-in-jq I tried:
$ cat json | jq 'with_entries(if (.key|test(".*foo.*$")) then ( {key: .key, value: .value } ) else empty end )'
{
"key-foo-1.0": [
{
"key1": "foo",
"key2": "bar",
"id": "01"
},
{
"key1": "foo",
"key2": "bar",
"id": "23"
}
]
}
But I don't really know how to continue.
I also think there is a better/simpler solution.
You could go with:
jq -r '.[keys_unsorted[] | select(test(".*foo.*|.bar.."))][].id'
01
23
45
67
This gathers all keys using keys_unsorted, then selects those matching the regular expression in test. The wrapping .[…] descends into them, the following [] iterates over the children, and .id outputs the values as raw text using the -r flag.
you can use the following JQ expression:
jq 'to_entries[] | select(.key | test(".*foo.*|.*bar.*")) | .value[] | .id'
JQ playground example
(I’m not sure the technical terms to use but can update the question if someone can clarify the terminology I’m lacking for what I'm trying to do. It might help someone find this answer in the future.)
Given the input JSON, how would I use jq to produce the expected output?
Input:
{
"items": {
"item1": {
"part1": {
"a": {
"key1": "value",
"key2": "value"
},
"b": {
"key1": "value",
"key2": "value"
}
},
"part2": {
"c": {
"key1": "value",
"key2": "value"
},
"d": {
"key1": "value",
"key2": "value"
}
}
},
"item2": {
"part3": {
"e": {
"key1": "value",
"key2": "value"
},
"f": {
"key1": "value",
"key2": "value"
}
},
"part4": {
"g": {
"key1": "value",
"key2": "value"
},
"h": {
"key1": "value",
"key2": "value"
}
}
}
}
}
Expected output:
{
"item1": [
"part1.a",
"part1.b",
"part2.c",
"part2.d"
]
"item2": [
"part3.e",
"part3.f"
"part4.g",
"part4.h"
]
}
Try this:
.items | map_values([path(.[][]) | join(".")])
Online demo
Each output path will contain as many path components as the number of []s in the .[][] part; in other words, if you change .[][] to .[][][], for example, you'll see part1.a.key1, part1.a.key2, etc.
This would do it:
# Output: a stream
def keyKey:
keys_unsorted[] as $k | $k + "." + (.[$k] | keys_unsorted[]);
.items | map_values([keyKey])
Some aspects are underspecified. For instance, you don't specify how deep the aggregation should go for the array items. Is it always two levels deep, or is it the whole tree but the last level?
Here's one way how you would go two levels deep with the keys sorted alphabetically:
jq '.items | .[] |= [keys[] as $k | $k + "." + (.[$k] | keys[])]'
Demo
Here's another way how to go down until the second-to-last level:
jq '.items | .[] |= ([path(.. | scalars)[:-1] | join(".")] | unique)'
Demo
Output:
{
"item1": [
"part1.a",
"part1.b",
"part2.c",
"part2.d"
],
"item2": [
"part3.e",
"part3.f",
"part4.g",
"part4.h"
]
}
the unique sequence of jq paths of 'keys' to each and every leaf
is returned from json2jqpath.jq
json2jqpath.jq dat.json
.
.items
.items|.item1
.items|.item1|.part1
.items|.item1|.part1|.a
.items|.item1|.part1|.a|.key1
.items|.item1|.part1|.a|.key2
.items|.item1|.part1|.b
.items|.item1|.part1|.b|.key1
.items|.item1|.part1|.b|.key2
.items|.item1|.part2
.items|.item1|.part2|.c
.items|.item1|.part2|.c|.key1
.items|.item1|.part2|.c|.key2
.items|.item1|.part2|.d
.items|.item1|.part2|.d|.key1
.items|.item1|.part2|.d|.key2
.items|.item2
.items|.item2|.part3
.items|.item2|.part3|.e
.items|.item2|.part3|.e|.key1
.items|.item2|.part3|.e|.key2
.items|.item2|.part3|.f
.items|.item2|.part3|.f|.key1
.items|.item2|.part3|.f|.key2
.items|.item2|.part4
.items|.item2|.part4|.g
.items|.item2|.part4|.g|.key1
.items|.item2|.part4|.g|.key2
.items|.item2|.part4|.h
.items|.item2|.part4|.h|.key1
.items|.item2|.part4|.h|.key2
It is not the output you asked for but as another noted, your question may be somewhat under specified. starting from a preprocessed structure such as this has the advantage of reducing every json file to its set of paths to start fiddling with.
json2jqpath
I have an json file as follows below-
{
"key1": [
"value1"
],
"key2": [
"value2"
],
"key3": [
"value3"
],
"key4": {
"name": "value4"
},
"key5": [
{
"field1": "abc",
"field2": "xyz"
}
]
}
I want to remove field 2 from key5 array and add another field say field3 in key5 array using jq
I tried various way but couldn't figure it how to do this in single command.
Can you please help . Thanks in advance
.key5[] |= ( ... ) allows you to modify each element of the array found at .key5.
Therein, we can use the usual commands for deleting and adding fields.
jq '.key5[] |= ( del( .field2 ) | .field3 = "foo" )'
Demo in jqplay
Here's a variation where the value to add is provided as a command-line argument:
jq --arg val foo '.key5[] |= ( del( .field2 ) | .field3 = $val )'
This assumes that the insert and deletion keys (field2 and field3) are also dynamic, not just the inserted value.
Use --arg and --argjson to initialize variables with values from outside the filter, here field2 as a string and {"field3": "new"} as a JSON object
Use the update operator |= on the element you want to change, here .key5[].
Note: .key5 is an array, not an object, thus it has no fields. But the array does contain objects as its elements. Using .key5[] to access the array elements will update all objects in the array. To update just one of them, say, the first one, use .key[0] instead.
Use delpaths to delete a given path, here the top-level field [$delkey].
Use simple addition + to add a new key/value pair as object, here $add.
jq --arg delkey 'field2' --argjson addobj '{"field3": "new"}' '
.key5[] |= delpaths([[$delkey]]) + $addobj
' input.json
{
"key1": [
"value1"
],
"key2": [
"value2"
],
"key3": [
"value3"
],
"key4": {
"name": "value4"
},
"key5": [
{
"field1": "abc",
"field3": "new"
}
]
}
Demo
If you want to provide the new object's key and value separately, i.e. as strings, not as a pre-composed JSON, you need a third input variable
jq --arg delkey 'field2' --arg addkey 'field3' --arg addval 'new' '
.key5[] |= (delpaths([[$delkey]]) | .[$addkey] = $addval)
' input.json
Demo
Try filter:
del(.key5[0].field2) | .key5[0] |= .+{field3:"foo"}
Output:
{
"key1": [
"value1"
],
"key2": [
"value2"
],
"key3": [
"value3"
],
"key4": {
"name": "value4"
},
"key5": [
{
"field1": "abc",
"field3": "foo"
}
]
}
Demo
https://jqplay.org/s/EB8MpWRxU2
I have a text file with following values in
input.txt
key1=value1\r
key2=value2
key3=value3\r
key4=value4
need the jq rexpression to convert it to below json format by removing "\r" also
output.json
{
"Environment": {
"Variables": {
"key1": "value1",
"key2": "value2",
"key3": "value3",
"key4": "value4"
}
}
}
I have tried the below expression and getting the
jq -Rs [ split("\n")[] | select(length > 0) | split("=") | {(.[0]): .[1]} ]
and getting the below output
[
{
"key1ey1": "Value1\r"
},
{
"key2": "value2"
},
{
"key3": "value3\r"
},
{
"key4": "value4"
}
]
jq solution:
jq -sR '{"Environment":
{"Variables": [split("\n")[:-1][] | rtrimstr("\\r")
| split("=") | {(.[0]): .[1]}
] | add
}
}' input.txt
The output:
{
"Environment": {
"Variables": {
"key1": "value1",
"key2": "value2",
"key3": "value3",
"key4": "value4"
}
}
}
Caveat
This solution assumes = does not appear in the "value" section of the input strings.
The following approach allows = to appear in the value portion of the key=value strings:
Invocation: jq -n -R -f program.jq input.txt
program.jq:
[inputs | sub("\\\\r$";"") | capture("^(?<key>[^=]*)=(?<value>.*)")]
| from_entries
| {Environment: { Variables: .}}
Using jq I would like to inverse a json object so that the property becomes the value and the value becomes the key.
Source:
{
"123": "Foobar"
"567": "Poit"
}
Goal:
{
"Foobar": "123"
"Poit": "567"
}
How can I achieve that?
In your particular case:
to_entries | map( {(.value) : .key } ) | add
More robustly:
to_entries | map( {(.value|tostring) : .key } ) | add
Or if you prefer:
with_entries( .key as $k | .key = (.value|tostring) | .value = $k )
Caveat: all these are potentially lossy.
If some keys have equal values then probably you would like to get an array of keys as value:
to_entries
| map( {(.value) : {(.key):null} } )
| reduce .[] as $item ({}; . * $item)
| to_entries
| map({key:.key, value:(.value|keys)})
| from_entries
input:
{
"key1": "val0",
"key2": "val1",
"key3": "val1"
}
output:
{
"val0": ["key1"],
"val1": ["key2", "key3"]
}
I would use an approach similar to #peak's answer but without using the add method
First use to_entries to get an output like this:
-> to_entries
Output:
[
{
"key": "123",
"value": "Foobar"
},
{
"key": "567",
"value": "Poit"
}
]
Then use map to swap the keys with values:
-> to_entries | map({key: .value|tostring, value: .key}) ###**tostring** method converts a number to string since keys can't be numbers
Output:
[
{
"key": "Foobar",
"value": "123"
},
{
"key": "Poit",
"value": "567"
}
]
Finally use from_entries to remove the key/value and return back to the original format:
-> to_entries | map({key: .value|tostring, value: .key}) | from_entries
Output:
{
"Foobar": "123",
"Poit": "567"
}