I have a json structured like this:
[
{
"name": "object1",
"prop": "prop1",
"props": [
{ "prop1": "value1" },
{ "prop2": "value2" },
{ "prop3": "value3" }
]
},
{
"name": "object2",
"prop": "prop2",
"props": [
{ "prop1": "value4" },
{ "prop2": "value5" },
{ "prop3": "value6" }
]
}
]
I would like to extract the content of the props variable and have it as extra object properties, looking like this:
[
{
"name": "object1",
"prop": "prop1",
"prop1": "value1",
"prop2": "value2",
"prop3": "value3"
},
{
"name": "object2",
"prop": "prop2",
"prop1": "value4",
"prop2": "value5",
"prop3": "value6"
},
]
I've been trying to use map but I can't seem to be able to get rid of the array.
Use add to merge the propN objects into one, del to remove the original props:
jq '[.[] | . + (.props | add) | del(.props)]' file.json
You can indeed use map to shorten it a bit:
jq 'map(. + (.props | add) | del(.props))' file.json
Related
Consider the below input
{
"name": "examplename1",
"Date1": "value1",
"Date2": "value2",
"Date3": "value3"
}
{
"name": "examplename1",
"Date1": "value4",
"Date2": "value5",
"Date3": "value6"
}
{
"name": "examplename2",
"Date1": "value7",
"Date2": "value8",
"Date3": "value9"
}
{
"name": "examplename2",
"Date1": "value10",
"Date2":"value11",
"Date3": "value12"
}
Require output as below
{
"names": "examplename1",
"availabledates1":[
"value1",
"value4"
],
"availabledates2":[
"value2",
"value5"
],
"availabledates3":[
"value3",
"value6"
]
}
{
"names": "examplename2",
"availabledates1":[
"value7",
"value10"
],
"availabledates2":[
"value8",
"valu11"
],
"availabledates3":[
"value9",
"value12"
]
}
Using JQ
[inputs] | group_by(.name)[] | [{names: .[].name, availabledates1: [.[].Date1], availabledates2: [.[].Date2], availabledates3: [.[].Date3]}] | unique_by(.names) | .[]
Getting output
{
"names": "examplename1",
"availabledates1": [
"value4"
],
"availabledates2": [
"value5"
],
"availabledates3": [
"value6"
]
}
{
"names": "examplename2",
"availabledates1": [
"value7",
"value10"
],
"availabledates2": [
"value8",
"value11"
],
"availabledates3": [
"value9",
"value12"
]
}
Issue 1: This JQ ignores the first row in inputs.
Issue 2: If the input data set is very large this jq takes too much memory and eventually fails to execute as its doing multiple iterations which needs parallel threads.
Can refer this: https://jqplay.org/s/OOHAuv72GAL
Need here more efficient jq which does not fail on large data set and also considers first row in inputs.
Using group_by:
jq -n '
[inputs] | group_by(.name)[] | {
names: first.name,
avaliableDates1: map(.Date1),
avaliableDates2: map(.Date2),
avaliableDates3: map(.Date3)
}
'
Demo
Using reduce:
jq -n '
(reduce inputs as $i ({}; .[$i.name] |= (
.names = $i.name
| .avaliableDates1 += [$i.Date1]
| .avaliableDates2 += [$i.Date2]
| .avaliableDates3 += [$i.Date3]
)))[]
'
Demo
Output:
{
"names": "examplename1",
"avaliableDates1": [
"value1",
"value4"
],
"avaliableDates2": [
"value2",
"value5"
],
"avaliableDates3": [
"value3",
"value6"
]
}
{
"names": "examplename2",
"avaliableDates1": [
"value7",
"value10"
],
"avaliableDates2": [
"value8",
"value11"
],
"avaliableDates3": [
"value9",
"value12"
]
}
Given the List:
[
{
"prop1": "prop11",
"prop2": "prop21",
"list": ["element1", "element2", "element3"]
},
{
"prop1": "prop12",
"prop2": "prop22",
"list": ["element1", "element3"]
},
{
"prop1": "prop13",
"prop2": "prop23",
"list": ["element2", "element3"]
}
]
While using jq, how to filter only the root's list elements by the children's lists.
For example, return all elements whose list contains element1. The output should be:
[
{
"prop1": "prop11",
"prop2": "prop21",
"list": ["element1", "element2", "element3"]
},
{
"prop1": "prop12",
"prop2": "prop22",
"list": ["element1", "element3"]
}
]
or
{"prop1": "prop11","prop2": "prop21","list": ["element1", "element2", "element3"]}
{"prop1": "prop12","prop2": "prop22","list": ["element1", "element3"]}
index makes for a short and in practice quite fast solution:
jq 'map(select(.list|index("element1")))'
or
jq -c '.[] | select(.list|index("element1"))'
Check if any of the list's element equals == the given string:
jq 'map(select(any(.list[]; . == "element1")))'
[
{
"prop1": "prop11",
"prop2": "prop21",
"list": [
"element1",
"element2",
"element3"
]
},
{
"prop1": "prop12",
"prop2": "prop22",
"list": [
"element1",
"element3"
]
}
]
Demo
Use .[] | … instead of map(…) to get rid of the outer array:
jq '.[] | select(any(.list[]; . == "element1"))'
{
"prop1": "prop11",
"prop2": "prop21",
"list": [
"element1",
"element2",
"element3"
]
}
{
"prop1": "prop12",
"prop2": "prop22",
"list": [
"element1",
"element3"
]
}
Demo
I need some help using jq to sort an array of elements where each element contains a nested
tags array of elements. My input JSON looks like this:
{
"result": [
{
"name": "ct-1",
"tags": [
{
"key": "service_name",
"value": "BaseCT"
},
{
"key": "sequence",
"value": "bb"
}
]
},
{
"name": "ct-2",
"tags": [
{
"key": "service_name",
"value": "BaseCT"
},
{
"key": "sequence",
"value": "aa"
}
]
}
]
}
I would like to sort using the value of the sequence tag in the nested tags array so that the output looks like this:
{
"result": [
{
"name": "ct-2",
"tags": [
{
"key": "service_name",
"value": "BaseCT"
},
{
"key": "sequence",
"value": "aa"
}
]
},
{
"name": "ct-1",
"tags": [
{
"key": "service_name",
"value": "BaseCT"
},
{
"key": "sequence",
"value": "bb"
}
]
}
]
}
I have tried the following jq command:
$ jq '.result |= ([.[] | .tags[] | select(.key == "sequence") | .value] | sort_by(.))' input.json
but I get the following result:
{
"result": [
"aa",
"bb"
]
}
Please let me know if you know how to deal with this scenario.
from_entries converts an array of key-value pairs to an object, you can use it with sort_by like this:
.result |= sort_by(.tags | from_entries | .sequence)
I've just started using jq json parser, is there anyway to choose multiple select?
I have this:
cat file | jq -r '.Instances[] | {ip: .PrivateIpAddress, name: .Tags[]}
| select(.name.Key == "Name")'
And I need to also include the .name.Key == "Type"
This is the JSON:
{
"Instances": [
{
"PrivateIpAddress": "1.1.1.1",
"Tags": [
{
"Value": "Daily",
"Key": "Backup"
},
{
"Value": "System",
"Key": "Name"
},
{
"Value": "YES",
"Key": "Is_in_Domain"
},
{
"Value": "PROD",
"Key": "Type"
}
]
}
]
}
And this is the current output:
{
"ip": "1.1.1.1",
"name": "System"
}
{
"ip": "2.2.2.2",
"name": "host"
}
{
"ip": "3.3.3.3",
"name": "slog"
}
Desired output:
{
"ip": "1.1.1.1",
"name": "System",
"type": "PROD"
}
{
"ip": "2.2.2.2",
"name": "host",
"type": "PROD"
}
{
"ip": "3.3.3.3",
"name": "slog",
"type": "PROD"
}
What is the right way to do it? Thanks.
There's no "right" way to do it, but there are approaches to take that can make things easier for you.
The tags are already in a format that makes converting to objects simple (they're object entries). Convert the tags to an object for easy access to the properties.
$ jq '.Instances[]
| .Tags |= from_entries
| {
ip: .PrivateIpAddress,
name: .Tags.Name,
type: .Tags.Type
}' file
I have an input json document as so:
[
{
"Name": "one",
"Tags": [
{
"Key": "Name",
"Value": "important"
},
{
"Key": "OtherTag",
"Value": "irrelevant"
}
]
},
{
"Name": "two",
"Tags": [
{
"Key": "OtherTag",
"Value": "irrelevant2"
},
{
"Key": "Name",
"Value": "important"
}
]
},
{
"Name": "three",
"Tags": [
{
"Key": "Name",
"Value": "important2"
},
{
"Key": "OtherTag",
"Value": "irrelevant3"
}
]
}
]
I want to use jq to group the three records by tag Value where the Key = "Name". The result would be two arrays, one with two records in it and one with one. The array with two records would have two because both records share the same tag with a value of "important". Here is what the result would look like:
[
[
{
"Name": "one",
"Tags": [
{
"Key": "Name",
"Value": "important"
},
{
"Key": "OtherTag",
"Value": "irrelevant"
}
]
},
{
"Name": "two",
"Tags": [
{
"Key": "OtherTag",
"Value": "irrelevant2"
},
{
"Key": "Name",
"Value": "important"
}
]
},
],
[
{
"Name": "three",
"Tags": [
{
"Key": "Name",
"Value": "important2"
},
{
"Key": "OtherTag",
"Value": "irrelevant3"
}
]
}
]
]
I just can't figure out how to do this with jq. Does anyone have any ideas?
Your proposed solution is fine, but if you don't mind converting the arrays of Key-Value pairs into objects, then the following can be used:
map( .Tags |= from_entries ) | group_by(.Tags.Name)
This at least makes the "group_by" easy to understand; furthermore, it would be easy to convert the .Tags objects back to key-value pairs (with lower-case "key" and "value"):
map( .Tags |= from_entries ) | group_by(.Tags.Name)
| map(map( .Tags |= to_entries))
Key/Value capitalization
One way to recover the capitalized Key/Value tags would be to tweak the above as follows:
def KV: map( {Key: .key, Value: .value} );
map( .Tags |= from_entries ) | group_by(.Tags.Name)
| map(map( .Tags |= (to_entries | KV)))