jq: output values of ids instead of numbers - json

Here's my input json:
{
"channels": [
{ "id": 1, "name": "Pop"},
{ "id": 2, "name": "Rock"}
],
"links": [
{ "id": 2, "streams": [ {"url": "http://example.com/rock"} ] },
{ "id": 1, "streams": [ {"url": "http://example.com/pop"} ] }
]
}
This is what I want as an output:
"http://example.com/pop"
"Pop"
"http://example.com/rock"
"Rock"
So I need jq to replace .channels[].id with .links[].streams[0].url based on .links[].id
I don't know if it's right, but this is how I managed to output the urls:
(.channels[].id | tostring) as $ids | [.links[]] | map({(.id | tostring): .streams[0].url}) | add as $urls | $urls[$ids]
"http://example.com/pop"
"http://example.com/rock"
The question is, how do I add .channels[].name to it?

You sometimes have to be careful what you ask for, but this will produce the result you said you want:
.channels[] as $channel
| $channel.name,
(.links[] | select(.id == $channel.id) | .streams[0].url)
Output for the given input:
"Pop"
"http://example.com/pop"
"Rock"
"http://example.com/rock"

Here is a solution which uses reduce and setpath to make a $urls lookup table from .links and then scans .channels generating corresponding urls and names.
(
reduce .links[] as $l (
{};
setpath([ $l.id|tostring ]; [$l.streams[].url])
)
) as $urls
| .channels[]
| $urls[ .id|tostring ][], .name
If multiple urls are present in the "streams" attribute this will
print them all before printing the name. e.g. if the input is
{
"channels": [
{ "id": 1, "name": "Pop"},
{ "id": 2, "name": "Rock"}
],
"links": [
{ "id": 2, "streams": [ {"url": "http://example.com/rock"},
{"url": "http://example.com/hardrock"} ] },
{ "id": 1, "streams": [ {"url": "http://example.com/pop"} ] }
]
}
the output will be
"http://example.com/pop"
"Pop"
"http://example.com/rock"
"http://example.com/hardrock"
"Rock"

Related

JQ: Merging array entities with same attribute name

I have a data structure like this:
[
{
"some_id": "123",
"items_1": [
{
"label": "my_name"
}
],
"items_2": []
},
{
"some_id": "123",
"items_1": [],
"items_2": [
"value_1",
"value_3"
]
},
{
"some_id": "123",
"items_1": [],
"items_2": [
"value_1",
"value_2"
]
}
]
And I want to modify the data into something like
[
{
"some_id": "123",
"items_1": [
{
"label": "my_name"
}
],
"items_2": [
"value_1",
"value_2",
"value_3"
]
}
]
Basically taking any fields that are the same and concatenating the arrays together. Similarly, items_1 can have some value for the same id down the line and I want to concatenate that array with another if needed.
I have tried using JQ with something like
jq -Mr '[ group_by(.media_url)[] | add | tojson ] | join(",\n")' test.json
However this doesnt seem to be working.
Would the following work for you?
group_by(.some_id) | map({
some_id: map(.some_id) | first,
items_1: map(.items_1) | add | unique,
items_2: map(.items_2) | add | unique })
demo

Fill arrays in the first input with elements from the second based on common field

I have two files and I would need to merge the elements of the second file into an object array in the first file based on searching the reference field.
The first file:
[
{
"reference": 25422,
"order_number": "10_1",
"details" : []
},
{
"reference": 25423,
"order_number": "10_2",
"details" : []
}
]
The second file:
[
{
"record_id" : 1,
"reference": 25422,
"row_description": "descr_1_0"
},
{
"record_id" : 2,
"reference": 25422,
"row_description": "descr_1_1"
},
{
"record_id" : 3,
"reference": 25423,
"row_description": "descr_2_0"
}
]
I would like to get:
[
{
"reference": 25422,
"order_number": "10_1",
"details" : [
{
"record_id" : 1,
"reference": 25422,
"row_description": "descr_1_0"
},
{
"record_id" : 2,
"reference": 25422,
"row_description": "descr_1_1"
}
]
},
{
"reference": 25423,
"order_number": "10_2",
"details" :[
{
"record_id" : 3,
"reference": 25423,
"row_description": "descr_2_0"
}
]
}
]
Below is my code in es_func.jq file launched by this command:
jq -n --argfile f1 es_file1.json --argfile f2 es_file2.json -f es_func.jq
INDEX($f2[] ; .reference) as $details
| $f1
| map( ($details[.reference|tostring]| .row_description) as $vn
| if $vn then .details = [{"row_description" : $vn}] else . end)
I get the result only for the last record in 25422 reference with "row description": "descr_1_1" and not have "row_description": "descr_1_0"
[
{
"reference": 25422,
"order_number": "10_1",
"details": [
{
"row_description": "descr_1_1"
}
]
},
{
"reference": 25423,
"order_number": "10_2",
"details": [
{
"row_description": "descr_2_0"
}
]
}
]
I think I'm close to the solution but something is still missing. Thank you
This would be way easier if you used reduce instead.
jq 'reduce inputs[] as $rec (INDEX(.reference);
.[$rec.reference | tostring].details += [$rec]
) | map(.)' es_file1.json es_file2.json
Online demo
Here's a straightforward, reduce-free solution:
jq '
group_by(.reference)
| INDEX(.[]; .[0]|.reference|tostring) as $dict
| input
| map_values(. + {details: $dict[.reference|tostring]})
' 2.json 1.json

jq to filter inner array elements but return the whole JSON

TL;DR
How can I return the whole JSON after filtering inner array elements of a top-level key?
Detailed explanation
I have a JSON describing the COCO image database and it is formatted as follows (irrelevant elements truncated as ...).
{
"info": {
"description": "COCO 2017 Dataset",
...
},
"licenses": [
{
"url": "http://creativecommons.org/licenses/by-nc-sa/2.0/",
...
},
...
],
"images": [
{
"license": 4,
...
},
"annotations": [
{
"segmentation": [
[
510.66,
...
]
],
"area": 702.1057499999998,
"iscrowd": 0,
"image_id": 289343,
"bbox": [
473.07,
395.93,
38.65,
28.67
],
"category_id": 18,
"id": 1768
},
"categories": [
{
"supercategory": "person",
...
},
]
}
I need to filter annotations where category_id has one of several values, for example 1, 2.
I can successfully filter such category_ids with
jq -C ' .annotations[] | select( .category_id == 1 or .category_id == 2 ) ' instances_val2017.json | less -R
However, what is returned are only the annotations element of the total JSON as below.
{
"segmentation": [
[
162.72,
...
]
],
"area": 426.9120499999995,
"iscrowd": 0,
"image_id": 45596,
"bbox": [
161.52,
507.18,
46.45,
19.16
],
"category_id": 2,
"id": 124742
}
{
...
{
I know it's possible to return these elements as an array by wrapping the expression in [] but how can I return the entire original JSON after filtering the specified category ids?
Okay I spent 3 hours trying to solve this yesterday then this morning I posted this question and subsequently figured it out!
Here is the solution which uses the |= operator which modifies an element in place.
jq '.annotations |= map(select(.category_id | contains(1,2)))' instances_val2017.json
As per the suggestion of #peak, here is the command with == instead of contains.
jq '.annotations |= map(select(.category_id == (1,2)))' instances_val2017.json

Change subelement with jq

I have a structure that looks like so
[
[
{
"ID": "grp1-001",
},
{
"ID": "grp1-002",
},
{
"ID": "grp1-003",
},
{
"ID": "grp1-004",
},
{
"ID": "grp1-005",
},
{
"ID": "grp1-006",
}
],
[
{
"ID": "grp2-001",
},
{
"ID": "grp2-002",
},
{
"ID": "grp2-003",
},
{
"ID": "grp2-004",
},
{
"ID": "grp2-005",
},
{
"ID": "grp2-006",
}
.......
what I need to get as a result of the modification is this
[
[
["1", "grp1-001"],
["2", "grp1-002"],
["3", "grp1-003"],
["4", "grp1-004"],
["5", "grp1-005"],
["6", "grp1-006"],
],
[
["1", "grp2-001"],
["2", "grp2-002"],
["3", "grp2-003"],
["4", "grp2-004"],
["5", "grp2-005"],
["6", "grp2-006"],
],
Which means I need to keep the external structure (outside array and an internal grouping) but convert the inner dict to an array and replace the "ID" key with a value (that will come from external source like --argjson). I am not even sure how to start - any ideas/resources are highly appreciated.
Assuming you're just taking the objects and transforming them to pairs of the index in the array and the ID value, you could do this:
map([to_entries[] | [.key + 1, .value.ID | tostring]])
https://jqplay.org/s/RBac7SPfdG
Using to_entries/0 on an array gives you an array of key/value (index/value) pairs. You could then shift the indices by 1 and convert to strings.

jq: extract a specific key from one object to another

I have two JSON files.
file1.json:
{
"Fruits": [
{
"name": "Apple",
"something_else": 123,
"id": 652090
},
{
"name": "Orange",
"something_else": 456,
"id": 28748
}
]}
file2.json:
{
"Fruits": [
{
"weight": 5,
"id": 652090
},
{
"weight": 7,
"id": 28748
}
]}
I want to combine objects from both files if they have a common key 'id', but to extract only 'name' property from file1. How do I do that using jq?
This is what I want to get:
{
"Fruits": [
{
"name": "Apple",
"weight": 5,
"id": 652090
},
{
"name": "Orange",
"weight": 7,
"id": 28748
},
]}
Combine Fruits arrays, group it by id, select groups with 2 elements because we want fruits present in both files. For each selected group; add name field from first group element to second, and collect results in an array.
jq -n '[inputs.Fruits[]]
| reduce (group_by(.id)[] | select(length==2)) as $f
([]; . + [$f[1] + ($f[0] | {name})])' file1.json file2.json
Note that the order files are given on the command line is important, the file with names should be given before the other.
Combining objects with same id and extracting a subset of fields is way much easier though:
jq -n '[inputs.Fruits[]]
| group_by(.id)
| map(select(length==2) | add | {name, id, weight})
' file1.json file2.json
There's plenty of ways this could be constructed. Here's another way:
$ jq '.Fruits |= (. + input.Fruits | [group_by(.id)[] | add | {name,weight,id}])' \
file1.json file2.json
{
"Fruits": [
{
"name": "Orange",
"weight": 7,
"id": 28748
},
{
"name": "Apple",
"weight": 5,
"id": 652090
}
]
}