merge two lists within a object conditionally - json

Hellow jq experts!
I'm a jq learner and have a json obect composed lists as follows:
{
"image_files": [
{
"id": "img_0001",
"width": 32,
"heigt": 32,
"file_name": "img_0001.png"
},
{
"id": "img_0002",
"width": 128,
"heigt": 32,
"file_name": "img_0002.png"
},
{
"id": "img_0003",
"width": 32,
"heigt": 32,
"file_name": "img_0003.png"
},
{
"id": "img_0004",
"width": 160,
"heigt": 32,
"file_name": "img_0004.png"
}
],
"annotations": [
{
"id": "ann_0001",
"image_id": "img_0001",
"label": "A",
"attributes": {
"type": "letter",
"augmented": false
}
},
{
"id": "ann_0002",
"image_id": "img_0002",
"label": "Good",
"attributes": {
"type": "word",
"augmented": false
}
},
{
"id": "ann_0003",
"image_id": "img_0003",
"label": "C",
"attributes": {
"type": "letter",
"augmented": false
}
},
{
"id": "ann_0004",
"image_id": "img_0004",
"label": "Hello",
"attributes": {
"type": "word",
"augmented": false
}
}
]
}
image_id in the annotations list are foreign key referencing the id in the image_files list.
I want to join image_files and annotations with condition of annotations.attribute.type == "letter".
Expecting following ouptut:
{
"letter_image_files_with_label": [
{
"id": "img_0001",
"width": 32,
"heigt": 32,
"file_name": "img_0001.png",
"label": "A"
},
{
"id": "img_0003",
"width": 32,
"heigt": 32,
"file_name": "img_0003.png",
"label": "C"
}
]
}
How can I produce above result from the json data input?
join explained in jq manual does not seem to use this kind task.
Is there a way for this? Please show me the rope.
Thanks for your generous reading.

Indexing image_files with ids makes this pretty trivial.
INDEX(.image_files[]; .id) as $imgs | [
.annotations[]
| select(.attributes.type == "letter")
| $imgs[.image_id] + {label: .label}
]
Online demo

Related

How to allocate dynamic Asset Id in Aws Sitewise monitoring dashboard definition Cloudformation?

We created a Cloudformation template for auto implementation of the AWS Sitewise monitoring dashboard. We would like to dynamically refer and assign the Asset logical id inside the below dashboard definition.
{\"widgets\":[{\"type\":\"sc-line-chart\",\"title\":\"power_all_plants_5m\",\"x\":0,\"y\":0,\"height\":3,\"width\":3,\"metrics\":[{\"type\":\"iotsitewise\",\"label\":\"power_all_plants_5m (All Power Plants)\",\"assetId\":\"0cd25cb9-89f9-4a93-b2bf-88050436f700\",\"propertyId\":\"fd34bba7-4ea2-4d62-9058-ab78b726b61a\",\"dataType\":\"DOUBLE\"}],\"alarms\":[],\"properties\":{\"colorDataAcrossThresholds\":true},\"annotations\":{\"y\":[]}},{\"type\":\"sc-line-chart\",\"title\":\"Generator-1\",\"x\":3,\"y\":0,\"height\":3,\"width\":3,\"metrics\":[{\"type\":\"iotsitewise\",\"label\":\"sum_watts_5m (Generator-1)\",\"assetId\":\"45b97aaa-3f0c-4312-a8a5-a00e4da8ec37\",\"propertyId\":\"e22d9a23-4ac8-432a-816b-cc4a2138b287\",\"dataType\":\"DOUBLE\"},{\"type\":\"iotsitewise\",\"label\":\"rpm (Generator-1)\",\"assetId\":\"45b97aaa-3f0c-4312-a8a5-a00e4da8ec37\",\"propertyId\":\"c6a40902-f07b-40ba-b6c5-3509b069dd4c\",\"dataType\":\"DOUBLE\"}],\"alarms\":[],\"properties\":{\"colorDataAcrossThresholds\":true},\"annotations\":{\"y\":[]}},{\"type\":\"sc-line-chart\",\"title\":\"Generator-2\",\"x\":0,\"y\":3,\"height\":3,\"width\":3,\"metrics\":[{\"type\":\"iotsitewise\",\"label\":\"sum_watts_5m (Generator-2)\",\"assetId\":\"b999319c-20ec-4060-b3b7-bc5ce7ef189c\",\"propertyId\":\"e22d9a23-4ac8-432a-816b-cc4a2138b287\",\"dataType\":\"DOUBLE\"},{\"type\":\"iotsitewise\",\"label\":\"rpm (Generator-2)\",\"assetId\":\"b999319c-20ec-4060-b3b7-bc5ce7ef189c\",\"propertyId\":\"c6a40902-f07b-40ba-b6c5-3509b069dd4c\",\"dataType\":\"DOUBLE\"}],\"alarms\":[],\"properties\":{\"colorDataAcrossThresholds\":true},\"annotations\":{\"y\":[]}}]}
This JSON literal is converted to YAML by adding backward slashes () along with the double quotes because we are using YAML as the default language of the Cloudformation template otherwise it looks like the below.
{
"widgets": [
{
"type": "sc-line-chart",
"title": "power_all_plants_5m",
"x": 0,
"y": 0,
"height": 3,
"width": 3,
"metrics": [
{
"type": "iotsitewise",
"label": "power_all_plants_5m (All Power Plants)",
"assetId": "0cd25cb9-89f9-4a93-b2bf-88050436f700",
"propertyId": "fd34bba7-4ea2-4d62-9058-ab78b726b61a",
"dataType": "DOUBLE"
}
],
"alarms": [],
"properties": {
"colorDataAcrossThresholds": true
},
"annotations": {
"y": []
}
},
{
"type": "sc-line-chart",
"title": "Generator-1",
"x": 3,
"y": 0,
"height": 3,
"width": 3,
"metrics": [
{
"type": "iotsitewise",
"label": "sum_watts_5m (Generator-1)",
"assetId": "45b97aaa-3f0c-4312-a8a5-a00e4da8ec37",
"propertyId": "e22d9a23-4ac8-432a-816b-cc4a2138b287",
"dataType": "DOUBLE"
},
{
"type": "iotsitewise",
"label": "rpm (Generator-1)",
"assetId": "45b97aaa-3f0c-4312-a8a5-a00e4da8ec37",
"propertyId": "c6a40902-f07b-40ba-b6c5-3509b069dd4c",
"dataType": "DOUBLE"
}
],
"alarms": [],
"properties": {
"colorDataAcrossThresholds": true
},
"annotations": {
"y": []
}
},
{
"type": "sc-line-chart",
"title": "Generator-2",
"x": 0,
"y": 3,
"height": 3,
"width": 3,
"metrics": [
{
"type": "iotsitewise",
"label": "sum_watts_5m (Generator-2)",
"assetId": "b999319c-20ec-4060-b3b7-bc5ce7ef189c",
"propertyId": "e22d9a23-4ac8-432a-816b-cc4a2138b287",
"dataType": "DOUBLE"
},
{
"type": "iotsitewise",
"label": "rpm (Generator-2)",
"assetId": "b999319c-20ec-4060-b3b7-bc5ce7ef189c",
"propertyId": "c6a40902-f07b-40ba-b6c5-3509b069dd4c",
"dataType": "DOUBLE"
}
],
"alarms": [],
"properties": {
"colorDataAcrossThresholds": true
},
"annotations": {
"y": []
}
}
]
}
We would like to assign Asset ID dynamically using "!Ref" with pre-created Asset. We have tried the below variations but no fate.
Existing value -> 0cd25cb9-89f9-4a93-b2bf-88050436f700
Tried below changes:-
[{\"Ref\":\"GeneratorAsset\"}]
!Ref GeneratorAsset
\"!Ref GeneratorAsset\"
{\"Ref\":GeneratorAsset}
{\"Ref\":\"GeneratorAsset\"}
\"{\"Ref\":\"GeneratorAsset\"}\"
\"{\"Fn::Sub\":${GeneratorAsset}}\"
\"{\"Fn::Sub\":${!GeneratorAsset}}\"
\"{\"Fn::Sub\":${!GeneratorAsset} }\"
Here GeneratorAsset is a resource that is already created before Dashboard. Request any Cloudformation expert to help us replace the id value with the correct dynamic string.
Ref link:- https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-iotsitewise-dashboard.html

jq return a json array in a very specifique way

I have this Json ( is a test database, no data is true here )
{
"pguid": "4EA979A2-E578-4DA3-89DB-24082F3092AA",
"lastEnrollTguid": "EA98B161-04D3-4F0A-920A-58DBFF3C2274",
"timestamp": 1016086888000,
"keys": [
{
"id": "gr",
"value": "1907971"
}
],
"biographics": [
{
"id": "localNascimento",
"value": "JOINVILLE SC"
},
{
"id": "dataNascimento",
"value": "1859-03-08"
},
{
"id": "mae",
"value": "ANTA MARCIA PINHEAD"
},
{
"id": "nome",
"value": "MIR PINHEAD"
}
],
"biometric": [
{
"source": "ORIGINAL",
"type": "FACE",
"format": "JPEG",
"properties": {
"width": 0,
"height": 0,
"resolution": 500,
"ratio": 0,
"matcherId": 0,
"extractorId": 0
},
"index": 10,
"content": "5215421547"
}
],
"labels": [
"SC",
"CIVIL",
"MASCULINO",
"JOINVILLE"
],
"history": {
"events": [
{
"type": "ENROLL",
"tguid": "3C1B0D1F-9143-4C24-A351-E88A19317AC9",
"timestamp": 1014086658288
},
{
"type": "UPDATE",
"tguid": "EA98B161-04D3-4F0A-920A-58DBFF3C2274",
"timestamp": 1016786888028
}
]
}
}
I want to retrive only de tguid in history array, and if exist a way to do this, use de index of the array to acomplish that.
Here I tryed to acomplish that ( and miserable failed in that )
example ( and it do not work ):
jq '.[].history.events.tguid[1]' /tmp/teste.json
I want to retrieve the pguid in a index to work with that.
Someone have any ideas?
try this
jq '.history.events | .[1].tguid' /tmp/teste.json
tnks everyone
jq '.[].history.events | .[0].tguid' /tmp/teste1.json

JSON-Path for objects that contain JSON array with specific entry

I have a JSON-Object containing a list of events related to soccer matches, as posted below.
I am using Newtonsoft's Json.NET framework. In said JSON-Object, what would be the JSON-Path (that works in Json.NET's JSONPath implementation) to find only events where there is a qualifier that has type.displayName 'Yellow'` (i.e., a JSON Path that returns only the second event, since this one has the desired qualifier)?
This is in .NET, so if there is a way to use LINQ for this, I'm open to that.
{
"events": [
{
"eventId": 490,
"minute": 54,
"second": 23,
"period": {
"value": 2,
"displayName": "SecondHalf"
},
"type": {
"value": 19,
"displayName": "SubstitutionOn"
},
"outcomeType": {
"value": 1,
"displayName": "Successful"
},
"qualifiers": [
{
"type": {
"value": 55,
"displayName": "RelatedEventId"
},
"value": "489"
},
{
"type": {
"value": 59,
"displayName": "JerseyNumber"
},
"value": "11"
}
],
"satisfiedEventsTypes": [
212
],
"isTouch": false
},
{
"eventId": 669,
"minute": 75,
"second": 5,
"period": {
"value": 2,
"displayName": "SecondHalf"
},
"type": {
"value": 17,
"displayName": "Card"
},
"outcomeType": {
"value": 1,
"displayName": "Successful"
},
"qualifiers": [
{
"type": {
"value": 13,
"displayName": "Foul"
},
"value": "243"
},
{
"type": {
"value": 31,
"displayName": "Yellow"
}
}
],
"isTouch": false
}
]
}
It turns out the correct JSON-Path in Json.Net for this would be
$.events[?(#.qualifiers..type.displayName == 'Yellow')]
Specifically, the deepscan operator .. is necessary, since the JSON in question may contain more than one qualifier per event.

Merge objects in same array on single key

I have an array of JSON objects formatted as follows:
[
{
"id": 1,
"names": [
{
"name": "Bulbasaur",
"language": {
"name": "en",
"url": "http://myserver.com:8000/api/v2/language/9/"
}
},
],
},
{
"id": 1,
"types": [
{
"slot": 1,
"type": {
"name": "grass",
"url": "http://myserver.com:8000/api/v2/type/12/"
}
},
{
"slot": 2,
"type": {
"name": "poison",
"url": "http://myserver.com:8000/api/v2/type/4/"
}
}
]
},
{
"id": 2,
"names": [
{
"name": "Ivysaur",
"language": {
"name": "en",
"url": "http://myserver.com:8000/api/v2/language/9/"
}
},
],
},
{
"id": 2,
"types": [
{
"slot": 1,
"type": {
"name": "ice",
"url": "http://myserver.com:8000/api/v2/type/10/"
}
},
{
"slot": 2,
"type": {
"name": "electric",
"url": "http://myserver.com:8000/api/v2/type/8/"
}
}
]
},
{
"id": 3,
"names": [
{
"name": "Venusaur",
"language": {
"name": "en",
"url": "http://myserver.com:8000/api/v2/language/9/"
}
},
],
},
{
"id": 3,
"types": [
{
"slot": 1,
"type": {
"name": "ground",
"url": "http://myserver.com:8000/api/v2/type/2/"
}
},
{
"slot": 2,
"type": {
"name": "rock",
"url": "http://myserver.com:8000/api/v2/type/3/"
}
}
]
}
]
Note that these are pairs of separate objects that appear sequentially in a JSON array, with each pair sharing an id field. This pattern repeats several hundred times in the array. What I need to accomplish is to "merge" each id-sharing pair into one object. So, the resultant output would be
[
{
"id": 1,
"names": [
{
"name": "Bulbasaur",
"language": {
"name": "en",
"url": "http://myserver.com:8000/api/v2/language/9/"
}
},
],
"types": [
{
"slot": 1,
"type": {
"name": "grass",
"url": "http://myserver.com:8000/api/v2/type/12/"
}
},
{
"slot": 2,
"type": {
"name": "poison",
"url": "http://myserver.com:8000/api/v2/type/4/"
}
}
]
},
{
"id": 2,
"names": [
{
"name": "Ivysaur",
"language": {
"name": "en",
"url": "http://myserver.com:8000/api/v2/language/9/"
}
},
],
"types": [
{
"slot": 1,
"type": {
"name": "ice",
"url": "http://myserver.com:8000/api/v2/type/10/"
}
},
{
"slot": 2,
"type": {
"name": "electric",
"url": "http://myserver.com:8000/api/v2/type/8/"
}
}
]
},
{
"id": 3,
"names": [
{
"name": "Venusaur",
"language": {
"name": "en",
"url": "http://myserver.com:8000/api/v2/language/9/"
}
},
],
"types": [
{
"slot": 1,
"type": {
"name": "ground",
"url": "http://myserver.com:8000/api/v2/type/2/"
}
},
{
"slot": 2,
"type": {
"name": "rock",
"url": "http://myserver.com:8000/api/v2/type/3/"
}
}
]
}
]
I've gotten these objects to appear next to each other via the group_by(.id) command, but I'm at a loss as to how I should actually combine them. I'm very much still a novice with jq so I'm a bit overwhelmed with the amount of possible solutions.
[Note: The following assumes that the data shown in the Q have been corrected so that they are valid JSON.]
The merging you want can be achieved by object addition (x + y). For example, given the two JSON objects as shown in the question (i.e., as a stream), you could write:
jq -s '.[0] + .[1]'
However, since the question also indicates these objects are actually in an array, let's next consider the case of an array with two objects. In that case, you could simply write:
jq add
Finally, if you have an array of arrays each of which is an array of objects, you could use map(add). Since you don't have a very large array, you could simply write:
group_by(.id) | map(add)
Please note that jq defines object addition in a non-commutative way. Specifically, there is a bias towards the right-most key.

Traversing json using with square brackets with jsx

I need some help with my syntax please in trying to get thevalue from a key. Given the following json object:
{
"records": [
{
"id": "recD17v5uIGpA7jYU",
"fields": {
"header h1": "Clouds",
"Email": 1,
"header figure": [
{
"id": "attmeLF6GWqLF3iOS",
"url": "https://dl.airtable.com/.attachments/d6c5f674aeb427dd533cd2c60ab2b29f/f2f99dc0/80__clouds.jpg",
"filename": "80__clouds.jpg",
"size": 2334688,
"type": "image/jpeg",
"thumbnails": {
"small": {
"url": "https://dl.airtable.com/.attachmentThumbnails/7c3dc033b1a7f77fcdeddb2f594a1feb/9a93ab8d",
"width": 36,
"height": 36
},
"large": {
"url": "https://dl.airtable.com/.attachmentThumbnails/6145b560b5c807787814fcf9b39e6130/0f7d0049",
"width": 519,
"height": 512
},
"full": {
"url": "https://dl.airtable.com/.attachmentThumbnails/ecebeb4705ba5531744d556242ddc081/20e10b2a",
"width": 2121,
"height": 2094
}
}
}
]
},
"createdTime": "2019-02-12T09:53:52.000Z"
},
{
"id": "recXiwDKIMSiGd2fn",
"fields": {
"header h1": "People",
"Email": 2,
"header figure": [
{
"id": "attr1iy10UnygtRx0",
"url": "https://dl.airtable.com/.attachments/67484295777d866d6610ef9f84f8bc53/81ae8e6b/A07.jpg",
"filename": "A07.jpg",
"size": 1014718,
"type": "image/jpeg",
"thumbnails": {
"small": {
"url": "https://dl.airtable.com/.attachmentThumbnails/786cc58d3f5b82f4a8414d89b5ea48fb/4a214afc",
"width": 14,
"height": 36
},
"large": {
"url": "https://dl.airtable.com/.attachmentThumbnails/3cb6be2be2b42be2c240f4f2bb9b7901/3bf2b62b",
"width": 512,
"height": 1288
},
"full": {
"url": "https://dl.airtable.com/.attachmentThumbnails/4e56dafdcf39da1f35c1e2f9e96fda6c/07d91945",
"width": 800,
"height": 2013
}
}
}
]
},
"createdTime": "2019-02-12T09:53:52.000Z"
},
{
"id": "recvSJHL3EkSBprs9",
"fields": {
"Email": 3
},
"createdTime": "2019-02-12T09:53:52.000Z"
}
],
"offset": "recvSJHL3EkSBprs9"
}
I'm able to extract the value for "header h1" key with the following jsx
<h2>{record.fields["header h1"]}</h2>
I'm now trying to extract the value from the "url" key. I've tried the following.
<img src={records.fields["header figure"]["url"]} />
and
<img src={records.fields["header figure"][0]["url"]} />
I'm getting TypeError: Cannot read property "url" of undefined.Need some help with my syntax please
List item
your records is an array, also you must specify wich position you want to read:
I tested your json with jq:
jq '.records[0].fields["header figure"][0]["url"]'
delivers
"https://dl.airtable.com/.attachments/d6c5f674aeb427dd533cd2c60ab2b29f/f2f99dc0/80__clouds.jpg"
and
jq '.records[1].fields["header figure"][0]["url"]'
delivers
"https://dl.airtable.com/.attachments/67484295777d866d6610ef9f84f8bc53/81ae8e6b/A07.jpg"