modify json with jq - json

I would like to use JQ to modify the following JSON input:
[
{
"description": "",
"created_on": "2021-12-27T11:32:03.171682Z",
"modified_on": "2021-12-27T11:32:03.171682Z",
"id": "test",
"enabled": true,
"minimum_origins": 1,
"monitor": "test",
"name": "test",
"notification_email": "",
"check_regions": null,
"latitude": 43.7417,
"longitude": -79.3733,
"notification_filter": {
"pool": {}
},
"healthy": true,
"origins": [
{
"name": "AAAA",
"address": "1.1.1.1",
"enabled": true,
"weight": 1,
"healthy": true,
"failure_reason": "No failures"
},
{
"name": "BBBB",
"address": "2.2.2.2",
"enabled": true,
"weight": 1,
"healthy": true,
"failure_reason": "No failures"
}
]
}
]
if address == 1.1.1.1 then weight 0
that's what i expect
[
{
"name": "test",
"origins": [
{
"name": "AAAA",
"address": "1.1.1.1",
"enabled": true,
"weight": 0,
"healthy": true,
"failure_reason": "No failures"
},
{
"name": "BBBB",
"address": "2.2.2.2",
"enabled": true,
"weight": 1,
"healthy": true,
"failure_reason": "No failures"
}
]
}
]
my best attempt but it doesn't fit my format. I tried many different options but could not find the correct code
(.[].origins[] | select(.address == "1.1.1.1") | .weight ) |= 0

Either perform setting the value first (where you could simply use = to set the new value as it does not depend on the old one), then use map({name, origins}) to get your expected output reduction:
(.[].origins[] | select(.address == "1.1.1.1")).weight = 0
| map({name, origins})
Demo
Or combine both within the map in one go:
map(
{name, origins}
| .origins[] |= (select(.address == "1.1.1.1").weight = 0)
)
Demo
Output:
[
{
"name": "test",
"origins": [
{
"name": "AAAA",
"address": "1.1.1.1",
"enabled": true,
"weight": 0,
"healthy": true,
"failure_reason": "No failures"
},
{
"name": "BBBB",
"address": "2.2.2.2",
"enabled": true,
"weight": 1,
"healthy": true,
"failure_reason": "No failures"
}
]
}
]

Here's a slightly different version, but pmf's answer is preferable.
map({
name,
origins: .origins | map(select(.address=="1.1.1.1").weight = 0)
})

If for whatever reason you want to avoid assignments with complex left-hand-sides (e.g., if you want portability across various versions of jq and jq look-alikes):
map( {name, origins}
| .origins |= map( if .address == "1.1.1.1" then .weight = 0 else . end) )

Related

Check if a key exists and return another key

I need help with jq syntax on how to return the Gitlab job ID if it contains an artifact. The JSON output looks like this (removed a lot of unrelated info from it and added [...]):
[{
"id": 3219589880,
"status": "success",
"stage": "test",
"name": "job_with_no_artifact",
"ref": "main",
"tag": false,
"coverage": null,
"allow_failure": false,
"created_at": "2022-10-24T18:21:25.119Z",
"started_at": "2022-10-24T18:21:25.986Z",
"finished_at": "2022-10-24T18:21:38.464Z",
"duration": 12.478682,
"queued_duration": 0.499786,
"user": {
"id": 123456789,
[...]
},
"commit": {
"id": "5e0e1f287d20daf2036a3ca71c656dce55999265",
[...]
"pipeline": {
"id": 123456789,
[...]
"project": {
"ci_job_token_scope_enabled": false
},
"artifacts": [],
"runner": {
"id": 12270859,
[...]
},
"artifacts_expire_at": null,
"tag_list": []
}, {
"id": 3219589878,
"status": "success",
"stage": "test",
"name": "create_artifact_job_2",
"ref": "main",
"tag": false,
"coverage": null,
"allow_failure": false,
"created_at": "2022-10-24T18:21:25.111Z",
"started_at": "2022-10-24T18:21:25.922Z",
"finished_at": "2022-10-24T18:21:39.090Z",
"duration": 13.168405,
"queued_duration": 0.464364,
"user": {
"id": 123456789,
[...]
},
"commit": {
"id": "5e0e1f287d20daf2036a3ca71c656dce55999265",
[...]
},
"pipeline": {
"id": 675641982,
[...],
"project": {
"ci_job_token_scope_enabled": false
},
"artifacts_file": {
"filename": "artifacts.zip",
"size": 223
},
"artifacts": [{
"file_type": "archive",
"size": 223,
"filename": "artifacts.zip",
"file_format": "zip"
}, {
"file_type": "metadata",
"size": 153,
"filename": "metadata.gz",
"file_format": "gzip"
}],
"runner": {
"id": 12270845,
[...]
},
"artifacts_expire_at": "2022-10-25T18:21:35.859Z",
"tag_list": []
}, {
"id": 3219589876,
"status": "success",
"stage": "test",
"name": "create_artifact_job_1",
"ref": "main",
"tag": false,
"coverage": null,
"allow_failure": false,
"created_at": "2022-10-24T18:21:25.103Z",
"started_at": "2022-10-24T18:21:25.503Z",
"finished_at": "2022-10-24T18:21:41.407Z",
"duration": 15.904028,
"queued_duration": 0.098837,
"user": {
"id": 123456789,
[...]
},
"commit": {
"id": "5e0e1f287d20daf2036a3ca71c656dce55999265",
[...]
},
"pipeline": {
"id": 123456789,
[...]
},
"web_url": "WEB_URL",
"project": {
"ci_job_token_scope_enabled": false
},
"artifacts_file": {
"filename": "artifacts.zip",
"size": 217
},
"artifacts": [{
"file_type": "archive",
"size": 217,
"filename": "artifacts.zip",
"file_format": "zip"
}, {
"file_type": "metadata",
"size": 152,
"filename": "metadata.gz",
"file_format": "gzip"
}],
"runner": {
"id": 12270857,
},
"artifacts_expire_at": "2022-10-25T18:21:37.808Z",
"tag_list": []
}]
I've been trying to do either of the following using jQ:
Either:
Check if artifacts_file key exists in each iteration and if it does return the (job) id (so .[].id)
Check if artifacts array is empty in each iteration and if it is empty return the (job) id.
In both cases I'm able to do the first part but I am not sure how to return the .id key.
Related stackoverflow questions that I've been trying to utilize and adapt to my case:
jq - return array value if its length is not null
How to check for presence of 'key' in jq before iterating over the values
What I have so far: jq '[.[].artifacts[]|select(length > 0)] | .[]' which returns all the artifacts found (but it doesn't contain the .id of the job).
Checking the existence of a field using has:
.[] | select(has("artifacts_file")).id
3219589878
3219589876
Demo
Checking if a field is an empty array by comparing it to []:
.[] | select(.artifacts == []).id
3219589880
Demo

Is there a way for omitting certain fields from matching array elements of JSON array of content of JSON response based on a criteria in spring boot

but it should retain the fields in other array elements which did not match the criteria.
I have json response like below
{
"content": [
{
"someId": 1498,
"someKey": {
"anotherKey": "Not Specified",
"keyOne": "",
"keyTwo": null,
"keyThree": null
},
"date": "2009-09-11",
"time": "17:14",
"location": "Not Specified"
},
{
"someId": 1498,
"someKey": {
"anotherKey": "criteria",
"keyOne": "some data",
"keyTwo": "some data for key two",
"keyThree": "some data for key three"
},
"date": "2009-09-12",
"time": "17:15",
"location": "Not Specified"
},...
],
"pageable": {
"sort": {
"sorted": false,
"unsorted": true,
"empty": true
},
"offset": 0,
"pageSize": 10,
"pageNumber": 0,
"unpaged": false,
"paged": true
},
"totalElements": 10630,
"totalPages": 1063,
"last": false,
"size": 10,
"number": 0,
"sort": {
"sorted": false,
"unsorted": true,
"empty": true
},
"first": true,
"numberOfElements": 10,
"empty": false
}
when "anotherKey" value is "criteria", the "keyOne","keyTwo","keyThree" should be omitted, else it should be retained. any solution to this? thanks in advance

select key names base on subject values

Hashicorp Vault provides me the following JSON:
{
"somekey1/": {
"accessor": "generic_123456,
"config": {
"default_lease_ttl": 0,
"force_no_cache": false,
"max_lease_ttl": 0
},
"description": "",
"external_entropy_access": false,
"local": false,
"options": {},
"seal_wrap": false,
"type": "generic",
"uuid": "1111111-2222-3333-4444-55555555555"
},
"somekey2/": {
"accessor": "aws_123456",
"config": {
"default_lease_ttl": 3600,
"force_no_cache": false,
"max_lease_ttl": 86400
},
"description": "",
"external_entropy_access": false,
"local": false,
"options": null,
"seal_wrap": false,
"type": "aws",
"uuid": "1111111-2222-3333-4444-55555555555"
},
"somekey3/": {
"accessor": "generic_1234567",
"config": {
"default_lease_ttl": 0,
"force_no_cache": false,
"max_lease_ttl": 0
},
"description": "",
"external_entropy_access": false,
"local": false,
"options": {},
"seal_wrap": false,
"type": "generic",
"uuid": "1111111-2222-3333-4444-55555555555"
}
}
I want to determine which top level keys have a sub-object K/V .type="generic".
In this case, "somekey1/" and "somekey3/"
Or "somekey2/" with a sub-object K/V .type="aws".
Can this be accomplished with JQ?
in this case, I use the following:
jq 'to_entries[] | select (.value.type == "generic") | .key' vault.json
which produces:
"somekey1/"
"somekey3/"
Which is exactly what I am looking for.

Modify value in the object containing a particular string using jq

I am trying to modify a large json file (a Grafana dashboard), replacing a single value, then output the whole file with the change. How can I do this?
You can see the value I want to edit here. The actual file is quite large, so there are many other top-level values, but I only need to edit a specific item under the "templating" block.
"templating": {
"list": [
{
"allValue": ".*",
"current": {},
"datasource": "$Source",
"hide": 0,
"includeAll": false,
"label": null,
"multi": true,
"name": "node",
"options": [],
"query": "label_values(node_boot_time{env=~\"$env\"}, instance)",
"refresh": 1,
"regex": "",
"sort": 0,
"tagValuesQuery": "",
"tags": [],
"tagsQuery": "",
"type": "query",
"useTags": false
},
{
"allValue": null,
"current": {
"tags": [],
"text": "",
"value": ""
},
"datasource": "$Source",
"definition": "label_values(env)",
"hide": 0,
"includeAll": true,
"label": "env",
"multi": false,
"name": "env",
"options": [],
"query": "label_values(env)",
"refresh": 1,
"regex": "",
"skipUrlSync": false,
"sort": 1,
"tagValuesQuery": "",
"tags": [],
"tagsQuery": "",
"type": "query",
"useTags": false
},
{
"current": {
"tags": [],
"text": "",
"value": ""
},
"hide": 0,
"includeAll": false,
"label": null,
"multi": false,
"name": "Source",
"options": [],
"query": "prometheus",
"refresh": 1,
"regex": "",
"skipUrlSync": false,
"type": "datasource"
}
]
},
The piece I need to change is the block containing "query": "label_values(env)", and I just need to change the value of "regex": "",
I have tried:
jq '.templating.list[] | select(.name == "env") |= . + {regex:"*"}' "dashboard.json" > test.json
The problem is then it only prints the ".list[]" elements instead of the whole file. I need to be able to make this change for multiple other files that will have the same block, but not necessarily in the same place so I can't just select by index number.
Output of above script:
{
"allValue": ".*",
"current": {},
"datasource": "$Source",
"hide": 0,
"includeAll": false,
"label": null,
"multi": true,
"name": "node",
"options": [],
"query": "label_values(node_boot_time{env=~\"$env\"}, instance)",
"refresh": 1,
"regex": "",
"sort": 0,
"tagValuesQuery": "",
"tags": [],
"tagsQuery": "",
"type": "query",
"useTags": false
}
{
"allValue": null,
"current": {
"tags": [],
"text": "",
"value": ""
},
"datasource": "$Source",
"definition": "label_values(env)",
"hide": 0,
"includeAll": true,
"label": "env",
"multi": false,
"name": "env",
"options": [],
"query": "label_values(env)",
"refresh": 1,
"regex": "*",
"skipUrlSync": false,
"sort": 1,
"tagValuesQuery": "",
"tags": [],
"tagsQuery": "",
"type": "query",
"useTags": false
}
{
"current": {
"tags": [],
"text": "",
"value": ""
},
"hide": 0,
"includeAll": false,
"label": null,
"multi": false,
"name": "Source",
"options": [],
"query": "prometheus",
"refresh": 1,
"regex": "",
"skipUrlSync": false,
"type": "datasource"
}
Position |= earlier to retain the original structure.
.templating.list[] |= (select(.name == "env") .regex = "*")
Online demo
Your expected output isn't quite matching with your description of your problem. If your requirement is to find inside templating list find the query containing "label_values(env)" and update the regex to "" you need below. To change it to *, use regex = "*"
.templating.list[] |= ( select(.query == "label_values(env)").regex = "")
The key is to use the right path and use the select operator to get the object to update using the |= operator
jq-play snippet

Rename JSON key field with value in object

Have the following json output:
[
{
"id": "47",
"canUpdate": true,
"canDelete": true,
"canArchive": true,
"info": [
{
"key": "problem_type",
"value": "PAN",
"valueCaption": "PAN",
"keyCaption": "Category"
},
{
"key": "status",
"value": 3,
"valueCaption": "Closed",
"keyCaption": "Status"
},
{
"key": "insert_time",
"value": 1466446314000,
"valueCaption": "2016-06-20 14:11:54.0",
"keyCaption": "Request time"
}
As you can see under "info" they actually label the key:value pair as "key": "problem_type" and "value": "PAN" and then "valueCaption": "PAN" "keyCaption": "Category". What I need to do is remap the file so that, in this example, it shows as "problem_type": "PAN" and "Category": "PAN". What would be the best method to iterate through the output to remap the key:value pairs in this manner?
How it needs to be:
[
{
"id": "47",
"canUpdate": true,
"canDelete": true,
"canArchive": true,
"info": [
{
"problem_type": "PAN",
"Category": "PAN"
},
{
"status": 3,
"Status": "Closed"
},
{
"insert_time": 1466446314000,
"Request time": "2016-06-20 14:11:54.0"
}
Here is a jq solution which uses Update assignment |=
.[].info[] |= {(.key):.value, (.keyCaption):.valueCaption}
Sample Run (assumes data in data.json)
$ jq -M '.[].info[] |= {(.key):.value, (.keyCaption):.valueCaption}' data.json
[
{
"id": "47",
"canUpdate": true,
"canDelete": true,
"canArchive": true,
"info": [
{
"problem_type": "PAN",
"Category": "PAN"
},
{
"status": 3,
"Status": "Closed"
},
{
"insert_time": 1466446314000,
"Request time": "2016-06-20 14:11:54.0"
}
]
}
]
Try it online at jqplay.org