I am trying to modify a large json file (a Grafana dashboard), replacing a single value, then output the whole file with the change. How can I do this?
You can see the value I want to edit here. The actual file is quite large, so there are many other top-level values, but I only need to edit a specific item under the "templating" block.
"templating": {
"list": [
{
"allValue": ".*",
"current": {},
"datasource": "$Source",
"hide": 0,
"includeAll": false,
"label": null,
"multi": true,
"name": "node",
"options": [],
"query": "label_values(node_boot_time{env=~\"$env\"}, instance)",
"refresh": 1,
"regex": "",
"sort": 0,
"tagValuesQuery": "",
"tags": [],
"tagsQuery": "",
"type": "query",
"useTags": false
},
{
"allValue": null,
"current": {
"tags": [],
"text": "",
"value": ""
},
"datasource": "$Source",
"definition": "label_values(env)",
"hide": 0,
"includeAll": true,
"label": "env",
"multi": false,
"name": "env",
"options": [],
"query": "label_values(env)",
"refresh": 1,
"regex": "",
"skipUrlSync": false,
"sort": 1,
"tagValuesQuery": "",
"tags": [],
"tagsQuery": "",
"type": "query",
"useTags": false
},
{
"current": {
"tags": [],
"text": "",
"value": ""
},
"hide": 0,
"includeAll": false,
"label": null,
"multi": false,
"name": "Source",
"options": [],
"query": "prometheus",
"refresh": 1,
"regex": "",
"skipUrlSync": false,
"type": "datasource"
}
]
},
The piece I need to change is the block containing "query": "label_values(env)", and I just need to change the value of "regex": "",
I have tried:
jq '.templating.list[] | select(.name == "env") |= . + {regex:"*"}' "dashboard.json" > test.json
The problem is then it only prints the ".list[]" elements instead of the whole file. I need to be able to make this change for multiple other files that will have the same block, but not necessarily in the same place so I can't just select by index number.
Output of above script:
{
"allValue": ".*",
"current": {},
"datasource": "$Source",
"hide": 0,
"includeAll": false,
"label": null,
"multi": true,
"name": "node",
"options": [],
"query": "label_values(node_boot_time{env=~\"$env\"}, instance)",
"refresh": 1,
"regex": "",
"sort": 0,
"tagValuesQuery": "",
"tags": [],
"tagsQuery": "",
"type": "query",
"useTags": false
}
{
"allValue": null,
"current": {
"tags": [],
"text": "",
"value": ""
},
"datasource": "$Source",
"definition": "label_values(env)",
"hide": 0,
"includeAll": true,
"label": "env",
"multi": false,
"name": "env",
"options": [],
"query": "label_values(env)",
"refresh": 1,
"regex": "*",
"skipUrlSync": false,
"sort": 1,
"tagValuesQuery": "",
"tags": [],
"tagsQuery": "",
"type": "query",
"useTags": false
}
{
"current": {
"tags": [],
"text": "",
"value": ""
},
"hide": 0,
"includeAll": false,
"label": null,
"multi": false,
"name": "Source",
"options": [],
"query": "prometheus",
"refresh": 1,
"regex": "",
"skipUrlSync": false,
"type": "datasource"
}
Position |= earlier to retain the original structure.
.templating.list[] |= (select(.name == "env") .regex = "*")
Online demo
Your expected output isn't quite matching with your description of your problem. If your requirement is to find inside templating list find the query containing "label_values(env)" and update the regex to "" you need below. To change it to *, use regex = "*"
.templating.list[] |= ( select(.query == "label_values(env)").regex = "")
The key is to use the right path and use the select operator to get the object to update using the |= operator
jq-play snippet
Related
I need help with jq syntax on how to return the Gitlab job ID if it contains an artifact. The JSON output looks like this (removed a lot of unrelated info from it and added [...]):
[{
"id": 3219589880,
"status": "success",
"stage": "test",
"name": "job_with_no_artifact",
"ref": "main",
"tag": false,
"coverage": null,
"allow_failure": false,
"created_at": "2022-10-24T18:21:25.119Z",
"started_at": "2022-10-24T18:21:25.986Z",
"finished_at": "2022-10-24T18:21:38.464Z",
"duration": 12.478682,
"queued_duration": 0.499786,
"user": {
"id": 123456789,
[...]
},
"commit": {
"id": "5e0e1f287d20daf2036a3ca71c656dce55999265",
[...]
"pipeline": {
"id": 123456789,
[...]
"project": {
"ci_job_token_scope_enabled": false
},
"artifacts": [],
"runner": {
"id": 12270859,
[...]
},
"artifacts_expire_at": null,
"tag_list": []
}, {
"id": 3219589878,
"status": "success",
"stage": "test",
"name": "create_artifact_job_2",
"ref": "main",
"tag": false,
"coverage": null,
"allow_failure": false,
"created_at": "2022-10-24T18:21:25.111Z",
"started_at": "2022-10-24T18:21:25.922Z",
"finished_at": "2022-10-24T18:21:39.090Z",
"duration": 13.168405,
"queued_duration": 0.464364,
"user": {
"id": 123456789,
[...]
},
"commit": {
"id": "5e0e1f287d20daf2036a3ca71c656dce55999265",
[...]
},
"pipeline": {
"id": 675641982,
[...],
"project": {
"ci_job_token_scope_enabled": false
},
"artifacts_file": {
"filename": "artifacts.zip",
"size": 223
},
"artifacts": [{
"file_type": "archive",
"size": 223,
"filename": "artifacts.zip",
"file_format": "zip"
}, {
"file_type": "metadata",
"size": 153,
"filename": "metadata.gz",
"file_format": "gzip"
}],
"runner": {
"id": 12270845,
[...]
},
"artifacts_expire_at": "2022-10-25T18:21:35.859Z",
"tag_list": []
}, {
"id": 3219589876,
"status": "success",
"stage": "test",
"name": "create_artifact_job_1",
"ref": "main",
"tag": false,
"coverage": null,
"allow_failure": false,
"created_at": "2022-10-24T18:21:25.103Z",
"started_at": "2022-10-24T18:21:25.503Z",
"finished_at": "2022-10-24T18:21:41.407Z",
"duration": 15.904028,
"queued_duration": 0.098837,
"user": {
"id": 123456789,
[...]
},
"commit": {
"id": "5e0e1f287d20daf2036a3ca71c656dce55999265",
[...]
},
"pipeline": {
"id": 123456789,
[...]
},
"web_url": "WEB_URL",
"project": {
"ci_job_token_scope_enabled": false
},
"artifacts_file": {
"filename": "artifacts.zip",
"size": 217
},
"artifacts": [{
"file_type": "archive",
"size": 217,
"filename": "artifacts.zip",
"file_format": "zip"
}, {
"file_type": "metadata",
"size": 152,
"filename": "metadata.gz",
"file_format": "gzip"
}],
"runner": {
"id": 12270857,
},
"artifacts_expire_at": "2022-10-25T18:21:37.808Z",
"tag_list": []
}]
I've been trying to do either of the following using jQ:
Either:
Check if artifacts_file key exists in each iteration and if it does return the (job) id (so .[].id)
Check if artifacts array is empty in each iteration and if it is empty return the (job) id.
In both cases I'm able to do the first part but I am not sure how to return the .id key.
Related stackoverflow questions that I've been trying to utilize and adapt to my case:
jq - return array value if its length is not null
How to check for presence of 'key' in jq before iterating over the values
What I have so far: jq '[.[].artifacts[]|select(length > 0)] | .[]' which returns all the artifacts found (but it doesn't contain the .id of the job).
Checking the existence of a field using has:
.[] | select(has("artifacts_file")).id
3219589878
3219589876
Demo
Checking if a field is an empty array by comparing it to []:
.[] | select(.artifacts == []).id
3219589880
Demo
but it should retain the fields in other array elements which did not match the criteria.
I have json response like below
{
"content": [
{
"someId": 1498,
"someKey": {
"anotherKey": "Not Specified",
"keyOne": "",
"keyTwo": null,
"keyThree": null
},
"date": "2009-09-11",
"time": "17:14",
"location": "Not Specified"
},
{
"someId": 1498,
"someKey": {
"anotherKey": "criteria",
"keyOne": "some data",
"keyTwo": "some data for key two",
"keyThree": "some data for key three"
},
"date": "2009-09-12",
"time": "17:15",
"location": "Not Specified"
},...
],
"pageable": {
"sort": {
"sorted": false,
"unsorted": true,
"empty": true
},
"offset": 0,
"pageSize": 10,
"pageNumber": 0,
"unpaged": false,
"paged": true
},
"totalElements": 10630,
"totalPages": 1063,
"last": false,
"size": 10,
"number": 0,
"sort": {
"sorted": false,
"unsorted": true,
"empty": true
},
"first": true,
"numberOfElements": 10,
"empty": false
}
when "anotherKey" value is "criteria", the "keyOne","keyTwo","keyThree" should be omitted, else it should be retained. any solution to this? thanks in advance
Hashicorp Vault provides me the following JSON:
{
"somekey1/": {
"accessor": "generic_123456,
"config": {
"default_lease_ttl": 0,
"force_no_cache": false,
"max_lease_ttl": 0
},
"description": "",
"external_entropy_access": false,
"local": false,
"options": {},
"seal_wrap": false,
"type": "generic",
"uuid": "1111111-2222-3333-4444-55555555555"
},
"somekey2/": {
"accessor": "aws_123456",
"config": {
"default_lease_ttl": 3600,
"force_no_cache": false,
"max_lease_ttl": 86400
},
"description": "",
"external_entropy_access": false,
"local": false,
"options": null,
"seal_wrap": false,
"type": "aws",
"uuid": "1111111-2222-3333-4444-55555555555"
},
"somekey3/": {
"accessor": "generic_1234567",
"config": {
"default_lease_ttl": 0,
"force_no_cache": false,
"max_lease_ttl": 0
},
"description": "",
"external_entropy_access": false,
"local": false,
"options": {},
"seal_wrap": false,
"type": "generic",
"uuid": "1111111-2222-3333-4444-55555555555"
}
}
I want to determine which top level keys have a sub-object K/V .type="generic".
In this case, "somekey1/" and "somekey3/"
Or "somekey2/" with a sub-object K/V .type="aws".
Can this be accomplished with JQ?
in this case, I use the following:
jq 'to_entries[] | select (.value.type == "generic") | .key' vault.json
which produces:
"somekey1/"
"somekey3/"
Which is exactly what I am looking for.
I have a JSON of the below format
{
"board_title": "test",
"read_only": false,
"isIntegration": false,
"board_bgtype": "board_graph",
"created": "2017-08-16T06:40:47.158868+00:00",
"original_title": "Revised_CID_Templating-test(cloned)",
"modified": "2017-08-31T11:52:22.115661+00:00",
"disableEditing": false,
"height": 111,
"width": "100%",
"template_variables": [
{
"default": "identity",
"prefix": "v1",
"name": "env"
}
],
"created_by": {
"disabled": false,
"handle": "xx.com",
"name": null,
"is_admin": false,
"role": null,
"access_role": "st",
"verified": true,
"email": "xx.com",
"icon": "https://secure.gravatar.com/avatar/86fd6c17deba27cfc4081134a5bc0c6a?s=48&d=retro"
},
...
}
I need to load this JSON using Python and edit the value of
"template_variables": [
{
"default": "identity"
To some other value, say:
default : "com"
I load it using Python script and traverse and print the key value pair but not able to understand how to modify it.
How can I assign a value to the first child of template variables and save the JSON in the same file and post it using request?
You can think of json object as a dictionary.
try this.
import json
jsonData = '''{
"board_title": "test",
"read_only": false,
"isIntegration": false,
"board_bgtype": "board_graph",
"created": "2017-08-16T06:40:47.158868+00:00",
"original_title": "Revised_CID_Templating-test(cloned)",
"modified": "2017-08-31T11:52:22.115661+00:00",
"disableEditing": false,
"height": 111,
"width": "100%",
"template_variables": [
{
"default": "identity",
"prefix": "v1",
"name": "env"
}
],
"created_by": {
"disabled": false,
"handle": "xx.com",
"name": null,
"is_admin": false,
"role": null,
"access_role": "st",
"verified": true,
"email": "xx.com",
"icon": "https://secure.gravatar.com/avatar/86fd6c17deba27cfc4081134a5bc0c6a?s=48&d=retro"
}
}'''
jsonToPython = json.loads(jsonData)
print (jsonToPython['template_variables'][0]['default'])
jsonToPython['template_variables'][0]['default'] = 'test'
print (jsonToPython['template_variables'][0]['default'])
as you can see jsonToPython is being modified.
i want to get access of "uuid" everytime,which is two times present in this code and also access to "cloud_uuid". the body is something like below:
{
"computes": [{
"uuid": "110c607a-231c-4724-be7f-db5ed388158",
"name": "9.4.98.33",
"description": null,
"version": "1.0",
"type": "compute",
"number_of_vms": 0,
"status": "ACTIVE",
"provisioning_status": {
"status": "COMPLETED",
"started_at": "",
"updated_at": "",
"status_data": null
},
"health_status": {
"status": "OK",
"alerts": [],
"updated_at": "2014-07-11T17:09:12.194000"
},
"compliance_status": {
"compliance_reasons": null,
"is_compliant": true,
"updated_at": null
},
"run_priority_order": null,
"created": "2014-07-11T16:01:32.837821",
"updated": "2014-07-11T17:08:16.031838",
"capability_categories": {
"v": [{
"name_key": "",
"description_key": "",
"version": "0",
"hidden": t,
"priority": 1,
"name_nls": "",
"description_nls": ""
}],
"monitoring": [{
"name_key": "m",
"description_key": null,
"version": "1.0",
"hidden": true,
"priority": 100,
"name_nls": "monitoring",
"description_nls": null
}],
"scheduler": [{
"name_key": "",
"description_key": null,
"version": "1.0",
"hidden": false,
"priority": 20,
"name_nls": "",
"description_nls": null
}],
"network": [{
"name_key": "",
"description_key": "",
"version": "1.0",
"hidden": false,
"priority": 10,
"name_nls": "",
"description_nls": ""
}]
},
"links": [{
"href": "",
"rel": "self"
}, {
"href": "",
"rel": "bookmark"
}],
"cloud_uuid": "b603e16e-38a6-435e-9359-79c27fee93a",
"operating_system_uuid": "70f605e7-6512-49b4-833c-b25d47823a4"
}, {
"uuid": "7383f4a5-dc0a-420b-806c-abbd49c1655a",
"name": "9.4.193.20",
"description": null,
"version": "1.0",
"type": "compute"
could you help with below,i tried the code as answered in comment:
suppose body is "clouds" instead of "computes"
i tried for getting cloud_uuid with something like:cloud_uuid = ((e['cloud_uuid'] for e in dict['clouds'] if e['name'] == name_to_find), None)
it throws error->
cloud_uuid = ((e['cloud_uuid'] for e in dict['clouds'] if e['name'] ==
name_to_find), None) TypeError: 'type' object is unsubscriptable
In python it is very simple:
[(e['uuid'], e['cloud_uuid']) for e in dict['computes']]
Edit:
It looks like I overlooked a pair square braces. Try:
[(_['computes'][0]['uuid'], _['cloud_uuid']) for _ in data]