I am trying to update "image_id" value in a json structure. Using the below command, how do I change ami-d8cf5cab to ami-a4df7gah So far, I have tried this
cat cog.test.tfstate | jq -r '.modules[].resources[] | select(.type == "aws_launch_configuration") | select(.primary.attributes.name_prefix == "pmsadmin-lc-")'
The JSON data is
{
"type": "aws_launch_configuration",
"primary": {
"id": "pmsadmin-lc-v47thk6rcrdgza6dujfzjatmju",
"attributes": {
"associate_public_ip_address": "false",
"ebs_block_device.#": "0",
"ebs_optimized": "false",
"enable_monitoring": "true",
"ephemeral_block_device.#": "0",
"iam_instance_profile": "cog-test-pmsadmin",
"id": "pmsadmin-lc-v47thk6rcrdgza6dujfzjatmju",
"image_id": "ami-d8cf5cab",
"instance_type": "t2.small",
"key_name": "cog-test-internal",
"name": "pmsadmin-lc-v47thk6rcrdgza6dujfzjatmju",
"name_prefix": "pmsadmin-lc-",
"root_block_device.#": "0",
"security_groups.#": "4",
"security_groups.1893851868": "sg-7ee7bf1a",
"security_groups.2774384192": "sg-e2e7bf86",
"security_groups.2825850029": "sg-86e6bee2",
"security_groups.3095009517": "sg-f4e7bf90",
"spot_price": "",
"user_data": "ed03ac6642af8c97562b065c0b37f211b58ad0a2"
}
}
}
Use the |= operator to assign to a property:
jq -r '.modules[].resources[] | select(.type == "aws_launch_configuration") | select(.primary.attributes.name_prefix == "pmsadmin-lc-")| .primary.attributes.image_id |= "ami-a4df7gah"
Related
I have a json output
{
"7": [
{
"devices": [
"/dev/sde"
],
"name": "osd-block-dcc9b386-529c-451e-9d84-8ccc4091102b",
"tags": {
"ceph.crush_device_class": "None",
"ceph.db_device": "/dev/nvme0n1p5",
"ceph.wal_device": "/dev/nvme0n1p6",
},
"type": "block",
"vg_name": "ceph-c4de9e90-853e-4569-b04f-8677ef9a8c7a"
},
{
"path": "/dev/nvme0n1p5",
"tags": {
"PARTUUID": "69712eb4-be52-4618-ba46-e317d6d3d76e"
},
"type": "db"
}
],
"41": [
{
"devices": [
"/dev/nvme1n1p13"
],
"name": "osd-block-97bce07f-ae98-4fdb-83a9-9fa2f35cee60",
"tags": {
"ceph.crush_device_class": "None",
},
"type": "block",
"vg_name": "ceph-c1d48671-2a33-4615-95e3-cc1b18783f0c"
}
],
"9": [
{
"devices": [
"/dev/sdf"
],
"name": "osd-block-35323eb8-17c1-460d-8cc5-565f549e6991",
"tags": {
"ceph.crush_device_class": "None",
"ceph.db_device": "/dev/nvme0n1p7",
"ceph.wal_device": "/dev/nvme0n1p8",
},
"type": "block",
"vg_name": "ceph-9488e8b8-ec18-4860-93d3-6a1ad91c698c"
},
{
"path": "/dev/nvme0n1p7",
"tags": {
"PARTUUID": "ef0e9588-2a20-4c2c-8b62-d73945e01322"
},
"type": "db"
}
]
}
Required output:
osd.7 /dev/sde /dev/nvme0n1p5 /dev/nvme0n1p6
osd.41 /dev/nvme1n1p13 n/a n/a
osd.9 /dev/sdf /dev/nvme0n1p7 /dev/nvme0n1p7
Problems:
When I try parsing using jq .[][].devices, I get null values:
$ cat json | jq .[][].devices
[
"/dev/sde"
]
null
[
"/dev/nvme1n1p13"
]
null
[
"/dev/sdf"
]
null
I can solve it via jq .[][].devices[]?.
However, this trick doesn't help me when I do want to see where there's no value (to print n/a instead):
$ cat json | jq '.[][].tags | ."ceph.db_device"'
"/dev/nvme0n1p5"
null
"/dev/nvme0n1p3"
null
null
"/dev/nvme0n1p7"
null
And finally, I try to create a table:
$ cat json | jq -r '["osd."+keys[]], [.[][].devices[]?], [.[][].tags."ceph.db_device" // ""] | #csv' | column -t -s,
"osd.7" "osd.41" "osd.9"
"/dev/sde" "/dev/nvme0n1p13" "/dev/sdf"
"/dev/nvme0n1p5" "/dev/nvme0n1p7"
So the obvious problem is that the 3rd row doesn't match the correct values.
And the final problem is how do I transpose it from columns to rows, as detailed in the required output?
Would this do what you want?
jq --raw-output '
to_entries[] | [
"osd." + .key,
( .value[0]
| .devices[],
( .tags
| ."ceph.db_device" // "n/a",
."ceph.wal_device" // "n/a"
)
)
]
| #tsv
'
osd.7 /dev/sde /dev/nvme0n1p5 /dev/nvme0n1p6
osd.41 /dev/nvme1n1p13 n/a n/a
osd.9 /dev/sdf /dev/nvme0n1p7 /dev/nvme0n1p8
Demo
I'm pasting here a JSON example data which would require some manipulation to get a desired output which is mentioned in the next section to be read after this piece of JSON code.
I want to use jq for parsing my desired data.
{
"MetricAlarms": [
{
"EvaluationPeriods": 3,
"ComparisonOperator": "GreaterThanOrEqualToThreshold",
"AlarmActions": [
"Unimportant:Random:alarm:ELK2[10.1.1.2]-Root-Disk-Alert"
],
"AlarmName": "Unimportant:Random:alarm:ELK1[10.1.1.0]-Root-Alert",
"Dimensions": [
{
"Name": "path",
"Value": "/"
},
{
"Name": "InstanceType",
"Value": "m5.2xlarge"
},
{
"Name": "fstype",
"Value": "ext4"
}
],
"DatapointsToAlarm": 3,
"MetricName": "disk_used_percent"
},
{
"EvaluationPeriods": 3,
"ComparisonOperator": "GreaterThanOrEqualToThreshold",
"AlarmActions": [
"Unimportant:Random:alarm:ELK2[10.1.1.2]"
],
"AlarmName": "Unimportant:Random:alarm:ELK2[10.1.1.2]",
"Dimensions": [
{
"Name": "path",
"Value": "/"
},
{
"Name": "InstanceType",
"Value": "r5.2xlarge"
},
{
"Name": "fstype",
"Value": "ext4"
}
],
"DatapointsToAlarm": 3,
"MetricName": "disk_used_percent"
}
]
}
So when I Pass some Key1 & value1 as a parameter "Name": "InstanceType", to the JQ probably using cat | jq and output expected should be as below
m5.2xlarge
r5.2xlarge
A generic approach to search for a key-value pair (sk-sv) in input recursively and extract another key's value (pv) from objects found:
jq -r --arg sk Name \
--arg sv InstanceType \
--arg pv Value \
'.. | objects | select(contains({($sk): $sv})) | .[$pv]' file
I have two questions:
How can I use jq to search for "name" fields that start with an underscore (like _RDS_PASSWORD) and remove the leading underscore (so it becomes RDS_PASSWORD)
How can I use jq for "name" fields that start with an underscore (like _RDS_PASSWORD) and pass the value of the value cGFzc3dvcmQK to be decoded via base64? (ex: "cGFzc3dvcmQK" | base64 --decode)
Input:
[
{
"name": "RDS_DB_NAME",
"value": "rds_db_name"
},
{
"name": "RDS_HOSTNAME",
"value": "rds_hostname"
},
{
"name": "RDS_PORT",
"value": "1234"
},
{
"name": "RDS_USERNAME",
"value": "rds_username"
},
{
"name": "_RDS_PASSWORD",
"value": "cGFzc3dvcmQK"
}
]
Desired output:
[
{
"name": "RDS_DB_NAME",
"value": "rds_db_name"
},
{
"name": "RDS_HOSTNAME",
"value": "rds_hostname"
},
{
"name": "RDS_PORT",
"value": "1234"
},
{
"name": "RDS_USERNAME",
"value": "rds_username"
},
{
"name": "RDS_PASSWORD",
"value": "password"
}
]
Q1
walk( if type=="object" and has("name") and .name[0:1] == "_"
then .name |= .[1:]
else .
end)
If your jq does not have walk/1 then you can either upgrade to a more recent version of jq than 1.5, or include its def, which can be found at https://github.com/stedolan/jq/blob/master/src/builtin.jq
Q2
.. | objects | select(has("name") and .name[0:1] == "_") | .value
If you are certain that the encoded string was a UTF-8 string, you could use jq's #base64d; otherwise, invoke jq with the -r option and pipe the results to a decoder as you indicated you planned to do.
jq strikes again. Trying to get the value of DATABASES_DEFAULT based on the name in a json file that has a whole lot of names and I'm completely lost.
My file looks like the following (output of an aws ecs describe-task-definition) only much more complex; I've stripped this to the most basic example I can where the structure is still intact.
{
"taskDefinition": {
"status": "bar",
"family": "bar2",
"volumes": [],
"taskDefinitionArn": "bar3",
"containerDefinitions": [
{
"dnsSearchDomains": [],
"environment": [
{
"name": "bar4",
"value": "bar5"
},
{
"name": "bar6",
"value": "bar7"
},
{
"name": "DATABASES_DEFAULT",
"value": "foo"
}
],
"name": "baz",
"links": []
},
{
"dnsSearchDomains": [],
"environment": [
{
"name": "bar4",
"value": "bar5"
},
{
"name": "bar6",
"value": "bar7"
},
{
"name": "DATABASES_DEFAULT",
"value": "foo2"
}
],
"name": "boo",
"links": []
}
],
"revision": 1
}
}
I need the value of DATABASES_DEFAULT where the name is baz. Note that there are a lot of keypairs with name, I'm specifically talking about the one outside of environment.
I've been tinkering with this but only got this far before realizing that I don't understand how to access nested values.
jq '.[] | select(.name==DATABASES_DEFAULT) | .value'
which is returning
jq: error: DATABASES_DEFAULT/0 is not defined at <top-level>, line 1:
.[] | select(.name==DATABASES_DEFAULT) | .value
jq: 1 compile error
Obviously this a) doesn't work, and b) even if it did, it's independant of the name value. My thought was to return all the db defaults and then identify the one with baz, but I don't know if that's the right approach.
I like to think of it as digging down into the structure, so first you open the outer layers:
.taskDefinition.containerDefinitions[]
Now select the one you want:
select(.name =="baz")
Open the inner structure:
.environment[]
Select the desired object:
select(.name == "DATABASES_DEFAULT")
Choose the key you want:
.value
Taken together:
parse.jq
.taskDefinition.containerDefinitions[] |
select(.name =="baz") |
.environment[] |
select(.name == "DATABASES_DEFAULT") |
.value
Run it like this:
<infile jq -f parse.jq
Output:
"foo"
The following seems to work:
.taskDefinition.containerDefinitions[] |
select(
select(
.environment[] | .name == "DATABASES_DEFAULT"
).name == "baz"
)
The output is the object with the name key mapped to "baz".
$ jq '.taskDefinition.containerDefinitions[] | select(select(.environment[]|.name == "DATABASES_DEFAULT").name=="baz")' tmp.json
{
"dnsSearchDomains": [],
"environment": [
{
"name": "bar4",
"value": "bar5"
},
{
"name": "bar6",
"value": "bar7"
},
{
"name": "DATABASES_DEFAULT",
"value": "foo"
}
],
"name": "baz",
"links": []
}
I have the following JSON:
[
{
"name": "InstanceA",
"tags": [
{
"key": "environment",
"value": "production"
},
{
"key": "group",
"value": "group1"
}
]
},
{
"name": "InstanceB",
"tags": [
{
"key": "group",
"value": "group2"
},
{
"key": "environment",
"value": "staging"
}
]
}
]
I'm trying to get a flat output of value based on the condition key == 'environment'. I already tried select(boolean_expression), but I cannot get the desired output, like:
"InstanceA, production"
"InstanceB, staging"
Does jq support this kind of output? If so, how to do it?
Yes.
For example:
$ jq '.[] | "\(.name), \(.tags | from_entries | .environment)"' input.json
Output:
"InstanceA, production"
"InstanceB, staging"
jq '.[] | .name + ", " + (.tags[] | select(.key == "environment").value)' f.json
Here is a solution using join
.[]
| [.name, (.tags[] | if .key == "environment" then .value else empty end)]
| join(", ")