I have this json output
{
"FileSystems": [
{
"CreationToken": "CreationToken1",
"OwnerId": "OwnerId1",
"SizeInBytes": {
"Timestamp": 1552377599.0,
"Value": 1550721024
},
"Name": "Name1",
"NumberOfMountTargets": 3,
"FileSystemId": "fs-1",
"LifeCycleState": "available",
"CreationTime": 1550506468.0,
"PerformanceMode": "generalPurpose"
},
{
"CreationToken": "CreationToken2",
"OwnerId": "OwnerId2",
"SizeInBytes": {
"Timestamp": 1552377599.0,
"Value": 2390339584
},
"Name": "Name2",
"NumberOfMountTargets": 3,
"FileSystemId": "fs-2",
"LifeCycleState": "available",
"CreationTime": 1547663741.0,
"PerformanceMode": "generalPurpose"
}
]
}
By using this command
aws efs describe-file-systems | jq -r ".FileSystems[] | .SizeInBytes.Value"
I can recieve the NFS sized values:
1550725120
2390339584
But I need to get an output with the filesystem Name, so it should be like that:
Name1: 1550725120
Name2: 2390339584
How can I do it?
You can use string interpolation, like this:
jq -r ".FileSystems[] | \"\(.Name) \(.SizeInBytes.Value)\""
Related
I have the following JSON input:
{
"paging": {
"count": 0,
"total": 0,
"offset": 0,
"max": 0
},
"executions": [
{
"id": 5,
"href": "https://localhost.com.br",
"permalink": "https://localhost.com.br",
"status": "succeeded",
"project": "PROJETO",
"executionType": "scheduled",
"date-started": {
"unixtime": 1660793400012,
"date": "2022-08-18T03:30:00Z"
},
"date-ended": {
"unixtime": 1660793409694,
"date": "2022-08-18T03:30:09Z"
},
"job": {
"id": "cdkwednweoi-8745bjdf-kcjkjr8745",
"averageDuration": 0,
"name": "routine",
"group": "",
"project": "PROJECT",
"description": "",
"href": "https://localhost.com.br",
"permalink": "https://localhost.com.br"
},
"description": "runner",
"argstring": null,
"serverUUID": "jdnsdnasldnaje382nf5ubv",
"successfulNodes": [
"84jsk937nf"
]
}
]
}
First I want to select an array by a property name. And then I want to select an object of the array by the value of the propertyes.
Example of the desired informations on output:
"href"
"status"
"project"
"date-started":
"unixtime": 48298437239847,
"date": "2022-07-17"
"date-ended":
"unixtime": 48298437239847,
"date": "2022-07-17"
"job":
"name": "cleaner"
I knew how to get the firts values:
jq -r '.executions[] | [.href, .status, .project']
But the other ones I don't know how to do, I've tried with:
jq '.executions[] | with_entries( select(.value | has("date-started") ) )'
But it doesn't works.
Your first query produces a JSON array, so in this response, I'll assume it will suffice to produce an array of the eight values of interest in the order you've specified.
With your input, the following invocation produces the eight values as shown below:
jq '.executions[]
| [.href, .status, .project,
(."date-started" | (.unixtime, .date)),
(."date-ended" | (.unixtime, .date)),
.job.name]'
Output:
[
"https://localhost.com.br/rundeck/api/40/execution/2340",
"succeeded",
"PROJETO",
1660793400012,
"2022-08-18T03:30:00Z",
1660793409694,
"2022-08-18T03:30:09Z",
"proc_limpeza_saft"
]
this is Json: I try to use
jq -r '.[] | .DBSnapshotIdentifier'
But it doesn't work for me
{
"DBSnapshots": [
{
"DBSnapshotIdentifier": "auto-lims-final-snapshot",
"DBInstanceIdentifier": "auto-lims",
"SnapshotCreateTime": "2018-08-15T09:59:23.332000+00:00",
"Engine": "postgres",
"AllocatedStorage": 50,
"Status": "available",
"Port": 5432,
"AvailabilityZone": "us-east-2a",
"VpcId": "vpc-e799fc8f",
"InstanceCreateTime": "2018-04-09T08:28:03.565000+00:00",
"MasterUsername": "postgres",
"EngineVersion": "9.6.6",
"LicenseModel": "postgresql-license",
"SnapshotType": "manual",
"OptionGroupName": "default:postgres-9-6",
"PercentProgress": 100,
"StorageType": "gp2",
"Encrypted": false,
"DBSnapshotArn": "arn:aws:rds:us-east-2:833682533595:snapshot:auto-lims-final-snapshot",
"IAMDatabaseAuthenticationEnabled": false,
"ProcessorFeatures": [],
"DbiResourceId": "db-ZL5T2TA2PJVG6CVOJRO7HUOAXQ",
"TagList": [
{
"Key": "Environment",
"Value": "auto"
},
{
"Key": "Application",
"Value": "LIMS"
}
],
"SnapshotTarget": "region"
},
{
"DBSnapshotIdentifier": "automation-lims-before-postgres-12-5",
"DBInstanceIdentifier": "automation-lims",
"Engine": "postgres",
"AllocatedStorage": 500,
"Status": "available",
"Port": 5432,
"AvailabilityZone": "us-east-2b",
"VpcId": "vpc-09fa88d2884ee2083",
"InstanceCreateTime": "2019-12-26T11:19:41.947000+00:00",
"MasterUsername": "lims",
"EngineVersion": "9.6.20",
"LicenseModel": "postgresql-license",
"SnapshotType": "manual",
"OptionGroupName": "default:postgres-9-6",
"PercentProgress": 100
}
]
}
How I can get only nested lines.
You first need to select the DBSnapshots object, try:
.DBSnapshots[] | .DBSnapshotIdentifier
This will output:
"auto-lims-final-snapshot"
"automation-lims-before-postgres-12-5"
As you can try in this online demo
To include the .TagList.Key we can add the following:
.DBSnapshots[] | .DBSnapshotIdentifier, .TagList[]?.Key
This will now output
"auto-lims-final-snapshot"
"Environment"
"Application"
"automation-lims-before-postgres-12-5"
As you can try in this online demo
If you'd like the DBSnapshotIdentifier to be on the same line as the TagList we can use string interpolation like so:
.DBSnapshots[] | "\(.DBSnapshotIdentifier) - \(.TagList[]?.Key)"
Thiis will output:
"auto-lims-final-snapshot - Environment"
"auto-lims-final-snapshot - Application"
As this demo demostrates
The .[]? from the above command mean:
Like .[], but no errors will be output if . is not an array or object.
More information in JQ's documentation
I have a file that looks as below:
{
"Job": {
"Name": "sample_job",
"Description": "",
"Role": "arn:aws:iam::00000000000:role/sample_role",
"CreatedOn": "2021-10-21T23:35:23.660000-03:00",
"LastModifiedOn": "2021-10-21T23:45:41.771000-03:00",
"ExecutionProperty": {
"MaxConcurrentRuns": 1
},
"Command": {
"Name": "glueetl",
"ScriptLocation": "s3://aws-sample-s3/scripts/sample.py",
"PythonVersion": "3"
},
"DefaultArguments": {
"--TempDir": "s3://aws-sample-s3/temporary/",
"--class": "GlueApp",
"--enable-continuous-cloudwatch-log": "true",
"--enable-glue-datacatalog": "true",
"--enable-metrics": "true",
"--enable-spark-ui": "true",
"--job-bookmark-option": "job-bookmark-enable",
"--job-insights-byo-rules": "",
"--job-language": "python",
"--spark-event-logs-path": "s3://aws-sample-s3/logs"
},
"MaxRetries": 0,
"AllocatedCapacity": 100,
"Timeout": 2880,
"MaxCapacity": 100.0,
"WorkerType": "G.1X",
"NumberOfWorkers": 100,
"GlueVersion": "2.0"
}
}
I want to get key/value from "Name", "--enable-continuous-cloudwatch-log": "" and "--enable-metrics": "". So, I need to show the info like this:
"Name" "sample_job"
"--enable-continuous-cloudwatch-log" ""
"--enable-metrics" ""
UPDATE
Follow the tips from #Inian and #0stone0 I came close to it:
jq -r '(.Job ) + (.Job.DefaultArguments | { "--enable-continuous-cloudwatch-log", "--enable-metrics"}) | to_entries[] | "\"\(.key)\" \"\(.value)\""'
This extract the values I need but show all another key/values.
Since you're JSON isn't valid, I've converted it into:
{
"Job": {
"Name": "sample_job",
"Role": "sample_role_job"
},
"DefaultArguments": {
"--enable-continuous-cloudwatch-log": "test_1",
"--enable-metrics": ""
},
"Timeout": 2880,
"NumberOfWorkers": 10
}
Using the following filter:
"Name \(.Job.Name)\n--enable-continuous-cloudwatch-log \(.DefaultArguments."--enable-continuous-cloudwatch-log")\n--enable-metrics \(.DefaultArguments."--enable-metrics")"
We use string interpolation to show the desired output:
Name sample_job
--enable-continuous-cloudwatch-log test_1
--enable-metrics
jq --raw-output '"Name \(.Job.Name)\n--enable-continuous-cloudwatch-log \(.DefaultArguments."--enable-continuous-cloudwatch-log")\n--enable-metrics \(.DefaultArguments."--enable-metrics")"'
Online Demo
I'm pasting here a JSON example data which would require some manipulation to get a desired output which is mentioned in the next section to be read after this piece of JSON code.
I want to use jq for parsing my desired data.
{
"MetricAlarms": [
{
"EvaluationPeriods": 3,
"ComparisonOperator": "GreaterThanOrEqualToThreshold",
"AlarmActions": [
"Unimportant:Random:alarm:ELK2[10.1.1.2]-Root-Disk-Alert"
],
"AlarmName": "Unimportant:Random:alarm:ELK1[10.1.1.0]-Root-Alert",
"Dimensions": [
{
"Name": "path",
"Value": "/"
},
{
"Name": "InstanceType",
"Value": "m5.2xlarge"
},
{
"Name": "fstype",
"Value": "ext4"
}
],
"DatapointsToAlarm": 3,
"MetricName": "disk_used_percent"
},
{
"EvaluationPeriods": 3,
"ComparisonOperator": "GreaterThanOrEqualToThreshold",
"AlarmActions": [
"Unimportant:Random:alarm:ELK2[10.1.1.2]"
],
"AlarmName": "Unimportant:Random:alarm:ELK2[10.1.1.2]",
"Dimensions": [
{
"Name": "path",
"Value": "/"
},
{
"Name": "InstanceType",
"Value": "r5.2xlarge"
},
{
"Name": "fstype",
"Value": "ext4"
}
],
"DatapointsToAlarm": 3,
"MetricName": "disk_used_percent"
}
]
}
So when I Pass some Key1 & value1 as a parameter "Name": "InstanceType", to the JQ probably using cat | jq and output expected should be as below
m5.2xlarge
r5.2xlarge
A generic approach to search for a key-value pair (sk-sv) in input recursively and extract another key's value (pv) from objects found:
jq -r --arg sk Name \
--arg sv InstanceType \
--arg pv Value \
'.. | objects | select(contains({($sk): $sv})) | .[$pv]' file
I have an existing json which have a from like this:
{
"arg1": "Admin",
"arg2": 0,
"data": [
{
"arg3": "11",
"user": "user1",
"age": 51,
"arg4": "11"
},
{
"arg3": "22",
"user": "user2",
"age": 52,
"arg4": "22"
},
{
"arg3": "33",
"user": "user3",
"age": 53,
"arg4": "33"
},
{
"arg3": "44",
"user": "user4",
"age": 54,
"arg4": "44"
}
]
}
Then I try this command:
$ cat tmp.json|jq '.data|.[]|{user,age}'
{
"user": "user1",
"age": 51,
}
{
"user": "user2",
"age": 52,
}
{
"user": "user3",
"age": 53,
}
{
"user": "user4",
"age": 54,
}
What I expect to output is:
{
"department": "Admin",
"user_age": {
"user1": "51",
"user2": "52",
"user3": "53",
"user4": "54"
},
"year": 2016
}
In jq's manual, my request is close to example 23.
So I tried to use from_entries function
cat tmp.json|jq '.data|.[]|{user,age}|from_entries'
but get this error:
jq: error (at :30): Cannot index string with string "key"
I know this is because of its format not equal an entry.
So, what should I do to convert to the expected output?
from_entries expects an array of objects with key and value properties. You are however generating separate objects with user and age properties which just would not work. First of all, you would need to make sure that the properties are in an array, then have the separate key/value pairs for each of the values you want in the array.
With that said, you don't really need to use entries to build out the result, it's not the right tool for the job. You just want to pull some properties from each of the data objects and add to another object mapping users to ages. There's many ways this can be achieved, I would use reduce to accomplish this.
reduce .data[] as $d ({}; .[$d.user] = $d.age)
To get your final result, just combine the parts you need into the object you're requiring. I'm not sure where you got that 2016 from, I'm assuming it's just hardcoded.
{department: .arg1, user_age: (reduce .data[] as $d ({}; .[$d.user] = $d.age)), year: 2016}
you can also try:
cat tmp.json|jq '{department: .arg1, user_age: (.data|map({(.user): .age})|add), year: 2016}'
or
cat tmp.json|jq '{department: .arg1, user_age: .data|map({(.user): .age})|add, year: 2016}'
As Jeff explained, from_entries expects input of the form
[ {key:xxx, value:yyy} ]
so Hao's filter
.data | .[] | {user,age} | from_entries
requires two small modifications to generate the desired user_age object:
the values must be {key,value} objects
the values need to be collected into an array
e.g.
.data | [.[]|{key:user, value:age|tostring}] | from_entries
(we include tostring because the values in the example output are strings)
But since [ .[]| ... ] is the same as map(...) this could be written
.data | map({key:user, value:age|tostring}) | from_entries
Here is a complete solution:
{
"department": .arg1,
"user_age": .data | map({key:.user, value:.age|tostring}) | from_entries,
"year": 2016
}
Sample output
{
"department": "Admin",
"user_age": {
"user1": "51",
"user2": "52",
"user3": "53",
"user4": "54"
},
"year": 2016
}