jq: extract value based on different (calculated) value - json

Question

I am trying to filter down a very large json file (AWS output from aws rds describe-db-snapshots) into just a list of snapshots for deletion.
The final list of snapshots should be older than 60 days. I can discern their age via their SnapshotCreateTime, but I need their DBSnapshotIdentifier value to be able to delete them.
Greatly stripped down for SO purposes, below is the input.json file.
{
"Engine": "postgres",
"SnapshotCreateTime": "2017-08-22T16:35:42.302Z",
"AvailabilityZone": "us-east-1b",
"DBSnapshotIdentifier": "alex2-20170822-0108-bkup",
"AllocatedStorage": 5
}
{
"Engine": "postgres",
"SnapshotCreateTime": "2017-06-02T16:35:42.302Z",
"AvailabilityZone": "us-east-1a",
"DBSnapshotIdentifier": "alex-dbs-16opfr84gq4h9-snapshot-rtsmdbinstance-fr84gq4h9",
"AllocatedStorage": 5
}
{
"Engine": "postgres",
"SnapshotCreateTime": "2017-04-22T16:35:42.302Z",
"AvailabilityZone": "us-east-1a",
"DBSnapshotIdentifier": "alex3-20170422-update",
"AllocatedStorage": 5
}
I know about select but from what I can tell it can't handle the math needed for the time comparison in a one-liner. I figured I'd need to branch out to bash, so I've been messing with the following (clunky) workaround. It's not working, but I figured I'd include it as proof of effort.
THEN=$(date +'%Y%m%d' -d "`date`-60days")
while IFS= read -r i
do
awsDate=$(jq -r '.SnapshotCreateTime' < $i) // get time
snapDate=$(date -d $awsDate +'%Y%m%d') //convert to correct format
if [ $snapDate -gt $THEN ] //compare times
then
// something to copy the ID
fi
done < input.json
In this case I'd be looking for an output of
alex-dbs-16opfr84gq4h9-snapshot-rtsmdbinstance-fr84gq4h9
alex3-20170422-update

Answer 1

Here is an all-jq solution (i.e. one that does not depend on calling the date command). You might like to try a variation, e.g. passing some form of the date in, using one of the command-line options such as --arg.
jq currently does not quite understand the SnapshotCreateTime format; that's where the call to sub comes in:
def ago(days): now - (days*24*3600);
select(.SnapshotCreateTime | sub("\\.[0-9]*";"") < (ago(60) | todate))
| .DBSnapshotIdentifier
After fixing the sample input so that it is valid JSON, the output would be:
"alex-dbs-16opfr84gq4h9-snapshot-rtsmdbinstance-fr84gq4h9"
"alex3-20170422-update"
To strip the quotation marks, use the -r command-line option.

Answer 2

Here is a solution which defines a filter function which uses select, sub, fromdate and now.
def too_old:
select( .SnapshotCreateTime
| sub("[.][0-9]+Z";"Z") # remove fractional seconds
| fromdate # convert to unix time
| now - . # convert to age in seconds
| . > (86400 * 60) # true if older than 60 days in seconds
)
;
too_old
| .DBSnapshotIdentifier
If you place this in a file filter.jq and run jq with the -r option e.g
jq -M -r -f filter.jq input.json
it will produce the output you requested:
alex-dbs-16opfr84gq4h9-snapshot-rtsmdbinstance-fr84gq4h9
alex3-20170422-update