JSON JQ filter by date older than bash - json

Question

I have a json with this format of data in a text.json file
[
{
"name": "page/page1.html",
"properties": {
"lastModified": "2021-08-10T18:00:45+00:00",
}
},
{
"name": "page/page2.html",
"properties": {
"lastModified": "2021-08-10T19:24:23+00:00",
}
},
{
"name": "page/page3.html",
"properties": {
"lastModified": "2021-08-10T20:36:21+00:00",
}
}
]
I want to make a list of all the names of files which are last modified more that 30 minutes ago. This is my query at the moment to get a list of file names as a variable which i can use later.
file_names=`cat text.json | jq -r .[].name`
How can I use jq to filter for lastModified more than 30 minutes ago based on the timestamp in the properties so I only get the relevant file names?

Answer 1

I'd typically calculate the target date in native bash.
#!/usr/bin/env bash
# make sure we have bash new enough for printf %(...)T time-formatting
# this makes our script work even without GNU date
case $BASH_VERSION in
''|[123].*|4.[012].*) echo "ERROR: bash 4.3+ required" >&2; exit 1;;
esac
export TZ=UTC # force all timestamps to be in UTC (+00:00 / Z)
# faster, bash-builtin now=$(date +%s)
printf -v now '%(%s)T' -1
# faster, bash-builtin start_date_iso8601=$(date +%s -d '30 minutes ago')
start_date_epoch=$((now - 60*30))
printf -v start_date_iso8601 '%(%Y-%m-%dT%H:%M:%S+00:00)T' "$start_date_epoch"
# read our resulting names into an array (not a string)
# jq -j suppresses newlines so we can use NUL delimiters
while IFS= read -r -d '' name; do
names+=( "$name" )
done < <(
jq -j --arg start_date "$start_date_iso8601" '
.[] |
select(.properties.lastModified < $start_date) |
(.name, "\u0000")
' <text.json
)
# print the content of the array we just read the names into
printf 'Matching name: %q\n' "${names[#]}"

Answer 2

This seems to work
date=`date +%Y-%m-%d'T'%H:%M'Z' -d "15 min ago"`
file_names=`jq -r --arg date "$date" '.[] | select(.properties.lastModified < $date) | .name' < text.json`

Answer 3

Let jq do all date computations:
With bash 4 and above with mapfile:
mapfile -d '' last_modified < <(
jq --join-output '(now - 1800) as $date | .[] | select((.properties.lastModified | .[:18] + "Z" | fromdate) < $date) | .name + "\u0000"' input_file.json
)
# For debug purpose
declare -p last_modified
Without mapfile, records are delimited with ASCII RS control character rather than a null byte:
IFS=$'\36' read -ra last_modified < <(jq -j '(now - 1800) as $date | .[] | select((.properties.lastModified | .[:18] + "Z" | fromdate) < $date) | .name + "\u001e"' input_file.json)
Here is the stand-alone jq script with comments:
#!/usr/bin/env -S jq -jf
# Store current timestamp minus 30 minutes (1800 seconds) as $date
(now - 1800) as $date |
.[] |
#
select(
(
# Strip the numerical timezone offset out from the timestamp
# and replace it with the Z for UTC iso8601
# to make it an iso8601 date string that jq understands
.properties.lastModified | .[:18] + "Z" | fromdate
) < $date
) |
.name + "\u0000"