Trouble optimizing MySQL query containing range expression, order by, and limit - mysql

Question

I'm having trouble optimizing what I think is a reasonable / straightforward query in MySQL. After spending a few late nights on this, I thought I'd post my question here as I'm sure the solution will be obvious to somebody.
Here are the columns in a simplified version of my table T:
id: varchar(32) not null (primary key)
timestamp: bigint(20) unsigned not null
family: char(32) not null
size: bigint(20) unsigned not null
And here's the query that I need to optimize:
select
id
from
T
where
family = 'some constant'
and
timestamp between T1 and T2
order by
size desc
limit
5
My table is fairly large (~630M rows) so I'm hoping that an index can do most of the work for me... but I'm having trouble picking the right columns for my index.
It seems that in order for MySQL to use an index to answer a range query (like what I'm doing w/ the timestamp), that column must be the last column in the index. But then there's the "order by", which is on a different column. I'm not sure which one of these columns should be last in my index, so I've tried creating the following indices:
i1, on (family, timestamp, size)
i2, on (family, size, timestamp)
... but neither of these seems to help.
Any idea what I'm doing wrong?
(BTW I'm running MySQL 8 in Amazon RDS, in case that makes a difference.)
Thanks in advance for any helpful suggestions you may have!
EDIT #1 ---------------------------------------
I just created this simplified table that I described above, and copied 10M rows worth of data from the original table to the simplified table, just to keep things clean. Then I ran the following query:
mysql> select
-> id, size
-> from
-> T
-> where
-> family = 'be0bf4a203797729f38c6355b6d80903'
-> and
-> timestamp between 1578460425887 and 1584710866343
-> order by
-> size desc;
... and took 1.27 seconds. I really need this to be faster, otherwise this sort of query (which I need to run many times per second) will take much too long on the real dataset.
Here are the results of an EXPLAIN on the query above:
+----+-------------+-------+------------+-------+---------------------------+---------------------------+---------+------+--------+----------+------------------------------------------+
| id | select_type | table | partitions | type | possible_keys | key | key_len | ref | rows | filtered | Extra |
+----+-------------+-------+------------+-------+---------------------------+---------------------------+---------+------+--------+----------+------------------------------------------+
| 1 | SIMPLE | T | NULL | range | idx_family_timestamp_size | idx_family_timestamp_size | 136 | NULL | 178324 | 100.00 | Using where; Using index; Using filesort |
+----+-------------+-------+------------+-------+---------------------------+---------------------------+---------+------+--------+----------+------------------------------------------+
I bet it's the filesort that's killing performance. Any ideas?
EDIT #2 ---------------------------------------
Oops, I just realized that I forgot the LIMIT in EDIT #1's query. I've fixed that here, and also grew T to 100M rows -- so now it's 10x the size that it was in my previous edit.
Now my query takes almost 10 sec. to run, and the results of the EXPLAIN are as follows:
mysql> explain
-> select
-> id, size
-> from
-> T
-> where
-> family = 'be0bf4a203797729f38c6355b6d80903'
-> and
-> timestamp between 1578460425887 and 1584710866343
-> order by
-> size desc
-> limit
-> 5;
+----+-------------+-------+------------+-------+-----------------------------------------------------+---------------------------+---------+------+--------+----------+--------------------------------------------------------+
| id | select_type | table | partitions | type | possible_keys | key | key_len | ref | rows | filtered | Extra |
+----+-------------+-------+------------+-------+-----------------------------------------------------+---------------------------+---------+------+--------+----------+--------------------------------------------------------+
| 1 | SIMPLE | T | NULL | range | idx_family_timestamp_size,idx_family_size_timestamp | idx_family_size_timestamp | 144 | NULL | 410744 | 100.00 | Using where; Using index for skip scan; Using filesort |
+----+-------------+-------+------------+-------+-----------------------------------------------------+---------------------------+---------+------+--------+----------+--------------------------------------------------------+