MySQL - Sort table efficiently when querying by another attribute - mysql

Question

On this JS fiddle:
http://sqlfiddle.com/#!9/4c7bc0/7
CREATE TABLE big_table(id INTEGER AUTO_INCREMENT PRIMARY KEY,
user_id INTEGER NOT NULL);
The second query
SELECT * FROM big_table WHERE user_id IN ( 1, 100, 1000 )
ORDER BY id DESC LIMIT 10;
Is planned as: Using where; Using index; Using filesort
When big_table contains tens of millions of rows, the filesort kills the performance.
How do I ORDER BY faster?

Answer 1

For this query:
SELECT *
FROM big_table
WHERE user_id IN ( 1, 100, 1000 )
ORDER BY id DESC
LIMIT 10;
You cannot easily get rid of the work for the ORDER BY (there is one possible way, see below). Most importantly, you need an index on big_table(user_id):
CREATE INDEX idx_bigtable_userid ON big_table(user_id);
If there are only a few dozen or hundreds of rows that match, then this should be fine.
Another possibility is to rewrite the query, and use an index on:
CREATE INDEX idx_bigtable_userid ON big_table(user_id, id);
I should start by saying that I'm not 100% sure that this approach will work because of the DESC on id. But, the following query should use the index:
SELECT *
FROM big_table
WHERE user_id = 1
ORDER BY id DESC;
So you can do:
SELECT *
FROM ((SELECT t.*
FROM big_table t
WHERE t.user_id = 1
ORDER BY id DESC
LIMIT 10
) UNION ALL
(SELECT t.*
FROM big_table t
WHERE t.user_id = 100
ORDER BY id DESC
LIMIT 10
) UNION ALL
(SELECT t.*
FROM big_table t
WHERE t.user_id = 1000
ORDER BY id DESC
LIMIT 10
)
) t
ORDER BY id DESC
LIMIT 10;