using an index when SELECTing from a MySQL join - mysql

Question

I have the following two MySQL/MariaDB tables:
CREATE TABLE requests (
request_id BIGINT UNSIGNED AUTO_INCREMENT PRIMARY KEY,
unix_timestamp DOUBLE NOT NULL,
[...]
INDEX unix_timestamp_index (unix_timestamp)
);
CREATE TABLE served_objects (
request_id BIGINT UNSIGNED NOT NULL,
object_name VARCHAR(255) NOT NULL,
[...]
FOREIGN KEY (request_id) REFERENCES requests (request_id)
);
There are several million columns in each table. There are zero or more served_objects per request. I have a view that provides a complete served_objects view by joining these two tables:
CREATE VIEW served_objects_view AS
SELECT
r.request_id AS request_id,
unix_timestamp,
object_name
FROM requests r
RIGHT JOIN served_objects so ON r.request_id=so.request_id;
This all seems pretty straightforward so far. But when I do a simple SELECT like this:
SELECT * FROM served_objects_view ORDER BY unix_timestamp LIMIT 5;
It takes a full minute or more. It's obviously not using the index. I've tried many different approaches, including flipping things around and using a LEFT or INNER join instead, but to no avail.
This is the output of the EXPLAIN for this SELECT:
+------+-------------+-------+--------+---------------+---------+---------+------------------+---------+---------------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+------+-------------+-------+--------+---------------+---------+---------+------------------+---------+---------------------------------+
| 1 | SIMPLE | so | ALL | NULL | NULL | NULL | NULL | 5196526 | Using temporary; Using filesort |
| 1 | SIMPLE | r | eq_ref | PRIMARY | PRIMARY | 8 | db.so.request_id | 1 | |
+------+-------------+-------+--------+---------------+---------+---------+------------------+---------+---------------------------------+
Is there something fundamental here that prevents the index from being used? I understand that it needs to use a temporary table to satisfy the view and that that's interfering with the ability to use the index. But I'm hoping that some trick exists that will allow me SELECT from the view while honouring the indexes in the requests table.

Answer 1

You're using a notorious performance antipattern.
SELECT * FROM served_objects_view ORDER BY unix_timestamp LIMIT 5;
You've told the query planner to make a copy of your whole view (in RAM or temp storage), sort it, and toss out all but five rows. So, it obeyed. It really didn't care how long it took.
SELECT * is generally considered harmful to query performance, and this is the kind of case why that's true.
Try this deferred-join optimization
SELECT a.*
FROM served_objects_view a
JOIN (
SELECT request_id
FROM served_objects_view
ORDER BY unix_timestamp
LIMIT 5
) b ON a.request_id = b.request_id
This sorts a smaller subset of data (just the request_id and timestamp values). It then fetches a small subset of the view's rows.
If it's still too slow for your purposes, try creating a compound index on request (unix_timestamp, request_id). But that's probably unnecessary. If it is necessary, concentrate on optimizing the subquery.
Remark: RIGHT JOIN? Really? Don't you want just JOIN?

Answer 2

VIEWs are not always well-optimized. Does the query run slow when you use the SELECT? Have you added the suggested index?
What version of MySQL/MariaDB are you using? There may have been optimization improvements in newer versions, and an upgrade might help.
My point is, you may have to abandon VIEW.

Answer 3

The answer provided by O. Jones was the right approach; thanks! The big saviour here is that if the inner SELECT refers only to columns from the requests table (such as the case when SELECTing only request_id), the optimizer can satisfy the view without performing a join, making it lickety-split.
I had to make two adjustments, though, to make it produce the same results as the original SELECT. First, if non-unique request_ids are returned by the inner SELECT, the outer JOIN creates a cross-product of these non-unique entries. These duplicate rows can be effectively discarded by changing the outer SELECT into a SELECT DISTINCT.
Second, if the ORDER BY column can contain non-unique values, the result can contain irrelevant rows. These can be effectively discarded by also SELECTing orderByCol and adding AND a.orderByCol = b.orderByCol to the JOIN rule.
So my final solution, which works well if orderByCol comes from the requests table, is the following:
SELECT DISTINCT a.*
FROM served_objects_view a
JOIN (
SELECT request_id, <orderByCol> FROM served_objects_view
<whereClause>
ORDER BY <orderByCol> LIMIT <startRow>,<nRows>
) b ON a.request_id = b.request_id AND a.<orderByCol> = b.<orderByCol>
ORDER BY <orderByCol>;
This is a more convoluted solution than I was hoping for, but it works, so I'm happy.
One final comment. An INNER JOIN and a RIGHT JOIN are effectively the same thing here, so I originally formulated it in terms of a RIGHT JOIN because that's the way I was conceptualizing it. However, after some experimentation (after your challenge) I discovered that an INNER join is much more efficient. (It's what allows the optimizer to satisfy the view without performing a join if the inner SELECT refers only to columns from the requests table.) Thanks again!