Slow MySQL query after adding multiple OR conditions to indexed column - mysql

Question

I have a query running on MySQL (v5.5 -- I know it's old but it's what I have to work with for now). The table A below has ~16 million rows and B has ~700,000. The query looks something like this:
SELECT A.id, A.x, A.y, A.z, B.foo FROM A STRAIGHT_JOIN B ON A.id = B.id
where A.x = 53 ORDER BY A.y desc LIMIT 0, 30;
There's an index setup on A.id as well as on B.id.
There's also an index setup on (A.x, A.y) (this key/index is called DocsByType).
This query has worked great so far, it's performance has always been sub-second or thereabouts. Recently though, I have a need to occasionally check against an additional possible value for A.x in the where clause. The following query is now performing very poorly, on average taking ~15 secs to complete:
SELECT A.id, A.x, A.y, A.z, B.foo FROM A STRAIGHT_JOIN B ON A.id = B.id
where (A.x = 18 or A.x = 53) ORDER BY A.y desc LIMIT 0, 30;
The explain for the fast query with only one comparison looks like this:
+----+-------------+-------+------+-----------------------------------------------------+------------+---------+----------------+---------+-------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------+------+-----------------------------------------------------+------------+---------+----------------+---------+-------------+
| 1 | SIMPLE | A | ref | Documents1,Documents2,Documents3,DocsByType,KEY_AID | DocsByType | 4 | const | 1870603 | Using where |
| 1 | SIMPLE | B | ref | KEY_BID | KEY_BID | 4 | mydb.B.id | 1 | |
+----+-------------+-------+------+-----------------------------------------------------+------------+---------+----------------+---------+-------------+
The explain for the multi-comparison query looks like this:
+----+-------------+-------+-------+-----------------------------------------------------+------------+---------+----------------+---------+-----------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------+-------+-----------------------------------------------------+------------+---------+----------------+---------+-----------------------------+
| 1 | SIMPLE | A | range | Documents1,Documents2,Documents3,DocsByType,KEY_AID | DocsByType | 4 | NULL | 1878693 | Using where; Using filesort |
| 1 | SIMPLE | B | ref | KEY_BID | KEY_BID | 4 | mydb.B.id | 1 | |
+----+-------------+-------+-------+-----------------------------------------------------+------------+---------+----------------+---------+-----------------------------+
I can see that there's a filesort operation that's not in the first query. Also the type is "range" instead of "ref", and the ref is "NULL" instead of "const". Removing the order by clause fixes it completely, so that it completes in less than a second, but it's important that the results are sorted.
Query optimization is not my strong suit. I would have thought that it would have worked exactly the same given that the column is already indexed. Can anyone explain why this behaves the way it does and suggest a way to optimize the query? Please also note that the new query might need to use 3, 4 or even 5 possible values for the where clause (but always against the same column).
I've also tried running the queries using MySQL 5.8 but the result is the same. My table is using the MyISAM engine.

Answer 1

Suppose you have a big list of people's names. And the goal is to find the first 30 Smiths (ordered by first name). The first query is fast because it is essentially doing the WHERE, ORDER BY and LIMIT all at once:
The second is messier because it is effectively done thus:
Find the first names of all the 'Smiths',
Find the first names of all the 'Joneses'
Sort the first names and show the first 30
There are two things to speed up your slow query:
( SELECT A.id, A.x, A.y, A.z, B.foo FROM A JOIN B ON A.id = B.id
where (A.x = 18)
ORDER BY A.y desc LIMIT 30 )
UNION ALL
( SELECT A.id, A.x, A.y, A.z, B.foo FROM A JOIN B ON A.id = B.id
where (A.x = 53) -- Note
ORDER BY A.y desc LIMIT 30 )
ORDER BY A.y desc LIMIT 0, 30 -- Yes, repeated
Comments:
STRAIGHT_JOIN is unnecessary, JOIN will happen to do the same thing
Each subquery will use INDEX(x,y) and make use of LIMIT.
ALL is faster than the default, and is appropriate in this case
If you need to "paginate", the limits need to be handled as described here: http://mysql.rjweb.org/doc.php/index_cookbook_mysql#or
Any number of UNIONs can be tacked together. However, at some point, the cost of all the unions will outweigh the benefit. (It is not practical to try to predict where the cutoff is.)
It would be faster to do the LIMIT 30 before JOINing to B. That way, you would do only 30 lookups in B; my way needs 60; your original query needed lots more.