analysing what to improve via an EXPLAIN query on MySQL - mysql

Question

I'm not that well versed in indexing in MySQL, and am having a hard time trying to understand how the EXPLAIN output works, and how it can be read to know if my query is optimised or not.
I have a fairly large table (1.1M records) and I am executing the below query:
SELECT * FROM `Member` this_ WHERE (this_._Temporary_Flag = 0 or this_._Temporary_Flag
is null) and (this_._Deleted = 0 or this_._Deleted is null) and
(this_.Username = 'XXXXXXXX' or this_.Email = 'XXXXXXXX')
ORDER BY this_.Priority asc;
It takes a very long time to execute, between 30 - 60 seconds most of the times. The output of the EXPLAIN query is as below:
id select_type table type possible_keys key key_len ref rows Extra
----------------------------------------------------------------------------------------------------------------------------------
1 SIMPLE this_ ref_or_null _Temporary_Flag,_Deleted,username,email _Temporary_Flag 2 const 33735 Using where; Using filesort
What does this statement exactly mean? Does it mean that this query can be optimised? The table has mostly single-column indexes. What are the important output from the EXPLAIN query which I should use?

Answer 1

It is saying that the index it has chosen to use is the one called _Temporary_Flag (which I assume is on the _Temporary_Flag column). This is not a great index to use (it still leaves it looking at 33k records), but the best it can use in the situation. It might be worth adding an index covering both the _Temporary_Flag and the _Deleted columns.
However I doubt that narrows things down much.
One issue is that MySQL can only use a single index on a table within a query. It is likely that the best indexes to use would be on Username and another on Email, but as your query has an OR there it would have to chose one or the other.
A way round this restriction on indexes is to use 2 queries unioned together, something like this:-
SELECT *
FROM `Member` this_
WHERE (this_._Temporary_Flag = 0
or this_._Temporary_Flag is null)
and (this_._Deleted = 0
or this_._Deleted is null)
and this_.Email = 'XXXXXXXX'
UNION
SELECT *
FROM `Member` this_
WHERE (this_._Temporary_Flag = 0
or this_._Temporary_Flag is null)
and (this_._Deleted = 0
or this_._Deleted is null)
and this_.Username = 'XXXXXXXX'
ORDER BY this_.Priority asc;

Answer 2

http://dev.mysql.com/doc/refman/5.5/en/explain-output.html
Explain tells you what MySQL is doing, it doesn't necessarily tell you or even imply what can be done to make things better.
That said, there are a few warning signs that generally imply that you can optimize a query; the biggest one in this case is the occurrence of Using filesort in the Extra column.
The documentation explains what happens in that case:
MySQL must do an extra pass to find out how to retrieve the rows in
sorted order. The sort is done by going through all rows according to
the join type and storing the sort key and pointer to the row for all
rows that match the WHERE clause. The keys then are sorted and the
rows are retrieved in sorted order.
Another warning sign in your case is the key that is used. While not necessarily true in your case, a well normalized structure will generally require unique values for Username and Email.
So, why does it take so long when you are specifying those two things? Shouldn't the optimizer be able to just go straight to those rows? Probably not, because you are specifying them with an OR, which makes it difficult for the optimizer to use indexes to find those rows.
Instead, the optimizer decided to _Temporary_Flag to look through all the results, which probably didn't narrow the result set much, especially given that the Explain says that approximately 33735 rows were looked at.
So, working on the assumption that email and username will be much more selective than this key, you could try rewriting your query as a UNION.
SELECT * FROM `Member` this_
WHERE
(this_._Temporary_Flag = 0 or this_._Temporary_Flag
is null)
and
(this_._Deleted = 0 or this_._Deleted is null)
and this_.Email = 'XXXXXXXX'
UNION
SELECT * FROM `Member` this_
WHERE (this_._Temporary_Flag = 0 or this_._Temporary_Flag
is null)
and (this_._Deleted = 0 or this_._Deleted is null)
and
this_.Username = 'XXXXXXXX'
ORDER BY this_.Priority asc;
So, those are a couple of warning signs from EXPLAIN: Look for Using filesort and strange key choices as indicators that you can probably improve things.