SQL query running really slow - mysql

Question

I am running this query to search the database:
SELECT
IFNULL(firstname, '') AS firstname,
IFNULL(lastname, '') AS lastname,
IFNULL(age, ' ') AS age,
email,
telephone,
comments,
ref
FROM person
RIGHT JOIN
order ON person.oID = order.ref
WHERE
LOWER(firstname) LIKE LOWER ('%{$search}%') OR
LOWER(lastname) LIKE LOWER ('%{$search}%') OR
LOWER(email) LIKE LOWER ('%{$search}%') OR
LOWER(telephone) LIKE LOWER ('%{$search}%') OR
LOWER(ref) LIKE LOWER ('%{$search}%');
It's doing a lot of processing, but how can I get these results faster? The page is taking about 6-7 seconds to load, If i run the query in PHPMyAdmin, the query takes 3-4 seconds to run. Its not a huge database, 3000 entries or so. I have added an index to the ref, email, firstname and lastname columns but that doesnt seem to have made any difference. Can anyone help?

Answer 1

The reason this query is slow is because you've combined two convenient but slow features of MySQL in the slowest possible way.
FUNCTION(column) LIKE %matchstring% requires a scan of the table; no ordered index can help satisfy this search because it's unanchored.
condition OR condition OR condition requires the table to be rescanned once per OR clause.
You also happen to be ignoring the fact that MySQL's searches are already case-insensitive if you have set up your column collations correctly.
Finally, it's not clear what you're doing with the RIGHT JOINed table data. Which columns of your result set come from that table? If you don't need data from that table get rid of it.
So, in summary, what you have is slow x many.
So, how can you fix this? The most important thing is for you to get rid of as many of these unanchored scans as possible. If you can change them to
email LIKE '{$search}%'
so the LOWER() functions and leading %s in the LIKE terms can be eliminated, you will have a big win.
If this sort of cast-a-wide-net search feature is critical to your application, you should consider using MySQL fulltext searching.
Or you could consider creating a new column in your table that's the concatenation of all the columns you presently search, so you can search it just once.
Edit to explain LIKE slowness
If the column haystack is indexed, the search haystack LIKE 'needle%' runs quite quickly. That's because the BTREE style index is inherently ordered. To search this way, MySQL can random-access the first possible match, and then scan sequentially to the last possible match.
But the search haystack LIKE '%needle%' can't use random access to find the first possible match in the index. The first possible match could be anywhere. So it has to scan all the values of the haystack one by one for the needle.

Answer 2

I would suggest that you change the right join to an inner join. The fields that you are looking for look like they are coming from the person table anyway, so the where clause is turning the query into an inner join.
SELECT
IFNULL(firstname, '') AS firstname,
IFNULL(lastname, '') AS lastname,
IFNULL(age, ' ') AS age,
email,
telephone,
comments,
ref
FROM person INNER JOIN
order
ON person.oID = order.ref
WHERE
LOWER(firstname) LIKE LOWER ('%{$search}%') OR
LOWER(lastname) LIKE LOWER ('%{$search}%') OR
LOWER(email) LIKE LOWER ('%{$search}%') OR
LOWER(telephone) LIKE LOWER ('%{$search}%') OR
LOWER(ref) LIKE LOWER ('%{$search}%');
Second, create an index on order(ref). This should greatly reduce the search space for the where clause. The syntax is:
create index order_ref on `order`(ref);
By the way, order is a bad name for a table, because it is a SQL reserved word. I would suggest orders instead.

Answer 3

why dont you use Full text search instead of bunch of OR and LOWER ?
SELECT
IFNULL(firstname, '') AS firstname,
IFNULL(lastname, '') AS lastname,
IFNULL(age, ' ') AS age,
email,
telephone,
comments,
ref
FROM person
RIGHT JOIN
order ON person.oID = order.ref
WHERE
MATCH (LOWER(firstname), LOWER(lastname),LOWER(email),LOWER(ref))
AGAINST ('$search' IN BOOLEAN MODE)
to run this faster you need to add an index .
ALTER TABLE person ADD FULLTEXT(firstname, lastname,email,ref);