Sending data taking too long but indexes aready create - mysql

Question

I am having problems with a query that is taking 20 seconds to return results :(
In table cases and cases_cstm, i have 960,000 rows
This is My query:
SELECT cases.id ,cases_cstm.assunto_c, cases.name , cases.case_number ,
cases.priority , accounts.name account_name ,
accounts.assigned_user_id account_name_owner ,
'Accounts' account_name_mod, cases.account_id ,
LTRIM(RTRIM(CONCAT(IFNULL(jt1.first_name,''),' ',IFNULL(jt1.last_name,'')))) assigned_user_name ,
jt1.created_by assigned_user_name_owner ,
'Users' assigned_user_name_mod, cases.status , cases.date_entered ,
cases.assigned_user_id
FROM cases
LEFT JOIN cases_cstm ON cases.id = cases_cstm.id_c
LEFT JOIN accounts accounts ON
cases.account_id=accounts.id AND accounts.deleted=0 AND
accounts.deleted=0
LEFT JOIN users jt1 ON
cases.assigned_user_id=jt1.id AND
jt1.deleted=0 AND jt1.deleted=0
where
(((LTRIM(RTRIM(CONCAT(IFNULL(accounts.name,'')))) LIKE 'rodrigo fernando%' OR
LTRIM(RTRIM(CONCAT(IFNULL(accounts.name,'')))) LIKE 'rodrigo fernando%'))) AND
cases.deleted=0 ORDER BY cases.date_entered DESC LIMIT 0,11;
here is the indexes of the table:
+-------+------------+--------------------+--------------+------------------
+-----------+-------------+----------+--------+------+------------+--------
| Table | Non_unique | Key_name | Seq_in_index | Column_name |Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment
+-------+------------+--------------------+--------------+------------------
+-----------+-------------+----------+--------+------+------------+---------
| cases | 0 | PRIMARY | 1 | id | A | 911472 | NULL | NULL | | BTREE |
| cases | 0 | case_number | 1 | case_number | A | 911472 | NULL | NULL | | BTREE |
|
| cases | 1 | idx_case_name | 1 | name | A | 911472 | NULL | NULL | YES | BTREE |
|
| cases | 1 | idx_account_id | 1 | account_id | A | 455736 | NULL | NULL | YES | BTREE |
|
| cases | 1 | idx_cases_stat_del | 1 | assigned_user_id| A | 106 | NULL | NULL | YES | BTREE |
|
| cases | 1 | idx_cases_stat_del | 2 | status | A | 197 | NULL | NULL | YES | BTREE |
|
| cases | 1 | idx_cases_stat_del | 3 | deleted | A | 214 | NULL | NULL | YES | BTREE |
|
| cases | 1 | idx_priority | 1 | priority | A | 455736 | NULL | NULL | YES | BTREE |
|
| cases | 1 | idx_date_entered | 1 | date_entered| A | 455736 | NULL | NULL | YES | BTREE |
+-------+------------+--------------------+--------------+------------------
+-----------+-------------+----------+--------+------+------------+---------
The Explain command of query(Image!)
this is the profile of query execution:
+--------------------+-----------+
| Status | Duration |
+--------------------+-----------+
| starting | 0.000122 |
| Opening tables | 0.000180 |
| System lock | 0.000005 |
| Table lock | 0.000005 |
| init | 0.000051 |
| optimizing | 0.000017 |
| statistics | 0.000071 |
| preparing | 0.000021 |
| executing | 0.000003 |
| Sorting result | 0.000004 |
| Sending data | 21.595455 |
| end | 0.000012 |
| query end | 0.000002 |
| freeing items | 0.000419 |
| logging slow query | 0.000005 |
| logging slow query | 0.000002 |
| cleaning up | 0.000004 |
Can someone help me undertang why the query is taking so long to execute?
Thanks!!

Answer 1

First, change your LEFT JOIN to accounts to an INNER JOIN I don't know if that will make a drastic change, but it makes a lot more sense if you understand the difference.
What you are saying with LEFT JOIN is "I want all cases, whether or not they have an associated account". An INNER JOIN here means "Give me all cases and return all accounts for them".
The end-result of your query is the same, because you are later on filtering things out with your WHERE clause, but I have a feeling that this might be why idx_account_id is being ignored.
A second, probably bigger problem is your where clause:
(((LTRIM(RTRIM(CONCAT(IFNULL(accounts.name,'')))) LIKE 'rodrigo fernando%' OR
LTRIM(RTRIM(CONCAT(IFNULL(accounts.name,'')))) LIKE 'rodrigo fernando%'))) AND
There's a ton of functions here, and MySQL can't optimize this using an index. Every record will be checked for this condition, and all functions you're using will be called for every record. This is most likely the biggest problem.
First, this can be simplified a bit. I think both sides of this OR statements are the same, so lets first turn this into one:
LTRIM(RTRIM(CONCAT(IFNULL(accounts.name,'')))) LIKE 'rodrigo fernando%'
Since you are adding a wildcard on one side of the switch statement, why bother with the RTRIM?
LTRIM(CONCAT(IFNULL(accounts.name,''))) LIKE 'rodrigo fernando%'
You don't need to CONCAT anything, if there's only one thing!
LTRIM(IFNULL(accounts.name,'')) LIKE 'rodrigo fernando%'
LTRIM works just fine on NULL values
LTRIM(accounts.name) LIKE 'rodrigo fernando%'
Alright, that saved us a bunch of functions. However, the last LTRIM is still a major problem as it completely still blocks mysql from using indexes. The solution is fairly simple though:
Update your accounts table, once:
UPDATE accounts SET name = LTRIM(name);
Make sure that whenever you insert new accounts, you trim before inserting. So you're really doing this now during INSERT time, not SELECT time.
Change your previous WHERE clause to:
accounts.name LIKE 'rodrigo fernando%'
Boom, you can now use an index on accounts.name and it will be fast as fuck.

Answer 2

I was able to solve this problem following the tips of Evert!
To be clear, this is a query that is being mounted dynamically by a system, I will still need to optimize the code to remove the functions that do not make sense in this case.
What helped me was to replace the LEFT by INNER joins in cases for cases_cstm and cases for accounts ... only with this change the query started being executed in 0.9 seconds!
Thanks for everyone's help!

Answer 3

You need to change your query as well as your indexing also. As you have indexed only on cases table, and you are using account and users table you should consider these table also while indexing. Make changes in your query as follows.
SELECT cases.id, cases_cstm.assunto_c, cases.name, cases.case_number, cases.priority, accounts.name account_name, accounts.assigned_user_id account_name_owner,
'Accounts' account_name_mod, cases.account_id, LTRIM(RTRIM(CONCAT(IFNULL(jt1.first_name,''),' ',IFNULL(jt1.last_name,'')))) assigned_user_name,
jt1.created_by assigned_user_name_owner,'Users' assigned_user_name_mod, cases.status, cases.date_entered, cases.assigned_user_id
FROM cases [enter link description here][1]
LEFT JOIN cases_cstm ON cases.id = cases_cstm.id_c
LEFT JOIN accounts accounts ON cases.account_id=accounts.id AND accounts.deleted=0 AND (TRIM(accounts.name) LIKE 'rodrigo fernando%' OR TRIM(accounts.name) LIKE 'rodrigo fernando%')
LEFT JOIN users jt1 ON cases.assigned_user_id=jt1.id AND jt1.deleted=0
WHERE cases.deleted=0 ORDER BY cases.date_entered DESC LIMIT 0,11;
Then first delete indexes that are not being used then create indexes on tables as follows:
Cases - deleted, date_entered (One index with multiple column)
accounts - deleted, name (One index with multiple column)
Users - deleted
Create these indexes and make sure sequence of columns selected and used in query must same, because MySql uses any leftmost prefix of the index. If you need more details go through these links:
Multiple-Column Indexes
MySQL ORDER BY / LIMIT performance