Closed. This question needs debugging details. It is not currently accepting answers.
Edit the question to include desired behavior, a specific problem or error, and the shortest code necessary to reproduce the problem. This will help others answer the question.
Closed 3 years ago.
Improve this question
I am using the following query which is very slow despite having indices, can someone help?
table a has close to 50k entries and table b about 2 million
i tried creating index on batchid in table 2 but it didnt help
select distinct x, y
from t1 a,t2 b
where b.batch_id = a.batch_id and b.sm_id = a.smid and b.status = 'new' and target_userid = *some value*
limit 10
here is the explain output
+----+-------------+-------+------+-----------------------------------+---------------+---------+------------------------------+---------+------------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------+------+-----------------------------------+---------------+---------+------------------------------+---------+------------------------------+
| 1 | SIMPLE | b | ALL | smreconhistory_sm_id_status_index | NULL | NULL | NULL | 2033644 | Using where; Using temporary |
| 1 | SIMPLE | a | ref | smid_batch_id,idx_batch_id | smid_batch_id | 16 | sync.b.sm_id,sync.b.batch_id | 1 | Using where; Distinct |
+----+-------------+-------+------+-----------------------------------+---------------+---------+------------------------------+---------+--------------------
index for table 1
+-----------------+------------+----------------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment |
+-----------------+------------+----------------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| SMReconJobEntry | 0 | PRIMARY | 1 | id | A | 49323 | NULL | NULL | | BTREE | | |
| SMReconJobEntry | 1 | smreconjobentry_ugid_index | 1 | ugid | A | 49323 | NULL | NULL | | BTREE | | |
| SMReconJobEntry | 1 | smid_batch_id | 1 | smid | A | 49323 | NULL | NULL | | BTREE | | |
| SMReconJobEntry | 1 | smid_batch_id | 2 | batch_id | A | 49323 | NULL | NULL | | BTREE | | |
| SMReconJobEntry | 1 | idx_batch_id | 1 | batch_id | A | 49323 | NULL | NULL | | BTREE | | |
+-----------------+------------+----------------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+--
for table 2
+----------------+------------+-----------------------------------+--------------+--------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment |
+----------------+------------+-----------------------------------+--------------+--------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| SMReconHistory | 0 | PRIMARY | 1 | id | A | 2033644 | NULL | NULL | | BTREE | | |
| SMReconHistory | 1 | smreconhistory_sm_id_status_index | 1 | sm_id | A | 18 | NULL | NULL | YES | BTREE | | |
| SMReconHistory | 1 | smreconhistory_sm_id_status_index | 2 | status | A | 18 | NULL | NULL | | BTREE | | |
| SMReconHistory | 1 | gmailunit_id_foreign | 1 | gmailunit_id | A | 2033644 | NULL | NULL | YES | BTREE | | |
+----------------+------------+-----------------------------------+--------------+--------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
Do not use find_in_set, replace function, it will calculate every time, I would suggest to use IN with static values.
x IN ('a', 'b', 'c')
Related
I need to speed up this query. What can I do?
select i.resp_id as id from int_result i, response_set rs, cx_store_child cbu
where rs.survey_id IN(5550512,5550516,5550521,5550520,5590351,5590384,5679615,5679646,5691634,5699259,5699266,5699270)
and i.q_id IN(52603091,52251250,52250724,52251333,52919541,52920117,54409178,54409806,54625102,54738933,54739117,54739221)
and rs.t >= '2017-08-30 00:00:00' and rs.t <= '2017-09-30 00:00:00'
and i.response_set_id = rs.id and rs.cx_business_unit_id = cbu.child_bu_id
and cbu.business_unit_id = 30850
group by rs.cx_business_unit_id, i.a_id, extract(day from rs.t)
------------+------------+-----------------+--------------+-----------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment |
+------------+------------+-----------------+--------------+-----------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| int_result | 0 | PRIMARY | 1 | id | A | 240843099 | NULL | NULL | | BTREE | | |
| int_result | 1 | q_id | 1 | q_id | A | 1442174 | NULL | NULL | | BTREE | | |
| int_result | 1 | a_id | 1 | a_id | A | 20070258 | NULL | NULL | | BTREE | | |
| int_result | 1 | resp_id | 1 | resp_id | A | 120421549 | NULL | NULL | | BTREE | | |
| int_result | 1 | response_set_id | 1 | response_set_id | A | 26760344 | NULL | NULL | | BTREE | | |
| int_result | 1 | survey_id | 1 | survey_id | A | 503855 | NULL | NULL | YES | BTREE | | |
| int_result | 1 | survey_id_2 | 1 | survey_id | A | 1459655 | NULL | NULL | YES | BTREE | | |
| int_result | 1 | survey_id_2 | 2 | q_id | A | 2736853 | NULL | NULL | | BTREE | | |
+------------+------------+-----------------+--------------+-----------------+-----------+-------------+----------+--------+------+------------+------
--+---------+---------------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment |
+--------------+------------+----------------------+--------------+---------------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| response_set | 0 | PRIMARY | 1 | id | A | 14307454 | NULL | NULL | | BTREE | | |
| response_set | 1 | survey_id | 1 | survey_id | A | 223553 | NULL | NULL | | BTREE | | |
| response_set | 1 | id | 1 | id | A | 14307454 | NULL | NULL | | BTREE | | |
| response_set | 1 | external_id | 1 | external_id | A | 2921 | NULL | NULL | YES | BTREE | | |
| response_set | 1 | panel_member_id | 1 | panel_member_id | A | 357686 | NULL | NULL | YES | BTREE | | |
| response_set | 1 | email_group | 1 | email_group | A | 21259 | NULL | NULL | YES | BTREE | | |
| response_set | 1 | survey_timestamp_idx | 1 | survey_id | A | 433559 | NULL | NULL | | BTREE | | |
| response_set | 1 | survey_timestamp_idx | 2 | t | A | 14307454 | NULL | NULL | YES | BTREE | | |
| response_set | 1 | bu_id | 1 | cx_business_unit_id | A | 2246 | NULL | NULL | YES | BTREE | | |
----------------+------------+------------------+--------------+------------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment |
+----------------+------------+------------------+--------------+------------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| cx_store_child | 0 | PRIMARY | 1 | id | A | 13667 | NULL | NULL | | BTREE | | |
| cx_store_child | 0 | bu_child_ref | 1 | business_unit_id | A | 13667 | NULL | NULL | YES | BTREE | | |
| cx_store_child | 0 | bu_child_ref | 2 | child_bu_id | A | 13667 | NULL | NULL | YES | BTREE | | |
| cx_store_child | 1 | cx_feedback_id | 1 | cx_feedback_id | A | 506 | NULL | NULL | YES | BTREE | | |
| cx_store_child | 1 | business_unit_id | 1 | business_unit_id | A | 13667 | NULL | NULL | YES | BTREE | | |
| cx_store_child | 1 | child_bu_id | 1 | child_bu_id | A | 13667 | NULL | NULL | YES | BTREE | | |
+----------------+------------+------------------+--------------+------------------+-----------+-------------+----------+--------+------+------------+
You seem to have an index on response_set(survey_id, t).
Try creating a so-called compound covering index on
response_set(t, survey_id, cx_business_unit_id)
This may help optimize the part of your query using that table. Why? Your query calls for a range scan on t, and a column used in a range scan must be first in its compound index.
Similarly, an index on int_result (q_id, resp_id, response_set_id) may help extract the data you need from that table.
Some notes:
It's hard to tell what your query does. Maybe some explanation will help you get better results here?
and rs.t >= '2017-08-30 00:00:00' and rs.t <= '2017-09-30 00:00:00' is probably incorrect as to the end of the time range. It probably contains an off-by-one error. Do you want < in place of <= ? What you have given includes the records with timestamps precisely at midnight on 30-Sep-2017, but no records on that day after midnight.
You have one index on int_result(survey_id, q_id) and another on int_result(survey_id). The latter index is entirely redundant with the former and you can drop it.
You seem to have lots of single-column indexes. Pro tip: don't add such indexes unless you know you need them. They rarely help speed up arbitrary queries and always slow down insertions and updates. Why might you need them? If you have a query you know needs them, or you need to enforce uniqueness. Drop the indexes you don't need.
Use 21st-century JOIN syntax instead of the old-timey comma-join syntax as follows. It's easier to read.
from int_result i
join response_set rs on i.response_set_id = rs.id
join cx_store_child cbu on rs.cx_business_unit_id = cbu.child_bu_id
Read this. You're maintaining a large database and it's worth your time to learn a lot about indexing. http://use-the-index-luke.com/
There are many ways that the Optimizer may attempt to perform the query. The following indexes give it some flexibility to find the optimal order of hitting the tables:
cbu: INDEX(business_unit_id, child_bu_id)
rs: INDEX(t, cx_business_unit_id, survey_id)
rs: INDEX(survey_id, t, cx_business_unit_id)
rs: INDEX(cx_business_unit_id, survey_id, t)
i: INDEX(response_set_id, q_id)
i: INDEX(q_id, response_set_id)
I arranged for rs and cbu to have "covering" indexes in all cases; this helps some.
(Yes, you should change to JOIN...ON as O. Jones suggests. And the rest of his suggestions.)
Before further discussion, please provide SHOW CREATE TABLE; the could be datatype issues, too.
A PRIMARY KEY is a UNIQUE key is an INDEX -- so INDEX(id) in rs is redundant.
I am working on large tables of my database (mysql). Some of my queries are running more than 5 minutes. Here is an example of my query that is running slow:
select b.DESCRIPTION collateral_type, a.description brand, a.year,
a.model, a.plate_number, d.description fuel, a.chassis_number,
a.engine_number, c.description as color, insurance_name
from lms_loan_application_collateral a inner join
lms_collateral_type b
on a.COLLATERAL_TYPE_ID = b.id left join
lms_color c
on a.color_id = c.id left join
lms_fuel_type d
on a.fuel_type_id = d.id inner join
lms_loan_application e
on e.id = a.loan_application_id inner join
lms_dlr_dtl f
on e.code = f.lano inner join
lms_loanapp_dtl g
on f.lano = g.lano
where b.description in ('Motorcycle', 'Automotive', 'Heavy Equipment');
here is a mysql explain of the query
+----+-------------+-------+-------+--------------------------------------------------------+-------------------------+---------+------------+-------+----------------------------------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------+-------+--------------------------------------------------------+-------------------------+---------+------------+-------+----------------------------------------------------+
| 1 | SIMPLE | g | index | lms_loanapp_dtl_lano | lms_loanapp_dtl_lano | 23 | NULL | 23432 | Using where; Using index |
| 1 | SIMPLE | f | ref | lano | lano | 23 | new.g.LANO | 1 | Using index |
| 1 | SIMPLE | b | ALL | lms_collateral_type_description,lms_collateral_type_id | NULL | NULL | NULL | 11 | Using where; Using join buffer (Block Nested Loop) |
| 1 | SIMPLE | a | ref | collateral_type_id | collateral_type_id | 5 | new.b.ID | 4067 | Using index condition |
| 1 | SIMPLE | e | ref | lms_loan_application_id | lms_loan_application_id | 153 | func | 1 | Using index condition; Using where |
| 1 | SIMPLE | c | ALL | LMS_color_id | NULL | NULL | NULL | 20 | Range checked for each record (index map: 0x1) |
| 1 | SIMPLE | d | ALL | LMS_fuel_type_id | NULL | NULL | NULL | 4 | Using where; Using join buffer (Block Nested Loop) |
+----+-------------+-------+-------+--------------------------------------------------------+-------------------------+---------+------------+-------+----------------------------------------------------+
under the key column, there are NULL values. I am not sure if mysql uses the index keys I have created.
Below is the list of indexes I created for the involved tables:
SHOW INDEX FROM lms_loan_application_collateral FROM new;
+---------------------------------+------------+---------------------+--------------+---------------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment |
+---------------------------------+------------+---------------------+--------------+---------------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| lms_loan_application_collateral | 1 | color_id | 1 | color_id | A | 36 | NULL | NULL | YES | BTREE | | |
| lms_loan_application_collateral | 1 | fuel_type_id | 1 | fuel_type_id | A | 6 | NULL | NULL | YES | BTREE | | |
| lms_loan_application_collateral | 1 | loan_application_id | 1 | loan_application_id | A | 89493 | NULL | NULL | YES | BTREE | | |
| lms_loan_application_collateral | 1 | collateral_type_id | 1 | collateral_type_id | A | 22 | NULL | NULL | YES | BTREE | | |
+---------------------------------+------------+---------------------+--------------+---------------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
SHOW INDEX FROM lms_collateral_type FROM new;
+---------------------+------------+---------------------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment |
+---------------------+------------+---------------------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| lms_collateral_type | 1 | lms_collateral_type_description | 1 | DESCRIPTION | A | 11 | NULL | NULL | YES | BTREE | | |
| lms_collateral_type | 1 | lms_collateral_type_id | 1 | ID | A | 11 | NULL | NULL | YES | BTREE | | |
+---------------------+------------+---------------------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
SHOW INDEX FROM lms_color FROM new;
+-----------+------------+--------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment |
+-----------+------------+--------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| lms_color | 1 | LMS_color_id | 1 | id | A | 20 | NULL | NULL | | BTREE | | |
+-----------+------------+--------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
SHOW INDEX FROM lms_fuel_type FROM new;
+---------------+------------+------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment |
+---------------+------------+------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| lms_fuel_type | 1 | LMS_fuel_type_id | 1 | id | A | 4 | NULL | NULL | YES | BTREE | | |
+---------------+------------+------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
SHOW INDEX FROM lms_loan_application FROM new;
+----------------------+------------+-------------------------+--------------+--------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment |
+----------------------+------------+-------------------------+--------------+--------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| lms_loan_application | 1 | lmspis_id | 1 | PIS_ID | A | 1878 | NULL | NULL | YES | BTREE | | |
| lms_loan_application | 1 | loan_type_id | 1 | LOAN_TYPE_ID | A | 6 | NULL | NULL | YES | BTREE | | |
| lms_loan_application | 1 | lms_loan_application_id | 1 | ID | A | 1878 | NULL | NULL | YES | BTREE | | |
+----------------------+------------+-------------------------+--------------+--------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
SHOW INDEX FROM lms_dlr_dtl FROM new;
+-------------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment |
+-------------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| lms_dlr_dtl | 0 | PRIMARY | 1 | LDDID | A | 90066 | NULL | NULL | | BTREE | | |
| lms_dlr_dtl | 1 | lano | 1 | LANO | A | 90066 | NULL | NULL | YES | BTREE | | |
+-------------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
SHOW INDEX FROM lms_loanapp_dtl FROM new;
+-----------------+------------+----------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment |
+-----------------+------------+----------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| lms_loanapp_dtl | 0 | PRIMARY | 1 | LLADID | A | 23432 | NULL | NULL | | BTREE | | |
| lms_loanapp_dtl | 1 | lms_loanapp_dtl_lano | 1 | LANO | A | 23432 | NULL | NULL | YES | BTREE | | |
+-----------------+------------+----------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
The tables which indicates the type as ALL does not utilise the indexes. Create indexes on id column on tables represents in c and d and one index on both description and id columns on b.
create index ix_id on lms_color(id);
create index ix_id on lms_fuel_type(id);
create index ix_description_id on lms_collateral_type(description,id);
MySQL is using ALL for your WHERE because of low cardinality. So MySQL thinks it is cheaper to just doing table scan over 11 entries than using binary search with index.
The real problem is that you do not filter using column whose index has higher cardinality (lms_loanapp_dtl_lano and collateral_type_id).
I am not sure why this query isn't using an index on the table world_cities. The indexes are the same and in the same order. The data types are of the same type and same length. The only difference is the key_name.
So why does it not use the key?
if I run the query I get: 318824 rows in set (2 min 51.30 sec)
Here is what I have done:
explain select * from zip_codes zc
inner join world_cities wc on
zc.city = wc.city
and zc.country = wc.country
and zc.region = wc.region;
+----+-------------+-------+------+---------------------+---------------------+---------+---------------------------------------------------------------+---------+-------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------+------+---------------------+---------------------+---------+---------------------------------------------------------------+---------+-------------+
| 1 | SIMPLE | wc | ALL | city_country_region | NULL | NULL | NULL | 3173958 | |
| 1 | SIMPLE | zc | ref | country_region_city | country_region_city | 165 | largedbapi.wc.city,largedbapi.wc.country,largedbapi.wc.region | 1 | Using where |
+----+-------------+-------+------+---------------------+---------------------+---------+---------------------------------------------------------------+---------+-------------+
mysql> show indexes from world_cities where Key_name like '%city%';
+--------------+------------+---------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment |
+--------------+------------+---------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| world_cities | 1 | city_country_region | 1 | city | A | NULL | NULL | NULL | YES | BTREE | |
| world_cities | 1 | city_country_region | 2 | country | A | NULL | NULL | NULL | YES | BTREE | |
| world_cities | 1 | city_country_region | 3 | region | A | NULL | NULL | NULL | YES | BTREE | |
+--------------+------------+---------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
mysql> show indexes from zip_codes where Key_name like '%city%';
+-----------+------------+---------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment |
+-----------+------------+---------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| zip_codes | 1 | country_region_city | 1 | city | A | 218424 | NULL | NULL | YES | BTREE | |
| zip_codes | 1 | country_region_city | 2 | country | A | 218424 | NULL | NULL | YES | BTREE | |
| zip_codes | 1 | country_region_city | 3 | region | A | 436849 | NULL | NULL | YES | BTREE | |
+-----------+------------+---------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
Your city_country_region key on world_cities has a NULL cardinality, while a non-NULL one is required for it to be use. Running analyze on it should fix it.
ANALYZE TABLE world_cities
Also have a look at this post for more info on NULL cardinality.
Closed. This question is off-topic. It is not currently accepting answers.
Want to improve this question? Update the question so it's on-topic for Stack Overflow.
Closed 10 years ago.
Improve this question
I have a database containing around 5million rows and am having issues with a query taking a long time (over a minute). I was hoping this information would be enough for someone to give me some suggestions, if I need to post more information just let me know.
Thank you in advance for any advice
EXPLAIN SELECT count( * ) AS count
FROM vtiger_time
INNER JOIN vtiger_crmentity ON vtiger_crmentity.crmid = vtiger_time.timeid
INNER JOIN vtiger_crmentityrel ON ( vtiger_crmentityrel.relcrmid = vtiger_crmentity.crmid
OR vtiger_crmentityrel.crmid = vtiger_crmentity.crmid )
LEFT JOIN vtiger_users ON vtiger_users.id = vtiger_crmentity.smownerid
LEFT JOIN vtiger_groups ON vtiger_groups.groupid = vtiger_crmentity.smownerid
WHERE vtiger_crmentity.deleted =0
AND (
vtiger_crmentityrel.crmid =211294
OR vtiger_crmentityrel.relcrmid =211294
)
+----+-------------+---------------------+-------------+-------------------------------------------------+----------------+---------+--------------------------------+-------+------------------------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+---------------------+-------------+-------------------------------------------------+----------------+---------+--------------------------------+-------+------------------------------------------+
| 1 | SIMPLE | vtiger_crmentityrel | index_merge | crmid,relcrmid | crmid,relcrmid | 4,4 | NULL | 5881 | Using union(crmid,relcrmid); Using where |
| 1 | SIMPLE | vtiger_crmentity | ref | PRIMARY,deleted,deleted_2,crmentity_multi_index | deleted_2 | 4 | const | 84424 | Using where; Using index |
| 1 | SIMPLE | vtiger_users | eq_ref | PRIMARY | PRIMARY | 4 | crm.vtiger_crmentity.smownerid | 1 | Using index |
| 1 | SIMPLE | vtiger_groups | eq_ref | PRIMARY | PRIMARY | 4 | crm.vtiger_crmentity.smownerid | 1 | Using index |
| 1 | SIMPLE | vtiger_time | eq_ref | PRIMARY,timeid | PRIMARY | 4 | crm.vtiger_crmentity.crmid | 1 | Using index |
+----+-------------+---------------------+-------------+-------------------------------------------------+----------------+---------+--------------------------------+-------+------------------------------------------+
Additionally these are my currently set indexes
+------------------+------------+----------------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment |
+------------------+------------+----------------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| vtiger_crmentity | 0 | PRIMARY | 1 | crmid | A | 755968 | NULL | NULL | | BTREE | |
| vtiger_crmentity | 0 | crmid | 1 | crmid | A | 755968 | NULL | NULL | | BTREE | |
| vtiger_crmentity | 1 | crmentity_IDX0 | 1 | smcreatorid | A | 15 | NULL | NULL | | BTREE | |
| vtiger_crmentity | 1 | crmentity_IDX1 | 1 | smownerid | A | 15 | NULL | NULL | | BTREE | |
| vtiger_crmentity | 1 | crmentity_IDX2 | 1 | modifiedby | A | 15 | NULL | NULL | | BTREE | |
| vtiger_crmentity | 1 | deleted | 1 | deleted | A | 15 | NULL | NULL | | BTREE | |
| vtiger_crmentity | 1 | deleted | 2 | smownerid | A | 15 | NULL | NULL | | BTREE | |
| vtiger_crmentity | 1 | smownerid | 1 | smownerid | A | 199 | NULL | NULL | | BTREE | |
| vtiger_crmentity | 1 | smownerid | 2 | deleted | A | 199 | NULL | NULL | | BTREE | |
| vtiger_crmentity | 1 | deleted_2 | 1 | deleted | A | 15 | NULL | NULL | | BTREE | |
| vtiger_crmentity | 1 | deleted_2 | 2 | smownerid | A | 15 | NULL | NULL | | BTREE | |
| vtiger_crmentity | 1 | smownerid_2 | 1 | smownerid | A | 385 | NULL | NULL | | BTREE | |
| vtiger_crmentity | 1 | smownerid_2 | 2 | deleted | A | 758 | NULL | NULL | | BTREE | |
| vtiger_crmentity | 1 | crm_ownerid_del_setype_idx | 1 | smownerid | A | 15 | NULL | NULL | | BTREE | |
| vtiger_crmentity | 1 | crm_ownerid_del_setype_idx | 2 | deleted | A | 15 | NULL | NULL | | BTREE | |
| vtiger_crmentity | 1 | crm_ownerid_del_setype_idx | 3 | setype | A | 613 | NULL | NULL | YES | BTREE | |
| vtiger_crmentity | 1 | crmentity_multi_index | 1 | crmid | A | 755968 | NULL | NULL | | BTREE | |
| vtiger_crmentity | 1 | crmentity_multi_index | 2 | smownerid | A | 755968 | NULL | NULL | | BTREE | |
| vtiger_crmentity | 1 | crmentity_multi_index | 3 | deleted | A | 755968 | NULL | NULL | | BTREE | |
+------------------+------------+----------------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
+---------------------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment |
+---------------------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| vtiger_crmentityrel | 1 | crmid | 1 | crmid | A | 223960 | NULL | NULL | | BTREE | |
| vtiger_crmentityrel | 1 | relcrmid | 1 | relcrmid | A | 12442 | NULL | NULL | | BTREE | |
+---------------------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
+--------------+------------+---------------+--------------+---------------+-----------+-------------+----------+--------+------+------------+---------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment |
+--------------+------------+---------------+--------------+---------------+-----------+-------------+----------+--------+------+------------+---------+
| vtiger_users | 0 | PRIMARY | 1 | id | A | 39 | NULL | NULL | | BTREE | |
| vtiger_users | 1 | idx_user_name | 1 | user_name | A | 39 | NULL | NULL | YES | BTREE | |
| vtiger_users | 1 | user_password | 1 | user_password | A | 39 | NULL | NULL | YES | BTREE | |
+--------------+------------+---------------+--------------+---------------+-----------+-------------+----------+--------+------+------------+---------+
+---------------+------------+---------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment |
+---------------+------------+---------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| vtiger_groups | 0 | PRIMARY | 1 | groupid | A | 5 | NULL | NULL | | BTREE | |
| vtiger_groups | 1 | groupname | 1 | groupname | A | 5 | NULL | NULL | YES | BTREE | |
| vtiger_groups | 1 | idx_groups_123group | 1 | groupname | A | 5 | NULL | NULL | YES | BTREE | |
+---------------+------------+---------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
+----------------+------------+-------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment |
+----------------+------------+-------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| vtiger_time | 0 | PRIMARY | 1 | timeid | A | 591772 | NULL | NULL | | BTREE | |
| vtiger_time | 0 | timeid | 1 | timeid | A | 591772 | NULL | NULL | | BTREE | |
| vtiger_time | 1 | relatedto | 1 | relatedto | A | 1405 | NULL | NULL | YES | BTREE | |
| vtiger_time | 1 | date_start | 1 | date_start | A | 7129 | NULL | NULL | YES | BTREE | |
| vtiger_time | 1 | relatedto_2 | 1 | relatedto | A | 3269 | NULL | NULL | YES | BTREE | |
+----------------+------------+-------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
+------------------+------------+-------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment |
+------------------+------------+-------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| vtiger_timecf | 0 | PRIMARY | 1 | timeid | A | 591324 | NULL | NULL | | BTREE | |
| vtiger_timecf | 0 | timeid | 1 | timeid | A | 591324 | NULL | NULL | | BTREE | |
| vtiger_timecf | 1 | timeid_2 | 1 | timeid | A | 591324 | NULL | NULL | | BTREE | |
+------------------+------------+-------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
After adding the following index
ALTER TABLE vtiger_crmentity ADD INDEX TMP_deletion_smownerid_crmid (smownerid, deleted,crmid);
The new explain:
+----+-------------+---------------------+-------------+--------------------------------------------------------------------------+----------------+---------+--------------------------------+-------+------------------------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+---------------------+-------------+--------------------------------------------------------------------------+----------------+---------+--------------------------------+-------+------------------------------------------+
| 1 | SIMPLE | vtiger_crmentityrel | index_merge | crmid,relcrmid | crmid,relcrmid | 4,4 | NULL | 5891 | Using union(crmid,relcrmid); Using where |
| 1 | SIMPLE | vtiger_crmentity | ref | PRIMARY,crmid,deleted,deleted_2,crmentity_multi_index,_deletion_crmid | deleted | 4 | const | 84424 | Using where; Using index |
| 1 | SIMPLE | vtiger_users | eq_ref | PRIMARY | PRIMARY | 4 | crm.vtiger_crmentity.smownerid | 1 | Using index |
| 1 | SIMPLE | vtiger_groups | eq_ref | PRIMARY | PRIMARY | 4 | crm.vtiger_crmentity.smownerid | 1 | Using index |
| 1 | SIMPLE | vtiger_time | eq_ref | PRIMARY,timeid | PRIMARY | 4 | crm.vtiger_crmentity.crmid | 1 | Using index |
+----+-------------+---------------------+-------------+--------------------------------------------------------------------------+----------------+---------+--------------------------------+-------+------------------------------------------+
Rewriting the query would help, but since you cannot do that, you may benefit from a composite index on vtiger_crmentity.crmid and vtiger_crmentity.deleted. But since it isn't using one of the the index you currently have on vtiger_crmentity.crmid, it might not use that new one. If it doesn't, try adding vtiger_crmentity.crmid to the vtiger_crmentity.deleted index. Since it is already using the index on vtiger_crmentity.deleted, this will make the index a covering index so the query will not need to read from the table.
As I mentioned in my comment, there are a lot of duplicate indexes. The won't affect the query for this question, but they will slow down inserts and increase table size. The following indexes are redundant:
vtiger_crmentity.crmid
vtiger_crmentity.smownerid
vtiger_crmentity.smownerid_2
vtiger_crmentity.deleted_2
vtiger_groups.idx_groups_123group
vtiger_time.timeid
vtiger_time.relatedto_2
vtiger_timecf.timeid
vtiger_timcfe.timeid_2
You also shouldn't need the index on the user password column since you shouldn't be looking for users by password.
Try adding this index:
ALTER TABLE vtiger_crmentityrel ADD INDEX ix_crmentityrel_crmid_relcrmid (crmid,relcrmid);
This should allow the first row in the explain to be satisfied completely by the two indexes on the crmentityrel table and should save you a few thousand lookups.
All other parts of the execution plan are using covering indexes so if this doesn't help, I'm not sure there is much more you can do without restructuring the data or the query.
I am trying to speed up a query on a large table with WHERE clauses on two columns, as far as I can, MySQL is only using the ALERT_ID column.
Is there a way to rewrite this query using both indices?
SHOW_INDEX and EXPLAIN output is below.
show index from alert_hit;
+-----------+------------+-------------------+--------------+-------------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment |
+-----------+------------+-------------------+--------------+-------------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| alert_hit | 0 | PRIMARY | 1 | id | A | 15181402 | NULL | NULL | | BTREE | | |
| alert_hit | 1 | alert_id | 1 | alert_id | A | 20 | NULL | NULL | YES | BTREE | | |
| alert_hit | 1 | timestamp | 1 | timestamp | A | 446511 | NULL | NULL | YES | BTREE | | |
| alert_hit | 1 | data_source_id | 1 | data_source_id | A | 20 | NULL | NULL | YES | BTREE | | |
| alert_hit | 1 | filter_syndicated | 1 | filter_syndicated | A | 20 | NULL | NULL | YES | BTREE | | |
| alert_hit | 1 | unique_id | 1 | unique_id | A | 5060467 | NULL | NULL | YES | BTREE | | |
| alert_hit | 1 | date_created | 1 | date_created | A | 281137 | NULL | NULL | | BTREE | | |
| alert_hit | 1 | language | 1 | language | A | 20 | NULL | NULL | YES | BTREE | | |
| alert_hit | 1 | region | 1 | region | A | 42406 | NULL | NULL | YES | BTREE | | |
| alert_hit | 1 | market_rank | 1 | market_rank | A | 20 | NULL | NULL | YES | BTREE | | |
+-----------+------------+-------------------+--------------+-------------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
explain select count(id) as history FROM alert_hit force index(alert_id, timestamp) where alert_id in (9045,9046,9047,9048,9049,9050,9051,9052,9330,9332) AND timestamp between DATE_SUB( NOW(), INTERVAL 1*2 day) and DATE_SUB( NOW(), INTERVAL 1 day);
+----+-------------+-----------+-------+--------------------+----------+---------+------+-------+-------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-----------+-------+--------------------+----------+---------+------+-------+-------------+
| 1 | SIMPLE | alert_hit | range | alert_id,timestamp | alert_id | 5 | NULL | 99578 | Using where |
+----+-------------+-----------+-------+--------------------+----------+---------+------+-------+-------------+
You need to have one index on both fields
ALTER TABLE alert_hit ADD INDEX `IDX-alert_id-timestamp` (`alert_id`, `timestamp`);
Also MySQL will use the multi column index up to the first field for which there is a range condition in the WHERE clause, so in this case order matters and timestamp should be last in the index.
As suggested by #spencer7593 selecting COUNT(1) instead of count(id) might also be better.