I need to speed up this query. What can I do?
select i.resp_id as id from int_result i, response_set rs, cx_store_child cbu
where rs.survey_id IN(5550512,5550516,5550521,5550520,5590351,5590384,5679615,5679646,5691634,5699259,5699266,5699270)
and i.q_id IN(52603091,52251250,52250724,52251333,52919541,52920117,54409178,54409806,54625102,54738933,54739117,54739221)
and rs.t >= '2017-08-30 00:00:00' and rs.t <= '2017-09-30 00:00:00'
and i.response_set_id = rs.id and rs.cx_business_unit_id = cbu.child_bu_id
and cbu.business_unit_id = 30850
group by rs.cx_business_unit_id, i.a_id, extract(day from rs.t)
------------+------------+-----------------+--------------+-----------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment |
+------------+------------+-----------------+--------------+-----------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| int_result | 0 | PRIMARY | 1 | id | A | 240843099 | NULL | NULL | | BTREE | | |
| int_result | 1 | q_id | 1 | q_id | A | 1442174 | NULL | NULL | | BTREE | | |
| int_result | 1 | a_id | 1 | a_id | A | 20070258 | NULL | NULL | | BTREE | | |
| int_result | 1 | resp_id | 1 | resp_id | A | 120421549 | NULL | NULL | | BTREE | | |
| int_result | 1 | response_set_id | 1 | response_set_id | A | 26760344 | NULL | NULL | | BTREE | | |
| int_result | 1 | survey_id | 1 | survey_id | A | 503855 | NULL | NULL | YES | BTREE | | |
| int_result | 1 | survey_id_2 | 1 | survey_id | A | 1459655 | NULL | NULL | YES | BTREE | | |
| int_result | 1 | survey_id_2 | 2 | q_id | A | 2736853 | NULL | NULL | | BTREE | | |
+------------+------------+-----------------+--------------+-----------------+-----------+-------------+----------+--------+------+------------+------
--+---------+---------------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment |
+--------------+------------+----------------------+--------------+---------------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| response_set | 0 | PRIMARY | 1 | id | A | 14307454 | NULL | NULL | | BTREE | | |
| response_set | 1 | survey_id | 1 | survey_id | A | 223553 | NULL | NULL | | BTREE | | |
| response_set | 1 | id | 1 | id | A | 14307454 | NULL | NULL | | BTREE | | |
| response_set | 1 | external_id | 1 | external_id | A | 2921 | NULL | NULL | YES | BTREE | | |
| response_set | 1 | panel_member_id | 1 | panel_member_id | A | 357686 | NULL | NULL | YES | BTREE | | |
| response_set | 1 | email_group | 1 | email_group | A | 21259 | NULL | NULL | YES | BTREE | | |
| response_set | 1 | survey_timestamp_idx | 1 | survey_id | A | 433559 | NULL | NULL | | BTREE | | |
| response_set | 1 | survey_timestamp_idx | 2 | t | A | 14307454 | NULL | NULL | YES | BTREE | | |
| response_set | 1 | bu_id | 1 | cx_business_unit_id | A | 2246 | NULL | NULL | YES | BTREE | | |
----------------+------------+------------------+--------------+------------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment |
+----------------+------------+------------------+--------------+------------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| cx_store_child | 0 | PRIMARY | 1 | id | A | 13667 | NULL | NULL | | BTREE | | |
| cx_store_child | 0 | bu_child_ref | 1 | business_unit_id | A | 13667 | NULL | NULL | YES | BTREE | | |
| cx_store_child | 0 | bu_child_ref | 2 | child_bu_id | A | 13667 | NULL | NULL | YES | BTREE | | |
| cx_store_child | 1 | cx_feedback_id | 1 | cx_feedback_id | A | 506 | NULL | NULL | YES | BTREE | | |
| cx_store_child | 1 | business_unit_id | 1 | business_unit_id | A | 13667 | NULL | NULL | YES | BTREE | | |
| cx_store_child | 1 | child_bu_id | 1 | child_bu_id | A | 13667 | NULL | NULL | YES | BTREE | | |
+----------------+------------+------------------+--------------+------------------+-----------+-------------+----------+--------+------+------------+
You seem to have an index on response_set(survey_id, t).
Try creating a so-called compound covering index on
response_set(t, survey_id, cx_business_unit_id)
This may help optimize the part of your query using that table. Why? Your query calls for a range scan on t, and a column used in a range scan must be first in its compound index.
Similarly, an index on int_result (q_id, resp_id, response_set_id) may help extract the data you need from that table.
Some notes:
It's hard to tell what your query does. Maybe some explanation will help you get better results here?
and rs.t >= '2017-08-30 00:00:00' and rs.t <= '2017-09-30 00:00:00' is probably incorrect as to the end of the time range. It probably contains an off-by-one error. Do you want < in place of <= ? What you have given includes the records with timestamps precisely at midnight on 30-Sep-2017, but no records on that day after midnight.
You have one index on int_result(survey_id, q_id) and another on int_result(survey_id). The latter index is entirely redundant with the former and you can drop it.
You seem to have lots of single-column indexes. Pro tip: don't add such indexes unless you know you need them. They rarely help speed up arbitrary queries and always slow down insertions and updates. Why might you need them? If you have a query you know needs them, or you need to enforce uniqueness. Drop the indexes you don't need.
Use 21st-century JOIN syntax instead of the old-timey comma-join syntax as follows. It's easier to read.
from int_result i
join response_set rs on i.response_set_id = rs.id
join cx_store_child cbu on rs.cx_business_unit_id = cbu.child_bu_id
Read this. You're maintaining a large database and it's worth your time to learn a lot about indexing. http://use-the-index-luke.com/
There are many ways that the Optimizer may attempt to perform the query. The following indexes give it some flexibility to find the optimal order of hitting the tables:
cbu: INDEX(business_unit_id, child_bu_id)
rs: INDEX(t, cx_business_unit_id, survey_id)
rs: INDEX(survey_id, t, cx_business_unit_id)
rs: INDEX(cx_business_unit_id, survey_id, t)
i: INDEX(response_set_id, q_id)
i: INDEX(q_id, response_set_id)
I arranged for rs and cbu to have "covering" indexes in all cases; this helps some.
(Yes, you should change to JOIN...ON as O. Jones suggests. And the rest of his suggestions.)
Before further discussion, please provide SHOW CREATE TABLE; the could be datatype issues, too.
A PRIMARY KEY is a UNIQUE key is an INDEX -- so INDEX(id) in rs is redundant.
Related
Closed. This question needs debugging details. It is not currently accepting answers.
Edit the question to include desired behavior, a specific problem or error, and the shortest code necessary to reproduce the problem. This will help others answer the question.
Closed 3 years ago.
Improve this question
I am using the following query which is very slow despite having indices, can someone help?
table a has close to 50k entries and table b about 2 million
i tried creating index on batchid in table 2 but it didnt help
select distinct x, y
from t1 a,t2 b
where b.batch_id = a.batch_id and b.sm_id = a.smid and b.status = 'new' and target_userid = *some value*
limit 10
here is the explain output
+----+-------------+-------+------+-----------------------------------+---------------+---------+------------------------------+---------+------------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------+------+-----------------------------------+---------------+---------+------------------------------+---------+------------------------------+
| 1 | SIMPLE | b | ALL | smreconhistory_sm_id_status_index | NULL | NULL | NULL | 2033644 | Using where; Using temporary |
| 1 | SIMPLE | a | ref | smid_batch_id,idx_batch_id | smid_batch_id | 16 | sync.b.sm_id,sync.b.batch_id | 1 | Using where; Distinct |
+----+-------------+-------+------+-----------------------------------+---------------+---------+------------------------------+---------+--------------------
index for table 1
+-----------------+------------+----------------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment |
+-----------------+------------+----------------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| SMReconJobEntry | 0 | PRIMARY | 1 | id | A | 49323 | NULL | NULL | | BTREE | | |
| SMReconJobEntry | 1 | smreconjobentry_ugid_index | 1 | ugid | A | 49323 | NULL | NULL | | BTREE | | |
| SMReconJobEntry | 1 | smid_batch_id | 1 | smid | A | 49323 | NULL | NULL | | BTREE | | |
| SMReconJobEntry | 1 | smid_batch_id | 2 | batch_id | A | 49323 | NULL | NULL | | BTREE | | |
| SMReconJobEntry | 1 | idx_batch_id | 1 | batch_id | A | 49323 | NULL | NULL | | BTREE | | |
+-----------------+------------+----------------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+--
for table 2
+----------------+------------+-----------------------------------+--------------+--------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment |
+----------------+------------+-----------------------------------+--------------+--------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| SMReconHistory | 0 | PRIMARY | 1 | id | A | 2033644 | NULL | NULL | | BTREE | | |
| SMReconHistory | 1 | smreconhistory_sm_id_status_index | 1 | sm_id | A | 18 | NULL | NULL | YES | BTREE | | |
| SMReconHistory | 1 | smreconhistory_sm_id_status_index | 2 | status | A | 18 | NULL | NULL | | BTREE | | |
| SMReconHistory | 1 | gmailunit_id_foreign | 1 | gmailunit_id | A | 2033644 | NULL | NULL | YES | BTREE | | |
+----------------+------------+-----------------------------------+--------------+--------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
Do not use find_in_set, replace function, it will calculate every time, I would suggest to use IN with static values.
x IN ('a', 'b', 'c')
I'm having an issue with the speed of a query I wan't to do.
The rooms table has 2685588 rows, and the room_active has less than 100, but it isn't slow with normal queries.
The query is:
SELECT rooms.*, room_active.active_users
FROM rooms LEFT JOIN room_active ON (room_active.roomid = rooms.id)
WHERE caption LIKE '%room%' ORDER BY room_active.active_users LIMIT 50
Runtime: 4.3s
It takes more than 5 seconds to execute when it should take less than one like does this one:
SELECT rooms.*, room_active.active_users
FROM rooms LEFT JOIN room_active ON (room_active.roomid = rooms.id)
WHERE caption LIKE '%room%' LIMIT 50
Runtime: 0.3s
The describe for the first query is this:
+----+-------------+-------------+--------+----------------+---------+---------+----------------+---------+----------------------------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------------+--------+----------------+---------+---------+----------------+---------+----------------------------------------------+
| 1 | SIMPLE | rooms | ALL | NULL | NULL | NULL | NULL | 2210576 | Using where; Using temporary; Using filesort |
| 1 | SIMPLE | room_active | eq_ref | PRIMARY,roomid | PRIMARY | 4 | xukys.rooms.id | 1 | NULL |
+----+-------------+-------------+--------+----------------+---------+---------+----------------+---------+----------------------------------------------+
Indexes:
+-------+------------+-------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment |
+-------+------------+-------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| rooms | 0 | PRIMARY | 1 | id | A | 2210613 | NULL | NULL | | BTREE | | |
| rooms | 1 | owner | 1 | owner | A | 2210613 | NULL | NULL | | BTREE | | |
| rooms | 1 | roomtype | 1 | roomtype | A | 18 | NULL | NULL | | BTREE | | |
| rooms | 1 | caption | 1 | caption | A | 2210613 | NULL | NULL | | BTREE | | |
| rooms | 1 | category | 1 | category | A | 18 | NULL | NULL | | BTREE | | |
| rooms | 1 | users_now | 1 | users_now | A | 18 | NULL | NULL | | BTREE | | |
| rooms | 1 | tags | 1 | tags | A | 18 | NULL | NULL | | BTREE | | |
| rooms | 1 | users_max | 1 | users_max | A | 18 | NULL | NULL | | BTREE | | |
| rooms | 1 | description | 1 | description | A | 368435 | NULL | NULL | | BTREE | | |
| rooms | 1 | state | 1 | state | A | 18 | NULL | NULL | | BTREE | | |
| rooms | 1 | model_name | 1 | model_name | A | 18 | NULL | NULL | | BTREE | | |
| rooms | 1 | public_ccts | 1 | public_ccts | A | 18 | NULL | NULL | | BTREE | | |
| rooms | 1 | score | 1 | score | A | 18 | NULL | NULL | | BTREE | | |
| rooms | 1 | password | 1 | password | A | 18 | NULL | NULL | | BTREE | | |
| rooms | 1 | icon_bg | 1 | icon_bg | A | 954 | NULL | NULL | | BTREE | | |
| rooms | 1 | icon_fg | 1 | icon_fg | A | 18 | NULL | NULL | | BTREE | | |
| rooms | 1 | badge_id | 1 | badge_id | A | 804 | NULL | NULL | | BTREE | | |
+-------+------------+-------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
+-------------+------------+--------------+--------------+--------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment |
+-------------+------------+--------------+--------------+--------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| room_active | 0 | PRIMARY | 1 | roomid | A | 116 | NULL | NULL | | BTREE | | |
| room_active | 0 | roomid | 1 | roomid | A | 116 | NULL | NULL | | BTREE | | |
| room_active | 1 | active_users | 1 | active_users | A | 29 | NULL | NULL | | BTREE | | |
+-------------+------------+--------------+--------------+--------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
The ORDER BY is removing the advantage of LIMIT 50 because to be able to order by the 50 first elements he has to fetch all the records.
maybe if you tell us what exactly you want, we can help with the query. For example do you really need to do a LEFT JOIN and get all the rooms who don't have a room_active ?
not sure if it helps, but try this one
SELECT rooms.*, room_active.active_users
FROM room_active ra JOIN rooms r ON (room_active.roomid = rooms.id)
WHERE r.caption LIKE '%room%' ORDER BY ra.active_users LIMIT 50
It will ignore the rooms that don't have a room_active, reducing the fetched records
Follow following steps and check performance.
Create index on field 'caption' for its first 4 characters and on 'active_users'.
Instead of "rooms.*", select fields explicitly.
Use "caption LIKE 'room%'" instead of "caption LIKE '%room%'".
If this still slow, you should create a view for your joined query and then perform simple select statement on your view.
1 you might want a index perhaps on the order?
2 You don't need %room% do you? room% would be faster, your effectivly searching all files if you don't
3 don't left join if you can and only want active? instead inner join because it will require less joins (it doesnt join the ones that are null)
4 dont select .* unless needed?
5. Maybe you can use a differnt function such as Contains if you are using oracle
6. Put a index on caption its being mass searched thats probably the problem
Closed. This question is off-topic. It is not currently accepting answers.
Want to improve this question? Update the question so it's on-topic for Stack Overflow.
Closed 10 years ago.
Improve this question
I have a database containing around 5million rows and am having issues with a query taking a long time (over a minute). I was hoping this information would be enough for someone to give me some suggestions, if I need to post more information just let me know.
Thank you in advance for any advice
EXPLAIN SELECT count( * ) AS count
FROM vtiger_time
INNER JOIN vtiger_crmentity ON vtiger_crmentity.crmid = vtiger_time.timeid
INNER JOIN vtiger_crmentityrel ON ( vtiger_crmentityrel.relcrmid = vtiger_crmentity.crmid
OR vtiger_crmentityrel.crmid = vtiger_crmentity.crmid )
LEFT JOIN vtiger_users ON vtiger_users.id = vtiger_crmentity.smownerid
LEFT JOIN vtiger_groups ON vtiger_groups.groupid = vtiger_crmentity.smownerid
WHERE vtiger_crmentity.deleted =0
AND (
vtiger_crmentityrel.crmid =211294
OR vtiger_crmentityrel.relcrmid =211294
)
+----+-------------+---------------------+-------------+-------------------------------------------------+----------------+---------+--------------------------------+-------+------------------------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+---------------------+-------------+-------------------------------------------------+----------------+---------+--------------------------------+-------+------------------------------------------+
| 1 | SIMPLE | vtiger_crmentityrel | index_merge | crmid,relcrmid | crmid,relcrmid | 4,4 | NULL | 5881 | Using union(crmid,relcrmid); Using where |
| 1 | SIMPLE | vtiger_crmentity | ref | PRIMARY,deleted,deleted_2,crmentity_multi_index | deleted_2 | 4 | const | 84424 | Using where; Using index |
| 1 | SIMPLE | vtiger_users | eq_ref | PRIMARY | PRIMARY | 4 | crm.vtiger_crmentity.smownerid | 1 | Using index |
| 1 | SIMPLE | vtiger_groups | eq_ref | PRIMARY | PRIMARY | 4 | crm.vtiger_crmentity.smownerid | 1 | Using index |
| 1 | SIMPLE | vtiger_time | eq_ref | PRIMARY,timeid | PRIMARY | 4 | crm.vtiger_crmentity.crmid | 1 | Using index |
+----+-------------+---------------------+-------------+-------------------------------------------------+----------------+---------+--------------------------------+-------+------------------------------------------+
Additionally these are my currently set indexes
+------------------+------------+----------------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment |
+------------------+------------+----------------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| vtiger_crmentity | 0 | PRIMARY | 1 | crmid | A | 755968 | NULL | NULL | | BTREE | |
| vtiger_crmentity | 0 | crmid | 1 | crmid | A | 755968 | NULL | NULL | | BTREE | |
| vtiger_crmentity | 1 | crmentity_IDX0 | 1 | smcreatorid | A | 15 | NULL | NULL | | BTREE | |
| vtiger_crmentity | 1 | crmentity_IDX1 | 1 | smownerid | A | 15 | NULL | NULL | | BTREE | |
| vtiger_crmentity | 1 | crmentity_IDX2 | 1 | modifiedby | A | 15 | NULL | NULL | | BTREE | |
| vtiger_crmentity | 1 | deleted | 1 | deleted | A | 15 | NULL | NULL | | BTREE | |
| vtiger_crmentity | 1 | deleted | 2 | smownerid | A | 15 | NULL | NULL | | BTREE | |
| vtiger_crmentity | 1 | smownerid | 1 | smownerid | A | 199 | NULL | NULL | | BTREE | |
| vtiger_crmentity | 1 | smownerid | 2 | deleted | A | 199 | NULL | NULL | | BTREE | |
| vtiger_crmentity | 1 | deleted_2 | 1 | deleted | A | 15 | NULL | NULL | | BTREE | |
| vtiger_crmentity | 1 | deleted_2 | 2 | smownerid | A | 15 | NULL | NULL | | BTREE | |
| vtiger_crmentity | 1 | smownerid_2 | 1 | smownerid | A | 385 | NULL | NULL | | BTREE | |
| vtiger_crmentity | 1 | smownerid_2 | 2 | deleted | A | 758 | NULL | NULL | | BTREE | |
| vtiger_crmentity | 1 | crm_ownerid_del_setype_idx | 1 | smownerid | A | 15 | NULL | NULL | | BTREE | |
| vtiger_crmentity | 1 | crm_ownerid_del_setype_idx | 2 | deleted | A | 15 | NULL | NULL | | BTREE | |
| vtiger_crmentity | 1 | crm_ownerid_del_setype_idx | 3 | setype | A | 613 | NULL | NULL | YES | BTREE | |
| vtiger_crmentity | 1 | crmentity_multi_index | 1 | crmid | A | 755968 | NULL | NULL | | BTREE | |
| vtiger_crmentity | 1 | crmentity_multi_index | 2 | smownerid | A | 755968 | NULL | NULL | | BTREE | |
| vtiger_crmentity | 1 | crmentity_multi_index | 3 | deleted | A | 755968 | NULL | NULL | | BTREE | |
+------------------+------------+----------------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
+---------------------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment |
+---------------------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| vtiger_crmentityrel | 1 | crmid | 1 | crmid | A | 223960 | NULL | NULL | | BTREE | |
| vtiger_crmentityrel | 1 | relcrmid | 1 | relcrmid | A | 12442 | NULL | NULL | | BTREE | |
+---------------------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
+--------------+------------+---------------+--------------+---------------+-----------+-------------+----------+--------+------+------------+---------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment |
+--------------+------------+---------------+--------------+---------------+-----------+-------------+----------+--------+------+------------+---------+
| vtiger_users | 0 | PRIMARY | 1 | id | A | 39 | NULL | NULL | | BTREE | |
| vtiger_users | 1 | idx_user_name | 1 | user_name | A | 39 | NULL | NULL | YES | BTREE | |
| vtiger_users | 1 | user_password | 1 | user_password | A | 39 | NULL | NULL | YES | BTREE | |
+--------------+------------+---------------+--------------+---------------+-----------+-------------+----------+--------+------+------------+---------+
+---------------+------------+---------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment |
+---------------+------------+---------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| vtiger_groups | 0 | PRIMARY | 1 | groupid | A | 5 | NULL | NULL | | BTREE | |
| vtiger_groups | 1 | groupname | 1 | groupname | A | 5 | NULL | NULL | YES | BTREE | |
| vtiger_groups | 1 | idx_groups_123group | 1 | groupname | A | 5 | NULL | NULL | YES | BTREE | |
+---------------+------------+---------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
+----------------+------------+-------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment |
+----------------+------------+-------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| vtiger_time | 0 | PRIMARY | 1 | timeid | A | 591772 | NULL | NULL | | BTREE | |
| vtiger_time | 0 | timeid | 1 | timeid | A | 591772 | NULL | NULL | | BTREE | |
| vtiger_time | 1 | relatedto | 1 | relatedto | A | 1405 | NULL | NULL | YES | BTREE | |
| vtiger_time | 1 | date_start | 1 | date_start | A | 7129 | NULL | NULL | YES | BTREE | |
| vtiger_time | 1 | relatedto_2 | 1 | relatedto | A | 3269 | NULL | NULL | YES | BTREE | |
+----------------+------------+-------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
+------------------+------------+-------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment |
+------------------+------------+-------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| vtiger_timecf | 0 | PRIMARY | 1 | timeid | A | 591324 | NULL | NULL | | BTREE | |
| vtiger_timecf | 0 | timeid | 1 | timeid | A | 591324 | NULL | NULL | | BTREE | |
| vtiger_timecf | 1 | timeid_2 | 1 | timeid | A | 591324 | NULL | NULL | | BTREE | |
+------------------+------------+-------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
After adding the following index
ALTER TABLE vtiger_crmentity ADD INDEX TMP_deletion_smownerid_crmid (smownerid, deleted,crmid);
The new explain:
+----+-------------+---------------------+-------------+--------------------------------------------------------------------------+----------------+---------+--------------------------------+-------+------------------------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+---------------------+-------------+--------------------------------------------------------------------------+----------------+---------+--------------------------------+-------+------------------------------------------+
| 1 | SIMPLE | vtiger_crmentityrel | index_merge | crmid,relcrmid | crmid,relcrmid | 4,4 | NULL | 5891 | Using union(crmid,relcrmid); Using where |
| 1 | SIMPLE | vtiger_crmentity | ref | PRIMARY,crmid,deleted,deleted_2,crmentity_multi_index,_deletion_crmid | deleted | 4 | const | 84424 | Using where; Using index |
| 1 | SIMPLE | vtiger_users | eq_ref | PRIMARY | PRIMARY | 4 | crm.vtiger_crmentity.smownerid | 1 | Using index |
| 1 | SIMPLE | vtiger_groups | eq_ref | PRIMARY | PRIMARY | 4 | crm.vtiger_crmentity.smownerid | 1 | Using index |
| 1 | SIMPLE | vtiger_time | eq_ref | PRIMARY,timeid | PRIMARY | 4 | crm.vtiger_crmentity.crmid | 1 | Using index |
+----+-------------+---------------------+-------------+--------------------------------------------------------------------------+----------------+---------+--------------------------------+-------+------------------------------------------+
Rewriting the query would help, but since you cannot do that, you may benefit from a composite index on vtiger_crmentity.crmid and vtiger_crmentity.deleted. But since it isn't using one of the the index you currently have on vtiger_crmentity.crmid, it might not use that new one. If it doesn't, try adding vtiger_crmentity.crmid to the vtiger_crmentity.deleted index. Since it is already using the index on vtiger_crmentity.deleted, this will make the index a covering index so the query will not need to read from the table.
As I mentioned in my comment, there are a lot of duplicate indexes. The won't affect the query for this question, but they will slow down inserts and increase table size. The following indexes are redundant:
vtiger_crmentity.crmid
vtiger_crmentity.smownerid
vtiger_crmentity.smownerid_2
vtiger_crmentity.deleted_2
vtiger_groups.idx_groups_123group
vtiger_time.timeid
vtiger_time.relatedto_2
vtiger_timecf.timeid
vtiger_timcfe.timeid_2
You also shouldn't need the index on the user password column since you shouldn't be looking for users by password.
Try adding this index:
ALTER TABLE vtiger_crmentityrel ADD INDEX ix_crmentityrel_crmid_relcrmid (crmid,relcrmid);
This should allow the first row in the explain to be satisfied completely by the two indexes on the crmentityrel table and should save you a few thousand lookups.
All other parts of the execution plan are using covering indexes so if this doesn't help, I'm not sure there is much more you can do without restructuring the data or the query.
I am trying to speed up a query on a large table with WHERE clauses on two columns, as far as I can, MySQL is only using the ALERT_ID column.
Is there a way to rewrite this query using both indices?
SHOW_INDEX and EXPLAIN output is below.
show index from alert_hit;
+-----------+------------+-------------------+--------------+-------------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment |
+-----------+------------+-------------------+--------------+-------------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| alert_hit | 0 | PRIMARY | 1 | id | A | 15181402 | NULL | NULL | | BTREE | | |
| alert_hit | 1 | alert_id | 1 | alert_id | A | 20 | NULL | NULL | YES | BTREE | | |
| alert_hit | 1 | timestamp | 1 | timestamp | A | 446511 | NULL | NULL | YES | BTREE | | |
| alert_hit | 1 | data_source_id | 1 | data_source_id | A | 20 | NULL | NULL | YES | BTREE | | |
| alert_hit | 1 | filter_syndicated | 1 | filter_syndicated | A | 20 | NULL | NULL | YES | BTREE | | |
| alert_hit | 1 | unique_id | 1 | unique_id | A | 5060467 | NULL | NULL | YES | BTREE | | |
| alert_hit | 1 | date_created | 1 | date_created | A | 281137 | NULL | NULL | | BTREE | | |
| alert_hit | 1 | language | 1 | language | A | 20 | NULL | NULL | YES | BTREE | | |
| alert_hit | 1 | region | 1 | region | A | 42406 | NULL | NULL | YES | BTREE | | |
| alert_hit | 1 | market_rank | 1 | market_rank | A | 20 | NULL | NULL | YES | BTREE | | |
+-----------+------------+-------------------+--------------+-------------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
explain select count(id) as history FROM alert_hit force index(alert_id, timestamp) where alert_id in (9045,9046,9047,9048,9049,9050,9051,9052,9330,9332) AND timestamp between DATE_SUB( NOW(), INTERVAL 1*2 day) and DATE_SUB( NOW(), INTERVAL 1 day);
+----+-------------+-----------+-------+--------------------+----------+---------+------+-------+-------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-----------+-------+--------------------+----------+---------+------+-------+-------------+
| 1 | SIMPLE | alert_hit | range | alert_id,timestamp | alert_id | 5 | NULL | 99578 | Using where |
+----+-------------+-----------+-------+--------------------+----------+---------+------+-------+-------------+
You need to have one index on both fields
ALTER TABLE alert_hit ADD INDEX `IDX-alert_id-timestamp` (`alert_id`, `timestamp`);
Also MySQL will use the multi column index up to the first field for which there is a range condition in the WHERE clause, so in this case order matters and timestamp should be last in the index.
As suggested by #spencer7593 selecting COUNT(1) instead of count(id) might also be better.
I am trying to optimize a MySQL select request:
SELECT * FROM `sales`
WHERE ((sales.private = false AND (sales.buyer_id IS NULL OR NOT sales.buyer_id=142)
AND (sales.merchand_id IS NULL OR NOT sales.merchand_id=142)
AND (sales.private_item = false) )
AND ((sales.buyer_id=32 OR sales.merchand_id=32)
AND (sales.admin=0 AND NOT sales.type IN ('book'))))
ORDER BY sales.created_at DESC, sales.id DESC LIMIT 0, 10;
The schema of the table is
mysql> SHOW columns from sales;
+------------------------+--------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+------------------------+--------------+------+-----+---------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
| type | varchar(255) | YES | MUL | NULL | |
| buyer_id | int(11) | YES | MUL | NULL | |
| merchand_id | int(11) | YES | MUL | NULL | |
| private | tinyint(1) | YES | | 0 | |
| admin | tinyint(1) | YES | | 0 | |
| created_at | datetime | YES | | NULL | |
| updated_at | datetime | YES | | NULL | |
| country_id | int(11) | YES | MUL | 0 | |
| private_item | tinyint(1) | YES | | 0 | |
+------------------------+--------------+------+-----+---------+----------------+
The indexes are:
mysql> show indexes from sales;
+-----------------+------------+--------------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment |
+-----------------+------------+--------------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| sales | 0 | PRIMARY | 1 | id | A | 286509 | NULL | NULL | | BTREE | |
| sales | 1 | index_sales_on_type | 1 | type | A | 123 | NULL | NULL | YES | BTREE | |
| sales | 1 | index_sales_on_buyer_id | 1 | buyer_id | A | 40929 | NULL | NULL | YES | BTREE | |
| sales | 1 | index_sales_on_merchand_id | 1 | merchand_id | A | 40929 | NULL | NULL | YES | BTREE | |
| sales | 1 | index_sales_on_country_id | 1 | country_id | A | 6 | NULL | NULL | YES | BTREE | |
| sales | 1 | index_sales_on_type_and_country_id | 1 | type | A | 151 | NULL | NULL | YES | BTREE | |
| sales | 1 | index_sales_on_type_and_country_id | 2 | country_id | A | 428 | NULL | NULL | YES | BTREE | |
| sales | 1 | index_sales_viewed | 1 | buyer_id | A | 35813 | NULL | NULL | YES | BTREE | |
| sales | 1 | index_sales_viewed | 2 | merchand_id | A | 286509 | NULL | NULL | YES | BTREE | |
| sales | 1 | index_sales_viewed | 3 | private_item| A | 285009 | NULL | NULL | YES | BTREE | |
| sales | 1 | index_sales_viewed | 4 | admin | A | 285009 | NULL | NULL | YES | BTREE | |
| sales | 1 | index_sales_viewed | 5 | type | A | 285009 | NULL | NULL | YES | BTREE | |
| sales | 1 | index_sales_viewed | 6 | private | A | 285009 | NULL | NULL | YES | BTREE | |
| sales | 1 | index_sales_viewed | 7 | created_at | A | 285009 | NULL | NULL | YES | BTREE | |
+-------+------------+------------------------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
When doing the query it uses the index_sales_on_type_and_country_id even if there is no country_id in the query...
The query takes 2.5 seconds with this index.
But when I use USE INDEX(index_sales_viewed) it goes down to 0.2 seconds.
Here is the EXPLAIN of the query:
+----+-------------+-----------------+------+----------------------------------------------+------+---------+------+--------+---------------------
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-----------------+------+----------------------------------------------+------+---------+------+--------+---------------------
| 1 | SIMPLE | sales | range | see bellow | index_sales_on_type_and_country_id | 258 | NULL | 208725 | Using where; Using filesort |
+----+-------------+-----------------+------+----------------------------------------------+------+---------+------+--------+---------------------
the possible keys are :
index_sales_on_type,
index_sales_on_buyer_id,
index_sales_on_merchand_id,
index_sales_on_type_and_country_id,
index_sales_public_recent_activity
Why doesn't MySQL use index_sales_viewed by default? Could there be a better index?
Thank you!
This is wrong use on NULL, please change all the column used in the index to be NOT NULL
refer this When to use NULL in MySQL tables
official documentation
If this column is NULL, there are no relevant indexes. In this case, you may be able to improve the performance of your query by examining the WHERE clause to check whether it refers to some column or columns that would be suitable for indexing. If so, create an appropriate index and check the query with EXPLAIN again
Mysql chosen index index_sales_on_type_and_country_id because you are not compare with NULL value