I have a table that was converted from myISAM to INNODB that is slowing down a query. This is a big table with lot of indexes. MyIsam (on mysql5.6) returns result instantly, INNODB (on mysql5.7) takes 2 to 3 seconds. fnota is float. serieid and epnumber are int. Any idea why this is taking more time when doing index_merge?
Explain on MYISAM TABLE:
explain SELECT count( fnota ) , avg( fnota ) FROM myrates
WHERE serieid =4376 AND epnumber ='149'\G
*************************** 1. row ***************************
id: 1
select_type: SIMPLE
table: myrates
type: ref
possible_keys: serieid,epnumber,serieid_2
key: serieid_2
key_len: 8
ref: const,const
rows: 8207
Extra: Using index condition
Explain on INNODB Table:
explain SELECT count( fnota ) , avg( fnota ) FROM myrates
WHERE serieid =4376 AND epnumber ='149'\G
*************************** 1. row ***************************
id: 1
select_type: SIMPLE
table: myrates
partitions: NULL
type: index_merge
possible_keys: serieid,epnumber,serieid_2
key: serieid,epnumber,serieid_2
key_len: 4,4,8
ref: NULL
rows: 2
filtered: 100.00
Extra: Using intersect(serieid,epnumber,serieid_2); Using where
Indexes:
SHOW INDEX FROM myrates;
+---------+------------+-----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment |
+---------+------------+-----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| myrates | 0 | fbid | 1 | userid | A | 1405506 | NULL | NULL | | BTREE | | |
| myrates | 0 | fbid | 2 | serieid | A | 8617224 | NULL | NULL | | BTREE | | |
| myrates | 0 | fbid | 3 | epnumber | A | 139638192 | NULL | NULL | | BTREE | | |
| myrates | 1 | serieid | 1 | serieid | A | 257656 | NULL | NULL | | BTREE | | |
| myrates | 1 | epnumber | 1 | epnumber | A | 93431 | NULL | NULL | | BTREE | | |
| myrates | 1 | serieid_2 | 1 | serieid | A | 186213 | NULL | NULL | | BTREE | | |
| myrates | 1 | serieid_2 | 2 | epnumber | A | 3309332 | NULL | NULL | | BTREE | | |
| myrates | 1 | userid | 1 | userid | A | 866339 | NULL | NULL | | BTREE | | |
| myrates | 1 | userid | 2 | serieid | A | 4656575 | NULL | NULL | | BTREE | | |
+---------+------------+-----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
SHOW CREATE TABLE MYRATES;
Table: myrates
Create Table: CREATE TABLE `myrates` (
`userid` bigint(10) NOT NULL,
`serieid` int(6) NOT NULL,
`epnumber` float NOT NULL,
`nota` int(6) NOT NULL DEFAULT '0',
`fnota` float NOT NULL,
`timestamp` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
UNIQUE KEY `fbid` (`userid`,`serieid`,`epnumber`),
KEY `serieid` (`serieid`),
KEY `epnumber` (`epnumber`),
KEY `serieid_2` (`serieid`,`epnumber`),
KEY `userid` (`userid`,`serieid`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1
1 row in set (0.00 sec)
DESCRIBE does not precisely spell out the indexes you have. Please provide SHOW CREATE TABLE. It sounds like it is getting confused over your long list of overlapping indexes.
When you have a composite index such as (serieid, epnumber) you don't also need (serieid). Drop the latter index to 'fix' the problem.
It seems that key "userid" is redundant also, given that "fbid" starts with both of its two columns in the same order.
Related
I've got the following mysql query taking approx 55 seconds to complete
SELECT this_.id AS y0_ FROM event this_
INNER JOIN member m1_ ON this_.member_id=m1_.id
INNER JOIN event_type et2_ ON this_.type_id=et2_.id
WHERE m1_.submission_id=40646 AND et2_.name IN ('Salary')
ORDER BY m1_.ni_number ASC, m1_.ident1 ASC, m1_.ident2 ASC, m1_.ident3 ASC, m1_.id ASC, et2_.name ASC LIMIT 15;
If I remove the join/where/order to the 'event_type' table, then the query runs in under 1 second.
So something clearly up with my join to the 'event_type' table, but a similar query in another database with similar database volumes runs absolutely fine. So my suspicion is something wrong with this 1 database.
The 'show create table' of the 'event' table is:
Create Table: CREATE TABLE `event` (
`id` bigint(20) NOT NULL AUTO_INCREMENT,
`version` bigint(20) NOT NULL,
`data_size` bigint(20) DEFAULT NULL,
`encoded_data` mediumblob,
`last_updated` datetime NOT NULL,
`member_id` bigint(20) NOT NULL,
`parent_event_id` bigint(20) DEFAULT NULL,
`status` varchar(255) DEFAULT NULL,
`type_id` bigint(20) NOT NULL,
`updated_by` varchar(255) NOT NULL,
`failed_workflow_case` varchar(255) DEFAULT NULL,
`failed_workflow_task` varchar(255) DEFAULT NULL,
PRIMARY KEY (`id`),
KEY `FK5C6729A2434DA80` (`member_id`),
KEY `FK5C6729AE4E22C6E` (`type_id`),
KEY `IND_parent_event_id` (`parent_event_id`),
CONSTRAINT `FK5C6729A2434DA80` FOREIGN KEY (`member_id`) REFERENCES `member` (`id`) ON DELETE NO ACTION ON UPDATE NO ACTION,
CONSTRAINT `FK5C6729AE4E22C6E` FOREIGN KEY (`type_id`) REFERENCES `event_type` (`id`) ON DELETE NO ACTION ON UPDATE NO ACTION
) ENGINE=InnoDB AUTO_INCREMENT=46241198 DEFAULT CHARSET=latin1
The EXPLAIN of the query is:
+----+-------------+-------+------------+--------+-------------------------------------+-------------------+---------+--------------------------+------+----------+----------------------------------------------+
| id | select_type | table | partitions | type | possible_keys | key | key_len | ref | rows | filtered | Extra |
+----+-------------+-------+------------+--------+-------------------------------------+-------------------+---------+--------------------------+------+----------+----------------------------------------------+
| 1 | SIMPLE | et2_ | NULL | ref | PRIMARY,IND_name | IND_name | 257 | const | 1 | 100.00 | Using index; Using temporary; Using filesort |
| 1 | SIMPLE | this_ | NULL | ref | FK5C6729A2434DA80,FK5C6729AE4E22C6E | FK5C6729AE4E22C6E | 8 | iconnect.et2_.id | 3303 | 100.00 | NULL |
| 1 | SIMPLE | m1_ | NULL | eq_ref | PRIMARY,IND_submission_id | PRIMARY | 8 | iconnect.this_.member_id | 1 | 5.00 | Using where |
+----+-------------+-------+------------+--------+-------------------------------------+-------------------+---------+--------------------------+------+----------+----------------------------------------------+
The indexes from the 'event' table are:
+-------+------------+---------------------+--------------+-----------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment |
+-------+------------+---------------------+--------------+-----------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| event | 0 | PRIMARY | 1 | id | A | 14307622 | NULL | NULL | | BTREE | | |
| event | 1 | FK5C6729A2434DA80 | 1 | member_id | A | 4680601 | NULL | NULL | | BTREE | | |
| event | 1 | FK5C6729AE4E22C6E | 1 | type_id | A | 4360 | NULL | NULL | | BTREE | | |
| event | 1 | IND_parent_event_id | 1 | parent_event_id | A | 114404 | NULL | NULL | YES | BTREE | | |
+-------+------------+---------------------+--------------+-----------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
What jumps out at me from this is:
1. why is the EXPLAIN going through 3303 rows and the FK5C6729AE4E22C6E index?
2. why does the FK5C6729AE4E22C6E index have a cardinality of 4360 when there are only 17 rows in the 'event_type' table? could this incorrect cardinality be affecting the query optimizer?
I've done an ANALYZE TABLE on both 'event' and 'event_type' and this has made no difference.
Any suggestions?
execute plan from other server with the same data (loaded from a dump file):
+----+-------------+-------+------------+------+-------------------------------------+-------------------+---------+-----------------+-------+----------+----------------------------------------------+
| id | select_type | table | partitions | type | possible_keys | key | key_len | ref | rows | filtered | Extra |
+----+-------------+-------+------------+------+-------------------------------------+-------------------+---------+-----------------+-------+----------+----------------------------------------------+
| 1 | SIMPLE | et2_ | NULL | ALL | PRIMARY | NULL | NULL | NULL | 17 | 10.00 | Using where; Using temporary; Using filesort |
| 1 | SIMPLE | m1_ | NULL | ref | PRIMARY,IND_submission_id | IND_submission_id | 8 | const | 27992 | 100.00 | NULL |
| 1 | SIMPLE | this_ | NULL | ref | FK5C6729A2434DA80,FK5C6729AE4E22C6E | FK5C6729A2434DA80 | 8 | iconnect.m1_.id | 3 | 11.11 | Using where |
+----+-------------+-------+------------+------+-------------------------------------+-------------------+---------+-----------------+-------+----------+----------------------------------------------+
Increasing innodb_stats_persistent_sample_pages from 20 to 100, then running ANALYZE TABLE on event/member tables changed the cardinality of the indexes and the execution plan, then the query ran in under 1 second. Thanks to Solarflare for the suggestion.
I'm new with mysql and am a little confused about what cardinality means, I read that it means the number or unique rows but I'd like to know what it does mean in this case, this is my table definition
+-------------------+--------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-------------------+--------------+------+-----+---------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
| revisado | varchar(10) | YES | MUL | NULL | |
| total | int(11) | NO | MUL | NULL | |
| busqueda | varchar(300) | NO | MUL | NULL | |
| clave | bigint(15) | NO | | NULL | |
| producto_servicio | varchar(300) | NO | | NULL | |
+-------------------+--------------+------+-----+---------+----------------+
the total of records right now is 13621
I have this query
SELECT clave, producto_servicio FROM buscador_claves2 WHERE busqueda = 'FERRETERIA' AND total = 2 AND revisado = 'APROBADO'
And this the index definition of the table
+------------------+------------+----------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment |
+------------------+------------+----------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| buscador_claves2 | 0 | PRIMARY | 1 | id | A | 14309 | NULL | NULL | | BTREE | |
| buscador_claves2 | 1 | idx_busqueda | 1 | busqueda | A | 14309 | 255 | NULL | | BTREE | |
| buscador_claves2 | 1 | idx_total | 1 | total | A | 3 | NULL | NULL | | BTREE | |
| buscador_claves2 | 1 | idx_revisado | 1 | revisado | A | 1 | NULL | NULL | YES | BTREE | |
| buscador_claves2 | 1 | idx_compuesto1 | 1 | revisado | A | 1 | NULL | NULL | YES | BTREE | |
| buscador_claves2 | 1 | idx_compuesto1 | 2 | total | A | 105 | NULL | NULL | | BTREE | |
| buscador_claves2 | 1 | idx_compuesto1 | 3 | busqueda | A | 14309 | 255 | NULL | | BTREE | |
| buscador_claves2 | 1 | idx_compuesto2 | 1 | busqueda | A | 14309 | 255 | NULL | | BTREE | |
| buscador_claves2 | 1 | idx_compuesto2 | 2 | total | A | 14309 | NULL | NULL | | BTREE | |
| buscador_claves2 | 1 | idx_compuesto2 | 3 | revisado | A | 14309 | NULL | NULL | YES | BTREE | |
+------------------+------------+----------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
the query is taking idx_compuesto1 as the index to find the data, what means the cardinality in this case for the revisado, total and busqueda columns as part of the index idx_compuesto1? and why it takes idx_compuesto1 instead of idx_compuesto2, I can see the cardinality is different in both indexes
This is the output of the query explain
*************************** 1. row ***************************
id: 1
select_type: SIMPLE
table: buscador_claves2
type: ref
possible_keys: idx_busqueda,idx_total,idx_revisado,idx_compuesto1,idx_compuesto2
key: idx_compuesto1
key_len: 804
ref: const,const,const
rows: 1
Extra: Using where
I hope you can help me to understand better this info, thank you.
In MySQL, the value of the index cardinality column is the storage engine estimate for the number of unique values in that index. It is used to determine how well this index can be used during joins. Generally MySQL optimizer prefers the index with a higher cardinality, because it usually means it is able to filter down to fewer rows. The ideal scenario is for the value of cardinality to be always equal to SELECT COUNT(DISTINCT the_key)..., but in practice it is usually off by some relatively small margin due to the difficulty of accurately computing this during normal database operations in an efficient manner that does not disrupt database performance. The value will be more accurate immediately after ANALYZE TABLE. Being off on cardinality begins to matter when the optimizer can choose more than one key for a particular join, it makes a huge difference in performance which one gets chosen, and the cardinality estimates for those keys are sufficiently off to cause the optimizer to choose the wrong key. Those situations are relatively rare, but do happen. In that case, the problem can be solved either with ANALYZE TABLE or - if you are always 100% sure which key is better for the join - by explicitly making the optimizer use it with FORCE KEY in the query.
As part of the upgrade process Alfresco preforms this query
INSERT INTO ACT_HI_VARINST(
ID_,
PROC_INST_ID_,
EXECUTION_ID_,
TASK_ID_,
NAME_,
VAR_TYPE_,
REV_,
BYTEARRAY_ID_,
DOUBLE_,
LONG_,
TEXT_,
TEXT2_
)
SELECT
(#cnt := #cnt + 1),
PROC_INST_ID_,
EXECUTION_ID_,
TASK_ID_,
NAME_,
VAR_TYPE_,
REV_,
BYTEARRAY_ID_,
DOUBLE_,
LONG_,
TEXT_,
TEXT2_
FROM ACT_HI_DETAIL AHD
CROSS JOIN (SELECT #cnt := 177401 + 1) AS dummy
WHERE AHD.PROC_INST_ID_ not in (select PROC_INST_ID_ from ACT_HI_VARINST)
AND
(AHD.PROC_INST_ID_ , AHD.NAME_, AHD.REV_, AHD.time_) IN
(SELECT PROC_INST_ID_, NAME_, MAX(REV_), MAX(time_)
FROM ACT_HI_DETAIL
GROUP BY PROC_INST_ID_ , NAME_);
It is taking over 12 hours to run.
Using explain on the query
explain SELECT
(#cnt := #cnt + 1),
PROC_INST_ID_,
EXECUTION_ID_,
TASK_ID_,
NAME_,
VAR_TYPE_,
REV_,
BYTEARRAY_ID_,
DOUBLE_,
LONG_,
TEXT_,
TEXT2_
FROM ACT_HI_DETAIL AHD
CROSS JOIN (SELECT #cnt := 177401 + 1) AS dummy
WHERE AHD.PROC_INST_ID_ not in (select PROC_INST_ID_ from ACT_HI_VARINST)
AND
(AHD.PROC_INST_ID_ , AHD.NAME_, AHD.REV_, AHD.time_) IN
(SELECT PROC_INST_ID_, NAME_, MAX(REV_), MAX(time_)
FROM ACT_HI_DETAIL
GROUP BY PROC_INST_ID_ , NAME_)\G
results in
*************************** 1. row ***************************
id: 1
select_type: PRIMARY
table: <derived2>
type: system
possible_keys: NULL
key: NULL
key_len: NULL
ref: NULL
rows: 1
Extra:
*************************** 2. row ***************************
id: 1
select_type: PRIMARY
table: AHD
type: ALL
possible_keys: ACT_IDX_HI_DETAIL_PROC_INST,ACT_IDX_HI_DETAIL_TIME,ACT_IDX_HI_DETAIL_NAME
key: NULL
key_len: NULL
ref: NULL
rows: 70669
Extra: Using where
*************************** 3. row ***************************
id: 1
select_type: PRIMARY
table: <subquery4>
type: eq_ref
possible_keys: distinct_key
key: distinct_key
key_len: 976
ref: alfresco.AHD.PROC_INST_ID_,alfresco.AHD.NAME_,alfresco.AHD.REV_,alfresco.AHD.TIME_
rows: 1
Extra:
*************************** 4. row ***************************
id: 4
select_type: MATERIALIZED
table: ACT_HI_DETAIL
type: ALL
possible_keys: NULL
key: NULL
key_len: NULL
ref: NULL
rows: 70669
Extra: Using temporary
*************************** 5. row ***************************
id: 3
select_type: MATERIALIZED
table: ACT_HI_VARINST
type: index
possible_keys: ACT_IDX_HI_PROCVAR_PROC_INST
key: ACT_IDX_HI_PROCVAR_PROC_INST
key_len: 197
ref: NULL
rows: 41504
Extra: Using index
*************************** 6. row ***************************
id: 2
select_type: DERIVED
table: NULL
type: NULL
possible_keys: NULL
key: NULL
key_len: NULL
ref: NULL
rows: NULL
Extra: No tables used
Adding an extra index
ALTER TABLE ACT_HI_DETAIL ADD INDEX `ACT_HI_DETAIL_MULTI` (PROC_INST_ID_, TIME_, NAME_);
resulted in halving the number of rows here
*************************** 2. row ***************************
id: 1
select_type: PRIMARY
table: AHD
type: range
possible_keys: act_hi_detail_multi_1
key: act_hi_detail_multi_1
key_len: 195
ref: NULL
rows: 35761
Extra: Using index condition; Using where
The database is using innodb.
Setting innodb_buffer_pool_size up to 4G made no difference.
The table definitions:
MariaDB [alfresco]> desc act_hi_detail;
+---------------+---------------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+---------------+---------------+------+-----+---------+-------+
| ID_ | varchar(64) | NO | PRI | NULL | |
| TYPE_ | varchar(255) | NO | | NULL | |
| PROC_INST_ID_ | varchar(64) | YES | MUL | NULL | |
| EXECUTION_ID_ | varchar(64) | YES | | NULL | |
| TASK_ID_ | varchar(64) | YES | MUL | NULL | |
| ACT_INST_ID_ | varchar(64) | YES | MUL | NULL | |
| NAME_ | varchar(255) | NO | MUL | NULL | |
| VAR_TYPE_ | varchar(255) | YES | | NULL | |
| REV_ | int(11) | YES | | NULL | |
| TIME_ | datetime | NO | MUL | NULL | |
| BYTEARRAY_ID_ | varchar(64) | YES | | NULL | |
| DOUBLE_ | double | YES | | NULL | |
| LONG_ | bigint(20) | YES | | NULL | |
| TEXT_ | varchar(4000) | YES | | NULL | |
| TEXT2_ | varchar(4000) | YES | | NULL | |
+---------------+---------------+------+-----+---------+-------+
15 rows in set (0.00 sec)
MariaDB [alfresco]> desc act_hi_varinst;
+---------------+---------------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+---------------+---------------+------+-----+---------+-------+
| ID_ | varchar(64) | NO | PRI | NULL | |
| PROC_INST_ID_ | varchar(64) | YES | MUL | NULL | |
| EXECUTION_ID_ | varchar(64) | YES | | NULL | |
| TASK_ID_ | varchar(64) | YES | | NULL | |
| NAME_ | varchar(255) | NO | MUL | NULL | |
| VAR_TYPE_ | varchar(100) | YES | | NULL | |
| REV_ | int(11) | YES | | NULL | |
| BYTEARRAY_ID_ | varchar(64) | YES | | NULL | |
| DOUBLE_ | double | YES | | NULL | |
| LONG_ | bigint(20) | YES | | NULL | |
| TEXT_ | varchar(4000) | YES | | NULL | |
| TEXT2_ | varchar(4000) | YES | | NULL | |
+---------------+---------------+------+-----+---------+-------+
12 rows in set (0.00 sec)
The current indexes are
MariaDB [alfresco]> show indexes from act_hi_detail;
+---------------+------------+-----------------------------+--------------+---------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment |
+---------------+------------+-----------------------------+--------------+---------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| act_hi_detail | 0 | PRIMARY | 1 | ID_ | A | 71244 | NULL | NULL | | BTREE | | |
| act_hi_detail | 1 | ACT_IDX_HI_DETAIL_PROC_INST | 1 | PROC_INST_ID_ | A | 3238 | NULL | NULL | YES | BTREE | | |
| act_hi_detail | 1 | ACT_IDX_HI_DETAIL_ACT_INST | 1 | ACT_INST_ID_ | A | 5937 | NULL | NULL | YES | BTREE | | |
| act_hi_detail | 1 | ACT_IDX_HI_DETAIL_TIME | 1 | TIME_ | A | 8905 | NULL | NULL | | BTREE | | |
| act_hi_detail | 1 | ACT_IDX_HI_DETAIL_NAME | 1 | NAME_ | A | 147 | NULL | NULL | | BTREE | | |
| act_hi_detail | 1 | ACT_IDX_HI_DETAIL_TASK_ID | 1 | TASK_ID_ | A | 5480 | NULL | NULL | YES | BTREE | | |
| act_hi_detail | 1 | act_hi_detail_multi_1 | 1 | PROC_INST_ID_ | A | 199 | NULL | NULL | YES | BTREE | | |
| act_hi_detail | 1 | act_hi_detail_multi_1 | 2 | TIME_ | A | 199 | NULL | NULL | | BTREE | | |
| act_hi_detail | 1 | act_hi_detail_multi_1 | 3 | NAME_ | A | 199 | NULL | NULL | | BTREE | | |
+---------------+------------+-----------------------------+--------------+---------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
9 rows in set (0.01 sec)
MariaDB [alfresco]> show indexes from act_hi_varinst;
+----------------+------------+------------------------------+--------------+---------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment |
+----------------+------------+------------------------------+--------------+---------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| act_hi_varinst | 0 | PRIMARY | 1 | ID_ | A | 42358 | NULL | NULL | | BTREE | | |
| act_hi_varinst | 1 | ACT_IDX_HI_PROCVAR_PROC_INST | 1 | PROC_INST_ID_ | A | 1925 | NULL | NULL | YES | BTREE | | |
| act_hi_varinst | 1 | ACT_IDX_HI_PROCVAR_NAME_TYPE | 1 | NAME_ | A | 184 | NULL | NULL | | BTREE | | |
| act_hi_varinst | 1 | ACT_IDX_HI_PROCVAR_NAME_TYPE | 2 | VAR_TYPE_ | A | 219 | NULL | NULL | YES | BTREE | | |
+----------------+------------+------------------------------+--------------+---------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
4 rows in set (0.01 sec)
Are there more indexes that can be added or modifications than can be made to the query to greatly improve the speed?
I have a table of over 9 million rows. I have a SELECT query that I'm using an index for. Here is the query:
SELECT `username`,`id`
FROM `04c1Tg0M`
WHERE `id` > 9259466
AND `tried` = 0
LIMIT 1;
That query executes very fast (0.00 sec). Here is the explain for that query:
+----+-------------+----------+-------+-----------------+---------+---------+------+-------+-------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+----------+-------+-----------------+---------+---------+------+-------+-------------+
| 1 | SIMPLE | 04c1Tg0M | range | PRIMARY,triedex | PRIMARY | 4 | NULL | 10822 | Using where |
+----+-------------+----------+-------+-----------------+---------+---------+------+-------+-------------+
Now here is the same query except that I'm going to change the id to 6259466:
SELECT `username`,`id`
FROM `04c1Tg0M`
WHERE `id` > 5986551
AND `tried` = 0
LIMIT 1;
That query took 4.78 seconds to complete. This is the problem. Here is the explain for that query:
+----+-------------+----------+------+-----------------+---------+---------+-------+---------+-------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+----------+------+-----------------+---------+---------+-------+---------+-------------+
| 1 | SIMPLE | 04c1Tg0M | ref | PRIMARY,triedex | triedex | 2 | const | 9275107 | Using where |
+----+-------------+----------+------+-----------------+---------+---------+-------+---------+-------------+
What is happening here and how can I fix it? Here are my indexes:
+----------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment |
+----------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| 04c1Tg0M | 0 | PRIMARY | 1 | id | A | 9275093 | NULL | NULL | | BTREE | |
| 04c1Tg0M | 1 | pdex | 1 | username | A | 9275093 | NULL | NULL | | BTREE | |
| 04c1Tg0M | 1 | pdex | 2 | id | A | 9275093 | NULL | NULL | | BTREE | |
| 04c1Tg0M | 1 | pdex | 3 | tried | A | 9275093 | NULL | NULL | YES | BTREE | |
| 04c1Tg0M | 1 | triedex | 1 | tried | A | 0 | NULL | NULL | YES | BTREE | |
| 04c1Tg0M | 1 | triedex | 2 | id | A | 9275093 | NULL | NULL | | BTREE | |
+----------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
And here is my table structure:
| 04c1Tg0M | CREATE TABLE `04c1Tg0M` (
`id` int(20) NOT NULL AUTO_INCREMENT,
`username` varchar(50) NOT NULL,
`tried` tinyint(1) DEFAULT '0',
PRIMARY KEY (`id`),
KEY `pdex` (`username`,`id`,`tried`),
KEY `triedex` (`tried`,`id`)
) ENGINE=MyISAM AUTO_INCREMENT=9275108 DEFAULT CHARSET=utf8 |
The first SQL returns 10822 rows, while the second one returns 9275107 rows!
The use of primary key "id" index in the second query isn't so useful because you have to do a full table scan anyway.
MySQL's cost-based optimizer thinks, in the case of the 2nd query, it's better off to use the index on 'tried'.
If you have to do a full table-scan, you're better off not using an index, as index constitutes additional disk reads.
You can use "use index" or "force index" in your query to hint to the optimizer whether to use an index.
Also update the statistics by analyzing your table periodically so the cost-based optimizer is working correctly.
Number of records in contacts: 245847
Number of records in teams sets teams: 348
mysql> explain SELECT distinct contacts.id FROM contacts INNER JOIN
team_sets_teams tst ON tst.team_set_id = contacts.team_set_id where
( contacts.phone_work LIKE '%487%1864%' or contacts.phone_mobile LIKE
'%487%1864%' or contacts.phone_other LIKE '%487%1864%' or
contacts.phone_home LIKE '%487%1864%' ) ORDER BY
contacts.last_name LIMIT 0,11 \G;
id: 1
select_type: SIMPLE
table: contacts
type: ALL
possible_keys: team_set_id
key: NULL
key_len: NULL
ref: NULL
rows: 245628
Extra: Using where; Using temporary; Using filesort
*************************** 2. row ***************************
id: 1
select_type: SIMPLE
table: tst
type: ref
possible_keys: idx_ud_set_id
key: idx_ud_set_id
key_len: 109
ref: db.contacts.team_set_id
rows: 2
Extra: Using where; Using index; Distinct
2 rows in set (0.00 sec)
Now without distinct:
mysql> explain SELECT distinct contacts.id FROM contacts INNER JOIN
team_sets_teams tst ON tst.team_set_id = contacts.team_set_id where
( contacts.phone_work LIKE '%487%1864%' or contacts.phone_mobile LIKE
'%487%1864%' or contacts.phone_other LIKE '%487%1864%' or
contacts.phone_home LIKE '%487%1864%' ) ORDER BY
contacts.last_name LIMIT 0,11 \G;
id: 1
select_type: SIMPLE
table: contacts
type: index
possible_keys: team_set_id
key: last_name
key_len: 303
ref: NULL
rows: 2
Extra: Using where
*************************** 2. row ***************************
id: 1
select_type: SIMPLE
table: tst
type: ref
possible_keys: idx_ud_set_id
key: idx_ud_set_id
key_len: 109
ref: karma.contacts.team_set_id
rows: 2
Extra: Using where; Using index
2 rows in set (0.00 sec)
What fields would you use as index. The query can not be changed as it's a third party application:
Teams Sets
mysql> show fields from team_sets_teams;
+---------------+------------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+---------------+------------+------+-----+---------+-------+
| id | char(36) | NO | PRI | | |
| team_set_id | char(36) | YES | MUL | NULL | |
| team_id | char(36) | YES | MUL | NULL | |
| date_modified | datetime | YES | | NULL | |
| deleted | tinyint(1) | YES | MUL | 0 | |
+---------------+------------+------+-----+---------+-------+
5 rows in set (0.00 sec)
Contacts
mysql> show fields from contacts;
+----------------------------+--------------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+----------------------------+--------------+------+-----+---------+-------+
| id | char(36) | NO | PRI | NULL | |
| date_entered | datetime | YES | MUL | NULL | |
| date_modified | datetime | YES | MUL | NULL | |
| modified_user_id | char(36) | YES | MUL | NULL | |
| created_by | char(36) | YES | MUL | NULL | |
| description | text | YES | | NULL | |
| deleted | tinyint(1) | YES | | 0 | |
| assigned_user_id | char(36) | YES | MUL | NULL | |
| team_id | char(36) | YES | MUL | NULL | |
| team_set_id | char(36) | YES | MUL | NULL | |
| salutation | varchar(100) | YES | | NULL | |
| first_name | varchar(100) | YES | MUL | NULL | |
| last_name | varchar(100) | YES | MUL | NULL | |
| title | varchar(100) | YES | | NULL | |
| department | varchar(255) | YES | | NULL | |
| do_not_call | tinyint(1) | YES | | 0 | |
| phone_home | varchar(100) | YES | MUL | NULL | |
| phone_mobile | varchar(100) | YES | MUL | NULL | |
| phone_work | varchar(100) | YES | MUL | NULL | |
| phone_other | varchar(100) | YES | MUL | NULL | |
| phone_fax | varchar(100) | YES | MUL | NULL | |
| assistant | varchar(75) | YES | MUL | NULL | |
+----------------------------+--------------+------+-----+---------+-------+