I running MySQL 5.1.30 on Solaris 10.
As I have a table which I partition into 12 according to each month.
The table structure as follow
mysql> desc my_events;
+-----------+--------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-----------+--------------+------+-----+---------+----------------+
| event_id | bigint(20) | NO | PRI | NULL | auto_increment |
| timestamp | timestamp | NO | PRI | NULL | |
| object | varchar(10) | YES | | NULL | |
| info | text | YES | | NULL | |
+-----------+--------------+------+-----+---------+----------------+
4 rows in set (0.00 sec)
mysql> SELECT PARTITION_NAME, TABLE_ROWS, PARTITION_EXPRESSION, PARTITION_DESCRIPTION FROM INFORMATION_SCHEMA.PARTITIONS WHERE TABLE_SCHEMA = 'test11' AND PARTITION_METHOD = 'RANGE';
+----------------+------------+----------------------+-----------------------+
| PARTITION_NAME | TABLE_ROWS | PARTITION_EXPRESSION | PARTITION_DESCRIPTION |
+----------------+------------+----------------------+-----------------------+
| p001 | 0 | to_days(timestamp) | 733773 |
| p002 | 0 | to_days(timestamp) | 733804 |
| p003 | 20863 | to_days(timestamp) | 733832 |
| p004 | 269336 | to_days(timestamp) | 733863 |
| p005 | 3094672 | to_days(timestamp) | 733893 |
| p006 | 2639348 | to_days(timestamp) | 733924 |
| p007 | 314010 | to_days(timestamp) | 733954 |
| p008 | 0 | to_days(timestamp) | 733985 |
| p009 | 0 | to_days(timestamp) | 734016 |
| p010 | 0 | to_days(timestamp) | 734046 |
| p011 | 0 | to_days(timestamp) | 734077 |
| p012 | 0 | to_days(timestamp) | 734107 |
+----------------+------------+----------------------+-----------------------+
12 rows in set (0.05 sec)
When I want to query for particular range of days.
mysql> DESCRIBE PARTITIONS SELECT * FROM my_events where timestamp > '2009-03-01' and timestamp < '2009-03-30';
+----+-------------+-------------------+-------------------------------------------------------------+------+---------------+------+---------+------+---------+-------------+
| id | select_type | table | partitions | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------------------+-------------------------------------------------------------+------+---------------+------+---------+------+---------+-------------+
| 1 | SIMPLE | my_events | p001,p002,p003,p004,p005,p006,p007,p008,p009,p010,p011,p012 | ALL | NULL | NULL | NULL | NULL | 6338229 | Using where |
+----+-------------+-------------------+-------------------------------------------------------------+------+---------------+------+---------+------+---------+-------------+
1 row in set (0.01 sec)
It query all the partitions when I use type timestamp for timestamp.
However when I change the type to datetime for timestamp, it query only 1 partition (p004).
mysql> DESCRIBE PARTITIONS SELECT * FROM my_events where timestamp > '2009-03-01' and timestamp < '2009-03-30';
+----+-------------+-------------------+------------+------+---------------+------+---------+------+---------+-------------+
| id | select_type | table | partitions | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------------------+------------+------+---------------+------+---------+------+---------+-------------+
| 1 | SIMPLE | my_events | p004 | ALL | NULL | NULL | NULL | NULL | 6338229 | Using where |
+----+-------------+-------------------+------------+------+---------------+------+---------+------+---------+-------------+
1 row in set (0.00 sec)
How can I query only 1 partition but with using type timestamp for timestamp?
I've just been researching this and the short answer is that you can't according to http://bugs.mysql.com/bug.php?id=24245
"pruning is not expected to work for tables partitioned on a
TIMESTAMP column, and you should use a DATETIME or DATE column for this instead."
this is a good article http://dev.mysql.com/tech-resources/articles/testing-partitions-large-db.html
personally i'd take an innodb clustered composite PK index over a partition until my data storage requirements got silly - really silly
Related
I need your help optimizing a query. One table is a log table which has millions of entries and I try to break my query to < 1s. My query should give an overall overview and should be quick therefore. I'm sure I could make multiple simple queries over the list with help of a script. But what a script can do, can mysql I think - I hope at least. And maybe not all parts are best used, but I'm stuck in a query which makes a temporary table and filesort (which I found out is really bad). As reading around I found out to use some neat and well placed indexes but now I'm stuck in a specific point.
Let me show you my final query with its results:
SELECT
ps.SERVER_ID,
ps.FULLNAME,
SUM(CASE WHEN pml.ID_TYPE = 3 THEN 1 ELSE 0 END) 'amount_warning',
SUM(CASE WHEN pml.ID_TYPE = 4 THEN 1 ELSE 0 END) 'amount_error',
SUM(CASE WHEN pml.ID_TYPE = 5 THEN 1 ELSE 0 END) 'amount_alert',
SUM(CASE WHEN pml.ID_TYPE = 7 THEN 1 ELSE 0 END) 'amount_critical'
FROM
PAR_SERVER ps
INNER JOIN
PAR_MONITORINGv2_LOG pml ON ps.SERVER_ID = pml.SERVER_ID
WHERE
pml.CREATED_DATE > date_sub( NOW( ) , INTERVAL 7 DAY )
GROUP BY
ps.SERVER_ID;
Here is what I get:
mysql> [thequeryabove]
[...]
59 rows in set (11.69 sec)
mysql> explain [thequeryabove]
+----+-------------+-------+--------+-----------------------------+---------+---------+---------------------------+---------+----------------------------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------+--------+-----------------------------+---------+---------+---------------------------+---------+----------------------------------------------+
| 1 | SIMPLE | pml | ALL | SERVER_ID,SERVER_ID-ID_TYPE | NULL | NULL | NULL | 4014447 | Using where; Using temporary; Using filesort |
| 1 | SIMPLE | ps | eq_ref | PRIMARY | PRIMARY | 4 | database.pml.SERVER_ID | 1 | |
+----+-------------+-------+--------+-----------------------------+---------+---------+---------------------------+---------+----------------------------------------------+
2 rows in set (0.00 sec)
Here's my current table setup:
mysql> describe PAR_SERVER;
+----------------+--------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+----------------+--------------+------+-----+---------+----------------+
| SERVER_ID | int(255) | NO | PRI | NULL | auto_increment |
| FULLNAME | varchar(255) | YES | | NULL | |
| SHORTNAME | varchar(255) | YES | MUL | NULL | |
+----------------+--------------+------+-----+---------+----------------+
3 rows in set (0.00 sec)
mysql> show indexes from PAR_SERVER;
+------------+------------+-----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment |
+------------+------------+-----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| PAR_SERVER | 0 | PRIMARY | 1 | SERVER_ID | A | 142 | NULL | NULL | | BTREE | |
| PAR_SERVER | 1 | shortname | 1 | SHORTNAME | A | 142 | NULL | NULL | YES | BTREE | |
+------------+------------+-----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
2 rows in set (0.00 sec)
mysql> select count(*) from PAR_SERVER;
+----------+
| count(*) |
+----------+
| 142 |
+----------+
1 row in set (0.00 sec)
mysql> describe PAR_MONITORINGv2_LOG;
+--------------+----------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+--------------+----------+------+-----+---------+----------------+
| ID | int(11) | NO | PRI | NULL | auto_increment |
| ID_TYPE | int(11) | NO | MUL | NULL | |
| ID_SERVICE | int(11) | NO | MUL | NULL | |
| SERVER_ID | int(11) | NO | MUL | NULL | |
| MESSAGE | tinytext | NO | | NULL | |
| CREATED_DATE | datetime | NO | | NULL | |
+--------------+----------+------+-----+---------+----------------+
6 rows in set (0.00 sec)
mysql> show indexes from PAR_MONITORINGv2_LOG;
+----------------------+------------+-------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment |
+----------------------+------------+-------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| PAR_MONITORINGv2_LOG | 0 | PRIMARY | 1 | ID | A | 3998188 | NULL | NULL | | BTREE | |
| PAR_MONITORINGv2_LOG | 1 | ID_TYPE | 1 | ID_TYPE | A | 7 | NULL | NULL | | BTREE | |
| PAR_MONITORINGv2_LOG | 1 | ID_SERVICE | 1 | ID_SERVICE | A | 5 | NULL | NULL | | BTREE | |
| PAR_MONITORINGv2_LOG | 1 | SERVER_ID | 1 | SERVER_ID | A | 66 | NULL | NULL | | BTREE | |
| PAR_MONITORINGv2_LOG | 1 | SERVER_ID-ID_TYPE | 1 | SERVER_ID | A | 66 | NULL | NULL | | BTREE | |
| PAR_MONITORINGv2_LOG | 1 | SERVER_ID-ID_TYPE | 2 | ID_TYPE | A | 258 | NULL | NULL | | BTREE | |
+----------------------+------------+-------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
6 rows in set (0.00 sec)
mysql> select count(*) from PAR_MONITORINGv2_LOG;
+----------+
| count(*) |
+----------+
| 3998386 |
+----------+
1 row in set (0.00 sec)
Here are time results breaking my query step by step down. I may going step by step up after fixing each part taking so long. But for now only the query with runtime of 2.30 sec is currently interesting for this question.
mysql> SELECT ps.SERVER_ID, ps.FULLNAME FROM PAR_SERVER ps INNER JOIN PAR_MONITORINGv2_LOG pml ON ps.SERVER_ID = pml.SERVER_ID WHERE pml.CREATED_DATE > date_sub( NOW( ) , INTERVAL 7 DAY ) GROUP BY ps.SERVER_ID;
[...]
59 rows in set (6.41 sec)
mysql> explain [thequeryabove]
+----+-------------+-------+--------+-----------------------------+---------+---------+---------------------------+---------+----------------------------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------+--------+-----------------------------+---------+---------+---------------------------+---------+----------------------------------------------+
| 1 | SIMPLE | pml | ALL | SERVER_ID,SERVER_ID-ID_TYPE | NULL | NULL | NULL | 4014788 | Using where; Using temporary; Using filesort |
| 1 | SIMPLE | ps | eq_ref | PRIMARY | PRIMARY | 4 | database.pml.SERVER_ID | 1 | |
+----+-------------+-------+--------+-----------------------------+---------+---------+---------------------------+---------+----------------------------------------------+
2 rows in set (0.00 sec)
mysql> SELECT ps.SERVER_ID, ps.FULLNAME FROM PAR_SERVER ps INNER JOIN PAR_MONITORINGv2_LOG pml ON ps.SERVER_ID = pml.SERVER_ID GROUP BY ps.SERVER_ID;
[...]
59 rows in set (2.30 sec)
mysql> explain [thequeryabove]
+----+-------------+-------+--------+-----------------------------+-----------+---------+---------------------------+---------+----------------------------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------+--------+-----------------------------+-----------+---------+---------------------------+---------+----------------------------------------------+
| 1 | SIMPLE | pml | index | SERVER_ID,SERVER_ID-ID_TYPE | SERVER_ID | 4 | NULL | 4015694 | Using index; Using temporary; Using filesort |
| 1 | SIMPLE | ps | eq_ref | PRIMARY | PRIMARY | 4 | database.pml.SERVER_ID | 1 | |
+----+-------------+-------+--------+-----------------------------+-----------+---------+---------------------------+---------+----------------------------------------------+
2 rows in set (0.00 sec)
mysql> SELECT pml.SERVER_ID FROM PAR_MONITORINGv2_LOG pml GROUP BY pml.SERVER_ID;
[...]
65 rows in set (0.00 sec)
mysql> explain [thequeryabove]
+----+-------------+-------+-------+---------------+-----------+---------+------+------+--------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------+-------+---------------+-----------+---------+------+------+--------------------------+
| 1 | SIMPLE | pml | range | NULL | SERVER_ID | 4 | NULL | 67 | Using index for group-by |
+----+-------------+-------+-------+---------------+-----------+---------+------+------+--------------------------+
1 row in set (0.00 sec)
I was able to improve the query a lot by defining an index for (SERVER_ID, ID_TYPE) as my following example query confirms:
mysql> SELECT count(*) 'count_warnings' FROM PAR_MONITORINGv2_LOG pml WHERE pml.SERVER_ID = 191 AND pml.ID_TYPE = 3 GROUP BY pml.SERVER_ID;
[...]
1 row in set (0.01 sec)
mysql> explain [thequeryabove]
+----+-------------+-------+------+-------------------------------------+-------------------+---------+-------------+-------+-------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------+------+-------------------------------------+-------------------+---------+-------------+-------+-------------+
| 1 | SIMPLE | pml | ref | ID_TYPE,SERVER_ID,SERVER_ID-ID_TYPE | SERVER_ID-ID_TYPE | 8 | const,const | 10254 | Using index |
+----+-------------+-------+------+-------------------------------------+-------------------+---------+-------------+-------+-------------+
1 row in set (0.00 sec)
I'm stuck now in the most broked down query with a long execution time of 2.30 sec. I don't know how to use indexes for such a query not having any where clause.
Your query will definitely benefit the most from adding composite index on PAR_MONITORINGv2_LOG(CREATED_DATE, SERVER_ID,ID_TYPE). However, I suggest even simple index on CREATED_DATE will improve performance a lot.
my tables are simple:
mysql> desc muralentry ;
+-----------------+------------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-----------------+------------------+------+-----+---------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
| user_src_id | int(11) | NO | MUL | NULL | |
| content | longtext | NO | | NULL | |
+-----------------+------------------+------+-----+---------+----------------+
mysql> desc muralentry_user ;
+-----------------+------------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-----------------+------------------+------+-----+---------+----------------+
| muralentry_id | int(11) | NO | PRI | NULL | auto_increment |
| userinfo_id | int(11) | NO | MUL | NULL | |
+-----------------+------------------+------+-----+---------+----------------+
Im doing the following query:
SELECT DISTINCT *
FROM muralentry
LEFT OUTER JOIN muralentry_user ON (muralentry.id = muralentry_user.muralentry_id)
WHERE user_src_id = 1
The explain:
+----+-------------+----------------------------+------+-------------------------------------------------------+-------------------------------------+---------+------------------------------------------+------+-----------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+----------------------------+------+-------------------------------------------------------+-------------------------------------+---------+------------------------------------------+------+-----------------+
| 1 | SIMPLE | muralentry | ref | muralentry_99bd10ae | muralentry_99bd10ae | 4 | const | 686 | Using temporary |
| 1 | SIMPLE | muralentry_user | ref | muralentry_id,muralentry_user_bcd7114e | muralentry_user_bcd7114e | 4 | muralentry.id | 15 | |
+----+-------------+----------------------------+------+-------------------------------------------------------+-------------------------------------+---------+------------------------------------------+------+-----------------+
Good result (for me :D)
But, when i add another where clause:
SELECT DISTINCT *
FROM muralentry
LEFT OUTER JOIN muralentry_user ON (muralentry.id = muralentry_user.muralentry_id)
WHERE user_src_id = 1 OR userinfo_id = 1;
The explain:
+----+-------------+----------------------------+------+-------------------------------------------------------+-------------------------------------+---------+------------------------------------------+---------+-----------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+----------------------------+------+-------------------------------------------------------+-------------------------------------+---------+------------------------------------------+---------+-----------------+
| 1 | SIMPLE | muralentry | ALL | muralentry_99bd10ae | NULL | NULL | NULL | 1140932 | Using temporary |
| 1 | SIMPLE | muralentry_user | ref | muralentry_id,muralentry_user_bcd7114e | muralentry_user_bcd7114e | 4 | muralentry.id | 15 | Using where |
+----+-------------+----------------------------+------+-------------------------------------------------------+-------------------------------------+---------+------------------------------------------+---------+-----------------+
Wow... the result if ALOT worst...
How can i "fix" this?
Should i create some index to do this job? Or recreate my query?
I'm expecting the following result: 'muralentry' rows where the user is 'user_src_id' AND the 'muralentry_user' rows where he is 'userinfo_id'.
-- edit --
I edited the question because when I wrote an AND actually wanted an OR... sorry for that!
I am going to join two tables by using a single position in one table to the range (represented by two columns) in another table.
However, the performance is too slow, which is about 20 mins.
I have tried adding the index on the table or changing the query.
But the performance is still poor.
So, I am asking for optimization of the joining speed.
The following is the query to MySQL.
mysql> SELECT `inVar`.chrom, `inVar`.pos, `openChrom_K562`.score
-> FROM `inVar`
-> LEFT JOIN `openChrom_K562`
-> ON (
-> `inVar`.chrom=`openChrom_K562`.chrom AND
-> `inVar`.pos BETWEEN `openChrom_K562`.chromStart AND `openChrom_K562`.chromEnd
-> );
inVar and openChrom_K562 are the tables I used.
inVar stores the single position in each row.
openChrom_K562 stores the range information indicated by chromStart and chromEnd.
inVar contains 57902 rows and openChrom_K562 has 137373 rows respectively.
Fields on the tables.
mysql> DESCRIBE inVar;
+-------+-------------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+-------+-------------+------+-----+---------+-------+
| chrom | varchar(31) | NO | PRI | NULL | |
| pos | int(10) | NO | PRI | NULL | |
+-------+-------------+------+-----+---------+-------+
mysql> DESCRIBE openChrom_K562;
+------------+-------------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+------------+-------------+------+-----+---------+-------+
| chrom | varchar(31) | NO | MUL | NULL | |
| chromStart | int(10) | NO | MUL | NULL | |
| chromEnd | int(10) | NO | | NULL | |
| score | int(10) | NO | | NULL | |
+------------+-------------+------+-----+---------+-------+
Index built in the tables
mysql> SHOW INDEX FROM inVar;
+-------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment |
+-------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| inVar | 0 | PRIMARY | 1 | chrom | A | NULL | NULL | NULL | | BTREE | |
| inVar | 0 | PRIMARY | 2 | pos | A | 57902 | NULL | NULL | | BTREE | |
+-------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
mysql> SHOW INDEX FROM openChrom_K562;
+----------------+------------+-------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment |
+----------------+------------+-------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| openChrom_K562 | 1 | start_end | 1 | chromStart | A | 137373 | NULL | NULL | | BTREE | |
| openChrom_K562 | 1 | start_end | 2 | chromEnd | A | 137373 | NULL | NULL | | BTREE | |
| openChrom_K562 | 1 | chrom_only | 1 | chrom | A | 22 | NULL | NULL | | BTREE | |
| openChrom_K562 | 1 | chrom_start | 1 | chrom | A | 22 | NULL | NULL | | BTREE | |
| openChrom_K562 | 1 | chrom_start | 2 | chromStart | A | 137373 | NULL | NULL | | BTREE | |
| openChrom_K562 | 1 | chrom_end | 1 | chrom | A | 22 | NULL | NULL | | BTREE | |
| openChrom_K562 | 1 | chrom_end | 2 | chromEnd | A | 137373 | NULL | NULL | | BTREE | |
+----------------+------------+-------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
Execution plan on MySQL
mysql> EXPLAIN SELECT `inVar`.chrom, `inVar`.pos, score FROM `inVar` LEFT JOIN `openChrom_K562` ON ( inVar.chrom=openChrom_K562.chrom AND `inVar`.pos BETWEEN chromStart AND chromEnd );
+----+-------------+----------------+-------+--------------------------------------------+------------+---------+-----------------+-------+-------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+----------------+-------+--------------------------------------------+------------+---------+-----------------+-------+-------------+
| 1 | SIMPLE | inVar | index | NULL | PRIMARY | 37 | NULL | 57902 | Using index |
| 1 | SIMPLE | openChrom_K562 | ref | start_end,chrom_only,chrom_start,chrom_end | chrom_only | 33 | tmp.inVar.chrom | 5973 | |
+----+-------------+----------------+-------+--------------------------------------------+------------+---------+-----------------+-------+-------------+
It seems it only optimizes by looking chrom in two tables. Then do the brute-force comparing in the tables.
Is there any ways to do the further optimization like indexing on the position?
(It is my first time posting the question, sorry for the poor posting quality.)
chrom_only is likely to be a bad index selection for your join as you only have chrom 22 values.
If I have interpreted this right the query should be faster if using start_end
SELECT `inVar`.chrom, `inVar`.pos, `openChrom_K562`.score
FROM `inVar`
LEFT JOIN `openChrom_K562`
USE INDEX (`start_end`)
ON (
`inVar`.chrom=`openChrom_K562`.chrom AND
`inVar`.pos BETWEEN `openChrom_K562`.chromStart AND `openChrom_K562`.chromEnd
)
I have inherited a database schema which has some design issues
Note that there are another 9 keys on the table which I haven't listed below, the keys in question look like
+-------+------------+----------------------------+--------------+--------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment |
+-------+------------+----------------------------+--------------+--------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| users | 0 | PRIMARY | 1 | userid | A | 604 | NULL | NULL | | BTREE | | |
| users | 1 | userid_2 | 1 | userid | A | 604 | NULL | NULL | | BTREE | | |
| users | 1 | userid_2 | 2 | age | A | 604 | NULL | NULL | YES | BTREE | | |
| users | 1 | userid_2 | 3 | image | A | 604 | 255 | NULL | YES | BTREE | | |
| users | 1 | userid_2 | 4 | gender | A | 604 | NULL | NULL | YES | BTREE | | |
| users | 1 | userid_2 | 5 | last_login | A | 604 | NULL | NULL | YES | BTREE | | |
| users | 1 | userid_2 | 6 | latitude | A | 604 | NULL | NULL | YES | BTREE | | |
| users | 1 | userid_2 | 7 | longitude | A | 604 | NULL | NULL | YES | BTREE | | |
+-------+------------+----------------------------+--------------+--------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
In a table with the following fields.
+--------------------------------+---------------------+------+-----+-------------------+----------------+
| Field | Type | Null | Key | Default | Extra |
+--------------------------------+---------------------+------+-----+-------------------+----------------+
| userid | int(11) | NO | PRI | NULL | auto_increment |
| age | int(11) | YES | | NULL | |
| image | varchar(500) | YES | | | |
| gender | varchar(10) | YES | | NULL | |
| last_login | timestamp | YES | MUL | NULL | |
| latitude | varchar(20) | YES | MUL | NULL | |
| longitude | varchar(20) | YES | | NULL | |
+--------------------------------+---------------------+------+-----+-------------------+----------------+
Running an explain statement and forcing it to use userid_2, it uses 522 rows
describe SELECT userid, age FROM users USE INDEX(userid_2) WHERE `userid` >=100 and age >27 limit 10 ;
+----+-------------+-------+-------+---------------+----------+---------+------+------+--------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------+-------+---------------+----------+---------+------+------+--------------------------+
| 1 | SIMPLE | users | index | userid_2 | userid_2 | 941 | NULL | 522 | Using where; Using index |
+----+-------------+-------+-------+---------------+----------+---------+------+------+--------------------------+
1 row in set (0.02 sec)
if I don't force it to use the index it is just using the primary key, which only consists of the userid and only uses 261 rows
mysql> describe SELECT userid, age FROM users WHERE userid >=100 and age >27 limit 10 ;
+----+-------------+-------+-------+--------------------------------------------+---------+---------+------+------+-------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------+-------+--------------------------------------------+---------+---------+------+------+-------------+
| 1 | SIMPLE | users | range | PRIMARY,users_user_ids_key,userid,userid_2 | PRIMARY | 4 | NULL | 261 | Using where |
+----+-------------+-------+-------+--------------------------------------------+---------+---------+------+------+-------------+
1 row in set (0.00 sec)
Questions
Why is it examining more rows when it uses the compound composite index?
Why isn't the query using the userid_2 index if its not specified in the query?
That row count is only an estimate based on indexed value distribution.
You have two options:
Execute ANALYZE TABLE mytable to recalculate distributions and then re-try the describe
Don't worry about stuff that doesn't matter... rows is just an estimate anyway
I found a very weird mysql behaviour : when I run a specific query twice, the explain of this query is different the second time :
query = SELECT `twstats_twwordstrend`.`id`, `twstats_twwordstrend`.`created`, `twstats_twwordstrend`.`freq`, `twstats_twwordstrend`.`word_id` FROM `twstats_twwordstrend` INNER JOIN `twstats_twwords` ON (`twstats_twwordstrend`.`word_id` = `twstats_twwords`.`id`) WHERE (`twstats_twwords`.`name` = '#ladygaga' AND `twstats_twwordstrend`.`created` > '2011-01-28 01:30:19' );
1st query execution and then run explain :
mysql> EXPLAIN SELECT `twstats_twwordstrend`.`id`, `twstats_twwordstrend`.`created`, `twstats_twwordstrend`.`freq`, `twstats_twwordstrend`.`word_id` FROM `twstats_twwordstrend` INNER JOIN `twstats_twwords` ON (`twstats_twwordstrend`.`word_id` = `twstats_twwords`.`id`) WHERE (`twstats_twwords`.`name` = '#ladygaga' AND `twstats_twwordstrend`.`created` > '2011-01-28 01:30:19' );
+----+-------------+----------------------+--------+-------------------------------+---------+---------+-------------------------------------------+---------+-------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+----------------------+--------+-------------------------------+---------+---------+-------------------------------------------+---------+-------------+
| 1 | SIMPLE | twstats_twwordstrend | ALL | twstats_twwordstrend_4b95d890 | NULL | NULL | NULL | 4877401 | Using where |
| 1 | SIMPLE | twstats_twwords | eq_ref | PRIMARY | PRIMARY | 4 | statweestics.twstats_twwordstrend.word_id | 1 | Using where |
+----+-------------+----------------------+--------+-------------------------------+---------+---------+-------------------------------------------+---------+-------------+
2 rows in set (0.00 sec)
2nd query execution and then run explain :
mysql> EXPLAIN SELECT `twstats_twwordstrend`.`id`, `twstats_twwordstrend`.`created`, `twstats_twwordstrend`.`freq`, `twstats_twwordstrend`.`word_id` FROM `twstats_twwordstrend` INNER JOIN `twstats_twwords` ON (`twstats_twwordstrend`.`word_id` = `twstats_twwords`.`id`) WHERE (`twstats_twwords`.`name` = '#ladygaga' AND `twstats_twwordstrend`.`created` > '2011-01-28 01:30:19' );
+----+-------------+----------------------+------+-------------------------------+-------------------------------+---------+---------------------------------+--------+-------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+----------------------+------+-------------------------------+-------------------------------+---------+---------------------------------+--------+-------------+
| 1 | SIMPLE | twstats_twwords | ALL | PRIMARY | NULL | NULL | NULL | 222994 | Using where |
| 1 | SIMPLE | twstats_twwordstrend | ref | twstats_twwordstrend_4b95d890 | twstats_twwordstrend_4b95d890 | 4 | statweestics.twstats_twwords.id | 15 | Using where |
+----+-------------+----------------------+------+-------------------------------+-------------------------------+---------+---------------------------------+--------+-------------+
2 rows in set (0.00 sec)
mysql> describe twstats_twwords;
+---------+--------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+---------+--------------+------+-----+---------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
| created | datetime | NO | | NULL | |
| name | varchar(140) | NO | | NULL | |
+---------+--------------+------+-----+---------+----------------+
3 rows in set (0.00 sec)
mysql> describe twstats_twwordstrend;
+---------+----------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+---------+----------+------+-----+---------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
| created | datetime | NO | | NULL | |
| freq | double | NO | | NULL | |
| word_id | int(11) | NO | MUL | NULL | |
+---------+----------+------+-----+---------+----------------+
4 rows in set (0.00 sec)
How this can be possible ??
Look at the rows column. The engine was able to gather more statistics -- so the next time it will try to use the better plan.
Happy coding.