I have an incredibly complex query being produced by an ORM, but here's the relevant portions for the sake of this question:
SELECT
...
FROM
`Broadcast` `t`
LEFT OUTER JOIN
`Site` `site`
ON (`t`.`site_id` = `site`.`id`)
LEFT OUTER JOIN
`Customer` `customer`
ON (`site`.`customer_id` = `customer`.`id`)
LEFT OUTER JOIN
`Domain` `domain`
ON (`customer`.`domain_id` = `domain`.`id`)
...
My fields are defined as follows:
>SHOW FIELDS FROM Broadcast;
+---------------------------+--------------------------------------+------+-----+------------------+----------------+
| Field | Type | Null | Key | Default | Extra |
+---------------------------+--------------------------------------+------+-----+------------------+----------------+
| id | int(10) unsigned | NO | PRI | NULL | auto_increment |
| site_id | int(10) unsigned | NO | MUL | NULL | |
...
>SHOW FIELDS FROM Site;
+---------------------------+---------------------+------+-----+------------------+----------------+
| Field | Type | Null | Key | Default | Extra |
+---------------------------+---------------------+------+-----+------------------+----------------+
| id | int(10) unsigned | NO | PRI | NULL | auto_increment |
| customer_id | int(11) | NO | MUL | 0 | |
...
>SHOW FIELDS FROM Customer;
+---------------------------+------------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+---------------------------+------------------+------+-----+---------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
| domain_id | int(10) unsigned | NO | MUL | 1 | |
...
>SHOW FIELDS FROM Domain;
+---------------------------------+------------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+---------------------------------+------------------+------+-----+---------+----------------+
| id | int(10) unsigned | NO | PRI | NULL | auto_increment |
...
Why some of our IDs are int(11) and others are int(10) unsigned is just a matter of historical context. What's important is that the X_id foreign key always shares a type with the relevant id column. We never attempt to link an int(11) with an int(10) unsigned. Also the id field is always the primary key for the table.
We also have indexes on all of the foreign keys:
>SHOW INDEXES FROM Broadcast;
+-----------+------------+---------------------------------------+--------------+---------------------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment |
+-----------+------------+---------------------------------------+--------------+---------------------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| Broadcast | 0 | PRIMARY | 1 | id | A | 139708 | NULL | NULL | | BTREE | | |
| Broadcast | 1 | site_id | 1 | site_id | A | 1060 | NULL | NULL | | BTREE | | |
...
>SHOW INDEXES FROM Site;
+-------+------------+----------------------------------+--------------+---------------------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment |
+-------+------------+----------------------------------+--------------+---------------------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| Site | 0 | PRIMARY | 1 | id | A | 1876 | NULL | NULL | | BTREE | | |
| Site | 1 | customer_id | 1 | customer_id | A | 250 | NULL | NULL | | BTREE | | |
...
>SHOW INDEXES FROM Customer;
+----------+------------+--------------------------------------+--------------+---------------------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment |
+----------+------------+--------------------------------------+--------------+---------------------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| Customer | 0 | PRIMARY | 1 | id | A | 292 | NULL | NULL | | BTREE | | |
| Customer | 1 | domain_id | 1 | domain_id | A | 2 | NULL | NULL | | BTREE | | |
...
>SHOW INDEXES FROM Domain;
+--------+------------+-------------------------------------------+--------------+---------------------------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment |
+--------+------------+-------------------------------------------+--------------+---------------------------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| Domain | 0 | PRIMARY | 1 | id | A | 2 | NULL | NULL | | BTREE | | |
...
Each of these indexes consists of a single field, so the field is the first sequence in the index.
When I run EXPLAIN ... on my query, the output I get looks as follows:
Site and Customer are joined with the eq_ref type, but Domain is joined with ALL type. Why is MySQL not using my index?
I can force MySQL to use my index by adding USE INDEX FOR JOIN (domain_id) after the Customer table name, but this causes a new issue:
The problem just moves up from the Customer->Domain relation to the Site->Customer relation. I can shift the problem up one further using USE INDEX FOR JOIN (customer_id) after the Site table name, and as you might expect this just causes the Site table to join with the ALL type instead of eq_ref
I can't for the life of me figure out why MySQL is refusing to use my indexes
Per #ysth in the comments:
MySQL will not use an index if the table you are joining has very few rows, as the overhead of reading an index from disk exceeds the benefits
Related
Well, yes I do. I am receiving the following error:
SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry '1-684-Answer' for key 'user_id' (SQL: insert into `votes` (`votable_id`, `votable_type`, `user_id`, `updated_at`, `created_at`) values (684, Answer, 1, 2016-03-17 12:16:53, 2016-03-17 12:16:53))
That would suggest I have a UNIQUE constraint on the user_id column? Looking at the DESCRIBE votes; I'm not sure whether this indicates that ot not. It shouldn't have though, one user should be able to vote many times:
mysql> describe votes;
+--------------+--------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+--------------+--------------+------+-----+---------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
| user_id | int(11) | NO | MUL | NULL | |
| votable_id | int(11) | NO | | NULL | |
| votable_type | varchar(255) | NO | | NULL | |
| value | int(11) | NO | | NULL | |
| created_at | datetime | NO | | NULL | |
| updated_at | datetime | NO | | NULL | |
| deleted_at | datetime | YES | | NULL | |
+--------------+--------------+------+-----+---------+----------------+
mysql> show indexes from votes where column_name = 'user_id';
+-------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment |
+-------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| votes | 0 | user_id | 1 | user_id | A | 51 | NULL | NULL | | BTREE | | |
+-------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
Does the above indicate that UNIQUE property is set? I'm looking at the migrations file and I can't see where the UNIQUE property was assigned. Perhaps someone put in manually.
This shows that there is an index user_id in your table.
you can use following code to see index that will list out all the indexes :
SHOW INDEX FROM votes;
if it doesnot solve your problem remove the unique index and recreate it again.
Non_unique = 0. It means it cannot contain duplicates. See SHOW INDEX documentation.
I am going to join two tables by using a single position in one table to the range (represented by two columns) in another table.
However, the performance is too slow, which is about 20 mins.
I have tried adding the index on the table or changing the query.
But the performance is still poor.
So, I am asking for optimization of the joining speed.
The following is the query to MySQL.
mysql> SELECT `inVar`.chrom, `inVar`.pos, `openChrom_K562`.score
-> FROM `inVar`
-> LEFT JOIN `openChrom_K562`
-> ON (
-> `inVar`.chrom=`openChrom_K562`.chrom AND
-> `inVar`.pos BETWEEN `openChrom_K562`.chromStart AND `openChrom_K562`.chromEnd
-> );
inVar and openChrom_K562 are the tables I used.
inVar stores the single position in each row.
openChrom_K562 stores the range information indicated by chromStart and chromEnd.
inVar contains 57902 rows and openChrom_K562 has 137373 rows respectively.
Fields on the tables.
mysql> DESCRIBE inVar;
+-------+-------------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+-------+-------------+------+-----+---------+-------+
| chrom | varchar(31) | NO | PRI | NULL | |
| pos | int(10) | NO | PRI | NULL | |
+-------+-------------+------+-----+---------+-------+
mysql> DESCRIBE openChrom_K562;
+------------+-------------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+------------+-------------+------+-----+---------+-------+
| chrom | varchar(31) | NO | MUL | NULL | |
| chromStart | int(10) | NO | MUL | NULL | |
| chromEnd | int(10) | NO | | NULL | |
| score | int(10) | NO | | NULL | |
+------------+-------------+------+-----+---------+-------+
Index built in the tables
mysql> SHOW INDEX FROM inVar;
+-------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment |
+-------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| inVar | 0 | PRIMARY | 1 | chrom | A | NULL | NULL | NULL | | BTREE | |
| inVar | 0 | PRIMARY | 2 | pos | A | 57902 | NULL | NULL | | BTREE | |
+-------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
mysql> SHOW INDEX FROM openChrom_K562;
+----------------+------------+-------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment |
+----------------+------------+-------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| openChrom_K562 | 1 | start_end | 1 | chromStart | A | 137373 | NULL | NULL | | BTREE | |
| openChrom_K562 | 1 | start_end | 2 | chromEnd | A | 137373 | NULL | NULL | | BTREE | |
| openChrom_K562 | 1 | chrom_only | 1 | chrom | A | 22 | NULL | NULL | | BTREE | |
| openChrom_K562 | 1 | chrom_start | 1 | chrom | A | 22 | NULL | NULL | | BTREE | |
| openChrom_K562 | 1 | chrom_start | 2 | chromStart | A | 137373 | NULL | NULL | | BTREE | |
| openChrom_K562 | 1 | chrom_end | 1 | chrom | A | 22 | NULL | NULL | | BTREE | |
| openChrom_K562 | 1 | chrom_end | 2 | chromEnd | A | 137373 | NULL | NULL | | BTREE | |
+----------------+------------+-------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
Execution plan on MySQL
mysql> EXPLAIN SELECT `inVar`.chrom, `inVar`.pos, score FROM `inVar` LEFT JOIN `openChrom_K562` ON ( inVar.chrom=openChrom_K562.chrom AND `inVar`.pos BETWEEN chromStart AND chromEnd );
+----+-------------+----------------+-------+--------------------------------------------+------------+---------+-----------------+-------+-------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+----------------+-------+--------------------------------------------+------------+---------+-----------------+-------+-------------+
| 1 | SIMPLE | inVar | index | NULL | PRIMARY | 37 | NULL | 57902 | Using index |
| 1 | SIMPLE | openChrom_K562 | ref | start_end,chrom_only,chrom_start,chrom_end | chrom_only | 33 | tmp.inVar.chrom | 5973 | |
+----+-------------+----------------+-------+--------------------------------------------+------------+---------+-----------------+-------+-------------+
It seems it only optimizes by looking chrom in two tables. Then do the brute-force comparing in the tables.
Is there any ways to do the further optimization like indexing on the position?
(It is my first time posting the question, sorry for the poor posting quality.)
chrom_only is likely to be a bad index selection for your join as you only have chrom 22 values.
If I have interpreted this right the query should be faster if using start_end
SELECT `inVar`.chrom, `inVar`.pos, `openChrom_K562`.score
FROM `inVar`
LEFT JOIN `openChrom_K562`
USE INDEX (`start_end`)
ON (
`inVar`.chrom=`openChrom_K562`.chrom AND
`inVar`.pos BETWEEN `openChrom_K562`.chromStart AND `openChrom_K562`.chromEnd
)
I have this query:
SELECT
s.last_spread, s.sd, s.mean, s.id
,c.id_ticker, c.coef
,t.ticker
,p.last, p.price
FROM (SELECT * FROM spreads WHERE spreads.id_check=1 LIMIT 100,500 ) as s
INNER JOIN coef as c
ON c.id_spread = s.id
INNER JOIN tickers AS t
ON t.id = c.id_ticker
LEFT JOIN (SELECT prices.id_ticker, MAX(prices.date) as last, prices.price FROM prices GROUP BY prices.id_ticker) AS p
ON p.id_ticker = t.id
These are the schemas of the tables:
mysql> desc spreads;
+-------------+---------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-------------+---------+------+-----+---------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
| id_check | int(11) | YES | MUL | NULL | |
| sd | double | YES | | NULL | |
| mean | double | YES | | NULL | |
| last_spread | double | YES | | NULL | |
+-------------+---------+------+-----+---------+----------------+
5 rows in set (0.00 sec)
mysql> desc coef;
+-----------+---------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-----------+---------+------+-----+---------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
| id_spread | int(11) | YES | MUL | NULL | |
| id_ticker | int(11) | YES | | NULL | |
| coef | double | YES | | NULL | |
| side | double | YES | | NULL | |
+-----------+---------+------+-----+---------+----------------+
5 rows in set (0.00 sec)
mysql> desc tickers;
+----------+------------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+----------+------------------+------+-----+---------+----------------+
| id | int(10) unsigned | NO | PRI | NULL | auto_increment |
| ticker | varchar(45) | NO | | NULL | |
| name | varchar(150) | NO | | NULL | |
| category | varchar(150) | NO | | NULL | |
| issuer | varchar(150) | NO | | NULL | |
+----------+------------------+------+-----+---------+----------------+
5 rows in set (0.00 sec)
mysql> desc prices;
+-----------+------------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-----------+------------------+------+-----+---------+----------------+
| id | int(10) unsigned | NO | PRI | NULL | auto_increment |
| id_ticker | int(10) unsigned | NO | MUL | NULL | |
| date | date | NO | | NULL | |
| price | double | NO | | NULL | |
+-----------+------------------+------+-----+---------+----------------+
4 rows in set (0.01 sec)
These are the indexes of the above tables;
mysql> show indexes from spreads;
+---------+------------+-----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment |
+---------+------------+-----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| spreads | 0 | PRIMARY | 1 | id | A | 2299 | NULL | NULL | | BTREE | |
| spreads | 1 | check_idx | 1 | id_check | A | 1 | NULL | NULL | YES | BTREE | |
+---------+------------+-----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
2 rows in set (0.00 sec)
mysql> show indexes from coef;
+-------+------------+-------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment |
+-------+------------+-------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| coef | 0 | PRIMARY | 1 | id | A | 9078 | NULL | NULL | | BTREE | |
| coef | 1 | spread_ticker_idx | 1 | id_spread | A | NULL | NULL | NULL | YES | BTREE | |
| coef | 1 | spread_ticker_idx | 2 | id_ticker | A | NULL | NULL | NULL | YES | BTREE | |
+-------+------------+-------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
3 rows in set (0.00 sec)
mysql> show indexes from tickers;
+---------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment |
+---------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| tickers | 0 | PRIMARY | 1 | id | A | 100 | NULL | NULL | | BTREE | |
+---------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
1 row in set (0.00 sec)
mysql> show indexes from prices;
+--------+------------+-----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment |
+--------+------------+-----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| prices | 0 | PRIMARY | 1 | id | A | 19962 | NULL | NULL | | BTREE | |
| prices | 1 | id_ticker | 1 | id_ticker | A | 19962 | NULL | NULL | | BTREE | |
+--------+------------+-----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
2 rows in set (0.15 sec)
And this is the explain of the query:
+----+-------------+------------+--------+-------------------+-------------------+---------+---------------------------+--------+-------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+------------+--------+-------------------+-------------------+---------+---------------------------+--------+-------------+
| 1 | PRIMARY | <derived2> | ALL | NULL | NULL | NULL | NULL | 500 | |
| 1 | PRIMARY | c | ref | spread_ticker_idx | spread_ticker_idx | 5 | s.id | 90 | Using where |
| 1 | PRIMARY | t | eq_ref | PRIMARY | PRIMARY | 4 | spreadtrading.c.id_ticker | 1 | Using where |
| 1 | PRIMARY | <derived3> | ALL | NULL | NULL | NULL | NULL | 100 | |
| 3 | DERIVED | prices | index | NULL | id_ticker | 4 | NULL | 119774 | |
| 2 | DERIVED | spreads | ref | check_idx | check_idx | 5 | | 2298 | Using where |
+----+-------------+------------+--------+-------------------+-------------------+---------+---------------------------+--------+-------------+
6 rows in set (0.27 sec)
Could I optimize it?
Thank you!
EDIT:
I would like to know if the INDEXES and the table's structure are optimized for the query I posted above. The results that I get using this query are good, it works well, but maybe I can optimize it to increse the "speed" of the query.
I think you may gain something by dropping the spreads subquery and moving the WHERE clause to the end, as in the following code. This loses your LIMIT restriction - perhaps you could put a LIMIT clause at the end as well, depending on what you're trying to achieve in terms of limiting the size of the output.
SELECT
s.last_spread, s.sd, s.mean, s.id
,c.id_ticker, c.coef
,t.ticker
,p.last, p.price
FROM spreads as s
INNER JOIN coef as c
ON c.id_spread = s.id
INNER JOIN tickers AS t
ON t.id = c.id_ticker
LEFT JOIN (SELECT prices.id_ticker, MAX(prices.date) as last, prices.price FROM prices GROUP BY prices.id_ticker) AS p
ON p.id_ticker = t.id
WHERE s.id_check = 1
I need to find the best index for this query:
SELECT c.id id, type
FROM Content c USE INDEX (type_proc_last_cat)
LEFT JOIN Battles b ON c.id = b.id
WHERE type = 1
AND processing_status = 1
AND category IN (13, 19)
AND status = 4
ORDER BY last_change DESC
LIMIT 100";
The tables look like this:
mysql> describe Content;
+-------------------+---------------------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+-------------------+---------------------+------+-----+---------+-------+
| id | bigint(20) unsigned | NO | PRI | NULL | |
| type | tinyint(3) unsigned | NO | MUL | NULL | |
| category | bigint(20) unsigned | NO | | NULL | |
| processing_status | tinyint(3) unsigned | NO | | NULL | |
| last_change | int(10) unsigned | NO | | NULL | |
+-------------------+---------------------+------+-----+---------+-------+
mysql> show indexes from Content;
+---------+------------+---------------------+--------------+-------------------+-----------+-------------+----------+--------+------+------------+---------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment |
+---------+------------+---------------------+--------------+-------------------+-----------+-------------+----------+--------+------+------------+---------+
| Content | 0 | PRIMARY | 1 | id | A | 4115 | NULL | NULL | | BTREE | |
| Content | 1 | type_proc_last_cat | 1 | type | A | 4 | NULL | NULL | | BTREE | |
| Content | 1 | type_proc_last_cat | 2 | processing_status | A | 20 | NULL | NULL | | BTREE | |
| Content | 1 | type_proc_last_cat | 3 | last_change | A | 4115 | NULL | NULL | | BTREE | |
| Content | 1 | type_proc_last_cat | 4 | category | A | 4115 | NULL | NULL | | BTREE | |
+---------+------------+---------------------+--------------+-------------------+-----------+-------------+----------+--------+------+------------+---------+
mysql> describe Battles;
+---------------------+---------------------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+---------------------+---------------------+------+-----+---------+-------+
| id | bigint(20) unsigned | NO | PRI | NULL | |
| status | tinyint(4) unsigned | NO | | NULL | |
| status_last_changed | int(11) unsigned | NO | | NULL | |
+---------------------+---------------------+------+-----+---------+-------+
mysql> show indexes from Battles;
+---------+------------+-----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment |
+---------+------------+-----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| Battles | 0 | PRIMARY | 1 | id | A | 1215 | NULL | NULL | | BTREE | |
| Battles | 0 | id_status | 1 | id | A | 1215 | NULL | NULL | | BTREE | |
| Battles | 0 | id_status | 2 | status | A | 1215 | NULL | NULL | | BTREE | |
+---------+------------+-----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
And I get output like this:
mysql> explain
-> SELECT c.id id, type
-> FROM Content c USE INDEX (type_proc_last_cat)
-> LEFT JOIN Battles b USE INDEX (id_status) ON c.id = b.id
-> WHERE type = 1
-> AND processing_status = 1
-> AND category IN (13, 19)
-> AND status = 4
-> ORDER BY last_change DESC
-> LIMIT 100;
+----+-------------+-------+--------+--------------------+--------------------+---------+-----------------------+------+--------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------+--------+--------------------+--------------------+---------+-----------------------+------+--------------------------+
| 1 | SIMPLE | c | ref | type_proc_last_cat | type_proc_last_cat | 2 | const,const | 1352 | Using where; Using index |
| 1 | SIMPLE | b | eq_ref | id_status | id_status | 9 | wtm_master.c.id,const | 1 | Using where; Using index |
+----+-------------+-------+--------+--------------------+--------------------+---------+-----------------------+------+--------------------------+
The trouble is the rows count for the Content table. It appears MySQL is unable to effectively use both last_change and category in the type_proc_last_cat index. If I switch the order of last_change and category, fewer rows are selected but it results in a filesort for the ORDER BY, meaning it pulls all the matching rows from the database. That is worse, since there are 100,000+ rows in both tables.
Tables are both InnoDB, so keep in mind that the PRIMARY key is appended to every other index. So the index the index type_proc_last_cat above behaves likes it's on (type, processing_status, last_change, category, id). I am aware I could change the PRIMARY key for Battles to (id, status) and drop the id_status index (and I may just do that).
Edit: Any value for type, category, processing_status, and status is less than 20% of the total values. last_change and status_last_change are unix timestamps.
Edit: If I use a different index with category and last_change in reverse order, I get this:
mysql> show indexes from Content;
+---------+------------+---------------------+--------------+-------------------+-----------+-------------+----------+--------+------+------------+---------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment |
+---------+------------+---------------------+--------------+-------------------+-----------+-------------+----------+--------+------+------------+---------+
| Content | 0 | PRIMARY | 1 | id | A | 4115 | NULL | NULL | | BTREE | |
| Content | 1 | type_proc_cat_last | 1 | type | A | 6 | NULL | NULL | | BTREE | |
| Content | 1 | type_proc_cat_last | 2 | processing_status | A | 26 | NULL | NULL | | BTREE | |
| Content | 1 | type_proc_cat_last | 3 | category | A | 228 | NULL | NULL | | BTREE | |
| Content | 1 | type_proc_cat_last | 4 | last_change | A | 4115 | NULL | NULL | | BTREE | |
+---------+------------+---------------------+--------------+-------------------+-----------+-------------+----------+--------+------+------------+---------+
mysql> explain SELECT c.id id, type FROM Content c USE INDEX (type_proc_cat_last) LEFT JOIN Battles b
USE INDEX (id_status) ON c.id = b.id WHERE type = 1 AND processing_status = 1 AND category IN (13, 19) AND status = 4 ORDER BY last_change DESC LIMIT 100;
+----+-------------+-------+-------+--------------------+--------------------+---------+-----------------------+------+------------------------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------+-------+--------------------+--------------------+---------+-----------------------+------+------------------------------------------+
| 1 | SIMPLE | c | range | type_proc_cat_last | type_proc_cat_last | 10 | NULL | 165 | Using where; Using index; Using filesort |
| 1 | SIMPLE | b | ref | id_status | id_status | 9 | wtm_master.c.id,const | 1 | Using where; Using index |
+----+-------------+-------+-------+--------------------+--------------------+---------+-----------------------+------+------------------------------------------+
The filesort worries me as it tells me MySQL pulls all the matching rows first, before sorting. This will be a big problem when there are 100,000+.
rows field in EXPLAIN doesn't reflect the number of rows that actually were read. It reflects the number of rows that possible will be affected. Also it doesn't rely on LIMIT, because LIMIT is applied after the plan has been calculated.
So you don't need to worry about it.
Also I would suggest you to swap last_change and category in type_proc_last_cat so mysql can try to use the last index part (last_change) for sorting purposes.
Whats query can be used to get the details of Indexes of any table? I need this to find out primarykey/autoincremented value of any table..
Please help/Guide me...
You can use
show indexes from your_table;
For more informations : 12.4.5.23. SHOW INDEX Syntax
As a quick demo (on a not-quite-optimized table) :
mysql> show indexes from post;
+-------+------------+-----------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment |
+-------+------------+-----------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| post | 0 | PRIMARY | 1 | id | A | 7 | NULL | NULL | | BTREE | |
| post | 1 | id_blog_idx | 1 | id_blog | A | 2 | NULL | NULL | | BTREE | |
| post | 1 | id_user_idx | 1 | id_user | A | 7 | NULL | NULL | | BTREE | |
| post | 1 | code_syntax_idx | 1 | code_syntax | A | 7 | NULL | NULL | | BTREE | |
| post | 1 | code_status_idx | 1 | code_status | A | 2 | NULL | NULL | | BTREE | |
| post | 1 | id_category_idx | 1 | id_category | A | 7 | NULL | NULL | | BTREE | |
+-------+------------+-----------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
6 rows in set (0,00 sec)
Note, though, that this will display indexes -- and auto_increment doesn't have much to do with indexes.
If you want to see the auto_increment of your table, you can use desc :
desc your_table;
For more informations : 12.8.1. DESCRIBE Syntax
And, for example, with the same table :
mysql> desc post;
+--------------------+------------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+--------------------+------------------+------+-----+---------+----------------+
| id | int(10) unsigned | NO | PRI | NULL | auto_increment |
| id_blog | int(10) unsigned | NO | MUL | NULL | |
| id_user | int(10) unsigned | NO | MUL | NULL | |
...
...
| nb_comments | smallint(6) | NO | | 0 | |
+--------------------+------------------+------+-----+---------+----------------+
17 rows in set (0,05 sec)
Also, an alternate way (which also shows you the current value of the AUTO_INCREMENT counter) is:
=> SHOW CREATE TABLE activations;
Yielding
CREATE TABLE `activations` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`user_id` int(11) DEFAULT NULL,
`created_at` datetime DEFAULT NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=104974 DEFAULT CHARSET=utf8