I have a table that stores tags in the JSON format for each page. Current table has about 1,000,000 records.
When I run
SELECT COUNT(*) FROM alltags WHERE tagtype = 'pages' AND JSON_SEARCH(tags, 'one', 'Hampden') IS NOT NULL;
the query takes about 20 seconds
+----------+
| COUNT(*) |
+----------+
| 23500 |
+----------+
1 row in set (19.20 sec)
The query is slow regardless if I use InnoDB or MyISAM engine.
I got indexes for tagtype, tags and tagtype with tags columns. I need the exact number of results for pagination. Is there a way I can speed up the SELECT COUNT query? Selecting actual data is pretty instant.
SELECT id FROM alltags WHERE tagtype = 'pages' AND JSON_SEARCH(tags, 'one', 'Hampden') IS NOT NULL LIMIT 0,10;
+------+
| id |
+------+
| 5072 |
| 5075 |
| 5078 |
| 5081 |
| 5084 |
| 5087 |
| 5090 |
| 5093 |
| 5096 |
| 5099 |
+------+
10 rows in set (0.01 sec)
These are the indexes
SHOW INDEX FROM alltags;
+---------+------------+-----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment |
+---------+------------+-----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| alltags | 0 | PRIMARY | 1 | id | A | 896649 | NULL | NULL | | BTREE | | |
| alltags | 1 | tagtype | 1 | tagtype | A | 2 | NULL | NULL | | BTREE | | |
| alltags | 1 | keyword | 1 | keyword | A | 33 | NULL | NULL | | BTREE | | |
| alltags | 1 | tags | 1 | tags | A | 89665 | NULL | NULL | YES | BTREE | | |
+---------+------------+-----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
EXPLAIN results:
Select data
EXPLAIN SELECT id FROM alltags WHERE tagtype = 'pages' AND JSON_SEARCH(tags, 'one', 'Hampden') IS NOT NULL;
+----+-------------+---------+------------+------+---------------+---------+---------+-------+--------+----------+-------------+
| id | select_type | table | partitions | type | possible_keys | key | key_len | ref | rows | filtered | Extra |
+----+-------------+---------+------------+------+---------------+---------+---------+-------+--------+----------+-------------+
| 1 | SIMPLE | alltags | NULL | ref | tagtype | tagtype | 7 | const | 892372 | 100.00 | Using where |
+----+-------------+---------+------------+------+---------------+---------+---------+-------+--------+----------+-------------+
1 row in set, 1 warning (0.00 sec)
Select count
EXPLAIN SELECT COUNT(*) FROM alltags WHERE tagtype = 'pages' AND JSON_SEARCH(tags, 'one', 'Hampden') IS NOT NULL;
+----+-------------+---------+------------+------+---------------+---------+---------+-------+--------+----------+-------------+
| id | select_type | table | partitions | type | possible_keys | key | key_len | ref | rows | filtered | Extra |
+----+-------------+---------+------------+------+---------------+---------+---------+-------+--------+----------+-------------+
| 1 | SIMPLE | alltags | NULL | ref | tagtype | tagtype | 7 | const | 892372 | 100.00 | Using where |
+----+-------------+---------+------------+------+---------------+---------+---------+-------+--------+----------+-------------+
1 row in set, 1 warning (0.00 sec)
Related
We have three tables users, stores and store_customer_associations, where a user can be a customer for many stores.
I've tried to place another index in primary key but without success (and other too).
(currently these are our indexes in each table)
mysql> SHOW INDEXES FROM store_customer_associations;
+-------+------------+-------------------------------------+--------------+----------------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment |
+-------+------------+-------------------------------------+--------------+----------------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| users | 0 | PRIMARY | 1 | id | A | 9386515 | NULL | NULL | | BTREE | | |
| users | 0 | tmp_idx_users_id | 1 | id | A | 9386515 | NULL | NULL | | BTREE | | |
+-------+------------+-------------------------------------+--------------+----------------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
10 rows in set (0.00 sec)
mysql> SHOW INDEXES FROM store_customer_associations;
+-----------------------------+------------+-----------------------------------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment |
+-----------------------------+------------+-----------------------------------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| store_customer_associations | 0 | PRIMARY | 1 | id | A | 51298761 | NULL | NULL | | BTREE | | |
| store_customer_associations | 1 | index_store_customer_associations_on_store_id | 1 | store_id | A | 50096 | NULL | NULL | YES | BTREE | | |
| store_customer_associations | 1 | index_store_customer_associations_on_user_id | 1 | user_id | A | 25649380 | NULL | NULL | YES | BTREE | | |
+-----------------------------+------------+-----------------------------------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
3 rows in set (0.00 sec)
We are using rails 5.1 (this is a big rails 4 legacy database).
Then we need to make this query fast:
SELECT COUNT(*)
FROM `users`
INNER JOIN `store_customer_associations`
ON `users`.`id` = `store_customer_associations`.`user_id`
WHERE `store_customer_associations`.`store_id` = STORE_ID;
+----------+
| COUNT(*) |
+----------+
| 1997632 |
+----------+
1 row in set (6.64 sec)
mysql> EXPLAIN SELECT COUNT(*)
FROM `users`
INNER JOIN `store_customer_associations`
ON `users`.`id` = `store_customer_associations`.`user_id`
WHERE `store_customer_associations`.`store_id` = STORE_ID;
+------+-------------+-----------------------------+--------+--------------------------------------------------------------------------------------------+-----------------------------------------------+---------+---------------------------------------------------------+---------+-------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+------+-------------+-----------------------------+--------+--------------------------------------------------------------------------------------------+-----------------------------------------------+---------+---------------------------------------------------------+---------+-------------+
| 1 | SIMPLE | store_customer_associations | ref | index_store_customer_associations_on_store_id,index_store_customer_associations_on_user_id | index_store_customer_associations_on_store_id | 5 | const | 4401812 | Using where |
| 1 | SIMPLE | users | eq_ref | PRIMARY,tmp_idx_users_id | PRIMARY | 4 | trustvox_production.store_customer_associations.user_id | 1 | Using index |
+------+-------------+-----------------------------+--------+--------------------------------------------------------------------------------------------+-----------------------------------------------+---------+---------------------------------------------------------+---------+-------------+
2 rows in set (0.00 sec)
UPDATE 1
mysql> EXPLAIN SELECT COUNT(*)
FROM `users`
INNER JOIN `store_customer_associations`
ON `users`.`id` = `store_customer_associations`.`user_id`
WHERE `store_customer_associations`.`store_id` = STORE_ID;
+------+-------------+-----------------------------+--------+-------------------------------------------------------------------------------------------------------------+-----------------------------------------------+---------+---------------------------------------------------------+---------+-------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+------+-------------+-----------------------------+--------+-------------------------------------------------------------------------------------------------------------+-----------------------------------------------+---------+---------------------------------------------------------+---------+-------------+
| 1 | SIMPLE | store_customer_associations | ref | index_store_customer_associations_on_store_id,index_store_customer_associations_on_user_id,user_id_store_id | index_store_customer_associations_on_store_id | 5 | const | 4401812 | Using where |
| 1 | SIMPLE | users | eq_ref | PRIMARY,tmp_idx_users_id | PRIMARY | 4 | trustvox_production.store_customer_associations.user_id | 1 | Using index |
+------+-------------+-----------------------------+--------+-------------------------------------------------------------------------------------------------------------+-----------------------------------------------+---------+---------------------------------------------------------+---------+-------------+
2 rows in set (0.00 sec)
UPDATE 2
mysql> SELECT COUNT(*)
FROM users
INNER JOIN store_customer_associations
FORCE INDEX FOR JOIN (PRIMARY, user_id_store_id)
ON users.id = store_customer_associations.user_id
WHERE store_customer_associations.store_id = STORE_ID;
+----------+
| COUNT(*) |
+----------+
| 1997632 |
+----------+
1 row in set (28.90 sec)
mysql> EXPLAIN SELECT COUNT(*)
FROM users
INNER JOIN store_customer_associations
FORCE INDEX FOR JOIN (PRIMARY, user_id_store_id)
ON users.id = store_customer_associations.user_id
WHERE store_customer_associations.store_id = STORE_ID;
+------+-------------+-----------------------------+-------+--------------------------+------------------+---------+------------------------------------+---------+-------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+------+-------------+-----------------------------+-------+--------------------------+------------------+---------+------------------------------------+---------+-------------+
| 1 | SIMPLE | users | index | PRIMARY,tmp_idx_users_id | tmp_idx_users_id | 4 | NULL | 8675689 | Using index |
| 1 | SIMPLE | store_customer_associations | ref | user_id_store_id | user_id_store_id | 10 | trustvox_production.users.id,const | 1 | Using index |
+------+-------------+-----------------------------+-------+--------------------------+------------------+---------+------------------------------------+---------+-------------+
2 rows in set (0.00 sec)
mysql>
UPDATE 3
mysql> EXPLAIN SELECT COUNT(*)
FROM users
INNER JOIN
( SELECT *
FROM store_customer_associations
WHERE store_id = 75856
) sca ON users.id = sca.user_id;
+------+-------------+-----------------------------+--------+-------------------------------------------------------------------------------------------------------------+-----------------------------------------------+---------+---------------------------------------------------------+---------+-------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+------+-------------+-----------------------------+--------+-------------------------------------------------------------------------------------------------------------+-----------------------------------------------+---------+---------------------------------------------------------+---------+-------------+
| 1 | SIMPLE | store_customer_associations | ref | index_store_customer_associations_on_store_id,index_store_customer_associations_on_user_id,user_id_store_id | index_store_customer_associations_on_store_id | 5 | const | 4401812 | Using where |
| 1 | SIMPLE | users | eq_ref | PRIMARY,tmp_idx_users_id | PRIMARY | 4 | trustvox_production.store_customer_associations.user_id | 1 | Using index |
+------+-------------+-----------------------------+--------+-------------------------------------------------------------------------------------------------------------+-----------------------------------------------+---------+---------------------------------------------------------+---------+-------------+
UPDATE 4
mysql> ALTER TABLE store_customer_associations DROP INDEX index_store_customer_associations_on_store_id;
Query OK, 0 rows affected (0.01 sec)
Records: 0 Duplicates: 0 Warnings: 0
mysql> ALTER TABLE store_customer_associations DROP INDEX index_store_customer_associations_on_user_id;
Query OK, 0 rows affected (0.02 sec)
Records: 0 Duplicates: 0 Warnings: 0
mysql> CREATE INDEX index_on_store_id_and_user_id ON store_customer_associations (store_id, user_id);
Query OK, 0 rows affected (45.95 sec)
Records: 0 Duplicates: 0 Warnings: 0
mysql> CREATE INDEX index_store_customer_associations_on_user_id ON store_customer_associations(user_id);
Query OK, 0 rows affected (33.06 sec)
Records: 0 Duplicates: 0 Warnings: 0
mysql> CREATE INDEX index_store_customer_associations_on_store_id ON store_customer_associations(store_id);
Query OK, 0 rows affected (38.58 sec)
Records: 0 Duplicates: 0 Warnings: 0
mysql> SHOW INDEXES FROM store_customer_associations;
+-----------------------------+------------+-----------------------------------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment |
+-----------------------------+------------+-----------------------------------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| store_customer_associations | 0 | PRIMARY | 1 | id | A | 10305620 | NULL | NULL | | BTREE | | |
| store_customer_associations | 1 | index_on_store_id_and_user_id | 1 | store_id | A | 8244 | NULL | NULL | YES | BTREE | | |
| store_customer_associations | 1 | index_on_store_id_and_user_id | 2 | user_id | A | 10305620 | NULL | NULL | YES | BTREE | | |
| store_customer_associations | 1 | index_store_customer_associations_on_user_id | 1 | user_id | A | 10305620 | NULL | NULL | YES | BTREE | | |
| store_customer_associations | 1 | index_store_customer_associations_on_store_id | 1 | store_id | A | 6424 | NULL | NULL | YES | BTREE | | |
+-----------------------------+------------+-----------------------------------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
5 rows in set (0.00 sec)
store_customer_associations: INDEX(store_id, user_id) -- "covering"
store_id is first -- to handle the WHERE; user_id is added so that the only the index need by touched.
A tutorial that lets you discover the INDEX yourself: http://mysql.rjweb.org/doc.php/index_cookbook_mysql
U need to make index for 2 parameters for store_curstometer_associations because in query index is not because where part.
create index user_id_store_id on store_customer_associations (user_id, store_id)
Should help u out
Example
mysql> show indexes from api_plexts;
+------------+------------+----------------------------------------+--------------+--------------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment |
+------------+------------+----------------------------------------+--------------+--------------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| api_plexts | 0 | PRIMARY | 1 | id | A | 206318 | NULL | NULL | | BTREE | | |
| api_plexts | 0 | index_api_plexts_on_guid | 1 | guid | A | 215475 | NULL | NULL | YES | BTREE | | |
| api_plexts | 1 | index_api_plexts_on_api_agent_id | 1 | api_agent_id | A | 1565 | NULL | NULL | YES | BTREE | | |
| api_plexts | 1 | index_api_plexts_on_from_api_portal_id | 1 | from_api_portal_id | A | 11401 | NULL | NULL | YES | BTREE | | |
| api_plexts | 1 | index_api_plexts_on_to_api_portal_id | 1 | to_api_portal_id | A | 9566 | NULL | NULL | YES | BTREE | | |
+------------+------------+----------------------------------------+--------------+--------------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
5 rows in set (0.00 sec)
mysql> show indexes from api_portals;
+-------------+------------+--------------------------------------+--------------+-----------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment |
+-------------+------------+--------------------------------------+--------------+-----------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| api_portals | 0 | PRIMARY | 1 | id | A | 20530 | NULL | NULL | | BTREE | | |
| api_portals | 0 | index_api_portals_on_guid | 1 | guid | A | 19891 | NULL | NULL | YES | BTREE | | |
| api_portals | 1 | index_api_portals_on_owner_agent_id | 1 | owner_agent_id | A | 718 | NULL | NULL | YES | BTREE | | |
| api_portals | 1 | index_api_portals_on_api_map_tile_id | 1 | api_map_tile_id | A | 5480 | NULL | NULL | YES | BTREE | | |
+-------------+------------+--------------------------------------+--------------+-----------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
explain select count(*) from api_portals ap JOIN api_plexts al FORCE INDEX FOR JOIN (PRIMARY) on ap.id = al.from_api_portal_id where al.api_agent_id = 2\G;
mysql> show indexes from api_plexts;
+------------+------------+----------------------------------------+--------------+--------------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment |
+------------+------------+----------------------------------------+--------------+--------------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| api_plexts | 0 | PRIMARY | 1 | id | A | 206318 | NULL | NULL | | BTREE | | |
| api_plexts | 0 | index_api_plexts_on_guid | 1 | guid | A | 215475 | NULL | NULL | YES | BTREE | | |
| api_plexts | 1 | index_api_plexts_on_api_agent_id | 1 | api_agent_id | A | 1565 | NULL | NULL | YES | BTREE | | |
| api_plexts | 1 | index_api_plexts_on_from_api_portal_id | 1 | from_api_portal_id | A | 11401 | NULL | NULL | YES | BTREE | | |
| api_plexts | 1 | index_api_plexts_on_to_api_portal_id | 1 | to_api_portal_id | A | 9566 | NULL | NULL | YES | BTREE | | |
+------------+------------+----------------------------------------+--------------+--------------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
5 rows in set (0.00 sec)
mysql> show indexes from api_portals;
+-------------+------------+--------------------------------------+--------------+-----------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment |
+-------------+------------+--------------------------------------+--------------+-----------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| api_portals | 0 | PRIMARY | 1 | id | A | 20530 | NULL | NULL | | BTREE | | |
| api_portals | 0 | index_api_portals_on_guid | 1 | guid | A | 19891 | NULL | NULL | YES | BTREE | | |
| api_portals | 1 | index_api_portals_on_owner_agent_id | 1 | owner_agent_id | A | 718 | NULL | NULL | YES | BTREE | | |
| api_portals | 1 | index_api_portals_on_api_map_tile_id | 1 | api_map_tile_id | A | 5480 | NULL | NULL | YES | BTREE | | |
+-------------+------------+--------------------------------------+--------------+-----------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
mysql> explain select count(*) from api_portals ap JOIN api_plexts al on ap.id = al.from_api_portal_id where al.api_agent_id = 2;
+----+-------------+-------+------------+--------+-------------------------------------------------------------------------+----------------------------------+---------+-------------------------------+------+----------+--------------------------+
| id | select_type | table | partitions | type | possible_keys | key | key_len | ref | rows | filtered | Extra |
+----+-------------+-------+------------+--------+-------------------------------------------------------------------------+----------------------------------+---------+-------------------------------+------+----------+--------------------------+
| 1 | SIMPLE | al | NULL | ref | index_api_plexts_on_api_agent_id,index_api_plexts_on_from_api_portal_id | index_api_plexts_on_api_agent_id | 5 | const | 271 | 100.00 | Using where |
| 1 | SIMPLE | ap | NULL | eq_ref | PRIMARY | PRIMARY | 8 | ingress.al.from_api_portal_id | 1 | 100.00 | Using where; Using index |
+----+-------------+-------+------------+--------+-------------------------------------------------------------------------+----------------------------------+---------+-------------------------------+------+----------+--------------------------+
this is exactly your case U can see using where
mysql> create index testime on api_plexts (api_agent_id, from_api_portal_id); Query OK, 0 rows affected (3.71 sec)
Records: 0 Duplicates: 0 Warnings: 0
mysql> show indexes from api_plexts;
+------------+------------+----------------------------------------+--------------+--------------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment |
+------------+------------+----------------------------------------+--------------+--------------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| api_plexts | 0 | PRIMARY | 1 | id | A | 200644 | NULL | NULL | | BTREE | | |
| api_plexts | 0 | index_api_plexts_on_guid | 1 | guid | A | 209549 | NULL | NULL | YES | BTREE | | |
| api_plexts | 1 | index_api_plexts_on_api_agent_id | 1 | api_agent_id | A | 1522 | NULL | NULL | YES | BTREE | | |
| api_plexts | 1 | index_api_plexts_on_from_api_portal_id | 1 | from_api_portal_id | A | 11087 | NULL | NULL | YES | BTREE | | |
| api_plexts | 1 | index_api_plexts_on_to_api_portal_id | 1 | to_api_portal_id | A | 9303 | NULL | NULL | YES | BTREE | | |
| api_plexts | 1 | testime | 1 | api_agent_id | A | 1817 | NULL | NULL | YES | BTREE | | |
| api_plexts | 1 | testime | 2 | from_api_portal_id | A | 52277 | NULL | NULL | YES | BTREE | | |
+------------+------------+----------------------------------------+--------------+--------------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
7 rows in set (0.00 sec)
VOILA!!!
mysql> explain select count(*) from api_portals ap JOIN api_plexts al on ap.id = al.from_api_portal_id where al.api_agent_id = 2\G; *************************** 1. row ***************************
id: 1
select_type: SIMPLE
table: al
partitions: NULL
type: ref
possible_keys: index_api_plexts_on_api_agent_id,index_api_plexts_on_from_api_portal_id,testime
key: testime
key_len: 5
ref: const
rows: 271
filtered: 100.00
Extra: Using where; Using index
*************************** 2. row ***************************
id: 1
select_type: SIMPLE
table: ap
partitions: NULL
type: eq_ref
possible_keys: PRIMARY
key: PRIMARY
key_len: 8
ref: ingress.al.from_api_portal_id
rows: 1
filtered: 100.00
Extra: Using where; Using index
2 rows in set, 1 warning (0.00 sec)
I need your help optimizing a query. One table is a log table which has millions of entries and I try to break my query to < 1s. My query should give an overall overview and should be quick therefore. I'm sure I could make multiple simple queries over the list with help of a script. But what a script can do, can mysql I think - I hope at least. And maybe not all parts are best used, but I'm stuck in a query which makes a temporary table and filesort (which I found out is really bad). As reading around I found out to use some neat and well placed indexes but now I'm stuck in a specific point.
Let me show you my final query with its results:
SELECT
ps.SERVER_ID,
ps.FULLNAME,
SUM(CASE WHEN pml.ID_TYPE = 3 THEN 1 ELSE 0 END) 'amount_warning',
SUM(CASE WHEN pml.ID_TYPE = 4 THEN 1 ELSE 0 END) 'amount_error',
SUM(CASE WHEN pml.ID_TYPE = 5 THEN 1 ELSE 0 END) 'amount_alert',
SUM(CASE WHEN pml.ID_TYPE = 7 THEN 1 ELSE 0 END) 'amount_critical'
FROM
PAR_SERVER ps
INNER JOIN
PAR_MONITORINGv2_LOG pml ON ps.SERVER_ID = pml.SERVER_ID
WHERE
pml.CREATED_DATE > date_sub( NOW( ) , INTERVAL 7 DAY )
GROUP BY
ps.SERVER_ID;
Here is what I get:
mysql> [thequeryabove]
[...]
59 rows in set (11.69 sec)
mysql> explain [thequeryabove]
+----+-------------+-------+--------+-----------------------------+---------+---------+---------------------------+---------+----------------------------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------+--------+-----------------------------+---------+---------+---------------------------+---------+----------------------------------------------+
| 1 | SIMPLE | pml | ALL | SERVER_ID,SERVER_ID-ID_TYPE | NULL | NULL | NULL | 4014447 | Using where; Using temporary; Using filesort |
| 1 | SIMPLE | ps | eq_ref | PRIMARY | PRIMARY | 4 | database.pml.SERVER_ID | 1 | |
+----+-------------+-------+--------+-----------------------------+---------+---------+---------------------------+---------+----------------------------------------------+
2 rows in set (0.00 sec)
Here's my current table setup:
mysql> describe PAR_SERVER;
+----------------+--------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+----------------+--------------+------+-----+---------+----------------+
| SERVER_ID | int(255) | NO | PRI | NULL | auto_increment |
| FULLNAME | varchar(255) | YES | | NULL | |
| SHORTNAME | varchar(255) | YES | MUL | NULL | |
+----------------+--------------+------+-----+---------+----------------+
3 rows in set (0.00 sec)
mysql> show indexes from PAR_SERVER;
+------------+------------+-----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment |
+------------+------------+-----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| PAR_SERVER | 0 | PRIMARY | 1 | SERVER_ID | A | 142 | NULL | NULL | | BTREE | |
| PAR_SERVER | 1 | shortname | 1 | SHORTNAME | A | 142 | NULL | NULL | YES | BTREE | |
+------------+------------+-----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
2 rows in set (0.00 sec)
mysql> select count(*) from PAR_SERVER;
+----------+
| count(*) |
+----------+
| 142 |
+----------+
1 row in set (0.00 sec)
mysql> describe PAR_MONITORINGv2_LOG;
+--------------+----------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+--------------+----------+------+-----+---------+----------------+
| ID | int(11) | NO | PRI | NULL | auto_increment |
| ID_TYPE | int(11) | NO | MUL | NULL | |
| ID_SERVICE | int(11) | NO | MUL | NULL | |
| SERVER_ID | int(11) | NO | MUL | NULL | |
| MESSAGE | tinytext | NO | | NULL | |
| CREATED_DATE | datetime | NO | | NULL | |
+--------------+----------+------+-----+---------+----------------+
6 rows in set (0.00 sec)
mysql> show indexes from PAR_MONITORINGv2_LOG;
+----------------------+------------+-------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment |
+----------------------+------------+-------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| PAR_MONITORINGv2_LOG | 0 | PRIMARY | 1 | ID | A | 3998188 | NULL | NULL | | BTREE | |
| PAR_MONITORINGv2_LOG | 1 | ID_TYPE | 1 | ID_TYPE | A | 7 | NULL | NULL | | BTREE | |
| PAR_MONITORINGv2_LOG | 1 | ID_SERVICE | 1 | ID_SERVICE | A | 5 | NULL | NULL | | BTREE | |
| PAR_MONITORINGv2_LOG | 1 | SERVER_ID | 1 | SERVER_ID | A | 66 | NULL | NULL | | BTREE | |
| PAR_MONITORINGv2_LOG | 1 | SERVER_ID-ID_TYPE | 1 | SERVER_ID | A | 66 | NULL | NULL | | BTREE | |
| PAR_MONITORINGv2_LOG | 1 | SERVER_ID-ID_TYPE | 2 | ID_TYPE | A | 258 | NULL | NULL | | BTREE | |
+----------------------+------------+-------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
6 rows in set (0.00 sec)
mysql> select count(*) from PAR_MONITORINGv2_LOG;
+----------+
| count(*) |
+----------+
| 3998386 |
+----------+
1 row in set (0.00 sec)
Here are time results breaking my query step by step down. I may going step by step up after fixing each part taking so long. But for now only the query with runtime of 2.30 sec is currently interesting for this question.
mysql> SELECT ps.SERVER_ID, ps.FULLNAME FROM PAR_SERVER ps INNER JOIN PAR_MONITORINGv2_LOG pml ON ps.SERVER_ID = pml.SERVER_ID WHERE pml.CREATED_DATE > date_sub( NOW( ) , INTERVAL 7 DAY ) GROUP BY ps.SERVER_ID;
[...]
59 rows in set (6.41 sec)
mysql> explain [thequeryabove]
+----+-------------+-------+--------+-----------------------------+---------+---------+---------------------------+---------+----------------------------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------+--------+-----------------------------+---------+---------+---------------------------+---------+----------------------------------------------+
| 1 | SIMPLE | pml | ALL | SERVER_ID,SERVER_ID-ID_TYPE | NULL | NULL | NULL | 4014788 | Using where; Using temporary; Using filesort |
| 1 | SIMPLE | ps | eq_ref | PRIMARY | PRIMARY | 4 | database.pml.SERVER_ID | 1 | |
+----+-------------+-------+--------+-----------------------------+---------+---------+---------------------------+---------+----------------------------------------------+
2 rows in set (0.00 sec)
mysql> SELECT ps.SERVER_ID, ps.FULLNAME FROM PAR_SERVER ps INNER JOIN PAR_MONITORINGv2_LOG pml ON ps.SERVER_ID = pml.SERVER_ID GROUP BY ps.SERVER_ID;
[...]
59 rows in set (2.30 sec)
mysql> explain [thequeryabove]
+----+-------------+-------+--------+-----------------------------+-----------+---------+---------------------------+---------+----------------------------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------+--------+-----------------------------+-----------+---------+---------------------------+---------+----------------------------------------------+
| 1 | SIMPLE | pml | index | SERVER_ID,SERVER_ID-ID_TYPE | SERVER_ID | 4 | NULL | 4015694 | Using index; Using temporary; Using filesort |
| 1 | SIMPLE | ps | eq_ref | PRIMARY | PRIMARY | 4 | database.pml.SERVER_ID | 1 | |
+----+-------------+-------+--------+-----------------------------+-----------+---------+---------------------------+---------+----------------------------------------------+
2 rows in set (0.00 sec)
mysql> SELECT pml.SERVER_ID FROM PAR_MONITORINGv2_LOG pml GROUP BY pml.SERVER_ID;
[...]
65 rows in set (0.00 sec)
mysql> explain [thequeryabove]
+----+-------------+-------+-------+---------------+-----------+---------+------+------+--------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------+-------+---------------+-----------+---------+------+------+--------------------------+
| 1 | SIMPLE | pml | range | NULL | SERVER_ID | 4 | NULL | 67 | Using index for group-by |
+----+-------------+-------+-------+---------------+-----------+---------+------+------+--------------------------+
1 row in set (0.00 sec)
I was able to improve the query a lot by defining an index for (SERVER_ID, ID_TYPE) as my following example query confirms:
mysql> SELECT count(*) 'count_warnings' FROM PAR_MONITORINGv2_LOG pml WHERE pml.SERVER_ID = 191 AND pml.ID_TYPE = 3 GROUP BY pml.SERVER_ID;
[...]
1 row in set (0.01 sec)
mysql> explain [thequeryabove]
+----+-------------+-------+------+-------------------------------------+-------------------+---------+-------------+-------+-------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------+------+-------------------------------------+-------------------+---------+-------------+-------+-------------+
| 1 | SIMPLE | pml | ref | ID_TYPE,SERVER_ID,SERVER_ID-ID_TYPE | SERVER_ID-ID_TYPE | 8 | const,const | 10254 | Using index |
+----+-------------+-------+------+-------------------------------------+-------------------+---------+-------------+-------+-------------+
1 row in set (0.00 sec)
I'm stuck now in the most broked down query with a long execution time of 2.30 sec. I don't know how to use indexes for such a query not having any where clause.
Your query will definitely benefit the most from adding composite index on PAR_MONITORINGv2_LOG(CREATED_DATE, SERVER_ID,ID_TYPE). However, I suggest even simple index on CREATED_DATE will improve performance a lot.
Here is the first table 'tbl1':
+---------+---------------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+---------+---------------------+------+-----+---------+----------------+
| val | varchar(45) | YES | MUL | NULL | |
| id | bigint(20) unsigned | NO | PRI | NULL | auto_increment |
+---------+---------------------+------+-----+---------+----------------+
With its indexes:
+-------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment |
+-------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| tbl1 | 0 | PRIMARY | 1 | id | A | 201826018 | NULL | NULL | | BTREE | |
| tbl1 | 1 | val | 1 | val | A | 2147085 | NULL | NULL | YES | BTREE | |
| tbl1 | 1 | id_val | 1 | id | A | 201826018 | NULL | NULL | | BTREE | |
| tbl1 | 1 | id_val | 2 | val | A | 201826018 | NULL | NULL | YES | BTREE | |
| tbl1 | 1 | val_id | 1 | val | A | 2147085 | NULL | NULL | YES | BTREE | |
| tbl1 | 1 | val_id | 2 | id | A | 201826018 | NULL | NULL | | BTREE | |
+-------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
(The reason for some extra indexing is this: http://bit.ly/KWx1Xz.)
The second table is just about the same. Here are its index cardinalities though:
+--------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment |
+--------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| tbl2 | 0 | PRIMARY | 1 | id | A | 201826018 | NULL | NULL | | BTREE | |
| tbl2 | 1 | val | 1 | val | A | 881336 | NULL | NULL | YES | BTREE | |
| tbl2 | 1 | id_val | 1 | id | A | 201826018 | NULL | NULL | | BTREE | |
| tbl2 | 1 | id_val | 2 | val | A | 201826018 | NULL | NULL | YES | BTREE | |
| tbl2 | 1 | val_id | 1 | val | A | 881336 | NULL | NULL | YES | BTREE | |
| tbl2 | 1 | val_id | 2 | id | A | 201826018 | NULL | NULL | | BTREE | |
+--------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
The task is to inner-join them on the val column and get the id's list (and to do it in 1 second).
Here is the 'join' approach:
SELECT tbl1.id FROM tbl1 JOIN tbl2 ON tbl1.val = 'iii' AND tbl2.val = 'iii' AND tbl1.id = tbl2.id;
Result: 10831 rows in set (55.15 sec)
Query explain:
+----+-------------+--------+--------+----------------------------------+---------+---------+---------------------------+------+--------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+--------+--------+----------------------------------+---------+---------+---------------------------+------+--------------------------+
| 1 | SIMPLE | tbl1 | ref | PRIMARY,val,id_val,val_id | val_id | 138 | const | 5160 | Using where; Using index |
| 1 | SIMPLE | tbl2 | eq_ref | PRIMARY,val,id_val,val_id | PRIMARY | 8 | search_test.tbl1.id | 1 | Using where |
+----+-------------+--------+--------+----------------------------------+---------+---------+---------------------------+------+--------------------------+
And here is the 'in' approach:
SELECT id FROM tbl1 WHERE val = 'iii' and id IN (SELECT id FROM tbl2 WHERE val = 'iii');
Result: 10831 rows in set (1 min 10.15 sec)
Explain:
+----+--------------------+--------+-----------------+---------------------------------+---------+---------+-------+------+--------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+--------------------+--------+-----------------+---------------------------------+---------+---------+-------+------+--------------------------+
| 1 | PRIMARY | tbl1 | ref | val,val_id | val_id | 138 | const | 8553 | Using where; Using index |
| 2 | DEPENDENT SUBQUERY | tbl2 | unique_subquery | PRIMARY,val,id_val,val_id | PRIMARY | 8 | func | 1 | Using where |
+----+--------------------+--------+-----------------+---------------------------------+---------+---------+-------+------+--------------------------+
So, here is the question: how to tweak this query to let MySQL accomplish it in a second?
OK I have tested this on 30,000+ records per table and it runs pretty darn quick.
As it currently stands you're performing a join on two massive tables presently but if you scan for matches on 'val' on each table first that will reduce the size of your join sets substantially.
I originally posted this answer as a set of subqueries but I didn't realize that MySQL is painfully slow at nested subqueries since it executes from the outside in. However if you define the subqueries as views it runs them from the inside out.
So, first create the views.
CREATE VIEW tbl1_iii AS (
SELECT * FROM tbl1 WHERE val='iii'
);
CREATE VIEW tbl2_iii AS (
SELECT * FROM tbl2 WHERE val='iii'
);
Then run the query.
SELECT tbl1_iii.id from tbl1_iii,tbl2_iii
WHERE tbl1_iii.id = tbl2_iii.id;
Lightning.
SELECT tbl1.id FROM tbl1 JOIN tbl2 ON tbl1.id = tbl2.id and tbl1.val = tbl2.val
where tbl1.val = 'iii';
I found a very weird mysql behaviour : when I run a specific query twice, the explain of this query is different the second time :
query = SELECT `twstats_twwordstrend`.`id`, `twstats_twwordstrend`.`created`, `twstats_twwordstrend`.`freq`, `twstats_twwordstrend`.`word_id` FROM `twstats_twwordstrend` INNER JOIN `twstats_twwords` ON (`twstats_twwordstrend`.`word_id` = `twstats_twwords`.`id`) WHERE (`twstats_twwords`.`name` = '#ladygaga' AND `twstats_twwordstrend`.`created` > '2011-01-28 01:30:19' );
1st query execution and then run explain :
mysql> EXPLAIN SELECT `twstats_twwordstrend`.`id`, `twstats_twwordstrend`.`created`, `twstats_twwordstrend`.`freq`, `twstats_twwordstrend`.`word_id` FROM `twstats_twwordstrend` INNER JOIN `twstats_twwords` ON (`twstats_twwordstrend`.`word_id` = `twstats_twwords`.`id`) WHERE (`twstats_twwords`.`name` = '#ladygaga' AND `twstats_twwordstrend`.`created` > '2011-01-28 01:30:19' );
+----+-------------+----------------------+--------+-------------------------------+---------+---------+-------------------------------------------+---------+-------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+----------------------+--------+-------------------------------+---------+---------+-------------------------------------------+---------+-------------+
| 1 | SIMPLE | twstats_twwordstrend | ALL | twstats_twwordstrend_4b95d890 | NULL | NULL | NULL | 4877401 | Using where |
| 1 | SIMPLE | twstats_twwords | eq_ref | PRIMARY | PRIMARY | 4 | statweestics.twstats_twwordstrend.word_id | 1 | Using where |
+----+-------------+----------------------+--------+-------------------------------+---------+---------+-------------------------------------------+---------+-------------+
2 rows in set (0.00 sec)
2nd query execution and then run explain :
mysql> EXPLAIN SELECT `twstats_twwordstrend`.`id`, `twstats_twwordstrend`.`created`, `twstats_twwordstrend`.`freq`, `twstats_twwordstrend`.`word_id` FROM `twstats_twwordstrend` INNER JOIN `twstats_twwords` ON (`twstats_twwordstrend`.`word_id` = `twstats_twwords`.`id`) WHERE (`twstats_twwords`.`name` = '#ladygaga' AND `twstats_twwordstrend`.`created` > '2011-01-28 01:30:19' );
+----+-------------+----------------------+------+-------------------------------+-------------------------------+---------+---------------------------------+--------+-------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+----------------------+------+-------------------------------+-------------------------------+---------+---------------------------------+--------+-------------+
| 1 | SIMPLE | twstats_twwords | ALL | PRIMARY | NULL | NULL | NULL | 222994 | Using where |
| 1 | SIMPLE | twstats_twwordstrend | ref | twstats_twwordstrend_4b95d890 | twstats_twwordstrend_4b95d890 | 4 | statweestics.twstats_twwords.id | 15 | Using where |
+----+-------------+----------------------+------+-------------------------------+-------------------------------+---------+---------------------------------+--------+-------------+
2 rows in set (0.00 sec)
mysql> describe twstats_twwords;
+---------+--------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+---------+--------------+------+-----+---------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
| created | datetime | NO | | NULL | |
| name | varchar(140) | NO | | NULL | |
+---------+--------------+------+-----+---------+----------------+
3 rows in set (0.00 sec)
mysql> describe twstats_twwordstrend;
+---------+----------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+---------+----------+------+-----+---------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
| created | datetime | NO | | NULL | |
| freq | double | NO | | NULL | |
| word_id | int(11) | NO | MUL | NULL | |
+---------+----------+------+-----+---------+----------------+
4 rows in set (0.00 sec)
How this can be possible ??
Look at the rows column. The engine was able to gather more statistics -- so the next time it will try to use the better plan.
Happy coding.
I need to find the best index for this query:
SELECT c.id id, type
FROM Content c USE INDEX (type_proc_last_cat)
LEFT JOIN Battles b ON c.id = b.id
WHERE type = 1
AND processing_status = 1
AND category IN (13, 19)
AND status = 4
ORDER BY last_change DESC
LIMIT 100";
The tables look like this:
mysql> describe Content;
+-------------------+---------------------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+-------------------+---------------------+------+-----+---------+-------+
| id | bigint(20) unsigned | NO | PRI | NULL | |
| type | tinyint(3) unsigned | NO | MUL | NULL | |
| category | bigint(20) unsigned | NO | | NULL | |
| processing_status | tinyint(3) unsigned | NO | | NULL | |
| last_change | int(10) unsigned | NO | | NULL | |
+-------------------+---------------------+------+-----+---------+-------+
mysql> show indexes from Content;
+---------+------------+---------------------+--------------+-------------------+-----------+-------------+----------+--------+------+------------+---------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment |
+---------+------------+---------------------+--------------+-------------------+-----------+-------------+----------+--------+------+------------+---------+
| Content | 0 | PRIMARY | 1 | id | A | 4115 | NULL | NULL | | BTREE | |
| Content | 1 | type_proc_last_cat | 1 | type | A | 4 | NULL | NULL | | BTREE | |
| Content | 1 | type_proc_last_cat | 2 | processing_status | A | 20 | NULL | NULL | | BTREE | |
| Content | 1 | type_proc_last_cat | 3 | last_change | A | 4115 | NULL | NULL | | BTREE | |
| Content | 1 | type_proc_last_cat | 4 | category | A | 4115 | NULL | NULL | | BTREE | |
+---------+------------+---------------------+--------------+-------------------+-----------+-------------+----------+--------+------+------------+---------+
mysql> describe Battles;
+---------------------+---------------------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+---------------------+---------------------+------+-----+---------+-------+
| id | bigint(20) unsigned | NO | PRI | NULL | |
| status | tinyint(4) unsigned | NO | | NULL | |
| status_last_changed | int(11) unsigned | NO | | NULL | |
+---------------------+---------------------+------+-----+---------+-------+
mysql> show indexes from Battles;
+---------+------------+-----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment |
+---------+------------+-----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| Battles | 0 | PRIMARY | 1 | id | A | 1215 | NULL | NULL | | BTREE | |
| Battles | 0 | id_status | 1 | id | A | 1215 | NULL | NULL | | BTREE | |
| Battles | 0 | id_status | 2 | status | A | 1215 | NULL | NULL | | BTREE | |
+---------+------------+-----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
And I get output like this:
mysql> explain
-> SELECT c.id id, type
-> FROM Content c USE INDEX (type_proc_last_cat)
-> LEFT JOIN Battles b USE INDEX (id_status) ON c.id = b.id
-> WHERE type = 1
-> AND processing_status = 1
-> AND category IN (13, 19)
-> AND status = 4
-> ORDER BY last_change DESC
-> LIMIT 100;
+----+-------------+-------+--------+--------------------+--------------------+---------+-----------------------+------+--------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------+--------+--------------------+--------------------+---------+-----------------------+------+--------------------------+
| 1 | SIMPLE | c | ref | type_proc_last_cat | type_proc_last_cat | 2 | const,const | 1352 | Using where; Using index |
| 1 | SIMPLE | b | eq_ref | id_status | id_status | 9 | wtm_master.c.id,const | 1 | Using where; Using index |
+----+-------------+-------+--------+--------------------+--------------------+---------+-----------------------+------+--------------------------+
The trouble is the rows count for the Content table. It appears MySQL is unable to effectively use both last_change and category in the type_proc_last_cat index. If I switch the order of last_change and category, fewer rows are selected but it results in a filesort for the ORDER BY, meaning it pulls all the matching rows from the database. That is worse, since there are 100,000+ rows in both tables.
Tables are both InnoDB, so keep in mind that the PRIMARY key is appended to every other index. So the index the index type_proc_last_cat above behaves likes it's on (type, processing_status, last_change, category, id). I am aware I could change the PRIMARY key for Battles to (id, status) and drop the id_status index (and I may just do that).
Edit: Any value for type, category, processing_status, and status is less than 20% of the total values. last_change and status_last_change are unix timestamps.
Edit: If I use a different index with category and last_change in reverse order, I get this:
mysql> show indexes from Content;
+---------+------------+---------------------+--------------+-------------------+-----------+-------------+----------+--------+------+------------+---------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment |
+---------+------------+---------------------+--------------+-------------------+-----------+-------------+----------+--------+------+------------+---------+
| Content | 0 | PRIMARY | 1 | id | A | 4115 | NULL | NULL | | BTREE | |
| Content | 1 | type_proc_cat_last | 1 | type | A | 6 | NULL | NULL | | BTREE | |
| Content | 1 | type_proc_cat_last | 2 | processing_status | A | 26 | NULL | NULL | | BTREE | |
| Content | 1 | type_proc_cat_last | 3 | category | A | 228 | NULL | NULL | | BTREE | |
| Content | 1 | type_proc_cat_last | 4 | last_change | A | 4115 | NULL | NULL | | BTREE | |
+---------+------------+---------------------+--------------+-------------------+-----------+-------------+----------+--------+------+------------+---------+
mysql> explain SELECT c.id id, type FROM Content c USE INDEX (type_proc_cat_last) LEFT JOIN Battles b
USE INDEX (id_status) ON c.id = b.id WHERE type = 1 AND processing_status = 1 AND category IN (13, 19) AND status = 4 ORDER BY last_change DESC LIMIT 100;
+----+-------------+-------+-------+--------------------+--------------------+---------+-----------------------+------+------------------------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------+-------+--------------------+--------------------+---------+-----------------------+------+------------------------------------------+
| 1 | SIMPLE | c | range | type_proc_cat_last | type_proc_cat_last | 10 | NULL | 165 | Using where; Using index; Using filesort |
| 1 | SIMPLE | b | ref | id_status | id_status | 9 | wtm_master.c.id,const | 1 | Using where; Using index |
+----+-------------+-------+-------+--------------------+--------------------+---------+-----------------------+------+------------------------------------------+
The filesort worries me as it tells me MySQL pulls all the matching rows first, before sorting. This will be a big problem when there are 100,000+.
rows field in EXPLAIN doesn't reflect the number of rows that actually were read. It reflects the number of rows that possible will be affected. Also it doesn't rely on LIMIT, because LIMIT is applied after the plan has been calculated.
So you don't need to worry about it.
Also I would suggest you to swap last_change and category in type_proc_last_cat so mysql can try to use the last index part (last_change) for sorting purposes.