I have a mysql table voucher_codes with the following schema:
mysql> describe voucher_codes;
+--------------------+------------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+--------------------+------------------+------+-----+---------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
| user_id | int(10) unsigned | YES | MUL | NULL | |
| promo_code | varchar(30) | YES | MUL | NULL | |
| voucher_value | double | NO | | NULL | |
| date_created | datetime(6) | NO | | NULL | |
| date_modified | datetime(6) | NO | | NULL | |
| redeem_flag | tinyint(1) | NO | | NULL | |
| fk_voucher_rule_id | int(11) | NO | MUL | NULL | |
+--------------------+------------------+------+-----+---------+----------------+
The indexes defined on the table are as follows:
mysql> show index from voucher_codes;
+---------------+------------+--------------------------+--------------+--------------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment |
+---------------+------------+--------------------------+--------------+--------------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| voucher_codes | 0 | PRIMARY | 1 | id | A | 15280692 | NULL | NULL | | BTREE | | |
| voucher_codes | 1 | voucher_codes_user_id | 1 | user_id | A | 2805369 | NULL | NULL | YES | BTREE | | |
| voucher_codes | 1 | voucher_codes_promo_code | 1 | promo_code | A | 7389780 | NULL | NULL | YES | BTREE | | |
| voucher_codes | 1 | fk_voucher_rule_id | 1 | fk_voucher_rule_id | A | 60 | NULL | NULL | | BTREE | | |
+---------------+------------+--------------------------+--------------+--------------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
The numbers of records with voucher_code 1 and redeem_flag as 0 and 1 are as follows:
mysql> select count(*) from voucher_codes where redeem_flag =0 and fk_voucher_rule_id=1;
+----------+
| count(*) |
+----------+
| 135114 |
+----------+
1 row in set (3.17 sec)
mysql> select count(*) from voucher_codes where redeem_flag =1 and fk_voucher_rule_id=1;
+----------+
| count(*) |
+----------+
| 1575024 |
+----------+
1 row in set (2.56 sec)
There are 1.5+ million records with redeem_flag as 1 and 100k+ are 0.
But the query with redeem_flag as 1 takes less time as compared with query with redeem_flag 0.
Below are the results:
mysql> select promo_code from voucher_codes where redeem_flag =0 and fk_voucher_rule_id=1 limit 1 ;
+--------------+
| promo_code |
+--------------+
| XXXXXXXXXX |
+--------------+
1 row in set (3.67 sec)
mysql> explain select promo_code from voucher_codes where redeem_flag =0 and fk_voucher_rule_id=1 limit 1 ;
+----+-------------+---------------+------------+------+--------------------+--------------------+---------+-------+---------+----------+-------------+
| id | select_type | table | partitions | type | possible_keys | key | key_len | ref | rows | filtered | Extra |
+----+-------------+---------------+------------+------+--------------------+--------------------+---------+-------+---------+----------+-------------+
| 1 | SIMPLE | voucher_codes | NULL | ref | fk_voucher_rule_id | fk_voucher_rule_id | 4 | const | 3258352 | 10.00 | Using where |
+----+-------------+---------------+------------+------+--------------------+--------------------+---------+-------+---------+----------+-------------+
1 row in set, 1 warning (0.00 sec)
Query with redeem_flag as 1:
mysql> select promo_code from voucher_codes where redeem_flag =1 and fk_voucher_rule_id=1 limit 1 ;
+------------+
| promo_code |
+------------+
| XXXXXXXXX |
+------------+
1 row in set (0.00 sec)
mysql> explain select promo_code from voucher_codes where redeem_flag =1 and fk_voucher_rule_id=1 limit 1 ;
+----+-------------+---------------+------------+------+--------------------+--------------------+---------+-------+---------+----------+-------------+
| id | select_type | table | partitions | type | possible_keys | key | key_len | ref | rows | filtered | Extra |
+----+-------------+---------------+------------+------+--------------------+--------------------+---------+-------+---------+----------+-------------+
| 1 | SIMPLE | voucher_codes | NULL | ref | fk_voucher_rule_id | fk_voucher_rule_id | 4 | const | 3258352 | 10.00 | Using where |
+----+-------------+---------------+------------+------+--------------------+--------------------+---------+-------+---------+----------+-------------+
1 row in set, 1 warning (0.00 sec)
Unable to understand this behaviour. Even though the index fk_voucher_rule_id is getting used, but it is taking more time with records which are more as compared to less number of records.
Update: Adding show create table result:
mysql> show create table voucher_codes;
+---------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Table | Create Table |
+---------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| voucher_codes | CREATE TABLE `voucher_codes` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`user_id` int(10) unsigned DEFAULT NULL,
`promo_code` varchar(30) DEFAULT NULL,
`voucher_value` double NOT NULL,
`date_created` datetime(6) NOT NULL,
`date_modified` datetime(6) NOT NULL,
`redeem_flag` tinyint(1) NOT NULL,
`fk_voucher_rule_id` int(11) NOT NULL,
PRIMARY KEY (`id`),
KEY `voucher_codes_user_id` (`user_id`),
KEY `voucher_codes_promo_code` (`promo_code`),
KEY `fk_voucher_rule_id` (`fk_voucher_rule_id`)
) ENGINE=InnoDB AUTO_INCREMENT=16305657 DEFAULT CHARSET=latin1 |
+---------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
1 row in set (0.00 sec)
First of all, test speed of your queries using SELECT SQL_NO_CACHE ... instead of just SELECT .... It is quite odd that query runs in 0.00 time.
Furthermore, create a composite index on redeem_flag, fk_voucher_rule and then retest your queries for comparison.
NOTE: You should use same order of fields in a query as it is in your index. Also, as #Raymod Nijland said, remove redundant indices. For example, if you have index fk_voucher_rule, redeem_flag it will be used both for queries that use only fk_voucher_rule and fk_voucher_rule, redeem_flag queries (other way around does not apply, you can't use it only for querying redeem_flag).
Related
I am creating a table that has a unique key constraint and I performing a select query on that table. MySQL does not use the index for the unique key for querying that. Why is that happening?
CREATE TABLE foo (
column1 int NOT NULL,
column2 int NOT NULL,
UNIQUE KEY idx_column1_column2 (column1, column2)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
mysql> explain select * from foo where column1=1 and column2=1;
+----+-------------+-------+------------+------+---------------+------+---------+------+------+----------+--------------------------------+
| id | select_type | table | partitions | type | possible_keys | key | key_len | ref | rows | filtered | Extra |
+----+-------------+-------+------------+------+---------------+------+---------+------+------+----------+--------------------------------+
| 1 | SIMPLE | NULL | NULL | NULL | NULL | NULL | NULL | NULL | NULL | NULL | no matching row in const table |
+----+-------------+-------+------------+------+---------------+------+---------+------+------+----------+--------------------------------+
1 row in set, 1 warning (0.00 sec)
mysql> show index from foo;
+-------+------------+---------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+---------+------------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment | Visible | Expression |
+-------+------------+---------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+---------+------------+
| foo | 0 | idx_column1_column2 | 1 | column1 | A | 0 | NULL | NULL | | BTREE | | | YES | NULL |
| foo | 0 | idx_column1_column2 | 2 | column2 | A | 0 | NULL | NULL | | BTREE | | | YES | NULL |
+-------+------------+---------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+---------+------------+
2 rows in set (0.01 sec)
I would expect here that the idx_column1_column2 index be used for my select query but it's not why is that happening, how can I get it to use that index automatically?
I'm facing and weird issue with a query, I never saw something like this in more than 10 years developing.
A InnoDB table with 8M of rows, this table is used to store references between 2 other tables (pages and keywords)
| Field | Type | Null | Key | Default | Extra |
+-----------------+---------------------+------+-----+---------------------+----------------+
| id | int(11) unsigned | NO | PRI | NULL | auto_increment |
| page_id | int(11) unsigned | NO | PRI | 0 | |
| keyword_id | int(11) unsigned | NO | PRI | 0 | |
| avg_kc | tinyint(3) | YES | | NULL | |
| rank_on_page | tinyint(2) | YES | | NULL | |
| created_at | timestamp | NO | | CURRENT_TIMESTAMP | |
| updated_at | timestamp | NO | | 0000-00-00 00:00:00 | |
+-----------------+---------------------+------+-----+---------------------+----------------+
The issue is, if you use SELECT * returns an Empty set
mysql> select * from pages_metrics where keyword_id=2385 and page_id=6004 ;
Empty set (0.00 sec)
But, if you select only some fields the query runs well.
mysql> select id, page_id, keyword_id from pages_metrics where keyword_id=2385 and page_id=6004;
+---------+---------+------------+
| id | page_id | keyword_id |
+---------+---------+------------+
| 8469199 | 6004 | 2385 |
+---------+---------+------------+
1 row in set (0.00 sec)
I run the analyze table and all looks good.
mysql> analyze table pages_metrics;
+------------------------+---------+----------+----------+
| Table | Op | Msg_type | Msg_text |
+------------------------+---------+----------+----------+
| keywords.pages_metrics | analyze | status | OK |
+------------------------+---------+----------+----------+
1 row in set (0.34 sec)
Any idea why is this happening?
Thanks in advance.
I have succesfully created a database in mySQL using the commandline and imported some data. It currently looks like this..
desc data;
+----------+-------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+----------+-------------+------+-----+---------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
| code | varchar(10) | YES | | NULL | |
+----------+-------------+------+-----+---------+----------------+
SELECT * FROM data;
+----+----------+
| id | code |
+----+----------+
| 1 | 123abc
| 2 | 234def
| 3 | 567ghi
| 4 | 890jkl
I would like to add a column to the table called timestamp, I am doing this with..
alter table data add timestamp VARCHAR(20);
But then my table looks like this...
desc data;
+-----------+-------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-----------+-------------+------+-----+---------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
| code | varchar(10) | YES | | NULL | |
| timestamp | varchar(20) | YES | | NULL | |
+-----------+-------------+------+-----+---------+----------------+
SELECT * FROM data;
+----+----------+-----------+
| id | code | timestamp |
+----+----------+-----------+
| NULL |
| NULL |
| NULL |
| NULL |
Where am I going wrong?
here you can see the backticks
alter table `data` add `timestamp` VARCHAR(20);
SAMPLE
MariaDB []> desc data;
+-------+----------------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-------+----------------------+------+-----+---------+----------------+
| id | int(11) unsigned | NO | PRI | NULL | auto_increment |
| e | enum('x1','x2','x3') | YES | | NULL | |
+-------+----------------------+------+-----+---------+----------------+
2 rows in set (0.01 sec)
MariaDB []> alter table `data` add `timestamp` VARCHAR(20);
Query OK, 0 rows affected (0.05 sec)
Records: 0 Duplicates: 0 Warnings: 0
MariaDB []> desc data;
+-----------+----------------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-----------+----------------------+------+-----+---------+----------------+
| id | int(11) unsigned | NO | PRI | NULL | auto_increment |
| e | enum('x1','x2','x3') | YES | | NULL | |
| timestamp | varchar(20) | YES | | NULL | |
+-----------+----------------------+------+-----+---------+----------------+
3 rows in set (0.01 sec)
Table Data
MariaDB [who]> select * from `data`;
+----+------+-----------+
| id | e | timestamp |
+----+------+-----------+
| 1 | x1 | NULL |
| 2 | x2 | NULL |
+----+------+-----------+
2 rows in set (0.00 sec)
MariaDB [who]>
This question already has answers here:
SELECT vs UPDATE performance with index
(7 answers)
Closed 8 years ago.
I'm trying to improve my query so that it doesn't take so long. Is there anything I can try?
I'm using InnoDB.
My table:
mysql> describe hunted_place_review_external_urls;
+--------------+--------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+--------------+--------------+------+-----+---------+----------------+
| id | bigint(20) | NO | PRI | NULL | auto_increment |
| worker_id | varchar(255) | YES | MUL | NULL | |
| queued_at | bigint(20) | YES | MUL | NULL | |
| external_url | varchar(255) | NO | | NULL | |
| place_id | varchar(63) | NO | MUL | NULL | |
| source_id | varchar(63) | NO | | NULL | |
| successful | tinyint(1) | NO | | 0 | |
+--------------+--------------+------+-----+---------+----------------+
mysql> show index from hunted_place_review_external_urls;
+-----------------------------------+------------+--------------------------------------------+--------------+--------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment |
+-----------------------------------+------------+--------------------------------------------+--------------+--------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| hunted_place_review_external_urls | 0 | PRIMARY | 1 | id | A | 5118685 | NULL | NULL | | BTREE | | |
| hunted_place_review_external_urls | 1 | worker_id | 1 | worker_id | A | 5118685 | NULL | NULL | YES | BTREE | | |
| hunted_place_review_external_urls | 1 | queued_at | 1 | queued_at | A | 5118685 | NULL | NULL | YES | BTREE | | |
| hunted_place_review_external_urls | 1 | worker_id_and_queued_at | 1 | worker_id | A | 5118685 | NULL | NULL | YES | BTREE | | |
| hunted_place_review_external_urls | 1 | worker_id_and_queued_at | 2 | queued_at | A | 5118685 | NULL | NULL | YES | BTREE | | |
| hunted_place_review_external_urls | 1 | place_id_source_id_external_url_successful | 1 | place_id | A | 5118685 | NULL | NULL | | BTREE | | |
| hunted_place_review_external_urls | 1 | place_id_source_id_external_url_successful | 2 | source_id | A | 5118685 | NULL | NULL | | BTREE | | |
| hunted_place_review_external_urls | 1 | place_id_source_id_external_url_successful | 3 | external_url | A | 5118685 | NULL | NULL | | BTREE | | |
| hunted_place_review_external_urls | 1 | place_id_source_id_external_url_successful | 4 | successful | A | 5118685 | NULL | NULL | | BTREE | | |
+-----------------------------------+------------+--------------------------------------------+--------------+--------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
My query:
mysql> select count(*) from hunted_place_review_external_urls;
+----------+
| count(*) |
+----------+
| 4217356 |
+----------+
1 row in set (0.96 sec)
mysql> select count(*) from hunted_place_review_external_urls where worker_id is null;
+----------+
| count(*) |
+----------+
| 772626 |
+----------+
1 row in set (0.27 sec)
mysql> update hunted_place_review_external_urls set worker_id = "123" where worker_id is null order by queued_at asc limit 1;
Query OK, 1 row affected (4.80 sec)
Rows matched: 1 Changed: 1 Warnings: 0
Why is the update query taking 4s even though I have both single and composite index on queued_at and worker_id? This never happened before when the number of rows with worker_id = null was much lower. With ~200k rows instead of 780k rows, it would only take a few milliseconds.
Note the equivalent query with SELECT instead of UPDATE is extremely fast:
mysql> select * from hunted_place_review_external_urls where worker_id is null order by queued_at asc limit 1;
1 row in set (0.00 sec)
My queued_at values are timestamps expressed in number of milliseconds, such as 1398210069531
I've tried dropping my single indices on worker_id and queued_at but the problem remains:
mysql> drop index queued_at on hunted_place_review_external_urls;
Query OK, 0 rows affected (3.75 sec)
mysql> drop index worker_id on hunted_place_review_external_urls;
Query OK, 0 rows affected (3.75 sec)
mysql> show index from hunted_place_review_external_urls;
+-----------------------------------+------------+--------------------------------------------+--------------+--------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment |
+-----------------------------------+------------+--------------------------------------------+--------------+--------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| hunted_place_review_external_urls | 0 | PRIMARY | 1 | id | A | 5118685 | NULL | NULL | | BTREE | | |
| hunted_place_review_external_urls | 1 | worker_id_and_queued_at | 1 | worker_id | A | 5118685 | NULL | NULL | YES | BTREE | | |
| hunted_place_review_external_urls | 1 | worker_id_and_queued_at | 2 | queued_at | A | 5118685 | NULL | NULL | YES | BTREE | | |
| hunted_place_review_external_urls | 1 | place_id_source_id_external_url_successful | 1 | place_id | A | 5118685 | NULL | NULL | | BTREE | | |
| hunted_place_review_external_urls | 1 | place_id_source_id_external_url_successful | 2 | source_id | A | 5118685 | NULL | NULL | | BTREE | | |
| hunted_place_review_external_urls | 1 | place_id_source_id_external_url_successful | 3 | external_url | A | 5118685 | NULL | NULL | | BTREE | | |
| hunted_place_review_external_urls | 1 | place_id_source_id_external_url_successful | 4 | successful | A | 5118685 | NULL | NULL | | BTREE | | |
+-----------------------------------+------------+--------------------------------------------+--------------+--------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
Here's my EXPLAIN SELECT statement. I'm using an old version of MYSQL that doesn't support EXPLAIN UPDATE:
mysql> explain select * from hunted_place_review_external_urls where worker_id is null order by queued_at asc limit 1;
+----+-------------+-----------------------------------+------+-------------------------+-------------------------+---------+-------+---------+-------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-----------------------------------+------+-------------------------+-------------------------+---------+-------+---------+-------------+
| 1 | SIMPLE | hunted_place_review_external_urls | ref | worker_id_and_queued_at | worker_id_and_queued_at | 768 | const | 1587282 | Using where |
+----+-------------+-----------------------------------+------+-------------------------+-------------------------+---------+-------+---------+-------------+
1 row in set (0.00 sec)
This is your query:
update hunted_place_review_external_urls
set worker_id = "123"
where worker_id is null
order by queued_at asc
limit 1;
It first has to find the row use for updating, and that requires applying the where clause and the order by clause. Either it does all the work (scanning the table and then sorting), or it uses an index. The proper index would be hunted_place_review_external_urls(worker_id, queued_at). You can add more columns at the end, but these must be the first two columns, and in that order.
EDIT:
Given that the select is fast, try this version:
update hunted_place_review_external_urls toupdate join
(select
from hunted_place_review_external_urls
where worker_id is null
order by queued_at asc
limit 1
) l
on toupdate.id = l.id
set toupdate.worker_id = '123';
I'm not sure why the indexes would be used properly here but not in the update, but hopefully this will work.
Compare the Time of Drop Index, Create Index and update.
You may notice a correlation.
When you are simple performing SELECT Queries, indexes are USEFUL, cause they speed things up.
When you are performing UPDATE or DELETE Statements - Indexes are bad, cause they slow things down! Whenever you change a value of an indexed column, MySQL needs to rebuild the index for any subsequent row. (assuming you are ALWAYS fetching the oldest entry - that means: Reindex ALL reamaining 772625 Rows.)
Try to drop the index on worker_id and see the Update Performance. If worker_id is not indexed, the update will be way faster. (Finding the entry to update is still as fast as before, because it mainly depends on the sorting performed on the indexed column queued_at and a very small subset of unindexed null values on the worker_id, matching the desired queued_at value)
I just created a dummy db and tested your setup:
With 1.000.000 Rows and both - the single index on worker_id and the composite index on worker_id|queued at, a select looks like:
SELECT * FROM `tbl` WHERE ISNULL( worker_ID ) ORDER BY queued_at ASC LIMIT 1
and the performance:
Query took 0.3360 sec
Trying to modify the worker_id in the way you do, results in:
UPDATE `tbl` SET worker_id=1 WHERE ISNULL(worker_ID) ORDER BY queued_at ASC LIMIT 1
performance:
1 row affected. (Query took 7.9592 sec)
Droping both the indexes (single and composite) on worker_id, then the same query results in:
1 row affected. (Query took 1.4364 sec)
(generated 50.000 rows per insert, so they have the same date, so the index is not "perfect" for searching, so "real" data may perform way better.)
I need your help optimizing a query. One table is a log table which has millions of entries and I try to break my query to < 1s. My query should give an overall overview and should be quick therefore. I'm sure I could make multiple simple queries over the list with help of a script. But what a script can do, can mysql I think - I hope at least. And maybe not all parts are best used, but I'm stuck in a query which makes a temporary table and filesort (which I found out is really bad). As reading around I found out to use some neat and well placed indexes but now I'm stuck in a specific point.
Let me show you my final query with its results:
SELECT
ps.SERVER_ID,
ps.FULLNAME,
SUM(CASE WHEN pml.ID_TYPE = 3 THEN 1 ELSE 0 END) 'amount_warning',
SUM(CASE WHEN pml.ID_TYPE = 4 THEN 1 ELSE 0 END) 'amount_error',
SUM(CASE WHEN pml.ID_TYPE = 5 THEN 1 ELSE 0 END) 'amount_alert',
SUM(CASE WHEN pml.ID_TYPE = 7 THEN 1 ELSE 0 END) 'amount_critical'
FROM
PAR_SERVER ps
INNER JOIN
PAR_MONITORINGv2_LOG pml ON ps.SERVER_ID = pml.SERVER_ID
WHERE
pml.CREATED_DATE > date_sub( NOW( ) , INTERVAL 7 DAY )
GROUP BY
ps.SERVER_ID;
Here is what I get:
mysql> [thequeryabove]
[...]
59 rows in set (11.69 sec)
mysql> explain [thequeryabove]
+----+-------------+-------+--------+-----------------------------+---------+---------+---------------------------+---------+----------------------------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------+--------+-----------------------------+---------+---------+---------------------------+---------+----------------------------------------------+
| 1 | SIMPLE | pml | ALL | SERVER_ID,SERVER_ID-ID_TYPE | NULL | NULL | NULL | 4014447 | Using where; Using temporary; Using filesort |
| 1 | SIMPLE | ps | eq_ref | PRIMARY | PRIMARY | 4 | database.pml.SERVER_ID | 1 | |
+----+-------------+-------+--------+-----------------------------+---------+---------+---------------------------+---------+----------------------------------------------+
2 rows in set (0.00 sec)
Here's my current table setup:
mysql> describe PAR_SERVER;
+----------------+--------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+----------------+--------------+------+-----+---------+----------------+
| SERVER_ID | int(255) | NO | PRI | NULL | auto_increment |
| FULLNAME | varchar(255) | YES | | NULL | |
| SHORTNAME | varchar(255) | YES | MUL | NULL | |
+----------------+--------------+------+-----+---------+----------------+
3 rows in set (0.00 sec)
mysql> show indexes from PAR_SERVER;
+------------+------------+-----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment |
+------------+------------+-----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| PAR_SERVER | 0 | PRIMARY | 1 | SERVER_ID | A | 142 | NULL | NULL | | BTREE | |
| PAR_SERVER | 1 | shortname | 1 | SHORTNAME | A | 142 | NULL | NULL | YES | BTREE | |
+------------+------------+-----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
2 rows in set (0.00 sec)
mysql> select count(*) from PAR_SERVER;
+----------+
| count(*) |
+----------+
| 142 |
+----------+
1 row in set (0.00 sec)
mysql> describe PAR_MONITORINGv2_LOG;
+--------------+----------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+--------------+----------+------+-----+---------+----------------+
| ID | int(11) | NO | PRI | NULL | auto_increment |
| ID_TYPE | int(11) | NO | MUL | NULL | |
| ID_SERVICE | int(11) | NO | MUL | NULL | |
| SERVER_ID | int(11) | NO | MUL | NULL | |
| MESSAGE | tinytext | NO | | NULL | |
| CREATED_DATE | datetime | NO | | NULL | |
+--------------+----------+------+-----+---------+----------------+
6 rows in set (0.00 sec)
mysql> show indexes from PAR_MONITORINGv2_LOG;
+----------------------+------------+-------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment |
+----------------------+------------+-------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| PAR_MONITORINGv2_LOG | 0 | PRIMARY | 1 | ID | A | 3998188 | NULL | NULL | | BTREE | |
| PAR_MONITORINGv2_LOG | 1 | ID_TYPE | 1 | ID_TYPE | A | 7 | NULL | NULL | | BTREE | |
| PAR_MONITORINGv2_LOG | 1 | ID_SERVICE | 1 | ID_SERVICE | A | 5 | NULL | NULL | | BTREE | |
| PAR_MONITORINGv2_LOG | 1 | SERVER_ID | 1 | SERVER_ID | A | 66 | NULL | NULL | | BTREE | |
| PAR_MONITORINGv2_LOG | 1 | SERVER_ID-ID_TYPE | 1 | SERVER_ID | A | 66 | NULL | NULL | | BTREE | |
| PAR_MONITORINGv2_LOG | 1 | SERVER_ID-ID_TYPE | 2 | ID_TYPE | A | 258 | NULL | NULL | | BTREE | |
+----------------------+------------+-------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
6 rows in set (0.00 sec)
mysql> select count(*) from PAR_MONITORINGv2_LOG;
+----------+
| count(*) |
+----------+
| 3998386 |
+----------+
1 row in set (0.00 sec)
Here are time results breaking my query step by step down. I may going step by step up after fixing each part taking so long. But for now only the query with runtime of 2.30 sec is currently interesting for this question.
mysql> SELECT ps.SERVER_ID, ps.FULLNAME FROM PAR_SERVER ps INNER JOIN PAR_MONITORINGv2_LOG pml ON ps.SERVER_ID = pml.SERVER_ID WHERE pml.CREATED_DATE > date_sub( NOW( ) , INTERVAL 7 DAY ) GROUP BY ps.SERVER_ID;
[...]
59 rows in set (6.41 sec)
mysql> explain [thequeryabove]
+----+-------------+-------+--------+-----------------------------+---------+---------+---------------------------+---------+----------------------------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------+--------+-----------------------------+---------+---------+---------------------------+---------+----------------------------------------------+
| 1 | SIMPLE | pml | ALL | SERVER_ID,SERVER_ID-ID_TYPE | NULL | NULL | NULL | 4014788 | Using where; Using temporary; Using filesort |
| 1 | SIMPLE | ps | eq_ref | PRIMARY | PRIMARY | 4 | database.pml.SERVER_ID | 1 | |
+----+-------------+-------+--------+-----------------------------+---------+---------+---------------------------+---------+----------------------------------------------+
2 rows in set (0.00 sec)
mysql> SELECT ps.SERVER_ID, ps.FULLNAME FROM PAR_SERVER ps INNER JOIN PAR_MONITORINGv2_LOG pml ON ps.SERVER_ID = pml.SERVER_ID GROUP BY ps.SERVER_ID;
[...]
59 rows in set (2.30 sec)
mysql> explain [thequeryabove]
+----+-------------+-------+--------+-----------------------------+-----------+---------+---------------------------+---------+----------------------------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------+--------+-----------------------------+-----------+---------+---------------------------+---------+----------------------------------------------+
| 1 | SIMPLE | pml | index | SERVER_ID,SERVER_ID-ID_TYPE | SERVER_ID | 4 | NULL | 4015694 | Using index; Using temporary; Using filesort |
| 1 | SIMPLE | ps | eq_ref | PRIMARY | PRIMARY | 4 | database.pml.SERVER_ID | 1 | |
+----+-------------+-------+--------+-----------------------------+-----------+---------+---------------------------+---------+----------------------------------------------+
2 rows in set (0.00 sec)
mysql> SELECT pml.SERVER_ID FROM PAR_MONITORINGv2_LOG pml GROUP BY pml.SERVER_ID;
[...]
65 rows in set (0.00 sec)
mysql> explain [thequeryabove]
+----+-------------+-------+-------+---------------+-----------+---------+------+------+--------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------+-------+---------------+-----------+---------+------+------+--------------------------+
| 1 | SIMPLE | pml | range | NULL | SERVER_ID | 4 | NULL | 67 | Using index for group-by |
+----+-------------+-------+-------+---------------+-----------+---------+------+------+--------------------------+
1 row in set (0.00 sec)
I was able to improve the query a lot by defining an index for (SERVER_ID, ID_TYPE) as my following example query confirms:
mysql> SELECT count(*) 'count_warnings' FROM PAR_MONITORINGv2_LOG pml WHERE pml.SERVER_ID = 191 AND pml.ID_TYPE = 3 GROUP BY pml.SERVER_ID;
[...]
1 row in set (0.01 sec)
mysql> explain [thequeryabove]
+----+-------------+-------+------+-------------------------------------+-------------------+---------+-------------+-------+-------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------+------+-------------------------------------+-------------------+---------+-------------+-------+-------------+
| 1 | SIMPLE | pml | ref | ID_TYPE,SERVER_ID,SERVER_ID-ID_TYPE | SERVER_ID-ID_TYPE | 8 | const,const | 10254 | Using index |
+----+-------------+-------+------+-------------------------------------+-------------------+---------+-------------+-------+-------------+
1 row in set (0.00 sec)
I'm stuck now in the most broked down query with a long execution time of 2.30 sec. I don't know how to use indexes for such a query not having any where clause.
Your query will definitely benefit the most from adding composite index on PAR_MONITORINGv2_LOG(CREATED_DATE, SERVER_ID,ID_TYPE). However, I suggest even simple index on CREATED_DATE will improve performance a lot.