I have two tables:
mysql> desc myps3t_gameusertrophyinfo;
+-----------+--------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-----------+--------------+------+-----+---------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
| user_id | int(11) | NO | MUL | NULL | |
| trophy_id | int(11) | NO | MUL | NULL | |
| date | datetime | NO | MUL | NULL | |
| date_read | varchar(100) | NO | | NULL | |
+-----------+--------------+------+-----+---------+----------------+
5 rows in set (0.19 sec)
mysql> show index from myps3t_gameusertrophyinfo;
+---------------------------+------------+------------------------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment |
+---------------------------+------------+------------------------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| myps3t_gameusertrophyinfo | 0 | PRIMARY | 1 | id | A | 4004589 | NULL | NULL | | BTREE | |
| myps3t_gameusertrophyinfo | 0 | user_id | 1 | user_id | A | 7686 | NULL | NULL | | BTREE | |
| myps3t_gameusertrophyinfo | 0 | user_id | 2 | trophy_id | A | 4004589 | NULL | NULL | | BTREE | |
| myps3t_gameusertrophyinfo | 1 | myps3t_gameusertrophyinfo_403f60f | 1 | user_id | A | 7686 | NULL | NULL | | BTREE | |
| myps3t_gameusertrophyinfo | 1 | myps3t_gameusertrophyinfo_61a683d8 | 1 | trophy_id | A | 22624 | NULL | NULL | | BTREE | |
| myps3t_gameusertrophyinfo | 1 | idx_gameusertrophyinfo_date | 1 | date | A | 4004589 | NULL | NULL | | BTREE | |
+---------------------------+------------+------------------------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
7 rows in set (0.06 sec)
the other table:
mysql> desc myps3t_gametrophyinfo ;
+-----------+--------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-----------+--------------+------+-----+---------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
| game_id | int(11) | NO | MUL | NULL | |
| name | varchar(500) | NO | | NULL | |
| desc | varchar(500) | NO | | NULL | |
| type | varchar(20) | NO | | NULL | |
| pic_url | varchar(200) | NO | | NULL | |
| desc_pt | varchar(500) | NO | | NULL | |
| name_pt | varchar(500) | NO | | NULL | |
| hidden_id | int(11) | NO | | NULL | |
| total_id | int(11) | NO | | NULL | |
| trophy_id | int(11) | NO | | NULL | |
| addon_id | int(11) | YES | | NULL | |
| points | double | NO | | 0 | |
| sony_id | int(11) | YES | | NULL | |
+-----------+--------------+------+-----+---------+----------------+
14 rows in set (0.00 sec)
mysql> show index from myps3t_gametrophyinfo;
+-----------------------+------------+-------------------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment |
+-----------------------+------------+-------------------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| myps3t_gametrophyinfo | 0 | PRIMARY | 1 | id | A | 25976 | NULL | NULL | | BTREE | |
| myps3t_gametrophyinfo | 1 | myps3t_gametrophyinfo_game_id | 1 | game_id | A | 764 | NULL | NULL | | BTREE | |
+-----------------------+------------+-------------------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
when i do this query:
mysql> explain select * from myps3t_gameusertrophyinfo a, myps3t_gametrophyinfo b where a.trophy_id = b.id and b.addon_id = 58; +----+-------------+-------+--------+--------------------------------------------------------------+---------+---------+-----------------------------+---------+-------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------+--------+--------------------------------------------------------------+---------+---------+-----------------------------+---------+-------------+
| 1 | SIMPLE | a | ALL | myps3t_gameusertrophyinfo_61a683d8,myps3t_gameusertrophyinfo | NULL | NULL | NULL | 4004592 | |
| 1 | SIMPLE | b | eq_ref | PRIMARY | PRIMARY | 4 | fabriciols_ps3t.a.trophy_id | 1 | Using where |
+----+-------------+-------+--------+--------------------------------------------------------------+---------+---------+-----------------------------+---------+-------------+
2 rows in set (0.00 sec)
why mysql do not use my keys ?
this query take more than 30 seconds, the first table has 4milion records ...
-- edit --
for quasnoi
mysql> SELECT COUNT(*), COUNT(DISTINCT addon_id), SUM(addon_id = 58) FROM myps3t_gametrophyinfo;
+----------+--------------------------+--------------------+
| COUNT(*) | COUNT(DISTINCT addon_id) | SUM(addon_id = 58) |
+----------+--------------------------+--------------------+
| 25976 | 160 | 6 |
+----------+--------------------------+--------------------+
1 row in set (0.00 sec)
MySQL chooses a as a leading table and b as a driven table. It does use a PRIMARY KEY on b for the joins.
Create an index on myps3t_gametrophyinfo (addon_id), this way b will be more probably chosen as a leading table.
You can try
select * from
myps3t_gametrophyinfo b
STRAIGHT_JOIN myps3t_gameusertrophyinfo a ON (a.trophy_id = b.id)
WHERE b.addon_id = 58;
I would probably rewrite the query to try to get a more sane execution path. I think something like below is more likely to get you the performance you want and is more clear in what you are doing to a human reader
SELECT * FROM myps3t_gametrophyinfo a LEFT JOIN myps3t_gameusertrophyinfo b ON a.id = b.trophy_id WHERE a.addon_id=58;
Related
im stuck with a query which take more than 13 secs to return output
i did create the indexes but the execution plan doesnt seem to use them. Below are some information regarding
Query
SELECT userinfo.username,userinfo.firstname,userinfo.lastname,userinfo.email,
radcheck.attribute,radcheck.value,radusergroup.groupname, userinfo.id,
userinfo.workphone,userinfo.homephone,
userinfo.mobilephone,userinfo.address, userinfo.zone,
userinfo.account_state,userinfo.device_owner,userinfo.link_type,userinfo.account_type
FROM userinfo
INNER JOIN radcheck ON userinfo.username = radcheck.username
INNER JOIN radusergroup ON userinfo.username = radusergroup.username
WHERE radcheck.attribute='Expiration' and
STR_TO_DATE(radcheck.value, '%d %M %Y') < CURDATE()
and radusergroup.groupname='Customer30M';
Explain output
+----+-------------+--------------+------+--------------------+-----------+---------+--------------------------------+------+-----------------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+--------------+------+--------------------+-----------+---------+--------------------------------+------+-----------------------------------+
| 1 | SIMPLE | radusergroup | ref | username,groupname | groupname | 66 | const | 5 | Using where with pushed condition |
| 1 | SIMPLE | radcheck | ref | username,attribute | attribute | 34 | const | 9 | Using where |
| 1 | SIMPLE | userinfo | ref | username | username | 131 | ctradius.radusergroup.username | 10 | Using where |
+----+-------------+--------------+------+--------------------+-----------+---------+--------------------------------+------+-----------------------------------+
3 rows in set (0.00 sec)
Table Info
Table: radusergroup
+-----------+-------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-----------+-------------+------+-----+---------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
| username | varchar(64) | NO | MUL | | |
| groupname | varchar(64) | NO | MUL | | |
| priority | int(11) | NO | | 1 | |
+-----------+-------------+------+-----+---------+----------------+
radusergroup table Index
+--------------+------------+-------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment |
+--------------+------------+-------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| radusergroup | 0 | PRIMARY | 1 | id | A | 11292 | NULL | NULL | | BTREE | | |
| radusergroup | 1 | username | 1 | username | A | NULL | NULL | NULL | | BTREE | | |
| radusergroup | 1 | groupname | 1 | groupname | A | NULL | NULL | NULL | | BTREE | | |
| radusergroup | 1 | indexradusergroup | 1 | username | A | NULL | NULL | NULL | | BTREE | | |
| radusergroup | 1 | indexradusergroup | 2 | groupname | A | NULL | NULL | NULL | | BTREE | | |
Table : userinfo
---------------------+------------------+------+-----+---------------------+----------------+
| Field | Type | Null | Key | Default | Extra |
+---------------------+------------------+------+-----+---------------------+----------------+
| id | int(11) unsigned | NO | PRI | NULL | auto_increment |
| username | varchar(128) | YES | MUL | NULL | |
| firstname | varchar(200) | YES | | NULL | |
| lastname | varchar(200) | YES | | NULL | |
| email | varchar(200) | YES | | NULL | |
| workphone | varchar(200) | YES | | NULL | |
| homephone | varchar(200) | YES | | NULL | |
| mobilephone | varchar(200) | YES | | NULL | |
| address | varchar(200) | YES | | NULL | |
| zone | varchar(20) | YES | | NULL | |
| account_state | varchar(10) | YES | | active | |
| change_commit | tinyint(1) | YES | | NULL | |
| link_type | varchar(8) | YES | | NULL | |
| account_type | varchar(255) | YES | | NULL | |
| device_owner | varchar(255) | YES | | NULL | |
| account_name | varchar(255) | YES | | NULL | |
| account_email | varchar(255) | YES | | NULL | |
| account_mobile | varchar(255) | YES | | NULL | |
| groupname | varchar(255) | YES | | NULL | |
+---------------------+------------------+------+-----+---------------------+----------------+
Indexes on userinfo
----------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment |
+----------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| userinfo | 0 | PRIMARY | 1 | id | A | 11282 | NULL | NULL | | BTREE | | |
| userinfo | 1 | username | 1 | username | A | NULL | NULL | NULL | YES | BTREE | | |
+----------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
Table: Radcheck
+-----------+------------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-----------+------------------+------+-----+---------+----------------+
| id | int(11) unsigned | NO | PRI | NULL | auto_increment |
| username | varchar(64) | NO | MUL | | |
| attribute | varchar(32) | NO | MUL | | |
| op | char(2) | NO | | == | |
| value | varchar(253) | NO | MUL | | |
+-----------+------------------+------+-----+---------+----------------+
indexes on radcheck
+----------+------------+---------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment |
+----------+------------+---------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| radcheck | 0 | PRIMARY | 1 | id | A | 33425 | NULL | NULL | | BTREE | | |
| radcheck | 1 | username | 1 | username | A | NULL | NULL | NULL | | BTREE | | |
| radcheck | 1 | value | 1 | value | A | NULL | NULL | NULL | | BTREE | | |
| radcheck | 1 | attribute | 1 | attribute | A | NULL | NULL | NULL | | BTREE | | |
The function invalidates the use of the index. Despite your cardinality showing 11k rows, the table scan ensues and it takes 13 seconds.
Though you have a join between Radcheck and userinfo, it is the STR_TO_DATE function in the where clause that kills the index use.
My feed_listingjob has 2 datetime fields:
+------------+-------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+------------+-------------+------+-----+---------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
| data | longtext | NO | | NULL | |
| meta_data | longtext | NO | | NULL | |
| state | varchar(25) | NO | | NULL | |
| error | longtext | NO | | NULL | |
| job_id | int(11) | NO | MUL | NULL | |
| created_at | datetime(6) | NO | MUL | NULL | |
| updated_at | datetime(6) | NO | MUL | NULL | |
| es_sync_at | datetime(6) | YES | MUL | NULL | |
+------------+-------------+------+-----+---------+----------------+
updated_at and es_sync_at both are indexed individually as below:
mysql> show indexes from feed_listingjob;
+-----------------+------------+--------------------------------------------------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment |
+-----------------+------------+--------------------------------------------------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| feed_listingjob | 0 | PRIMARY | 1 | id | A | 64534 | NULL | NULL | | BTREE | | |
| feed_listingjob | 1 | feed_listingjob_job_id_4c3b1b514481f269_fk_feed_importjob_id | 1 | job_id | A | 2081 | NULL | NULL | | BTREE | | |
| feed_listingjob | 1 | feed_listingjob_fde81f11 | 1 | created_at | A | 64534 | NULL | NULL | | BTREE | | |
| feed_listingjob | 1 | feed_listingjob_afd1a1a8 | 1 | updated_at | A | 64534 | NULL | NULL | | BTREE | | |
| feed_listingjob | 1 | feed_listingjob_381895a2 | 1 | es_sync_at | A | 2 | NULL | NULL | YES | BTREE | | |
+-----------------+------------+--------------------------------------------------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
5 rows in set (0.00 sec)
And my query is not using the indexes:
mysql> explain SELECT `feed_listingjob`.`id` FROM `feed_listingjob` WHERE `feed_listingjob`.`es_sync_at` < `feed_listingjob`.`updated_at` LIMIT 10;
+----+-------------+-----------------+------+---------------+------+---------+------+-------+-------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-----------------+------+---------------+------+---------+------+-------+-------------+
| 1 | SIMPLE | feed_listingjob | ALL | NULL | NULL | NULL | NULL | 53534 | Using where |
+----+-------------+-----------------+------+---------------+------+---------+------+-------+-------------+
1 row in set (0.01 sec)
Can someone please tell me why and how to optimise this query?
The index can't be used here because feed_listingjob.updated_at is not a constant. Which means every row would have to be examined to check the condition. One way to use the index would be to have another column which stores the difference between es_sync_at and updated_at timestamps. If you store it as difference = (es_sync_at - updated_at timestamps) then the query becomes (difference < 0).
running this query on 5Gb sized messages table.
The problem is that the execution takes > 3 minutes.
SELECT m.id FROM messages m
LEFT JOIN dialog d on m.id=d.mid
WHERE (SELECT count(*)
FROM dialog
WHERE (m.from_id=uid1 and m.user_id=uid2)
OR (m.from_id=uid2 and m.user_id=uid1))=0 && read_state=0
LIMIT 100
I understand, that this is a bad practice to search by NESTED SELECT IN WHERE CLAUSE, but yet didn`t find another way to select such rows. Tried to split OR to 2 UNION statements, but it was long either.
messages table structure:
+------------+-------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+------------+-------------+------+-----+---------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
| from_id | int(11) | NO | MUL | NULL | |
| user_id | int(11) | NO | MUL | NULL | |
| group_id | int(11) | NO | | NULL | |
| to_number | varchar(30) | NO | MUL | NULL | |
| msg | text | NO | | NULL | |
| image | varchar(20) | NO | | NULL | |
| date | bigint(20) | NO | | NULL | |
| read_state | tinyint(1) | NO | | 0 | |
| removed | tinyint(1) | NO | | NULL | |
+------------+-------------+------+-----+---------+----------------+
dialog table structure
+-----------+------------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-----------+------------------+------+-----+---------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
| uid1 | int(11) | NO | MUL | NULL | |
| uid2 | int(11) | NO | MUL | NULL | |
| mid | int(11) | NO | | NULL | |
| anonym_id | int(10) unsigned | NO | | NULL | |
+-----------+------------------+------+-----+---------+----------------+
mysql> show index from messages;
+----------+------------+-----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment |
+----------+------------+-----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| messages | 0 | PRIMARY | 1 | id | A | 12560908 | NULL | NULL | | BTREE | | |
| messages | 1 | to_number | 1 | to_number | A | 161037 | NULL | NULL | | BTREE | | |
| messages | 1 | from_id | 1 | from_id | A | 157011 | NULL | NULL | | BTREE | | |
| messages | 1 | from_id | 2 | to_number | A | 169742 | NULL | NULL | | BTREE | | |
| messages | 1 | user_id_2 | 1 | user_id | A | 314022 | NULL | NULL | | BTREE | | |
| messages | 1 | user_id_2 | 2 | read_state | A | 380633 | NULL | NULL | | BTREE | | |
| messages | 1 | user_id_2 | 3 | removed | A | 392528 | NULL | NULL | | BTREE | | |
+----------+------------+-----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
mysql> show index from dialog;
+--------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment |
+--------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| dialog | 0 | PRIMARY | 1 | id | A | 3125615 | NULL | NULL | | BTREE | | |
| dialog | 1 | uid1 | 1 | uid1 | A | 520935 | NULL | NULL | | BTREE | | |
| dialog | 1 | uid1 | 2 | uid2 | A | 3125615 | NULL | NULL | | BTREE | | |
| dialog | 1 | uid2 | 1 | uid2 | A | 1562807 | NULL | NULL | | BTREE | | |
+--------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
EXPLAIN EXTENDED
+----+--------------------+--------+-------+---------------+------+---------+------+----------+----------+--------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | filtered | Extra |
+----+--------------------+--------+-------+---------------+------+---------+------+----------+----------+--------------------------+
| 1 | PRIMARY | m | ALL | NULL | NULL | NULL | NULL | 22190398 | 100.00 | Using where |
| 1 | PRIMARY | d | ALL | NULL | NULL | NULL | NULL | 3125621 | 100.00 | |
| 2 | DEPENDENT SUBQUERY | dialog | index | uid1,uid2 | uid1 | 8 | NULL | 3125621 | 100.00 | Using where; Using index |
+----+--------------------+--------+-------+---------------+------+---------+------+----------+----------+--------------------------+
The first thought is to change the subquery query to use not exists instead of count(*). The second is to split this into two separate subqueries. The third is to add indexes:
create index idx_messages_read_state_4 on messages_read_state(user_id, from_id, user_id, id);
create index idx_dialog_2 on dialog(uid1, uid2)
And the fourth is to remove the left join to dialog in the outer query. No fields from dialog are being used and the left join means it is not being used for filtering.
The query is then:
select m.id
from messages m
where m.read_state = 0 and
not exists (select 1
from dialog d
where m.from_id = d.uid1 and m.user_id = d.uid2
) and
not exists (select 1
from dialog d
where m.from_id = d.uid2 and m.user_id = d.uid1
)
limit 100;
I have this query:
SELECT
s.last_spread, s.sd, s.mean, s.id
,c.id_ticker, c.coef
,t.ticker
,p.last, p.price
FROM (SELECT * FROM spreads WHERE spreads.id_check=1 LIMIT 100,500 ) as s
INNER JOIN coef as c
ON c.id_spread = s.id
INNER JOIN tickers AS t
ON t.id = c.id_ticker
LEFT JOIN (SELECT prices.id_ticker, MAX(prices.date) as last, prices.price FROM prices GROUP BY prices.id_ticker) AS p
ON p.id_ticker = t.id
These are the schemas of the tables:
mysql> desc spreads;
+-------------+---------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-------------+---------+------+-----+---------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
| id_check | int(11) | YES | MUL | NULL | |
| sd | double | YES | | NULL | |
| mean | double | YES | | NULL | |
| last_spread | double | YES | | NULL | |
+-------------+---------+------+-----+---------+----------------+
5 rows in set (0.00 sec)
mysql> desc coef;
+-----------+---------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-----------+---------+------+-----+---------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
| id_spread | int(11) | YES | MUL | NULL | |
| id_ticker | int(11) | YES | | NULL | |
| coef | double | YES | | NULL | |
| side | double | YES | | NULL | |
+-----------+---------+------+-----+---------+----------------+
5 rows in set (0.00 sec)
mysql> desc tickers;
+----------+------------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+----------+------------------+------+-----+---------+----------------+
| id | int(10) unsigned | NO | PRI | NULL | auto_increment |
| ticker | varchar(45) | NO | | NULL | |
| name | varchar(150) | NO | | NULL | |
| category | varchar(150) | NO | | NULL | |
| issuer | varchar(150) | NO | | NULL | |
+----------+------------------+------+-----+---------+----------------+
5 rows in set (0.00 sec)
mysql> desc prices;
+-----------+------------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-----------+------------------+------+-----+---------+----------------+
| id | int(10) unsigned | NO | PRI | NULL | auto_increment |
| id_ticker | int(10) unsigned | NO | MUL | NULL | |
| date | date | NO | | NULL | |
| price | double | NO | | NULL | |
+-----------+------------------+------+-----+---------+----------------+
4 rows in set (0.01 sec)
These are the indexes of the above tables;
mysql> show indexes from spreads;
+---------+------------+-----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment |
+---------+------------+-----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| spreads | 0 | PRIMARY | 1 | id | A | 2299 | NULL | NULL | | BTREE | |
| spreads | 1 | check_idx | 1 | id_check | A | 1 | NULL | NULL | YES | BTREE | |
+---------+------------+-----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
2 rows in set (0.00 sec)
mysql> show indexes from coef;
+-------+------------+-------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment |
+-------+------------+-------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| coef | 0 | PRIMARY | 1 | id | A | 9078 | NULL | NULL | | BTREE | |
| coef | 1 | spread_ticker_idx | 1 | id_spread | A | NULL | NULL | NULL | YES | BTREE | |
| coef | 1 | spread_ticker_idx | 2 | id_ticker | A | NULL | NULL | NULL | YES | BTREE | |
+-------+------------+-------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
3 rows in set (0.00 sec)
mysql> show indexes from tickers;
+---------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment |
+---------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| tickers | 0 | PRIMARY | 1 | id | A | 100 | NULL | NULL | | BTREE | |
+---------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
1 row in set (0.00 sec)
mysql> show indexes from prices;
+--------+------------+-----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment |
+--------+------------+-----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| prices | 0 | PRIMARY | 1 | id | A | 19962 | NULL | NULL | | BTREE | |
| prices | 1 | id_ticker | 1 | id_ticker | A | 19962 | NULL | NULL | | BTREE | |
+--------+------------+-----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
2 rows in set (0.15 sec)
And this is the explain of the query:
+----+-------------+------------+--------+-------------------+-------------------+---------+---------------------------+--------+-------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+------------+--------+-------------------+-------------------+---------+---------------------------+--------+-------------+
| 1 | PRIMARY | <derived2> | ALL | NULL | NULL | NULL | NULL | 500 | |
| 1 | PRIMARY | c | ref | spread_ticker_idx | spread_ticker_idx | 5 | s.id | 90 | Using where |
| 1 | PRIMARY | t | eq_ref | PRIMARY | PRIMARY | 4 | spreadtrading.c.id_ticker | 1 | Using where |
| 1 | PRIMARY | <derived3> | ALL | NULL | NULL | NULL | NULL | 100 | |
| 3 | DERIVED | prices | index | NULL | id_ticker | 4 | NULL | 119774 | |
| 2 | DERIVED | spreads | ref | check_idx | check_idx | 5 | | 2298 | Using where |
+----+-------------+------------+--------+-------------------+-------------------+---------+---------------------------+--------+-------------+
6 rows in set (0.27 sec)
Could I optimize it?
Thank you!
EDIT:
I would like to know if the INDEXES and the table's structure are optimized for the query I posted above. The results that I get using this query are good, it works well, but maybe I can optimize it to increse the "speed" of the query.
I think you may gain something by dropping the spreads subquery and moving the WHERE clause to the end, as in the following code. This loses your LIMIT restriction - perhaps you could put a LIMIT clause at the end as well, depending on what you're trying to achieve in terms of limiting the size of the output.
SELECT
s.last_spread, s.sd, s.mean, s.id
,c.id_ticker, c.coef
,t.ticker
,p.last, p.price
FROM spreads as s
INNER JOIN coef as c
ON c.id_spread = s.id
INNER JOIN tickers AS t
ON t.id = c.id_ticker
LEFT JOIN (SELECT prices.id_ticker, MAX(prices.date) as last, prices.price FROM prices GROUP BY prices.id_ticker) AS p
ON p.id_ticker = t.id
WHERE s.id_check = 1
I'm trying to run the following query:
SELECT formatted_journeys.*, MAX(speed)
FROM formatted_journeys
JOIN tracker.g_log
ON imeiid = vehicle
AND g_logid BETWEEN start_g_log AND end_g_log
GROUP BY id
however, it seems to be extremely slow. Here's the explain for that query.
+----+-------------+--------------------+------+-------------------------------+-------+---------+-----------------------------------+------+----------+---------------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | filtered | Extra |
+----+-------------+--------------------+------+-------------------------------+-------+---------+-----------------------------------+------+----------+---------------------------------+
| 1 | SIMPLE | formatted_journeys | ALL | vehicle,start_g_log,end_g_log | NULL | NULL | NULL | 824 | 100.00 | Using temporary; Using filesort |
| 1 | SIMPLE | g_log | ref | PRIMARY,Dupes | Dupes | 4 | motrak.formatted_journeys.vehicle | 1985 | 100.00 | Using where |
+----+-------------+--------------------+------+-------------------------------+-------+---------+-----------------------------------+------+----------+---------------------------------+
2 rows in set, 1 warning (0.02 sec)
The tables are as follows:
formatted_journeys:
+-----------------+------------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-----------------+------------------+------+-----+---------+----------------+
| id | int(11) unsigned | NO | PRI | NULL | auto_increment |
| start_g_log | int(11) | YES | MUL | NULL | |
| end_g_log | int(11) | YES | MUL | NULL | |
| start_latitude | decimal(18,12) | YES | | NULL | |
| start_longitude | decimal(18,12) | YES | | NULL | |
| end_latitude | decimal(18,12) | YES | | NULL | |
| end_longitude | decimal(18,12) | YES | | NULL | |
| start_location | text | YES | | NULL | |
| end_location | text | YES | | NULL | |
| distance | decimal(10,5) | YES | | NULL | |
| start_date | datetime | YES | | NULL | |
| end_date | datetime | YES | | NULL | |
| vehicle | int(11) | YES | MUL | NULL | |
| private | bit(1) | NO | | b'0' | |
+-----------------+------------------+------+-----+---------+----------------+
Indices:
+--------------------+------------+-------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment |
+--------------------+------------+-------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| formatted_journeys | 0 | PRIMARY | 1 | id | A | 830 | NULL | NULL | | BTREE | |
| formatted_journeys | 1 | vehicle | 1 | vehicle | A | 4 | NULL | NULL | YES | BTREE | |
| formatted_journeys | 1 | start_g_log | 1 | start_g_log | A | 830 | NULL | NULL | YES | BTREE | |
| formatted_journeys | 1 | end_g_log | 1 | end_g_log | A | 830 | NULL | NULL | YES | BTREE | |
+--------------------+------------+-------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
tracker.g_log:
+-----------+------------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-----------+------------------+------+-----+---------+----------------+
| g_logid | int(10) unsigned | NO | PRI | NULL | auto_increment |
| imeiid | int(10) unsigned | NO | MUL | NULL | |
| latitude | decimal(18,12) | YES | | NULL | |
| longitude | decimal(18,12) | YES | | NULL | |
| speed | int(4) | YES | MUL | NULL | |
| bearing | int(4) | YES | | NULL | |
| distance | decimal(10,5) | YES | | NULL | |
| eventcode | int(10) | YES | | NULL | |
| status | int(10) | YES | | NULL | |
| date | datetime | YES | | NULL | |
+-----------+------------------+------+-----+---------+----------------+
Indices:
+-------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment |
+-------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| g_log | 0 | PRIMARY | 1 | g_logid | A | 31760 | NULL | NULL | | BTREE | |
| g_log | 0 | Dupes | 1 | imeiid | A | 16 | NULL | NULL | | BTREE | |
| g_log | 0 | Dupes | 2 | date | A | 31760 | NULL | NULL | YES | BTREE | |
| g_log | 0 | Dupes | 3 | eventcode | A | 31760 | NULL | NULL | YES | BTREE | |
| g_log | 1 | speed | 1 | speed | A | 423 | NULL | NULL | YES | BTREE | |
+-------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
Now I know that the filesort isn't a good thing but how would I get rid of it?
You will hardly get rid of Using temporary; Using filesort, but the query might be faster.
First, try to rewrite the query as:
SELECT *
FROM (
SELECT id, MAX(speed) as max_speed
FROM formatted_journeys
JOIN tracker.g_log
ON imeiid = vehicle
AND g_logid BETWEEN start_g_log AND end_g_log
GROUP BY id) as maxspeeds
JOIN formatted_journeys USING (id);
Then, you can try to force the query to use a coverage index, though that is not easy for the query.
First try: add a composite index (vehicle, start_g_log, end_g_log) and look in explain whether it is used (you should see "Using index" there)