im stuck with a query which take more than 13 secs to return output
i did create the indexes but the execution plan doesnt seem to use them. Below are some information regarding
Query
SELECT userinfo.username,userinfo.firstname,userinfo.lastname,userinfo.email,
radcheck.attribute,radcheck.value,radusergroup.groupname, userinfo.id,
userinfo.workphone,userinfo.homephone,
userinfo.mobilephone,userinfo.address, userinfo.zone,
userinfo.account_state,userinfo.device_owner,userinfo.link_type,userinfo.account_type
FROM userinfo
INNER JOIN radcheck ON userinfo.username = radcheck.username
INNER JOIN radusergroup ON userinfo.username = radusergroup.username
WHERE radcheck.attribute='Expiration' and
STR_TO_DATE(radcheck.value, '%d %M %Y') < CURDATE()
and radusergroup.groupname='Customer30M';
Explain output
+----+-------------+--------------+------+--------------------+-----------+---------+--------------------------------+------+-----------------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+--------------+------+--------------------+-----------+---------+--------------------------------+------+-----------------------------------+
| 1 | SIMPLE | radusergroup | ref | username,groupname | groupname | 66 | const | 5 | Using where with pushed condition |
| 1 | SIMPLE | radcheck | ref | username,attribute | attribute | 34 | const | 9 | Using where |
| 1 | SIMPLE | userinfo | ref | username | username | 131 | ctradius.radusergroup.username | 10 | Using where |
+----+-------------+--------------+------+--------------------+-----------+---------+--------------------------------+------+-----------------------------------+
3 rows in set (0.00 sec)
Table Info
Table: radusergroup
+-----------+-------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-----------+-------------+------+-----+---------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
| username | varchar(64) | NO | MUL | | |
| groupname | varchar(64) | NO | MUL | | |
| priority | int(11) | NO | | 1 | |
+-----------+-------------+------+-----+---------+----------------+
radusergroup table Index
+--------------+------------+-------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment |
+--------------+------------+-------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| radusergroup | 0 | PRIMARY | 1 | id | A | 11292 | NULL | NULL | | BTREE | | |
| radusergroup | 1 | username | 1 | username | A | NULL | NULL | NULL | | BTREE | | |
| radusergroup | 1 | groupname | 1 | groupname | A | NULL | NULL | NULL | | BTREE | | |
| radusergroup | 1 | indexradusergroup | 1 | username | A | NULL | NULL | NULL | | BTREE | | |
| radusergroup | 1 | indexradusergroup | 2 | groupname | A | NULL | NULL | NULL | | BTREE | | |
Table : userinfo
---------------------+------------------+------+-----+---------------------+----------------+
| Field | Type | Null | Key | Default | Extra |
+---------------------+------------------+------+-----+---------------------+----------------+
| id | int(11) unsigned | NO | PRI | NULL | auto_increment |
| username | varchar(128) | YES | MUL | NULL | |
| firstname | varchar(200) | YES | | NULL | |
| lastname | varchar(200) | YES | | NULL | |
| email | varchar(200) | YES | | NULL | |
| workphone | varchar(200) | YES | | NULL | |
| homephone | varchar(200) | YES | | NULL | |
| mobilephone | varchar(200) | YES | | NULL | |
| address | varchar(200) | YES | | NULL | |
| zone | varchar(20) | YES | | NULL | |
| account_state | varchar(10) | YES | | active | |
| change_commit | tinyint(1) | YES | | NULL | |
| link_type | varchar(8) | YES | | NULL | |
| account_type | varchar(255) | YES | | NULL | |
| device_owner | varchar(255) | YES | | NULL | |
| account_name | varchar(255) | YES | | NULL | |
| account_email | varchar(255) | YES | | NULL | |
| account_mobile | varchar(255) | YES | | NULL | |
| groupname | varchar(255) | YES | | NULL | |
+---------------------+------------------+------+-----+---------------------+----------------+
Indexes on userinfo
----------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment |
+----------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| userinfo | 0 | PRIMARY | 1 | id | A | 11282 | NULL | NULL | | BTREE | | |
| userinfo | 1 | username | 1 | username | A | NULL | NULL | NULL | YES | BTREE | | |
+----------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
Table: Radcheck
+-----------+------------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-----------+------------------+------+-----+---------+----------------+
| id | int(11) unsigned | NO | PRI | NULL | auto_increment |
| username | varchar(64) | NO | MUL | | |
| attribute | varchar(32) | NO | MUL | | |
| op | char(2) | NO | | == | |
| value | varchar(253) | NO | MUL | | |
+-----------+------------------+------+-----+---------+----------------+
indexes on radcheck
+----------+------------+---------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment |
+----------+------------+---------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| radcheck | 0 | PRIMARY | 1 | id | A | 33425 | NULL | NULL | | BTREE | | |
| radcheck | 1 | username | 1 | username | A | NULL | NULL | NULL | | BTREE | | |
| radcheck | 1 | value | 1 | value | A | NULL | NULL | NULL | | BTREE | | |
| radcheck | 1 | attribute | 1 | attribute | A | NULL | NULL | NULL | | BTREE | | |
The function invalidates the use of the index. Despite your cardinality showing 11k rows, the table scan ensues and it takes 13 seconds.
Though you have a join between Radcheck and userinfo, it is the STR_TO_DATE function in the where clause that kills the index use.
Related
Parent Table,
mysql> desc sattool_testing;
+------------------+--------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+------------------+--------------+------+-----+---------+----------------+
| id | bigint(20) | NO | PRI | NULL | auto_increment |
| circuit_id | varchar(30) | NO | | NULL | |
| coff_id | varchar(100) | NO | | NULL | |
| result | int(11) | YES | | NULL | |
| result_details | text | NO | | NULL | |
| details | longtext | NO | | NULL | |
| reverse_response | varchar(100) | NO | | NULL | |
| start_date | datetime | NO | | NULL | |
| end_date | datetime | YES | | NULL | |
| Modules | varchar(100) | YES | | NULL | |
| rehit | datetime | YES | | NULL | |
| isAuto | tinyint(1) | NO | | 0 | |
+------------------+--------------+------+-----+---------+----------------+
12 rows in set (0.00 sec)
1st Child Table,
mysql> desc sattool_desc;
+--------------------+--------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+--------------------+--------------+------+-----+---------+----------------+
| sattool_desc_id | bigint(20) | NO | PRI | NULL | auto_increment |
| ceinterface | varchar(50) | YES | | NULL | |
| cehostname | varchar(50) | YES | | NULL | |
| vprnno | varchar(50) | YES | | NULL | |
| policyname | varchar(50) | YES | | NULL | |
| cosno | varchar(10) | YES | | NULL | |
| peipaddress | varchar(17) | YES | | NULL | |
| router_type | varchar(5) | YES | | NULL | |
| sattool_testing_id | bigint(20) | YES | | NULL | |
| epipeid | varchar(20) | YES | | NULL | |
| cerouter | varchar(10) | YES | | NULL | |
| service_type | varchar(10) | YES | | NULL | |
| scope_of_manage | varchar(50) | YES | | NULL | |
| service_name | varchar(10) | YES | | NULL | |
| ce_vrf_name | varchar(150) | YES | | NULL | |
+--------------------+--------------+------+-----+---------+----------------+
15 rows in set (0.01 sec)
2nd Child Table (Contain multiple records against one parent table record),
mysql> desc sattool_error_log;
+----------------+--------------+------+-----+-------------------+-----------------------------+
| Field | Type | Null | Key | Default | Extra |
+----------------+--------------+------+-----+-------------------+-----------------------------+
| id | bigint(20) | NO | PRI | NULL | auto_increment |
| service_id | varchar(30) | NO | | NULL | |
| coff_id | varchar(50) | NO | | NULL | |
| error_name | varchar(100) | NO | | NULL | |
| error_desc | text | NO | | NULL | |
| error_occurred | timestamp | NO | | CURRENT_TIMESTAMP | on update CURRENT_TIMESTAMP |
| error_type | int(11) | YES | | NULL | |
| error_code | varchar(10) | YES | | NULL | |
| sattool_id | bigint(20) | YES | | 0 | |
| module_no | int(11) | YES | | 0 | |
+----------------+--------------+------+-----+-------------------+-----------------------------+
10 rows in set (0.00 sec)
SQL Query :-
SELECT GROUP_CONCAT(sattool_error_log.error_desc) AS err_desc, GROUP_CONCAT(sattool_error_log.error_type) AS err_type, sattool_testing.*, sattool_desc.service_type, sattool_desc.service_name, sattool_desc.scope_of_manage
FROM sattool_testing
LEFT JOIN sattool_desc ON sattool_testing.id = sattool_desc.sattool_testing_id
LEFT JOIN sattool_error_log ON sattool_testing.id = sattool_error_log.sattool_id
WHERE sattool_testing.isAuto = 1 GROUP BY sattool_testing.id ORDER BY sattool_testing.id DESC limit 100
Current Execution Time is (22.76 sec)
Explain Query :-
+----+-------------+-------------------+------+---------------+------+---------+------+------+----------------------------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------------------+------+---------------+------+---------+------+------+----------------------------------------------+
| 1 | SIMPLE | sattool_testing | ALL | NULL | NULL | NULL | NULL | 3578 | Using where; Using temporary; Using filesort |
| 1 | SIMPLE | sattool_desc | ALL | NULL | NULL | NULL | NULL | 4009 | |
| 1 | SIMPLE | sattool_error_log | ALL | NULL | NULL | NULL | NULL | 8904 | |
+----+-------------+-------------------+------+---------------+------+---------+------+------+----------------------------------------------+
3 rows in set (0.00 sec)
I want to increase speed to above query execution.
Please give me suggestion on that.
Thanks!!!
I have added index to the columns used in the join/where clauses.
Like :-
ALTER TABLE sattool_desc ADD INDEX sattool_testing_id (sattool_testing_id);
ALTER TABLE sattool_error_log ADD INDEX sattool_id (sattool_id);
ALTER TABLE sattool_testing ADD INDEX isAuto (isAuto);
Now the execution time is 100 rows in set (0.00 sec).
Thanks again.
running this query on 5Gb sized messages table.
The problem is that the execution takes > 3 minutes.
SELECT m.id FROM messages m
LEFT JOIN dialog d on m.id=d.mid
WHERE (SELECT count(*)
FROM dialog
WHERE (m.from_id=uid1 and m.user_id=uid2)
OR (m.from_id=uid2 and m.user_id=uid1))=0 && read_state=0
LIMIT 100
I understand, that this is a bad practice to search by NESTED SELECT IN WHERE CLAUSE, but yet didn`t find another way to select such rows. Tried to split OR to 2 UNION statements, but it was long either.
messages table structure:
+------------+-------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+------------+-------------+------+-----+---------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
| from_id | int(11) | NO | MUL | NULL | |
| user_id | int(11) | NO | MUL | NULL | |
| group_id | int(11) | NO | | NULL | |
| to_number | varchar(30) | NO | MUL | NULL | |
| msg | text | NO | | NULL | |
| image | varchar(20) | NO | | NULL | |
| date | bigint(20) | NO | | NULL | |
| read_state | tinyint(1) | NO | | 0 | |
| removed | tinyint(1) | NO | | NULL | |
+------------+-------------+------+-----+---------+----------------+
dialog table structure
+-----------+------------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-----------+------------------+------+-----+---------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
| uid1 | int(11) | NO | MUL | NULL | |
| uid2 | int(11) | NO | MUL | NULL | |
| mid | int(11) | NO | | NULL | |
| anonym_id | int(10) unsigned | NO | | NULL | |
+-----------+------------------+------+-----+---------+----------------+
mysql> show index from messages;
+----------+------------+-----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment |
+----------+------------+-----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| messages | 0 | PRIMARY | 1 | id | A | 12560908 | NULL | NULL | | BTREE | | |
| messages | 1 | to_number | 1 | to_number | A | 161037 | NULL | NULL | | BTREE | | |
| messages | 1 | from_id | 1 | from_id | A | 157011 | NULL | NULL | | BTREE | | |
| messages | 1 | from_id | 2 | to_number | A | 169742 | NULL | NULL | | BTREE | | |
| messages | 1 | user_id_2 | 1 | user_id | A | 314022 | NULL | NULL | | BTREE | | |
| messages | 1 | user_id_2 | 2 | read_state | A | 380633 | NULL | NULL | | BTREE | | |
| messages | 1 | user_id_2 | 3 | removed | A | 392528 | NULL | NULL | | BTREE | | |
+----------+------------+-----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
mysql> show index from dialog;
+--------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment |
+--------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| dialog | 0 | PRIMARY | 1 | id | A | 3125615 | NULL | NULL | | BTREE | | |
| dialog | 1 | uid1 | 1 | uid1 | A | 520935 | NULL | NULL | | BTREE | | |
| dialog | 1 | uid1 | 2 | uid2 | A | 3125615 | NULL | NULL | | BTREE | | |
| dialog | 1 | uid2 | 1 | uid2 | A | 1562807 | NULL | NULL | | BTREE | | |
+--------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
EXPLAIN EXTENDED
+----+--------------------+--------+-------+---------------+------+---------+------+----------+----------+--------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | filtered | Extra |
+----+--------------------+--------+-------+---------------+------+---------+------+----------+----------+--------------------------+
| 1 | PRIMARY | m | ALL | NULL | NULL | NULL | NULL | 22190398 | 100.00 | Using where |
| 1 | PRIMARY | d | ALL | NULL | NULL | NULL | NULL | 3125621 | 100.00 | |
| 2 | DEPENDENT SUBQUERY | dialog | index | uid1,uid2 | uid1 | 8 | NULL | 3125621 | 100.00 | Using where; Using index |
+----+--------------------+--------+-------+---------------+------+---------+------+----------+----------+--------------------------+
The first thought is to change the subquery query to use not exists instead of count(*). The second is to split this into two separate subqueries. The third is to add indexes:
create index idx_messages_read_state_4 on messages_read_state(user_id, from_id, user_id, id);
create index idx_dialog_2 on dialog(uid1, uid2)
And the fourth is to remove the left join to dialog in the outer query. No fields from dialog are being used and the left join means it is not being used for filtering.
The query is then:
select m.id
from messages m
where m.read_state = 0 and
not exists (select 1
from dialog d
where m.from_id = d.uid1 and m.user_id = d.uid2
) and
not exists (select 1
from dialog d
where m.from_id = d.uid2 and m.user_id = d.uid1
)
limit 100;
I have a query that is using a group by and an order by, but it's very slow! I need some help to get the index correct. This is the query I'm running:
select *
from puresen_mv_shop.cache_deals
where feature_ids REGEXP 'i,t,d'
and phone_cost > 100.00
group by handset_numeric_id
order by popularity
LIMIT 0,10;
explain:
1 SIMPLE cache_deals index popularity 5 10 6635320.00 Using where; Using temporary
This runs in about 3 seconds, but I need it under a second. There are over 600k rows in the table. Here is the table structure:
mysql> describe cache_deals
-> ;
+-------------------------+---------------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+-------------------------+---------------+------+-----+---------+-------+
| deal_id | varchar(100) | NO | | NULL | |
| deal_id_replication | varchar(150) | NO | PRI | | |
| handset_numeric_id | int(10) | YES | MUL | NULL | |
| handset_url | varchar(100) | YES | | NULL | |
| image_url | varchar(50) | YES | MUL | NULL | |
| handset_id | varchar(50) | YES | | NULL | |
| phone_cost | decimal(10,2) | YES | | NULL | |
| tariff_numeric_id | int(10) | YES | MUL | NULL | |
| tariff_id | varchar(100) | YES | MUL | NULL | |
| tariff_name | varchar(100) | YES | | NULL | |
| network_name | varchar(20) | YES | | NULL | |
| network_numeric_id | int(5) | YES | MUL | NULL | |
| term | int(5) | YES | MUL | NULL | |
| minutes | int(5) | YES | MUL | NULL | |
| texts | int(5) | YES | MUL | NULL | |
| data | int(5) | YES | MUL | NULL | |
| org_line_rental | decimal(10,2) | YES | MUL | NULL | |
| effective_monthly_cost | decimal(10,2) | YES | MUL | NULL | |
| free_gift_id | int(10) | YES | MUL | NULL | |
| free_gift_name | varchar(50) | YES | | NULL | |
| cashback | int(5) | YES | MUL | NULL | |
| free_lr | int(5) | YES | MUL | NULL | |
| half_lr | int(5) | YES | MUL | NULL | |
| clearance_flag | int(5) | YES | MUL | NULL | |
| manufacturer_numeric_id | int(5) | YES | MUL | NULL | |
| manufacturer_name | varchar(50) | YES | | NULL | |
| full_handset_name | varchar(100) | YES | | NULL | |
| popularity | int(20) | YES | MUL | NULL | |
| handset_colour | varchar(50) | YES | MUL | NULL | |
| feature_ids | varchar(100) | YES | | NULL | |
| operating_system | varchar(30) | YES | MUL | NULL | |
+-------------------------+---------------+------+-----+---------+-------+
Here are the indexes I've currently got on this table:
mysql> show index from cache_deals;
+-------------+------------+-------------------------+--------------+-------------------------+-----------+-------------+----------+--------+------+------------
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type
+-------------+------------+-------------------------+--------------+-------------------------+-----------+-------------+----------+--------+------+------------
| cache_deals | 0 | PRIMARY | 1 | deal_id_replication | A | 663532 | NULL | NULL | | BTREE
| | |
| cache_deals | 1 | handset_numeric_id | 1 | handset_numeric_id | A | 759 | NULL | NULL | YES | BTREE
| | |
| cache_deals | 1 | network_numeric_id | 1 | network_numeric_id | A | 8 | NULL | NULL | YES | BTREE
| | |
| cache_deals | 1 | tariff_numeric_id | 1 | tariff_numeric_id | A | 1091 | NULL | NULL | YES | BTREE
| | |
| cache_deals | 1 | manufacturer_numeric_id | 1 | manufacturer_numeric_id | A | 23 | NULL | NULL | YES | BTREE
| | |
| cache_deals | 1 | operating_system | 1 | operating_system | A | 42 | NULL | NULL | YES | BTREE
| | |
| cache_deals | 1 | image_url | 1 | image_url | A | 755 | NULL | NULL | YES | BTREE
| | |
| cache_deals | 1 | tariff_id | 1 | tariff_id | A | 1091 | NULL | NULL | YES | BTREE
| | |
| cache_deals | 1 | term | 1 | term | A | 7 | NULL | NULL | YES | BTREE
| | |
| cache_deals | 1 | minutes | 1 | minutes | A | 26 | NULL | NULL | YES | BTREE
| | |
| cache_deals | 1 | texts | 1 | texts | A | 14 | NULL | NULL | YES | BTREE
| | |
| cache_deals | 1 | data | 1 | data | A | 14 | NULL | NULL | YES | BTREE
| | |
| cache_deals | 1 | org_line_rental | 1 | org_line_rental | A | 128 | NULL | NULL | YES | BTREE
| | |
| cache_deals | 1 | effective_monthly_cost | 1 | effective_monthly_cost | A | 2147 | NULL | NULL | YES | BTREE
| | |
| cache_deals | 1 | free_gift_id | 1 | free_gift_id | A | 105 | NULL | NULL | YES | BTREE
| | |
| cache_deals | 1 | cashback | 1 | cashback | A | 2 | NULL | NULL | YES | BTREE
| | |
| cache_deals | 1 | free_lr | 1 | free_lr | A | 2 | NULL | NULL | YES | BTREE
| | |
| cache_deals | 1 | half_lr | 1 | half_lr | A | 2 | NULL | NULL | YES | BTREE
| | |
| cache_deals | 1 | clearance_flag | 1 | clearance_flag | A | 2 | NULL | NULL | YES | BTREE
| | |
| cache_deals | 1 | handset_colour | 1 | handset_colour | A | 17 | NULL | NULL | YES | BTREE
| | |
| cache_deals | 1 | popularity | 1 | popularity | A | 718 | NULL | NULL | YES | BTREE
| | |
+-------------+------------+-------------------------+--------------+-------------------------+-----------+-------------+----------+--------+------+------------
Am I indexing this table correctly?
Thanks for you help.
That REGEXP is killing the potential performance since it can't use an index. Can that part of the query be written some other way or possibly normalized to another table? If you can, that will probably help. Other than that, a composite index on phone_cost and handset_numeric_id should help. Also adding popularity to that index might help, but MySQL is already unhappy with the query since you're not really supposed to be able to work with columns that aren't in the GROUP BY clause.
As a side note, the data types of the columns could be more efficient. int(5) works the same as MEDIUMINT due to the limit on the INT except that MEDIUMINT is one byte smaller. Also, int(20) doesn't make sense since the INT type can't be that large.
select [name the columns you actually want returned]
from puresen_mv_shop.cache_deals
where feature_ids = 'i,t,d'
and phone_cost > 100.00
order
by popularity
LIMIT 0,10;
Index features_id or (features_id,phone_cost)
I'm trying to run the following query:
SELECT formatted_journeys.*, MAX(speed)
FROM formatted_journeys
JOIN tracker.g_log
ON imeiid = vehicle
AND g_logid BETWEEN start_g_log AND end_g_log
GROUP BY id
however, it seems to be extremely slow. Here's the explain for that query.
+----+-------------+--------------------+------+-------------------------------+-------+---------+-----------------------------------+------+----------+---------------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | filtered | Extra |
+----+-------------+--------------------+------+-------------------------------+-------+---------+-----------------------------------+------+----------+---------------------------------+
| 1 | SIMPLE | formatted_journeys | ALL | vehicle,start_g_log,end_g_log | NULL | NULL | NULL | 824 | 100.00 | Using temporary; Using filesort |
| 1 | SIMPLE | g_log | ref | PRIMARY,Dupes | Dupes | 4 | motrak.formatted_journeys.vehicle | 1985 | 100.00 | Using where |
+----+-------------+--------------------+------+-------------------------------+-------+---------+-----------------------------------+------+----------+---------------------------------+
2 rows in set, 1 warning (0.02 sec)
The tables are as follows:
formatted_journeys:
+-----------------+------------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-----------------+------------------+------+-----+---------+----------------+
| id | int(11) unsigned | NO | PRI | NULL | auto_increment |
| start_g_log | int(11) | YES | MUL | NULL | |
| end_g_log | int(11) | YES | MUL | NULL | |
| start_latitude | decimal(18,12) | YES | | NULL | |
| start_longitude | decimal(18,12) | YES | | NULL | |
| end_latitude | decimal(18,12) | YES | | NULL | |
| end_longitude | decimal(18,12) | YES | | NULL | |
| start_location | text | YES | | NULL | |
| end_location | text | YES | | NULL | |
| distance | decimal(10,5) | YES | | NULL | |
| start_date | datetime | YES | | NULL | |
| end_date | datetime | YES | | NULL | |
| vehicle | int(11) | YES | MUL | NULL | |
| private | bit(1) | NO | | b'0' | |
+-----------------+------------------+------+-----+---------+----------------+
Indices:
+--------------------+------------+-------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment |
+--------------------+------------+-------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| formatted_journeys | 0 | PRIMARY | 1 | id | A | 830 | NULL | NULL | | BTREE | |
| formatted_journeys | 1 | vehicle | 1 | vehicle | A | 4 | NULL | NULL | YES | BTREE | |
| formatted_journeys | 1 | start_g_log | 1 | start_g_log | A | 830 | NULL | NULL | YES | BTREE | |
| formatted_journeys | 1 | end_g_log | 1 | end_g_log | A | 830 | NULL | NULL | YES | BTREE | |
+--------------------+------------+-------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
tracker.g_log:
+-----------+------------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-----------+------------------+------+-----+---------+----------------+
| g_logid | int(10) unsigned | NO | PRI | NULL | auto_increment |
| imeiid | int(10) unsigned | NO | MUL | NULL | |
| latitude | decimal(18,12) | YES | | NULL | |
| longitude | decimal(18,12) | YES | | NULL | |
| speed | int(4) | YES | MUL | NULL | |
| bearing | int(4) | YES | | NULL | |
| distance | decimal(10,5) | YES | | NULL | |
| eventcode | int(10) | YES | | NULL | |
| status | int(10) | YES | | NULL | |
| date | datetime | YES | | NULL | |
+-----------+------------------+------+-----+---------+----------------+
Indices:
+-------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment |
+-------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| g_log | 0 | PRIMARY | 1 | g_logid | A | 31760 | NULL | NULL | | BTREE | |
| g_log | 0 | Dupes | 1 | imeiid | A | 16 | NULL | NULL | | BTREE | |
| g_log | 0 | Dupes | 2 | date | A | 31760 | NULL | NULL | YES | BTREE | |
| g_log | 0 | Dupes | 3 | eventcode | A | 31760 | NULL | NULL | YES | BTREE | |
| g_log | 1 | speed | 1 | speed | A | 423 | NULL | NULL | YES | BTREE | |
+-------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
Now I know that the filesort isn't a good thing but how would I get rid of it?
You will hardly get rid of Using temporary; Using filesort, but the query might be faster.
First, try to rewrite the query as:
SELECT *
FROM (
SELECT id, MAX(speed) as max_speed
FROM formatted_journeys
JOIN tracker.g_log
ON imeiid = vehicle
AND g_logid BETWEEN start_g_log AND end_g_log
GROUP BY id) as maxspeeds
JOIN formatted_journeys USING (id);
Then, you can try to force the query to use a coverage index, though that is not easy for the query.
First try: add a composite index (vehicle, start_g_log, end_g_log) and look in explain whether it is used (you should see "Using index" there)
I have two tables:
mysql> desc myps3t_gameusertrophyinfo;
+-----------+--------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-----------+--------------+------+-----+---------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
| user_id | int(11) | NO | MUL | NULL | |
| trophy_id | int(11) | NO | MUL | NULL | |
| date | datetime | NO | MUL | NULL | |
| date_read | varchar(100) | NO | | NULL | |
+-----------+--------------+------+-----+---------+----------------+
5 rows in set (0.19 sec)
mysql> show index from myps3t_gameusertrophyinfo;
+---------------------------+------------+------------------------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment |
+---------------------------+------------+------------------------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| myps3t_gameusertrophyinfo | 0 | PRIMARY | 1 | id | A | 4004589 | NULL | NULL | | BTREE | |
| myps3t_gameusertrophyinfo | 0 | user_id | 1 | user_id | A | 7686 | NULL | NULL | | BTREE | |
| myps3t_gameusertrophyinfo | 0 | user_id | 2 | trophy_id | A | 4004589 | NULL | NULL | | BTREE | |
| myps3t_gameusertrophyinfo | 1 | myps3t_gameusertrophyinfo_403f60f | 1 | user_id | A | 7686 | NULL | NULL | | BTREE | |
| myps3t_gameusertrophyinfo | 1 | myps3t_gameusertrophyinfo_61a683d8 | 1 | trophy_id | A | 22624 | NULL | NULL | | BTREE | |
| myps3t_gameusertrophyinfo | 1 | idx_gameusertrophyinfo_date | 1 | date | A | 4004589 | NULL | NULL | | BTREE | |
+---------------------------+------------+------------------------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
7 rows in set (0.06 sec)
the other table:
mysql> desc myps3t_gametrophyinfo ;
+-----------+--------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-----------+--------------+------+-----+---------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
| game_id | int(11) | NO | MUL | NULL | |
| name | varchar(500) | NO | | NULL | |
| desc | varchar(500) | NO | | NULL | |
| type | varchar(20) | NO | | NULL | |
| pic_url | varchar(200) | NO | | NULL | |
| desc_pt | varchar(500) | NO | | NULL | |
| name_pt | varchar(500) | NO | | NULL | |
| hidden_id | int(11) | NO | | NULL | |
| total_id | int(11) | NO | | NULL | |
| trophy_id | int(11) | NO | | NULL | |
| addon_id | int(11) | YES | | NULL | |
| points | double | NO | | 0 | |
| sony_id | int(11) | YES | | NULL | |
+-----------+--------------+------+-----+---------+----------------+
14 rows in set (0.00 sec)
mysql> show index from myps3t_gametrophyinfo;
+-----------------------+------------+-------------------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment |
+-----------------------+------------+-------------------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| myps3t_gametrophyinfo | 0 | PRIMARY | 1 | id | A | 25976 | NULL | NULL | | BTREE | |
| myps3t_gametrophyinfo | 1 | myps3t_gametrophyinfo_game_id | 1 | game_id | A | 764 | NULL | NULL | | BTREE | |
+-----------------------+------------+-------------------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
when i do this query:
mysql> explain select * from myps3t_gameusertrophyinfo a, myps3t_gametrophyinfo b where a.trophy_id = b.id and b.addon_id = 58; +----+-------------+-------+--------+--------------------------------------------------------------+---------+---------+-----------------------------+---------+-------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------+--------+--------------------------------------------------------------+---------+---------+-----------------------------+---------+-------------+
| 1 | SIMPLE | a | ALL | myps3t_gameusertrophyinfo_61a683d8,myps3t_gameusertrophyinfo | NULL | NULL | NULL | 4004592 | |
| 1 | SIMPLE | b | eq_ref | PRIMARY | PRIMARY | 4 | fabriciols_ps3t.a.trophy_id | 1 | Using where |
+----+-------------+-------+--------+--------------------------------------------------------------+---------+---------+-----------------------------+---------+-------------+
2 rows in set (0.00 sec)
why mysql do not use my keys ?
this query take more than 30 seconds, the first table has 4milion records ...
-- edit --
for quasnoi
mysql> SELECT COUNT(*), COUNT(DISTINCT addon_id), SUM(addon_id = 58) FROM myps3t_gametrophyinfo;
+----------+--------------------------+--------------------+
| COUNT(*) | COUNT(DISTINCT addon_id) | SUM(addon_id = 58) |
+----------+--------------------------+--------------------+
| 25976 | 160 | 6 |
+----------+--------------------------+--------------------+
1 row in set (0.00 sec)
MySQL chooses a as a leading table and b as a driven table. It does use a PRIMARY KEY on b for the joins.
Create an index on myps3t_gametrophyinfo (addon_id), this way b will be more probably chosen as a leading table.
You can try
select * from
myps3t_gametrophyinfo b
STRAIGHT_JOIN myps3t_gameusertrophyinfo a ON (a.trophy_id = b.id)
WHERE b.addon_id = 58;
I would probably rewrite the query to try to get a more sane execution path. I think something like below is more likely to get you the performance you want and is more clear in what you are doing to a human reader
SELECT * FROM myps3t_gametrophyinfo a LEFT JOIN myps3t_gameusertrophyinfo b ON a.id = b.trophy_id WHERE a.addon_id=58;