SELECT au.*
FROM users au
INNER JOIN friends fa ON au.id = fa.to_user_id
WHERE fa.from_user_id = 369 AND fa.persona_id IN('1241')
GROUP BY au.id
ORDER BY id DESC
LIMIT 0, 9999999999;
Here is the explain
mysql> EXPLAIN SELECT au.* FROM users au INNER JOIN friends fa ON au.id = fa.to_user_id WHERE fa.from_user_id = 369 AND fa.persona_id IN('1241') GROUP BY au.id ORDER BY id DESC LIMIT 0, 9999999999;
+----+-------------+-------+-------------+------------------------------------------------------------+------------------------------------+---------+--------------------+------+---------------------------------------------------------------------------------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------+-------------+------------------------------------------------------------+------------------------------------+---------+--------------------+------+---------------------------------------------------------------------------------------------------+
| 1 | SIMPLE | fa | index_merge | from_user_id,to_user_id,persona_id,from_user_id_persona_id | persona_id,from_user_id_persona_id | 4,8 | NULL | 49 | Using intersect(persona_id,from_user_id_persona_id); Using where; Using temporary; Using filesort |
| 1 | SIMPLE | au | eq_ref | PRIMARY | PRIMARY | 4 | kjdb.fa.to_user_id | 1 | |
+----+-------------+-------+-------------+------------------------------------------------------------+------------------------------------+---------+--------------------+------+---------------------------------------------------------------------------------------------------+
2 rows in set (0.00 sec)
I only have 300,000 rows in this table, but it's taking forever. (about .75 seconds to run it)
mysql> desc friends;
+-------------------+------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-------------------+------------+------+-----+---------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
| from_user_id | int(11) | NO | MUL | NULL | |
| to_user_id | int(11) | NO | MUL | NULL | |
| persona_id | int(11) | NO | MUL | NULL | |
| action_by_user_id | int(11) | NO | MUL | NULL | |
| is_disabled | tinyint(1) | NO | | 0 | |
| created_at | datetime | YES | | NULL | |
| updated_at | datetime | YES | | NULL | |
+-------------------+------------+------+-----+---------+----------------+
8 rows in set (0.01 sec)
> desc users;
+---------------------+--------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+---------------------+--------------+------+-----+---------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
| username | varchar(255) | NO | UNI | NULL | |
| first_name | varchar(255) | NO | MUL | NULL | |
| last_name | varchar(255) | YES | MUL | NULL | |
| email | varchar(255) | YES | UNI | NULL | |
here are my indexes:
mysql> show indexes from friends;
+-------------------+------------+-------------------------+--------------+-------------------+-----------+-------------+----------+--------+------+------------+---------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment |
+-------------------+------------+-------------------------+--------------+-------------------+-----------+-------------+----------+--------+------+------------+---------+
| friends | 0 | PRIMARY | 1 | id | A | 388926 | NULL | NULL | | BTREE | |
| friends | 0 | from_user_id | 1 | from_user_id | A | 17 | NULL | NULL | | BTREE | |
| friends | 0 | from_user_id | 2 | to_user_id | A | 388926 | NULL | NULL | | BTREE | |
| friends | 0 | from_user_id | 3 | persona_id | A | 388926 | NULL | NULL | | BTREE | |
| friends | 1 | to_user_id | 1 | to_user_id | A | 19446 | NULL | NULL | | BTREE | |
| friends | 1 | persona_id | 1 | persona_id | A | 32410 | NULL | NULL | | BTREE | |
| friends | 1 | action_by_user_id | 1 | action_by_user_id | A | 9972 | NULL | NULL | | BTREE | |
| friends | 1 | from_user_id_persona_id | 1 | from_user_id | A | 9486 | NULL | NULL | | BTREE | |
| friends | 1 | from_user_id_persona_id | 2 | persona_id | A | 35356 | NULL | NULL | | BTREE | |
+-------------------+------------+-------------------------+--------------+-------------------+-----------+-------------+----------+--------+------+------------+---------+
Add this index:
ALTER TABLE friends ADD INDEX ix_from_persona_to (from_user_id,persona_id,to_user_id);
and see if that helps.
If not, you may have to add a FORCE INDEX:
SELECT
au.*
FROM
users au
INNER JOIN
friends fa ON au.id = fa.to_user_id
FORCE INDEX
(ix_from_persona_to)
WHERE
fa.from_user_id = 369 AND
fa.persona_id IN ('1241')
GROUP BY
au.id
ORDER BY
id DESC
LIMIT
0, 9999999999;
the order by on a group by is causing a temp table + filesort which is killing your performance.
try subselects?
SELECT au.*
FROM users au
WHERE au.id IN (
SELECT DISTINCT fa.to_user_id
FRoM friends fa
WHERE fa.from_user_id = 369 AND fa.persona_id IN('1241')
)
ORDER BY id DESC
LIMIT 0, 9999999999;
then add indexes on friends.from_user_id , persona_id
Related
Below is the output of "show full processlist" on my-sql command line. this query calculates the users total upload and download bandwidth for specified period of time. (the database is from freeradius). How can i make the query faster. [ Memory:150G , processr:8 centos 6.8]
15260415|radiusremote|panel.example.com:57526|radius|Query|35|Copying to tmp table
SELECT sum(acctinputoctets) as upload,sum(acctoutputoctets) as download
FROM radacct a
INNER JOIN
( SELECT acctuniqueid, MIN( radacctid ) radacctid
FROM radacct
WHERE username='nyjohan'
and acctstarttime between '2016-01-15 13:50:05'
AND '2016-08-07 13:16:36'
GROUP BY acctuniqueid
) b ON a.acctuniqueid = b.acctuniqueid
AND a.radacctid = b.radacctid
There are indexes created on the table , below is the output of indexes on table
mysql> show index from radacct;
+---------+------------+-----------------+--------------+-----------------+-----------+-------------+----------+--------+------+------------+---------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment |
+---------+------------+-----------------+--------------+-----------------+-----------+-------------+----------+--------+------+------------+---------+
| radacct | 0 | PRIMARY | 1 | radacctid | A | 161791738 | NULL | NULL | | BTREE | |
| radacct | 1 | username | 1 | username | A | 15 | NULL | NULL | | BTREE | |
| radacct | 1 | framedipaddress | 1 | framedipaddress | A | 458333 | NULL | NULL | | BTREE | |
| radacct | 1 | acctsessionid | 1 | acctsessionid | A | 161791738 | NULL | NULL | | BTREE | |
| radacct | 1 | acctsessiontime | 1 | acctsessiontime | A | 46332 | NULL | NULL | YES | BTREE | |
| radacct | 1 | acctuniqueid | 1 | acctuniqueid | A | 161791738 | NULL | NULL | | BTREE | |
| radacct | 1 | acctstarttime | 1 | acctstarttime | A | 40447934 | NULL | NULL | YES | BTREE | |
| radacct | 1 | acctstoptime | 1 | acctstoptime | A | 80895869 | NULL | NULL | YES | BTREE | |
| radacct | 1 | nasipaddress | 1 | nasipaddress | A | 15 | NULL | NULL | | BTREE | |
+---------+------------+-----------------+--------------+-----------------+-----------+-------------+----------+--------+------+------------+---------+
Schema of table
mysql> describe radacct;
+----------------------+-------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+----------------------+-------------+------+-----+---------+----------------+
| radacctid | bigint(21) | NO | PRI | NULL | auto_increment |
| acctsessionid | varchar(32) | NO | MUL | | |
| acctuniqueid | varchar(32) | NO | MUL | | |
| username | varchar(64) | NO | MUL | | |
| groupname | varchar(64) | NO | | | |
| realm | varchar(64) | YES | | | |
| nasipaddress | varchar(15) | NO | MUL | | |
| nasportid | varchar(15) | YES | | NULL | |
| nasporttype | varchar(32) | YES | | NULL | |
| acctstarttime | datetime | YES | MUL | NULL | |
| acctstoptime | datetime | YES | MUL | NULL | |
| acctsessiontime | int(12) | YES | MUL | NULL | |
| acctauthentic | varchar(32) | YES | | NULL | |
| connectinfo_start | varchar(50) | YES | | NULL | |
| connectinfo_stop | varchar(50) | YES | | NULL | |
| acctinputoctets | bigint(20) | YES | | NULL | |
| acctoutputoctets | bigint(20) | YES | | NULL | |
| calledstationid | varchar(50) | NO | | | |
| callingstationid | varchar(50) | NO | | | |
| acctterminatecause | varchar(32) | NO | | | |
| servicetype | varchar(32) | YES | | NULL | |
| framedprotocol | varchar(32) | YES | | NULL | |
| framedipaddress | varchar(15) | NO | MUL | | |
| acctstartdelay | int(12) | YES | | NULL | |
| acctstopdelay | int(12) | YES | | NULL | |
| xascendsessionsvrkey | varchar(10) | YES | | NULL | |
+----------------------+-------------+------+-----+---------+----------------+
Explain output:-
explain SELECT sum(acctinputoctets) as upload,sum(acctoutputoctets) as download
FROM radacct a
INNER JOIN (
SELECT acctuniqueid, MIN( radacctid ) radacctid
FROM radacct
WHERE username='dave137' and acctstarttime between '2016-08-03 00:00:00' and '2016-08-07 14:47:54' GROUP BY acctuniqueid
)b ON a.acctuniqueid = b.acctuniqueid
AND a.radacctid = b.radacctid ;
+----+-------------+------------+--------+------------------------+----------+---------+-------------+-------+----------------------------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+------------+--------+------------------------+----------+---------+-------------+-------+----------------------------------------------+
| 1 | PRIMARY | <derived2> | ALL | NULL | NULL | NULL | NULL | 10 | |
| 1 | PRIMARY | a | eq_ref | PRIMARY,acctuniqueid | PRIMARY | 8 | b.radacctid | 1 | Using where |
| 2 | DERIVED | radacct | ref | username,acctstarttime | username | 66 | | 10164 | Using where; Using temporary; Using filesort |
+----+-------------+------------+--------+------------------------+----------+---------+-------------+-------+----------------------------------------------+
3 rows in set (9.91 sec)
For this query:
SELECT sum(acctinputoctets) as upload,sum(acctoutputoctets) as download
FROM radacct a INNER JOIN
(SELECT acctuniqueid, MIN( radacctid ) as radacctid
FROM radacct
WHERE username = 'dave137' and acctstarttime between '2016-08-03 00:00:00' and '2016-08-07 14:47:54'
GROUP BY acctuniqueid
) b
ON a.acctuniqueid = b.acctuniqueid AND a.radacctid = b.radacctid ;
I would recommend two indexes, one is already present: radacct(radacctid) and radacct(username, acctstarttime, acctuniqueid).
In addition, I would simplify the ON clause to:
ON a.radacctid = b.radacctid ;
radacctid is unique so there is no need for another condition.
Using a JOIN on a sub-query will slow down your query regardless of indexes. From what I can see this should do the same thing without needing to do the sub-join:
SELECT acctinputoctets as upload, acctoutputoctets as download
FROM radacct a
WHERE username = 'dave137'
and acctstarttime between '2016-08-03 00:00:00' and '2016-08-07 14:47:54'
ORDER BY radacctid
LIMIT 1
running this query on 5Gb sized messages table.
The problem is that the execution takes > 3 minutes.
SELECT m.id FROM messages m
LEFT JOIN dialog d on m.id=d.mid
WHERE (SELECT count(*)
FROM dialog
WHERE (m.from_id=uid1 and m.user_id=uid2)
OR (m.from_id=uid2 and m.user_id=uid1))=0 && read_state=0
LIMIT 100
I understand, that this is a bad practice to search by NESTED SELECT IN WHERE CLAUSE, but yet didn`t find another way to select such rows. Tried to split OR to 2 UNION statements, but it was long either.
messages table structure:
+------------+-------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+------------+-------------+------+-----+---------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
| from_id | int(11) | NO | MUL | NULL | |
| user_id | int(11) | NO | MUL | NULL | |
| group_id | int(11) | NO | | NULL | |
| to_number | varchar(30) | NO | MUL | NULL | |
| msg | text | NO | | NULL | |
| image | varchar(20) | NO | | NULL | |
| date | bigint(20) | NO | | NULL | |
| read_state | tinyint(1) | NO | | 0 | |
| removed | tinyint(1) | NO | | NULL | |
+------------+-------------+------+-----+---------+----------------+
dialog table structure
+-----------+------------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-----------+------------------+------+-----+---------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
| uid1 | int(11) | NO | MUL | NULL | |
| uid2 | int(11) | NO | MUL | NULL | |
| mid | int(11) | NO | | NULL | |
| anonym_id | int(10) unsigned | NO | | NULL | |
+-----------+------------------+------+-----+---------+----------------+
mysql> show index from messages;
+----------+------------+-----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment |
+----------+------------+-----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| messages | 0 | PRIMARY | 1 | id | A | 12560908 | NULL | NULL | | BTREE | | |
| messages | 1 | to_number | 1 | to_number | A | 161037 | NULL | NULL | | BTREE | | |
| messages | 1 | from_id | 1 | from_id | A | 157011 | NULL | NULL | | BTREE | | |
| messages | 1 | from_id | 2 | to_number | A | 169742 | NULL | NULL | | BTREE | | |
| messages | 1 | user_id_2 | 1 | user_id | A | 314022 | NULL | NULL | | BTREE | | |
| messages | 1 | user_id_2 | 2 | read_state | A | 380633 | NULL | NULL | | BTREE | | |
| messages | 1 | user_id_2 | 3 | removed | A | 392528 | NULL | NULL | | BTREE | | |
+----------+------------+-----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
mysql> show index from dialog;
+--------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment |
+--------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| dialog | 0 | PRIMARY | 1 | id | A | 3125615 | NULL | NULL | | BTREE | | |
| dialog | 1 | uid1 | 1 | uid1 | A | 520935 | NULL | NULL | | BTREE | | |
| dialog | 1 | uid1 | 2 | uid2 | A | 3125615 | NULL | NULL | | BTREE | | |
| dialog | 1 | uid2 | 1 | uid2 | A | 1562807 | NULL | NULL | | BTREE | | |
+--------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
EXPLAIN EXTENDED
+----+--------------------+--------+-------+---------------+------+---------+------+----------+----------+--------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | filtered | Extra |
+----+--------------------+--------+-------+---------------+------+---------+------+----------+----------+--------------------------+
| 1 | PRIMARY | m | ALL | NULL | NULL | NULL | NULL | 22190398 | 100.00 | Using where |
| 1 | PRIMARY | d | ALL | NULL | NULL | NULL | NULL | 3125621 | 100.00 | |
| 2 | DEPENDENT SUBQUERY | dialog | index | uid1,uid2 | uid1 | 8 | NULL | 3125621 | 100.00 | Using where; Using index |
+----+--------------------+--------+-------+---------------+------+---------+------+----------+----------+--------------------------+
The first thought is to change the subquery query to use not exists instead of count(*). The second is to split this into two separate subqueries. The third is to add indexes:
create index idx_messages_read_state_4 on messages_read_state(user_id, from_id, user_id, id);
create index idx_dialog_2 on dialog(uid1, uid2)
And the fourth is to remove the left join to dialog in the outer query. No fields from dialog are being used and the left join means it is not being used for filtering.
The query is then:
select m.id
from messages m
where m.read_state = 0 and
not exists (select 1
from dialog d
where m.from_id = d.uid1 and m.user_id = d.uid2
) and
not exists (select 1
from dialog d
where m.from_id = d.uid2 and m.user_id = d.uid1
)
limit 100;
When GROUP BY uid and GROUP BY anonym_id are placed into queries the execution time problem appears > 2sec.
The question is - how to optimize/override this query with effect like GROUP BY statement?
( SELECT u.id as uid, u.name, u.avatar, u.avatar_date, u.driver,
u.vehicle, m.msg, m.removed, (m.date DIV 1000) AS date, m.from_id = 1 AS outbox,
CASE WHEN m.read_state IS NULL THEN 1 ELSE 0 END AS read_state, d.anonym_id
FROM dialog as d CROSS JOIN messages AS m CROSS JOIN users AS u
WHERE (m.id=d.mid AND ((d.uid1 = 1 and u.id = d.uid2) or (d.uid2 = 1 and u.id = d.uid1)))
GROUP BY uid
ORDER BY d.id DESC )
UNION
( SELECT u.id as uid, u.name, u.avatar, u.avatar_date, u.driver, u.vehicle,
m.msg, m.removed, (m.date DIV 1000) AS date, m.from_id = 1 AS outbox,
CASE WHEN m.read_state IS NULL THEN 1 ELSE 0 END AS read_state,
d.anonym_id
FROM dialog as d FORCE KEY(anonym_id) CROSS JOIN messages AS m ON d.mid=m.id
CROSS JOIN users AS u ON u.id=d.uid2
WHERE d.anonym_id=100001
GROUP BY d.uid2
ORDER BY d.id DESC )
LIMIT 0, 20
mysql> show index from users;
+-------+------------+-------------+--------------+--------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment |
+-------+------------+-------------+--------------+--------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| users | 0 | PRIMARY | 1 | id | A | 606396 | NULL | NULL | | BTREE | | |
| users | 1 | number | 1 | number | A | 606396 | NULL | NULL | | BTREE | | |
| users | 1 | name | 1 | name | A | 121279 | NULL | NULL | | BTREE | | |
| users | 1 | show_phone | 1 | show_phone | A | 10 | NULL | NULL | | BTREE | | |
| users | 1 | show_mail | 1 | show_mail | A | 14 | NULL | NULL | | BTREE | | |
| users | 1 | show_on_map | 1 | show_on_map | A | 18 | NULL | NULL | | BTREE | | |
| users | 1 | show_on_map | 2 | map_activity | A | 606396 | NULL | NULL | | BTREE | | |
| users | 1 | udid | 1 | udid | A | 606396 | NULL | NULL | YES | BTREE | | |
| users | 1 | blocked | 1 | blocked | A | 10 | NULL | NULL | | BTREE | | |
+-------+------------+-------------+--------------+--------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
mysql> show index from dialog;
+--------+------------+-----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment |
+--------+------------+-----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| dialog | 0 | PRIMARY | 1 | id | A | 2964484 | NULL | NULL | | BTREE | | |
| dialog | 1 | uid1 | 1 | uid1 | A | 211748 | NULL | NULL | | BTREE | | |
| dialog | 1 | uid1 | 2 | uid2 | A | 2964484 | NULL | NULL | | BTREE | | |
| dialog | 1 | uid2 | 1 | uid2 | A | 1482242 | NULL | NULL | | BTREE | | |
| dialog | 1 | anonym_id | 1 | anonym_id | A | 18 | NULL | NULL | | BTREE | | |
| dialog | 1 | idx_mid | 1 | mid | A | 200 | NULL | NULL | | BTREE | | |
+--------+------------+-----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
mysql> show index from messages;
+----------+------------+-----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment |
+----------+------------+-----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| messages | 0 | PRIMARY | 1 | id | A | 23227116 | NULL | NULL | | BTREE | | |
| messages | 1 | user_id_2 | 1 | user_id | A | 188 | NULL | NULL | | BTREE | | |
| messages | 1 | user_id_2 | 2 | read_state | A | 188 | NULL | NULL | | BTREE | | |
| messages | 1 | user_id_2 | 3 | removed | A | 188 | NULL | NULL | | BTREE | | |
| messages | 1 | from_id | 1 | from_id | A | 188 | NULL | NULL | | BTREE | | |
| messages | 1 | from_id | 2 | to_number | A | 188 | NULL | NULL | | BTREE | | |
+----------+------------+-----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
EXPLAIN EXTENDED
1 PRIMARY d index_merge uid1,uid2,idx_mid uid1,uid2 4,4 26 100.00 Using sort_union(uid1,uid2); Using where; Using temporary; Using filesort
1 PRIMARY u ALL PRIMARY 625866 100.00 Using where; Using join buffer
1 PRIMARY m eq_ref PRIMARY PRIMARY 4 numbers.d.mid 1 100.00
2 UNION d ref anonym_id anonym_id 4 const 1 100.00
2 UNION u eq_ref PRIMARY PRIMARY 4 numbers.d.uid2 1 100.00
2 UNION m eq_ref PRIMARY PRIMARY 4 numbers.d.mid 1 100.00
UNION RESULT <union1,2> ALL
I have this query:
SELECT
s.last_spread, s.sd, s.mean, s.id
,c.id_ticker, c.coef
,t.ticker
,p.last, p.price
FROM (SELECT * FROM spreads WHERE spreads.id_check=1 LIMIT 100,500 ) as s
INNER JOIN coef as c
ON c.id_spread = s.id
INNER JOIN tickers AS t
ON t.id = c.id_ticker
LEFT JOIN (SELECT prices.id_ticker, MAX(prices.date) as last, prices.price FROM prices GROUP BY prices.id_ticker) AS p
ON p.id_ticker = t.id
These are the schemas of the tables:
mysql> desc spreads;
+-------------+---------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-------------+---------+------+-----+---------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
| id_check | int(11) | YES | MUL | NULL | |
| sd | double | YES | | NULL | |
| mean | double | YES | | NULL | |
| last_spread | double | YES | | NULL | |
+-------------+---------+------+-----+---------+----------------+
5 rows in set (0.00 sec)
mysql> desc coef;
+-----------+---------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-----------+---------+------+-----+---------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
| id_spread | int(11) | YES | MUL | NULL | |
| id_ticker | int(11) | YES | | NULL | |
| coef | double | YES | | NULL | |
| side | double | YES | | NULL | |
+-----------+---------+------+-----+---------+----------------+
5 rows in set (0.00 sec)
mysql> desc tickers;
+----------+------------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+----------+------------------+------+-----+---------+----------------+
| id | int(10) unsigned | NO | PRI | NULL | auto_increment |
| ticker | varchar(45) | NO | | NULL | |
| name | varchar(150) | NO | | NULL | |
| category | varchar(150) | NO | | NULL | |
| issuer | varchar(150) | NO | | NULL | |
+----------+------------------+------+-----+---------+----------------+
5 rows in set (0.00 sec)
mysql> desc prices;
+-----------+------------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-----------+------------------+------+-----+---------+----------------+
| id | int(10) unsigned | NO | PRI | NULL | auto_increment |
| id_ticker | int(10) unsigned | NO | MUL | NULL | |
| date | date | NO | | NULL | |
| price | double | NO | | NULL | |
+-----------+------------------+------+-----+---------+----------------+
4 rows in set (0.01 sec)
These are the indexes of the above tables;
mysql> show indexes from spreads;
+---------+------------+-----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment |
+---------+------------+-----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| spreads | 0 | PRIMARY | 1 | id | A | 2299 | NULL | NULL | | BTREE | |
| spreads | 1 | check_idx | 1 | id_check | A | 1 | NULL | NULL | YES | BTREE | |
+---------+------------+-----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
2 rows in set (0.00 sec)
mysql> show indexes from coef;
+-------+------------+-------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment |
+-------+------------+-------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| coef | 0 | PRIMARY | 1 | id | A | 9078 | NULL | NULL | | BTREE | |
| coef | 1 | spread_ticker_idx | 1 | id_spread | A | NULL | NULL | NULL | YES | BTREE | |
| coef | 1 | spread_ticker_idx | 2 | id_ticker | A | NULL | NULL | NULL | YES | BTREE | |
+-------+------------+-------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
3 rows in set (0.00 sec)
mysql> show indexes from tickers;
+---------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment |
+---------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| tickers | 0 | PRIMARY | 1 | id | A | 100 | NULL | NULL | | BTREE | |
+---------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
1 row in set (0.00 sec)
mysql> show indexes from prices;
+--------+------------+-----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment |
+--------+------------+-----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| prices | 0 | PRIMARY | 1 | id | A | 19962 | NULL | NULL | | BTREE | |
| prices | 1 | id_ticker | 1 | id_ticker | A | 19962 | NULL | NULL | | BTREE | |
+--------+------------+-----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
2 rows in set (0.15 sec)
And this is the explain of the query:
+----+-------------+------------+--------+-------------------+-------------------+---------+---------------------------+--------+-------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+------------+--------+-------------------+-------------------+---------+---------------------------+--------+-------------+
| 1 | PRIMARY | <derived2> | ALL | NULL | NULL | NULL | NULL | 500 | |
| 1 | PRIMARY | c | ref | spread_ticker_idx | spread_ticker_idx | 5 | s.id | 90 | Using where |
| 1 | PRIMARY | t | eq_ref | PRIMARY | PRIMARY | 4 | spreadtrading.c.id_ticker | 1 | Using where |
| 1 | PRIMARY | <derived3> | ALL | NULL | NULL | NULL | NULL | 100 | |
| 3 | DERIVED | prices | index | NULL | id_ticker | 4 | NULL | 119774 | |
| 2 | DERIVED | spreads | ref | check_idx | check_idx | 5 | | 2298 | Using where |
+----+-------------+------------+--------+-------------------+-------------------+---------+---------------------------+--------+-------------+
6 rows in set (0.27 sec)
Could I optimize it?
Thank you!
EDIT:
I would like to know if the INDEXES and the table's structure are optimized for the query I posted above. The results that I get using this query are good, it works well, but maybe I can optimize it to increse the "speed" of the query.
I think you may gain something by dropping the spreads subquery and moving the WHERE clause to the end, as in the following code. This loses your LIMIT restriction - perhaps you could put a LIMIT clause at the end as well, depending on what you're trying to achieve in terms of limiting the size of the output.
SELECT
s.last_spread, s.sd, s.mean, s.id
,c.id_ticker, c.coef
,t.ticker
,p.last, p.price
FROM spreads as s
INNER JOIN coef as c
ON c.id_spread = s.id
INNER JOIN tickers AS t
ON t.id = c.id_ticker
LEFT JOIN (SELECT prices.id_ticker, MAX(prices.date) as last, prices.price FROM prices GROUP BY prices.id_ticker) AS p
ON p.id_ticker = t.id
WHERE s.id_check = 1
I have two tables:
mysql> desc myps3t_gameusertrophyinfo;
+-----------+--------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-----------+--------------+------+-----+---------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
| user_id | int(11) | NO | MUL | NULL | |
| trophy_id | int(11) | NO | MUL | NULL | |
| date | datetime | NO | MUL | NULL | |
| date_read | varchar(100) | NO | | NULL | |
+-----------+--------------+------+-----+---------+----------------+
5 rows in set (0.19 sec)
mysql> show index from myps3t_gameusertrophyinfo;
+---------------------------+------------+------------------------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment |
+---------------------------+------------+------------------------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| myps3t_gameusertrophyinfo | 0 | PRIMARY | 1 | id | A | 4004589 | NULL | NULL | | BTREE | |
| myps3t_gameusertrophyinfo | 0 | user_id | 1 | user_id | A | 7686 | NULL | NULL | | BTREE | |
| myps3t_gameusertrophyinfo | 0 | user_id | 2 | trophy_id | A | 4004589 | NULL | NULL | | BTREE | |
| myps3t_gameusertrophyinfo | 1 | myps3t_gameusertrophyinfo_403f60f | 1 | user_id | A | 7686 | NULL | NULL | | BTREE | |
| myps3t_gameusertrophyinfo | 1 | myps3t_gameusertrophyinfo_61a683d8 | 1 | trophy_id | A | 22624 | NULL | NULL | | BTREE | |
| myps3t_gameusertrophyinfo | 1 | idx_gameusertrophyinfo_date | 1 | date | A | 4004589 | NULL | NULL | | BTREE | |
+---------------------------+------------+------------------------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
7 rows in set (0.06 sec)
the other table:
mysql> desc myps3t_gametrophyinfo ;
+-----------+--------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-----------+--------------+------+-----+---------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
| game_id | int(11) | NO | MUL | NULL | |
| name | varchar(500) | NO | | NULL | |
| desc | varchar(500) | NO | | NULL | |
| type | varchar(20) | NO | | NULL | |
| pic_url | varchar(200) | NO | | NULL | |
| desc_pt | varchar(500) | NO | | NULL | |
| name_pt | varchar(500) | NO | | NULL | |
| hidden_id | int(11) | NO | | NULL | |
| total_id | int(11) | NO | | NULL | |
| trophy_id | int(11) | NO | | NULL | |
| addon_id | int(11) | YES | | NULL | |
| points | double | NO | | 0 | |
| sony_id | int(11) | YES | | NULL | |
+-----------+--------------+------+-----+---------+----------------+
14 rows in set (0.00 sec)
mysql> show index from myps3t_gametrophyinfo;
+-----------------------+------------+-------------------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment |
+-----------------------+------------+-------------------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| myps3t_gametrophyinfo | 0 | PRIMARY | 1 | id | A | 25976 | NULL | NULL | | BTREE | |
| myps3t_gametrophyinfo | 1 | myps3t_gametrophyinfo_game_id | 1 | game_id | A | 764 | NULL | NULL | | BTREE | |
+-----------------------+------------+-------------------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
when i do this query:
mysql> explain select * from myps3t_gameusertrophyinfo a, myps3t_gametrophyinfo b where a.trophy_id = b.id and b.addon_id = 58; +----+-------------+-------+--------+--------------------------------------------------------------+---------+---------+-----------------------------+---------+-------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------+--------+--------------------------------------------------------------+---------+---------+-----------------------------+---------+-------------+
| 1 | SIMPLE | a | ALL | myps3t_gameusertrophyinfo_61a683d8,myps3t_gameusertrophyinfo | NULL | NULL | NULL | 4004592 | |
| 1 | SIMPLE | b | eq_ref | PRIMARY | PRIMARY | 4 | fabriciols_ps3t.a.trophy_id | 1 | Using where |
+----+-------------+-------+--------+--------------------------------------------------------------+---------+---------+-----------------------------+---------+-------------+
2 rows in set (0.00 sec)
why mysql do not use my keys ?
this query take more than 30 seconds, the first table has 4milion records ...
-- edit --
for quasnoi
mysql> SELECT COUNT(*), COUNT(DISTINCT addon_id), SUM(addon_id = 58) FROM myps3t_gametrophyinfo;
+----------+--------------------------+--------------------+
| COUNT(*) | COUNT(DISTINCT addon_id) | SUM(addon_id = 58) |
+----------+--------------------------+--------------------+
| 25976 | 160 | 6 |
+----------+--------------------------+--------------------+
1 row in set (0.00 sec)
MySQL chooses a as a leading table and b as a driven table. It does use a PRIMARY KEY on b for the joins.
Create an index on myps3t_gametrophyinfo (addon_id), this way b will be more probably chosen as a leading table.
You can try
select * from
myps3t_gametrophyinfo b
STRAIGHT_JOIN myps3t_gameusertrophyinfo a ON (a.trophy_id = b.id)
WHERE b.addon_id = 58;
I would probably rewrite the query to try to get a more sane execution path. I think something like below is more likely to get you the performance you want and is more clear in what you are doing to a human reader
SELECT * FROM myps3t_gametrophyinfo a LEFT JOIN myps3t_gameusertrophyinfo b ON a.id = b.trophy_id WHERE a.addon_id=58;