My feed_listingjob has 2 datetime fields:
+------------+-------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+------------+-------------+------+-----+---------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
| data | longtext | NO | | NULL | |
| meta_data | longtext | NO | | NULL | |
| state | varchar(25) | NO | | NULL | |
| error | longtext | NO | | NULL | |
| job_id | int(11) | NO | MUL | NULL | |
| created_at | datetime(6) | NO | MUL | NULL | |
| updated_at | datetime(6) | NO | MUL | NULL | |
| es_sync_at | datetime(6) | YES | MUL | NULL | |
+------------+-------------+------+-----+---------+----------------+
updated_at and es_sync_at both are indexed individually as below:
mysql> show indexes from feed_listingjob;
+-----------------+------------+--------------------------------------------------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment |
+-----------------+------------+--------------------------------------------------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| feed_listingjob | 0 | PRIMARY | 1 | id | A | 64534 | NULL | NULL | | BTREE | | |
| feed_listingjob | 1 | feed_listingjob_job_id_4c3b1b514481f269_fk_feed_importjob_id | 1 | job_id | A | 2081 | NULL | NULL | | BTREE | | |
| feed_listingjob | 1 | feed_listingjob_fde81f11 | 1 | created_at | A | 64534 | NULL | NULL | | BTREE | | |
| feed_listingjob | 1 | feed_listingjob_afd1a1a8 | 1 | updated_at | A | 64534 | NULL | NULL | | BTREE | | |
| feed_listingjob | 1 | feed_listingjob_381895a2 | 1 | es_sync_at | A | 2 | NULL | NULL | YES | BTREE | | |
+-----------------+------------+--------------------------------------------------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
5 rows in set (0.00 sec)
And my query is not using the indexes:
mysql> explain SELECT `feed_listingjob`.`id` FROM `feed_listingjob` WHERE `feed_listingjob`.`es_sync_at` < `feed_listingjob`.`updated_at` LIMIT 10;
+----+-------------+-----------------+------+---------------+------+---------+------+-------+-------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-----------------+------+---------------+------+---------+------+-------+-------------+
| 1 | SIMPLE | feed_listingjob | ALL | NULL | NULL | NULL | NULL | 53534 | Using where |
+----+-------------+-----------------+------+---------------+------+---------+------+-------+-------------+
1 row in set (0.01 sec)
Can someone please tell me why and how to optimise this query?
The index can't be used here because feed_listingjob.updated_at is not a constant. Which means every row would have to be examined to check the condition. One way to use the index would be to have another column which stores the difference between es_sync_at and updated_at timestamps. If you store it as difference = (es_sync_at - updated_at timestamps) then the query becomes (difference < 0).
Related
I am going to join two tables by using a single position in one table to the range (represented by two columns) in another table.
However, the performance is too slow, which is about 20 mins.
I have tried adding the index on the table or changing the query.
But the performance is still poor.
So, I am asking for optimization of the joining speed.
The following is the query to MySQL.
mysql> SELECT `inVar`.chrom, `inVar`.pos, `openChrom_K562`.score
-> FROM `inVar`
-> LEFT JOIN `openChrom_K562`
-> ON (
-> `inVar`.chrom=`openChrom_K562`.chrom AND
-> `inVar`.pos BETWEEN `openChrom_K562`.chromStart AND `openChrom_K562`.chromEnd
-> );
inVar and openChrom_K562 are the tables I used.
inVar stores the single position in each row.
openChrom_K562 stores the range information indicated by chromStart and chromEnd.
inVar contains 57902 rows and openChrom_K562 has 137373 rows respectively.
Fields on the tables.
mysql> DESCRIBE inVar;
+-------+-------------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+-------+-------------+------+-----+---------+-------+
| chrom | varchar(31) | NO | PRI | NULL | |
| pos | int(10) | NO | PRI | NULL | |
+-------+-------------+------+-----+---------+-------+
mysql> DESCRIBE openChrom_K562;
+------------+-------------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+------------+-------------+------+-----+---------+-------+
| chrom | varchar(31) | NO | MUL | NULL | |
| chromStart | int(10) | NO | MUL | NULL | |
| chromEnd | int(10) | NO | | NULL | |
| score | int(10) | NO | | NULL | |
+------------+-------------+------+-----+---------+-------+
Index built in the tables
mysql> SHOW INDEX FROM inVar;
+-------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment |
+-------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| inVar | 0 | PRIMARY | 1 | chrom | A | NULL | NULL | NULL | | BTREE | |
| inVar | 0 | PRIMARY | 2 | pos | A | 57902 | NULL | NULL | | BTREE | |
+-------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
mysql> SHOW INDEX FROM openChrom_K562;
+----------------+------------+-------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment |
+----------------+------------+-------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| openChrom_K562 | 1 | start_end | 1 | chromStart | A | 137373 | NULL | NULL | | BTREE | |
| openChrom_K562 | 1 | start_end | 2 | chromEnd | A | 137373 | NULL | NULL | | BTREE | |
| openChrom_K562 | 1 | chrom_only | 1 | chrom | A | 22 | NULL | NULL | | BTREE | |
| openChrom_K562 | 1 | chrom_start | 1 | chrom | A | 22 | NULL | NULL | | BTREE | |
| openChrom_K562 | 1 | chrom_start | 2 | chromStart | A | 137373 | NULL | NULL | | BTREE | |
| openChrom_K562 | 1 | chrom_end | 1 | chrom | A | 22 | NULL | NULL | | BTREE | |
| openChrom_K562 | 1 | chrom_end | 2 | chromEnd | A | 137373 | NULL | NULL | | BTREE | |
+----------------+------------+-------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
Execution plan on MySQL
mysql> EXPLAIN SELECT `inVar`.chrom, `inVar`.pos, score FROM `inVar` LEFT JOIN `openChrom_K562` ON ( inVar.chrom=openChrom_K562.chrom AND `inVar`.pos BETWEEN chromStart AND chromEnd );
+----+-------------+----------------+-------+--------------------------------------------+------------+---------+-----------------+-------+-------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+----------------+-------+--------------------------------------------+------------+---------+-----------------+-------+-------------+
| 1 | SIMPLE | inVar | index | NULL | PRIMARY | 37 | NULL | 57902 | Using index |
| 1 | SIMPLE | openChrom_K562 | ref | start_end,chrom_only,chrom_start,chrom_end | chrom_only | 33 | tmp.inVar.chrom | 5973 | |
+----+-------------+----------------+-------+--------------------------------------------+------------+---------+-----------------+-------+-------------+
It seems it only optimizes by looking chrom in two tables. Then do the brute-force comparing in the tables.
Is there any ways to do the further optimization like indexing on the position?
(It is my first time posting the question, sorry for the poor posting quality.)
chrom_only is likely to be a bad index selection for your join as you only have chrom 22 values.
If I have interpreted this right the query should be faster if using start_end
SELECT `inVar`.chrom, `inVar`.pos, `openChrom_K562`.score
FROM `inVar`
LEFT JOIN `openChrom_K562`
USE INDEX (`start_end`)
ON (
`inVar`.chrom=`openChrom_K562`.chrom AND
`inVar`.pos BETWEEN `openChrom_K562`.chromStart AND `openChrom_K562`.chromEnd
)
I have this query:
SELECT
s.last_spread, s.sd, s.mean, s.id
,c.id_ticker, c.coef
,t.ticker
,p.last, p.price
FROM (SELECT * FROM spreads WHERE spreads.id_check=1 LIMIT 100,500 ) as s
INNER JOIN coef as c
ON c.id_spread = s.id
INNER JOIN tickers AS t
ON t.id = c.id_ticker
LEFT JOIN (SELECT prices.id_ticker, MAX(prices.date) as last, prices.price FROM prices GROUP BY prices.id_ticker) AS p
ON p.id_ticker = t.id
These are the schemas of the tables:
mysql> desc spreads;
+-------------+---------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-------------+---------+------+-----+---------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
| id_check | int(11) | YES | MUL | NULL | |
| sd | double | YES | | NULL | |
| mean | double | YES | | NULL | |
| last_spread | double | YES | | NULL | |
+-------------+---------+------+-----+---------+----------------+
5 rows in set (0.00 sec)
mysql> desc coef;
+-----------+---------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-----------+---------+------+-----+---------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
| id_spread | int(11) | YES | MUL | NULL | |
| id_ticker | int(11) | YES | | NULL | |
| coef | double | YES | | NULL | |
| side | double | YES | | NULL | |
+-----------+---------+------+-----+---------+----------------+
5 rows in set (0.00 sec)
mysql> desc tickers;
+----------+------------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+----------+------------------+------+-----+---------+----------------+
| id | int(10) unsigned | NO | PRI | NULL | auto_increment |
| ticker | varchar(45) | NO | | NULL | |
| name | varchar(150) | NO | | NULL | |
| category | varchar(150) | NO | | NULL | |
| issuer | varchar(150) | NO | | NULL | |
+----------+------------------+------+-----+---------+----------------+
5 rows in set (0.00 sec)
mysql> desc prices;
+-----------+------------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-----------+------------------+------+-----+---------+----------------+
| id | int(10) unsigned | NO | PRI | NULL | auto_increment |
| id_ticker | int(10) unsigned | NO | MUL | NULL | |
| date | date | NO | | NULL | |
| price | double | NO | | NULL | |
+-----------+------------------+------+-----+---------+----------------+
4 rows in set (0.01 sec)
These are the indexes of the above tables;
mysql> show indexes from spreads;
+---------+------------+-----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment |
+---------+------------+-----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| spreads | 0 | PRIMARY | 1 | id | A | 2299 | NULL | NULL | | BTREE | |
| spreads | 1 | check_idx | 1 | id_check | A | 1 | NULL | NULL | YES | BTREE | |
+---------+------------+-----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
2 rows in set (0.00 sec)
mysql> show indexes from coef;
+-------+------------+-------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment |
+-------+------------+-------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| coef | 0 | PRIMARY | 1 | id | A | 9078 | NULL | NULL | | BTREE | |
| coef | 1 | spread_ticker_idx | 1 | id_spread | A | NULL | NULL | NULL | YES | BTREE | |
| coef | 1 | spread_ticker_idx | 2 | id_ticker | A | NULL | NULL | NULL | YES | BTREE | |
+-------+------------+-------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
3 rows in set (0.00 sec)
mysql> show indexes from tickers;
+---------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment |
+---------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| tickers | 0 | PRIMARY | 1 | id | A | 100 | NULL | NULL | | BTREE | |
+---------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
1 row in set (0.00 sec)
mysql> show indexes from prices;
+--------+------------+-----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment |
+--------+------------+-----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| prices | 0 | PRIMARY | 1 | id | A | 19962 | NULL | NULL | | BTREE | |
| prices | 1 | id_ticker | 1 | id_ticker | A | 19962 | NULL | NULL | | BTREE | |
+--------+------------+-----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
2 rows in set (0.15 sec)
And this is the explain of the query:
+----+-------------+------------+--------+-------------------+-------------------+---------+---------------------------+--------+-------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+------------+--------+-------------------+-------------------+---------+---------------------------+--------+-------------+
| 1 | PRIMARY | <derived2> | ALL | NULL | NULL | NULL | NULL | 500 | |
| 1 | PRIMARY | c | ref | spread_ticker_idx | spread_ticker_idx | 5 | s.id | 90 | Using where |
| 1 | PRIMARY | t | eq_ref | PRIMARY | PRIMARY | 4 | spreadtrading.c.id_ticker | 1 | Using where |
| 1 | PRIMARY | <derived3> | ALL | NULL | NULL | NULL | NULL | 100 | |
| 3 | DERIVED | prices | index | NULL | id_ticker | 4 | NULL | 119774 | |
| 2 | DERIVED | spreads | ref | check_idx | check_idx | 5 | | 2298 | Using where |
+----+-------------+------------+--------+-------------------+-------------------+---------+---------------------------+--------+-------------+
6 rows in set (0.27 sec)
Could I optimize it?
Thank you!
EDIT:
I would like to know if the INDEXES and the table's structure are optimized for the query I posted above. The results that I get using this query are good, it works well, but maybe I can optimize it to increse the "speed" of the query.
I think you may gain something by dropping the spreads subquery and moving the WHERE clause to the end, as in the following code. This loses your LIMIT restriction - perhaps you could put a LIMIT clause at the end as well, depending on what you're trying to achieve in terms of limiting the size of the output.
SELECT
s.last_spread, s.sd, s.mean, s.id
,c.id_ticker, c.coef
,t.ticker
,p.last, p.price
FROM spreads as s
INNER JOIN coef as c
ON c.id_spread = s.id
INNER JOIN tickers AS t
ON t.id = c.id_ticker
LEFT JOIN (SELECT prices.id_ticker, MAX(prices.date) as last, prices.price FROM prices GROUP BY prices.id_ticker) AS p
ON p.id_ticker = t.id
WHERE s.id_check = 1
I'm trying to run the following query:
SELECT formatted_journeys.*, MAX(speed)
FROM formatted_journeys
JOIN tracker.g_log
ON imeiid = vehicle
AND g_logid BETWEEN start_g_log AND end_g_log
GROUP BY id
however, it seems to be extremely slow. Here's the explain for that query.
+----+-------------+--------------------+------+-------------------------------+-------+---------+-----------------------------------+------+----------+---------------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | filtered | Extra |
+----+-------------+--------------------+------+-------------------------------+-------+---------+-----------------------------------+------+----------+---------------------------------+
| 1 | SIMPLE | formatted_journeys | ALL | vehicle,start_g_log,end_g_log | NULL | NULL | NULL | 824 | 100.00 | Using temporary; Using filesort |
| 1 | SIMPLE | g_log | ref | PRIMARY,Dupes | Dupes | 4 | motrak.formatted_journeys.vehicle | 1985 | 100.00 | Using where |
+----+-------------+--------------------+------+-------------------------------+-------+---------+-----------------------------------+------+----------+---------------------------------+
2 rows in set, 1 warning (0.02 sec)
The tables are as follows:
formatted_journeys:
+-----------------+------------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-----------------+------------------+------+-----+---------+----------------+
| id | int(11) unsigned | NO | PRI | NULL | auto_increment |
| start_g_log | int(11) | YES | MUL | NULL | |
| end_g_log | int(11) | YES | MUL | NULL | |
| start_latitude | decimal(18,12) | YES | | NULL | |
| start_longitude | decimal(18,12) | YES | | NULL | |
| end_latitude | decimal(18,12) | YES | | NULL | |
| end_longitude | decimal(18,12) | YES | | NULL | |
| start_location | text | YES | | NULL | |
| end_location | text | YES | | NULL | |
| distance | decimal(10,5) | YES | | NULL | |
| start_date | datetime | YES | | NULL | |
| end_date | datetime | YES | | NULL | |
| vehicle | int(11) | YES | MUL | NULL | |
| private | bit(1) | NO | | b'0' | |
+-----------------+------------------+------+-----+---------+----------------+
Indices:
+--------------------+------------+-------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment |
+--------------------+------------+-------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| formatted_journeys | 0 | PRIMARY | 1 | id | A | 830 | NULL | NULL | | BTREE | |
| formatted_journeys | 1 | vehicle | 1 | vehicle | A | 4 | NULL | NULL | YES | BTREE | |
| formatted_journeys | 1 | start_g_log | 1 | start_g_log | A | 830 | NULL | NULL | YES | BTREE | |
| formatted_journeys | 1 | end_g_log | 1 | end_g_log | A | 830 | NULL | NULL | YES | BTREE | |
+--------------------+------------+-------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
tracker.g_log:
+-----------+------------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-----------+------------------+------+-----+---------+----------------+
| g_logid | int(10) unsigned | NO | PRI | NULL | auto_increment |
| imeiid | int(10) unsigned | NO | MUL | NULL | |
| latitude | decimal(18,12) | YES | | NULL | |
| longitude | decimal(18,12) | YES | | NULL | |
| speed | int(4) | YES | MUL | NULL | |
| bearing | int(4) | YES | | NULL | |
| distance | decimal(10,5) | YES | | NULL | |
| eventcode | int(10) | YES | | NULL | |
| status | int(10) | YES | | NULL | |
| date | datetime | YES | | NULL | |
+-----------+------------------+------+-----+---------+----------------+
Indices:
+-------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment |
+-------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| g_log | 0 | PRIMARY | 1 | g_logid | A | 31760 | NULL | NULL | | BTREE | |
| g_log | 0 | Dupes | 1 | imeiid | A | 16 | NULL | NULL | | BTREE | |
| g_log | 0 | Dupes | 2 | date | A | 31760 | NULL | NULL | YES | BTREE | |
| g_log | 0 | Dupes | 3 | eventcode | A | 31760 | NULL | NULL | YES | BTREE | |
| g_log | 1 | speed | 1 | speed | A | 423 | NULL | NULL | YES | BTREE | |
+-------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
Now I know that the filesort isn't a good thing but how would I get rid of it?
You will hardly get rid of Using temporary; Using filesort, but the query might be faster.
First, try to rewrite the query as:
SELECT *
FROM (
SELECT id, MAX(speed) as max_speed
FROM formatted_journeys
JOIN tracker.g_log
ON imeiid = vehicle
AND g_logid BETWEEN start_g_log AND end_g_log
GROUP BY id) as maxspeeds
JOIN formatted_journeys USING (id);
Then, you can try to force the query to use a coverage index, though that is not easy for the query.
First try: add a composite index (vehicle, start_g_log, end_g_log) and look in explain whether it is used (you should see "Using index" there)
I am trying to optimize a MySQL select request:
SELECT * FROM `sales`
WHERE ((sales.private = false AND (sales.buyer_id IS NULL OR NOT sales.buyer_id=142)
AND (sales.merchand_id IS NULL OR NOT sales.merchand_id=142)
AND (sales.private_item = false) )
AND ((sales.buyer_id=32 OR sales.merchand_id=32)
AND (sales.admin=0 AND NOT sales.type IN ('book'))))
ORDER BY sales.created_at DESC, sales.id DESC LIMIT 0, 10;
The schema of the table is
mysql> SHOW columns from sales;
+------------------------+--------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+------------------------+--------------+------+-----+---------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
| type | varchar(255) | YES | MUL | NULL | |
| buyer_id | int(11) | YES | MUL | NULL | |
| merchand_id | int(11) | YES | MUL | NULL | |
| private | tinyint(1) | YES | | 0 | |
| admin | tinyint(1) | YES | | 0 | |
| created_at | datetime | YES | | NULL | |
| updated_at | datetime | YES | | NULL | |
| country_id | int(11) | YES | MUL | 0 | |
| private_item | tinyint(1) | YES | | 0 | |
+------------------------+--------------+------+-----+---------+----------------+
The indexes are:
mysql> show indexes from sales;
+-----------------+------------+--------------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment |
+-----------------+------------+--------------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| sales | 0 | PRIMARY | 1 | id | A | 286509 | NULL | NULL | | BTREE | |
| sales | 1 | index_sales_on_type | 1 | type | A | 123 | NULL | NULL | YES | BTREE | |
| sales | 1 | index_sales_on_buyer_id | 1 | buyer_id | A | 40929 | NULL | NULL | YES | BTREE | |
| sales | 1 | index_sales_on_merchand_id | 1 | merchand_id | A | 40929 | NULL | NULL | YES | BTREE | |
| sales | 1 | index_sales_on_country_id | 1 | country_id | A | 6 | NULL | NULL | YES | BTREE | |
| sales | 1 | index_sales_on_type_and_country_id | 1 | type | A | 151 | NULL | NULL | YES | BTREE | |
| sales | 1 | index_sales_on_type_and_country_id | 2 | country_id | A | 428 | NULL | NULL | YES | BTREE | |
| sales | 1 | index_sales_viewed | 1 | buyer_id | A | 35813 | NULL | NULL | YES | BTREE | |
| sales | 1 | index_sales_viewed | 2 | merchand_id | A | 286509 | NULL | NULL | YES | BTREE | |
| sales | 1 | index_sales_viewed | 3 | private_item| A | 285009 | NULL | NULL | YES | BTREE | |
| sales | 1 | index_sales_viewed | 4 | admin | A | 285009 | NULL | NULL | YES | BTREE | |
| sales | 1 | index_sales_viewed | 5 | type | A | 285009 | NULL | NULL | YES | BTREE | |
| sales | 1 | index_sales_viewed | 6 | private | A | 285009 | NULL | NULL | YES | BTREE | |
| sales | 1 | index_sales_viewed | 7 | created_at | A | 285009 | NULL | NULL | YES | BTREE | |
+-------+------------+------------------------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
When doing the query it uses the index_sales_on_type_and_country_id even if there is no country_id in the query...
The query takes 2.5 seconds with this index.
But when I use USE INDEX(index_sales_viewed) it goes down to 0.2 seconds.
Here is the EXPLAIN of the query:
+----+-------------+-----------------+------+----------------------------------------------+------+---------+------+--------+---------------------
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-----------------+------+----------------------------------------------+------+---------+------+--------+---------------------
| 1 | SIMPLE | sales | range | see bellow | index_sales_on_type_and_country_id | 258 | NULL | 208725 | Using where; Using filesort |
+----+-------------+-----------------+------+----------------------------------------------+------+---------+------+--------+---------------------
the possible keys are :
index_sales_on_type,
index_sales_on_buyer_id,
index_sales_on_merchand_id,
index_sales_on_type_and_country_id,
index_sales_public_recent_activity
Why doesn't MySQL use index_sales_viewed by default? Could there be a better index?
Thank you!
This is wrong use on NULL, please change all the column used in the index to be NOT NULL
refer this When to use NULL in MySQL tables
official documentation
If this column is NULL, there are no relevant indexes. In this case, you may be able to improve the performance of your query by examining the WHERE clause to check whether it refers to some column or columns that would be suitable for indexing. If so, create an appropriate index and check the query with EXPLAIN again
Mysql chosen index index_sales_on_type_and_country_id because you are not compare with NULL value
I have two tables:
mysql> desc myps3t_gameusertrophyinfo;
+-----------+--------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-----------+--------------+------+-----+---------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
| user_id | int(11) | NO | MUL | NULL | |
| trophy_id | int(11) | NO | MUL | NULL | |
| date | datetime | NO | MUL | NULL | |
| date_read | varchar(100) | NO | | NULL | |
+-----------+--------------+------+-----+---------+----------------+
5 rows in set (0.19 sec)
mysql> show index from myps3t_gameusertrophyinfo;
+---------------------------+------------+------------------------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment |
+---------------------------+------------+------------------------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| myps3t_gameusertrophyinfo | 0 | PRIMARY | 1 | id | A | 4004589 | NULL | NULL | | BTREE | |
| myps3t_gameusertrophyinfo | 0 | user_id | 1 | user_id | A | 7686 | NULL | NULL | | BTREE | |
| myps3t_gameusertrophyinfo | 0 | user_id | 2 | trophy_id | A | 4004589 | NULL | NULL | | BTREE | |
| myps3t_gameusertrophyinfo | 1 | myps3t_gameusertrophyinfo_403f60f | 1 | user_id | A | 7686 | NULL | NULL | | BTREE | |
| myps3t_gameusertrophyinfo | 1 | myps3t_gameusertrophyinfo_61a683d8 | 1 | trophy_id | A | 22624 | NULL | NULL | | BTREE | |
| myps3t_gameusertrophyinfo | 1 | idx_gameusertrophyinfo_date | 1 | date | A | 4004589 | NULL | NULL | | BTREE | |
+---------------------------+------------+------------------------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
7 rows in set (0.06 sec)
the other table:
mysql> desc myps3t_gametrophyinfo ;
+-----------+--------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-----------+--------------+------+-----+---------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
| game_id | int(11) | NO | MUL | NULL | |
| name | varchar(500) | NO | | NULL | |
| desc | varchar(500) | NO | | NULL | |
| type | varchar(20) | NO | | NULL | |
| pic_url | varchar(200) | NO | | NULL | |
| desc_pt | varchar(500) | NO | | NULL | |
| name_pt | varchar(500) | NO | | NULL | |
| hidden_id | int(11) | NO | | NULL | |
| total_id | int(11) | NO | | NULL | |
| trophy_id | int(11) | NO | | NULL | |
| addon_id | int(11) | YES | | NULL | |
| points | double | NO | | 0 | |
| sony_id | int(11) | YES | | NULL | |
+-----------+--------------+------+-----+---------+----------------+
14 rows in set (0.00 sec)
mysql> show index from myps3t_gametrophyinfo;
+-----------------------+------------+-------------------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment |
+-----------------------+------------+-------------------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| myps3t_gametrophyinfo | 0 | PRIMARY | 1 | id | A | 25976 | NULL | NULL | | BTREE | |
| myps3t_gametrophyinfo | 1 | myps3t_gametrophyinfo_game_id | 1 | game_id | A | 764 | NULL | NULL | | BTREE | |
+-----------------------+------------+-------------------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
when i do this query:
mysql> explain select * from myps3t_gameusertrophyinfo a, myps3t_gametrophyinfo b where a.trophy_id = b.id and b.addon_id = 58; +----+-------------+-------+--------+--------------------------------------------------------------+---------+---------+-----------------------------+---------+-------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------+--------+--------------------------------------------------------------+---------+---------+-----------------------------+---------+-------------+
| 1 | SIMPLE | a | ALL | myps3t_gameusertrophyinfo_61a683d8,myps3t_gameusertrophyinfo | NULL | NULL | NULL | 4004592 | |
| 1 | SIMPLE | b | eq_ref | PRIMARY | PRIMARY | 4 | fabriciols_ps3t.a.trophy_id | 1 | Using where |
+----+-------------+-------+--------+--------------------------------------------------------------+---------+---------+-----------------------------+---------+-------------+
2 rows in set (0.00 sec)
why mysql do not use my keys ?
this query take more than 30 seconds, the first table has 4milion records ...
-- edit --
for quasnoi
mysql> SELECT COUNT(*), COUNT(DISTINCT addon_id), SUM(addon_id = 58) FROM myps3t_gametrophyinfo;
+----------+--------------------------+--------------------+
| COUNT(*) | COUNT(DISTINCT addon_id) | SUM(addon_id = 58) |
+----------+--------------------------+--------------------+
| 25976 | 160 | 6 |
+----------+--------------------------+--------------------+
1 row in set (0.00 sec)
MySQL chooses a as a leading table and b as a driven table. It does use a PRIMARY KEY on b for the joins.
Create an index on myps3t_gametrophyinfo (addon_id), this way b will be more probably chosen as a leading table.
You can try
select * from
myps3t_gametrophyinfo b
STRAIGHT_JOIN myps3t_gameusertrophyinfo a ON (a.trophy_id = b.id)
WHERE b.addon_id = 58;
I would probably rewrite the query to try to get a more sane execution path. I think something like below is more likely to get you the performance you want and is more clear in what you are doing to a human reader
SELECT * FROM myps3t_gametrophyinfo a LEFT JOIN myps3t_gameusertrophyinfo b ON a.id = b.trophy_id WHERE a.addon_id=58;