very simple but quite confusing to me:
SELECT `start`, `stop` FROM loadtime
WHERE utilisateur_id = '202931999'
AND `type` = 'stat'
AND `stop` !=0
ORDER BY id DESC
LIMIT 0,1
12 sec
SELECT `start`, `stop` FROM loadtime
WHERE utilisateur_id = '202931999'
AND `type` = 'stat'
AND `stop` !=0
ORDER BY id DESC
LIMIT 0,2 or xxx
0.07 sec
explain says
limit 0, 1
id select_type table type possible_keys
1 SIMPLE loadtime index utilisateur_id_2,utilisateur_id,type
key key_len ref rows Extra
PRIMARY 4 NULL 10089 Using where
limit 0, x
id select_type table type possible_keys
1 SIMPLE loadtime ref utilisateur_id_2,utilisateur_id,type
key key_len ref rows Extra
utilisateur_id 62 const 12103 Using index condition; Using where; Using filesort
so the first query doesn't use the index naturally.
server is on MySQL 5.5.32
The problem is simple: for some reason (which can be many) MySQL server decides that in the limit 0,1 case it is a better idea (i.e. the server expects that it would take less time and computation) to do a full table scan in order to give you the right result, instead of using an index.
Since this is a prediction, it could happen that it may be wrong and using an index would work better (as in the limit 0,x case). Since you noticed the performance difference, you can force the query to use the index you think is the best, as follows (and as suggested here):
SELECT `start`, `stop` FORCE INDEX (condition) FROM loadtime
WHERE utilisateur_id = '202931999'
AND `type` = 'stat'
AND `stop` !=0
ORDER BY id DESC
LIMIT 0,1
Related
SELECT `f`.*
FROM `files_table` `f`
WHERE f.`application_id` IN(6)
AND `f`.`project_id` IN(130418)
AND `f`.`is_last_version` = 1
AND `f`.`temporary` = 0
AND f.deleted_by is null
ORDER BY `f`.`date` DESC
LIMIT 5
When I remove the ORDER BY, query executes in 0.1 seconds. With the ORDER BY it takes 3 seconds.
There is an index on every WHERE column and there is also an index on ORDER BY field (date).
What can I do to make this query faster? Why is ORDER BY slowing it down so much? Table has 3M rows.
instead of an index on each column in where be sure you have a composite index that cover all the columns in where
eg
create index idx1 on files_table (application_id, project_id,is_last_version,temporary,deleted_by)
avoid IN clause for single value use = for these
SELECT `f`.*
FROM `files_table` `f`
WHERE f.`application_id` = 6
AND `f`.`project_id` = 130418
AND `f`.`is_last_version` = 1
AND `f`.`temporary` = 0
AND f.deleted_by is null
ORDER BY `f`.`date` DESC
LIMIT 5
the date or others column in select could be useful retrive all info using the index and avoiding the access to the table data .. but for select all (select *)
you probably need severl columns an then the access to the table data is done however .. but you can try an eval the performance ..
be careful to place the data non involved in where at the right of all the column involved in where
create index idx1 on files_table (application_id, project_id,is_last_version,temporary,deleted_by, date)
I have the following query and plan:
SELECT data.*
FROM data
WHERE channel_id = 1
AND timestamp >= IFNULL((
SELECT UNIX_TIMESTAMP(DATE_ADD(FROM_UNIXTIME(MAX(timestamp) / 1000, "%Y-%m-%d"), INTERVAL 1 day)) * 1000
FROM aggregate
WHERE type = '3' AND aggregate.channel_id = data.channel_id
), 0)
AND timestamp < UNIX_TIMESTAMP(DATE_FORMAT(NOW(), "%Y-%m-%d")) * 1000
id select_type table partitions type possible_keys key key_len ref rows filtered Extra
'1' 'PRIMARY' 'data' NULL 'ref' 'data_unique IDX_ADF3F36372F5A1AA' 'IDX_ADF3F36372F5A1AA' '5' 'const' '860512' '11.11' 'Using where'
'2' 'DEPENDENT SUBQUERY' 'aggregate' NULL 'ref' 'aggregate_unique IDX_B77949FF72F5A1AA' 'aggregate_unique' '7' 'volkszaehler.data.channel_id const' '1473' '100.00' 'Using index'
The data table has a couple of million rows, all tables are indexed:
data: `channel_id`, `timestamp`
aggregate: `type`, `channel_id`, `timestamp`
The query becomes fast when the aggregate.channel_id = data.channel_id is replaced with the actual value of the channel_id of the outer query. And the dependent subquery becomes a simple subquery.
However, I'd rather not do this to allow the query to operate on >1 channel_ids at a time.
Why doesn't MySQL (5.7 homebrew) recognize that this subquery really is not dependent (or is it?) and how could it be optimized?
I've already verified that removing the IFNULL function or pushing it inwards does not cure the problem. I was also not successful in pushing down the subquery another level as suggested in Can i force mysql to perform subquery first? since the data.channel_id reference is no longer known then.
Please consider a table with queue_name, priority and message_timestamp columns.
Here is a compound index for that:
CREATE INDEX STATE_QUEUENAME_PRIORITY_TIMESTAMP ON
`queue_messages` (queue_name, state, priority, message_timestamp);
EXPLAIN for SELECT:
EXPLAIN SELECT message_timestamp
from queue_messages
WHERE queue_name = 'folder'
AND state = 0
ORDER BY priority DESC, message_timestamp DESC
LIMIT 1;
Returns Using where; Using index
EXPLAIN for UPDATE (with the same WHERE and ORDER BY!):
EXPLAIN UPDATE queue_messages
SET state = 1
WHERE queue_name = 'folder'
AND state = 0
ORDER BY priority DESC, message_timestamp DESC
LIMIT 1;
Returns Using where; Using filesort
--
It causes significant performance impact (20ms SELECT vs 90ms UPDATE on 50k rows).
How can I force MariaDB (MySQL) to get rid of filesort in UPDATE statement?
For the update, you need a second index:
CREATE INDEX STATE_QUEUENAME_TIMESTAMP_2 ON `queue_messages` (queue_name, state, priority, message_timestamp);
Your existing index doesn't include priority, so it cannot be used for the order by.
My problem is this query:
(SELECT
ID1,ID2,ID3,ID4,ID5,ID10,ID11,ID13,ID14,ID454,ID453,
TIME,TEMP_ID,'ID_AUTO',PREDAJCA,VYTVORIL,MAIL,TEMP_ID_HASH,ID_SEND
FROM `load_send_calc`
WHERE `TEMP_ID` LIKE '$find%'
AND ACTIVE = 1 AND TEMP_ID > 0)
UNION ALL
(SELECT
ID1,ID2,ID3,ID4,ID5,ID10,ID11,ID13,ID14,ID454,ID453,TIME,'',
ID_AUTO,'','','','',''
FROM `temp`
WHERE `ID_AUTO` LIKE '$find%'
AND `ID_AUTO` NOT IN (SELECT TEMP_ID
FROM `load_send_calc`
WHERE `load_send_calc`.ACTIVE = 1)
)
ORDER BY TIME DESC LIMIT $limitFrom,$limitTo;
There are 18000 records in table load_send_calc and 3000 table temp. The query itself take more than 2 minutes to execute. Is there any way to optimize this time?
I already tried to put order into each subqueries but it didnt help significantly. I am really desperate so I really appreciate any kind of help.
EDIT:
Here is EXPLAIN result :
id select_type table type possible_keys key key_len ref rows Extra
1 PRIMARY load_send_calc ALL NULL NULL NULL NULL 18394 Using where
2 UNION temp ALL NULL NULL NULL NULL 1918 Using where
3 DEPENDENT SUBQUERY load_send_calc ALL NULL NULL NULL NULL 18394 Using where
NULL UNION RESULT <union1,2> ALL NULL NULL NULL NULL NULL Using filesort
Thanks for adding your explain output - it tells us a lot. The query isn't using a single index, which is very bad for performance. A very simple optimisation would be add indexes on the fields that are used in the join, and also in the where clauses. In your case those fields would be:
load_send_calc.temp_id
load_send_calc.active
temp.id_auto
In addition to these, you have an unnecessary AND TEMP_ID > 0, since you are already limiting on the same field with WHERE TEMP_ID LIKE '$find%'
3 things to speed it up:
INDEX(active, temp_id) -- significantly better than two separate indexes
IN ( SELECT ... ) performs poorly, especially in old versions of MySQL. Turn it into a JOIN.
Add a LIMIT to each SELECT. For example:
( SELECT ... ORDER BY ... LIMIT 80 )
UNION ALL
( SELECT ... ORDER BY ... LIMIT 80 )
ORDER BY ... LIMIT 70, 10;
The inner ones have a limit of the max needed -- the outer's offset + limit.
Please suggest indexes to optimize below query. I couldn't allowed to rewrite the query but create indexes:
SELECT
`ADV`.`inds` as `c0`,
sum(`ADVpost`.`clk`) as `m0`
FROM
(SELECT *
FROM advts
WHERE comp_id =
(SELECT comp_id
FROM comp
WHERE name = 'abc')) as `ADV`,
(SELECT dt_id,
comp_id,
b_id,
ad_id,
clk,
resp
FROM advts_post
WHERE comp_id =
(SELECT comp_id
FROM comp
WHERE name = 'abc')) as `ADVpost`
WHERE
`ADVpost`.`ad_id` = `ADV`.`ad_id`
GROUP BY
`ADV`.`inds`
ORDER BY
ISNULL(`ADV`.`inds`), `ADV`.`inds` ASC
The explain for the query is as:
select_type table type possible_keys Extra
PRIMARY <derived2> ALL null Using temporary; Using filesort
PRIMARY <derived4> ALL null Using where; Using join buffer
DERIVED ADVpost ALL null Using where
SUBQUERY comp ALL null Using where
DERIVED advts ALL null Using where
SUBQUERY comp ALL null Using where
Existing indexes are as follows:
ADVpost > PRIMARY KEY (`dt_id`,`comp_id`,`b_id`,`ad_id`)
comp > PRIMARY KEY (`comp_id`)
advts > PRIMARY KEY (`ad_id`)
Thanks in advance.
Ok, maybe I am not an expert with MySQL optimization, but:
if it is possible and reasonable, try to avoid subselects where possible (instead it may be better to make separate query and then pass the retrieved ID, like comp_id, to the containing query),
put index on comp.name,
put index on advts_post.comp_id (single one),
put index on advts_post.ad_id (single one),
Maybe it is rather simple, but should help at least slightly make it faster. Tell us about the results.
That query is a dogs dinner - whoever wrote it should be severely punished, and the person who said you can't rewrite it but must make it run faster should be shot.
Loose the sub-selects!
MySQL does not do push-predicates very well (at all?).
Use proper joins instead and state implied joins:
SELECT ap.inds, SUM(ap.clk)
FROM advts_post AS ap
, comp AS co
, advts ad
WHERE ap.comp_id = co.comp_id
AND ad.comp_id = co.comp_id
AND ap.comp_id = ad.comp_id
AND co.name='abc'
GROUP BY ap.inds
ORDER BY ISNULL(ap.inds), ap.inds ASC