Please consider a table with queue_name, priority and message_timestamp columns.
Here is a compound index for that:
CREATE INDEX STATE_QUEUENAME_PRIORITY_TIMESTAMP ON
`queue_messages` (queue_name, state, priority, message_timestamp);
EXPLAIN for SELECT:
EXPLAIN SELECT message_timestamp
from queue_messages
WHERE queue_name = 'folder'
AND state = 0
ORDER BY priority DESC, message_timestamp DESC
LIMIT 1;
Returns Using where; Using index
EXPLAIN for UPDATE (with the same WHERE and ORDER BY!):
EXPLAIN UPDATE queue_messages
SET state = 1
WHERE queue_name = 'folder'
AND state = 0
ORDER BY priority DESC, message_timestamp DESC
LIMIT 1;
Returns Using where; Using filesort
--
It causes significant performance impact (20ms SELECT vs 90ms UPDATE on 50k rows).
How can I force MariaDB (MySQL) to get rid of filesort in UPDATE statement?
For the update, you need a second index:
CREATE INDEX STATE_QUEUENAME_TIMESTAMP_2 ON `queue_messages` (queue_name, state, priority, message_timestamp);
Your existing index doesn't include priority, so it cannot be used for the order by.
Related
I want to improve my current query. So I have this table called Incomes. Where I have a sourceId varchar field. I have a single SELECT for the fields I need, but I needed to add an extra field called isFirstTime to represent if it was the first time on the row on what that sourceId was used. This is my current query:
SELECT DISTINCT
`income`.*,
CASE WHEN (
SELECT
`income2`.id
FROM
`income` as `income2`
WHERE
`income2`."sourceId" = `income`."sourceId"
ORDER BY
`income2`.created asc
LIMIT 1
) = `income`.id THEN true ELSE false END
as isFirstIncome
FROM
`income` as `income`
WHERE `income`.incomeType IN ('passive', 'active') AND `income`.status = 'paid'
ORDER BY `income`.created desc
LIMIT 50
The query works but slows down if I keep increasing the LIMIT or OFFSET. Any suggestions?
UPDATE 1:
Added WHERE statements used on the original query
UPDATE 2:
MYSQL version 5.7.22
You can achieve it using Ordered Analytical Function.
You can use ROW_NUMBER or RANK to get the desired result.
Below query will give the desired output.
SELECT *,
CASE
WHEN Row_number()
OVER(
PARTITION BY sourceid
ORDER BY created ASC) = 1 THEN true
ELSE false
END AS isFirstIncome
FROM income
WHERE incomeType IN ('passive', 'active') AND status = 'paid'
ORDER BY created desc
DB Fiddle: See the result here
My first thought is that isFirstIncome should be an extra column in the table. It should be populated as the data is inserted.
If you don't like that, let's try to optimize the query...
Let's avoid doing the subquery more than 50 times. This requires turning the query inside-out. (It's like "explode-implode", where the query gathers lots of stuff, then sorts it and throws most of the rows away.)
To summarize:
do the least amount of effort to just identify the 5 rows.
JOIN to whatever tables are needed (including itself if appropriate); this is to get any other columns desired (including isFirstIncome).
SELECT i3.*,
( ... using i3 ... ) as isFirstIncome
FROM (
SELECT i1.id, i1.sourceId
FROM `income` AS i1
WHERE i1.incomeType IN ('passive', 'active')
AND i1.status = 'paid'
ORDER BY i1.created DESC
LIMIT 50
) AS i2
JOIN income AS i3 USING(id)
ORDER BY i2.created DESC -- yes, repeated
(I left out the computation of isFirstIncome; it is discussed in other Answers. But note that it will be executed at most 50 times.)
(The aliases -- i1, i2, i3 -- are numbered in the order they will be "used"; this is to assist in following the SQL.)
To assist in performance, add
INDEX(status, incomeType, created, id, sourceId)
It should help with my formulation, but probably not for the other versions. Your version would benefit from
INDEX(sourceId, created, id)
SELECT `f`.*
FROM `files_table` `f`
WHERE f.`application_id` IN(6)
AND `f`.`project_id` IN(130418)
AND `f`.`is_last_version` = 1
AND `f`.`temporary` = 0
AND f.deleted_by is null
ORDER BY `f`.`date` DESC
LIMIT 5
When I remove the ORDER BY, query executes in 0.1 seconds. With the ORDER BY it takes 3 seconds.
There is an index on every WHERE column and there is also an index on ORDER BY field (date).
What can I do to make this query faster? Why is ORDER BY slowing it down so much? Table has 3M rows.
instead of an index on each column in where be sure you have a composite index that cover all the columns in where
eg
create index idx1 on files_table (application_id, project_id,is_last_version,temporary,deleted_by)
avoid IN clause for single value use = for these
SELECT `f`.*
FROM `files_table` `f`
WHERE f.`application_id` = 6
AND `f`.`project_id` = 130418
AND `f`.`is_last_version` = 1
AND `f`.`temporary` = 0
AND f.deleted_by is null
ORDER BY `f`.`date` DESC
LIMIT 5
the date or others column in select could be useful retrive all info using the index and avoiding the access to the table data .. but for select all (select *)
you probably need severl columns an then the access to the table data is done however .. but you can try an eval the performance ..
be careful to place the data non involved in where at the right of all the column involved in where
create index idx1 on files_table (application_id, project_id,is_last_version,temporary,deleted_by, date)
I'm new in sql and I have some problems with complex selection. How can I delete two rows with smallest base_amount with base_currency 'USD'?
In MySQL, you can use ORDER BY and LIMIT in a DELETE:
delete from t
where base_currency = 'USD'
order by base_amount asc
limit 2;
Suppose my query is:
SELECT adstable.adid FROM `adstable`
inner join userstable on
(adstable.adid = userstable.adid)
WHERE adstable.desktopimp > 100
and adstable.mobileimp > 100
and adstable.userbal > 0.02
and adstable.realbal> 0.02
order by adstable.imptotal asc
Which columns should I index, to ensure a 'covering index' for this query?
Can there be more than one "user" per "ad"? Can there be zero? If neither, then
SELECT a.adid
FROM `adstable` AS a
WHERE a.desktopimp > 100
and a.mobileimp > 100
and a.userbal > 0.02
and a.realbal> 0.02
AND EXISTS (
SELECT *
FROM userstable
WHERE adid = a.adid
)
order by a.imptotal asc
Whether or not that is a better formulation, have these indexes:
userstable: INDEX(adid)
adstable: INDEX(imptotal) -- in case it is better to avoid the sort
INDEX(desktopimp) -- in case it is most selective
INDEX(mobileimp)
INDEX(userbal)
INDEX(realbal)
A "covering" index would include all 6 columns of adstable that are used in the query. That is rather bulky, and of dubious use.
Please provide SHOW CREATE TABLE -- it may be instructive to see the datatypes, engine, etc.
When building an optimal index, start with any columns compared with =. If that takes care of all the WHERE, then move on to the ORDER BY columns. You don't have that case, since you are not using =.
I have a SQL query like this, and it takes high amount of loading...
Not sure what is happening here...if anyone could help me with this.
SELECT `posts`.* FROM `posts`
WHERE `posts`.`type`
IN ('MySubDomainSitePost')
AND `posts`.`aasm_state` = 'published'
AND (published_at <= '2015-05-12 01:01:01')
AND `posts`.`on_frontpage` = 1
AND `posts`.`is_pinned` = 0
ORDER BY published_at DESC LIMIT 16
You need to check the query performance using explain select. Now for large data-set the query will perform very poorly if the columns are not indexed.
From the given query you may need to add the following indexes
alter table posts
add index p_search_idx(type,aasm_state,published_at,on_frontpage,is_pinned);
This will boost up the speed of the query.
Make sure to take a backup of the table before applying the index.
And no need to use IN in the query it could be as
SELECT `posts`.* FROM `posts`
WHERE `posts`.`type` = 'MySubDomainSitePost'
AND `posts`.`aasm_state` = 'published'
AND (published_at <= '2015-05-12 01:01:01')
AND `posts`.`on_frontpage` = 1
AND `posts`.`is_pinned` = 0
ORDER BY published_at DESC LIMIT 16