do you think a query like this will create problem in the execution of my software?
I need to delete the all the table, except the last 2 groups of entries, grouped by the same time of insert.
delete from tableA WHERE time not in
(
SELECT time FROM
(select distinct time from tableA order by time desc limit 2
) AS tmptable
);
Do you have better solution? I'm using mysql 5.5
I don't see anything wrong with your query, but I prefer using an OUTER JOIN/NULL check (plus it alleviates the need for one of the nested subqueries):
delete a
from tableA a
left join
(
select distinct time
from tableA
order by time desc
limit 2
) b on a.time = b.time
where b.time is null
SQL Fiddle Demo
Related
I have a big table of messages with date and room columns. and 2 billion rows.
now i want keep only last 50 messages for every room and delete previous messages.
can i do it with a fast query ?
this question is unique , i didn't found any other question for delete rows over a grouped and ordered selection
You cannot do it in a fast query. You have a lot of data.
I would suggest creating a new table. You can then replace the data in your first table, if necessary.
Possibly the most efficient method to get the 50 rows -- assuming that date is unique for each room:
select t.*
from t
where t.date >= coalesce((select t2.date
from t t2
where t2.room = t.room
order by t2.date desc
limit 1
), t.date
);
For this to have any hope of performance you want an index on (room, date).
You can also try row_number() in MySQL 8+:
select . . . -- list the columns
from (select t.*, row_number() over (partition by room order by date desc) as seqnum
from t
) t
where seqnum <= 50;
Then you can replace the data by doing:
create table temp_t as
select . . . -- one of the select queries here;
truncate table t; -- this gets rid of all the data, so be careful
insert into t
select *
from temp_t;
Massive inserts are much more efficient than massive updates, because the old data does not need to be logged (nor the pages locked and other things).
You can use Rank() function to get top 50 results for each group ordered by date desc, so the last entries will be in top.
http://www.mysqltutorial.org/mysql-window-functions/mysql-rank-function/
Then you left join that subquery on your table on id ( or room and date, if those are unique and you don’t have id in your table)
The last step would be to filter all such result that have null in subquery and delete those.
The full code will look something like this:
DELETE T FROM YOURTABLE T
LEFT JOIN (
SELECT *,
RANK() OVER (PARTITION BY
ROOM
ORDER BY
[DATE] DESC
) DATE_RANK
) AS T2
ON T.[DATE] = T2.[DATE]
AND T.ROOM = T2.ROOM
AND T2.DATE_RANK<=50
WHERE T2.DATE IS NULL
Im trying to do two queries on the same table to get the Count(*) value.
I have this
SELECT `a`.`name`, `a`.`points` FROM `rank` AS a WHERE `id` = 1
And in the same query I want to do this
SELECT `b`.`Count(*)` FROM `rank` as b WHERE `b`.`points` >= `a`.`points`
I tried searching but did not find how to do a Count(*) in the same query.
Typically you would not intermingle a non aggregate and aggregate query together in MySQL. You might do this in databases which support analytic functions, such as SQL Server, but not in (the current version of) MySQL. That being said, your second query can be handled using a correlated subquery in the select clause the first query. So you may try the following:
SELECT
a.name,
a.points,
(SELECT COUNT(*) FROM rank b WHERE b.points >= a.points) AS cnt
FROM rank a
WHERE a.id = 1;
As I understand from the question, you want to find out in a table for a given id how many rows have the points greater than this row. This can be achieved using full join.
select count(*) from rank a join rank b on(a.id != b.id) where a.id=1 and b.points >= a.points;
I have 3 tables (ex. a,b,c) which indicates activities for different items (ex. commenting, liking, etc) as well as the time for each activity. I am trying to essentially do a sort of news feed that shows the most recent activities first. I constructed a UNION ALL for all three tables to group all the activities together and then a GROUP BY to ensure that activities for the same items are not shown twice and order by time DESC. This function uses an infinite scroll so the query must also be able to shift appropriately.
I am wondering if there is any way to optimize this (Each table is about 500-900K and growing). Truncated code is shown below.
SELECT time,item_id FROM (
SELECT a.time AS time, a.item_id FROM a
UNION ALL
SELECT b.time AS time, b.item_id FROM b
UNION ALL
SELECT c.time AS time, c.item_id FROM c
) temp
GROUP BY item_id
ORDER BY time DESC
LIMIT 10
The query you've written will create a very large temporary table. You're then sorting by a column in that temporary table. You should try to limit each table, perhaps like this:
SELECT time,item_id FROM (
SELECT a.time AS time, a.item_id FROM a LIMIT 10 ORDER BY time DESC
UNION ALL
SELECT b.time AS time, b.item_id FROM b LIMIT 10 ORDER BY time DESC
UNION ALL
SELECT c.time AS time, c.item_id FROM c LIMIT 10 ORDER BY time DESC
) temp
GROUP BY item_id
ORDER BY time DESC
LIMIT 10
You'll want to make sure time has an index on each table.
I don't really like doing this though, as it may be difficult to "scroll" through the results accurately.
When going to the "next page" you may want to consider adding a WHERE clause like WHERE a/b/c.item_id > num instead of LIMIT offset, length. That will help with the accuracy.
When writing the query you should prefix the query with EXPLAIN to see how the query is being handled. This will give you a better idea of what's happening: Are temporary tables being created? How large is it? What indexes are being used? etc...
Another approach could be to use a MySQL trigger to populate a single "feed" table.
I get a complicate query:
SELECT * FROM
(
SELECT Transaction
FROM table1
WHERE
Transaction IN (SELECT Transaction FROM table2 WHERE Plugin='XXX' AND Server='XXX')
AND
Transaction NOT IN (SELECT Transaction FROM table1 WHERE Detail IN ('Monitor','Version','monitor','version'))
ORDER BY Date DESC, Millisecond DESC LIMIT 10)
AS res
I get indexes on table1:Detail and the "Transaction" is the primary key of table2.
It will take a while(5-10 secs) for the database to return result. So I create another index on table2:Plugin, the query is fasted now, but a preparing state shows up and also takes 5-10 secs. So after I create a new index, the time does not change at all.
Can someone tell me what`s going on and how can I optimize this query? Thank you!
Could you not simply rewrite the query as follows:
SELECT a.Transaction
FROM table1 a
INNER JOIN table2 b ON b.Transaction = a.Transaction
WHERE (b.Plugin='XXX' AND b.Server='XXX')
AND a.Detail NOT IN ('Monitor','Version','monitor','version')
ORDER BY a.Date DESC, a.Millisecond DESC LIMIT 10
So you join the table2 (which will be faster) and remove all the subqueries.
This should be much faster.
i want to get last two row of a table in one query as new data and previous data
i got
select tbl.x , tbl2.x as last_x
from tbl left join tbl tbl2 ON tbl.id!= tbl2.id
order by tbl.id desc , tbl2.id desc limit 1
it works fine but i think it might get slow in a big DB
is there any way to make this faster ?
A LIMIT should work in a basic subquery, and so the following will possibly be more efficient
SELECT Sub1.x , Sub2.x as last_x
FROM (SELECT x FROM tbl ORDER BY tbl.id DESC LIMIT 1) Sub1
CROSS JOIN (SELECT x FROM tbl ORDER BY tbl.id DESC LIMIT 2, 1) Sub2
You can take a look at the execution plan and try to optimize your query, but usually you do this when you face a problem so you can determine which parts are taking long.
Chick this thread to: How to optimise MySQL queries based on EXPLAIN plan
But as saied i would not try to solve a problm which still does not exist, i do not actually see a problem aith your query.