Delete duplicate records in MYSQL with condition - mysql

I am using a command that runs every second on node.js It has the function of excluding any duplicate records for a given item, which is specified as in the example by: AND t1.auction_id = 1335.
DELETE FROM bid_account t1
WHERE t1.id < (Select max(t1.id) FROM bid_account t2 WHERE t1.bidding_price = t2.bidding_price) AND t1.auction_id = 1335;
I need it to delete a record that has an equal value in the bidding_price column, and keep only one. But it is important that he does this search not across the table, but rather for a certain item as I reported at the beginning, through the column auction_id.
I tried to run the above command, but it returns the following error:
#1064 - You have a syntax error in your SQL next to 't1
WHERE t1.id < (Select max(t1.id) FROM bid_account t2 WHERE t1.bidding_price ' na linha 1
What is wrong with this query?
I use the MYSQL database, and the table bid_account has the id column as index and primary.
If I use SELECT below, it returns the values in duplicity normally.
SELECT bidding_price, count(*) FROM bid_account WHERE `auction_id` = 1335 GROUP BY bidding_price Having Count(*) > 1

You have two quirks from MySQL. First, when you use an alias for a table, it needs to follow the DELETE. So, this is what you intend:
DELETE ba FROM bid_account ba
WHERE ba.id < (Select max(ba2.id) -- I assume you want the max from the innter reference
FROM bid_account ba2
WHERE ba2.bidding_price = ba.bidding_price
) AND
ba.auction_id = 1335;
However, this still will not work, because you are referencing bid_account in the table.
So, I think you want something more like this:
DELETE ba
FROM bid_account ba JOIN
(SELECT ba2.auction_id, ba2.bidding_price, MAX(ba2.id) as max_id -- I assume you want the max from the innter reference
FROM bid_account ba2
WHERE ba2.auction_id = 1335
GROUP BY ba2.auction_id, ba2.bidding_price
) ba2
ON ba2.auction_id = ba.ba2.auction_id AND
ba2.bidding_price = ba.bidding_price AND
ba2.max_id > ba.id
WHERE ba.auction_id = 1335;
This still doesn't strike me as useful. Matching rows based on bidding_price seems unusual. I would expect auction_id to be part of the match. I suspect you want to GROUP BY and JOIN on both bidding_price and auction_id.

You can delete row by using alias before from keyword in sql query
for example
delete t from test t where t.id = 3
So your query will work fine
DELETE t1 FROM bid_account t1
WHERE t1.id < (Select max(t1.id) FROM bid_account t2 WHERE t1.bidding_price = t2.bidding_price) AND t1.auction_id = 1335;

Related

Using group by in SET clause

I'm trying to update a column of a table so that is equal to the count of something in another table. Like this:
UPDATE TABLE
SET TOTAL = (SELECT COUNT(f1)
FROM TABLE2
GROUP BY f2);
But I keep getting sub query returns more than 1 row, and I can't think of how to fix it.
UPDATE (copied from the comment)
f2 is the relation between TABLE and TABLE2 – Thomasd d
Based on your comment
f2 is the relation between TABLE and TABLE2
you probably want something like this
UPDATE TABLE T1, (SELECT f2, COUNT(F1) cnt FROM TABLE2 GROUP BY f2) T2
SET T1.TOTAL = T2.cnt
WHERE T1.f2=T2.f2
adapt T1.f2 if necessary
UPDATE t1
SET total = ( SELECT COUNT(f1)
FROM t2
WHERE t1.f2 = t2.f2 );
https://dbfiddle.uk/?rdbms=mysql_8.0&fiddle=91de17deff657f66fa54b42fe20ed3c5
Add WHERE total IS NULL if you do not need to recalculate values for rows which have a value already.
Your subquery is returning multiple values and your SET statement is only expecting one. This might fix your code if that is what you are looking for.
UPDATE TABLE
SET TOTAL = (SELECT COUNT(f1)
FROM TABLE2)

Deleting duplicate values from a mysql table but keep one

I'm trying to delete duplicate rows from a mysql table, but still keep one.
However the following query seemingly deletes every duplicate row and I'm not sure why. Basically I want to delete the row if the outputID, title and type all matches.
DELETE DupRows.*
FROM output AS DupRows
INNER JOIN (
SELECT MIN(Output_ID) AS Output_ID, Title, Type
FROM output
GROUP BY Title, Type
HAVING COUNT(*) > 1
) AS SaveRows
ON SaveRows.Title = DupRows.Title
AND SaveRows.Type = DupRows.Type
AND SaveRows.Output_ID = DupRows.Output_ID;
Just :
DELETE DupRows
FROM output AS DupRows
INNER JOIN output AS SaveRows
ON SaveRows.Title = DupRows.Title
AND SaveRows.Type = DupRows.Type
AND DupRows.Output_ID > SaveRows.Output_ID
This will delete all duplicates on Title and Type while keeping the record with the lowest value.
If you are running MySQL 8.0, you can use window function ROW_NUMBER() to assign a rank to each record in Title/Type groups, ordered by id. Then you can delete all records whose row number is not 1.
DELETE FROM output
WHERE Output_ID IN (
SELECT Output_ID
FROM (
SELECT Output_ID, ROW_NUMBER() OVER(PARTITION BY Title, Type ORDER BY Output_ID) rn
FROM output
) x
WHERE rn > 1
)
Delete From output Where Output_ID NOT IN (
Select MIN(Output_ID) from output Group By Title, Type Having COUNT(*)>1
)
By below query duplicate rows with matching condition get deleted and keeps one oldest unique row.
NOTE:- In my query I used id column is auto increment column.
DELETE t1
FROM output t1, output t2
WHERE t1.Title = t2.Title
AND t1.Type = t2.Type
AND t1.Output_ID = t2.Output_ID
AND t1.id>t2.id
If you want to keep newly inserted unique row just change the last condition as:
DELETE t1
FROM output t1, output t2
WHERE t1.Title = t2.Title
AND t1.Type = t2.Type
AND t1.Output_ID = t2.Output_ID
AND t1.id<t2.id

trouble updating column with subselect in mysql

I have dups in my mysql database that I can find with this query:
select checksum, count(*) c from MY_DATA group by checksum having c > 1;
I get a list of about 200 dups. What I want to do is update a column in the same table to mark them as dups.
I tried this:
update MY_DATA SET DONT_PARSE=1
where (select count(*) c from MY_DATA group by checksum having c > 1);
I get the error: You can't specify target table 'MY_DATA' for update in FROM clause
Anyone have a solution for this?
Use join:
update my_data d join
(select checksum, count(*) as cnt
from MY_DATA
group by checksum
having cnt > 1
) c
on d.checksum = c.checksum
set DONT_PARSE = 1 ;
Unfortunately, MySQL doesn't allow you to reference the table being updated in a subquery in the update (or delete). However, join can usually be used to get around this.
your query seems also not correct (seems that the where clause is not complete) so you could use a INNER JOIN on the subquery that retrieve duplicated
You could try using
update MY_DATA m
INNER JOIN (
select checksum, count(*) c
from MY_DATA
group by checksum
having c > 1
) t on t.checksum = m.checksum
SET DONT_PARSE=1

Delete records with multiple conditions

I need to exclude duplicate records in the bidding_price column with the following conditions:
Table: bid_account
Columns to check:
id = PRIMARY KEY auction_id = ID of each product
bidding_price = inserted value (this must be checked for duplicity for each product)
bid_flag = must always equal the value of: 'd' bidding_type = must always equal the value of:: 's'
It will always exist equal records in the bidding_price column, which it can not have is equal records with the same product ID (auction_id).
Example of how it should not have:
auction_id | bidding_price
------10------------0.02
------10------------0.02
------11------------0.02
------11------------0.02
The correct would be:
auction_id | bidding_price
------10------------0.02
------11------------0.02
I tried with the following command:
DELETE ba
FROM bid_account ba JOIN
(SELECT ba2.auction_id, ba2.bidding_price, MAX(ba2.id) as max_id
FROM bid_account ba2
WHERE ba2.bid_flag = 'd' AND ba2.bidding_type = 's'
GROUP BY ba2.auction_id, ba2.bidding_price
) ba2
ON ba2.auction_id = ba.auction_id AND
ba2.bidding_price = ba.bidding_price AND
ba2.max_id < ba.id
WHERE ba.bid_flag = 'd' AND ba.bidding_type = 's' AND ba.auction_id = ba2.auction_id
The problem is that it deleted multiple records that it should not delete, did not do the validations correctly. How can I do it?
ID is your PRIMARY KEY in the table, so you can get MAX(id) to be your
Reservation ID,then use NOT IN to delete by ID without MAX(id)
You can try this.
DELETE ba FROM bid_account ba
WHERE ba.id NOT IN
(
SELECT max_id FROM
(
SELECT auction_id, bidding_price, MAX(id) max_id
FROM bid_account
WHERE bid_flag = 'd' AND bidding_type = 's'
GROUP BY auction_id, bidding_price
) t
)
sqlfiddle:http://sqlfiddle.com/#!9/0f2e5/1
EDIT
If you want to get lowest-value ID you could use MIN(id) in the subquery in the where clause
DELETE ba FROM bid_account ba
WHERE ba.id NOT IN
(
SELECT min_id FROM
(
SELECT auction_id, bidding_price, MIN(id) min_id
FROM bid_account
WHERE bid_flag = 'd' AND bidding_type = 's'
GROUP BY auction_id, bidding_price
) t
)
sqlfiddle:http://sqlfiddle.com/#!9/ffe92/1
You can delete it by using your PRIMARY KEY: ID.
It is used to uniquely identify the record for delete action.
See demo here: http://sqlfiddle.com/#!9/603d56/1
It makes use of selecting the ID within a subquery(selecting it twice will AVOID the error: target table cannot be specified for update). The subquery is similar to your query and it selects the id that will be retained. Using NOT IN means delete the rest of the rows not equal to these ids in the subquery.
delete e.*
from bid_account e
where e.id not in (
select id from (
select a.id
from bid_account a
join bid_account b
on a.auction_id=b.auction_id
and a.bidding_price=b.bidding_price
and a.bid_flag=b.bid_flag
and a.bidding_type=b.bidding_type
where a.bid_flag='d' and a.bidding_type='s'
and a.id < b.id) tt);
The below statement should give you all the records that you need.
SELECT ba2.auction_id, ba2.bidding_price, MAX(ba2.id) as max_id
FROM bid_account ba2
WHERE ba2.bid_flag = 'd' AND ba2.bidding_type = 's'
GROUP BY ba2.auction_id, ba2.bidding_price;
A not-in clause should give you all the records that you don't need.
So, delete from bid_account where id not in (#sub query to fetch required ids#) and ba.bid_flag = 'd' AND ba.bidding_type = 's'; should delete the duplicate records.

ORDER BY with DELETE not working

I have written this delete query to remove the duplicate entries from table but when I run this query it will keep the old rows and delete the newest one. I want to keep the newest inserted rows and want to delete the old ones. I have appended ORDER BY clause in delete query but it's not working.
DELETE t2
FROM user_deprovision t1
JOIN user_deprovision t2 ON (
t2.UserName = t1.UserName AND
t2.GroupName = t1.Groupname AND
t2.id > t1.id)
ORDER BY t1.DeletedDate;
How can I achieve this.
Let's write a query that returns the latest time for a user name and group name.
SELECT
UserName, GroupName, MAX(deprovision_time)
FROM
user_deprovision
GROUP BY
UserName, GroupName
Now, let's adapt this to your purpose. In Oracle, I might do something to turn that into a temporary table, but I'll use correlated subqueries here instead.
DELETE FROM user_deprovision AS dp1
WHERE dp1.deprovision_time <
ANY (SELECT deprovision_time FROM user_deprovision AS dp2
WHERE dp1.UserName = dp2.UserName
AND dp1.GroupName = dp2.GroupName);
We can turn this into a join instead.
SELECT dp1.id
FROM user_deprovision AS dp1
JOIN user_deprovision AS dp2
ON dp1.UserName = dp2.UserName
AND dp1.GroupName = dp2.GroupName
WHERE
dp1.deprovision_time < dp2.deprovision_time
And the corresponding delete is:
DELETE FROM user_deprovision
WHERE id IN (
SELECT dp1.id
FROM user_deprovision AS dp1
JOIN user_deprovision AS dp2
ON dp1.UserName = dp2.UserName
AND dp1.GroupName = dp2.GroupName
WHERE
dp1.deprovision_time < dp2.deprovision_time);