I'm trying to delete duplicate rows from a mysql table, but still keep one.
However the following query seemingly deletes every duplicate row and I'm not sure why. Basically I want to delete the row if the outputID, title and type all matches.
DELETE DupRows.*
FROM output AS DupRows
INNER JOIN (
SELECT MIN(Output_ID) AS Output_ID, Title, Type
FROM output
GROUP BY Title, Type
HAVING COUNT(*) > 1
) AS SaveRows
ON SaveRows.Title = DupRows.Title
AND SaveRows.Type = DupRows.Type
AND SaveRows.Output_ID = DupRows.Output_ID;
Just :
DELETE DupRows
FROM output AS DupRows
INNER JOIN output AS SaveRows
ON SaveRows.Title = DupRows.Title
AND SaveRows.Type = DupRows.Type
AND DupRows.Output_ID > SaveRows.Output_ID
This will delete all duplicates on Title and Type while keeping the record with the lowest value.
If you are running MySQL 8.0, you can use window function ROW_NUMBER() to assign a rank to each record in Title/Type groups, ordered by id. Then you can delete all records whose row number is not 1.
DELETE FROM output
WHERE Output_ID IN (
SELECT Output_ID
FROM (
SELECT Output_ID, ROW_NUMBER() OVER(PARTITION BY Title, Type ORDER BY Output_ID) rn
FROM output
) x
WHERE rn > 1
)
Delete From output Where Output_ID NOT IN (
Select MIN(Output_ID) from output Group By Title, Type Having COUNT(*)>1
)
By below query duplicate rows with matching condition get deleted and keeps one oldest unique row.
NOTE:- In my query I used id column is auto increment column.
DELETE t1
FROM output t1, output t2
WHERE t1.Title = t2.Title
AND t1.Type = t2.Type
AND t1.Output_ID = t2.Output_ID
AND t1.id>t2.id
If you want to keep newly inserted unique row just change the last condition as:
DELETE t1
FROM output t1, output t2
WHERE t1.Title = t2.Title
AND t1.Type = t2.Type
AND t1.Output_ID = t2.Output_ID
AND t1.id<t2.id
Related
Thanks to this question Rename Mysql Duplicate Value I was able to come up with this queryn to elminate the duplicate rows.
UPDATE table1
inner join (SELECT OBJECTID,CONCAT(IDENT,'_1') as IDENT FROM table1
GROUP BY IDENT HAVING COUNT(*) > 1) t
on t.OBJECTID = table1.OBJECTID
SET table1.IDENT = t.IDENT;
This works well but I want to only rename the rows where the column IDENT is duplicated and the NAME column is different. Any ideas how to do this?
Change the grouping to be both NAME and IDENT.
UPDATE table1
JOIN (
SELECT MAX(objectid) AS max_id, name, CONCAT(ident, '_1') AS new_ident
FROM table1
GROUP BY name, ident
HAVING COUNT(*) > 1
) AS t ON t.max_id = table1.objectid
SET table1.ident = t.new_ident
I'm trying to update a column of a table so that is equal to the count of something in another table. Like this:
UPDATE TABLE
SET TOTAL = (SELECT COUNT(f1)
FROM TABLE2
GROUP BY f2);
But I keep getting sub query returns more than 1 row, and I can't think of how to fix it.
UPDATE (copied from the comment)
f2 is the relation between TABLE and TABLE2 – Thomasd d
Based on your comment
f2 is the relation between TABLE and TABLE2
you probably want something like this
UPDATE TABLE T1, (SELECT f2, COUNT(F1) cnt FROM TABLE2 GROUP BY f2) T2
SET T1.TOTAL = T2.cnt
WHERE T1.f2=T2.f2
adapt T1.f2 if necessary
UPDATE t1
SET total = ( SELECT COUNT(f1)
FROM t2
WHERE t1.f2 = t2.f2 );
https://dbfiddle.uk/?rdbms=mysql_8.0&fiddle=91de17deff657f66fa54b42fe20ed3c5
Add WHERE total IS NULL if you do not need to recalculate values for rows which have a value already.
Your subquery is returning multiple values and your SET statement is only expecting one. This might fix your code if that is what you are looking for.
UPDATE TABLE
SET TOTAL = (SELECT COUNT(f1)
FROM TABLE2)
I want to update a column in a MySQL table that contains string values with values from another table that contains unique names, but without repeating the same values.
Each name in the 'names' table has a unique id from 1 - 1001.
There are 161 names in the target table.
I tried something like this, but it fetches duplicate entries:
UPDATE table_to_update
SET name =
( SELECT name FROM `names`
WHERE id >= (SELECT FLOOR(RAND()*(SELECT MAX(id) FROM `names`)) )
LIMIT 1 ) ;
Here is one option if you are using MySQL 8+, which supports ROW_NUMBER:
UPDATE table_to_update t1
INNER JOIN
(
SELECT id, ROW_NUMBER() OVER (ORDER BY id) rn
FROM table_to_update
) t2
ON t1.id = t2.id
INNER JOIN
(
SELECT name, ROW_NUMBER() OVER (ORDER BY id) rn
FROM (SELECT name, id FROM names ORDER BY RAND() LIMIT 161) t
) t3
ON t2.rn = t3.rn
SET
t1.name = t3.name;
Note that I hard-coded the LIMIT value to be 161 in the subquery, given that you know the size of the target table. This value could be made dynamic as well, but would require more work.
I have a table named consignment which has some duplicate rows against column "service" where service='CLRC'.
select * from consignment where service='CLRC'
When i select the rows, i have total 2023 rows which includes duplicates.
I wrote the below query to delete the rows but i want to select them first to make sure its deleting the correct records.
When the select runs it returns 64431 records. Is that correct?
select t1.hawb FROM consignment t1
INNER JOIN consignment t2
WHERE
t1.id < t2.id AND
t1.hawb = t2.hawb
and t1.service='CLRC'
If you expect your query to return the number of duplicates then no it is not correct.
The condition t1.id < t2.id will join every id of t1 with all ids from t2 that are greater resulting on more rows or less rows (in the case of only 2 duplicates) and rarely in the expected number.
See the demo.
If you want to see all the duplicates:
select * from consignment t
where t.service = 'CLRC'
and exists (
select 1 from consignment
where service = t.service and id <> t.id and hawb = t.hawb
)
See the demo.
If you want to delete the duplicates and keep only the one ones with the max id for each hawb then:
delete from consignment
where service='CLRC'
and id not in (
select id from (
select max(id) id from consignment
where service='CLRC'
group by hawb
) t
);
See the demo.
Include all the columns in the matching condition except id column, as being primary key :
delete t1
from consignment t1
join consignment t2
where t1.id < t2.id
and t1.hawb = t2.hawb
and t1.col1=t2.col1
and t1.col2=t2.col2
......
and t1.service='CLRC';
Demo
You can check the number of duplicates by
select count(*) from
(
select distinct hawb, col1, col2, service -- (all columns except `id`)
from consignment
) q
check whether this number equals number of deleted records just before commiting the changes.
I have a query in MySQL based on which I am finding duplicate records of some columns.
select max(id), count(*) as cnt
from table group by start_id, end_id, mysqltable
having cnt>1;
This above query gives me the max(id) and the count of number of records that have start_id,end_id,mysqltable column values same.
I want to delete all the records that match the max(id) column of the above query
How can I do that?
I have tried like below
delete from table
where (select max(id), count(*) as cnt
from table group by start_id,end_id,mysqltable
having cnt>1)
But Unable to delete records
You can remove duplicate records using JOIN.
DELETE t1 FROM table t1
INNER JOIN
table t2
WHERE
t1.id > t2.id AND t1.start_id = t2.start_id AND t1.end_id = t2.end_id AND t1.mysqltable = t2.mysqltable;
This query keeps the lowest id and remove the highest.
I think so this command should work:
delete from table
where id in
( select max(id) from table
group by start_id, end_id, mysqltable
having count(*) > 1
);