Hi i have a mysql table without primary key and i need to delete the duplicated rows. how can i do so?
user_id category_id
1 2
1 3
1 4
1 2
2 2
2 3
2 2
CREATE TABLE temp SELECT DISTINCT * FROM tablename;
ALTER TABLE tablename RENAME junk;
ALTER TABLE temp RENAME tablename;
Since you cannot differentiate 2 identical rows, you cannot delete just one of them. The way you need to think about it is like this:
insert into new_better_table
select user_id, category_id from old_table group by user_id, category_id
It's possible to use a dirty flag with default value 1 and copy only single records with the flag set to 0, then simply remove all dirty records. This way you don't need another table.
Assuming you already created a dirty flag with default value 1:
insert into mytable
select fld1,fld2,fldN,0 as dirty
from mytable
group by duplicate_field
Then you can just delete dirty records:
delete from mytable where dirty = 1
Don't forget to remove the dirty flag. You're done.
The select distinct * solution proposed in another answer will not work in cases where your rows contain null values. A better solution that also prevents future duplicates from appearing in my_table in the first place is as follows:
create table no_dupes like my_table;
alter table no_dupes add unique index(unique_column,or_columns);
insert ignore into no_dupes select * from my_table;
rename table my_table to junk, no_dupes to my_table;
Related
I have a table called leads with duplicate records
Leads:
*account_id
*campaign_id
I want to remove all the duplicate account_id where campaign_id equal to "51"
For example, if account_id = 1991 appears two times in the table then remove the one with campaign_id = "51" and keep the other one.
You could use a delete join:
DELETE t1
FROM yourTable t1
INNER JOIN yourTable t2
ON t2.account_id = t1.account_id AND
t2.campaign_id <> 51
WHERE
t1.campaign_id = 51;
There's no problem to delete from a table provided that:
You use the correct syntax.
You have done a backup of the table BEFORE you do any deleting.
However, I would suggest a different method:
Create a new table based on the existing table:
CREATE TABLE mytable_new LIKE mytable;
Add unique constraint (or PRIMARY KEY) on column(s) you don't want to have duplicates:
ALTER TABLE mytable_new ADD UNIQUE(column1,[column2]);
Note: if you want to identify a combination of two (or more) columns as unique, place all the column names in the UNIQUE() separated by comma. Maybe in your case, the constraint would be UNIQUE(account_id, campaign_id).
Insert data from original table to new table:
INSERT IGNORE INTO mytable_new SELECT * FROM mytable;
Note: the IGNORE will insert only non-duplicate values that match with the UNIQUE() constraint. If you have an app that runs a MySQL INSERT query to the table, you have to update the query by adding IGNORE.
Check data consistency and once you're satisfied, rename both tables:
RENAME TABLE mytable TO mytable_old;
RENAME TABLE mytable_new TO mytable;
The best thing about this is that in case that if you see anything wrong with the new table, you still have the original table.
Changing the name of the tables only take less than a second, the probable issue here is that it might take a while to do the INSERT IGNORE if you have a large data.
Demo fiddle
DELETE t1
FROM yourTable t1
INNER JOIN yourTable t2
ON t2.account_id = t1.account_id AND
t2.campaign_id <> 51
WHERE
t1.campaign_id = 51;
I have table which looks like this. I want to delete duplicates and leave one row for each user. How do I go about this?
Table*
id user
Thango 1
Thango 1
Samg 2
Samg 2
Results
id user
Thango 1
Samg 2
For this dataset, it is probably simpler to just empty and refill the table:
-- deduplicate into a temporary table
create table mytmp as select distinct id, user from mytable;
-- empty the original table (backup your data first!)
truncate table mytable;
-- refill the table from the temporary table
insert into mytable(id, user) select id, user from mytmp;
-- drop the temporary table
drop table mytemp;
Once this is done, you might consider creating a unique constraint on the table to avoid further duplicates:
alter table mytable
add constraint myconstraint
unique (id, user);
You use Common Table Expressions (CTE). For a better explanation I advise you to take a look into this url
A possible solution:
WITH CTE([user],
duplicatecount)
AS (SELECT [user],
ROW_NUMBER() OVER(PARTITION BY [user]
ORDER BY [id]) AS DuplicateCount
FROM dbo.[YourDataBase])
DELETE FROM CTE
WHERE DuplicateCount > 1
I am a Linux admin with only basic knowledge in Mysql Queries
I want to delete many table entries which are ip address from my table using id,
currently i am using
DELETE from tablename where id=1;
DELETE from tablename where id=2;
but i have to delete 254 entries,so this method is going to take hours,how can i tell mysql to delete rows that i specify,coz i want to skip deleting some entries out of this 254.
Deleting whole table and importing needed entries is not an option.
The best way is to use IN statement :
DELETE from tablename WHERE id IN (1,2,3,...,254);
You can also use BETWEEN if you have consecutive IDs :
DELETE from tablename WHERE id BETWEEN 1 AND 254;
You can of course limit for some IDs using other WHERE clause :
DELETE from tablename WHERE id BETWEEN 1 AND 254 AND id<>10;
how about using IN
DELETE FROM tableName
WHERE ID IN (1,2) -- add as many ID as you want.
if you need to keep only a few rows, consider
DELETE FROM tablename WHERE id NOT IN (5,124,221);
This will keep only some records and discard others.
Something like this might make it a bit easier, you could obviously use a script to generate this, or even excel
DELETE FROM tablename WHERE id IN (
1,
2,
3,
4,
5,
6
);
If you have some 'condition' in your data to figure out the 254 ids, you could use:
delete from tablename
where id in
(select id from tablename where <your-condition>)
or simply:
delete from tablename where <your-condition>
Simply hard coding the 254 values of id column would be very tough in any case.
Others have suggested IN, this is fine. You can also use a range:
DELETE from tablename where id<254 and id>3;
If the ids to delete are contiguous.
Use IN Clause
DELETE from tablename where id IN (1,2);
OR you can merge the use of BETWEEN and NOT IN to decrease the numbers you have to mention.
DELETE from tablename
where (id BETWEEN 1 AND 255)
AND (id NOT IN (254));
DELETE FROM table_name WHERE id BETWEEN 1 AND 256;
Try This.
Hope it helps:
DELETE FROM tablename
WHERE tablename.id = ANY (SELECT id FROM tablename WHERE id = id);
DELETE FROM tablename WHERE id > 0;
OR
DELETE FROM tablename WHERE id <255;
It deletes id from 1 to 254
This question already has answers here:
Remove duplicate rows in MySQL
(26 answers)
Closed 5 years ago.
I have a table with a few thousand rows. The table contains two columns, name and email. I have several duplicate rows, for example:
John Smith | john#smith.com
John Smith | john#smith.com
Erica Smith | erica#smith.com
Erica Smith | erica#smith.com
What would be the easiest way to delete all duplicate results. For example, such that the table's content would = SELECT name, DISTINCT(email) FROM table.
You could pretty easily do this by selecting that query into another table, then renaming it to replace the original.
CREATE TABLE `table2` (
`name` varchar(255),
`email` varchar(255),
UNIQUE KEY `email` (`email`));
INSERT INTO `table2` SELECT `name`, DISTINCT(`email`) FROM `table`;
RENAME TABLE `table` TO `table1`;
RENAME TABLE `table2` TO `table`;
Note that this CREATE should be adjusted to your actual table format. I added the unique key on the email field as a suggestion on how you would prevent duplicates in the first place.
Alternatively, you could loop over this
DELETE FROM `table`
WHERE `email` IN (
SELECT `email` FROM `table` GROUP BY `email` HAVING count(*) > 1
) LIMIT 1
Which would delete one duplicate record per call. The importance of the limit is to not remove both rows for any duplicate
The easiest way would be to copy all distinct values into a new table:
select distinct *
into NewTable
from MyTable
DELETE FROM table
WHERE id
NOT IN
(SELECT A.id
FROM
(
SELECT name,MAX(id) AS id
FROM table
GROUP BY name
) A
)
Add an auto-increment field to the table. I believe that when you add it, it will be 'backfilled' for you. Since MySql doesn't allow a delete based on a subquery against the same table, the easiest solution is to then dump the whole dataset into a temptable for use in processing. Assuming you called the new field RowId and the temp table tempTable, you can then use code like this:
DELETE FROM NameAndEmail
LEFT JOIN
( SELECT name, email, Max(RowId) as MaxRowId
FROM temptable
GROUP BY name, email
) as MaxId
WHERE NameAndEmail.Email = MaxId.Email
and NameAndEmail.Name = MaxId.Name
and NameAndEmail.RowId <> MaxId.RowId
Add a unique index
The simplest way to clean up a table with duplicate data - is to just add a unique index:
set session old_alter_table=1;
ALTER IGNORE TABLE `table` ADD UNIQUE INDEX (name, email);
Pay particular attention to the first sql statement, without it the IGNORE flag is ignored and the alter table statement will fail with an error.
I have a table with 1000 rows.
I need to delete rows from 5 to 1000, so the first 4 remain intact and eveythin after 4 is deleted.
However if I write something like:
DELETE FROM table_name WHERE id > 4
the numbering of the next record after INSERT is 1001, which I do not want.
How to delete rows, but force the numbering of all next rows to the number of my last id?
P.S. I can not use ALTER TABLE to drop all the table.
Thanks in advance.
Can you do this?
ALTER TABLE theTableInQuestion AUTO_INCREMENT=5
If your id - auto_increment field you need to use ALTER TABLE to change it's value...
But if it's impossible you can try not to delete rows but to set there value to NULL or "" or 0 and then just to update there value
not INSERT but UPDATE ... WHERE id = 5
With MySQL? You don't. AUTO_INCREMENT never... decrements unless you call ALTER TABLE. Have you considered use of a trigger DELETE, however?
(You may need to debug this)
delimiter |
CREATE TRIGGER testref AFTER DELETE ON theTableInQuestion
FOR EACH ROW BEGIN
IF SELECT COUNT(*) FROM test1 < 5 THEN
ALTER TABLE theTableInQuestion AUTO_INCREMENT=5
END IF;
END;
|
DBCC CHECKIDENT ( 'table_name', RESEED, new_reseed_value ) ==> Works with MS-SQL
whereas
ALTER TABLE table_name AUTO_INCREMENT = 5 ==> Works with MySQL
Think the table is in auto_increment . If you cant ALTER TABLE you need to manualy give the id in each query, something like
INSERT INTO table (id,value) VALUES ((SELECT MAX(id)+1 FROM table), value);
(see comment vis-a-vis lock table)