I have a table containing some duplicate values for 1 column, ie with table emails
id email
1 test#test.com
2 test#test.com
3 more#most.many
4 cook#sheep.com
I'd like to remove row with id '2'. And I'd like to do this by creating a unique index of email thus forcing the table to drop redundancies.
I have seen this method referenced here(http://www.it-iss.com/mysql/sql-removing-duplicate-records/) and https://stackoverflow.com/questions/19000050/how-to-delete-first-of-a-double-record-with-alter-ignore-command-in-mysql
But when I attempt the statement
alter ignore table emails_test add unique index(email)
I get a duplicate entry error for test#test.com, as if I never included the ignore keyword
Is there something I'm missing here? If this is not possible, what are alternative methods of deleting duplicates that are simpler than, say, using temporary tables MySQL Error 1093 - Can't specify target table for update in FROM clause
You could try doing this as a few separate steps:
CREATE TABLE _emails LIKE emails
ALTER TABLE _emails ADD UNIQUE INDEX(email)
INSERT IGNORE INTO _emails SELECT * FROM emails
RENAME TABLE emails TO emails_old, _emails TO emails
Related
I get database table which contain postal numbers and regions for my country. That table have all information but i need to change it for my purpose.
I need to eliminate all rows that have duplicate content in specific column.
Check screenshot to see result
I want to remove all duplicate rows which have postanski_broj (postal_number) the some. That number need to be unique. I try manualy to set that column to unique but i get duplicate entry when i try to execute statment.
ID is primary key with auto increment.
postanski_broj column is VARCHAR which represent postal_code
naselje column is VARCHAR which represent region
One region can have one postal_code
I try
ALTER TABLE poste ADD UNIQUE INDEX idx_postanski_br (postanski_broj);
00:03:20 ALTER TABLE poste ADD UNIQUE INDEX idx_postanski_br
(postanski_broj) Error Code: 1062. Duplicate entry '11158' for key
'idx_postanski_br' 0.118 sec
ALTER IGNORE TABLE poste ADD UNIQUE INDEX idx_postanski_br (postanski_broj);
00:04:17 ALTER IGNORE TABLE poste ADD UNIQUE INDEX idx_postanski_br
(postanski_broj) Error Code: 1064. You have an error in your SQL
syntax; check the manual that corresponds to your MySQL server version
for the right syntax to use near 'IGNORE TABLE poste ADD UNIQUE INDEX
idx_postanski_br (postanski_broj)' at line 1 0.00037 sec
Anyone have sugestion? Thanks
If you have other columns with different values than the ones you've shown there (except for id), deleting should be your last choice.
I usually would duplicate the table first:
CREATE TABLE poste_new LIKE poste;
add unique index to the newly created poste_new table:
ALTER TABLE poste_new ADD UNIQUE INDEX idx_postanski_br (postanski_broj);
insert the data from poste into poste_new with IGNORE option to skip duplicates based on the unique index:
INSERT IGNORE INTO poste_new SELECT * FROM poste;
rename the tables:
RENAME TABLE poste TO poste_old;
RENAME TABLE poste_new TO poste;
The good thing about this is that you've minimized the risk of wrong delete and if you're not satisfied with the new table, you still have the old table intact - effectively making it a backup.
This solution can take too much time for big tables. Best way of solving this is: Remove duplicate rows in MySQL
You have to delete the rows before applying the unique constraint. Be careful applying this:
DELETE p1 FROM poste p1
INNER JOIN poste p2
WHERE
p1.id < p2.id AND
p1.postanski_broj = p2.postanski_broj;
This should remove the duplicated ones and will keep only the ones with the higher id (id=168044 in your example).
I have table with 7 columns where I want to combine three column and want to remove duplicates. Your help will be appreciated.
I'm guessing you want to update the table SERIAL_NUMBERS with just a single row of combined warranty_indicator, account, date_sold data. If that's true then I will suggest the following.
Duplicate SERIAL_NUMBERS table:
CREATE TABLE SERIAL_NUMBERS_NEW LIKE SERIAL_NUMBERS;
Add unique constraint - combination of warranty_indicator, account, date_sold:
ALTER TABLE SERIAL_NUMBERS_NEW
ADD CONSTRAINT Index1 UNIQUE (warranty_indicator,account,date_sold);
Insert data from SERIAL_NUMBERS table to SERIAL_NUMBERS_NEW using INSERT IGNORE ; to ignore duplicates:
INSERT IGNORE INTO SERIAL_NUMBERS_NEW
SELECT * FROM SERIAL_NUMBERS;
Rename old and new table:
RENAME TABLE SERIAL_NUMBERS TO SERIAL_NUMBERS_OLD;
RENAME TABLE SERIAL_NUMBERS_NEW TO SERIAL_NUMBERS;
Check data:
SELECT * FROM SERIAL_NUMBERS_OLD;
SELECT * FROM SERIAL_NUMBERS;
Keep in mind that any future data inserted will treat duplicates according to the unique constraint. Therefore, if you have program running the INSERT syntax, make sure you update it to INSERT IGNORE.
Demo fiddle
I have an issue with duplicate entries in a database and due to the nature of the problem the easiest way to fix it would be to remove current duplicate rows and prevent further duplicates from being added .
Here is the table structure :
| a | b | c |
user url1 token1
photo url1 token2
action action1 token3
user url1 token4
photo url1 token5
action action2 token6
I want to prevent duplicate entries only when 2 columns are duplicated, in this case a and b .
So here we have user | url1 and photo | url1 duplicated twice.
I want to prevent any further duplicates from being added when both columns match another row at same time but the queries I found so far will consider each column separately and preventing any further duplicates to be added to any of them .
Can I achieve this with a mysql query using unique index ?
I tried using the following code :
Using ALTER TABLE `targets` ADD UNIQUE (
`a` ,
`b`
);
Your question:"Can I achieve this with a mysql query using unique index ?"
Answer is 100% yes.
There are two ways of creating index:
1. CREATE UNIQUE INDEX index_name
ON table_name (column1, column2, ...);
2. ALTER TABLE table_name
ADD UNIQUE index_name (column1, column2, ...);
However, this will only work if your table doesn't have existing duplicate data. Otherwise you'll receive an error message like this:
Query: CREATE UNIQUE INDEX index_name ON targets (a, b)
Error Code: 1062
Duplicate entry 'photo-url1' for key 'index_name'
Therefore, you need to:
create a new empty table similar to your targets table.
create unique index.
INSERT IGNORE data from the old table.
Rename targets to targets_old and targets_new to targets.
Example:
CREATE TABLE targets_new LIKE targets;
CREATE UNIQUE INDEX index_name
ON targets_new (a, b);
INSERT IGNORE INTO targets_new SELECT * FROM targets;
RENAME TABLE targets TO targets_old;
RENAME TABLE targets_new TO targets;
Thanks for the replies guys, but I found the solution in the meantime and it was much simpler !
It's called unique composite key and it allows to do exactly what I wanted :
ALTER TABLE targets ADD UNIQUE KEY `uidx` (a, b, c);
Problem fixed :)
i don't think you can specify a unique doublet property.
edit : mention by Salmon : An index specification of the form (key_part1, key_part2, ...) creates an index with multiple key parts
https://dev.mysql.com/doc/refman/8.0/en/create-index.html#create-index-unique
you could try :
SELECT * FROM database1.table1 WHERE a='user' AND b='url1';
if got return a number of rows then don't add to the database.
this way you can fully control what goes into the database instead of letting the table automatically ignore the fail condition.
I want to copy all the rows from one table to another and change the ID if there is a duplicate.
I'm using phpmyadmin and tried the operations tab.
Copy table to (database.table):
Data Only
Add Auto Increment
This is the SQL it gives me:
INSERT INTO `wsuca2_dbwsuca2`.`cxtb4_menu` SELECT * FROM `wsuca2_dbwsuca2`.`j25_menu`
This is the error I'm getting:
#1062 - Duplicate entry '0-0-root-*' for key 'idx_client_id_parent_id_alias_language'
what you need is
INSERT ...... ON DUPLICATE KEY UPDATE
also, "change the ID if there is a duplicate" is not enough it seems. Your table have a complex unique key of client_id,parent_id,alias,language.
both your table had an entry of with the above field set to 0-0-root-* so it throws a error because MySQL doesn't know how to handle it.
either
update those entry manually before copying over
use INSERT ...... ON DUPLICATE KEY UPDATE to specify how to update those entries when found.
use INSERT IGNORE to ignore all duplicate entries (probably not what you want)
Having Duplicate key error is clear notification that you did not use the same structure and keys in the both tables. So first recreate the structure:
DROP TABLE `wsuca2_dbwsuca2`.`cxtb4_menu`;
SHOW CREATE TABLE `wsuca2_dbwsuca2`.`j25_menu`; //old table structure
and paste the structure of j25_menu as structure od cxtb4_menu (change the name of the table). After that insert the data with your INSERT clause.
Here is the scenario:
I have 2 tables and 2 temporary tables. Before I insert user data to the official tables, I insert them to a temp table to let them do the checks. There is a company table with company info, and a contact table that has contact info. The contact table has a field called company_id which is a foreign key index for the company table.
Temp tables are set up the same way.
I want to do something like: INSERT INTO company () SELECT * FROM temp_company; and INSERT INTO contact () SELECT * FROM temp_contact
My question is, how do I transfer the foreign key from the temp_company to the newly inserted id on the company table using a statement like this? Is there a way to do it?
Currently I am:
grabbing the temp rows
going one by one and inserting them
grabbing the last insert id
then inserting the contacts afterwards with the new last insert id
I just don't know if that is the most efficient way. Thanks!
if you have the same number of columns in both tables and then you should just be able to use the syntax you have there? Just take out the (). Just make sure there aren't any duplicate primary keys:
INSERT INTO company SELECT * FROM temp_company;
INSERT INTO contact SELECT * FROM temp_contact;
You can also specifically specify the columns that get inserted, this way you can specify exactly which column you insert as the new ID.
INSERT INTO company (`ID`,`col_1`,...,`last_col`) SELECT `foreign_key_col`,`col_1`,...,`last_col` FROM temp_company;
INSERT INTO contact (`ID`,`col_1`,...,`last_col`) SELECT `foreign_key_col`,`col_1`,...,`last_col` FROM temp_contact;
Just make sure you are selecting the right # of columns.