I get database table which contain postal numbers and regions for my country. That table have all information but i need to change it for my purpose.
I need to eliminate all rows that have duplicate content in specific column.
Check screenshot to see result
I want to remove all duplicate rows which have postanski_broj (postal_number) the some. That number need to be unique. I try manualy to set that column to unique but i get duplicate entry when i try to execute statment.
ID is primary key with auto increment.
postanski_broj column is VARCHAR which represent postal_code
naselje column is VARCHAR which represent region
One region can have one postal_code
I try
ALTER TABLE poste ADD UNIQUE INDEX idx_postanski_br (postanski_broj);
00:03:20 ALTER TABLE poste ADD UNIQUE INDEX idx_postanski_br
(postanski_broj) Error Code: 1062. Duplicate entry '11158' for key
'idx_postanski_br' 0.118 sec
ALTER IGNORE TABLE poste ADD UNIQUE INDEX idx_postanski_br (postanski_broj);
00:04:17 ALTER IGNORE TABLE poste ADD UNIQUE INDEX idx_postanski_br
(postanski_broj) Error Code: 1064. You have an error in your SQL
syntax; check the manual that corresponds to your MySQL server version
for the right syntax to use near 'IGNORE TABLE poste ADD UNIQUE INDEX
idx_postanski_br (postanski_broj)' at line 1 0.00037 sec
Anyone have sugestion? Thanks
If you have other columns with different values than the ones you've shown there (except for id), deleting should be your last choice.
I usually would duplicate the table first:
CREATE TABLE poste_new LIKE poste;
add unique index to the newly created poste_new table:
ALTER TABLE poste_new ADD UNIQUE INDEX idx_postanski_br (postanski_broj);
insert the data from poste into poste_new with IGNORE option to skip duplicates based on the unique index:
INSERT IGNORE INTO poste_new SELECT * FROM poste;
rename the tables:
RENAME TABLE poste TO poste_old;
RENAME TABLE poste_new TO poste;
The good thing about this is that you've minimized the risk of wrong delete and if you're not satisfied with the new table, you still have the old table intact - effectively making it a backup.
This solution can take too much time for big tables. Best way of solving this is: Remove duplicate rows in MySQL
You have to delete the rows before applying the unique constraint. Be careful applying this:
DELETE p1 FROM poste p1
INNER JOIN poste p2
WHERE
p1.id < p2.id AND
p1.postanski_broj = p2.postanski_broj;
This should remove the duplicated ones and will keep only the ones with the higher id (id=168044 in your example).
Related
I have an issue with duplicate entries in a database and due to the nature of the problem the easiest way to fix it would be to remove current duplicate rows and prevent further duplicates from being added .
Here is the table structure :
| a | b | c |
user url1 token1
photo url1 token2
action action1 token3
user url1 token4
photo url1 token5
action action2 token6
I want to prevent duplicate entries only when 2 columns are duplicated, in this case a and b .
So here we have user | url1 and photo | url1 duplicated twice.
I want to prevent any further duplicates from being added when both columns match another row at same time but the queries I found so far will consider each column separately and preventing any further duplicates to be added to any of them .
Can I achieve this with a mysql query using unique index ?
I tried using the following code :
Using ALTER TABLE `targets` ADD UNIQUE (
`a` ,
`b`
);
Your question:"Can I achieve this with a mysql query using unique index ?"
Answer is 100% yes.
There are two ways of creating index:
1. CREATE UNIQUE INDEX index_name
ON table_name (column1, column2, ...);
2. ALTER TABLE table_name
ADD UNIQUE index_name (column1, column2, ...);
However, this will only work if your table doesn't have existing duplicate data. Otherwise you'll receive an error message like this:
Query: CREATE UNIQUE INDEX index_name ON targets (a, b)
Error Code: 1062
Duplicate entry 'photo-url1' for key 'index_name'
Therefore, you need to:
create a new empty table similar to your targets table.
create unique index.
INSERT IGNORE data from the old table.
Rename targets to targets_old and targets_new to targets.
Example:
CREATE TABLE targets_new LIKE targets;
CREATE UNIQUE INDEX index_name
ON targets_new (a, b);
INSERT IGNORE INTO targets_new SELECT * FROM targets;
RENAME TABLE targets TO targets_old;
RENAME TABLE targets_new TO targets;
Thanks for the replies guys, but I found the solution in the meantime and it was much simpler !
It's called unique composite key and it allows to do exactly what I wanted :
ALTER TABLE targets ADD UNIQUE KEY `uidx` (a, b, c);
Problem fixed :)
i don't think you can specify a unique doublet property.
edit : mention by Salmon : An index specification of the form (key_part1, key_part2, ...) creates an index with multiple key parts
https://dev.mysql.com/doc/refman/8.0/en/create-index.html#create-index-unique
you could try :
SELECT * FROM database1.table1 WHERE a='user' AND b='url1';
if got return a number of rows then don't add to the database.
this way you can fully control what goes into the database instead of letting the table automatically ignore the fail condition.
I have SQL code, how should I change it to insert datas only if they aren't exist in my table. thanks for your answers:)
INSERT INTO stages_done(id_booked_proj, id_stage)
SELECT booked_proj.id, stages.id_st
FROM booked_proj, stages
WHERE booked_proj.name='home' AND stages.name_st= 'sm'"
You can use INSERT IGNORE, this way the row won't be inserted if it results in a duplicate key:
INSERT IGNORE INTO stages_done(id_booked_proj, id_stage)
SELECT booked_proj.id, stages.id_st
FROM booked_proj, stages
WHERE booked_proj.name='home' AND stages.name_st= 'sm'"
Any duplicate key in columns either with PRIMARY KEY or UNIQUE constraints will be ignored.
If you have writing access, you can add a constraint to your table, like this:
ALTER TABLE `stages_done` ADD UNIQUE `id_booked_proj_id_stage_index` (`id_booked_proj`, `id_stage`);
Where id_booked_proj_id_stage_index can be any name you pick as long as it is unique.
I've created multiple indexed tables that I want to tie into a new normalized version of an old table. I get everything indexed and the relations set and I get a "Duplicate entry '11' for key 'Primary' " error message.
Here's the code I'm using to populate the new table.
insert into dvdNormal(dvdId, dvdTitle, year, publicRating, dvdStudioId,
dvdStatusId, dvdGenreId)
(
select dvdId, dvdTitle, year, publicRating, studioId, statusId, genreId
from dvd d
join dvdStudio on d.studio = dvdStudio.studioName
join dvdStatus on d.status = dvdStatus.dvdStatus
join dvdGenre on d.genre = dvdGenre.genre);
I'm going to assume you were asking a question, and not just giving a status report.
The behavior you observe is (most likely) due to the insert statement attempting to insert a row that violates a UNIQUE (or PRIMARY KEY) constraint defined on the dvdId column in the target table (the table the statment is inserting rows into.)
And either 1) the dvdId column is not unique in the table it's being retrieved from, or 2) there is more than one "matching" row in one of the other three tables.
For example, if dvdId is a column in dvd, and it's defined as UNIQUE, then case 1) doesn't apply.
But if that row from dvd has more than one "matching" row from one (or more) of the other three tables, then we'd expect the SELECT to generate "duplicate" values for dvdId.
For example, if the genre column is not unique in dvdGenre table, or studioName column is not unique in dvdStudio, we'd expect the query to return multiple copies of the row from dvd. The redundant data (duplicated values) is expected when we "denormalize" data.
If we want to get the table loaded from the query, there's a couple of options.
If we want to store every row returned by the query, we would remove the UNIQUE constraint from the dvdId column. (There may also be other UNIQUE constraints that need to be removed from the target table.)
If we only want to store one copy of the row from dvd, along with values from one matching row from each of the other tables, we could leave the UNIQUE constraint, and use an INSERT IGNORE statement to avoid throwing a "duplicate key error". Any rows where that error would have been thrown will be discarded, and won't be inserted into the target table.
Because the column references aren't qualified, we can't actually tell which table the dvdId column is beint returned from. We can't tell which table any of the columns are returned from. We can "guess" that genreId is being returned from the dvdGenre table, but for us to figure that out, we'd need to investigate the schema definition. It's not a problem for MySQL, it can lookup the table definitions a whole lot faster than we can.
We could aid to the future reader of that SQL statement by qualifying the column references with the tablename, or a table alias.
I want to copy all the rows from one table to another and change the ID if there is a duplicate.
I'm using phpmyadmin and tried the operations tab.
Copy table to (database.table):
Data Only
Add Auto Increment
This is the SQL it gives me:
INSERT INTO `wsuca2_dbwsuca2`.`cxtb4_menu` SELECT * FROM `wsuca2_dbwsuca2`.`j25_menu`
This is the error I'm getting:
#1062 - Duplicate entry '0-0-root-*' for key 'idx_client_id_parent_id_alias_language'
what you need is
INSERT ...... ON DUPLICATE KEY UPDATE
also, "change the ID if there is a duplicate" is not enough it seems. Your table have a complex unique key of client_id,parent_id,alias,language.
both your table had an entry of with the above field set to 0-0-root-* so it throws a error because MySQL doesn't know how to handle it.
either
update those entry manually before copying over
use INSERT ...... ON DUPLICATE KEY UPDATE to specify how to update those entries when found.
use INSERT IGNORE to ignore all duplicate entries (probably not what you want)
Having Duplicate key error is clear notification that you did not use the same structure and keys in the both tables. So first recreate the structure:
DROP TABLE `wsuca2_dbwsuca2`.`cxtb4_menu`;
SHOW CREATE TABLE `wsuca2_dbwsuca2`.`j25_menu`; //old table structure
and paste the structure of j25_menu as structure od cxtb4_menu (change the name of the table). After that insert the data with your INSERT clause.
I have a category say ecommerce.add ebay and amazon.when i update ebay as amazon,it should n't update.How do i d it?
I suggest you check out unique indexes and primary keys. These will cause an insert or update to fail rather that allow duplicate entries to be made.
CREATE UNIQUE INDEX name_unique ON tablename (name(10));
Replace name_unique with the name you want for the index, tablename with the name of your table, and name(10) with the column name and how many characters you want to be unique (the length of the column if you want the entire value to be unique).