I have db with multiple entries.
I Google out something like this
SELECT COUNT(*), item_id, text, number FROM ads
GROUP BY item_id, text, number
HAVING COUNT(*)>1;
this select (I think) all my multiple entries, I use SQLyog, ... and there is no option to press button and delete all results of this query.
but even if I select all one by one and delete, I would also delete original one, right?
I basically want to keep all unique entries and keep one of multiple items.
simple example
('1' 'Novi Sad' '123');
('1' 'Novi Sad' '123');
('3' 'Beograd' '124');
I want to keep
('1' 'Novi Sad' '123');
('3' 'Beograd' '124');
I know only basic mysql.
When you do delete entries make sure to reset your ID increment
ALTER TABLE 'table_name' AUTO_INCREMENT = 1
Can you just copy, drop and delete?
CREATE TABLE Copy_Temp as
SELECT item_id, text, number
FROM ads
GROUP BY item_id, text, number;
DROP Table ads;
RENAME TABLE Copy_Temp TO ads;
Select all unique records into a temp table.
Delete all records from original table.
Insert all records from your temp table into original table.
DELETE emp FROM employee emp, employee emp2
WHERE emp.id > emp2.id
AND emp.name = emp2.name
For example, you having the table employee in which there are duplicate records (having the same name multiple times) then this query will delete all the duplicate records.
Related
I need to validate the last inserted row in a table every time using some constraints given by the user on a particular column in that table( Ex: age > 50...etc.).
I thought of using a temporary table to insert the row in both of my temporary table and my normal table. After then, I'll use query like Select * from tb_name USER_CONSTRAINTS [ Ex: select * from student where age>50 ] on temporary table. If the result is not null, then the last row satisfies the user constraint else it fails. After I'll delete the last added row from the temporary table and repeat the process for next row.
I don't know if this way is good or bad or is there some other efficient way to do this?
Edit:
The table has 5 columns
stud_id,
subject_id,
age,
dob,
marks
Here stud_id and subject_id act as foreign keys to two tables tbl_student and tbl_subject
I'm trying to update all the prices in one database with the prices from another where the product code matches (which isn't the primary key), whilst leaving the other fields columns untouched.
INSERT INTO inventory
SELECT * FROM temporary_table
ON DUPLICATE KEY UPDATE price = VALUES(price)
This is just duplicating the whole product where the code matches
Thanks in advance
Try using replace instead of insert. select all the columns from inventory table except the one you want to update(price)
REPLACE INTO inventory
SELECT b.col1, b.col2......... a.price
FROM temporary_table a, inventory b
where a.product_code = b.product_code
I've seen a number of variations on this but nothing quite matches what I'm trying to accomplish.
I have a table, TableA, which contain the answers given by users to configurable questionnaires. The columns are member_id, quiz_num, question_num, answer_num.
Somehow a few members got their answers submitted twice. So I need to remove the duplicated records, but make sure that one row is left behind.
There is no primary column so there could be two or three rows all with the exact same data.
Is there a query to remove all the duplicates?
Add Unique Index on your table:
ALTER IGNORE TABLE `TableA`
ADD UNIQUE INDEX (`member_id`, `quiz_num`, `question_num`, `answer_num`);
Another way to do this would be:
Add primary key in your table then you can easily remove duplicates from your table using the following query:
DELETE FROM member
WHERE id IN (SELECT *
FROM (SELECT id FROM member
GROUP BY member_id, quiz_num, question_num, answer_num HAVING (COUNT(*) > 1)
) AS A
);
Instead of drop table TableA, you could delete all registers (delete from TableA;) and then populate original table with registers coming from TableA_Verify (insert into TAbleA select * from TAbleA_Verify). In this way you won't lost all references to original table (indexes,... )
CREATE TABLE TableA_Verify AS SELECT DISTINCT * FROM TableA;
DELETE FROM TableA;
INSERT INTO TableA SELECT * FROM TAbleA_Verify;
DROP TABLE TableA_Verify;
This doesn't use TEMP Tables, but real tables instead. If the problem is just about temp tables and not about table creation or dropping tables, this will work:
SELECT DISTINCT * INTO TableA_Verify FROM TableA;
DROP TABLE TableA;
RENAME TABLE TableA_Verify TO TableA;
Thanks to jveirasv for the answer above.
If you need to remove duplicates of a specific sets of column, you can use this (if you have a timestamp in the table that vary for example)
CREATE TABLE TableA_Verify AS SELECT * FROM TableA WHERE 1 GROUP BY [COLUMN TO remove duplicates BY];
DELETE FROM TableA;
INSERT INTO TableA SELECT * FROM TAbleA_Verify;
DROP TABLE TableA_Verify;
Add Unique Index on your table:
ALTER IGNORE TABLE TableA
ADD UNIQUE INDEX (member_id, quiz_num, question_num, answer_num);
is work very well
If you are not using any primary key, then execute following queries at one single stroke. By replacing values:
# table_name - Your Table Name
# column_name_of_duplicates - Name of column where duplicate entries are found
create table table_name_temp like table_name;
insert into table_name_temp select distinct(column_name_of_duplicates),value,type from table_name group by column_name_of_duplicates;
delete from table_name;
insert into table_name select * from table_name_temp;
drop table table_name_temp
create temporary table and store distinct(non duplicate) values
make empty original table
insert values to original table from temp table
delete temp table
It is always advisable to take backup of database before you play with it.
As noted in the comments, the query in Saharsh Shah's answer must be run multiple times if items are duplicated more than once.
Here's a solution that doesn't delete any data, and keeps the data in the original table the entire time, allowing for duplicates to be deleted while keeping the table 'live':
alter table tableA add column duplicate tinyint(1) not null default '0';
update tableA set
duplicate=if(#member_id=member_id
and #quiz_num=quiz_num
and #question_num=question_num
and #answer_num=answer_num,1,0),
member_id=(#member_id:=member_id),
quiz_num=(#quiz_num:=quiz_num),
question_num=(#question_num:=question_num),
answer_num=(#answer_num:=answer_num)
order by member_id, quiz_num, question_num, answer_num;
delete from tableA where duplicate=1;
alter table tableA drop column duplicate;
This basically checks to see if the current row is the same as the last row, and if it is, marks it as duplicate (the order statement ensures that duplicates will show up next to each other). Then you delete the duplicate records. I remove the duplicate column at the end to bring it back to its original state.
It looks like alter table ignore also might go away soon: http://dev.mysql.com/worklog/task/?id=7395
An alternative way would be to create a new temporary table with same structure.
CREATE TABLE temp_table AS SELECT * FROM original_table LIMIT 0
Then create the primary key in the table.
ALTER TABLE temp_table ADD PRIMARY KEY (primary-key-field)
Finally copy all records from the original table while ignoring the duplicate records.
INSERT IGNORE INTO temp_table AS SELECT * FROM original_table
Now you can delete the original table and rename the new table.
DROP TABLE original_table
RENAME TABLE temp_table TO original_table
Tested in mysql 5.Dont know about other versions.
If you want to keep the row with the lowest id value:
DELETE n1 FROM 'yourTableName' n1, 'yourTableName' n2 WHERE n1.id > n2.id AND n1.member_id = n2.member_id and n1.answer_num =n2.answer_num
If you want to keep the row with the highest id value:
DELETE n1 FROM 'yourTableName' n1, 'yourTableName' n2 WHERE n1.id < n2.id AND n1.member_id = n2.member_id and n1.answer_num =n2.answer_num
I entered a query that introduced some duplicates into my database. The table is straight forward.
It has an id (int) column and a phrase column which is varchar(255). In order to find duplicates, my query looks like the following:
SELECT phrase from foo GROUP BY phrase HAVING (count(phrase) > 1)
My question is, how do I delete the duplicate entries without manually having to do it? I want to use the query above to generate the list of entries that need to be deleted at least once. This way only one version of 'phrase' exists in table foo.
This would keep one row (the one with the lowest ID) per phrase.
DELETE FROM foo
WHERE id NOT IN (
SELECT id FROM (
SELECT MIN(id) id
FROM foo
GROUP BY phrase
) _
);
As dan pointed out in comments, with MySQL you need that weird inner query.
You should use:
SELECT max(id) from foo GROUP BY phrase HAVING (count(phrase) > 1)
To establish what the ids that need to be deleted.
To delete the entries you can do something like:
delete from foo where id in (select id from (SELECT max(id) from foo GROUP BY phrase HAVING (count(phrase) > 1)) foo);
You will be able to execute the delete statement multiple times, to delete duplicates that are more then one.
You need to create a temporary table, add unique values, add just one of the duplicated values, and then rename your temporary table to your original one.
create table tmp like foo;
alter table tmp add unique (phrase);
insert into tmp select * from foo
on duplicate key update phrase=ifnull(phrase, values(phrase));
rename table foo to deleteme, tmp to foo;
drop table deleteme;
You can do a JOIN and decide if you want to delete the first (min) or last (max) duplicate.
DELETE phrase FROM phrase JOIN
(SELECT max(id),COUNT(id) cnt from foo GROUP BY phrase
HAVING cnt>1) AS dups
ON phrase.id=dups.id
You need to run it multiple times if you have more that more than 1 duplicate of each record.
I have a table in my database which has duplicate records that I want to delete. I don't want to create a new table with distinct entries for this. What I want is to delete duplicate entries from the existing table without the creation of any new table. Is there any way to do this?
id action
L1_name L1_data
L2_name L2_data
L3_name L3_data
L4_name L4_data
L5_name L5_data
L6_name L6_data
L7_name L7_data
L8_name L8_data
L9_name L9_data
L10_name L10_data
L11_name L11_data
L12_name L12_data
L13_name L13_data
L14_name L14_data
L15_name L15_data
see these all are my fields :
id is unique for every row.
L11_data is unique for respective action field.
L11_data is having company names while action is having name of the industries.
So in my data I'm having duplicate name of the companies in L11_data for their respective industries.
What I want is to have is unique name and other data of the companies in the particular industry stored in action. I hope I have stated my problem in a way that you people can understand it.
Yes, assuming you have a unique ID field, you can delete all records that are the same except for the ID, but don't have "the minimum ID" for their group of values.
Example query:
DELETE FROM Table
WHERE ID NOT IN
(
SELECT MIN(ID)
FROM Table
GROUP BY Field1, Field2, Field3, ...
)
Notes:
I freely chose "Table" and "ID" as representative names
The list of fields ("Field1, Field2, ...") should include all fields except for the ID
This may be a slow query depending on the number of fields and rows, however I expect it would be okay compared to alternatives
EDIT: In case you don't have a unique index, my recommendation is to simply add an auto-incremental unique index. Mainly because it's good design, but also because it will allow you to run the query above.
ALTER IGNORE TABLE 'table' ADD UNIQUE INDEX(your cols);
Duplicates get NULL, then you can delete them
DELETE
FROM table_x a
WHERE rowid < ANY (
SELECT rowid
FROM table_x b
WHERE a.someField = b.someField
AND a.someOtherField = b.someOtherField
)
WHERE (
a.someField,
a.someOtherField
) IN (
SELECT c.someField,
c.someOtherField
FROM table_x c
GROUP BY c.someField,
c.someOtherField
HAVING count(*) > 1
)
In above query the combination of someField and someOtherField must identify the duplicates distinctively.