SQL query to remove multiple duplicate entries from table - mysql

I have table containing following entries
Id | Accno | Name | Hash
----+----------+-----------+---------
1 | 11 | ABC | 01110
2 | 11 | ABC |
3 | 22 | PQT |
4 | 33 | XYZ | 03330
5 | 44 | LMN | 04440
6 | 33 | XYZ |
I need SQL query to remove duplicate entry from table and keep atleast single entry in table whose hash value is present. and for those entries which are not duplicate should also remain in table.

I think you guys overcomplicate things a lot. This should work just dandy:
DELETE FROM
YourTable
WHERE Hash IS NULL
AND Accno IN
(
SELECT Accno
FROM YourTable
GROUP BY Accno
HAVING COUNT(Name) > 1
)
;

Probably the easiest way to do it is to create a new table and copy non duplicate entries.
create table table_name2 as select distinct * from table_name1;
drop table table_name1;
rename table_name2 to table_name1;
Something like this.

Create table temp2 as SELECT *
FROM temp where id in (select id from temp group by accno having count(accno)>=1 and hash<>'');
drop table old_table;
rename table temp2 to old_table;
Check SQL Fiddle

Related

How to combine column from other table

I have created 2 temporary tables:
-------------- ------------------
| withlabfee | | without lab fee|
-------------- ------------------
| 6 | | 3 |
-------------- ------------------
My code returned the following result with single column.
-------------------------------
| withlabfee , without lab fee|
-------------------------------
| 6 , 3 |
------------------------------
My attempted code as below
drop table if exists withlabFee;
create temporary table withlabFee(labFee1 int);
insert into withlabFee
select count(*)
from course_relation
where lab_fee is not null;
drop table if exists withoutLabfee;
create temporary table withoutLabfee(labFee2 int);
insert into withoutLabfee
select count(*)
from course_relation
where lab_fee is null;
select concat(labfee1,',',labfee2) as `With lab fee , Without lab fee`
from withlabFee, withoutLabfee;
drop table labFee;
drop table withoutLabfee;
How do i combine the column from my temp table withlabfee and withoutlabfee into 1 table with 2 columns like below:
-------------------------------
| withlabfee | without lab fee|
-------------------------------
| 6 | 3 |
-------------------------------
Any helps are very much appreciated!
Solution even without temporary tables:
SELECT
SUM(lab_fee IS NOT NULL) AS 'With lab fee',
SUM(lab_fee IS NULL) AS 'Without lab fee'
FROM
course_relation
Try the below query (without creating temp tables).
SELECT
COUNT(CASE WHEN lab_fee IS NOT NULL THEN 1 END) AS withlabFee,
COUNT(CASE WHEN lab_fee IS NULL THEN 1 END) AS withoutLabfee
FROM course_relation

Deleting almost duplicate rows in MySQL?

I have seen a few different answers for this question, but none really hit exactly what I needed to do in MySQL.
I did find a thread for MS SQL that is exactly to what I need to do here but nothing min MySQL.
Data Example
+--------+----------+--------+
| Col1 | Col2 | UniqueID |
+--------+----------+--------+
| Peaches| Outdoor | 1 |
| Peaches| Outdoor | 2 |
| Apples | Indoor | 3 |
| Apples | Indoor | 4 |
+--------+----------+--------+
Desired Output
+--------+----------+--------+
| Col1 | Col2 | UniqueID |
+--------+----------+--------+
| Peaches| Outdoor | 1 |
| Apples | Indoor | 3 |
+--------+----------+--------+
Your way is OK. You only forgot the KEYWORD TABLE
CREATE TABLE NewTable AS SELECT Col1,Col2 ,MAX(col3) FROM t GROUP BY Col1,col2
but the structure can be different from the original table
Do this way:
CREATE TABLE NewTable like t;
then add a unique key:
ALTER TABLE NewTable ADD KEY (Col1,col2);
and now copy old data in new table with ON DUPLICATE KEY UPDATE
INSERT INTO NewTable
SELECT *
from t
ON DUPLICATE KEY UPDATE Col3=GREATEST(Col3,VALUES(Col3));
so you copy every row and the duplicates tests for maximum
Im going to post the answer to the answer provided above so its clear...it is just one simple query:
CREATE NewTable AS SELECT Col1,Col2 ,MAX(col3) FROM t GROUP BY Col1,col2
Just querying max was the trick...so simple.
Thank you!

Handle dynamic missed columns in MySQL

I have a small doubt in MySQL. While loading data from one table to another table I faced one issue
first table: emp
id | name | sal | deptno | loc | referby
1 | abc | 100 | 10 | hyd | xyz
2 | mnc | 200 | 20 |chen | pqr
second table:emprefers
id | name | sal | deptno | loc | referby
Now I want to load the emp table data into the emprefers table. I wrote a query like
insert into emprefers select * from emp after
I ran the query, the data was loaded into the emprefers table like below:
id | name | sal |deptno | loc |referby
1 | abc | 100 | 10 | hyd | xyz
2 | mnc | 200 | 20 | chen | pqr
Now I ran the same query a second time. It has failed. The reason is the name column is deleted from the emp table.
I edited the query like:
insert into emprefers select id,'null'as name,sal,deptno,loc,referby from emp
After I ran the edited query again, now records are loading into the emprefers table and the data looks like:
id | name | sal |deptno | loc |referby
1 | null | 100 |10 | hyd | xyz
2 | null | 200 |20 |chen | pqr
Every time before loading the emprefers table I truncate the emprefers table data. And the emprefers table structure never changed.
Again, a third time I ran the same query again. The query has failed, the reason was that the sal and deptno columns were missing in the emp table.
I don't want to edit the query again, reason is we don't know which columns are/get deleted from the emp table.
This time we want solve the issue.
We want to load the data into the second table if the columns are available in the emp table, then load the data - otherwise we need to pass null or empty values for those columns.
Please tell me how to write a query to check if a column exist or not, and if it exists to retrieve the same column, otherwise assign null values for that column.
Rather than changing the existing query and truncating the table, it might be a better idea to make delete the whole table, make a copy of the original emp table and then insert the data into it. That way they'll always be the same.
DROP TABLE emprefers IF EXISTS
CREATE TABLE emprefers LIKE emp;
INSERT INTO emprefers SELECT * FROM emp
This statement will create the table over the fly.
CREATE TABLE databasename.emprefers SELECT * FROM databasename.emp;

Remove duplicate rows on many to many table (Mysql)

I have a table which is many to many and my table looks like this
+----+--------+
| Customers |
+----+--------+
| id | name |
+----+--------+
| 1 | john |
| 1 | john |
| 1 | james |
| 2 | george |
| 2 | michael|
+----+--------+
What i want is to remove the duplicate rows with the same name.
Unfortunately, you have no way to distinguish one row from another. So, the easiest way to do this is the temporary table approach:
create table temp as
select distinct id, name
from customers;
truncate table customers;
insert into customers(id, name)
select id, name
from temp;
drop table temp;
Take a look at GROUP BY aggregate function
Well a small variation #Gordon Linoff answer, is by avoiding the "Insert into" and doing a "Rename table" and making the queries to work on any table.
Solution-1: By using a temp table
CREATE TABLE table_name_clean AS SELECT DISTINCT
*
FROM
table_name;
DROP TABLE table_name;
RENAME TABLE table_name_clean TO table_name;
Solution-2: Adding a UNIQUE INDEX (recommended as it will prevent the creation of duplicate entries in your table)
ALTER IGNORE TABLE table_name ADD UNIQUE INDEX u_id (id,name);

MySQL: Copy data from TableA:FieldA to TableB:FieldB matching common UID field

I didn't see this exact question asked. Most people seem to want to sync. I just need a one-time copy.
MySQL version is 5.5.35.
In my MySQL database I need to one-time copy data from TableA:FieldA to TableB:FieldB while matching a common UID field — TableA:UID and TableB:UID are the related fields.
More specifically I am copying Employee ID from one table to a different field in another table, and both tables have Contact ID in common. So obviously I need TableA:UID=1's Employee Number to appear in TableB on the correct row where TableB:UID=1.
Thanks.
UPDATED: Tested the solution, got an error 1442
UPDATE civicrm_value_member_fields_1
SET civicrm_value_member_fields_1.aft_id_43 =
(SELECT civicrm_contact.external_identifier FROM civicrm_contact
WHERE civicrm_contact.id = civicrm_value_member_fields_1.entity_id)
alt version of the above:
UPDATE `civicrm_value_member_fields_1`
SET `aft_id_43` =
(SELECT `external_identifier` FROM `civicrm_contact`
WHERE `id` = `entity_id`)
both error 1442:
#1442 - Can't update table 'civicrm_contact' in stored function/trigger because it is already used by statement which invoked this stored function/trigger.
The following example should help.
create table A(id int,cid int);
create table B(id int,cid int);
insert into A values(6,1);
insert into A values(7,2);
insert into A values(8,3);
insert into A values(9,4);
insert into B(cid) values(1);
insert into B(cid) values(3);
insert into B(cid) values(4);
Table A
| ID | CID |
|----|-----|
| 6 | 1 |
| 7 | 2 |
| 8 | 3 |
| 9 | 4 |
Table B
| ID | CID |
|--------|-----|
| (null) | 1 |
| (null) | 3 |
| (null) | 4 |
The UPDATE query will update B's id field referring A's id field.
update B set B.id = (select A.id from A where A.cid = B.cid);
Table B
| ID | CID |
|----|-----|
| 6 | 1 |
| 8 | 3 |
| 9 | 4 |
The error can be sidestepped by first creating a temporary table - it will have all the data you need but not have the functions/triggers that impede the step.
Also, once the table is set then a JOIN statement is a more efficient representation of the action and should run more quickly as well.
CREATE TEMPORARY TABLE contact_dupe SELECT id, external_identifier FROM civicrm_contact;
UPDATE civicrm_value_member_fields_1 AS cs
JOIN contact_dupe AS cd ON cd.id=cs.entity_id
SET cs.aft_id_43=cc.external_identifier;