Deleting almost duplicate rows in MySQL? - mysql

I have seen a few different answers for this question, but none really hit exactly what I needed to do in MySQL.
I did find a thread for MS SQL that is exactly to what I need to do here but nothing min MySQL.
Data Example
+--------+----------+--------+
| Col1 | Col2 | UniqueID |
+--------+----------+--------+
| Peaches| Outdoor | 1 |
| Peaches| Outdoor | 2 |
| Apples | Indoor | 3 |
| Apples | Indoor | 4 |
+--------+----------+--------+
Desired Output
+--------+----------+--------+
| Col1 | Col2 | UniqueID |
+--------+----------+--------+
| Peaches| Outdoor | 1 |
| Apples | Indoor | 3 |
+--------+----------+--------+

Your way is OK. You only forgot the KEYWORD TABLE
CREATE TABLE NewTable AS SELECT Col1,Col2 ,MAX(col3) FROM t GROUP BY Col1,col2
but the structure can be different from the original table
Do this way:
CREATE TABLE NewTable like t;
then add a unique key:
ALTER TABLE NewTable ADD KEY (Col1,col2);
and now copy old data in new table with ON DUPLICATE KEY UPDATE
INSERT INTO NewTable
SELECT *
from t
ON DUPLICATE KEY UPDATE Col3=GREATEST(Col3,VALUES(Col3));
so you copy every row and the duplicates tests for maximum

Im going to post the answer to the answer provided above so its clear...it is just one simple query:
CREATE NewTable AS SELECT Col1,Col2 ,MAX(col3) FROM t GROUP BY Col1,col2
Just querying max was the trick...so simple.
Thank you!

Related

UPDATE/INSERT INTO/DELETE FROM table in MYSQL

I have a table in MYSQL database TABLE1, with columns COLA, COLB, COLC and (COLA,COLB) are composite primary keys. Something like this
-----------------------
| COLA | COLB | COLC |
-----------------------
| A | B | C |
-----------------------
| A | Q | D |
-----------------------
| A | E | J |
-----------------------
| B | W | P |
-----------------------
Also there is background script which passes data to a java program which should update the table under following conditions :
If new values have any new pair for PRIMARY KEYs then INSERT new row into the table.
If new values have any common pair for PRIMARY KEYs then UPDATE the table.
DELETE all other rows Where COLA value matches with new values.
If new vaues are ('A','B','L'),('A','Y','D'),('A','Q','Z') then it should :
UPDATE 1st and 2nd row.
INSERT a new row ('A','Y','D').
DELETE only 3rd row.
So table should look like
-----------------------
| COLA | COLB | COLC |
-----------------------
| A | B | L |
-----------------------
| A | Q | Z |
-----------------------
| B | W | P |
-----------------------
| A | Y | D |
-----------------------
To implement this I was running two queries :
INSERT INTO TABLE1 VALUES('A','B','L'),('A','Y','D'),('A','Q','Z') ON DUPLICATE KEY UPDATE COLC=VALUES(COLC);
Which is working the way I want. But when I try to delete other rows I am getting into problem what I am trying is :
DELETE FROM TABLE1 WHERE NOT ((COLA='A' AND COLB='B') OR (COLA='A' AND COLB='Y') OR (COLA='A' AND COLB='Q'));
But it does not work. As it deletes the last row as well.
So
How to implement the query?
Can it be clubbed into one query?
THANKS IN ADVANCE :)
I also couldn't find one query solution to this issue but for the second query a bit optimised version can be:
DELETE FROM TABLE1 WHERE COLA='A' AND COLB NOT IN ('B','Y','Q');
or
DELETE FROM TABLE1 WHERE COLA='A' AND COLC NOT IN ('L','Z','D');
Any of the above can be used and it can be a bit scalable than the one you provided.
I got the answer to first question. query should be
DELETE FROM TABLE1 WHERE COLA='A' AND NOT ((COLA='A' AND COLB='B') OR (COLA='A' AND COLB='Y') OR (COLA='A' AND COLB='Q'));

Select multiple distinct columns from same table separately

Let's have this sample data:
+-------+---------+
| col1 | col2 |
+-------+---------+
| 111 | hello |
| 222 | hello |
| 111 | blabla |
| 444 | there |
| 555 | blabla |
| 555 | there |
+-------+---------+
I need a SQL returning distinct values for each columns separately (as feed for dropdown values for filtering).
Thus the result should be:
+-------+---------+
| col1 | col2 |
+-------+---------+
| 111 | hello |
| 222 | blabla |
| 444 | there |
| 555 | |
+-------+---------+
The results need not be in this format; it's more important that I have the distinct values for easy access and iteration.
The closest I got is from here:
https://stackoverflow.com/a/12188117/169252
select (SELECT group_concat(DISTINCT col1) FROM testtable) as col1, (SELECT group_concat(DISTINCT col2) FROM testtable) as col2;
This returns:
+-----------------+-------------------+
| col1 | col2 |
+-----------------+-------------------+
| 111,222,444,555 | velo,hallo,blabla |
+-----------------+-------------------+
That's pretty close, and I'll choose this one if no better solution comes up; it's not optimal as values are comma separated and I need to split the results.
I also tried:
SELECT DISTINCT from col1,col2 FROM testtable;
This returns the distint values of BOTH columns, not what I want.
Also:
select col1,col2 from testtable group by col1,col2;
which has been suggested elsewhere doesn't return what I need, but returns each column in-distinct :)
One problem with what you're asking is that you're expected resultset doesn't really make any sense from a relational database standpoint. Every column in a row of data should have a relationship with the other columns in the row.
The best way to approach this, IMO, is to return two result sets and process each one for each of your drop down boxes:
SELECT DISTINCT column_1 FROM My_Table
and
SELECT DISTINCT column_2 FROM My_Table
I'd also look into why you have that data in the same table to begin with if the two columns are not related. If they are related and you're trying to have a drop down for one column that then filters the items in the second drop down list then you really should return the full set of rows and let your front end application handle the filtering (and displaying unique results). Most drop down widgets should allow this kind of linking.
Try
Select (SELECT DISTINCT col1 from testtable) AS col1, (SELECT DISTINCT col2 from testtable) AS col2;
I think the group_concat is putting the commas into the returned table.

MySQL recursive update based on values in the same table

I am having trouble implementing the following structure in MySQL.
Table1:
ID | Val
1 | 10
2 | 20
Table2:
ID | LeftTableType | LeftID | LeftVal | RightTableType | RightID | RightVal | Operation | Result
1 | Table1 | 1 | (10) | Table1 | 2 | (20) | + | (30)
2 | Table2 | 1 | (30) | Table1 | 2 | (20) | + | (50)
I tried to use a trigger system where an update to Table1 would update the values of Table2. Unfortunately, I needed to then update subsequent values of Table2, which caused a recursive trigger system that MySQL did not like.
I have also been looking into nested sets and tree structures. It seems like they might be what I am looking for, or at least very close.
Is there something obvious that I am missing to implement something like this? This seems like it might lead me to a messy mixture of cursors, recursion, triggers, procedures, and tree structures.
Any hints would be greatly appreciated!

SQL query to remove multiple duplicate entries from table

I have table containing following entries
Id | Accno | Name | Hash
----+----------+-----------+---------
1 | 11 | ABC | 01110
2 | 11 | ABC |
3 | 22 | PQT |
4 | 33 | XYZ | 03330
5 | 44 | LMN | 04440
6 | 33 | XYZ |
I need SQL query to remove duplicate entry from table and keep atleast single entry in table whose hash value is present. and for those entries which are not duplicate should also remain in table.
I think you guys overcomplicate things a lot. This should work just dandy:
DELETE FROM
YourTable
WHERE Hash IS NULL
AND Accno IN
(
SELECT Accno
FROM YourTable
GROUP BY Accno
HAVING COUNT(Name) > 1
)
;
Probably the easiest way to do it is to create a new table and copy non duplicate entries.
create table table_name2 as select distinct * from table_name1;
drop table table_name1;
rename table_name2 to table_name1;
Something like this.
Create table temp2 as SELECT *
FROM temp where id in (select id from temp group by accno having count(accno)>=1 and hash<>'');
drop table old_table;
rename table temp2 to old_table;
Check SQL Fiddle

INSERT / UPDATE SQL random & unique VARCHAR

I need to be able to INSERT/UPDATE UNIQUE RANDOM UTF8 ALPHANUMERICAL VARCHAR 55 into a table field called 'key'.
Can't find out any good query example, does anyone can show me or link me something?
This answer is based on mysql.
This select will create 55 char long random strings:
select substr(concat(md5(rand()),md5(rand())),1,55);
to fill your table column you might want to try out:
create table example (keycol varchar(55));
insert into example (keycol) values (substr(concat(md5(rand()),md5(rand())),1,55));
The result will be:
select keycol from example;
+---------------------------------------------------------+
| keycol |
+---------------------------------------------------------+
| 4517f4be669301a4a529b53fc18d646dec42d4d07d911d33a67c863 |
| 3caa1c98f0f9ee39515aa6f4ddb3f84fa41abd5392f610c5d24bcd9 |
| 8e52cb4ce29e58514671c9b68f19832f26ddf53f277621ac420bd2e |
| 3adcccfb6cb729ce1c0a14fb75f6fd54f58992dc0751527c969e007 |
| c28c5879589dc90f4fb0963673e5668fa5789d325423ba043e0243b |
| 8f7a2af97d73261008f0d0d7249480fde56a3a91f2ce6e8bf0b0070 |
| ff4f74f25b92da3eaab282218c23a75d4cfa77c8f8bfdf74d7ebdf9 |
+---------------------------------------------------------+