So I have a database with more than 2000 line and I wanted to delete the duplicate value based on 2 rows.For Example :
no server name
1 serv1 a
2 serv1 b
3 serv1 b
4 serv1 b
5 serv2 a
6 serv2 b
7 serv2 c
8 serv2 c
So basically I wanted to remove the duplicate IF two of the rows have a duplicateBUT I dont want to remove them if just one of the row has duplicate.
Expected Output:
no server name
1 serv1 a
2 serv1 b
3 serv2 a
4 serv2 b
5 serv2 c
Any answer would be appreciated.
There is many ways to do what you want.
If you 're looking for just SELECTing the data without duplicates then you could use:
DISTINCT
SELECT DISTINCT Server,
Name
FROM YourTableName
GROUP BY
SELECT Server,
Name
FROM YourTableName
GROUP BY Server, Name
Window function (ROW_NUMBER) in a subquery
SELECT Server,
Name
FROM
(
SELECT *,
ROW_NUMBER() OVER(PARTITION BY Server, Name ORDER BY Server) RN
FROM YourTableName
) TT
WHERE TT.RN = 1
You delete the duplicates as
DELETE T1
FROM
(
SELECT ROW_NUMBER() OVER(PARTITION BY Server, Name ORDER BY Server) RN
FROM T --YourTableName
) T1 JOIN
(
SELECT ROW_NUMBER() OVER(PARTITION BY Server, Name ORDER BY Server) RN
FROM T --YourTableName
) T2 ON T1.RN = T2.RN
WHERE T1.RN > 1;
Use select distinct:
select distinct server, name
from t;
If you have a lot of duplicates, the simplest way is probably to recreate the table:
create table temp_t as
select min(no) as no, server, name
from t
group by server, name;
truncate table t;
insert into t (no, server, name)
select no, server, name
from temp_t;
If you don't have many rows, then delete is fine:
delete t
from t join
(select server, name, min(no) as min_no
from t
group by server, name
) tt
on t.server = tt.server and t.name = tt.name
where t.no > tt.min_no;
You can use the following query to remove duplicates
DELETE t1 FROM tablename t1 INNER JOIN tablename t2
WHERE t1.id < t2.id AND t1.server = t2.server AND t1.name = t2.name;
Related
Please consider the table below
Id F1 F2
---------------------------
1 Nima a
2 Eli a
3 Arian a
4 Ava b
5 Arsha b
6 Rozhan c
7 Zhina c
I want to display records by sorting COLUMN F2 to display one record from each string category (a,b,c) in order
Id F1 F2
---------------------------
1 Nima a
5 Arsha b
6 Rozhan c
2 Eli a
4 Ava b
7 Zhina c
3 Arian a
NOTE: a,b,c could be anything... it should take one record from one entry and then 2nd from 2nd entry.
I have used join, or group by records but no success.
MySQL version 5.7 – Syed Saqlain
SELECT id, f1, f2
FROM ( SELECT t1.id, t1.f1, t1.f2, COUNT(*) cnt
FROM test t1
JOIN test t2 ON t1.f2 = t2.f2 AND t1.id >= t2.id
GROUP BY t1.id, t1.f1, t1.f2 ) t3
ORDER BY cnt, f2;
https://dbfiddle.uk/?rdbms=mysql_5.7&fiddle=8138bd9ab5be36ba534a258d20b2e555
ROW_NUMBER() alternative for lower version of MYSQL. This query will work for version 5.5, 5.6 & 5.7.
-- MySQL (v5.7)
SELECT t.id, t.f1, t.f2
FROM (SELECT #row_no:=CASE WHEN #db_names=d.f2 THEN #row_no+1 ELSE 1 END AS row_number
, #db_names:= d.f2
, d.f2
, d.f1
, d.id
FROM test d,
(SELECT #row_no := 0,#db_names:='') x
ORDER BY d.f2) t
ORDER BY t.row_number, t.f2
Please check from url https://dbfiddle.uk/?rdbms=mysql_5.7&fiddle=02dbb0086a6dd7c926d55a690bffbd06
You can use window functions in the order by:
select t.*
from t
order by row_number() over (partition by f2 order by id),
f2;
The row_number() function (as used above) assigns a sequential number starting with 1 to each value of f2.
In older versions of MySQL, you can use a correlated subquery instead:
order by (select count(*) from t t2 where t2.f2 = t.f2 and t2.id <= t.id),
f2;
For performance, you want an index on (f2, id).
This question is an extension of mysql compress order_by values.
My table has groups of ordered numbers, with undesired gaps. How can I renumber each of these groups, while keeping the original order?
Group Order Desired Order
A 1 1
A 3 2
A 6 3
A 7 4
B 2 1
B 3 2
B 8 3
C 1 1
C 7 2
C 8 3
You can also get the desired order by using a correlated sub query without using and variables
select t1.*,
(
select count(*)
from demo t2
where t1.`group` = t2.`group`
and t1.`order` > t2.`order`
) + 1 desiredorder
from demo t1
DEMO
Or to update same table with your desired order you can use below query
update demo a
join (select t1.*,(
select count(*)
from demo t2
where t1.`group` = t2.`group`
and t1.`order` > t2.`order`
) + 1 desiredorder
from demo t1
) b on a.`group` = b.`group`
and a.`order` = b.`order`
set a.`order` = b.desiredorder
DEMO
Note make sure to add an index on group and order column for better performance.
You can create a variable to store the previous value of group and another one for the desiredorder. Essentially this is a mimic of a Window Function ROW_NUMBER which MySql doesn't support.
I renamed the columns as group=col1, order=col2
select col1,
col2,
neworder
from (
select t.col1,
t.col2,
case when #prev=t.col1 then #id:=#id+1 else #id:=1 end as neworder,
#prev:=t.col1
from (select #prev:=null, #id:=0) a,
toorder t
order by t.col1
) a
See it working here: http://sqlfiddle.com/#!9/9d8528/2
The external query is only needed if you want to pull out only those three columns.
I'm trying to create a search function in different tables using UNION and what happened is that the id's are duplicating making the search go wrong. How can I merge different tables into one while no id's are in common?
Here is the example
table1
id name desc
1 henry post
2 albert doth
3 jun cloth
table2
id name desc
1 kin revenge
2 pot eve
The result SHOULD be like this
id name desc
1 henry post
2 albert doth
3 jun cloth
4 kin revenge
5 pot eve
Please help me. Thanks.
In most databases, you would add a new id using the ANSI standard row_number() function:
select row_number() over (order by which, id) as newid, name, description
from (select 1 as which, t1.* from table1 t1 union all
select 2 as which, t2.* from table2 t2
) t;
Note that desc is a really bad name for a column, because it is a SQL keyword and usually a reserved word.
EDIT:
MySQL doesn't support this ANSI standard functionality. Instead, use variables:
select (#rn := #rn + 1) as newid, name, description
from (select 1 as which, t1.* from table1 t1 union all
select 2 as which, t2.* from table2 t2
) t cross join
(select #rn := 0) vars
order by which, id;
I've include the order by so the rows remain in the same order that you seem to want them in -- rows from the first table followed by rows from the second table. If you don't care about the order, just drop the order by.
For SQLite, the calculation is much more painful:
with cte as (
select 1 as which, t1.* from table1 t1 union all
select 2 as which, t2.* from table2 t2
)
select (select count(*)
from cte cte2
where cte2.which < cte.which or (ct2.which = cte.which and cte2.id <= cte.id
) as id,
name, description
from cte;
In MySql, you can simulate the row_number() function of Sql Server and Oracle using a mutating variable hack:
set #rownum := 0;
SELECT #rownum:=#rownum+1 AS` row_number`, `name`, `desc`
FROM
(
SELECT `name`, `desc` FROM table1
UNION
SELECT `name`, `desc` FROM table2
) AS x;
SqlFiddle
It looks like you have to Generate Id's so you can make you Union query as Sub select and generate Id's in Outer Query
MySQL does not have any system function like SQL Server’s row_number () to generate the row number for each row. However, it can be generated using the variable in the SELECT statement
SET #row_number:=0;
SELECT #row_number:=#row_number+1 As Id,
NAME,
desc
FROM (SELECT NAME,desc
FROM table1
UNION ALL
SELECT NAME,desc
FROM table2
UNION ALL
........
........) A
Order by NAME -- Change the column in Order by in which order you want to create New ID's
I'm trying to figure out best way to take required rows from database.
Database table:
id user cat time
1 5 1 123
2 5 1 150
3 5 2 160
4 5 3 100
I want to take DISTINCT cat ... WHERE user=5 with MAX time value. How should I do that in efficient way?
You will want to use an aggregate function with a GROUP BY:
select user, cat, max(time) as Time
from yourtable
group by user, cat
See SQL Fiddle with Demo
If you want to include the id column, then you can use a subquery:
select t1.id,
t1.user,
t1.cat,
t1.time
from yourtable t1
inner join
(
select max(time) Time, user, cat
from yourtable
group by user, cat
) t2
on t1.time = t2.time
and t1.user = t2.user
and t1.cat = t2.cat
See SQL Fiddle with Demo. I used a subquery to be sure that the id value that is returned with each max(time) row is the correct id.
I am trying to clean up records stored in a MySQL table. If a row contains %X%, I need to delete that row and the row immediately below it, regardless of content. E.g. (sorry if the table is insulting anyone's intelligence):
| 1 | leave alone
| 2 | Contains %X% - Delete
| 3 | This row should also be deleted
| 4 | leave alone
| 5 | Contains %X% - Delete
| 6 | This row should also be deleted
| 7 | leave alone
Is there a way to do this using only a couple of queries? Or am I going to have to execute a SELECT query first (using the %x% search parameter) then loop through those results and execute a DELETE...WHERE for each index returned + 1
This should work although its a bit clunky (might want to check the LIKE argument as it uses pattern matching (see comments)
DELETE FROM table.db
WHERE idcol IN
( SELECT idcol FROM db.table WHERE col LIKE '%X%')
OR idcolIN
( SELECTidcol+1 FROMdb.tableWHEREcol` LIKE '%X%')
Let's assume the table was named test and contained to columns named id and data.
We start with a SELECT that gives us the id of all rows that have a preceding row (highest id of all ids lower than id of our current row):
SELECT t1.id FROM test t1
JOIN test t2 ON
( t2.id, true )
=
( SELECT t3.id, t3.data LIKE '%X%' FROM test t3
WHERE t3.id < t1.id ORDER BY id DESC LIMIT 1 )
That gives us the ids 3 and 6. Their preceding rows 2 and 5 contain %X%, so that's good.
Now lets get the ids of the rows that contain %X% and combine them with the previous ones, via UNION:
(SELECT t1.id FROM test t1
JOIN test t2 ON
( t2.id, true )
=
( SELECT t3.id, t3.data LIKE '%X%' FROM test t3
WHERE t3.id < t1.id ORDER BY id DESC LIMIT 1 )
)
UNION
(
SELECT id FROM test WHERE data LIKE '%X%'
)
That gives us 3, 6, 2, 5 - nice!
Now, we can't delete from a table and select from the same table in MySQL - so lets use a temporary table, store our ids that are to be deleted in there, and then read from that temporary table to delete from our original table:
CREATE TEMPORARY TABLE deleteids (id INT);
INSERT INTO deleteids
(SELECT t1.id FROM test t1
JOIN test t2 ON
( t2.id, true )
=
( SELECT t3.id, t3.data LIKE '%X%' FROM test t3
WHERE t3.id < t1.id ORDER BY id DESC LIMIT 1 )
)
UNION
(
SELECT id FROM test WHERE data LIKE '%X%'
);
DELETE FROM test WHERE id in (SELECT * FROM deleteids);
... and we are left with the ids 1, 4 and 7 in our test table!
(And since the previous rows are selected using <, ORDER BY and LIMIT, this also works if the ids are not continuous.)
You can do it all in a single DELETE statement:
Assuming the "row immediately after" is based on the order of your INT-based ID column, you can use MySQL variables to assign row numbers which accounts for gaps in your IDs:
DELETE a FROM tbl a
JOIN (
SELECT a.id, b.id AS nextid
FROM (
SELECT a.id, a.text, #rn:=#rn+1 AS rownum
FROM tbl a
CROSS JOIN (SELECT #rn:=1) rn_init
ORDER BY a.id
) a
LEFT JOIN (
SELECT a.id, #rn2:=#rn2+1 AS rownum
FROM tbl a
CROSS JOIN (SELECT #rn2:=0) rn_init
ORDER BY a.id
) b ON a.rownum = b.rownum
WHERE a.text LIKE '%X%'
) b ON a.id IN (b.id, b.nextid)
SQL Fiddle Demo (added additional data for example)
What this does is it first takes your data and ranks it based on your ID column, then we do an offset LEFT JOIN on an almost identical result set except that the rank column is behind by 1. This gets the rows and their immediate "next" rows side by side so that we can pull both of their id's at the same time in the parent DELETE statement:
SELECT a.id, a.text, b.id AS nextid, b.text AS nexttext
FROM (
SELECT a.id, a.text, #rn:=#rn+1 AS rownum
FROM tbl a
CROSS JOIN (SELECT #rn:=1) rn_init
ORDER BY a.id
) a
LEFT JOIN (
SELECT a.id, a.text, #rn2:=#rn2+1 AS rownum
FROM tbl a
CROSS JOIN (SELECT #rn2:=0) rn_init
ORDER BY a.id
) b ON a.rownum = b.rownum
WHERE a.text LIKE '%X%'
Yields:
ID | TEXT | NEXTID | NEXTTEXT
2 | Contains %X% - Delete | 3 | This row should also be deleted
5 | Contains %X% - Delete | 6 | This row should also be deleted
257 | Contains %X% - Delete | 3434 | This row should also be deleted
4000 | Contains %X% - Delete | 4005 | Contains %X% - Delete
4005 | Contains %X% - Delete | 6000 | Contains %X% - Delete
6000 | Contains %X% - Delete | 6534 | This row should also be deleted
We then JOIN-DELETE that entire statement on the condition that it deletes rows whose IDs are either the "subselected" ID or NEXTID.
There is no reasonable way of doing this in a single query. (It may be possible, but the query you end up having to use will be unreasonably complex, and will almost certainly not be portable to other SQL engines.)
Use the SELECT-then-DELETE approach you described in your question.