i have table like :
id IP Subnet Duplicates Valid
1 foo 16 1
2 bar 24 1
3 foo 28 1
4 foo 32 1
i want update description with id of duplicated row . something like:
id IP Subnet Duplicates Valid
1 foo 16 3,4 0
2 bar 24 1
3 foo 28 1,4 0
4 foo 32 1,3 0
Here is my query :
update tblSample inner join (
select
t1.Id,
group_concat(t2.Id) dups
from
tblSample t1 inner join tblSample t2
on t1.Id<>t2.Id ) AND
((t1.IP >> (32-LEAST(t1.Subnet,t2.Subnet))
<< (32-LEAST(t1.Subnet,t2.Subnet))
=
((t2.IP >> (32-LEAST(t1.Subnet,t2.Subnet))
<< 32-LEAST(t1.Subnet,t2.Subnet)))
group by
t1.Id
) s on tblSample.Id = s.Id
set
Valid=0 ,Duplicates=dups
my code works but its very slow ( about 53 second for 10000 record )
How can i improve speed ?
is there any way that i can decrease comparison operation.
Here is a solution without self join in your sub query, maybe will not improve performance greatly, but try it, and also, try to explain it and yours.
update tblSample t1
join (
select name, group_concat(id order by id) as description
from tblSample
group by name
) t2
on t1.name = t2.name and cast(t1.id as char) <> t2.description
set t1.description = replace(
replace(
replace(
t2.description,
concat(',', t1.id, ','),
',')
, concat(',', t1.id)
, '')
, concat(t1.id, ',')
, '')
;
Demo Here
you can also use this query for test:
UPDATE dupli d
SET description = (
SELECT CONCAT('duplicate in ',GROUP_CONCAT(`id` ORDER BY id))
FROM (SELECT * FROM dupli) AS d1
WHERE `name` = d.`name` AND id <> d.id ) ;
sample
MariaDB [yourSchema]> UPDATE dupli d
-> SET description = (
-> SELECT CONCAT('duplicate in ',GROUP_CONCAT(`id` ORDER BY id))
-> FROM (SELECT * FROM dupli) AS d1
-> WHERE `name` = d.`name` AND id <> d.id ) ;
Query OK, 0 rows affected, 1 warning (0.00 sec)
Rows matched: 4 Changed: 0 Warnings: 1
MariaDB [yourSchema]> select * from dupli;
+----+------+------------------+
| id | name | description |
+----+------+------------------+
| 1 | foo | duplicate in 3,4 |
| 2 | bar | NULL |
| 3 | foo | duplicate in 1,4 |
| 4 | foo | duplicate in 1,3 |
+----+------+------------------+
4 rows in set (0.00 sec)
MariaDB [yourSchema]>
Related
Is there a way to filter result as follows:
Dataset:
ID NAME VALUE
-----------------------------------------
23 Test TRUE
24 Test FALSE
25 Test FALSE
26 Test TRUE
27 Test FALSE
28 Test FALSE
Result should be:
ID NAME VALUE
-----------------------------------------
23 Test TRUE
24 Test FALSE
26 Test TRUE
27 Test FALSE
The idea is to return all rows that has value TRUE and first row that has value of false after true
I guess this could be done with a self join, something like:
SELECT ID, NAME, VALUE
FROM table
WHERE VALUE = "TRUE"
UNION ALL
SELECT t2.ID, t2.NAME, t2.VALUE
FROM table t1
JOIN table t2
ON t1.ID +1 = t2.ID
AND t1.VALUE = "TRUE"
AND t2.VALUE = "FALSE"
EDIT: this also only works for nonebreaking IDs, so not usable for OPs case
E.g.:
DROP TABLE IF EXISTS my_table;
CREATE TABLE my_table
(id INT NOT NULL AUTO_INCREMENT PRIMARY KEY
,name VARCHAR(12) NOT NULL
,value TINYINT NOT NULL
);
INSERT INTO my_table VALUES
(23,'Test',TRUE),
(24,'Test',FALSE),
(25,'Test',FALSE),
(26,'Test',TRUE),
(27,'Test',FALSE),
(28,'Test',FALSE);
SELECT a.*
FROM my_table a
JOIN
( SELECT x.*
, MIN(y.id) minid
FROM my_table x
JOIN my_table y
ON y.id > x.id
AND y.value = FALSE
WHERE x.value = TRUE
GROUP
BY x.id
) b
ON a.id IN (b.id,b.minid);
+----+------+-------+
| id | name | value |
+----+------+-------+
| 23 | Test | 1 |
| 24 | Test | 0 |
| 26 | Test | 1 |
| 27 | Test | 0 |
+----+------+-------+
Probably only with joining table to same table again, only problem with this sql is that it needs to have non-breaking ID
Data
id name value
1 TEST 1
2 TEST 0
3 TEST 0
4 TEST 0
5 TEST 1
6 TEST 1
7 TEST 0
8 TEST 0
9 TEST 1
Join same table based on id from next row and then all rows from table where value true/1 or current row is t1.value=0 and previous t2.value = 1
SELECT t1.* FROM table as t1
LEFT JOIN table t2 ON t1.id = t2.id + 1
WHERE ( ( t1.value = 1 ) OR ( t1.value = 0 AND t2.value = 1 ) )
ORDER BY t1.id
Result
id name value
1 TEST 1
2 TEST 0
5 TEST 1
6 TEST 1
7 TEST 0
9 TEST 1
EDIT: My last chance, though not sure performance but noncontinous id are working with this
SELECT * FROM my_table as t1
LEFT JOIN my_table t2 ON
t2.id = (
SELECT MAX(my_table.id) FROM my_table
WHERE my_table.id < t1.id ORDER BY id asc LIMIT 1
)
WHERE (t1.value=1) OR (t1.value=0 AND t2.value=1) order by t1.id asc
I have a table like the following:
| ID | Short Name | Long Name |
|----|------------|-----------|
| 1 | s1 | l2 |
| 2 | s1 | l2 |
| 3 | s1 | l2 |
| 4 | s5 | l6 |
| .. | ... | |
I want to get all records that share the same Short Name and Long Name. I need their shared short name, long name, and 3 duplicates' IDs. For this particular example, I want {s1, l2, 1,2,3}
This is a fairly simple problem to solve. Basically what you want to do is write a subquery that counts the number of rows that match on your specified field for each row in your query. I have included a few examples below.
Find all rows that are duplicates and match on both name fields
SELECT * FROM TableName WHERE (SELECT COUNT(*) FROM TableName AS T2 WHERE T2.ShortName = TableName.ShortName AND T2.LongName = TableName.LongName) > 1;
Find all rows that are duplicates and match on the short name
SELECT * FROM TableName WHERE (SELECT COUNT(*) FROM TableName AS T2 WHERE T2.ShortName = TableName.ShortName) > 1;
Find all rows that are duplicates and match on the long name
SELECT * FROM TableName WHERE (SELECT COUNT(*) FROM TableName AS T2 WHERE T2.LongName = TableName.LongName) > 1;
Simply use a a self join of your table and select those rows where the two names are equal and the ids differ:
SELECT *
FROM <table> t1, <table> t2
WHERE t1.id <> t2.id
AND t1.short_name = t2.short_name
AND t1.long_name = t2.long_name;
You can use exits to see if there are any other data exists with the same condition where ID is not same.
select t1.* from table_name t1
where exists (
select 1 from table_name t2
where
t1.ID <> t2.ID
and t1.`Short Name` = t2.`Short Name`
and t1.`Long Name` = t2.`Long Name`
);
For example I have a table like given below .. I want to have separate columns on the basis of even/odd ids
-----------------------------------------------------
| ID | Names
-----------------------------------------------------
| 1 | Name1
-----------------------------------------------------
| 2 | Name2
-----------------------------------------------------
| 3 | Name3
-----------------------------------------------------
| 4 | Name4
-----------------------------------------------------
I want to design a query that could give me
-------------------
| Even | Odd |
-------------------
| Name2 | Name1 |
-------------------
| Name4 | Name3 |
-------------------
select
max(case id%2 when 1 then name end) as odd,
max(case id%2 when 0 then name end) as even
from your_table
group by floor((id+1)/2)
SQL Fiddle Demo
If you want to get odd or even, use next queries:
Select for odd records:
SELECT * FROM table WHERE ID % 2 = 1
Select, for even
SELECT * FROM table WHERE ID % 2 = 0
And if you want to decorate as two columns, try next solution:
SELECT
odd.name as Odd,
(SELECT name FROM table WHERE ID = odd.ID + 1 ) as Even
FROM
table as odd
WHERE
odd.ID % 2 = 1
If your Id column contain sequential number without any gap between number then :
SELECT t1.name as ODD,
t2.name as EVEN
FROM YourTable t1
left outer JOIN YourTable t2
ON t1.Id + 1 = t2.Id
where t1.Id%2 = 0
Note : if there are gaps between number then some of the ODD will be shown as NULL or it may skip that name if it has gap of more than 3.
Find the parity (property of even or odd) using modulo operator %.
...where id%2 equals 0;
This will give you even id in result.
The ones which are not equal to 0 are the odd id.
If you ID is sequential then
SELECT tb1.ODD, tb2.EVEN
FROM
(
SELECT Id, name ODD
FROM YourTable
where (`Id` % 2) = 1
) AS tb1
JOIN
(SELECT Id, name EVEN
FROM YourTable
WHERE (`Id` % 2) = 0
) AS tb2
ON (tb1.Id + 1 = tb2.Id)
WHERE tb1.ODD IS NOT NULL
ORDER BY tb1.Id
The above result set can be achieved by the following code -
SELECT Even, Odd
FROM (SELECT *, ROW_NUMBER()OVER(ORDER BY Even) AS ROW
FROM (SELECT CASE WHEN ID % 2 = 0 THEN Names ELSE NULL END AS 'Even'
FROM TableName)TAB1
WHERE Even IS NOT NULL)T1
FULL OUTER JOIN
(SELECT *, ROW_NUMBER()OVER(ORDER BY Odd) AS ROW FROM
(SELECT CASE WHEN ID % 2 = 1 THEN Names ELSE NULL END AS 'Odd'
FROM TableName) TAB2
WHERE Odd IS NOT NULL) T2
ON T1.ROW=T2.ROW
I'm trying to select just records which changed values compared to previous record,
my table is looking like this
Id(int) | value(boolean) |
-------------------------
1 | 0 |
-------------------------
2 | 1 |
-------------------------
3 | 1 |
-------------------------
4 | 1 |
-------------------------
5 | 0 |
-------------------------
6 | 0 |
-------------------------
I must get id:2,5
thanks for your help
I did a self-join, but instead of joining identical ids, I join t1.id = t2.id-1 and then compare the values:
select t2.id
from thetable t1
inner join thetable t2 on t1.id = t2.id-1
where t1.value != t2.value
sqlfiddle: http://sqlfiddle.com/#!2/6d626/4
Edit to add:
Thanks to #Ravinder, I figured out a way to do this even if the ids aren't sequential.
I used this related question.
SET #a :=0;
SET #b :=1;
SELECT t1.id, t1.rownum
FROM
(SELECT if(#a, #a:=#a+1, #a:=1) as rownum, id, value FROM thetable) AS t1
LEFT JOIN
(SELECT if(#b, #b:=#b+1, #b:=1) as rownum, id, value FROM thetable) AS t2
ON t1.rownum = t2.rownum
where t2.value != t1.value
SQLFiddle with non-sequential ids
Basically if you don't have sequential ids you create your own. I called them rownum.
You can use mysql variable in query to check if its new or unchanged
SELECT
`Id`,
`value`,
(CASE when #test=value then 'unchanged'
else 'changed' end) occurance,
#test:=`value` testvar
FROM
Table1
,(SELECT #test:='new') t
HAVING occurance='changed'
ORDER BY `Id`
See fiddle demo
I need to count repeated rows but only show them in results if at least one of them has status = 'new'.
______________
URL | Status
--------------
A new
A seen
B new
C seen
should echo:
___________
URL | SUM
-----------
A 2 (counts both the seen one and the new one because there is at least one nwe)
B 1
My idea is basically to count the repeated URLs and RIGHT JOIN it with the same table but only rows with Status = 'new' so that remaining rows disappear.
SELECT `userFlags` distinct(URL) WHERE Status = "new"
How do I add these conditions to the joining table and how is it called?
EDIT
I added Status = "new" to the query, how can I add distinct(URL) or nest the whole query on it?
SELECT userFlags.URL, COUNT( * ) AS SUM
FROM `userFlags`
RIGHT JOIN `userFlags` as u2 ON u2.Status = "new" AND userFlags.URL = u2.URL
GROUP BY u2.URL
ORDER BY SUM DESC
One possible answer is:
SELECT userFlags.URL, COUNT( * ) AS SUM
FROM `userFlags`
JOIN (select distinct URL from userflags where status = 'new') as u2 on u2.url = userflags.url
GROUP BY u2.URL
ORDER BY SUM DESC **strong text**
Try this :
Select userFlags.URL,Count(Status) AS SUM from userFlags
where Status = "new"
group by userFlags.URL
Edit
Select userFlags.URL,Count(Status) AS SUM from userFlags
where userFlags.URL in
(Select userFlags.URL from userFlags where Status = "new" )
group by userFlags.URL
Instead of an OUTER JOIN, do an INNER JOIN :
SELECT U1.URL, COUNT(*) AS Foo
FROM userFlags U1
INNER JOIN userFlags U2
ON U1.URL = U2.URL
AND U2.Status = 'new'
GROUP BY U1.URL;
If there is no row matching the join condition, then the row will not be added to the resultset. This filters out any URL that does not have at least one row where Status = new.
Edit: Removed the HAVING, for some reason I thought you only wanted rows where URL appeared more than once.
No need for a right join. I think that's just complicating things for you. Using a subquery to find the 'new' rows is one strategy.
select u2.URL, count(*) as SUM
from userFlags as u2
where u2.URL in (
select distinct u1.URL from userFlags as u1 where u1.Status = 'new'
) group by u2.URL;
Alternatively, this could be written as a join instead of a subquery.
select u2.URL, count(*) as SUM
from userFlags as u2
inner join (
select distinct u1.URL from userFlags as u1 where u1.Status = 'new'
) as subq on subq.URL = u2.URL
group by u2.URL;
Either one works. Here is a quick test to prove it:
mysql> create table userFlags (
-> URL varchar(255) not null,
-> Status enum('new', 'seen') not null,
-> index(URL),
-> index(Status)
-> ) engine=innodb;
Query OK, 0 rows affected (0.01 sec)
mysql> insert into userFlags (URL, Status) values ('A', 'new'), ('A', 'seen'), ('B', 'new'), ('C', 'seen');
Query OK, 4 rows affected (0.01 sec)
Records: 4 Duplicates: 0 Warnings: 0
mysql> select * from userFlags;
+-----+--------+
| URL | Status |
+-----+--------+
| A | new |
| A | seen |
| B | new |
| C | seen |
+-----+--------+
4 rows in set (0.00 sec)
mysql> select u2.URL, count(*) as SUM
-> from userFlags as u2
-> where u2.URL in (
-> select distinct u1.URL from userFlags as u1 where u1.Status = 'new'
-> ) group by u2.URL;
+-----+-----+
| URL | SUM |
+-----+-----+
| A | 2 |
| B | 1 |
+-----+-----+
2 rows in set (0.00 sec)
mysql> select u2.URL, count(*) as SUM
-> from userFlags as u2
-> inner join (
-> select distinct u1.URL from userFlags as u1 where u1.Status = 'new'
-> ) as subq on subq.URL = u2.URL
-> group by u2.URL;
+-----+-----+
| URL | SUM |
+-----+-----+
| A | 2 |
| B | 1 |
+-----+-----+
2 rows in set (0.01 sec)