mysql: delete rows between repeating specific values - mysql

Question

SQL rookie here. I have a broken punch in/out type table with millions of records fed by a legacy bad app that did not check for previous logins/logouts before merrily inserting another duplicate record. The app is fixed but I need to sanitize the table to retain the historical data so it can be fed into future reports.
In a nutshell, I'm trying to keep each minimum login row followed by the next minimum logout row and discard everything else between. The bad app allowed both duplicate logins AND logouts... grrrr.
Every "duplicate row" type question I've searched for here doesn't seem to apply to this type of grouping situation. From being a long time SO lurker I know you guys would like to see what I've already tried but have already tried tens of goofy query attempts that aren't coming close. Any guidance would be greatly appreciated.
Here's the table and what I'm trying to do and the fiddle with schema
+---------------------+-------+-------------+---------------+
| calldate | agent | etype | uniqueid |
+---------------------+-------+-------------+---------------+
| 2018-02-02 19:26:47 | 501 | agentlogin | 1517599607.71 |
| 2018-02-02 19:26:55 | 501 | agentlogin | 1517599615.72 |<-- delete
| 2018-02-02 19:27:32 | 501 | agentlogoff | 1517599652.73 |
| 2018-02-02 19:27:43 | 501 | agentlogin | 1517599663.74 |
| 2018-02-02 19:28:24 | 501 | agentlogoff | 1517599704.75 |
| 2018-02-02 19:29:02 | 501 | agentlogoff | 1517599742.76 |<-- delete
| 2018-02-02 19:29:39 | 501 | agentlogoff | 1517599778.77 |<-- delete
| 2018-02-02 19:34:54 | 501 | agentlogin | 1517600094.80 |
| 2018-02-02 19:35:23 | 501 | agentlogin | 1517600122.81 |<-- delete
| 2018-02-02 19:35:49 | 501 | agentlogin | 1517600149.82 |<-- delete
| 2018-02-02 19:36:04 | 501 | agentlogoff | 1517600164.83 |
| 2018-02-02 19:36:08 | 501 | agentlogoff | 1517600168.84 |<-- delete
+---------------------+-------+-------------+---------------+

Answer 1

I would create a copy of the table with an auto_increment column. This way you can compare two neighbor rows more easily and more efficiently.
Find in the new table the rows which have the same agent and etype as in the previous row and join the result with the original table using the unique column in a DELETE statement.
create table tmp (
`id` int unsigned auto_increment primary key,
`calldate` datetime,
`uniqueid` varchar(32),
`agent` varchar(80),
`etype` varchar(80)
) as
select null as id, calldate, uniqueid, agent, etype
from test
order by agent, calldate, uniqueid
;
delete t
from tmp t1
join tmp t2
on t2.id = t1.id + 1
and t2.agent = t1.agent
and t2.etype = t1.etype
join test t on t.uniqueid = t2.uniqueid;
drop table tmp;
Demo: http://sqlfiddle.com/#!9/3e96b/2
You should however first have an index on uniqueid.

Answer 2

Here you go:
select calldate,agent,etype,uniqueid
from test as t1
where not exists
(select *
from
test as t2
where t2.agent=t1.agent
and t2.etype=t1.etype
and t2.uniqueid<t1.uniqueid
and t2.uniqueid>ifnull((select max(uniqueid )
from test t3
where t3.agent=t1.agent
and t3.etype<>t1.etype
and t3.uniqueid<t1.uniqueid),0)
)
order by uniqueid;
http://sqlfiddle.com/#!9/149802/16