Problem with this SQL that finds duplicate records - mysql

Question

Please help me with a problem with this SQL in phpmyadmin that finds duplicate records for cleanup:
SELECT userid, MIN(logged_time),`logged_time`, count(email) as duplicated_count
FROM tbl_users
GROUP BY email HAVING count(email) > 1
ORDER BY `duplicated_count` DESC
Here is the GOAL:
Trying to list the duplicate email records in a user database that also tracks when they last used the system under the account.
Trying to identify the records with the oldest logged_time so those duplicates ONLY can be deleted.
PROBLEM:
The SQL above WORKS to identify the duplicates. AND, it WORKS to display the oldest (min) login. BUT>>> The one userID it returns is NOT the user id with the oldest login.
In other words, the data returned is from different records in the duplicate search for the same user email.
The results look like:
userid, MIN(logged_time), logged time,
"111" "2013-04-10 22:35:21", "2017-10-01 04:17:49"
SO...... User "111" is not the one I want to delete! That userid matches the most RECENT logged time. I want the userid for the record that matched MIN(logged_time).
Thanks for the help! I know this may be confusing.
MySQL version is 5.5, so the code posted doesn't seem to work with this version. Any other suggestions?

Answer 1

The one userID it returns is NOT the user id with the oldest login.
Selecting min(logged_time) does not say to only return the oldest login. It just gives you the oldest logged_time for each email.
What you asked for is invalid SQL, but MySQL allows it. It's invalid because it leads to exactly the trouble you're having.
Say we have this.
userid email logged_time
-----------------------------
1 email1 2021-01-01
2 email2 2021-01-01
3 email1 2021-02-01
4 email3 2021-01-01
If we select userid, min(logged_time), count(email) from tbl_users group by email, MySQL can only show you one row per email. For email2 and email3 that's fine, there's only one option. But for email1 it has to choose either 1 or 3.
Normally this would make the query invalid and you would get an error. You can't select a non-aggregated column (userid) which is not in the group by.
But MySQL allows this. It picks a userid at random. And we get the problem you're having.
userid min(logged_time) count(email)
--------------------------------------
2 2021-01-01 1
3 2021-01-01 2
4 2021-01-01 1
See MySQL Handling of GROUP BY for more, and I'd suggest you turn it off with ONLY_FULL_GROUP_BY.
What you're asking for is a bit tricky.
You need to both find emails with duplicates AND pick the oldest row for each email. You can't do all that in a single query, you need two queries joined together.
You know how to find duplicate emails.
-- Note that if you have two entries with the same userid and email
-- it will count that as a duplicate.
select email
from tbl_users
group by email
having count(email) > 1
To order the emails by their logged_time use the row_number window function.
select
userid,
email,
logged_time,
row_number() over(partition by email order by logged_time asc) as login_order
from tbl_users
That will return all the rows with each ranked.
userid email logged_time login_order
--------------------------------------------
1 email1 2021-01-01 1
2 email2 2021-01-01 1
3 email1 2021-02-01 2
4 email3 2021-01-01 1
Join them together and select only where login_order = 1.
-- Get the emails with dups.
with users_with_duplicates as (
select email
from tbl_users
group by email
having count(email) > 1
),
-- Get the logins by email ranked by ascending logged_time
users_in_login_order as (
select
userid,
email,
logged_time,
row_number() over(partition by email order by logged_time asc) as login_order
from tbl_users
)
select
users_in_login_order.*
from users_in_login_order
-- The join will constrain users_in_login_order to only emails with dups
join users_with_duplicates
on users_in_login_order.email = users_with_duplicates.email
where login_order = 1
Try it.