MySQL sorting by date with GROUP BY - mysql

My table titles looks like this
id |group|date |title
---+-----+--------------------+--------
1 |1 |2012-07-26 18:59:30 | Title 1
2 |1 |2012-07-26 19:01:20 | Title 2
3 |2 |2012-07-26 19:18:15 | Title 3
4 |2 |2012-07-26 20:09:28 | Title 4
5 |2 |2012-07-26 23:59:52 | Title 5
I need latest result from each group ordered by date in descending order. Something like this
id |group|date |title
---+-----+--------------------+--------
5 |2 |2012-07-26 23:59:52 | Title 5
2 |1 |2012-07-26 19:01:20 | Title 2
I tried
SELECT *
FROM `titles`
GROUP BY `group`
ORDER BY MAX( `date` ) DESC
but I'm geting first results from groups. Like this
id |group|date |title
---+-----+--------------------+--------
3 |2 |2012-07-26 18:59:30 | Title 3
1 |1 |2012-07-26 19:18:15 | Title 1
What am I doing wrong?
Is this query going to be more complicated if I use LEFT JOIN?

This page was very helpful to me; it taught me how to use self-joins to get the max/min/something-n rows per group.
In your situation, it can be applied to the effect you want like so:
SELECT * FROM
(SELECT group, MAX(date) AS date FROM titles GROUP BY group)
AS x JOIN titles USING (group, date);

I found this topic via Google, looked like I had the same issue.
Here's my own solution if, like me, you don't like subqueries :
-- Create a temporary table like the output
CREATE TEMPORARY TABLE titles_tmp LIKE titles;
-- Add a unique key on where you want to GROUP BY
ALTER TABLE titles_tmp ADD UNIQUE KEY `group` (`group`);
-- Read the result into the tmp_table. Duplicates won't be inserted.
INSERT IGNORE INTO titles_tmp
SELECT *
FROM `titles`
ORDER BY `date` DESC;
-- Read the temporary table as output
SELECT *
FROM titles_tmp
ORDER BY `group`;
It has a way better performance. Here's how to increase speed if the date_column has the same order as the auto_increment_one (you then don't need an ORDER BY statement) :
-- Create a temporary table like the output
CREATE TEMPORARY TABLE titles_tmp LIKE titles;
-- Add a unique key on where you want to GROUP BY
ALTER TABLE titles_tmp ADD UNIQUE KEY `group` (`group`);
-- Read the result into the tmp_table, in the natural order. Duplicates will update the temporary table with the freshest information.
INSERT INTO titles_tmp
SELECT *
FROM `titles`
ON DUPLICATE KEY
UPDATE `id` = VALUES(`id`),
`date` = VALUES(`date`),
`title` = VALUES(`title`);
-- Read the temporary table as output
SELECT *
FROM titles_tmp
ORDER BY `group`;
Result :
+----+-------+---------------------+---------+
| id | group | date | title |
+----+-------+---------------------+---------+
| 2 | 1 | 2012-07-26 19:01:20 | Title 2 |
| 5 | 2 | 2012-07-26 23:59:52 | Title 5 |
+----+-------+---------------------+---------+
On large tables this method makes a significant point in terms of performance.

Well, if dates are unique in a group this would work (if not, you'll see several rows that match the max date in a group). (Also, bad naming of columns, 'group', 'date' might give you syntax errors and such specially 'group')
select t1.* from titles t1, (select group, max(date) date from titles group by group) t2
where t2.date = t1.date
and t1.group = t2.group
order by date desc

Another approach is to make use of MySQL user variables to identify a "control break" in the group values.
If you can live with an extra column being returned, something like this will work:
SELECT IF(s.group = #prev_group,0,1) AS latest_in_group
, s.id
, #prev_group := s.group AS `group`
, s.date
, s.title
FROM (SELECT t.id,t.group,t.date,t.title
FROM titles t
ORDER BY t.group DESC, t.date DESC, t.id DESC
) s
JOIN (SELECT #prev_group := NULL) p
HAVING latest_in_group = 1
ORDER BY s.group DESC
What this is doing is ordering all the rows by group and by date in descending order. (We specify DESC on all the columns in the ORDER BY, in case there is an index on (group,date,id) that MySQL can do a "reverse scan" on. The inclusion of the id column gets us deterministic (repeatable) behavior, in the case when there are more than one row with the latest date value.) That's the inline view aliased as s.
The "trick" we use is to compare the group value to the group value from the previous row. Whenever we have a different value, we know that we are starting a "new" group, and that this row is the "latest" row (we have the IF function return a 1). Otherwise (when the group values match), it's not the latest row (and we have the IF function returns a 0).
Then, we filter out all the rows that don't have that latest_in_group set as a 1.
It's possible to remove that extra column by wrapping that query (as an inline view) in another query:
SELECT r.id
, r.group
, r.date
, r.title
FROM ( SELECT IF(s.group = #prev_group,0,1) AS latest_in_group
, s.id
, #prev_group := s.group AS `group`
, s.date
, s.title
FROM (SELECT t.id,t.group,t.date,t.title
FROM titles t
ORDER BY t.group DESC, t.date DESC, t.id DESC
) s
JOIN (SELECT #prev_group := NULL) p
HAVING latest_in_group = 1
) r
ORDER BY r.group DESC

If your id field is an auto-incrementing field, and it's safe to say that the highest value of the id field is also the highest value for the date of any group, then this is a simple solution:
SELECT b.*
FROM (SELECT MAX(id) AS maxid FROM titles GROUP BY group) a
JOIN titles b ON a.maxid = b.id
ORDER BY b.date DESC

Use the below mysql query to get latest updated/inserted record from table.
SELECT * FROM
(
select * from `titles` order by `date` desc
) as tmp_table
group by `group`
order by `date` desc

Use the following query to get the most recent record from each group
SELECT
T1.* FROM
(SELECT
MAX(ID) AS maxID
FROM
T2
GROUP BY Type) AS aux
INNER JOIN
T2 AS T2 ON T1.ID = aux.maxID ;
Where ID is your auto increment field and Type is the type of records, you wanted to group by.

MySQL uses an dumb extension of GROUP BY which is not reliable if you want to get such results therefore, you could use
select id, group, date, title from titles as t where id =
(select id from titles where group = a.group order by date desc limit 1);
In this query, each time the table is scanned full for each group so it can find the most recent date. I could not find any better alternate for this. Hope this will help someone.

Related

SQL Formula for mysql Table

Hello – I have a DB table (MySQL ver 5.6.41-84.1-log) that has about 92,000 entries, with columns for:
id (incremental unique ID)
post_type (not important)
post_id (not important, but shows relation to another table)
user_id (not important)
vote (not important)
ip (IP Address, ie. 123.123.123.123)
voted (Datestamp in GMT, ie. 2018-12-03 04:50:05)
I recently ran a contest and we had a rule that no single IP could vote more than 60 times per day. So now I need to run a custom SQL formula that applies the following rule:
For each IP address, for each day, if there are > 60 rows, delete those additional rows.
Thank you for your help!
This is a complicated one, and I think it is hard to provide a 100% sure answer without actual table and data to play with.
However let me try to describe the logic, and build the query step by step so you can paly around with it and possibly fix lurking erros.
1) We start with selecting all ip adresses that posted more than 60 votes on a given day. For this we use a group by on the voting day and on the ip adress, combined with a having clause
select date(voted), ip_adress
from table
group by date(voted), ip_adress
having count(*) > 60
2) From then, we go back to the table and select the first 60 ids corresponding to each voting day / ip adress couple. id is an autoincremented field so we just sort using this field and the use the mysql limit instruction
select id, ip_adress, date(voted) as day_voted
from table
where ip_adress, date(voted) in (
select date(voted), ip_adress
from table
group by date(voted), ip_adress
having count(*) > 60
)
order by id
limit 60
3) Finally, we go back once again to the table and search for the all ids whose ip adress and day of vote belong to the above list, but whose id is greater than the max id of the list. This is achieved with a join and requires a group by clause.
select t1.id
from
table t1
join (
select id, ip_adress, date(voted) as day_voted
from table
where ip_adress, date(voted) in (
select date(voted), ip_adress
from table
group by date(voted), ip_adress
having count(*) > 60
)
order by id
limit 60
) t2
on t1.ip_adress = t2.ip_adress
and date(t1.voted) = t2.day_voted and t1.id > max(t2.id)
group by t1.id
That should return the list of all ids that we need to delete. Test if before you go further.
4) The very last step is to delete those ids. There are limitations in mysql that make a delete with subquery condition quite uneasy to achieve. See the following SO question for more information on the technical background. You can either use a temporary table to store the selected ids, or try to outsmart mysql by wrapping the subquery and aliasing it. Let us try with the second option :
delete t.* from table t where id in ( select id from (
select t1.id
from
table t1
join (
select id, ip_adress, date(voted) as day_voted
from table
where ip_adress, date(voted) in (
select date(voted), ip_adress
from table
group by date(voted), ip_adress
having count(*) > 60
)
order by id
limit 60
) t2
on t1.ip_adress = t2.ip_adress
and date(t1.voted) = t2.day_voted
and t1.id > max(t2.id)
group by t1.id
) x );
Hope this helps !
You could approach this by vastly simplifying your sample data and using row number simulation for mysql version prior to 8.0 or window function for versions 8.0 or above. I assume you are not on version 8 or above in the following example
drop table if exists t;
create table t(id int auto_increment primary key,ip varchar(2));
insert into t (ip) values
(1),(1),(3),(3),
(2),
(3),(3),(1),(2);
delete t1 from t t1 join
(
select id,rownumber from
(
select t.*,
if(ip <> #p,#r:=1,#r:=#r+1) rownumber,
#p:=ip p
from t
cross join (select #r:=0,#p:=0) r
order by ip,id
)s
where rownumber > 2
) a on a.id = t1.id;
Working in to out the sub query s allocates a row number per ip, sub query a then picks row numbers > 2 and the outer multi-table delete deletes from t joined to a to give
+----+------+
| id | ip |
+----+------+
| 1 | 1 |
| 2 | 1 |
| 3 | 3 |
| 4 | 3 |
| 5 | 2 |
| 9 | 2 |
+----+------+
6 rows in set (0.00 sec)
I had someone help me write the following query, which addressed my question.
SET SQL_SAFE_UPDATES = 0;
create table temp( SELECT id, ip, voted
FROM
(SELECT id, ip, voted,
#ip_rank := IF(#current_ip = ip, #ip_rank + 1, 1) AS ip_rank,
#current_ip := ip
FROM `table_name` where ip in (SELECT ip from `table_name` group by date(voted),ip having count(*) >60)
ORDER BY ip, voted desc
) ranked
WHERE ip_rank <= 2);
DELETE FROM `table_name`
WHERE id not in (select id from temp) and ip in (select ip from temp);
drop table temp;

SELECT value BEFORE max mysql

so, i have this table generated annually :
+----+------+
| id | name |
+----+------+
| 1 | 20162|
| 2 | 20162|
| 3 | 20171|
| 4 | 20171|<<<||| "how do i get this bfore max value"
| 5 | 20172|
| 6 | 20172|
+----+------+
If i query :
SELECT name FROM table WHERE where name=(SELECT max(name))
The result is 20172
How do i get the value before that (20171)?
This will get second max.
SELECT *
FROM table
WHERE name NOT IN (SELECT MAX(name) FROM table )
ORDER BY id DESC LIMIT 1
You can do the little trick:
SELECT name FROM table ORDER BY name DESC LIMIT 1,1
If you want a portable solution (since the above will only work in MySQL probably):
SELECT name FROM table t1 where
1 = (SELECT count(distinct name)
from table t2
where t2.name > t1.name )
This selects the name which has exactly one other name which is greater than it. It will have issues when e.g. all values are the same but that may actually be the intention sometimes.
Also you can try this:
SELECT name
FROM (SELECT DISTINCT name FROM yourtable) TMP
ORDER BY name DESC LIMIT 1,1
how about this:
select b.second_id,b.name from
(select id, max(name) as max from tbl_max) as a
LEFT JOIN
(select id as second_id, name from tbl_max ) as b
on a.id <> second_id where name < a.max order by name desc limit 1;
Another solution much better:
set #max= 0;
select max(name) into #max from tbl_max;
select * from tbl_max where name < #max ORDER BY id desc limit 1
it return a result of
4 20171
If you want Max(name)
SELECT id,MAX(name) FROM table
WHERE name < (SELECT MAX(name) FROM table )
or
SELECT id,MAX(name) FROM table
WHERE name <> (SELECT MAX(name) FROM table )
And there is an article for Find nth high value
(as Question changed later, this tricks is not useful for this problem)
can you use this simple order by ?
SELECT * FROM tablename
group by name
order by name desc
limit 1,1
if you have multiple equal values in MAX(name) you need group by, even this query is not general solution!
You can try below code.It will give you nth highest record.
SELECT NAME FROM table ORDER BY name DESC LIMIT n,1
Hope this will helps
SELECT DISTINCT name FROM table ORDER BY name DESC LIMIT 1,1;
The LIMIT clause accepts two values, an offset and a count. When only 1 value is provided, it assumes the offset is 0
DISTINCT is necessary as you have duplicate values for name. If this was unintentional, it's not needed.
Example at http://sqlfiddle.com/#!9/ea6e2/2

Fetch 2nd Higest value from MySql DB with GROUP BY

I have a table tbl_patient and I want to fetch last 2 visit of each patient in order to compare whether patient condition is improving or degrading.
tbl_patient
id | patient_ID | visit_ID | patient_result
1 | 1 | 1 | 5
2 | 2 | 1 | 6
3 | 2 | 3 | 7
4 | 1 | 2 | 3
5 | 2 | 3 | 2
6 | 1 | 3 | 9
I tried the query below to fetch the last visit of each patient as,
SELECT MAX(id), patient_result FROM `tbl_patient` GROUP BY `patient_ID`
Now i want to fetch the 2nd last visit of each patient with query but it give me error
(#1242 - Subquery returns more than 1 row)
SELECT id, patient_result FROM `tbl_patient` WHERE id <(SELECT MAX(id) FROM `tbl_patient` GROUP BY `patient_ID`) GROUP BY `patient_ID`
Where I'm wrong
select p1.patient_id, p2.maxid id1, max(p1.id) id2
from tbl_patient p1
join (select patient_id, max(id) maxid
from tbl_patient
group by patient_id) p2
on p1.patient_id = p2.patient_id and p1.id < p2.maxid
group by p1.patient_id
id11 is the ID of the last visit, id2 is the ID of the 2nd to last visit.
Your first query doesn't get the last visits, since it gives results 5 and 6 instead of 2 and 9.
You can try this query:
SELECT patient_ID,visit_ID,patient_result
FROM tbl_patient
where id in (
select max(id)
from tbl_patient
GROUP BY patient_ID)
union
SELECT patient_ID,visit_ID,patient_result
FROM tbl_patient
where id in (
select max(id)
from tbl_patient
where id not in (
select max(id)
from tbl_patient
GROUP BY patient_ID)
GROUP BY patient_ID)
order by 1,2
SELECT id, patient_result FROM `tbl_patient` t1
JOIN (SELECT MAX(id) as max, patient_ID FROM `tbl_patient` GROUP BY `patient_ID`) t2
ON t1.patient_ID = t2.patient_ID
WHERE id <max GROUP BY t1.`patient_ID`
There are a couple of approaches to getting the specified resultset returned in a single SQL statement.
Unfortunately, most of those approaches yield rather unwieldy statements.
The more elegant looking statements tend to come with poor (or unbearable) performance when dealing with large sets. And the statements that tend to have better performance are more un-elegant looking.
Three of the most common approaches make use of:
correlated subquery
inequality join (nearly a Cartesian product)
two passes over the data
Here's an approach that uses two passes over the data, using MySQL user variables, which basically emulates the analytic RANK() OVER(PARTITION ...) function available in other DBMS:
SELECT t.id
, t.patient_id
, t.visit_id
, t.patient_result
FROM (
SELECT p.id
, p.patient_id
, p.visit_id
, p.patient_result
, #rn := if(#prev_patient_id = patient_id, #rn + 1, 1) AS rn
, #prev_patient_id := patient_id AS prev_patient_id
FROM tbl_patients p
JOIN (SELECT #rn := 0, #prev_patient_id := NULL) i
ORDER BY p.patient_id DESC, p.id DESC
) t
WHERE t.rn <= 2
Note that this involves an inline view, which means there's going to be a pass over all the data in the table to create a "derived tabled". Then, the outer query will run against the derived table. So, this is essentially two passes over the data.
This query can be tweaked a bit to improve performance, by eliminating the duplicated value of the patient_id column returned by the inline view. But I show it as above, so we can better understand what is happening.
This approach can be rather expensive on large sets, but is generally MUCH more efficient than some of the other approaches.
Note also that this query will return a row for a patient_id if there is only one id value exists for that patient; it does not restrict the return to just those patients that have at least two rows.
It's also possible to get an equivalent resultset with a correlated subquery:
SELECT t.id
, t.patient_id
, t.visit_id
, t.patient_result
FROM tbl_patients t
WHERE ( SELECT COUNT(1) AS cnt
FROM tbl_patients p
WHERE p.patient_id = t.patient_id
AND p.id >= t.id
) <= 2
ORDER BY t.patient_id ASC, t.id ASC
Note that this is making use of a "dependent subquery", which basically means that for each row returned from t, MySQL is effectively running another query against the database. So, this will tend to be very expensive (in terms of elapsed time) on large sets.
As another approach, if there are relatively few id values for each patient, you might be able to get by with an inequality join:
SELECT t.id
, t.patient_id
, t.visit_id
, t.patient_result
FROM tbl_patients t
LEFT
JOIN tbl_patients p
ON p.patient_id = t.patient_id
AND t.id < p.id
GROUP
BY t.id
, t.patient_id
, t.visit_id
, t.patient_result
HAVING COUNT(1) <= 2
Note that this will create a nearly Cartesian product for each patient. For a limited number of id values for each patient, this won't be too bad. But if a patient has hundreds of id values, the intermediate result can be huge, on the order of (O)n**2.
Try this..
SELECT id, patient_result FROM tbl_patient AS tp WHERE id < ((SELECT MAX(id) FROM tbl_patient AS tp_max WHERE tp_max.patient_ID = tp.patient_ID) - 1) GROUP BY patient_ID
Why not use simply...
GROUP BY `patient_ID` DESC LIMIT 2
... and do the rest in the next step?

Only Return Highest Value for another unique Column

Preferably without downloading the entire database and scanning each item, is there a MySQL query that could access this table:
__________________________________
| ColID | Timestamp |
|___________|_____________________|
|_____2_____|_2012-08-01_12:00:00_|
|_____1_____|_2012-08-01_12:01:00_|
|_____3_____|_2012-08-01_12:02:00_|
|_____3_____|_2012-08-01_12:03:00_|
|_____2_____|_2012-08-01_12:04:00_|
|_____3_____|_2012-08-01_12:05:00_|
And return only these rows:
__________________________________
| ColID | Timestamp |
|___________|_____________________|
|_____1_____|_2012-08-01_12:01:00_|
|_____2_____|_2012-08-01_12:04:00_|
|_____3_____|_2012-08-01_12:05:00_|
So as to extract only one of each ColID with the highest Timestamp
Use GROUP BY with MAX:
SELECT ColID, MAX(Timestamp) AS Timestamp
FROM tbl
GROUP BY ColID
Also, just a tip you may want to keep in mind for the future: if you wanted to also select other columns that might be in the same table for each maximum ColID, you cannot select it directly in the above query due to the nature of GROUP BY. You will need to wrap the query in a joined subselect joining on both the id and date columns:
SELECT b.*
FROM tbl a
JOIN (
SELECT ColID, MAX(Timestamp) AS max_timestamp
FROM tbl
GROUP BY ColID
) b ON a.ColID = b.ColID AND a.max_timestamp = b.Timestamp
yes you can achive this by using GROUP BY with MAX as:
SELECT ColID, MAX(Timestamp) AS max_Timestamp
FROM my_table
GROUP BY ColID;

Get most recent row for given ID

In the table below, how do I get just the most recent row with id=1 based on the signin column, and not all 3 rows?
+----+---------------------+---------+
| id | signin | signout |
+----+---------------------+---------+
| 1 | 2011-12-12 09:27:24 | NULL |
| 1 | 2011-12-13 09:27:31 | NULL |
| 1 | 2011-12-14 09:27:34 | NULL |
| 2 | 2011-12-14 09:28:21 | NULL |
+----+---------------------+---------+
Use the aggregate MAX(signin) grouped by id. This will list the most recent signin for each id.
SELECT
id,
MAX(signin) AS most_recent_signin
FROM tbl
GROUP BY id
To get the whole single record, perform an INNER JOIN against a subquery which returns only the MAX(signin) per id.
SELECT
tbl.id,
signin,
signout
FROM tbl
INNER JOIN (
SELECT id, MAX(signin) AS maxsign FROM tbl GROUP BY id
) ms ON tbl.id = ms.id AND signin = maxsign
WHERE tbl.id=1
SELECT *
FROM tbl
WHERE id = 1
ORDER BY signin DESC
LIMIT 1;
The obvious index would be on (id), or a multicolumn index on (id, signin DESC).
Conveniently for the case, MySQL sorts NULL values last in descending order. That's what you typically want if there can be NULL values: the row with the latest not-null signin.
To get NULL values first:
ORDER BY signin IS NOT NULL, signin DESC
You may want to append more expressions to ORDER BY to get a deterministic pick from (potentially) multiple rows with NULL.
The same applies without NULL if signin is not defined UNIQUE.
Related:
mysql order by, null first, and DESC after
The SQL standard does not explicitly define a default sort order for NULL values. The behavior varies quite a bit across different RDBMS. See:
https://docs.mendix.com/refguide/null-ordering-behavior
But there are the NULLS FIRST / NULLS LAST clauses defined in the SQL standard and supported by most major RDBMS, but not by MySQL. See:
SQL how to make null values come last when sorting ascending
Sort by column ASC, but NULL values first?
Building on #xQbert's answer's, you can avoid the subquery AND make it generic enough to filter by any ID
SELECT id, signin, signout
FROM dTable
INNER JOIN(
SELECT id, MAX(signin) AS signin
FROM dTable
GROUP BY id
) AS t1 USING(id, signin)
Select [insert your fields here]
from tablename
where signin = (select max(signin) from tablename where ID = 1)
SELECT * FROM (SELECT * FROM tb1 ORDER BY signin DESC) GROUP BY id;
I had a similar problem. I needed to get the last version of page content translation, in other words - to get that specific record which has highest number in version column. So I select all records ordered by version and then take the first row from result (by using LIMIT clause).
SELECT *
FROM `page_contents_translations`
ORDER BY version DESC
LIMIT 1
Simple Way To Achieve
I know it's an old question
You can also do something like
SELECT * FROM Table WHERE id=1 ORDER BY signin DESC
In above, query the first record will be the most recent record.
For only one record you can use something like
SELECT top(1) * FROM Table WHERE id=1 ORDER BY signin DESC
Above query will only return one latest record.
Cheers!