ORDER BY and LIMIT in GROUP BY - mysql

I'm trying to get a subset of records in a GROUP BY, I've seen a lot of crazy solutions out there, but they just seem too complicated, is there any more efficient way to do this.
SELECT user_id, GROUP_CONCAT(item_id ORDER BY `timestamp`) AS items
FROM wb_user_book_current_item GROUP BY user_id
So this will return me all the current items for all users which is okay so far. But I only want the ten most recent items. Adding ORDER BY to the GROUP_CONCAT helps, but it still doesn't give me the last ten records.
EDIT
If I do something like this and hard code the user_id then I can get the results I want for that one user, problem is combining it so that I don't need to hard code the user_id and can for instance just get ALL users last ten items
SELECT GROUP_CONCAT(cp2.item_id) AS items
FROM (SELECT cp.user_id, cp.item_id
FROM wb_user_book_current_item cp
WHERE cp.user_id=1 ORDER BY cp.`timestamp`
LIMIT 10) AS cp2
GROUP BY cp2.user_id

This is a difficult problem, but how about this:
SELECT user_id, GROUP_CONCAT(item_id ORDER BY `timestamp`) AS items
FROM wb_user_book_current_item T
WHERE NOT EXISTS
(
SELECT 1
FROM wb_user_book_current_item T2
WHERE T2.user_id = T.user_id
ORDER BY T2.`timestamp` DESC
LIMIT 10,1
)
OR T.`timestamp` > (
SELECT T2.`timestamp`
FROM wb_user_book_current_item T2
WHERE T2.user_id = T.user_id
ORDER BY T2.`timestamp` DESC
LIMIT 10,1
)
GROUP BY user_id
This of course assumes you won't have two rows with the same timestamp for the same user.
If your timestamp field is always a positive integer, you can also replace the NOT EXISTS...OR with a COALESCE:
SELECT user_id, GROUP_CONCAT(item_id ORDER BY `timestamp`) AS items
FROM wb_user_book_current_item T
WHERE T.`timestamp` > COALESCE((
SELECT T2.`timestamp`
FROM wb_user_book_current_item T2
WHERE T2.user_id = T.user_id
ORDER BY T2.`timestamp` DESC
LIMIT 10,1
), 0)
GROUP BY user_id
Original answer, but apparently MySQL doesn't understand how to do this properly and complains the subselect returns multiple rows. Of course we want multiple rows; it's a GROUP_CONCAT. Grr.
Unfortunately, I think there's no real way around using a subquery:
SELECT T.user_id,
GROUP_CONCAT((SELECT T2.item_id
FROM wb_user_book_current_item T2
WHERE T2.user_id = T.user_id
ORDER BY T2.`timestamp`
LIMIT 10)) AS items
FROM wb_user_book_current_item T
GROUP BY user_id
Otherwise, adding LIMIT anywhere else will either limit the number of groups, or limit from the total recordset over the table (and not the group) - neither of which are what you are trying to achieve.

So came across a nice solution here that works pretty well.
http://www.xaprb.com/blog/2006/12/07/how-to-select-the-firstleastmax-row-per-group-in-sql/
It's something like this put all together:
SET #num := 0, #user_id := '';
SELECT cp2.user_id, CONCAT(cp2.item_id) AS items
FROM (
SELECT cp.user_id, cp.item_id,
#num := IF(#user_id = cp.user_id, #num + 1, 1) AS row_number,
#user_id := cp.user_id AS dummy
FROM wb_user_curent_item AS cp
ORDER BY cp.user_id ASC, cp.`timestamp` DESC
) AS cp2 WHERE cp2.row_number <= 10
GROUP BY cp2.user_id
So basically it just uses the num increment to limit the records rather than using LIMIT

SELECT
i.user_id,
GROUP_CONCAT(i.item_id ORDER BY i.timestamp) AS items
FROM
( SELECT DISTINCT user_id
FROM wb_user_book_current_item
) AS du
JOIN
wb_user_book_current_item AS i
ON i.user_id = du.user_id
AND i.timestamp <= COALESCE(
( SELECT i2.item_id
FROM wb_user_book_current_item AS i2
WHERE i2.user_id = du.user_id
ORDER BY i2.timestamp ASC
LIMIT 1 OFFSET 9
)
, '2038-01-19 03:14:07')
GROUP BY
i.user_id ;
An index on (user_id, timestamp, item_id) will help efficiency.

Try this:
SELECT
user_id,
GROUP_CONCAT(item_id ORDER BY `timestamp`) AS items
FROM wb_user_book_current_item
GROUP BY user_id
LIMIT 0, 10

UPDATE: I didn't notice the GROUP_CONCAT so you will have to use sub query in conunction with LIMIT
use LIMIT
SELECT column_name(s)
FROM table_name
LIMIT number

Related

MySQL group by only on specific amount of records (query inside JOIN expression)

I have a History table that represents messages that are stored by some sort of chat program.
It has a user id, message & datetime. (specified on: http://sqlfiddle.com/#!9/49fcefb/5)
Now, what I want is to have a result which contains:
The max amount of messages typed by date
What user has the most messages and howmany messages this user typed)
I got it working, but I find the query rather slow, I think its because of the last GROUP BY (GROUP BY b.cnt) as this will group over all the records found.
Query:
SELECT b.cnt as dayCount, a.cnt as userCount, a.userid as userId, b.date
FROM (
select date_added as date, user_id as userid, count(*) as cnt from history group by userid, day(date_added),month(date_added),year(date_added)
) a
INNER JOIN (
select date_added as date, count(*) as cnt from history group by day(date_added),month(date_added),year(date_added)
) b ON year(a.date) = year(b.date)
AND month(a.date) = month(b.date)
AND day(a.date) = day(b.date)
GROUP BY b.cnt
ORDER BY dayCount desc, userCount desc limit 10;
Can someone advice me on how to solve this? Maybe with another sort of query?
Thanks in advance!
You can try using windows function with Subquery to get the same result.
select SQ.daycount,SQ.usercount,SQ.user_id, SQ.date1 from (
select sum(count(Message)) over (partition by date(date_added)) as daycount, --to get max amount of msg by date
max(count(Message)) over (partition by date(date_added)) as maxuser, -- to identify user with most msg
count(Message) as usercount,
date(date_added) as date1,user_id from history
group by date1,user_id ) SQ
where SQ.usercount=SQ.maxuser
Check the Fiddle here
seems you are joining with wrong criteria.
first table is getting the # of users, 2nd table is getting the #of days per user_id. see dbfiddle
SELECT b.cnt as dayCount, a.cnt as userCount, a.userid as userId, b.date
FROM (
select
user_id as userid
, count(*) as cnt
from history
group by userid
) a
LEFT JOIN (
select cast(date_added as date) as date
, count(*) as cnt
, user_id as userid
from history
group by cast(date_added as date)
) b ON a.userid = b.userid
ORDER BY dayCount desc, userCount desc limit 10;

sql join Query the database ,There is no desired result,

my demand is:from database query newest data,according to time field as a standard,query alarmtime this field ,but alarmtime < time
executive sql:
SELECT
c.entryTime AS alarmtime ,
a.time AS time
FROM (
SELECT
t.DB33,
MAX( t.Time ) AS time,
t.Stream,
t.Coil,
t.`View`
FROM (
SELECT DB33, Time, Stream, Coil, `View`
FROM running_check
ORDER BY id DESC
LIMIT 1000 ) as t
GROUP BY t.DB33 ) AS a
LEFT JOIN (
select cameraID,cameraName
from monitor_link_info) as b ON a.db33 = b.cameraID
LEFT JOIN (
SELECT entryTime, cameraID
FROM result_video_v2
ORDER BY id DESC
limit 1000 ) AS c ON a.db33 = c.cameraId
GROUP BY a.db33, b.cameraName
ORDER BY a.time DESC, alarmtime DESC
execute result is unamiable,please look at image
now time column not what i want,Because he didn't change.
SELECT
c.entryTime AS alarmtime ,
a.time AS TIME
FROM
(
SELECT DB33, MAX(TIME) TIME, Stream, Coil, `View`
FROM running_check
ORDER BY id DESC
GROUP BY t.DB33
LIMIT 1000
) AS a
LEFT JOIN
(
SELECT cameraID,cameraName
FROM monitor_link_info
GROUP BY cameraID
LIMIT 1000
) AS b ON (a.db33 = b.cameraID)
LEFT JOIN (
SELECT entryTime, cameraID
FROM result_video_v2
GROUP BY cameraID
ORDER BY id DESC
LIMIT 1000
) AS c ON (a.db33 = c.cameraID)
GROUP BY a.db33, b.cameraName
ORDER BY a.time DESC, c.entryTime DESC

MySQL converting subquery to join

Is it possible to convert this subquery to join?
SELECT `news`.`newsId`,
(SELECT `comments`.`text`
FROM `comments`
WHERE `comments`.`newsId` = `news`.`newsId`
order by `comments`.`date` desc
limit 1)
FROM `news` , `comments`
where `news`.`newsId` = `comments`.`newsId`
GROUP BY `news`.`newsId`
order by news.date desc;
I assume newsId is unique.
SELECT `news`.`newsId`,
`comments`.`text`
FROM `news`
CROSS APPLY (SELECT `comments`.`text`
FROM `comments`
WHERE `comments`.`newsId` = `news`.`newsId`
order by `comments`.`date` desc
limit 1) cm
order by news.date desc;
I think that what you're trying to do is:
SELECT n.newsId FROM news n
INNER JOIN comments c ON c.newsId = n.newsId
ORDER BY c.date DESC, n.date
LIMIT 1
The GROUP BY is not necessary as you are not using any aggregation function. You can have unique entries with DISTINCT

LIMITing a SQL JOIN, with JOIN conditions

I have a problem similar to LIMITing a SQL JOIN, but with a slightly more complex requirement.
I want to search for Users and associated Transactions, which lie within a time range:
SELECT u.*, t.*
FROM User u
JOIN Transaction t ON t.user_id = u.id
WHERE t.timestamp >= ? and t.timestamp <= ?;
So far, so good. Now I want to repeat the query, but with a LIMIT on the number of users returned. There should be no limit on the number of transactions returned for a given user, though.
If I follow the approach suggested in the other question, this would translate into:
SELECT u.*, t.*
FROM (SELECT * FROM User LIMIT 10) u
JOIN Transaction t ON t.user_id = u.id
WHERE t.timestamp >= ? and t.timestamp <= ?;
This will not produce what I want: it will return the first 10 users, who might not have any transactions associated.
I want to return 10 users who have at least one associated transaction in the given time range.
How can I achieve this using MySQL?
You can use variables for this:
SELECT *
FROM (
SELECT *,
#rn := IF(#uid = user_id, #rn,
IF(#uid := user_id, #rn +1, #rn + 1)) AS rn
FROM (
SELECT u.*, t.*
FROM User u
JOIN Transaction t ON t.user_id = u.id
WHERE t.timestamp >= x and t.timestamp <= y) AS t
CROSS JOIN (SELECT #rn := 0, #uid := 0) AS vars
ORDER BY user_id) AS x
WHERE x.rn <= 10
Variable #rn is incremented by 1 every time a new user is returned by the query. So we can control the number of users returned using #rn <= 10.
You can do this without variables, but it requires repeating the join logic:
SELECT u.*, t.*
FROM (SELECT *
FROM User
WHERE EXISTS (SELECT 1
FROM Transaction t
WHERE t.user_id = u.id AND
t.timestamp >= ? and t.timestamp <= ?
)
LIMIT 10
) u JOIN
Transaction t
ON t.user_id = u.id
WHERE t.timestamp >= ? and t.timestamp <= ?;
EDIT:
Probably the fastest answer is something like this:
select u.*, t.*
from (select user_id
from (select user_id
from transaction t
where t.timestamp >= ? and t.timestamp <= ?
limit 1000
) t
limit 30
) tt join
user u
on tt.userid = u.id join
transaction t
on tt.userid = t.userid and t.timestamp >= ? and t.timestamp <= ?;
The first subquery chooses 1,000 matching records in the transaction table. My guess is that this is more than enough to get 30 users. This list is then joined to the user and transaction table to get the final results. By limiting the list without having to do a full table scan, the first query should be pretty fast . . . especially with an additional index on (timestamp, user).

MySQL group by last entry

My table payment_status have these fields:
id
payment_id
status
created
created_by
Many entries could have the same payment_id... So, I want to get the last status for one payment_id...
I have this request that work but take too much time to load... I would like to have an optimize version to group by payment_id and take the last status.
SELECT pstatus.*
FROM `payment_status` AS pstatus
WHERE pstatus.id = (
SELECT id
FROM `payment_status`
WHERE pstatus.status = '200'
ORDER BY created DESC
LIMIT 1
)
GROUP BY pstatus.payment_id
ORDER BY pstatus.payment_id DESC
Try this query -
SELECT t1.* FROM payment_status t1
JOIN (SELECT payment_id, MAX(created) max_created
FROM payment_status
GROUP BY payment_id
) t2
ON t1.payment_id = t2.payment_id AND t1.created = t2.max_created;
...then add WHERE conditions you need.
Try to use JOIN:
SELECT p1.*
FROM payment_status p1
INNER JOIN
(
SELECT id, MAX(created) MaxCreated
FROM payment_status
WHERE status = '200'
GROUP BY id
) p2 ON p1.id = p2.id AND p1.created = p2.MaxCreated
ORDER BY p1.payment_id DESC
this should work
SELECT *
FROM payment_status
WHERE status = '200'
ORDER BY created DESC
LIMIT 1
shouldn't yo ujust be able to do this?:
(assuming that created is the timestamp, so "last" = "most recent")
SELECT pstatus.*
FROM `payment_status` AS pstatus
GROUP BY pstatus.payment_id, pstatus.status
ORDER BY pstatus.payment_id DESC, pstatus.created DESC
each row returns should have the payment_id with the most recent status.
I suppose you are using MySQL database.
Here I have a solution is fast and readable:
select
substring_index(
group_concat(status order by created desc)
, ',', 1
) as status_latest
from payment_status
group by payment_id
I'm quite sure it's fast than others SQL statement,
You may try it.