Group datetime within range - mysql

I have some troubles in grouping some datasets. Here is my query and query result.
SELECT COUNT(*),salarie_id, created_at
FROM enregistrement
GROUP BY salarie_id, created_at
My point is to group similar created_at rows, in an interval of +/- 3 sec. I didn't manage it, even in using INTERVAL, in such criterias as HAVING
, WHERE ... BETWEEN.
How can I group these rows to get, for example 36 (33+3) in my count result, as shown in the image ?
Didn't found any suitable solution... Let me know if you want some additional information.
UPDATE 1 : Looks like #fancypants solution is on the right way.
SELECT COUNT(*),salarie_id, DATE_FORMAT(created_at, CONCAT('%Y-%m-%d %H:%i:', LEFT(RIGHT(created_at, 2), 1), RIGHT(created_at, 1) % 3)) AS 'date'
FROM enregistrement
GROUP BY salarie_id, DATE_FORMAT(created_at, CONCAT('%Y-%m-%d %H:%i:', LEFT(RIGHT(created_at, 2), 1), RIGHT(created_at, 1) % 3))

SELECT COUNT(*),salarie_id, created_at
FROM enregistrement
GROUP BY salarie_id,
DATE_FORMAT(created_at, CONCAT('%Y-%m-%d %H:%i:', LEFT(RIGHT(created_at, 2), 1),
CASE WHEN RIGHT(created_at, 1) BETWEEN 1 AND 4 THEN 0
WHEN RIGHT(created_at, 1) BETWEEN 5 AND 7 THEN 1
ELSE 2 END
));
You can use DATE_FORMAT() to bring your datetime/timestamp column in certain shapes. In your case we specify the format with
CONCAT('%Y-%m-%d %H:%i:', LEFT(RIGHT(created_at, 2), 1), RIGHT(created_at, 1) % 3))
That is, we take the year, the month, the day, the hour, the minute. Then we concatenate the date format string with the power of ten of seconds.
For example you have this seconds:
21
33
36
40
Then we take 2, 3, 3 and 4 with LEFT(RIGHT(created_at, 2), 1). The 1, 3, 6 and 0 we get with RIGHT(created_at, 1). Then you just have to put them into self-defined groups with CASE WHEN.

You need to create the column in your select, and then you can group by this column. Here is an example:
SELECT
COUNT(*),
salarie_id,
IF(created_at BETWEEN "2014-10-01 00:00:00" AND "2014-10-01 00:00:00", "2014-10-01 00:00:00"
ELSEIF(created_at BETWEEN "2015-02-18 11:05:43" AND "2015-02-18 11:05:46", "2015-02-18 11:05:43",
ELSEIF( ...
))) as buckets
FROM enregistrement
GROUP BY salarie_id, bucket
I am not sure how practical this may be in your case though, since you probably do not know the intervals beforehand.
An easy way to get through in your example would be to simply group by date and time without the seconds:
SELECT
DATE_FORMAT(created_at,"%Y-%m-%d %H:%i") as short_date
.....
GROUP BY salarie_id, short_date

Related

Get percentage of total when using GROUP BY in SQL query

I have a SQL query that I'm using to return the number of training sessions recorded by a client on each day of the week (during the last year).
SELECT COUNT(*) total_sessions
, DAYNAME(log_date) day_name
FROM programmes_results
WHERE log_date >= DATE_SUB(CURDATE(), INTERVAL 1 YEAR)
AND log_date <= CURDATE()
AND client_id = 7171
GROUP
BY day_name
ORDER
BY FIELD(day_name, 'MONDAY', 'TUESDAY', 'WEDNESDAY', 'THURSDAY', 'FRIDAY', 'SATURDAY', 'SUNDAY')
I would like to then plot a table showing these values as a percentage of the total, as opposed to as a 'count' for each day. However I'm at a bit of a loss as to how to do that without another query (which I'd like to avoid).
Any thoughts?
Use a derived table
select day_name, total_sessions, total_sessions / sum(total_sessions) * 100 percentage
from (
query from your question goes here
) temp
group by day_name, total_sessions
You can add the number of trainings per day in your client application to get the total count. This way you definitely avoid having a 2nd query to get the total.
Use the with rollup modifier in the query to get the total returned in the last row:
...GROUP BY day_name WITH ROLLUP ORDER BY ...
Use a subquery to return the overall count within each row
SELECT ..., t.total_count
...FROM programmes_results INNER JOIN (SELECT COUNT(*) as total_count FROM programmes_results WHERE <same where criteria>) as t --NO join condition
...
This will have the largest performance impact on the database, however, it enables you to have the total number in each row.

MySql query histogram for time intervals data

I have an event input of this type
event user
event start
event end
event type
Inserted to MySql table, each in its own row with user+start as primary key.
I need to query an histogram for a type by time interval (say minute) counting events occurred on each time interval.
something like:
SELECT count(*) as hits FROM events
WHERE type="browsing"
GROUP BY time_diff("2015-1-1" AND "2015-1-2") / 60 * second
but I could not find any way to do that in SQL besides writing code, any idea?
Sample data
user, start, end, type
1, 2015-1-1 12:00:00, 2015-1-1 12:03:59, browsing
2, 2015-1-1 12:03:00, 2015-1-1 12:06:00, browsing
2, 2015-1-1 12:03:00, 2015-1-1 12:06:00, eating
3, 2015-1-1 12:03:00, 2015-1-1 12:08:00, browsing
the result should look like this:
^
count |
browsing |
users | *
| * * * *
| * * * * * * * *
--|--|--|--|--|--|--|--|--|--> minute
0 1 2 3 4 5 6 7 8 9
You can do this using group by with the level that you want. Here is an example using the data you gave:
First the SQL to create the table and populate it. The ID column here isn't "needed" but it is recommended if the table will be large or have indexes on it.
CREATE TABLE `test`.`events` (
`id` INT NOT NULL AUTO_INCREMENT,
`user` INT NULL,
`start` DATETIME NULL,
`end` DATETIME NULL,
`type` VARCHAR(45) NULL,
PRIMARY KEY (`id`));
INSERT INTO events (user, start, end, type) VALUES
(1, '2015-1-1 12:00:00', '2015-1-1 12:03:59', 'browsing'),
(2, '2015-1-1 12:03:00', '2015-1-1 12:06:00', 'browsing'),
(2, '2015-1-1 12:03:00', '2015-1-1 12:06:00', 'eating'),
(3, '2015-1-1 12:03:00', '2015-1-1 12:08:00', 'browsing');
To get a list of ordered pairs of number of minutes duration to number of events:
The query can then be easily written using the timestampdiff fuction, as shown below:
SELECT
TIMESTAMPDIFF(MINUTE, start, end) as minutes,
COUNT(*) AS numEvents
FROM
test.events
GROUP BY TIMESTAMPDIFF(MINUTE, start, end)
The output:
minutes numEvents
3 3
5 1
The first parameter in the select can be one of FRAC_SECOND, SECOND, MINUTE, HOUR, DAY, WEEK, MONTH, QUARTER, or YEAR.
Here are some more examples of queries you can do:
Events by hour (floor function is applied)
SELECT
TIMESTAMPDIFF(HOUR, start, end) as hours,
COUNT(*) AS numEvents
FROM
test.events
GROUP BY TIMESTAMPDIFF(HOUR, start, end)
**Events by hour with better formatting **
SELECT
CONCAT("<", TIMESTAMPDIFF(HOUR, start, end) + 1) as hours,
COUNT(*) AS numEvents
FROM
test.events
GROUP BY TIMESTAMPDIFF(HOUR, start, end)
You can group by a variety of options, but this should definitely get you started. Most plotting packages will allow you to specify arbitrary x y coordinates, so you don't need to worry about the missing values on the x axis.
To get a list of ordered pairs of number of events at a specific time (for logging):
Note that this is left for reference.
Now for the queries. First you have to pick which item you want to use for the grouping. For example, a task might take more than a minute, so the start and end would be in different minutes. For all these examples, I am basing them off of the start time, since that is when the event actually took place.
To group event counts by minute, you can use a query like this:
SELECT
DATE_FORMAT(start, '%M %e, %Y %h:%i %p') as minute,
count(*) AS numEvents
FROM test.events
GROUP BY YEAR(start), MONTH(start), DAYOFMONTH(start), HOUR(start), MINUTE(start);
Note how this groups by all the items, starting with year, going the minute. I also have the minute displayed as a label. The resulting output looks like this:
minute numEvents
January 1, 2015 12:00 PM 1
January 1, 2015 12:03 PM 3
This is data that you could then take using php and prepare it for display by one of the many graphing libraries out there, plotting the minute column on the x axis, and plotting the numEvents on the y axis.
Here are some more examples of queries you can do:
Events by hour
SELECT
DATE_FORMAT(start, '%M %e, %Y %h %p') as hour,
count(*) AS numEvents
FROM test.events
GROUP BY YEAR(start), MONTH(start), DAYOFMONTH(start), HOUR(start);
Events by date
SELECT
DATE_FORMAT(start, '%M %e, %Y') as date,
count(*) AS numEvents
FROM test.events
GROUP BY YEAR(start), MONTH(start), DAYOFMONTH(start);
Events by month
SELECT
DATE_FORMAT(start, '%M %Y') as date,
count(*) AS numEvents
FROM test.events
GROUP BY YEAR(start), MONTH(start);
Events by year
SELECT
DATE_FORMAT(start, '%Y') as date,
count(*) AS numEvents
FROM test.events
GROUP BY YEAR(start);
I should also point out that if you have an index on the start column for this table, these queries will complete quickly, even with hundreds of millions of rows.
Hope this helps! Let me know if you have any other questions about this.
I am going to assume that you have a numbers table that contains integers. You also have $starttime and $endtime.
This is one way to get the values you want:
select ($starttime + interval n.n - 1 minute) as thetime, n.n as minutes,
count(sd.user)
from numbers n left join
sampledata sd
on $starttime + interval n.n - 1 minute between sd.start and sd.end
where $starttime + interval n.n - 1 minute <= $endtime and
sd.end >= $starttime and
sd.start <= $endtime
group by n.n
order by n.n;

Last date in quarter MySQL

I have a table with some dates. I need a query which will return the max (last) date from this table and last date of quarter this max date belongs to.
So for data i table
ID| EDATE
--+----------
1|2014-03-06
2|2014-10-12
this query should return 2014-10-12 and 2014-12-31.
As I understand you want the last day of the quarter, so 31 March, 30 June, 30 Sept, 31 Dec? So you can use the answer from Gordon Linoff and adjust it to do that.
You only need a case statement on month(date) and concat that with the year.
http://dev.mysql.com/doc/refman/5.1/de/control-flow-functions.html
str_to_date(
concat(
year(edate),
(case
when month(edate) in (1, 2, 3) then '-03-31'
when month(edate) in (4, 5, 6) then '-06-30'
when month(edate) in (7, 8, 9) then '-09-30'
else '-12-31'
end)
),
'%Y-%m-%d'
)
Getting the day of the last quarter for the date is a bit yucky, but possible. Here is a sort of brute force solution:
select edate,
str_to_date(concat(year(edate), '-', 1 + floor((month(edate) - 1)/ 3)) * 3, '-',
(case when month(edate) in (1, 2, 3, 10, 11, 12) then 31 else 30 end)),
'%Y-%m-%d'
)
from table t
order by edate desc
limit 1;
Here is a SQL Fiddle that demonstrates it.
You can use LAST_DAY to select the last day of a specific month depending on where your quarters end you may have to change the 3,6,9,12 to different months.
select t1.max_date,
(
case
when month(max_date) <= 3
then last_day(concat(year(max_date),'-3-1'))
when month(max_date) <= 6
then last_day(concat(year(max_date),'-6-1'))
when month(max_date) <= 9
then last_day(concat(year(max_date),'-9-1'))
else last_day(concat(year(max_date),'-12-1'))
end
) last_quarter_day
from (
select max(EDATE) max_date from myTable
) t1
I found the simplest answer:
SELECT MAKEDATE(YEAR(edate),1)
+ INTERVAL QUARTER(edate) QUARTER
- INTERVAL 1 DAY
This query takes the first day of year, adds quarters to it and subtracts 1 day to get the last day in wanted quarter. So the required query should look like:
SELECT MAX(edate),
MAKEDATE(YEAR(MAX(edate)),1)
+ INTERVAL QUARTER(MAX(edate)) QUARTER
- INTERVAL 1 DAY
FROM table

Group mysql results by "every 30 days"

I have a query:
SELECT
COUNT(id) as amount,
DATEDIFF(expire, buydate) as days
FROM `vw8mv_orders`
GROUP BY MONTH(expire)
The result is:
amount days
1 22
1 30
1 105
1 161
I'd like to see these results in a group (every 30 days). If days value is between 1 and 30 days, then put this in 30days group, if bet 31-60, put to 60days group, etc.
For example:
amount time
2 30 days
0 60 days
1 90 days
You will need to create a calculated column to group by. There are several approaches you could use for the calculation, but a good option might be integer division using the DIV operator:
SELECT
COUNT(id) as amount,
(((datediff(expire, buydate) DIV 30) + 1) * 30) as timegroup
FROM
table
GROUP BY timegroup;
The reason I like this approach, rather than using for example some fancy arithmetic with ROUND(), is that it's a little more clear what you're trying to do. datediff(expire, buydate) DIV 30 says, take the difference of these dates, and tell me "how many 30s" are in that number.
That's all you need for your grouping; the rest is there to make the column display the way you want it, as 30, 60, 90, ... instead of as 0, 1, 2, ....
Another option, if you're not comfortable with integer division, would be the CEILING function:
SELECT
COUNT(id) as amount,
30 * CEILING(datediff(expire, buydate) / 30) as timegroup
FROM
table
GROUP BY timegroup;
Mathematically speaking, CEILING(x / N) is equivalent to ((x DIV N) + 1), but it's a little less busy with CEILING().
You can do a subselect over the result returned from your query,below is the example query
SELECT COUNT(`amount`) as amount,
CONCAT(ROUND(`days` / 30) * 30, ' Days')
as `time`
FROM `t`
GROUP BY `time`
ORDER BY ROUND(`days` / 30)
Demo
For your query you can do so
SELECT COUNT(`amount`) as amount,
CONCAT(ROUND(`days` / 30) * 30, ' Days')
as `time`
FROM(
SELECT COUNT(id) as amount,
datediff(expire, buydate) as days
FROM `vw8mv_orders`
GROUP BY MONTH(expire)
) t
GROUP BY `time`
ORDER BY ROUND(`days` / 30)

order by with group by sub query not working

I have a simple stats query in mysql.
I had it working using php but was keen to keep it in the database so I found another question on it here. Thanks to that I have it working in mysql now.
I need to get a running total count of records submitted per day.
I have the running total working but no matter what I change it simply will not order by day.
It will order by the running total column but not the day.
Any input is appreciated.
SET #runtot:=0;
SELECT
q1.`Day` AS 'Days of month',
q1.`Record count` AS 'Record count',
(#runtot := #runtot + q1.`Record count`) AS 'rt'
FROM
(SELECT
FROM_UNIXTIME(tr.`tr_last_update`, '%e') AS 'Day',
COUNT(tr.tr_id_pk) AS 'Record Count'
FROM records_table AS tr
GROUP BY FROM_UNIXTIME(tr.`tr_last_update`,'%e')
) AS q1
ORDER BY 'Days of month';
Days of month Record count rt
10 13 13
11 2 15
7 255 270
8 173 443
9 166 609
It's being treated as a string. Try casting to an int:
SELECT
q1.`Day` AS 'Days of month',
...
ORDER BY CAST(q1.`Day` AS signed integer)
Unless I'm much mistaken, isn't your whole query the same as:
SELECT DAYOFMONTH(FROM_UNIXTIME(tr_last_update)) AS `Days of month`,
COUNT(tr_id_pk) AS `Record Count`,
#runtot := #runtot + COUNT(tr_id_pk) AS rt
FROM records_table, (SELECT #runtot:=0) AS z
GROUP BY `Days of month`
ORDER BY `Days of month`