I want to count the number of actions per day in my dataset.
date action_id
2010-01-01 id00
2010-01-03 id01
2010-01-05 id02
This is just a sample, but the point is that my data does not include actions for every day and I want to include days where there are zero actions in my result.
My plan is to do this.
with dates as (
select [sequence of dates from 2010-01-01 to 2010-02-01] as day)
select day, coalesce(count(distinct action_id), 0) as actions
from dates
left join my_table
on dates.date = my_table.date
How do I create the sequence of dates?
You example shows a CTE. So, you can use a recursive CTE:
with recursive dates as (
select date('2010-01-01') as day
union all
select day + interval 1 day
from dates
where day < '2010-02-01'
)
select d.day, count(distinct t.action_id) as actions
from dates d left join
my_table t
on d.day = my_table.date
group by d.day;
Note that COUNT() never returns NULL, so COALESCE() is unnecessary.
In older versions, you can use a calendar table or generate the data on the fly. Assuming your table has enough rows:
select d.day, count(distinct t.action_id) as actions
from (select date('2010-01-01') + interval (#rn := #rn + 1) - 1 day as day
from my_table cross join
(select #rn := 0) params
limit 31
) d left join
my_table t
on d.day = my_table.date
group by d.day;
it seems just you need group by and count
select date, count(distinct action_id) as action
from my_table left join
dates on dates.date = my_table.date
group by date
with dates as
(
select a.Date
from (
select curdate() - INTERVAL (a.a + (10 * b.a) + (100 * c.a) + (1000 * d.a) ) DAY as Date
from (select 0 as a union all select 1 union all select 2 union all select 3 union all select 4 union all select 5 union all select 6 union all select 7 union all select 8 union all select 9) as a
cross join (select 0 as a union all select 1 union all select 2 union all select 3 union all select 4 union all select 5 union all select 6 union all select 7 union all select 8 union all select 9) as b
cross join (select 0 as a union all select 1 union all select 2 union all select 3 union all select 4 union all select 5 union all select 6 union all select 7 union all select 8 union all select 9) as c
cross join (select 0 as a union all select 1 union all select 2 union all select 3 union all select 4 union all select 5 union all select 6 union all select 7 union all select 8 union all select 9) as d
) a
where a.Date between '<start_date>' and '<end_date>' )
select day, count(distinct action_id) as actions
from dates
left join my_table
on dates.date = my_table.date
I have a MySql table containing events having a DATETIME timestamp. I want to count each day's events. On some days, e.g. on Sundays, events are missing. The result should contain these days too with a count of zero.
My query is the following:
SELECT
COUNT(1) AS mycount,
DATE_FORMAT(DATE(evaluations.timestamp),"%a, %d.%m.%Y") AS date
FROM Events
GROUP BY DATE(timestamp)
ORDER BY DATE(timestamp) DESC
Can I modify the query without using a helper table containing all dates?
A single query (no procedere, no function) would be fine.
The query would somehow look like this if you don't have any calendar table:
SELECT
dateTable.day,
COALESCE(t.mycount,0) AS cnt
FROM
(
SELECT ADDDATE((SELECT MIN(DATE(timestamp)) FROM Events), INTERVAL #i:=#i+1 DAY) AS DAY
FROM (
SELECT a.a
FROM (SELECT 0 AS a UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) AS a
CROSS JOIN (SELECT 0 AS a UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) AS b
CROSS JOIN (SELECT 0 AS a UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) AS c
) a
JOIN (SELECT #i := -1) r1
WHERE
#i < DATEDIFF((SELECT MAX(DATE(timestamp)) FROM Events), (SELECT MIN(DATE(timestamp)) FROM Events))
) AS dateTable
LEFT JOIN
(
SELECT
COUNT(1) AS mycount,
DATE_FORMAT(DATE(evaluations.timestamp),"%a, %d.%m.%Y") AS date
FROM Events
GROUP BY DATE(timestamp)
ORDER BY DATE(timestamp) DESC
) AS t
ON dateTable.day = t.date
ORDER BY dateTable.day DESC;
Note:
If you think you will need this kind of query too often then you can create a table where all the dates would reside. Newer dates can be added through mysql event periodically .
Then the work is simple. Just need to make a LEFT JOIN between the calendar table and the result of your query.
I've a table named userActivity where each active period is recorded.
Here's the table structure:
Table: userActivity
ID user_id start_time end_time
When a user comes into online the start time is recorded and whenever the online status is changed the end time is recorded in the corresponding row.
Now I've to generate a report where a day wise available time of the users will be shown.
Sample Input:
ID user_id start_time end_time
'1' '1' '2016-02-28 10:00:00' '2016-02-28 19:00:00'
'2' '1' '2016-02-28 22:00:00' '2016-02-29 10:00:00'
'3' '1' '2016-03-02 10:00:00' '2016-03-02 19:00:00'
'4' '1' '2016-03-02 22:00:00' '2016-03-06 19:00:00'
Expected output:
Date AvailableTime(Hours)
2016-02-28 11
2016-02-29 10
2016-03-02 11
2016-03-03 24
2016-03-04 24
2016-03-05 24
2016-03-06 19
So far what I've tried:
SELECT
DATE_FORMAT(start_time,"%Y-%m-%d") `date`,
TIMESTAMPDIFF(HOUR,start_time,end_time) availableTime
FROM useractivity
GROUP BY `date`
Got output:
Date availableTime(Hours)
2016-02-28 9
2016-03-02 9
Here's the SQL FIDDLE
Note:
Please ignore the user_id for the time being. I can solve it in application level but I want to deal with it in MySQL.
The time interval can start one day and end more than one day later
In a word, the available time is just the projection in the day axis (from start time and end time). If the start time doesn't project into the same day as end time then the start time would be considered the start_time of that particular day where the end time projects into.
Pictorial View:
So available time will be calculated from this screenshot as follows:
28 Feb = (t2-t1) + (t4- t3)
29 Feb = (t5 - t4)
02 Mar = (t7 - t6)
You could use a date table to cross join to get the real start and end time in the day you want to split from the log time.
CREATE TABLE `dates` (
`date` date ,
`start_time` timestamp ,
`end_time` timestamp
);
INSERT INTO `dates` VALUES('20160228','2016-02-28 00:00:00', '2016-02-29 00:00:00');
INSERT INTO `dates` VALUES('20160229','2016-02-29 00:00:00', '2016-03-01 00:00:00');
INSERT INTO `dates` VALUES('20160301','2016-03-01 00:00:00', '2016-03-02 00:00:00');
INSERT INTO `dates` VALUES('20160302','2016-03-02 00:00:00', '2016-03-03 00:00:00');
INSERT INTO `dates` VALUES('20160303','2016-03-03 00:00:00', '2016-03-04 00:00:00');
INSERT INTO `dates` VALUES('20160304','2016-03-04 00:00:00', '2016-03-05 00:00:00');
INSERT INTO `dates` VALUES('20160305','2016-03-05 00:00:00', '2016-03-06 00:00:00');
INSERT INTO `dates` VALUES('20160306','2016-03-06 00:00:00', '2016-03-07 00:00:00');
SELECT
u.*,
d.date,
case when u.start_time<= d.start_time then d.start_time
else u.start_time end as `start_time_in_the_day`,
case when u.end_time> d.end_time then d.end_time
else u.end_time end as `end_time_in_the_day`
FROM useractivity u
INNER JOIN dates d
ON u.start_time< d.end_time
and u.end_time>= d.start_time
Then you just need to sum the hours between end_time_in_the_day and start_time_in_the_day.
SELECT
user_id,
date,
sum(TIMESTAMPDIFF(HOUR,start_time_in_the_day,end_time_in_the_day)) as `availableTime`
FROM(
SELECT
u.*,
d.date,
case when u.start_time<= d.start_time then d.start_time
else u.start_time end as `start_time_in_the_day`,
case when u.end_time> d.end_time then d.end_time
else u.end_time end as `end_time_in_the_day`
FROM useractivity u
INNER JOIN dates d
ON u.start_time< d.end_time
and u.end_time>= d.start_time) as t
group by user_id,date
My SqlFiddle here.
And I think use TIMESTAMPDIFF(SECOND... instead TIMESTAMPDIFF(HOUR... would be better.
I have created a query help of UNION ALL
SELECT sub_query.`date`, SUM(sub_query.available_time) FROM (
SELECT
DATE_FORMAT(start_time,"%Y-%m-%d") `date`,
IF(TIMESTAMPDIFF(day,date(start_time),date(end_time))= 0,
TIMESTAMPDIFF(HOUR,start_time,end_time),0) AS available_time
FROM useractivity
UNION ALL
SELECT
DATE_FORMAT(start_time,"%Y-%m-%d") `date`,
IF(TIMESTAMPDIFF(day,date(start_time),date(end_time)) > 0,
TIMESTAMPDIFF(HOUR,start_time, date_add(date(start_time),interval 24 hour)),0) AS available_time
FROM useractivity
UNION ALL
SELECT
DATE_FORMAT(end_time,"%Y-%m-%d") `date`,
IF(TIMESTAMPDIFF(day,date(start_time),date(end_time)) > 0,
TIMESTAMPDIFF(HOUR,date(end_time), end_time) , 0) AS available_time
FROM useractivity
) AS sub_query
GROUP BY sub_query.`date`
UNION
SELECT SELECTed_date `date`, 24 FROM
(SELECT adddate('1970-01-01',t4.i*10000 + t3.i*1000 + t2.i*100 + t1.i*10 + t0.i) SELECTed_date FROM
(SELECT 0 i UNION SELECT 1 UNION SELECT 2 UNION SELECT 3 UNION SELECT 4 UNION SELECT 5 UNION SELECT 6 UNION SELECT 7 UNION SELECT 8 UNION SELECT 9) t0,
(SELECT 0 i UNION SELECT 1 UNION SELECT 2 UNION SELECT 3 UNION SELECT 4 UNION SELECT 5 UNION SELECT 6 UNION SELECT 7 UNION SELECT 8 UNION SELECT 9) t1,
(SELECT 0 i UNION SELECT 1 UNION SELECT 2 UNION SELECT 3 UNION SELECT 4 UNION SELECT 5 UNION SELECT 6 UNION SELECT 7 UNION SELECT 8 UNION SELECT 9) t2,
(SELECT 0 i UNION SELECT 1 UNION SELECT 2 UNION SELECT 3 UNION SELECT 4 UNION SELECT 5 UNION SELECT 6 UNION SELECT 7 UNION SELECT 8 UNION SELECT 9) t3,
(SELECT 0 i UNION SELECT 1 UNION SELECT 2 UNION SELECT 3 UNION SELECT 4 UNION SELECT 5 UNION SELECT 6 UNION SELECT 7 UNION SELECT 8 UNION SELECT 9) t4) v
WHERE SELECTed_date between (SELECT min(date(start_time)) FROM useractivity) and (SELECT max(date(end_time)) FROM useractivity)
AND SELECTed_date NOT IN(
SELECT miss_date FROM (
SELECT date(start_time) AS miss_date FROM useractivity
UNION
SELECT date(end_time) AS miss_date FROM useractivity
) AS miss
) ORDER BY `date`;
SQLFiddle
I've modified #Vipin Jain's query to fulfill the requirement:
SELECT sub_query.`date`, SUM(sub_query.available_time) FROM (
SELECT
DATE_FORMAT(start_time,"%Y-%m-%d") `date`,
IF(TIMESTAMPDIFF(day,date(start_time),date(end_time))= 0,
TIMESTAMPDIFF(HOUR,start_time,end_time),0) AS available_time
FROM useractivity
UNION ALL
SELECT
DATE_FORMAT(start_time,"%Y-%m-%d") `date`,
IF(TIMESTAMPDIFF(day,date(start_time),date(end_time)) > 0,
TIMESTAMPDIFF(HOUR,start_time, date_add(date(start_time),interval 24 hour)),0) AS available_time
FROM useractivity
UNION ALL
SELECT
DATE_FORMAT(end_time,"%Y-%m-%d") `date`,
IF(TIMESTAMPDIFF(day,date(start_time),date(end_time)) > 0,
TIMESTAMPDIFF(HOUR,date(end_time), end_time) , 0) AS available_time
FROM useractivity
) AS sub_query
GROUP BY sub_query.`date`
UNION
SELECT SELECTed_date `date`, 24 FROM
(SELECT adddate('1970-01-01',t4.i*10000 + t3.i*1000 + t2.i*100 + t1.i*10 + t0.i) SELECTed_date FROM
(SELECT 0 i UNION SELECT 1 UNION SELECT 2 UNION SELECT 3 UNION SELECT 4 UNION SELECT 5 UNION SELECT 6 UNION SELECT 7 UNION SELECT 8 UNION SELECT 9) t0,
(SELECT 0 i UNION SELECT 1 UNION SELECT 2 UNION SELECT 3 UNION SELECT 4 UNION SELECT 5 UNION SELECT 6 UNION SELECT 7 UNION SELECT 8 UNION SELECT 9) t1,
(SELECT 0 i UNION SELECT 1 UNION SELECT 2 UNION SELECT 3 UNION SELECT 4 UNION SELECT 5 UNION SELECT 6 UNION SELECT 7 UNION SELECT 8 UNION SELECT 9) t2,
(SELECT 0 i UNION SELECT 1 UNION SELECT 2 UNION SELECT 3 UNION SELECT 4 UNION SELECT 5 UNION SELECT 6 UNION SELECT 7 UNION SELECT 8 UNION SELECT 9) t3,
(SELECT 0 i UNION SELECT 1 UNION SELECT 2 UNION SELECT 3 UNION SELECT 4 UNION SELECT 5 UNION SELECT 6 UNION SELECT 7 UNION SELECT 8 UNION SELECT 9) t4) v
WHERE SELECTed_date between (SELECT min(date(start_time)) FROM useractivity) and (SELECT max(date(end_time)) FROM useractivity)
AND SELECTed_date NOT IN(
SELECT miss_date FROM (
SELECT date(start_time) AS miss_date FROM useractivity
UNION
SELECT date(end_time) AS miss_date FROM useractivity
) AS miss
)
AND EXISTS (SELECT 1 FROM useractivity WHERE SELECTed_date BETWEEN start_time AND end_time)
ORDER BY `date`;
Can I list somehow the dates of last 30 days in MySQL? Not from a table!
For example I think about like this:
SELECT date WHERE date BETWEEN SUBDATE(NOW(), INTERVAL 30 DAY) AND NOW();
Is that possible?
I hacked this together from someone else's code, but it seems to work:
SELECT DATE_FORMAT(m1, '%d %b %Y')
FROM (
SELECT SUBDATE( NOW() , INTERVAL 30 DAY) + INTERVAL m DAY AS m1
FROM (
select #rownum:=#rownum+1 as m from
(select 1 union select 2 union select 3 union select 4) t1,
(select 1 union select 2 union select 3 union select 4) t2,
(select 1 union select 2 union select 3 union select 4) t3,
(select 1 union select 2 union select 3 union select 4) t4,
(select #rownum:=-1) t0
) d1
) d2
WHERE m1 <= now()
ORDER BY m1
The original code by valex is here:
How to get a list of months between two dates in mysql
You can do this the "explicit" way. That is, generate a series of numbers and calculate the date:
select date(date_sub(now(), interval n.n day) as thedate
from (select 1 as n union all
select 2 union all
. . .
select 30
) n
I have users in db that I want to sort by hour and display count of users registered at that hour.
select
date_format(create_time, '%Y-%m-%d %h%p') as date,
count(id) as 'Number of registrations'
from users
group by 1
order by 1 desc
;
The above code will work; however, what I am trying to do is display 0's for the hours that have no user registrations. For example, if there were no registrations at 5pm, this will skip row for 5pm, which is logical. Is there a way to achieve what I am trying?
You could use a query like this:
select
date_format(t.d + INTERVAL t.h HOUR, '%Y-%m-%d %h%p') as date,
count(id) as 'Number of registrations'
from (
SELECT *
FROM
(SELECT DISTINCT DATE(create_time) d FROM users) dates,
(SELECT 0 h UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3
UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7
UNION ALL SELECT 8 UNION ALL SELECT 9 UNION ALL SELECT 10 UNION ALL SELECT 11
UNION ALL SELECT 12 UNION ALL SELECT 13 UNION ALL SELECT 14 UNION ALL SELECT 15
UNION ALL SELECT 16 UNION ALL SELECT 17 UNION ALL SELECT 18 UNION ALL SELECT 19
UNION ALL SELECT 20 UNION ALL SELECT 21 UNION ALL SELECT 22 UNION ALL SELECT 23) hours
) t LEFT JOIN users
ON DATE(users.create_time)=t.d AND HOUR(users.create_time)=t.h
group by t.d, t.h
order by t.d, t.h
Please see fiddle here.
You need to generate all possible day and hour combinations.
Assuming that you have at least one record on each day and one record for each hour, you can do:
select concat(d.theday, ' ', h.thehour) as date,
count(id) as 'Number of registrations'
from (select distinct date_format(create_time, '%Y-%m-%d') as theday from users
) d cross join
(select distinct date_format(create_time, '%h%p') as thehour from users
) h left outer join
users u
on date_format(u.create_time, '%Y-%m-%d %h%p) = concat(d.theday, ' ', h.thehour)
group by concat(d.theday, ' ', h.thehour)
order by 1 desc;