Elapsed Time Between Two Dates for specified time range - mysql

I have a MYSQL table with a TIMESTAMP column 'Start' and a TIMESTAMP column 'End'. I want to return the number of minutes between the start and the end (End is always after than Start). Usually I'd just use 'TIMESTAMPDIFF()' but this time I need to get the minutes from 9am until 22pm, of each day in that date range.
If a row has a Start '2017-01-01 07:15:00' and an End of '2017-01-02 11:30:00' - the elapsed time should be 15.5 hours (930 minutes).
I'm having trouble coming up with a decent way of doing this and my searching online hasn't found quite what I'm looking for. Can someone help me along?
Edit:
CREATE TABLE date_ranges (
Start TIMESTAMP,
End TIMESTAMP
);
INSERT INTO date_ranges VALUES('2017-01-01 07:15:00','2017-01-02 11:30:00');
I came up with this:
SELECT Start, End, TIMESTAMPDIFF(MINUTE, Start, End) AS MinutesElapsed
FROM date_ranges;
I'm missing the part where the time in minutes is calculated only in the specified time range (9am until 22pm). Any ideas?

Here you go:
SELECT t1, t2, (TIMESTAMPDIFF(MINUTE, t1, t2) - TIMESTAMPDIFF(DAY, t1, t2)*660) FROM
(SELECT CASE WHEN t1 < STR_TO_DATE(concat(date_format(t1, '%Y-%m-%d'), ' 09:00:00'), '%Y-%m-%d %h:%i:%s')
THEN STR_TO_DATE(concat(date_format(t1, '%Y-%m-%d'), ' 09:00:00'), '%Y-%m-%d %h:%i:%s')
ELSE t1
END AS t1 FROM test) test1,
(SELECT CASE WHEN t2 > STR_TO_DATE(concat(date_format(t2, '%Y-%m-%d'), ' 22:00:00'), '%Y-%m-%d %h:%i:%s')
THEN STR_TO_DATE(concat(date_format(t2, '%Y-%m-%d'), ' 22:00:00'), '%Y-%m-%d %h:%i:%s')
ELSE t2
END AS t2 FROM test) test2;
660 = number of minutes between 22:00 and 09:00 (11 hours)
Here's the SQL Fiddle.

It's not very concise, but this should give you the results you want:
select started_at,ended_at,
(case
when date(ended_at) = date(started_at)
then
timestampdiff(
minute,
greatest(started_at,concat(date(started_at),' 09:00:00')),
least(ended_at,concat(date(ended_at),' 22:00:00'))
)
else
timestampdiff(
minute,
least(greatest(started_at,concat(date(started_at),' 09:00:00')),concat(date(started_at),' 22:00:00')),
concat(date(started_at),' 22:00:00')
)
+
timestampdiff(
minute,
concat(date(ended_at),' 09:00:00'),
greatest(least(ended_at,concat(date(ended_at),' 22:00:00')),concat(date(ended_at),' 09:00:00'))
)
+ ((datediff(ended_at,started_at)-1)*780)
end) as total_minutes
from your_table;

--Generating all dates in 2017.
CREATE TABLE CALENDAR AS --Use a different table name if CALENDAR already exists
SELECT '2017-12-31 09:00:00' - INTERVAL c.number DAY AS start_datetime,'2017-12-31 22:00:00' - INTERVAL c.number DAY AS end_datetime
FROM (SELECT singles + tens + hundreds number FROM
(SELECT 0 singles
UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3
UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6
UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9
) singles JOIN
(SELECT 0 tens
UNION ALL SELECT 10 UNION ALL SELECT 20 UNION ALL SELECT 30
UNION ALL SELECT 40 UNION ALL SELECT 50 UNION ALL SELECT 60
UNION ALL SELECT 70 UNION ALL SELECT 80 UNION ALL SELECT 90
) tens JOIN
(SELECT 0 hundreds
UNION ALL SELECT 100 UNION ALL SELECT 200 UNION ALL SELECT 300
UNION ALL SELECT 400 UNION ALL SELECT 500 UNION ALL SELECT 600
UNION ALL SELECT 700 UNION ALL SELECT 800 UNION ALL SELECT 900
) hundreds
ORDER BY number DESC) c
WHERE c.number BETWEEN 0 and 364
;
--End of table creation
--Actual query begins here
SELECT D.`START`,
D.`END`,
SUM(TIMESTAMPDIFF(MINUTE,GREATEST(D.`START`,C.START_DATETIME), LEAST(D.`END`,C.END_DATETIME))) AS TOTAL_TIME
FROM CALENDAR C
LEFT JOIN DATE_RANGES D ON DATE(C.START_DATETIME) >= DATE(D.`START`)
AND DATE(C.START_DATETIME) <= DATE(D.`END`)
WHERE D.`START` IS NOT NULL
AND D.`END` IS NOT NULL
GROUP BY D.`START`,
D.`END`
;
Construct a calendar table with a dates for a specified number of years. Each date having a start time of 09:00 and an end time of 22:00.
Left join on this table to get one row per date from the date ranges table.
Sum up the differences each day to get the total time worked.
Sample Demo

Day 1 Day 2 Day 3
|--********--|--********--|--********--|
|__________________________|
The question, IMHO is to know how many minutes the first day, and how many minutes the last day, the intermediate days have 780 minutes.
I've used a subquery just to help in the intermediate calculations.
select
if(hour(t1) < 9, date(t1) + interval 9 hour , t1) as tIni1,
date(t1) + interval 22 hour as tFin1,
date(t2) + interval 9 hour as tIni2,
if(hour(t2) > 22, date(t2) + interval 22 hour, t2) as tFin2,
TIMESTAMPDIFF(day, date(t1), date(t2)) numDays
from
tdt
tIni1 and tFin1 is the period of the first day, and tIni2, tFin2 the period of the last day, obviously first and last day can be the same.
Then calculate minutes of first day + minutes of second day + 780 minutes for every intermediate day.
select numDays, tIni1, tFin1, tIni2, tFin2,
if (numDays = 0,
TIMESTAMPDIFF(minute, tIni1, tFin2),
TIMESTAMPDIFF(minute, tIni1, tFin1)
+ TIMESTAMPDIFF(minute, tIni2, tFin2)
+ (numDays - 1) * 780
) as Minutes
from (
select
if(hour(t1) < 9, date(t1) + interval 9 hour , t1) as tIni1,
date(t1) + interval 22 hour as tFin1,
date(t2) + interval 9 hour as tIni2,
if(hour(t2) > 22, date(t2) + interval 22 hour, t2) as tFin2,
TIMESTAMPDIFF(day, date(t1), date(t2)) numDays
from
tdt
) ti
;
Try it here: http://rextester.com/GDHAB78973

Related

How to supplement the result set with dates for Saturday and Sunday when they are not in the table [duplicate]

I have a table with 2 columns, date and score. It has at most 30 entries, for each of the last 30 days one.
date score
-----------------
1.8.2010 19
2.8.2010 21
4.8.2010 14
7.8.2010 10
10.8.2010 14
My problem is that some dates are missing - I want to see:
date score
-----------------
1.8.2010 19
2.8.2010 21
3.8.2010 0
4.8.2010 14
5.8.2010 0
6.8.2010 0
7.8.2010 10
...
What I need from the single query is to get: 19,21,9,14,0,0,10,0,0,14... That means that the missing dates are filled with 0.
I know how to get all the values and in server side language iterating through dates and missing the blanks. But is this possible to do in mysql, so that I sort the result by date and get the missing pieces.
EDIT: In this table there is another column named UserID, so I have 30.000 users and some of them have the score in this table. I delete the dates every day if date < 30 days ago because I need last 30 days score for each user. The reason is I am making a graph of the user activity over the last 30 days and to plot a chart I need the 30 values separated by comma. So I can say in query get me the USERID=10203 activity and the query would get me the 30 scores, one for each of the last 30 days. I hope I am more clear now.
MySQL doesn't have recursive functionality, so you're left with using the NUMBERS table trick -
Create a table that only holds incrementing numbers - easy to do using an auto_increment:
DROP TABLE IF EXISTS `example`.`numbers`;
CREATE TABLE `example`.`numbers` (
`id` int(10) unsigned NOT NULL auto_increment,
PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
Populate the table using:
INSERT INTO `example`.`numbers`
( `id` )
VALUES
( NULL )
...for as many values as you need.
Use DATE_ADD to construct a list of dates, increasing the days based on the NUMBERS.id value. Replace "2010-06-06" and "2010-06-14" with your respective start and end dates (but use the same format, YYYY-MM-DD) -
SELECT `x`.*
FROM (SELECT DATE_ADD('2010-06-06', INTERVAL `n`.`id` - 1 DAY)
FROM `numbers` `n`
WHERE DATE_ADD('2010-06-06', INTERVAL `n`.`id` -1 DAY) <= '2010-06-14' ) x
LEFT JOIN onto your table of data based on the time portion:
SELECT `x`.`ts` AS `timestamp`,
COALESCE(`y`.`score`, 0) AS `cnt`
FROM (SELECT DATE_FORMAT(DATE_ADD('2010-06-06', INTERVAL `n`.`id` - 1 DAY), '%m/%d/%Y') AS `ts`
FROM `numbers` `n`
WHERE DATE_ADD('2010-06-06', INTERVAL `n`.`id` - 1 DAY) <= '2010-06-14') x
LEFT JOIN TABLE `y` ON STR_TO_DATE(`y`.`date`, '%d.%m.%Y') = `x`.`ts`
If you want to maintain the date format, use the DATE_FORMAT function:
DATE_FORMAT(`x`.`ts`, '%d.%m.%Y') AS `timestamp`
I'm not a fan of the other answers, requiring tables to be created and such. This query does it efficiently without helper tables.
SELECT
IF(score IS NULL, 0, score) AS score,
b.Days AS date
FROM
(SELECT a.Days
FROM (
SELECT curdate() - INTERVAL (a.a + (10 * b.a) + (100 * c.a)) DAY AS Days
FROM (SELECT 0 AS a UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) AS a
CROSS JOIN (SELECT 0 AS a UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) AS b
CROSS JOIN (SELECT 0 AS a UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) AS c
) a
WHERE a.Days >= curdate() - INTERVAL 30 DAY) b
LEFT JOIN your_table
ON date = b.Days
ORDER BY b.Days;
So lets dissect this.
SELECT
IF(score IS NULL, 0, score) AS score,
b.Days AS date
The if will detect days that had no score and set them to 0. b.Days is the configured amount of days you chose to get from the current date, up to 1000.
(SELECT a.Days
FROM (
SELECT curdate() - INTERVAL (a.a + (10 * b.a) + (100 * c.a)) DAY AS Days
FROM (SELECT 0 AS a UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) AS a
CROSS JOIN (SELECT 0 AS a UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) AS b
CROSS JOIN (SELECT 0 AS a UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) AS c
) a
WHERE a.Days >= curdate() - INTERVAL 30 DAY) b
This subquery is something I saw on stackoverflow. It efficiently generates a list of the past 1000 days from the current date. The interval (currently 30) in the WHERE clause at the end determines which days are returned; the maximum is 1000. This query could be easily modified to return 100s of years worth of dates, but 1000 should be good for most things.
LEFT JOIN your_table
ON date = b.Days
ORDER BY b.Days;
This is the part that brings your table that contains the score into it. You compare to the selected date range from the date generator query to be able to fill in 0s where needed (the score will be set to NULL initially, because it is a LEFT JOIN; this is fixed in the select statement). I also order it by the dates, just because. This is preference, you could also order by score.
Before the ORDER BY you could easily join with your table about user info you mentioned with your edit, to add that last requirement.
I hope this version of the query helps someone. Thanks for reading.
Time went by since this question was asked. MySQL 8.0 was released in 2018 and added support for recursive common table expressions, which provide an elegant, state-of-the-art solution to this question.
The following query can be used to generate a list of dates, say for the first 15 days of August 2010:
with recursive all_dates(dt) as (
-- anchor
select '2010-08-01' dt
union all
-- recursion with stop condition
select dt + interval 1 day from all_dates where dt < '2010-08-15'
)
select * from all_dates order by dt
You can then left join this resultset with your table to generate the expected output:
with recursive all_dates(dt) as (
select '2010-08-01' dt
union all
select dt + interval 1 day from all_dates where dt < '2010-08-15'
)
select d.dt date, coalesce(t.score, 0) score
from all_dates d
left join mytable t on t.date = d.dt
order by d.dt
Demo on DB Fiddle:
date | score
:--------- | ----:
2010-08-01 | 19
2010-08-02 | 21
2010-08-03 | 0
2010-08-04 | 14
2010-08-05 | 0
2010-08-06 | 0
2010-08-07 | 10
2010-08-08 | 0
2010-08-09 | 0
2010-08-10 | 14
2010-08-11 | 0
2010-08-12 | 0
2010-08-13 | 0
2010-08-14 | 0
2010-08-15 | 0
Note that it is very easy to adapt the recursive CTE for other intervals or periods. As an example, say we want a row every 15 minutes from 4 AM to 8 AM on August 1st, 2010 ; we can do :
with recursive all_dates(dt) as (
select '2010-08-01 04:00:00' dt
union all
select dt + interval 15 minute from all_dates where dt < '2010-08-01 08:00:00'
)
...
You can accomplish this by using a Calendar Table. That's a table which you create once and fill with a date range (e.g. one dataset for each day 2000-2050; that depends on your data). Then you can make an outer join of your table against the calendar table. If a date is missing in your table, you return 0 for the score.
Michael Conard answer is great but I needed intervals of 15 minutes where the time must always start at the top of every 15th minute:
SELECT a.Days
FROM (
SELECT FROM_UNIXTIME( FLOOR( UNIX_TIMESTAMP() / (15 * 60) ) * (15 * 60)) - INTERVAL 15 * (a.a + (10 * b.a) + (100 * c.a)) MINUTE AS Days
FROM (SELECT 0 AS a UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) AS a
CROSS JOIN (SELECT 0 AS a UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) AS b
CROSS JOIN (SELECT 0 AS a UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) AS c
) a
WHERE a.Days >= curdate() - INTERVAL 30 DAY
This will set the current time to the previous round 15th minute:
FROM_UNIXTIME( FLOOR( UNIX_TIMESTAMP() / (15 * 60) ) * (15 * 60))
And this will remove time with a 15 minute step:
- INTERVAL 15 * (a.a + (10 * b.a) + (100 * c.a)) MINUTE
If there's a simpler way to do it, please let me know.
you can user direct from start date up to today with insertion
with recursive all_dates(dt) as (
-- anchor
select '2021-01-01' dt
union all
-- recursion with stop condition
INSERT IGNORE INTO mytable (date,score) VALUES (dt + interval 1 day ,0 ) where dt + interval 1 day <= curdate()
)
select * from all_dates

Mysql command to.return zero if date isnt found [duplicate]

I have a table with 2 columns, date and score. It has at most 30 entries, for each of the last 30 days one.
date score
-----------------
1.8.2010 19
2.8.2010 21
4.8.2010 14
7.8.2010 10
10.8.2010 14
My problem is that some dates are missing - I want to see:
date score
-----------------
1.8.2010 19
2.8.2010 21
3.8.2010 0
4.8.2010 14
5.8.2010 0
6.8.2010 0
7.8.2010 10
...
What I need from the single query is to get: 19,21,9,14,0,0,10,0,0,14... That means that the missing dates are filled with 0.
I know how to get all the values and in server side language iterating through dates and missing the blanks. But is this possible to do in mysql, so that I sort the result by date and get the missing pieces.
EDIT: In this table there is another column named UserID, so I have 30.000 users and some of them have the score in this table. I delete the dates every day if date < 30 days ago because I need last 30 days score for each user. The reason is I am making a graph of the user activity over the last 30 days and to plot a chart I need the 30 values separated by comma. So I can say in query get me the USERID=10203 activity and the query would get me the 30 scores, one for each of the last 30 days. I hope I am more clear now.
MySQL doesn't have recursive functionality, so you're left with using the NUMBERS table trick -
Create a table that only holds incrementing numbers - easy to do using an auto_increment:
DROP TABLE IF EXISTS `example`.`numbers`;
CREATE TABLE `example`.`numbers` (
`id` int(10) unsigned NOT NULL auto_increment,
PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
Populate the table using:
INSERT INTO `example`.`numbers`
( `id` )
VALUES
( NULL )
...for as many values as you need.
Use DATE_ADD to construct a list of dates, increasing the days based on the NUMBERS.id value. Replace "2010-06-06" and "2010-06-14" with your respective start and end dates (but use the same format, YYYY-MM-DD) -
SELECT `x`.*
FROM (SELECT DATE_ADD('2010-06-06', INTERVAL `n`.`id` - 1 DAY)
FROM `numbers` `n`
WHERE DATE_ADD('2010-06-06', INTERVAL `n`.`id` -1 DAY) <= '2010-06-14' ) x
LEFT JOIN onto your table of data based on the time portion:
SELECT `x`.`ts` AS `timestamp`,
COALESCE(`y`.`score`, 0) AS `cnt`
FROM (SELECT DATE_FORMAT(DATE_ADD('2010-06-06', INTERVAL `n`.`id` - 1 DAY), '%m/%d/%Y') AS `ts`
FROM `numbers` `n`
WHERE DATE_ADD('2010-06-06', INTERVAL `n`.`id` - 1 DAY) <= '2010-06-14') x
LEFT JOIN TABLE `y` ON STR_TO_DATE(`y`.`date`, '%d.%m.%Y') = `x`.`ts`
If you want to maintain the date format, use the DATE_FORMAT function:
DATE_FORMAT(`x`.`ts`, '%d.%m.%Y') AS `timestamp`
I'm not a fan of the other answers, requiring tables to be created and such. This query does it efficiently without helper tables.
SELECT
IF(score IS NULL, 0, score) AS score,
b.Days AS date
FROM
(SELECT a.Days
FROM (
SELECT curdate() - INTERVAL (a.a + (10 * b.a) + (100 * c.a)) DAY AS Days
FROM (SELECT 0 AS a UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) AS a
CROSS JOIN (SELECT 0 AS a UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) AS b
CROSS JOIN (SELECT 0 AS a UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) AS c
) a
WHERE a.Days >= curdate() - INTERVAL 30 DAY) b
LEFT JOIN your_table
ON date = b.Days
ORDER BY b.Days;
So lets dissect this.
SELECT
IF(score IS NULL, 0, score) AS score,
b.Days AS date
The if will detect days that had no score and set them to 0. b.Days is the configured amount of days you chose to get from the current date, up to 1000.
(SELECT a.Days
FROM (
SELECT curdate() - INTERVAL (a.a + (10 * b.a) + (100 * c.a)) DAY AS Days
FROM (SELECT 0 AS a UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) AS a
CROSS JOIN (SELECT 0 AS a UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) AS b
CROSS JOIN (SELECT 0 AS a UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) AS c
) a
WHERE a.Days >= curdate() - INTERVAL 30 DAY) b
This subquery is something I saw on stackoverflow. It efficiently generates a list of the past 1000 days from the current date. The interval (currently 30) in the WHERE clause at the end determines which days are returned; the maximum is 1000. This query could be easily modified to return 100s of years worth of dates, but 1000 should be good for most things.
LEFT JOIN your_table
ON date = b.Days
ORDER BY b.Days;
This is the part that brings your table that contains the score into it. You compare to the selected date range from the date generator query to be able to fill in 0s where needed (the score will be set to NULL initially, because it is a LEFT JOIN; this is fixed in the select statement). I also order it by the dates, just because. This is preference, you could also order by score.
Before the ORDER BY you could easily join with your table about user info you mentioned with your edit, to add that last requirement.
I hope this version of the query helps someone. Thanks for reading.
Time went by since this question was asked. MySQL 8.0 was released in 2018 and added support for recursive common table expressions, which provide an elegant, state-of-the-art solution to this question.
The following query can be used to generate a list of dates, say for the first 15 days of August 2010:
with recursive all_dates(dt) as (
-- anchor
select '2010-08-01' dt
union all
-- recursion with stop condition
select dt + interval 1 day from all_dates where dt < '2010-08-15'
)
select * from all_dates order by dt
You can then left join this resultset with your table to generate the expected output:
with recursive all_dates(dt) as (
select '2010-08-01' dt
union all
select dt + interval 1 day from all_dates where dt < '2010-08-15'
)
select d.dt date, coalesce(t.score, 0) score
from all_dates d
left join mytable t on t.date = d.dt
order by d.dt
Demo on DB Fiddle:
date | score
:--------- | ----:
2010-08-01 | 19
2010-08-02 | 21
2010-08-03 | 0
2010-08-04 | 14
2010-08-05 | 0
2010-08-06 | 0
2010-08-07 | 10
2010-08-08 | 0
2010-08-09 | 0
2010-08-10 | 14
2010-08-11 | 0
2010-08-12 | 0
2010-08-13 | 0
2010-08-14 | 0
2010-08-15 | 0
Note that it is very easy to adapt the recursive CTE for other intervals or periods. As an example, say we want a row every 15 minutes from 4 AM to 8 AM on August 1st, 2010 ; we can do :
with recursive all_dates(dt) as (
select '2010-08-01 04:00:00' dt
union all
select dt + interval 15 minute from all_dates where dt < '2010-08-01 08:00:00'
)
...
You can accomplish this by using a Calendar Table. That's a table which you create once and fill with a date range (e.g. one dataset for each day 2000-2050; that depends on your data). Then you can make an outer join of your table against the calendar table. If a date is missing in your table, you return 0 for the score.
Michael Conard answer is great but I needed intervals of 15 minutes where the time must always start at the top of every 15th minute:
SELECT a.Days
FROM (
SELECT FROM_UNIXTIME( FLOOR( UNIX_TIMESTAMP() / (15 * 60) ) * (15 * 60)) - INTERVAL 15 * (a.a + (10 * b.a) + (100 * c.a)) MINUTE AS Days
FROM (SELECT 0 AS a UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) AS a
CROSS JOIN (SELECT 0 AS a UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) AS b
CROSS JOIN (SELECT 0 AS a UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) AS c
) a
WHERE a.Days >= curdate() - INTERVAL 30 DAY
This will set the current time to the previous round 15th minute:
FROM_UNIXTIME( FLOOR( UNIX_TIMESTAMP() / (15 * 60) ) * (15 * 60))
And this will remove time with a 15 minute step:
- INTERVAL 15 * (a.a + (10 * b.a) + (100 * c.a)) MINUTE
If there's a simpler way to do it, please let me know.
you can user direct from start date up to today with insertion
with recursive all_dates(dt) as (
-- anchor
select '2021-01-01' dt
union all
-- recursion with stop condition
INSERT IGNORE INTO mytable (date,score) VALUES (dt + interval 1 day ,0 ) where dt + interval 1 day <= curdate()
)
select * from all_dates

MySQL list number of results per date including days with no results [duplicate]

I have a table with 2 columns, date and score. It has at most 30 entries, for each of the last 30 days one.
date score
-----------------
1.8.2010 19
2.8.2010 21
4.8.2010 14
7.8.2010 10
10.8.2010 14
My problem is that some dates are missing - I want to see:
date score
-----------------
1.8.2010 19
2.8.2010 21
3.8.2010 0
4.8.2010 14
5.8.2010 0
6.8.2010 0
7.8.2010 10
...
What I need from the single query is to get: 19,21,9,14,0,0,10,0,0,14... That means that the missing dates are filled with 0.
I know how to get all the values and in server side language iterating through dates and missing the blanks. But is this possible to do in mysql, so that I sort the result by date and get the missing pieces.
EDIT: In this table there is another column named UserID, so I have 30.000 users and some of them have the score in this table. I delete the dates every day if date < 30 days ago because I need last 30 days score for each user. The reason is I am making a graph of the user activity over the last 30 days and to plot a chart I need the 30 values separated by comma. So I can say in query get me the USERID=10203 activity and the query would get me the 30 scores, one for each of the last 30 days. I hope I am more clear now.
MySQL doesn't have recursive functionality, so you're left with using the NUMBERS table trick -
Create a table that only holds incrementing numbers - easy to do using an auto_increment:
DROP TABLE IF EXISTS `example`.`numbers`;
CREATE TABLE `example`.`numbers` (
`id` int(10) unsigned NOT NULL auto_increment,
PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
Populate the table using:
INSERT INTO `example`.`numbers`
( `id` )
VALUES
( NULL )
...for as many values as you need.
Use DATE_ADD to construct a list of dates, increasing the days based on the NUMBERS.id value. Replace "2010-06-06" and "2010-06-14" with your respective start and end dates (but use the same format, YYYY-MM-DD) -
SELECT `x`.*
FROM (SELECT DATE_ADD('2010-06-06', INTERVAL `n`.`id` - 1 DAY)
FROM `numbers` `n`
WHERE DATE_ADD('2010-06-06', INTERVAL `n`.`id` -1 DAY) <= '2010-06-14' ) x
LEFT JOIN onto your table of data based on the time portion:
SELECT `x`.`ts` AS `timestamp`,
COALESCE(`y`.`score`, 0) AS `cnt`
FROM (SELECT DATE_FORMAT(DATE_ADD('2010-06-06', INTERVAL `n`.`id` - 1 DAY), '%m/%d/%Y') AS `ts`
FROM `numbers` `n`
WHERE DATE_ADD('2010-06-06', INTERVAL `n`.`id` - 1 DAY) <= '2010-06-14') x
LEFT JOIN TABLE `y` ON STR_TO_DATE(`y`.`date`, '%d.%m.%Y') = `x`.`ts`
If you want to maintain the date format, use the DATE_FORMAT function:
DATE_FORMAT(`x`.`ts`, '%d.%m.%Y') AS `timestamp`
I'm not a fan of the other answers, requiring tables to be created and such. This query does it efficiently without helper tables.
SELECT
IF(score IS NULL, 0, score) AS score,
b.Days AS date
FROM
(SELECT a.Days
FROM (
SELECT curdate() - INTERVAL (a.a + (10 * b.a) + (100 * c.a)) DAY AS Days
FROM (SELECT 0 AS a UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) AS a
CROSS JOIN (SELECT 0 AS a UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) AS b
CROSS JOIN (SELECT 0 AS a UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) AS c
) a
WHERE a.Days >= curdate() - INTERVAL 30 DAY) b
LEFT JOIN your_table
ON date = b.Days
ORDER BY b.Days;
So lets dissect this.
SELECT
IF(score IS NULL, 0, score) AS score,
b.Days AS date
The if will detect days that had no score and set them to 0. b.Days is the configured amount of days you chose to get from the current date, up to 1000.
(SELECT a.Days
FROM (
SELECT curdate() - INTERVAL (a.a + (10 * b.a) + (100 * c.a)) DAY AS Days
FROM (SELECT 0 AS a UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) AS a
CROSS JOIN (SELECT 0 AS a UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) AS b
CROSS JOIN (SELECT 0 AS a UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) AS c
) a
WHERE a.Days >= curdate() - INTERVAL 30 DAY) b
This subquery is something I saw on stackoverflow. It efficiently generates a list of the past 1000 days from the current date. The interval (currently 30) in the WHERE clause at the end determines which days are returned; the maximum is 1000. This query could be easily modified to return 100s of years worth of dates, but 1000 should be good for most things.
LEFT JOIN your_table
ON date = b.Days
ORDER BY b.Days;
This is the part that brings your table that contains the score into it. You compare to the selected date range from the date generator query to be able to fill in 0s where needed (the score will be set to NULL initially, because it is a LEFT JOIN; this is fixed in the select statement). I also order it by the dates, just because. This is preference, you could also order by score.
Before the ORDER BY you could easily join with your table about user info you mentioned with your edit, to add that last requirement.
I hope this version of the query helps someone. Thanks for reading.
Time went by since this question was asked. MySQL 8.0 was released in 2018 and added support for recursive common table expressions, which provide an elegant, state-of-the-art solution to this question.
The following query can be used to generate a list of dates, say for the first 15 days of August 2010:
with recursive all_dates(dt) as (
-- anchor
select '2010-08-01' dt
union all
-- recursion with stop condition
select dt + interval 1 day from all_dates where dt < '2010-08-15'
)
select * from all_dates order by dt
You can then left join this resultset with your table to generate the expected output:
with recursive all_dates(dt) as (
select '2010-08-01' dt
union all
select dt + interval 1 day from all_dates where dt < '2010-08-15'
)
select d.dt date, coalesce(t.score, 0) score
from all_dates d
left join mytable t on t.date = d.dt
order by d.dt
Demo on DB Fiddle:
date | score
:--------- | ----:
2010-08-01 | 19
2010-08-02 | 21
2010-08-03 | 0
2010-08-04 | 14
2010-08-05 | 0
2010-08-06 | 0
2010-08-07 | 10
2010-08-08 | 0
2010-08-09 | 0
2010-08-10 | 14
2010-08-11 | 0
2010-08-12 | 0
2010-08-13 | 0
2010-08-14 | 0
2010-08-15 | 0
Note that it is very easy to adapt the recursive CTE for other intervals or periods. As an example, say we want a row every 15 minutes from 4 AM to 8 AM on August 1st, 2010 ; we can do :
with recursive all_dates(dt) as (
select '2010-08-01 04:00:00' dt
union all
select dt + interval 15 minute from all_dates where dt < '2010-08-01 08:00:00'
)
...
You can accomplish this by using a Calendar Table. That's a table which you create once and fill with a date range (e.g. one dataset for each day 2000-2050; that depends on your data). Then you can make an outer join of your table against the calendar table. If a date is missing in your table, you return 0 for the score.
Michael Conard answer is great but I needed intervals of 15 minutes where the time must always start at the top of every 15th minute:
SELECT a.Days
FROM (
SELECT FROM_UNIXTIME( FLOOR( UNIX_TIMESTAMP() / (15 * 60) ) * (15 * 60)) - INTERVAL 15 * (a.a + (10 * b.a) + (100 * c.a)) MINUTE AS Days
FROM (SELECT 0 AS a UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) AS a
CROSS JOIN (SELECT 0 AS a UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) AS b
CROSS JOIN (SELECT 0 AS a UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) AS c
) a
WHERE a.Days >= curdate() - INTERVAL 30 DAY
This will set the current time to the previous round 15th minute:
FROM_UNIXTIME( FLOOR( UNIX_TIMESTAMP() / (15 * 60) ) * (15 * 60))
And this will remove time with a 15 minute step:
- INTERVAL 15 * (a.a + (10 * b.a) + (100 * c.a)) MINUTE
If there's a simpler way to do it, please let me know.
you can user direct from start date up to today with insertion
with recursive all_dates(dt) as (
-- anchor
select '2021-01-01' dt
union all
-- recursion with stop condition
INSERT IGNORE INTO mytable (date,score) VALUES (dt + interval 1 day ,0 ) where dt + interval 1 day <= curdate()
)
select * from all_dates

Hour Wise data in mySql

I have the table with following fields
Createdon(datetime)
Amount(double)
I need to find the sum of amounts for next 24 hours of the given date. If there are no results then the sum should be zero.
e.g
duration sum
0000-0001 25.43
0001-0002 36.85
0002-0003 0
.
.
.
.
0022-0023 38.56
Can you please help me creating a query to find the required solution
The key to your query is the ability to take any datetime value and truncate it to the nearest preceding hour. You can do that with this expression:
DATE_FORMAT(Createdon, '%Y-%m-%d %H:00')
Given, for example, 2015-04-21 14:22:05, this gives back 2015-04-21 14:00:00.
Then you use that in GROUP BY
SELECT DATE_FORMAT(Createdon, '%Y-%m-%d %H:00') Createdhour,
SUM(Amount) sum
FROM theTable
GROUP BY DATE_FORMAT(Createdon, '%Y-%m-%d %H:00')
ORDER BY DATE_FORMAT(Createdon, '%Y-%m-%d %H:00')
Finally, I think you wanted one day's worth of results. You need to add a WHERE clause to get that. The one shown here will take yesterday's results -- that is, all results from [midnight yesterday -- midnight today).
SELECT DATE_FORMAT(Createdon, '%Y-%m-%d %H:00') Createdhour,
SUM(Amount) sum
FROM theTable
WHERE CreatedOn >= DATE(NOW()) - INTERVAL 1 DAY
AND CreatedOn < DATE(NOW())
GROUP BY DATE_FORMAT(Createdon, '%Y-%m-%d %H:00')
ORDER BY DATE_FORMAT(Createdon, '%Y-%m-%d %H:00')
This is explained in greater detail at http://www.plumislandmedia.net/mysql/sql-reporting-time-intervals/
To include all hours of the day, you will need an independent source of distinct DATETIME items.
Here's a query that will do such a thing.
SELECT mintime + INTERVAL seq.seq HOUR AS CreatedHour
FROM (
SELECT DATE(NOW()) - INTERVAL 1 DAY AS mintime,
DATE(NOW()) AS maxtime
) AS minmax
JOIN seq_0_to_23 AS seq
ON seq.seq < TIMESTAMPDIFF(HOUR,mintime,maxtime)
You then need to use LEFT JOIN to pick up your data.
SELECT a.Createdhour,
SUM(Amount) sum
FROM (
SELECT mintime + INTERVAL seq.seq HOUR AS CreatedHour
FROM (
SELECT DATE(NOW()) - INTERVAL 1 DAY AS mintime,
DATE(NOW()) AS maxtime
) AS minmax
JOIN seq_0_to_23 AS seq
ON seq.seq < TIMESTAMPDIFF(HOUR,mintime,maxtime)
) a
LEFT JOIN theTable t
ON a.CreatedHour = DATE_FORMAT(t.Createdon, '%Y-%m-%d %H:00')
GROUP BY DATE_FORMAT(t.Createdon, '%Y-%m-%d %H:00')
ORDER BY DATE_FORMAT(t.Createdon, '%Y-%m-%d %H:00')
Finally, you need to somehow get that table seq_0_to_23. If you're running MariaDB, it's built in. If not...
CREATE TABLE seq_0_to_23 AS
SELECT 0 AS seq
UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3
UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6
UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9
UNION ALL SELECT 10 UNION ALL SELECT 11 UNION ALL SELECT 12
UNION ALL SELECT 13 UNION ALL SELECT 14 UNION ALL SELECT 15
UNION ALL SELECT 16 UNION ALL SELECT 17 UNION ALL SELECT 18
UNION ALL SELECT 19 UNION ALL SELECT 20 UNION ALL SELECT 21
UNION ALL SELECT 22 UNION ALL SELECT 23
This is written up in more general form at http://www.plumislandmedia.net/mysql/filling-missing-data-sequences-cardinal-integers/

MySQL showing missing dates with group by [duplicate]

I have a table with 2 columns, date and score. It has at most 30 entries, for each of the last 30 days one.
date score
-----------------
1.8.2010 19
2.8.2010 21
4.8.2010 14
7.8.2010 10
10.8.2010 14
My problem is that some dates are missing - I want to see:
date score
-----------------
1.8.2010 19
2.8.2010 21
3.8.2010 0
4.8.2010 14
5.8.2010 0
6.8.2010 0
7.8.2010 10
...
What I need from the single query is to get: 19,21,9,14,0,0,10,0,0,14... That means that the missing dates are filled with 0.
I know how to get all the values and in server side language iterating through dates and missing the blanks. But is this possible to do in mysql, so that I sort the result by date and get the missing pieces.
EDIT: In this table there is another column named UserID, so I have 30.000 users and some of them have the score in this table. I delete the dates every day if date < 30 days ago because I need last 30 days score for each user. The reason is I am making a graph of the user activity over the last 30 days and to plot a chart I need the 30 values separated by comma. So I can say in query get me the USERID=10203 activity and the query would get me the 30 scores, one for each of the last 30 days. I hope I am more clear now.
MySQL doesn't have recursive functionality, so you're left with using the NUMBERS table trick -
Create a table that only holds incrementing numbers - easy to do using an auto_increment:
DROP TABLE IF EXISTS `example`.`numbers`;
CREATE TABLE `example`.`numbers` (
`id` int(10) unsigned NOT NULL auto_increment,
PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
Populate the table using:
INSERT INTO `example`.`numbers`
( `id` )
VALUES
( NULL )
...for as many values as you need.
Use DATE_ADD to construct a list of dates, increasing the days based on the NUMBERS.id value. Replace "2010-06-06" and "2010-06-14" with your respective start and end dates (but use the same format, YYYY-MM-DD) -
SELECT `x`.*
FROM (SELECT DATE_ADD('2010-06-06', INTERVAL `n`.`id` - 1 DAY)
FROM `numbers` `n`
WHERE DATE_ADD('2010-06-06', INTERVAL `n`.`id` -1 DAY) <= '2010-06-14' ) x
LEFT JOIN onto your table of data based on the time portion:
SELECT `x`.`ts` AS `timestamp`,
COALESCE(`y`.`score`, 0) AS `cnt`
FROM (SELECT DATE_FORMAT(DATE_ADD('2010-06-06', INTERVAL `n`.`id` - 1 DAY), '%m/%d/%Y') AS `ts`
FROM `numbers` `n`
WHERE DATE_ADD('2010-06-06', INTERVAL `n`.`id` - 1 DAY) <= '2010-06-14') x
LEFT JOIN TABLE `y` ON STR_TO_DATE(`y`.`date`, '%d.%m.%Y') = `x`.`ts`
If you want to maintain the date format, use the DATE_FORMAT function:
DATE_FORMAT(`x`.`ts`, '%d.%m.%Y') AS `timestamp`
I'm not a fan of the other answers, requiring tables to be created and such. This query does it efficiently without helper tables.
SELECT
IF(score IS NULL, 0, score) AS score,
b.Days AS date
FROM
(SELECT a.Days
FROM (
SELECT curdate() - INTERVAL (a.a + (10 * b.a) + (100 * c.a)) DAY AS Days
FROM (SELECT 0 AS a UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) AS a
CROSS JOIN (SELECT 0 AS a UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) AS b
CROSS JOIN (SELECT 0 AS a UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) AS c
) a
WHERE a.Days >= curdate() - INTERVAL 30 DAY) b
LEFT JOIN your_table
ON date = b.Days
ORDER BY b.Days;
So lets dissect this.
SELECT
IF(score IS NULL, 0, score) AS score,
b.Days AS date
The if will detect days that had no score and set them to 0. b.Days is the configured amount of days you chose to get from the current date, up to 1000.
(SELECT a.Days
FROM (
SELECT curdate() - INTERVAL (a.a + (10 * b.a) + (100 * c.a)) DAY AS Days
FROM (SELECT 0 AS a UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) AS a
CROSS JOIN (SELECT 0 AS a UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) AS b
CROSS JOIN (SELECT 0 AS a UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) AS c
) a
WHERE a.Days >= curdate() - INTERVAL 30 DAY) b
This subquery is something I saw on stackoverflow. It efficiently generates a list of the past 1000 days from the current date. The interval (currently 30) in the WHERE clause at the end determines which days are returned; the maximum is 1000. This query could be easily modified to return 100s of years worth of dates, but 1000 should be good for most things.
LEFT JOIN your_table
ON date = b.Days
ORDER BY b.Days;
This is the part that brings your table that contains the score into it. You compare to the selected date range from the date generator query to be able to fill in 0s where needed (the score will be set to NULL initially, because it is a LEFT JOIN; this is fixed in the select statement). I also order it by the dates, just because. This is preference, you could also order by score.
Before the ORDER BY you could easily join with your table about user info you mentioned with your edit, to add that last requirement.
I hope this version of the query helps someone. Thanks for reading.
Time went by since this question was asked. MySQL 8.0 was released in 2018 and added support for recursive common table expressions, which provide an elegant, state-of-the-art solution to this question.
The following query can be used to generate a list of dates, say for the first 15 days of August 2010:
with recursive all_dates(dt) as (
-- anchor
select '2010-08-01' dt
union all
-- recursion with stop condition
select dt + interval 1 day from all_dates where dt < '2010-08-15'
)
select * from all_dates order by dt
You can then left join this resultset with your table to generate the expected output:
with recursive all_dates(dt) as (
select '2010-08-01' dt
union all
select dt + interval 1 day from all_dates where dt < '2010-08-15'
)
select d.dt date, coalesce(t.score, 0) score
from all_dates d
left join mytable t on t.date = d.dt
order by d.dt
Demo on DB Fiddle:
date | score
:--------- | ----:
2010-08-01 | 19
2010-08-02 | 21
2010-08-03 | 0
2010-08-04 | 14
2010-08-05 | 0
2010-08-06 | 0
2010-08-07 | 10
2010-08-08 | 0
2010-08-09 | 0
2010-08-10 | 14
2010-08-11 | 0
2010-08-12 | 0
2010-08-13 | 0
2010-08-14 | 0
2010-08-15 | 0
Note that it is very easy to adapt the recursive CTE for other intervals or periods. As an example, say we want a row every 15 minutes from 4 AM to 8 AM on August 1st, 2010 ; we can do :
with recursive all_dates(dt) as (
select '2010-08-01 04:00:00' dt
union all
select dt + interval 15 minute from all_dates where dt < '2010-08-01 08:00:00'
)
...
You can accomplish this by using a Calendar Table. That's a table which you create once and fill with a date range (e.g. one dataset for each day 2000-2050; that depends on your data). Then you can make an outer join of your table against the calendar table. If a date is missing in your table, you return 0 for the score.
Michael Conard answer is great but I needed intervals of 15 minutes where the time must always start at the top of every 15th minute:
SELECT a.Days
FROM (
SELECT FROM_UNIXTIME( FLOOR( UNIX_TIMESTAMP() / (15 * 60) ) * (15 * 60)) - INTERVAL 15 * (a.a + (10 * b.a) + (100 * c.a)) MINUTE AS Days
FROM (SELECT 0 AS a UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) AS a
CROSS JOIN (SELECT 0 AS a UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) AS b
CROSS JOIN (SELECT 0 AS a UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) AS c
) a
WHERE a.Days >= curdate() - INTERVAL 30 DAY
This will set the current time to the previous round 15th minute:
FROM_UNIXTIME( FLOOR( UNIX_TIMESTAMP() / (15 * 60) ) * (15 * 60))
And this will remove time with a 15 minute step:
- INTERVAL 15 * (a.a + (10 * b.a) + (100 * c.a)) MINUTE
If there's a simpler way to do it, please let me know.
you can user direct from start date up to today with insertion
with recursive all_dates(dt) as (
-- anchor
select '2021-01-01' dt
union all
-- recursion with stop condition
INSERT IGNORE INTO mytable (date,score) VALUES (dt + interval 1 day ,0 ) where dt + interval 1 day <= curdate()
)
select * from all_dates