I want to get all the Mondays in the month of MAY 2015
(using mysql query)
OUTPUT:
MON
04
11
18
25
select row+1 as Mon from
( SELECT #row := #row + 1 as row FROM
(select 0 union all select 1 union all select 3 union all select 4 union all select 5 union all select 6) t1,
(select 0 union all select 1 union all select 3 union all select 4 union all select 5 union all select 6) t2,
(SELECT #row:=-1) t3 limit 31 ) b where
DATE_ADD('2015-05-01', INTERVAL ROW DAY) between '2015-05-01' and '2015-05-31' and DAYOFWEEK(DATE_ADD('2015-05-01', INTERVAL ROW DAY))=2;
Output
+------------+
| Mon |
+------------+
| 4 |
| 11 |
| 18 |
| 25 |
+------------+
Tweaking a bit this query
For reference, here's another solution - note that the last entry may be null, can be changed to another value if necessary, or wrap in a sub-select and filter on not null.
SET #date='2015-05-01';
SET #offset=7 - WeekDay(#date);
SELECT DAY(DATE_ADD(#date,INTERVAL #offset DAY)) AS 'MON'
UNION SELECT DAY(DATE_ADD(#date,INTERVAL #offset+7 DAY))
UNION SELECT DAY(DATE_ADD(#date,INTERVAL #offset+14 DAY))
UNION SELECT DAY(DATE_ADD(#date,INTERVAL #offset+21 DAY))
UNION DISTINCT SELECT IF(DAY(DATE_ADD(#date,INTERVAL #offset+28 DAY))>21,
DAY(DATE_ADD(#date,INTERVAL #offset+28 DAY)),
DAY(DATE_ADD(#date,INTERVAL #offset+21 DAY)))
;
SQL Fiddle: http://sqlfiddle.com/#!9/fa4ce/4
This query returns the two digit day value of the Mondays in a month.
This requires the "month" as a date of the first day of the month, as a value in the SELECT list of the first inline view (d0). (This inline view query could be tweaked to handle any date value within a month as the specification for a month.)
SELECT DATE_FORMAT(d0.dt + INTERVAL d1.i*6+d2.i DAY,'%d') AS dd
-- , d0.dt + INTERVAL d1.i*6+d2.i DAY AS dt
FROM ( SELECT '2015-05-01' + INTERVAL 0 DAY AS dt
) d0
CROSS
JOIN ( SELECT 0 AS i UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5
) d1
CROSS
JOIN ( SELECT 0 AS i UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5
) d2
WHERE d0.dt + INTERVAL d1.i*6+d2.i DAY < d0.dt + INTERVAL 1 MONTH
AND NOT WEEKDAY(d0.dt + INTERVAL d1.i*6+d2.i DAY)
ORDER BY 1
NOTE: This requires the month to be specified only once, in the first inline view (d0). Everything else is handled in expression that reference this one value.
The WEEKDAY function returns 0 for a date value that is a Monday, so a NOT on the return from the WEEKDAY function will return TRUE for a Monday.
For a supplied date value of '2015-05-01', this returns:
dd
--
04
11
18
25
Related
I have a table with 2 columns, date and score. It has at most 30 entries, for each of the last 30 days one.
date score
-----------------
1.8.2010 19
2.8.2010 21
4.8.2010 14
7.8.2010 10
10.8.2010 14
My problem is that some dates are missing - I want to see:
date score
-----------------
1.8.2010 19
2.8.2010 21
3.8.2010 0
4.8.2010 14
5.8.2010 0
6.8.2010 0
7.8.2010 10
...
What I need from the single query is to get: 19,21,9,14,0,0,10,0,0,14... That means that the missing dates are filled with 0.
I know how to get all the values and in server side language iterating through dates and missing the blanks. But is this possible to do in mysql, so that I sort the result by date and get the missing pieces.
EDIT: In this table there is another column named UserID, so I have 30.000 users and some of them have the score in this table. I delete the dates every day if date < 30 days ago because I need last 30 days score for each user. The reason is I am making a graph of the user activity over the last 30 days and to plot a chart I need the 30 values separated by comma. So I can say in query get me the USERID=10203 activity and the query would get me the 30 scores, one for each of the last 30 days. I hope I am more clear now.
MySQL doesn't have recursive functionality, so you're left with using the NUMBERS table trick -
Create a table that only holds incrementing numbers - easy to do using an auto_increment:
DROP TABLE IF EXISTS `example`.`numbers`;
CREATE TABLE `example`.`numbers` (
`id` int(10) unsigned NOT NULL auto_increment,
PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
Populate the table using:
INSERT INTO `example`.`numbers`
( `id` )
VALUES
( NULL )
...for as many values as you need.
Use DATE_ADD to construct a list of dates, increasing the days based on the NUMBERS.id value. Replace "2010-06-06" and "2010-06-14" with your respective start and end dates (but use the same format, YYYY-MM-DD) -
SELECT `x`.*
FROM (SELECT DATE_ADD('2010-06-06', INTERVAL `n`.`id` - 1 DAY)
FROM `numbers` `n`
WHERE DATE_ADD('2010-06-06', INTERVAL `n`.`id` -1 DAY) <= '2010-06-14' ) x
LEFT JOIN onto your table of data based on the time portion:
SELECT `x`.`ts` AS `timestamp`,
COALESCE(`y`.`score`, 0) AS `cnt`
FROM (SELECT DATE_FORMAT(DATE_ADD('2010-06-06', INTERVAL `n`.`id` - 1 DAY), '%m/%d/%Y') AS `ts`
FROM `numbers` `n`
WHERE DATE_ADD('2010-06-06', INTERVAL `n`.`id` - 1 DAY) <= '2010-06-14') x
LEFT JOIN TABLE `y` ON STR_TO_DATE(`y`.`date`, '%d.%m.%Y') = `x`.`ts`
If you want to maintain the date format, use the DATE_FORMAT function:
DATE_FORMAT(`x`.`ts`, '%d.%m.%Y') AS `timestamp`
I'm not a fan of the other answers, requiring tables to be created and such. This query does it efficiently without helper tables.
SELECT
IF(score IS NULL, 0, score) AS score,
b.Days AS date
FROM
(SELECT a.Days
FROM (
SELECT curdate() - INTERVAL (a.a + (10 * b.a) + (100 * c.a)) DAY AS Days
FROM (SELECT 0 AS a UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) AS a
CROSS JOIN (SELECT 0 AS a UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) AS b
CROSS JOIN (SELECT 0 AS a UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) AS c
) a
WHERE a.Days >= curdate() - INTERVAL 30 DAY) b
LEFT JOIN your_table
ON date = b.Days
ORDER BY b.Days;
So lets dissect this.
SELECT
IF(score IS NULL, 0, score) AS score,
b.Days AS date
The if will detect days that had no score and set them to 0. b.Days is the configured amount of days you chose to get from the current date, up to 1000.
(SELECT a.Days
FROM (
SELECT curdate() - INTERVAL (a.a + (10 * b.a) + (100 * c.a)) DAY AS Days
FROM (SELECT 0 AS a UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) AS a
CROSS JOIN (SELECT 0 AS a UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) AS b
CROSS JOIN (SELECT 0 AS a UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) AS c
) a
WHERE a.Days >= curdate() - INTERVAL 30 DAY) b
This subquery is something I saw on stackoverflow. It efficiently generates a list of the past 1000 days from the current date. The interval (currently 30) in the WHERE clause at the end determines which days are returned; the maximum is 1000. This query could be easily modified to return 100s of years worth of dates, but 1000 should be good for most things.
LEFT JOIN your_table
ON date = b.Days
ORDER BY b.Days;
This is the part that brings your table that contains the score into it. You compare to the selected date range from the date generator query to be able to fill in 0s where needed (the score will be set to NULL initially, because it is a LEFT JOIN; this is fixed in the select statement). I also order it by the dates, just because. This is preference, you could also order by score.
Before the ORDER BY you could easily join with your table about user info you mentioned with your edit, to add that last requirement.
I hope this version of the query helps someone. Thanks for reading.
Time went by since this question was asked. MySQL 8.0 was released in 2018 and added support for recursive common table expressions, which provide an elegant, state-of-the-art solution to this question.
The following query can be used to generate a list of dates, say for the first 15 days of August 2010:
with recursive all_dates(dt) as (
-- anchor
select '2010-08-01' dt
union all
-- recursion with stop condition
select dt + interval 1 day from all_dates where dt < '2010-08-15'
)
select * from all_dates order by dt
You can then left join this resultset with your table to generate the expected output:
with recursive all_dates(dt) as (
select '2010-08-01' dt
union all
select dt + interval 1 day from all_dates where dt < '2010-08-15'
)
select d.dt date, coalesce(t.score, 0) score
from all_dates d
left join mytable t on t.date = d.dt
order by d.dt
Demo on DB Fiddle:
date | score
:--------- | ----:
2010-08-01 | 19
2010-08-02 | 21
2010-08-03 | 0
2010-08-04 | 14
2010-08-05 | 0
2010-08-06 | 0
2010-08-07 | 10
2010-08-08 | 0
2010-08-09 | 0
2010-08-10 | 14
2010-08-11 | 0
2010-08-12 | 0
2010-08-13 | 0
2010-08-14 | 0
2010-08-15 | 0
Note that it is very easy to adapt the recursive CTE for other intervals or periods. As an example, say we want a row every 15 minutes from 4 AM to 8 AM on August 1st, 2010 ; we can do :
with recursive all_dates(dt) as (
select '2010-08-01 04:00:00' dt
union all
select dt + interval 15 minute from all_dates where dt < '2010-08-01 08:00:00'
)
...
You can accomplish this by using a Calendar Table. That's a table which you create once and fill with a date range (e.g. one dataset for each day 2000-2050; that depends on your data). Then you can make an outer join of your table against the calendar table. If a date is missing in your table, you return 0 for the score.
Michael Conard answer is great but I needed intervals of 15 minutes where the time must always start at the top of every 15th minute:
SELECT a.Days
FROM (
SELECT FROM_UNIXTIME( FLOOR( UNIX_TIMESTAMP() / (15 * 60) ) * (15 * 60)) - INTERVAL 15 * (a.a + (10 * b.a) + (100 * c.a)) MINUTE AS Days
FROM (SELECT 0 AS a UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) AS a
CROSS JOIN (SELECT 0 AS a UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) AS b
CROSS JOIN (SELECT 0 AS a UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) AS c
) a
WHERE a.Days >= curdate() - INTERVAL 30 DAY
This will set the current time to the previous round 15th minute:
FROM_UNIXTIME( FLOOR( UNIX_TIMESTAMP() / (15 * 60) ) * (15 * 60))
And this will remove time with a 15 minute step:
- INTERVAL 15 * (a.a + (10 * b.a) + (100 * c.a)) MINUTE
If there's a simpler way to do it, please let me know.
you can user direct from start date up to today with insertion
with recursive all_dates(dt) as (
-- anchor
select '2021-01-01' dt
union all
-- recursion with stop condition
INSERT IGNORE INTO mytable (date,score) VALUES (dt + interval 1 day ,0 ) where dt + interval 1 day <= curdate()
)
select * from all_dates
I have a database of users. I would like to create a graph based on userbase growth. The query I have now is:
SELECT DATE(datecreated), count(*) AS number FROM users
WHERE DATE(datecreated) > '2009-06-21' AND DATE(datecreated) <= DATE(NOW())
GROUP BY DATE(datecreated) ORDER BY datecreated ASC
This returns almost what I want. If we get 0 users one day, that day is not returned as a 0 value, it is just skipped and the next day that has at least one user is returned. How can I get something like (psuedo-response):
date1 5
date2 8
date3 0
date4 0
date5 9
etc...
where the dates with zero show up in sequential order with the rest of the dates?
Thanks!
I hope you will figure out the rest.
select * from (
select date_add('2003-01-01 00:00:00.000', INTERVAL n5.num*10000+n4.num*1000+n3.num*100+n2.num*10+n1.num DAY ) as date from
(select 0 as num
union all select 1
union all select 2
union all select 3
union all select 4
union all select 5
union all select 6
union all select 7
union all select 8
union all select 9) n1,
(select 0 as num
union all select 1
union all select 2
union all select 3
union all select 4
union all select 5
union all select 6
union all select 7
union all select 8
union all select 9) n2,
(select 0 as num
union all select 1
union all select 2
union all select 3
union all select 4
union all select 5
union all select 6
union all select 7
union all select 8
union all select 9) n3,
(select 0 as num
union all select 1
union all select 2
union all select 3
union all select 4
union all select 5
union all select 6
union all select 7
union all select 8
union all select 9) n4,
(select 0 as num
union all select 1
union all select 2
union all select 3
union all select 4
union all select 5
union all select 6
union all select 7
union all select 8
union all select 9) n5
) a
where date >'2011-01-02 00:00:00.000' and date < NOW()
order by date
With
select n3.num*100+n2.num*10+n1.num as date
you will get a column with numbers from 0 to max(n3)*100+max(n2)*10+max(n1)
Since here we have max n3 as 3, SELECT will return 399, plus 0 -> 400 records (dates in calendar).
You can tune your dynamic calendar by limiting it, for example, from min(date) you have to now().
This question asks the same thing I think. Generally the accepted answer seems to be that you either do it in your application logic (read in what you have into an array, then loop through the array and create the missing dates), or you use temporary tables filled with the dates you wish to join.
This is better to do as:
-- 7 Days:
set #n:=date(now() + interval 1 day);
SELECT qb.day_series as days , COALESCE(col_byte, 0) as Bytes from tbl1 qa
right join (
select (select #n:= #n - interval 1 day) day_series from tbl1 limit 7 ) as qb
on date(qa.Timestamp) = qb.day_series and
qa.Timestamp > DATE_SUB(curdate(), INTERVAL 7 day) order by qb.day_series asc
-- 30 Days:
set #n:=date(now() + interval 1 day);
SELECT qb.day_series as days , COALESCE(col_byte, 0) as Bytes from tbl1 qa
right join (
select (select #n:= #n - interval 1 day) day_series from tbl1 limit 30 ) as qb
on date(qa.Timestamp) = qb.day_series and
qa.Timestamp > DATE_SUB(curdate(), INTERVAL 30 day) order by qb.day_series asc;
or without variable like this:
SELECT qb.day_series as days , COALESCE(col_byte, 0) as Bytes from tbl1 qa
right join (
select curdate() - INTERVAL a.a day as day_series from(
select 0 as a union all select 1 union all select 2 union all
select 3 union all select 4 union all
select 5 union all select 6 union all select 7
) as a ) as qb
on date(qa.Timestamp) = qb.day_series and
qa.Timestamp > DATE_SUB(curdate(), INTERVAL 7 day) order by qb.day_series asc;
Do a right outer join to a table, call it tblCalendar, that is pre-populated with the dates you wish to report on. And join on the date field.
Paul
Query is:
SELECT qb.dy as yourday, COALESCE(count(yourcolumn), 0) as yourcount from yourtable qa
right join (
select curdate() as dy union
select DATE_SUB(curdate(), INTERVAL 1 day) as dy union
select DATE_SUB(curdate(), INTERVAL 2 day) as dy union
select DATE_SUB(curdate(), INTERVAL 3 day) as dy union
select DATE_SUB(curdate(), INTERVAL 4 day) as dy union
select DATE_SUB(curdate(), INTERVAL 5 day) as dy union
select DATE_SUB(curdate(), INTERVAL 6 day) as dy
) as qb
on qa.dates = qb.dy
and qa.dates > DATE_SUB(curdate(), INTERVAL 7 day)
order by qb.dy asc;
and the result is:
+------------+-----------+
| yourday | yourcount |
+------------+-----------+
| 2015-06-24 | 274339 |
| 2015-06-25 | 0 |
| 2015-06-26 | 0 |
| 2015-06-27 | 0 |
| 2015-06-28 | 134703 |
| 2015-06-29 | 87613 |
| 2015-06-30 | 0 |
+------------+-----------+
On further thought, something like this should be what you want:
CREATE TEMPORARY TABLE DateSummary1 ( datenew timestamp ) SELECT DISTINCT(DATE(datecreated)) as datenew FROM users;
CREATE TEMPORARY TABLE DateSummary2 ( datenew timestamp, number int ) SELECT DATE(datecreated) as datenew, count(*) AS number FROM users
WHERE DATE(datecreated) > '2009-06-21' AND DATE(datecreated) <= DATE(NOW())
GROUP BY DATE(datecreated) ORDER BY datecreated ASC;
SELECT ds1.datenew,ds2.number FROM DateSummary1 ds1 LEFT JOIN DateSummary2 ds2 on ds1.datenew=ds2.datenew;
This gives you all the dates in the first table, and the count summary data in the second table. You might need to replace ds2.number with IF(ISNULL(ds2.number),0,ds2.number) or something similar.
I have a table with 2 columns, date and score. It has at most 30 entries, for each of the last 30 days one.
date score
-----------------
1.8.2010 19
2.8.2010 21
4.8.2010 14
7.8.2010 10
10.8.2010 14
My problem is that some dates are missing - I want to see:
date score
-----------------
1.8.2010 19
2.8.2010 21
3.8.2010 0
4.8.2010 14
5.8.2010 0
6.8.2010 0
7.8.2010 10
...
What I need from the single query is to get: 19,21,9,14,0,0,10,0,0,14... That means that the missing dates are filled with 0.
I know how to get all the values and in server side language iterating through dates and missing the blanks. But is this possible to do in mysql, so that I sort the result by date and get the missing pieces.
EDIT: In this table there is another column named UserID, so I have 30.000 users and some of them have the score in this table. I delete the dates every day if date < 30 days ago because I need last 30 days score for each user. The reason is I am making a graph of the user activity over the last 30 days and to plot a chart I need the 30 values separated by comma. So I can say in query get me the USERID=10203 activity and the query would get me the 30 scores, one for each of the last 30 days. I hope I am more clear now.
MySQL doesn't have recursive functionality, so you're left with using the NUMBERS table trick -
Create a table that only holds incrementing numbers - easy to do using an auto_increment:
DROP TABLE IF EXISTS `example`.`numbers`;
CREATE TABLE `example`.`numbers` (
`id` int(10) unsigned NOT NULL auto_increment,
PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
Populate the table using:
INSERT INTO `example`.`numbers`
( `id` )
VALUES
( NULL )
...for as many values as you need.
Use DATE_ADD to construct a list of dates, increasing the days based on the NUMBERS.id value. Replace "2010-06-06" and "2010-06-14" with your respective start and end dates (but use the same format, YYYY-MM-DD) -
SELECT `x`.*
FROM (SELECT DATE_ADD('2010-06-06', INTERVAL `n`.`id` - 1 DAY)
FROM `numbers` `n`
WHERE DATE_ADD('2010-06-06', INTERVAL `n`.`id` -1 DAY) <= '2010-06-14' ) x
LEFT JOIN onto your table of data based on the time portion:
SELECT `x`.`ts` AS `timestamp`,
COALESCE(`y`.`score`, 0) AS `cnt`
FROM (SELECT DATE_FORMAT(DATE_ADD('2010-06-06', INTERVAL `n`.`id` - 1 DAY), '%m/%d/%Y') AS `ts`
FROM `numbers` `n`
WHERE DATE_ADD('2010-06-06', INTERVAL `n`.`id` - 1 DAY) <= '2010-06-14') x
LEFT JOIN TABLE `y` ON STR_TO_DATE(`y`.`date`, '%d.%m.%Y') = `x`.`ts`
If you want to maintain the date format, use the DATE_FORMAT function:
DATE_FORMAT(`x`.`ts`, '%d.%m.%Y') AS `timestamp`
I'm not a fan of the other answers, requiring tables to be created and such. This query does it efficiently without helper tables.
SELECT
IF(score IS NULL, 0, score) AS score,
b.Days AS date
FROM
(SELECT a.Days
FROM (
SELECT curdate() - INTERVAL (a.a + (10 * b.a) + (100 * c.a)) DAY AS Days
FROM (SELECT 0 AS a UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) AS a
CROSS JOIN (SELECT 0 AS a UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) AS b
CROSS JOIN (SELECT 0 AS a UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) AS c
) a
WHERE a.Days >= curdate() - INTERVAL 30 DAY) b
LEFT JOIN your_table
ON date = b.Days
ORDER BY b.Days;
So lets dissect this.
SELECT
IF(score IS NULL, 0, score) AS score,
b.Days AS date
The if will detect days that had no score and set them to 0. b.Days is the configured amount of days you chose to get from the current date, up to 1000.
(SELECT a.Days
FROM (
SELECT curdate() - INTERVAL (a.a + (10 * b.a) + (100 * c.a)) DAY AS Days
FROM (SELECT 0 AS a UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) AS a
CROSS JOIN (SELECT 0 AS a UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) AS b
CROSS JOIN (SELECT 0 AS a UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) AS c
) a
WHERE a.Days >= curdate() - INTERVAL 30 DAY) b
This subquery is something I saw on stackoverflow. It efficiently generates a list of the past 1000 days from the current date. The interval (currently 30) in the WHERE clause at the end determines which days are returned; the maximum is 1000. This query could be easily modified to return 100s of years worth of dates, but 1000 should be good for most things.
LEFT JOIN your_table
ON date = b.Days
ORDER BY b.Days;
This is the part that brings your table that contains the score into it. You compare to the selected date range from the date generator query to be able to fill in 0s where needed (the score will be set to NULL initially, because it is a LEFT JOIN; this is fixed in the select statement). I also order it by the dates, just because. This is preference, you could also order by score.
Before the ORDER BY you could easily join with your table about user info you mentioned with your edit, to add that last requirement.
I hope this version of the query helps someone. Thanks for reading.
Time went by since this question was asked. MySQL 8.0 was released in 2018 and added support for recursive common table expressions, which provide an elegant, state-of-the-art solution to this question.
The following query can be used to generate a list of dates, say for the first 15 days of August 2010:
with recursive all_dates(dt) as (
-- anchor
select '2010-08-01' dt
union all
-- recursion with stop condition
select dt + interval 1 day from all_dates where dt < '2010-08-15'
)
select * from all_dates order by dt
You can then left join this resultset with your table to generate the expected output:
with recursive all_dates(dt) as (
select '2010-08-01' dt
union all
select dt + interval 1 day from all_dates where dt < '2010-08-15'
)
select d.dt date, coalesce(t.score, 0) score
from all_dates d
left join mytable t on t.date = d.dt
order by d.dt
Demo on DB Fiddle:
date | score
:--------- | ----:
2010-08-01 | 19
2010-08-02 | 21
2010-08-03 | 0
2010-08-04 | 14
2010-08-05 | 0
2010-08-06 | 0
2010-08-07 | 10
2010-08-08 | 0
2010-08-09 | 0
2010-08-10 | 14
2010-08-11 | 0
2010-08-12 | 0
2010-08-13 | 0
2010-08-14 | 0
2010-08-15 | 0
Note that it is very easy to adapt the recursive CTE for other intervals or periods. As an example, say we want a row every 15 minutes from 4 AM to 8 AM on August 1st, 2010 ; we can do :
with recursive all_dates(dt) as (
select '2010-08-01 04:00:00' dt
union all
select dt + interval 15 minute from all_dates where dt < '2010-08-01 08:00:00'
)
...
You can accomplish this by using a Calendar Table. That's a table which you create once and fill with a date range (e.g. one dataset for each day 2000-2050; that depends on your data). Then you can make an outer join of your table against the calendar table. If a date is missing in your table, you return 0 for the score.
Michael Conard answer is great but I needed intervals of 15 minutes where the time must always start at the top of every 15th minute:
SELECT a.Days
FROM (
SELECT FROM_UNIXTIME( FLOOR( UNIX_TIMESTAMP() / (15 * 60) ) * (15 * 60)) - INTERVAL 15 * (a.a + (10 * b.a) + (100 * c.a)) MINUTE AS Days
FROM (SELECT 0 AS a UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) AS a
CROSS JOIN (SELECT 0 AS a UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) AS b
CROSS JOIN (SELECT 0 AS a UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) AS c
) a
WHERE a.Days >= curdate() - INTERVAL 30 DAY
This will set the current time to the previous round 15th minute:
FROM_UNIXTIME( FLOOR( UNIX_TIMESTAMP() / (15 * 60) ) * (15 * 60))
And this will remove time with a 15 minute step:
- INTERVAL 15 * (a.a + (10 * b.a) + (100 * c.a)) MINUTE
If there's a simpler way to do it, please let me know.
you can user direct from start date up to today with insertion
with recursive all_dates(dt) as (
-- anchor
select '2021-01-01' dt
union all
-- recursion with stop condition
INSERT IGNORE INTO mytable (date,score) VALUES (dt + interval 1 day ,0 ) where dt + interval 1 day <= curdate()
)
select * from all_dates
I have a table with 2 columns, date and score. It has at most 30 entries, for each of the last 30 days one.
date score
-----------------
1.8.2010 19
2.8.2010 21
4.8.2010 14
7.8.2010 10
10.8.2010 14
My problem is that some dates are missing - I want to see:
date score
-----------------
1.8.2010 19
2.8.2010 21
3.8.2010 0
4.8.2010 14
5.8.2010 0
6.8.2010 0
7.8.2010 10
...
What I need from the single query is to get: 19,21,9,14,0,0,10,0,0,14... That means that the missing dates are filled with 0.
I know how to get all the values and in server side language iterating through dates and missing the blanks. But is this possible to do in mysql, so that I sort the result by date and get the missing pieces.
EDIT: In this table there is another column named UserID, so I have 30.000 users and some of them have the score in this table. I delete the dates every day if date < 30 days ago because I need last 30 days score for each user. The reason is I am making a graph of the user activity over the last 30 days and to plot a chart I need the 30 values separated by comma. So I can say in query get me the USERID=10203 activity and the query would get me the 30 scores, one for each of the last 30 days. I hope I am more clear now.
MySQL doesn't have recursive functionality, so you're left with using the NUMBERS table trick -
Create a table that only holds incrementing numbers - easy to do using an auto_increment:
DROP TABLE IF EXISTS `example`.`numbers`;
CREATE TABLE `example`.`numbers` (
`id` int(10) unsigned NOT NULL auto_increment,
PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
Populate the table using:
INSERT INTO `example`.`numbers`
( `id` )
VALUES
( NULL )
...for as many values as you need.
Use DATE_ADD to construct a list of dates, increasing the days based on the NUMBERS.id value. Replace "2010-06-06" and "2010-06-14" with your respective start and end dates (but use the same format, YYYY-MM-DD) -
SELECT `x`.*
FROM (SELECT DATE_ADD('2010-06-06', INTERVAL `n`.`id` - 1 DAY)
FROM `numbers` `n`
WHERE DATE_ADD('2010-06-06', INTERVAL `n`.`id` -1 DAY) <= '2010-06-14' ) x
LEFT JOIN onto your table of data based on the time portion:
SELECT `x`.`ts` AS `timestamp`,
COALESCE(`y`.`score`, 0) AS `cnt`
FROM (SELECT DATE_FORMAT(DATE_ADD('2010-06-06', INTERVAL `n`.`id` - 1 DAY), '%m/%d/%Y') AS `ts`
FROM `numbers` `n`
WHERE DATE_ADD('2010-06-06', INTERVAL `n`.`id` - 1 DAY) <= '2010-06-14') x
LEFT JOIN TABLE `y` ON STR_TO_DATE(`y`.`date`, '%d.%m.%Y') = `x`.`ts`
If you want to maintain the date format, use the DATE_FORMAT function:
DATE_FORMAT(`x`.`ts`, '%d.%m.%Y') AS `timestamp`
I'm not a fan of the other answers, requiring tables to be created and such. This query does it efficiently without helper tables.
SELECT
IF(score IS NULL, 0, score) AS score,
b.Days AS date
FROM
(SELECT a.Days
FROM (
SELECT curdate() - INTERVAL (a.a + (10 * b.a) + (100 * c.a)) DAY AS Days
FROM (SELECT 0 AS a UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) AS a
CROSS JOIN (SELECT 0 AS a UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) AS b
CROSS JOIN (SELECT 0 AS a UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) AS c
) a
WHERE a.Days >= curdate() - INTERVAL 30 DAY) b
LEFT JOIN your_table
ON date = b.Days
ORDER BY b.Days;
So lets dissect this.
SELECT
IF(score IS NULL, 0, score) AS score,
b.Days AS date
The if will detect days that had no score and set them to 0. b.Days is the configured amount of days you chose to get from the current date, up to 1000.
(SELECT a.Days
FROM (
SELECT curdate() - INTERVAL (a.a + (10 * b.a) + (100 * c.a)) DAY AS Days
FROM (SELECT 0 AS a UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) AS a
CROSS JOIN (SELECT 0 AS a UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) AS b
CROSS JOIN (SELECT 0 AS a UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) AS c
) a
WHERE a.Days >= curdate() - INTERVAL 30 DAY) b
This subquery is something I saw on stackoverflow. It efficiently generates a list of the past 1000 days from the current date. The interval (currently 30) in the WHERE clause at the end determines which days are returned; the maximum is 1000. This query could be easily modified to return 100s of years worth of dates, but 1000 should be good for most things.
LEFT JOIN your_table
ON date = b.Days
ORDER BY b.Days;
This is the part that brings your table that contains the score into it. You compare to the selected date range from the date generator query to be able to fill in 0s where needed (the score will be set to NULL initially, because it is a LEFT JOIN; this is fixed in the select statement). I also order it by the dates, just because. This is preference, you could also order by score.
Before the ORDER BY you could easily join with your table about user info you mentioned with your edit, to add that last requirement.
I hope this version of the query helps someone. Thanks for reading.
Time went by since this question was asked. MySQL 8.0 was released in 2018 and added support for recursive common table expressions, which provide an elegant, state-of-the-art solution to this question.
The following query can be used to generate a list of dates, say for the first 15 days of August 2010:
with recursive all_dates(dt) as (
-- anchor
select '2010-08-01' dt
union all
-- recursion with stop condition
select dt + interval 1 day from all_dates where dt < '2010-08-15'
)
select * from all_dates order by dt
You can then left join this resultset with your table to generate the expected output:
with recursive all_dates(dt) as (
select '2010-08-01' dt
union all
select dt + interval 1 day from all_dates where dt < '2010-08-15'
)
select d.dt date, coalesce(t.score, 0) score
from all_dates d
left join mytable t on t.date = d.dt
order by d.dt
Demo on DB Fiddle:
date | score
:--------- | ----:
2010-08-01 | 19
2010-08-02 | 21
2010-08-03 | 0
2010-08-04 | 14
2010-08-05 | 0
2010-08-06 | 0
2010-08-07 | 10
2010-08-08 | 0
2010-08-09 | 0
2010-08-10 | 14
2010-08-11 | 0
2010-08-12 | 0
2010-08-13 | 0
2010-08-14 | 0
2010-08-15 | 0
Note that it is very easy to adapt the recursive CTE for other intervals or periods. As an example, say we want a row every 15 minutes from 4 AM to 8 AM on August 1st, 2010 ; we can do :
with recursive all_dates(dt) as (
select '2010-08-01 04:00:00' dt
union all
select dt + interval 15 minute from all_dates where dt < '2010-08-01 08:00:00'
)
...
You can accomplish this by using a Calendar Table. That's a table which you create once and fill with a date range (e.g. one dataset for each day 2000-2050; that depends on your data). Then you can make an outer join of your table against the calendar table. If a date is missing in your table, you return 0 for the score.
Michael Conard answer is great but I needed intervals of 15 minutes where the time must always start at the top of every 15th minute:
SELECT a.Days
FROM (
SELECT FROM_UNIXTIME( FLOOR( UNIX_TIMESTAMP() / (15 * 60) ) * (15 * 60)) - INTERVAL 15 * (a.a + (10 * b.a) + (100 * c.a)) MINUTE AS Days
FROM (SELECT 0 AS a UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) AS a
CROSS JOIN (SELECT 0 AS a UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) AS b
CROSS JOIN (SELECT 0 AS a UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) AS c
) a
WHERE a.Days >= curdate() - INTERVAL 30 DAY
This will set the current time to the previous round 15th minute:
FROM_UNIXTIME( FLOOR( UNIX_TIMESTAMP() / (15 * 60) ) * (15 * 60))
And this will remove time with a 15 minute step:
- INTERVAL 15 * (a.a + (10 * b.a) + (100 * c.a)) MINUTE
If there's a simpler way to do it, please let me know.
you can user direct from start date up to today with insertion
with recursive all_dates(dt) as (
-- anchor
select '2021-01-01' dt
union all
-- recursion with stop condition
INSERT IGNORE INTO mytable (date,score) VALUES (dt + interval 1 day ,0 ) where dt + interval 1 day <= curdate()
)
select * from all_dates
I have a database of users. I would like to create a graph based on userbase growth. The query I have now is:
SELECT DATE(datecreated), count(*) AS number FROM users
WHERE DATE(datecreated) > '2009-06-21' AND DATE(datecreated) <= DATE(NOW())
GROUP BY DATE(datecreated) ORDER BY datecreated ASC
This returns almost what I want. If we get 0 users one day, that day is not returned as a 0 value, it is just skipped and the next day that has at least one user is returned. How can I get something like (psuedo-response):
date1 5
date2 8
date3 0
date4 0
date5 9
etc...
where the dates with zero show up in sequential order with the rest of the dates?
Thanks!
I hope you will figure out the rest.
select * from (
select date_add('2003-01-01 00:00:00.000', INTERVAL n5.num*10000+n4.num*1000+n3.num*100+n2.num*10+n1.num DAY ) as date from
(select 0 as num
union all select 1
union all select 2
union all select 3
union all select 4
union all select 5
union all select 6
union all select 7
union all select 8
union all select 9) n1,
(select 0 as num
union all select 1
union all select 2
union all select 3
union all select 4
union all select 5
union all select 6
union all select 7
union all select 8
union all select 9) n2,
(select 0 as num
union all select 1
union all select 2
union all select 3
union all select 4
union all select 5
union all select 6
union all select 7
union all select 8
union all select 9) n3,
(select 0 as num
union all select 1
union all select 2
union all select 3
union all select 4
union all select 5
union all select 6
union all select 7
union all select 8
union all select 9) n4,
(select 0 as num
union all select 1
union all select 2
union all select 3
union all select 4
union all select 5
union all select 6
union all select 7
union all select 8
union all select 9) n5
) a
where date >'2011-01-02 00:00:00.000' and date < NOW()
order by date
With
select n3.num*100+n2.num*10+n1.num as date
you will get a column with numbers from 0 to max(n3)*100+max(n2)*10+max(n1)
Since here we have max n3 as 3, SELECT will return 399, plus 0 -> 400 records (dates in calendar).
You can tune your dynamic calendar by limiting it, for example, from min(date) you have to now().
This question asks the same thing I think. Generally the accepted answer seems to be that you either do it in your application logic (read in what you have into an array, then loop through the array and create the missing dates), or you use temporary tables filled with the dates you wish to join.
This is better to do as:
-- 7 Days:
set #n:=date(now() + interval 1 day);
SELECT qb.day_series as days , COALESCE(col_byte, 0) as Bytes from tbl1 qa
right join (
select (select #n:= #n - interval 1 day) day_series from tbl1 limit 7 ) as qb
on date(qa.Timestamp) = qb.day_series and
qa.Timestamp > DATE_SUB(curdate(), INTERVAL 7 day) order by qb.day_series asc
-- 30 Days:
set #n:=date(now() + interval 1 day);
SELECT qb.day_series as days , COALESCE(col_byte, 0) as Bytes from tbl1 qa
right join (
select (select #n:= #n - interval 1 day) day_series from tbl1 limit 30 ) as qb
on date(qa.Timestamp) = qb.day_series and
qa.Timestamp > DATE_SUB(curdate(), INTERVAL 30 day) order by qb.day_series asc;
or without variable like this:
SELECT qb.day_series as days , COALESCE(col_byte, 0) as Bytes from tbl1 qa
right join (
select curdate() - INTERVAL a.a day as day_series from(
select 0 as a union all select 1 union all select 2 union all
select 3 union all select 4 union all
select 5 union all select 6 union all select 7
) as a ) as qb
on date(qa.Timestamp) = qb.day_series and
qa.Timestamp > DATE_SUB(curdate(), INTERVAL 7 day) order by qb.day_series asc;
Do a right outer join to a table, call it tblCalendar, that is pre-populated with the dates you wish to report on. And join on the date field.
Paul
Query is:
SELECT qb.dy as yourday, COALESCE(count(yourcolumn), 0) as yourcount from yourtable qa
right join (
select curdate() as dy union
select DATE_SUB(curdate(), INTERVAL 1 day) as dy union
select DATE_SUB(curdate(), INTERVAL 2 day) as dy union
select DATE_SUB(curdate(), INTERVAL 3 day) as dy union
select DATE_SUB(curdate(), INTERVAL 4 day) as dy union
select DATE_SUB(curdate(), INTERVAL 5 day) as dy union
select DATE_SUB(curdate(), INTERVAL 6 day) as dy
) as qb
on qa.dates = qb.dy
and qa.dates > DATE_SUB(curdate(), INTERVAL 7 day)
order by qb.dy asc;
and the result is:
+------------+-----------+
| yourday | yourcount |
+------------+-----------+
| 2015-06-24 | 274339 |
| 2015-06-25 | 0 |
| 2015-06-26 | 0 |
| 2015-06-27 | 0 |
| 2015-06-28 | 134703 |
| 2015-06-29 | 87613 |
| 2015-06-30 | 0 |
+------------+-----------+
On further thought, something like this should be what you want:
CREATE TEMPORARY TABLE DateSummary1 ( datenew timestamp ) SELECT DISTINCT(DATE(datecreated)) as datenew FROM users;
CREATE TEMPORARY TABLE DateSummary2 ( datenew timestamp, number int ) SELECT DATE(datecreated) as datenew, count(*) AS number FROM users
WHERE DATE(datecreated) > '2009-06-21' AND DATE(datecreated) <= DATE(NOW())
GROUP BY DATE(datecreated) ORDER BY datecreated ASC;
SELECT ds1.datenew,ds2.number FROM DateSummary1 ds1 LEFT JOIN DateSummary2 ds2 on ds1.datenew=ds2.datenew;
This gives you all the dates in the first table, and the count summary data in the second table. You might need to replace ds2.number with IF(ISNULL(ds2.number),0,ds2.number) or something similar.