I have a MySql table containing events having a DATETIME timestamp. I want to count each day's events. On some days, e.g. on Sundays, events are missing. The result should contain these days too with a count of zero.
My query is the following:
SELECT
COUNT(1) AS mycount,
DATE_FORMAT(DATE(evaluations.timestamp),"%a, %d.%m.%Y") AS date
FROM Events
GROUP BY DATE(timestamp)
ORDER BY DATE(timestamp) DESC
Can I modify the query without using a helper table containing all dates?
A single query (no procedere, no function) would be fine.
The query would somehow look like this if you don't have any calendar table:
SELECT
dateTable.day,
COALESCE(t.mycount,0) AS cnt
FROM
(
SELECT ADDDATE((SELECT MIN(DATE(timestamp)) FROM Events), INTERVAL #i:=#i+1 DAY) AS DAY
FROM (
SELECT a.a
FROM (SELECT 0 AS a UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) AS a
CROSS JOIN (SELECT 0 AS a UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) AS b
CROSS JOIN (SELECT 0 AS a UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) AS c
) a
JOIN (SELECT #i := -1) r1
WHERE
#i < DATEDIFF((SELECT MAX(DATE(timestamp)) FROM Events), (SELECT MIN(DATE(timestamp)) FROM Events))
) AS dateTable
LEFT JOIN
(
SELECT
COUNT(1) AS mycount,
DATE_FORMAT(DATE(evaluations.timestamp),"%a, %d.%m.%Y") AS date
FROM Events
GROUP BY DATE(timestamp)
ORDER BY DATE(timestamp) DESC
) AS t
ON dateTable.day = t.date
ORDER BY dateTable.day DESC;
Note:
If you think you will need this kind of query too often then you can create a table where all the dates would reside. Newer dates can be added through mysql event periodically .
Then the work is simple. Just need to make a LEFT JOIN between the calendar table and the result of your query.
Related
I have a reports table with the following structure :
I want a SQL Query to get the report for the last 365 days by following conditions :
Group dates if the same date is repeated.
The days which the report is not available for the last 365 days, I need those days added to the result rows with 0 as their success and failed recipients.
I tried to get it by group by report dates
SELECT report_date, SUM(success_recipient) as success_recipient, SUM(failed_recipient) as failed_recipient FROM reports GROUP BY report_date;
and I have got the grouped result which satisfies the first condition
Now I need to append the rest of the days in the last 365 days to this result in which 0 as their success and failure recipients.
Expected result :
and so on ..
MYSQL VERSION : 5.6
One way to achieve this is using "with recursive" to generate all dates you need in you output and then outer join to the rest of your query. Note: I use the number 356 as it is in your description but it seems more appropriate to use date difference as this approach does not take into account leap years. Using the query below you will get NULL values in case you have no data. If you need the value 0 you can use coalesce(sum(...), 0).
with recursive
dates as (
select curdate()-356 dt
union all
select dt+1 from dates
where dt < curdate()
)
select
dt report_date,
sum(success_recipient) success_recipient,
sum(failed_recipient) failed_recipient
from dates
left join reports on report_date = dt
group by report_date;
From the above comments and the answer, I could write this query which gave me the expected outcome :
SELECT a.date, SUM(COALESCE(r.success_recipient, 0)), SUM(COALESCE(r.failed_recipient, 0))
FROM (
SELECT curdate() - INTERVAL (a.a + (10 * b.a) + (100 * c.a) + (1000 * d.a) ) DAY AS date
FROM (select 0 as a union all select 1 union all select 2 union all select 3 union all select 4 union all select 5 union all select 6 union all select 7 union all select 8 union all select 9) as a
cross join (select 0 as a union all select 1 union all select 2 union all select 3 union all select 4 union all select 5 union all select 6 union all select 7 union all select 8 union all select 9) as b
cross join (select 0 as a union all select 1 union all select 2 union all select 3 union all select 4 union all select 5 union all select 6 union all select 7 union all select 8 union all select 9) as c
cross join (select 0 as a union all select 1 union all select 2 union all select 3 union all select 4 union all select 5 union all select 6 union all select 7 union all select 8 union all select 9) as d
) a
LEFT JOIN reports r ON a.date = r.report_date
WHERE a.date between DATE_SUB(CURDATE(), INTERVAL 1 YEAR) and now()
GROUP BY a.date;
I want to count the number of actions per day in my dataset.
date action_id
2010-01-01 id00
2010-01-03 id01
2010-01-05 id02
This is just a sample, but the point is that my data does not include actions for every day and I want to include days where there are zero actions in my result.
My plan is to do this.
with dates as (
select [sequence of dates from 2010-01-01 to 2010-02-01] as day)
select day, coalesce(count(distinct action_id), 0) as actions
from dates
left join my_table
on dates.date = my_table.date
How do I create the sequence of dates?
You example shows a CTE. So, you can use a recursive CTE:
with recursive dates as (
select date('2010-01-01') as day
union all
select day + interval 1 day
from dates
where day < '2010-02-01'
)
select d.day, count(distinct t.action_id) as actions
from dates d left join
my_table t
on d.day = my_table.date
group by d.day;
Note that COUNT() never returns NULL, so COALESCE() is unnecessary.
In older versions, you can use a calendar table or generate the data on the fly. Assuming your table has enough rows:
select d.day, count(distinct t.action_id) as actions
from (select date('2010-01-01') + interval (#rn := #rn + 1) - 1 day as day
from my_table cross join
(select #rn := 0) params
limit 31
) d left join
my_table t
on d.day = my_table.date
group by d.day;
it seems just you need group by and count
select date, count(distinct action_id) as action
from my_table left join
dates on dates.date = my_table.date
group by date
with dates as
(
select a.Date
from (
select curdate() - INTERVAL (a.a + (10 * b.a) + (100 * c.a) + (1000 * d.a) ) DAY as Date
from (select 0 as a union all select 1 union all select 2 union all select 3 union all select 4 union all select 5 union all select 6 union all select 7 union all select 8 union all select 9) as a
cross join (select 0 as a union all select 1 union all select 2 union all select 3 union all select 4 union all select 5 union all select 6 union all select 7 union all select 8 union all select 9) as b
cross join (select 0 as a union all select 1 union all select 2 union all select 3 union all select 4 union all select 5 union all select 6 union all select 7 union all select 8 union all select 9) as c
cross join (select 0 as a union all select 1 union all select 2 union all select 3 union all select 4 union all select 5 union all select 6 union all select 7 union all select 8 union all select 9) as d
) a
where a.Date between '<start_date>' and '<end_date>' )
select day, count(distinct action_id) as actions
from dates
left join my_table
on dates.date = my_table.date
I need zero values also in group by clause .Have read almost all question related to this on Stackoverflow, but none of the solutions have worked.
My Table is
Need to get sum of score grouped by day of month.But I am not getting zero against the days not present in the table
SELECT SUM(engagement_score), DAY(creation_dt)
FROM qee_emp_engagement_index
RIGHT JOIN (
SELECT 1 AS index1 UNION ALL
SELECT 2 UNION ALL
SELECT 3 UNION ALL
SELECT 4 UNION ALL
SELECT 5 UNION ALL
SELECT 6 UNION ALL
SELECT 7 UNION ALL
SELECT 8 UNION ALL
SELECT 9) AS a ON a.index1 = DAY(creation_dt)
WHERE org_id = 1
GROUP BY a.index1
ORDER BY a.index1 ASC
I would write the query using a left join rather than a right join (the logic of left join makes more sense to me: keep all the rows in the first table). But, your problem is the where clause. That logic should go in the on clause:
SELECT COALESCE(SUM(eei.engagement_score), 0), a.index1
FROM (SELECT 1 AS index1 UNION ALL
SELECT 2 UNION ALL
SELECT 3 UNION ALL
SELECT 4 UNION ALL
SELECT 5 UNION ALL
SELECT 6 UNION ALL
SELECT 7 UNION ALL
SELECT 8 UNION ALL
SELECT 9
) a LEFT JOIN
qee_emp_engagement_index eei
ON a.index1 = DAY(eei.creation_dt) AND eei.org_id = 1
GROUP BY a.index1
ORDER BY a.index1 ASC;
In addition, the GROUP BY and SELECT should use the column from the driving table.
You can do right join between your table and a temp table containing all dates in a range.
Dates can be generated using below query
select a.Date
from (
select curdate() - INTERVAL (a.a + (10 * b.a) + (100 * c.a)) DAY as Date
from (select 0 as a union all select 1 union all select 2 union all select 3 union all select 4 union all select 5 union all select 6 union all select 7 union all select 8 union all select 9) as a
cross join (select 0 as a union all select 1 union all select 2 union all select 3 union all select 4 union all select 5 union all select 6 union all select 7 union all select 8 union all select 9) as b
cross join (select 0 as a union all select 1 union all select 2 union all select 3 union all select 4 union all select 5 union all select 6 union all select 7 union all select 8 union all select 9) as c
) a
where a.Date between '2015-01-01' and '2017-01-01'
SELECT DATE(creation_date), SUM(engagement_score) FROM table_name GROUP BY DATE(creation_date)
There is no need to do that weird right join in your code
If you want to group it by the day of month and not by the date, use DAY() instead of DATE()
EDIT:
If you want a zero when the day of month doesn't exist, try this:
SELECT d, SUM(score) FROM
((SELECT creation_date AS d, engagement_score AS score FROM table_name)
UNION ALL
(SELECT d, 0 AS engagement_score FROM table_name WHERE d BETWEEN 1 AND 31))
GROUP BY d
I have a table for slots that have the following Schema
SlotId
FromDate
ToDate
I want to get the remaining days in that given month available for slot booking.
To be clear, I am trying to retrieve all the dates apart from the dates stored in the database(as those are already booked) for a given month.
For example, if a record have FromDate is equal to 2014-04-02 and ToDate is equal to 2014-04-06 I am expecting the following result:
2014-04-01
2014-04-07
...
2014-04-30
Although i am scripting in PHP, I am little curious about the query to accomplish this.
So it is not an easy thing to do in mysql but here is something that should work. this gets the dates in any given month that are not booked... see fiddle for working example
SELECT *, union_month.day_date
FROM (
SELECT 1 AS day_date UNION ALL
SELECT 2 UNION ALL
SELECT 3 UNION ALL
SELECT 4 UNION ALL
SELECT 5 UNION ALL
SELECT 6 UNION ALL
SELECT 7 UNION ALL
SELECT 8 UNION ALL
SELECT 9 UNION ALL
SELECT 10 UNION ALL
SELECT 11 UNION ALL
SELECT 12 UNION ALL
SELECT 13 UNION ALL
SELECT 14 UNION ALL
SELECT 15 UNION ALL
SELECT 16 UNION ALL
SELECT 17 UNION ALL
SELECT 18 UNION ALL
SELECT 19 UNION ALL
SELECT 20 UNION ALL
SELECT 21 UNION ALL
SELECT 22 UNION ALL
SELECT 23 UNION ALL
SELECT 24 UNION ALL
SELECT 25 UNION ALL
SELECT 26 UNION ALL
SELECT 27 UNION ALL
SELECT 28 UNION ALL
SELECT 29 UNION ALL
SELECT 30 UNION ALL
SELECT 31
) AS union_month
LEFT JOIN myTable AS t ON union_month.day_date <> DAY(t.to_date) OR union_month.day_date <> DAY(t.from_date)
WHERE union_month.day_date <= DAY(LAST_DAY(t.to_date))
AND union_month.day_date NOT BETWEEN DAY(t.from_date) AND DAY(t.to_date)
GROUP BY union_month.day_date
for multiple dates in a month change the WHERE clause to this
WHERE
union_month.day_date <= DAY(LAST_DAY(t.to_date))
AND union_month.day_date not BETWEEN (select DAY(from_date) from myTable limit 0,1) AND (select DAY(to_date) from myTable limit 0,1)
AND union_month.day_date not BETWEEN (select DAY(from_date) from myTable limit 1,1) AND (select DAY(to_date) from myTable limit 1,1)
AND union_month.day_date not BETWEEN (select DAY(from_date) from myTable limit 2,1) AND (select DAY(to_date) from myTable limit 2,1)
GROUP BY union_month.day_date
working fiddle for multiple dates
You need remaining days count or dates which are free?
I have tried this. May it will help you.
You need to use this query in loop. with some variables.
In example I have consider April month only. You can do it for all months.
SELECT distinct * FROM (SELECT DATE_ADD('2014-04-01', INTERVAL t4+t16+t64+t256+t1024 DAY) freedays FROM
(SELECT 0 t4 UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3 ) t4,
(SELECT 0 t16 UNION ALL SELECT 4 UNION ALL SELECT 8 UNION ALL SELECT 12 ) t16,
(SELECT 0 t64 UNION ALL SELECT 16 UNION ALL SELECT 32 UNION ALL SELECT 48 ) t64,
(SELECT 0 t256 UNION ALL SELECT 64 UNION ALL SELECT 128 UNION ALL SELECT 192) t256,
(SELECT 0 t1024 UNION ALL SELECT 256 UNION ALL SELECT 512 UNION ALL SELECT 768) t1024
) b
WHERE freedays not between (select FrmDate from slotbooking limit 1) and (select ToDate from
slotbooking limit 1) and freedays < '2014-04-30';
This question already has answers here:
generate days from date range
(30 answers)
Closed 5 years ago.
What is the best way in mysql to generate a series of dates in a given range?
The application I have in mind is to write a report query that returns a row for every date, regardless of whether there is any data to report. In its simplest form:
select dates.date, sum(sales.amount)
from <series of dates between X and Y> dates
left join sales on date(sales.created) = dates.date
group by 1
I have tried creating a table with lots of dates, but that seems like a poor workaround.
if you're in a situation like me where creating temporary tables is prohibited, and setting variables is also not allowed, but you want to generate a list of dates in a specific period, say current year to do some aggregation, use this
select * from
(select adddate('1970-01-01',t4*10000 + t3*1000 + t2*100 + t1*10 + t0) gen_date from
(select 0 t0 union select 1 union select 2 union select 3 union select 4 union select 5 union select 6 union select 7 union select 8 union select 9) t0,
(select 0 t1 union select 1 union select 2 union select 3 union select 4 union select 5 union select 6 union select 7 union select 8 union select 9) t1,
(select 0 t2 union select 1 union select 2 union select 3 union select 4 union select 5 union select 6 union select 7 union select 8 union select 9) t2,
(select 0 t3 union select 1 union select 2 union select 3 union select 4 union select 5 union select 6 union select 7 union select 8 union select 9) t3,
(select 0 t4 union select 1 union select 2 union select 3 union select 4 union select 5 union select 6 union select 7 union select 8 union select 9) t4) v
where gen_date between '2017-01-01' and '2017-12-31'
I think having a calendar table is a good idea; you can gain a lot of reporting and query functionality, especially when filling sparse data ranges.
I found this article with what seems to be a good example.
You may use a variable generate date series:
Set #i:=0;
SELECT DATE(DATE_ADD(X,
INTERVAL #i:=#i+1 DAY) ) AS datesSeries
FROM yourtable, (SELECT #i:=0) r
where #i < DATEDIFF(now(), date Y)
;
Not sure if this is what you have tried :) though.
Next use above generated query as a table to left join:
set #i:=0;
select
d.dates,
sum(s.amount) as TotalAmount
from(
SELECT DATE(DATE_ADD(X,
INTERVAL #i:=#i+1 DAY) ) AS dateSeries
FROM Sales, (SELECT #i:=0) r
where #i < DATEDIFF(now(), date Y)
) dates d
left join Sales s
on Date(s.Created) = Date(d.dateSeries)
group by 1
;
You can also use Temporary Table to generate date series. Check below query:
CREATE TEMPORARY TABLE daterange (dte DATE);
SET #counter := -1;
WHILE (#counter < DATEDIFF(DATE(_todate), DATE(_fromdate))) DO
INSERT daterange VALUES (DATE_ADD(_fromdate, INTERVAL #counter:=#counter + 1 DAY));
END WHILE;
SELECT dates.dte, SUM(sales.amount)
FROM daterange dates
LEFT JOIN sales ON DATE(sales.created) = dates.date
GROUP BY dates.dte;