I want to count the number of actions per day in my dataset.
date action_id
2010-01-01 id00
2010-01-03 id01
2010-01-05 id02
This is just a sample, but the point is that my data does not include actions for every day and I want to include days where there are zero actions in my result.
My plan is to do this.
with dates as (
select [sequence of dates from 2010-01-01 to 2010-02-01] as day)
select day, coalesce(count(distinct action_id), 0) as actions
from dates
left join my_table
on dates.date = my_table.date
How do I create the sequence of dates?
You example shows a CTE. So, you can use a recursive CTE:
with recursive dates as (
select date('2010-01-01') as day
union all
select day + interval 1 day
from dates
where day < '2010-02-01'
)
select d.day, count(distinct t.action_id) as actions
from dates d left join
my_table t
on d.day = my_table.date
group by d.day;
Note that COUNT() never returns NULL, so COALESCE() is unnecessary.
In older versions, you can use a calendar table or generate the data on the fly. Assuming your table has enough rows:
select d.day, count(distinct t.action_id) as actions
from (select date('2010-01-01') + interval (#rn := #rn + 1) - 1 day as day
from my_table cross join
(select #rn := 0) params
limit 31
) d left join
my_table t
on d.day = my_table.date
group by d.day;
it seems just you need group by and count
select date, count(distinct action_id) as action
from my_table left join
dates on dates.date = my_table.date
group by date
with dates as
(
select a.Date
from (
select curdate() - INTERVAL (a.a + (10 * b.a) + (100 * c.a) + (1000 * d.a) ) DAY as Date
from (select 0 as a union all select 1 union all select 2 union all select 3 union all select 4 union all select 5 union all select 6 union all select 7 union all select 8 union all select 9) as a
cross join (select 0 as a union all select 1 union all select 2 union all select 3 union all select 4 union all select 5 union all select 6 union all select 7 union all select 8 union all select 9) as b
cross join (select 0 as a union all select 1 union all select 2 union all select 3 union all select 4 union all select 5 union all select 6 union all select 7 union all select 8 union all select 9) as c
cross join (select 0 as a union all select 1 union all select 2 union all select 3 union all select 4 union all select 5 union all select 6 union all select 7 union all select 8 union all select 9) as d
) a
where a.Date between '<start_date>' and '<end_date>' )
select day, count(distinct action_id) as actions
from dates
left join my_table
on dates.date = my_table.date
Related
I have two tables user_profile and tracked_search. The user_profile table has user details and tracked_search tracks searches made by each user.
Whenever a user makes a search this search entry goes in the tracked_search table. If nothing is searched for a particular date nothing is added in tracked_search.
I need to develop a report where in I need to show on all days of month how many users made searches.
For example:
CREATE TABLE tracked_search (
id int NOT NULL AUTO_INCREMENT PRIMARY KEY,
created DATE,
user_id int NOT NULL
);
INSERT INTO tracked_search(created, user_id) VALUES
('2017-10-01', 1000),
('2017-10-01', 1000),
('2017-10-01', 2000),
('2017-10-01', 3000),
('2017-10-01', 4000),
('2017-10-04', 1000),
('2017-10-04', 2000),
('2017-10-04', 2000),
('2017-10-04', 2000),
('2017-10-04', 2000),
('2017-10-04', 3000),
('2017-10-31', 1000),
('2017-10-31', 2000),
('2017-10-31', 3000),
('2017-10-31', 4000),
('2017-10-31', 5000);
Desired output:
Date user_count
2017-10-01 4
2017-10-02 0
2017-10-03 0
2017-10-04 3
2017-10-05 0
...
2017-10-30 0
2017-10-31 5
I have written following query
SELECT ts.created , count( distinct ts.user_id) FROM tracked_search ts, user_profile u
WHERE ts.created>=(CURDATE()-INTERVAL 1 MONTH) AND u.id = ts.user_id
group by ts.created;
but i get
Date user_count
2017-10-01 4
2017-10-04 3
2017-10-31 5
I need to print all days values if no entry is there for a particular date it should be zero.
I am using MySQL.
By the way, you don't need the join on user_profile.
If you have a dates table with the relevant dates, this is pretty easy:
SELECT dates.day AS `Date`, COUNT(DISTINCT ts.user_id) AS user_count
FROM dates
LEFT OUTER JOIN tracked_search AS ts
ON ts.created = dates.day
GROUP BY dates.day;
Since you probably don't have a dates table and might not want to create and maintain one, you could use one of the solutions for generating the list of dates on the fly. e.g. Get a list of dates between two dates or How to get list of dates between two dates in mysql select query
SELECT dates.day AS `Date`, COUNT(DISTINCT ts.user_id) AS user_count
FROM (
SELECT ADDDATE('1970-01-01', t4.i * 10000 + t3.i * 1000 + t2.i * 100 + t1.i * 10 + t0.i) AS day
FROM (SELECT 0 AS i UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) AS t0,
(SELECT 0 AS i UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) AS t1,
(SELECT 0 AS i UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) AS t2,
(SELECT 0 AS i UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) AS t3,
(SELECT 0 AS i UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) AS t4
) AS dates
LEFT OUTER JOIN tracked_search AS ts
ON ts.created = dates.day
WHERE dates.day >= '2017-10-01'
AND dates.day < '2017-11-01'
GROUP BY dates.day;
You need to write without AND u.id = ts.user_id.
SELECT ts.created , count( distinct ts.user_id) FROM tracked_search ts, user_profile u
WHERE ts.created>=(CURDATE()-INTERVAL 1 MONTH)
group by ts.created;
I was able to solve this using the following logic hope it will help someone
select
t1.attempt_date,
coalesce(SUM(t1.attempt_count+t2.attempt_count), 0) AS attempt_count
from
(
select DATE_FORMAT(a.Date,'%Y/%m/%d') as attempt_date,
'0' as attempt_count
from (
select curdate() - INTERVAL (a.a + (10 * b.a) + (100 * c.a)) DAY as Date
from (select 0 as a union all select 1 union all select 2 union all select 3 union all select 4 union all select 5 union all select 6 union all select 7 union all select 8 union all select 9) as a
cross join (select 0 as a union all select 1 union all select 2 union all select 3 union all select 4 union all select 5 union all select 6 union all select 7 union all select 8 union all select 9) as b
cross join (select 0 as a union all select 1 union all select 2 union all select 3 union all select 4 union all select 5 union all select 6 union all select 7 union all select 8 union all select 9) as c
) a
where a.Date BETWEEN NOW() - INTERVAL 1 MONTH AND NOW()
)t1
left join
(
SELECT DATE_FORMAT(ts.created,'%Y/%m/%d') AS attempt_date,
count( distinct ts.user_id) AS attempt_count
FROM tracked_search ts, user_profile u
WHERE ts.user_id = u.id and
DATE_SUB(ts.created, INTERVAL 1 DAY) > DATE_SUB(DATE(NOW()), INTERVAL 1 MONTH)
GROUP BY DAY(ts.created) DESC
)t2
on t2.attempt_date = t1.attempt_date
group by DAY(t1.attempt_date)
order by t1.attempt_date desc;
I have a MySql table containing events having a DATETIME timestamp. I want to count each day's events. On some days, e.g. on Sundays, events are missing. The result should contain these days too with a count of zero.
My query is the following:
SELECT
COUNT(1) AS mycount,
DATE_FORMAT(DATE(evaluations.timestamp),"%a, %d.%m.%Y") AS date
FROM Events
GROUP BY DATE(timestamp)
ORDER BY DATE(timestamp) DESC
Can I modify the query without using a helper table containing all dates?
A single query (no procedere, no function) would be fine.
The query would somehow look like this if you don't have any calendar table:
SELECT
dateTable.day,
COALESCE(t.mycount,0) AS cnt
FROM
(
SELECT ADDDATE((SELECT MIN(DATE(timestamp)) FROM Events), INTERVAL #i:=#i+1 DAY) AS DAY
FROM (
SELECT a.a
FROM (SELECT 0 AS a UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) AS a
CROSS JOIN (SELECT 0 AS a UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) AS b
CROSS JOIN (SELECT 0 AS a UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) AS c
) a
JOIN (SELECT #i := -1) r1
WHERE
#i < DATEDIFF((SELECT MAX(DATE(timestamp)) FROM Events), (SELECT MIN(DATE(timestamp)) FROM Events))
) AS dateTable
LEFT JOIN
(
SELECT
COUNT(1) AS mycount,
DATE_FORMAT(DATE(evaluations.timestamp),"%a, %d.%m.%Y") AS date
FROM Events
GROUP BY DATE(timestamp)
ORDER BY DATE(timestamp) DESC
) AS t
ON dateTable.day = t.date
ORDER BY dateTable.day DESC;
Note:
If you think you will need this kind of query too often then you can create a table where all the dates would reside. Newer dates can be added through mysql event periodically .
Then the work is simple. Just need to make a LEFT JOIN between the calendar table and the result of your query.
I need zero values also in group by clause .Have read almost all question related to this on Stackoverflow, but none of the solutions have worked.
My Table is
Need to get sum of score grouped by day of month.But I am not getting zero against the days not present in the table
SELECT SUM(engagement_score), DAY(creation_dt)
FROM qee_emp_engagement_index
RIGHT JOIN (
SELECT 1 AS index1 UNION ALL
SELECT 2 UNION ALL
SELECT 3 UNION ALL
SELECT 4 UNION ALL
SELECT 5 UNION ALL
SELECT 6 UNION ALL
SELECT 7 UNION ALL
SELECT 8 UNION ALL
SELECT 9) AS a ON a.index1 = DAY(creation_dt)
WHERE org_id = 1
GROUP BY a.index1
ORDER BY a.index1 ASC
I would write the query using a left join rather than a right join (the logic of left join makes more sense to me: keep all the rows in the first table). But, your problem is the where clause. That logic should go in the on clause:
SELECT COALESCE(SUM(eei.engagement_score), 0), a.index1
FROM (SELECT 1 AS index1 UNION ALL
SELECT 2 UNION ALL
SELECT 3 UNION ALL
SELECT 4 UNION ALL
SELECT 5 UNION ALL
SELECT 6 UNION ALL
SELECT 7 UNION ALL
SELECT 8 UNION ALL
SELECT 9
) a LEFT JOIN
qee_emp_engagement_index eei
ON a.index1 = DAY(eei.creation_dt) AND eei.org_id = 1
GROUP BY a.index1
ORDER BY a.index1 ASC;
In addition, the GROUP BY and SELECT should use the column from the driving table.
You can do right join between your table and a temp table containing all dates in a range.
Dates can be generated using below query
select a.Date
from (
select curdate() - INTERVAL (a.a + (10 * b.a) + (100 * c.a)) DAY as Date
from (select 0 as a union all select 1 union all select 2 union all select 3 union all select 4 union all select 5 union all select 6 union all select 7 union all select 8 union all select 9) as a
cross join (select 0 as a union all select 1 union all select 2 union all select 3 union all select 4 union all select 5 union all select 6 union all select 7 union all select 8 union all select 9) as b
cross join (select 0 as a union all select 1 union all select 2 union all select 3 union all select 4 union all select 5 union all select 6 union all select 7 union all select 8 union all select 9) as c
) a
where a.Date between '2015-01-01' and '2017-01-01'
SELECT DATE(creation_date), SUM(engagement_score) FROM table_name GROUP BY DATE(creation_date)
There is no need to do that weird right join in your code
If you want to group it by the day of month and not by the date, use DAY() instead of DATE()
EDIT:
If you want a zero when the day of month doesn't exist, try this:
SELECT d, SUM(score) FROM
((SELECT creation_date AS d, engagement_score AS score FROM table_name)
UNION ALL
(SELECT d, 0 AS engagement_score FROM table_name WHERE d BETWEEN 1 AND 31))
GROUP BY d
I have two tables: users, user_source_history.
In table users, all the users who have registered are saved. And, in table user_source_history, their activities are stored. These activities may include log in to the system, ordering something. So one user can exist multiple times in user_source_history.
Now the problem is I need to find all the active users in the bucket of 15 days.
So, for example, a user purchased something on 2014-12-03. Now, this user will appear active user till 2014-12-18.
I need to find all the active users in date range.
So, for example, in this date range-
2014-12-03 - 10 users are active.
2014-12-04 - 10(active on 2014-12-03) + (new users for this date)
2014-12-05 - Number on active user on this date + all users in 15 day bucket
2014-12-06 - Number on active user on this date + all users in 15 day bucket
The query:
SELECT CAST(`user_last_interaction_date` as Date)
FROM `users` U
LEFT JOIN `user_source_history` USH on U.user_id = USH.user_id
WHERE `user_last_interaction_date` >
(SELECT DATE_ADD(`user_last_interaction_date`, INTERVAL -15 DAY)
FROM `users`
GROUP BY CAST(`user_last_interaction_date` as Date)
)
GROUP BY CAST(`user_last_interaction_date` as Date)
I tried this query, but SQL says Subquery returns more than one row
How can I split the query to run for more than one row??
Here is the query to find no active users from '2015-05-03' to '2015-05-18'
You would need to generate all the dates using calendar_date, which can be your date field in your database schema.
SELECT c.calendar_date, count(*) active_users
FROM `users` U
LEFT JOIN `user_source_history` USH
on( U.user_id = USH.user_id )
CROSS JOIN (select * from
(select adddate('1970-01-01',t4.i*10000 + t3.i*1000 + t2.i*100 + t1.i*10 + t0.i) `calendar_date` from
(select 0 i union select 1 union select 2 union select 3 union select 4 union select 5 union select 6 union select 7 union select 8 union select 9) t0,
(select 0 i union select 1 union select 2 union select 3 union select 4 union select 5 union select 6 union select 7 union select 8 union select 9) t1,
(select 0 i union select 1 union select 2 union select 3 union select 4 union select 5 union select 6 union select 7 union select 8 union select 9) t2,
(select 0 i union select 1 union select 2 union select 3 union select 4 union select 5 union select 6 union select 7 union select 8 union select 9) t3,
(select 0 i union select 1 union select 2 union select 3 union select 4 union select 5 union select 6 union select 7 union select 8 union select 9) t4) v
where calendar_date between '2015-05-03' and '2015-05-18') as c
on
(date(USH.user_last_interaction_date) < c.calendar_date
and date(USH.user_last_interaction_date) >= (SELECT DATE_ADD(c.calendar_date, INTERVAL -15 DAY)))
group by c.calendar_date
Here's an example query that looks for users that were active in the last 15 days:
select distinct name
from users u
join user_source_history uh
on uh.user_id = u.user_id
where user_last_interaction_date between
now() and now() - interval 15 day
This question already has answers here:
generate days from date range
(30 answers)
Closed 5 years ago.
What is the best way in mysql to generate a series of dates in a given range?
The application I have in mind is to write a report query that returns a row for every date, regardless of whether there is any data to report. In its simplest form:
select dates.date, sum(sales.amount)
from <series of dates between X and Y> dates
left join sales on date(sales.created) = dates.date
group by 1
I have tried creating a table with lots of dates, but that seems like a poor workaround.
if you're in a situation like me where creating temporary tables is prohibited, and setting variables is also not allowed, but you want to generate a list of dates in a specific period, say current year to do some aggregation, use this
select * from
(select adddate('1970-01-01',t4*10000 + t3*1000 + t2*100 + t1*10 + t0) gen_date from
(select 0 t0 union select 1 union select 2 union select 3 union select 4 union select 5 union select 6 union select 7 union select 8 union select 9) t0,
(select 0 t1 union select 1 union select 2 union select 3 union select 4 union select 5 union select 6 union select 7 union select 8 union select 9) t1,
(select 0 t2 union select 1 union select 2 union select 3 union select 4 union select 5 union select 6 union select 7 union select 8 union select 9) t2,
(select 0 t3 union select 1 union select 2 union select 3 union select 4 union select 5 union select 6 union select 7 union select 8 union select 9) t3,
(select 0 t4 union select 1 union select 2 union select 3 union select 4 union select 5 union select 6 union select 7 union select 8 union select 9) t4) v
where gen_date between '2017-01-01' and '2017-12-31'
I think having a calendar table is a good idea; you can gain a lot of reporting and query functionality, especially when filling sparse data ranges.
I found this article with what seems to be a good example.
You may use a variable generate date series:
Set #i:=0;
SELECT DATE(DATE_ADD(X,
INTERVAL #i:=#i+1 DAY) ) AS datesSeries
FROM yourtable, (SELECT #i:=0) r
where #i < DATEDIFF(now(), date Y)
;
Not sure if this is what you have tried :) though.
Next use above generated query as a table to left join:
set #i:=0;
select
d.dates,
sum(s.amount) as TotalAmount
from(
SELECT DATE(DATE_ADD(X,
INTERVAL #i:=#i+1 DAY) ) AS dateSeries
FROM Sales, (SELECT #i:=0) r
where #i < DATEDIFF(now(), date Y)
) dates d
left join Sales s
on Date(s.Created) = Date(d.dateSeries)
group by 1
;
You can also use Temporary Table to generate date series. Check below query:
CREATE TEMPORARY TABLE daterange (dte DATE);
SET #counter := -1;
WHILE (#counter < DATEDIFF(DATE(_todate), DATE(_fromdate))) DO
INSERT daterange VALUES (DATE_ADD(_fromdate, INTERVAL #counter:=#counter + 1 DAY));
END WHILE;
SELECT dates.dte, SUM(sales.amount)
FROM daterange dates
LEFT JOIN sales ON DATE(sales.created) = dates.date
GROUP BY dates.dte;