How to select count different values by different conditions - mysql

I have issues to count data by different criteria and to display them on a chart. In the database, I have millions of entries when I try to count the total, the completed, the rejected and expired in daily, weekly and monthly bases, it doesn't show the correct answer. I have 189 on 2019-09-07 but on daily it shows 0 or the current date 2019-09-08 must be with 0 doesn't appear or a different value which doesn't match with the actual value selected directly by the workbench.
SELECT
D.dates AS dates,
IFNULL(V.total,0) AS total,
IFNULL(V.completed,0) AS complete,
IFNULL(V.rejected,0) AS rejected,
IFNULL(EX.e,0) AS expired
FROM (
SELECT *
FROM (
SELECT CURDATE() - INTERVAL (a.a + (10 * b.a) + (100 * c.a)) DAY AS dates
FROM (
SELECT 0 AS a UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL
SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL
SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9
) AS a
CROSS JOIN (
SELECT 0 AS a UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL
SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL
SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9
) AS b
CROSS JOIN (
SELECT 0 AS a UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL
SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL
SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9
) AS c
) AS generator
WHERE generator.dates BETWEEN CURDATE() - INTERVAL 90 DAY AND CURDATE()
) AS D
LEFT JOIN (
SELECT
DATE(live_date) AS live_date,
COUNT(*) as total,
SUM(CASE WHEN status = 'complete' THEN 1 ELSE 0 END) AS completed,
SUM(CASE WHEN status = 'rejected' THEN 1 ELSE 0 END) AS rejected
FROM agreement
WHERE DATE(live_date) BETWEEN CURDATE() - INTERVAL 90 DAY AND CURDATE()
GROUP BY DATE(live_date)
ORDER BY DATE(live_date) DESC
) AS V ON D.dates = V.live_date
LEFT JOIN (
SELECT DATE(expire_date) AS dEx, COUNT(*) AS e
FROM agreement
WHERE status = 'open'
AND DATE(expire_date) BETWEEN CURDATE() - INTERVAL 90 DAY AND CURDATE()
GROUP BY dEx
ORDER BY DATE(expire_date) DESC
) AS EX ON D.dates = EX.dEx;
Expecting to have the correct value for total, completed, rejected and expired in single SELECT to avoid high complexity and time to load the data instead of multiple SELECT with JOIN.

Related

SQL query for last 365 days report

I have a reports table with the following structure :
I want a SQL Query to get the report for the last 365 days by following conditions :
Group dates if the same date is repeated.
The days which the report is not available for the last 365 days, I need those days added to the result rows with 0 as their success and failed recipients.
I tried to get it by group by report dates
SELECT report_date, SUM(success_recipient) as success_recipient, SUM(failed_recipient) as failed_recipient FROM reports GROUP BY report_date;
and I have got the grouped result which satisfies the first condition
Now I need to append the rest of the days in the last 365 days to this result in which 0 as their success and failure recipients.
Expected result :
and so on ..
MYSQL VERSION : 5.6
One way to achieve this is using "with recursive" to generate all dates you need in you output and then outer join to the rest of your query. Note: I use the number 356 as it is in your description but it seems more appropriate to use date difference as this approach does not take into account leap years. Using the query below you will get NULL values in case you have no data. If you need the value 0 you can use coalesce(sum(...), 0).
with recursive
dates as (
select curdate()-356 dt
union all
select dt+1 from dates
where dt < curdate()
)
select
dt report_date,
sum(success_recipient) success_recipient,
sum(failed_recipient) failed_recipient
from dates
left join reports on report_date = dt
group by report_date;
From the above comments and the answer, I could write this query which gave me the expected outcome :
SELECT a.date, SUM(COALESCE(r.success_recipient, 0)), SUM(COALESCE(r.failed_recipient, 0))
FROM (
SELECT curdate() - INTERVAL (a.a + (10 * b.a) + (100 * c.a) + (1000 * d.a) ) DAY AS date
FROM (select 0 as a union all select 1 union all select 2 union all select 3 union all select 4 union all select 5 union all select 6 union all select 7 union all select 8 union all select 9) as a
cross join (select 0 as a union all select 1 union all select 2 union all select 3 union all select 4 union all select 5 union all select 6 union all select 7 union all select 8 union all select 9) as b
cross join (select 0 as a union all select 1 union all select 2 union all select 3 union all select 4 union all select 5 union all select 6 union all select 7 union all select 8 union all select 9) as c
cross join (select 0 as a union all select 1 union all select 2 union all select 3 union all select 4 union all select 5 union all select 6 union all select 7 union all select 8 union all select 9) as d
) a
LEFT JOIN reports r ON a.date = r.report_date
WHERE a.date between DATE_SUB(CURDATE(), INTERVAL 1 YEAR) and now()
GROUP BY a.date;

How do I create a sequence of dates?

I want to count the number of actions per day in my dataset.
date action_id
2010-01-01 id00
2010-01-03 id01
2010-01-05 id02
This is just a sample, but the point is that my data does not include actions for every day and I want to include days where there are zero actions in my result.
My plan is to do this.
with dates as (
select [sequence of dates from 2010-01-01 to 2010-02-01] as day)
select day, coalesce(count(distinct action_id), 0) as actions
from dates
left join my_table
on dates.date = my_table.date
How do I create the sequence of dates?
You example shows a CTE. So, you can use a recursive CTE:
with recursive dates as (
select date('2010-01-01') as day
union all
select day + interval 1 day
from dates
where day < '2010-02-01'
)
select d.day, count(distinct t.action_id) as actions
from dates d left join
my_table t
on d.day = my_table.date
group by d.day;
Note that COUNT() never returns NULL, so COALESCE() is unnecessary.
In older versions, you can use a calendar table or generate the data on the fly. Assuming your table has enough rows:
select d.day, count(distinct t.action_id) as actions
from (select date('2010-01-01') + interval (#rn := #rn + 1) - 1 day as day
from my_table cross join
(select #rn := 0) params
limit 31
) d left join
my_table t
on d.day = my_table.date
group by d.day;
it seems just you need group by and count
select date, count(distinct action_id) as action
from my_table left join
dates on dates.date = my_table.date
group by date
with dates as
(
select a.Date
from (
select curdate() - INTERVAL (a.a + (10 * b.a) + (100 * c.a) + (1000 * d.a) ) DAY as Date
from (select 0 as a union all select 1 union all select 2 union all select 3 union all select 4 union all select 5 union all select 6 union all select 7 union all select 8 union all select 9) as a
cross join (select 0 as a union all select 1 union all select 2 union all select 3 union all select 4 union all select 5 union all select 6 union all select 7 union all select 8 union all select 9) as b
cross join (select 0 as a union all select 1 union all select 2 union all select 3 union all select 4 union all select 5 union all select 6 union all select 7 union all select 8 union all select 9) as c
cross join (select 0 as a union all select 1 union all select 2 union all select 3 union all select 4 union all select 5 union all select 6 union all select 7 union all select 8 union all select 9) as d
) a
where a.Date between '<start_date>' and '<end_date>' )
select day, count(distinct action_id) as actions
from dates
left join my_table
on dates.date = my_table.date

mysql show count as 0 for non-existing records

Although I have researched similar other questions, however could not reach the solution by following those, hence posting my following question, and apologies for a long question in an attempt to make my question more clear.
The image shows my table structure.
I want to run such a query to extract the 3 information,i.e
userId, count(), Date(viewTime)
i.e the no of counts of id that a user has viewed on daily basis in a interval of last 14 days,
also show count as 0 if there are no records for a user on a particular day
select userId, count(userId), Date(viewTime) from user_views
where DATE(viewTime) between DATE_SUB(DATE(NOW()), INTERVAL 90 DAY) AND now()
group by userId, date(viewTime);
By using the above query I am getting only the non-zero records, see in the following image:
However I want to show count as 0 for those days when there are no transaction of users. How do I achieve this?
You need to generate the dates dynamically for this and then use left join. Also note that since you are displaying the user_id it might be needed a cross join of distinct user_id with the dynamically generated dates.
From my previous answers related to showing missing dates MySql Single Table, Select last 7 days and include empty rows
Here is one for your case
select
t1.user_id,
coalesce(t2.cnt,0) as cnt,
t1.view_date
from
(
select DATE_FORMAT(a.Date,'%Y-%m-%d') as view_date,
x.user_id
from (
select curdate() - INTERVAL (a.a + (10 * b.a) + (100 * c.a)) DAY as Date
from (select 0 as a union all select 1 union all select 2 union all select 3 union all select 4 union all select 5 union all select 6 union all select 7 union all select 8 union all select 9) as a
cross join (select 0 as a union all select 1 union all select 2 union all select 3 union all select 4 union all select 5 union all select 6 union all select 7 union all select 8 union all select 9) as b
cross join (select 0 as a union all select 1 union all select 2 union all select 3 union all select 4 union all select 5 union all select 6 union all select 7 union all select 8 union all select 9) as c
) a,(select distinct user_id from user_views)x
where a.Date between DATE_SUB(DATE(NOW()), INTERVAL 90 DAY) AND now()
)t1
left join
(
select user_id, count(user_id) as cnt, Date(view_time) as view_time from user_views
where DATE(view_time) between DATE_SUB(DATE(NOW()), INTERVAL 90 DAY) AND now()
group by user_id, date(view_time)
)t2
on t2.view_time = t1.view_date
and t1.user_id = t2.user_id
order by t1.view_date,t1.user_id
http://sqlfiddle.com/#!2/4136e/5

To find all the active users in a bucket of 15 days in SQL

I have two tables: users, user_source_history.
In table users, all the users who have registered are saved. And, in table user_source_history, their activities are stored. These activities may include log in to the system, ordering something. So one user can exist multiple times in user_source_history.
Now the problem is I need to find all the active users in the bucket of 15 days.
So, for example, a user purchased something on 2014-12-03. Now, this user will appear active user till 2014-12-18.
I need to find all the active users in date range.
So, for example, in this date range-
2014-12-03 - 10 users are active.
2014-12-04 - 10(active on 2014-12-03) + (new users for this date)
2014-12-05 - Number on active user on this date + all users in 15 day bucket
2014-12-06 - Number on active user on this date + all users in 15 day bucket
The query:
SELECT CAST(`user_last_interaction_date` as Date)
FROM `users` U
LEFT JOIN `user_source_history` USH on U.user_id = USH.user_id
WHERE `user_last_interaction_date` >
(SELECT DATE_ADD(`user_last_interaction_date`, INTERVAL -15 DAY)
FROM `users`
GROUP BY CAST(`user_last_interaction_date` as Date)
)
GROUP BY CAST(`user_last_interaction_date` as Date)
I tried this query, but SQL says Subquery returns more than one row
How can I split the query to run for more than one row??
Here is the query to find no active users from '2015-05-03' to '2015-05-18'
You would need to generate all the dates using calendar_date, which can be your date field in your database schema.
SELECT c.calendar_date, count(*) active_users
FROM `users` U
LEFT JOIN `user_source_history` USH
on( U.user_id = USH.user_id )
CROSS JOIN (select * from
(select adddate('1970-01-01',t4.i*10000 + t3.i*1000 + t2.i*100 + t1.i*10 + t0.i) `calendar_date` from
(select 0 i union select 1 union select 2 union select 3 union select 4 union select 5 union select 6 union select 7 union select 8 union select 9) t0,
(select 0 i union select 1 union select 2 union select 3 union select 4 union select 5 union select 6 union select 7 union select 8 union select 9) t1,
(select 0 i union select 1 union select 2 union select 3 union select 4 union select 5 union select 6 union select 7 union select 8 union select 9) t2,
(select 0 i union select 1 union select 2 union select 3 union select 4 union select 5 union select 6 union select 7 union select 8 union select 9) t3,
(select 0 i union select 1 union select 2 union select 3 union select 4 union select 5 union select 6 union select 7 union select 8 union select 9) t4) v
where calendar_date between '2015-05-03' and '2015-05-18') as c
on
(date(USH.user_last_interaction_date) < c.calendar_date
and date(USH.user_last_interaction_date) >= (SELECT DATE_ADD(c.calendar_date, INTERVAL -15 DAY)))
group by c.calendar_date
Here's an example query that looks for users that were active in the last 15 days:
select distinct name
from users u
join user_source_history uh
on uh.user_id = u.user_id
where user_last_interaction_date between
now() and now() - interval 15 day

Select records of previous month and show 0 if no records are there in specific time-window

Hello I have this query to get a list of drives that occurred for a specific month in a time window of 1 year back.
SELECT COUNT( drives.id ) AS drives, DATE_FORMAT( drives.timestamp, '%d-%m-%Y' ) AS mdate
FROM drives, users
WHERE drives.user = '146'
AND DATE_FORMAT( drives.timestamp, '%b' ) = 'Feb'
AND drives.timestamp > DATE_SUB(now(), INTERVAL 12 MONTH)
GROUP BY DATE(drives.timestamp) ORDER BY drives.timestamp ASC
I get the following result:
drives mdate
1 14-02-2013
2 17-02-2013
However I would like a result with every date of the month even if no records are found for that date, and display 0 next to the date that no drives took place.The tricky part for me is how to get the exact dates of the specific month in 1 year back timewindow.
I could implement this with php but I would prefer a cleaner solution.
Not sure where the users table comes into this (you are cross joining it, but not actually using it anywhere), but something like this should do what you require (not tested).
SELECT Sub1.aDay, COUNT( drives.id ) AS drives
FROM
(
SELECT DATE_ADD('2013-02-01', INTERVAL units.i + tens.i * 10 DAY) AS aDay
FROM
(SELECT 0 AS i UNION SELECT 1 UNION SELECT 2 UNION SELECT 3 UNION SELECT 4 UNION SELECT 5 UNION SELECT 6 UNION SELECT 7 UNION SELECT 8 UNION SELECT 9) units
CROSS JOIN
(SELECT 0 AS i UNION SELECT 1 UNION SELECT 2 UNION SELECT 3 UNION SELECT 4 UNION SELECT 5 UNION SELECT 6 UNION SELECT 7 UNION SELECT 8 UNION SELECT 9) tens
HAVING MONTH(aDay) = 2
) Sub1
LEFT OUTER JOIN drives
ON Sub1.aDay = DATE(drives.timestamp)
WHERE drives.user = '146'
GROUP BY DATE(drives.timestamp)
ORDER BY drives.timestamp ASC