Getting all previous records of table by date MySQL - mysql

My table currently has 21000 records, it's daily updated and almost 300 entries are inserted. Now, what I want is to have a query which will fetch the counts of elements that my table had for the previous 10 days, so it returns:
26000
21300
21000
etc
Right now, I wrote this:
"SELECT COUNT(*) from tbl_task where `task_start_time` < '2020-12-01'"
And it returns 21000 but only for 1 day. I want by query to return records according to 10 days.
However, this does it for only 1 day.
edit : database flavor is mysql and date column is date not datetime

The most efficient method may be aggregation and cumulative sums:
select date(task_start_time) as dte, count(*) as cnt_on_day,
sum(count(*)) over (order by date(task_start_time)) as running_cnt
from tbl_task
group by dte
order by dte desc
limit 10;
This returns the last 10 days in the data. You can easily adjust to more days if you like -- in fact all of them -- without much trouble.

I don't know if I'm wrong, but could you not simple add a GROUP BY - statement? Like:
"SELECT COUNT(*) from tbl_task where `task_start_time` < '2020-12-01' GROUP
BY task_start_time"
EDIT:
This should only work if task_start_time is a date, not if it is a datetime
EDIT2:
If it is a datetime you could use the date function:
SELECT COUNT(*) from tbl_task where `task_start_time` < '2020-12-01' GROUP
BY DATE(task_start_time)

You can use UNION ALL and date arithmetic.
SELECT count(*)
FROM tbl_task
WHERE task_start_time < current_date
UNION ALL
SELECT count(*)
FROM tbl_task
WHERE task_start_time < date_sub(current_date, INTERVAL 1 DAY)
...
UNION ALL
SELECT count(*)
FROM tbl_task
WHERE task_start_time < date_sub(current_date, INTERVAL 9 DAY);
Edit:
You might also join a derived table that uses FROM-less SELECTs and UNION ALL to get the days to look back and then aggregate. This might be a little easier to construct dynamically. (But it may be slower I suspect.)
SELECT count(*)
FROM (SELECT 0 x
UNION ALL
SELECT 1
...
UNION ALL
SELECT 9)
INNER JOIN tbl_task t
ON t.task_start_time < date_sub(current_date, INTERVAL x.x DAY)
GROUP BY x.x;
In MySQL version 8+ you can even use a recursive CTE to construct the table with the days.
WITH RECURSIVE x
AS
(
SELECT 0 x
UNION ALL
SELECT x + 1
FROM x
WHERE x + 1 < 10
)
SELECT count(*)
FROM x
INNER JOIN tbl_task t
ON t.task_start_time < date_sub(current_date, INTERVAL x.x DAY)
GROUP BY x.x;

Related

MySQL SELECT all rows between date time with interval

I have a column in my sql table called loggedTime which is a datetime field and I want to select between two dates startDate and endDate along with the interval may be 5 minutes, 10 minutes, 1 hour etc. I tried to write the SQL query but it says You have syntax error next interval, I am not sure what wrong with my query. If I remove INTERVAL 5 MINUTE my query works fine but I want to pass the Interval along with the date so it will select all rows between two dates and also with interval
Here is SQL
SELECT * FROM mytable WHERE loggedTime BETWEEN '2021-06-01' and '2021-06-03' INTERVAL 5 MINUTE
If you have any unique consecutively increasing column like id, then you can use an INNER JOIN as done followingly:
SELECT *
FROM mytable a
INNER JOIN mytable b
ON a.ID = b.ID + 1
WHERE TIMESTAMPDIFF(minute, a.timestamp, b.timestamp) = 5;
If you do not have that column in your table then use this code :
SELECT *
FROM (SELECT mt.*,
TIMESTAMPDIFF(minute, #prevTS, `loggedTime`) AS timeinterval,
#prevTS:=mt.`loggedTime`
FROM mytable mt,
(SELECT #prevTS := (SELECT MIN(`loggedTime`)
FROM yourTable)) vars
ORDER BY ID)subquery_alias
WHERE loggedTime BETWEEN '2021-06-01' AND '2021-06-03'
AND timeinterval = 5
Check this thread as reference too.

How ot return 0 instead of null on mysql query?

The following query returns the visitors and pageviews of last 7 days. However, if there are no results (let's say it is a fresh account), nothing is returned.
How to edit this in order to return 0 in days that there are no entries?
SELECT Date(timestamp) AS day,
Count(DISTINCT hash) AS visitors,
Count(*) AS pageviews
FROM behaviour
WHERE company_id = 1
AND timestamp >= Subdate(Curdate(), 7)
GROUP BY day
Assuming that you always have at least one record in the table for each of the last 7 days (regardless of the company_id), then you can use conditional aggregation as follows:
select
date(timestamp) as day,
count(distinct case when company_id = 1 then hash end) as visitors,
sum(company_id = 1) as pageviews
from behaviour
where timestamp >= curdate() - interval 7 day
group by day
Note that I changed you query to use standard date arithmetics, which I find easier to understand that date functions.
Otherwise, you would need to move the condition on the date from the where clause to the aggregate functions:
select
date(timestamp) as day,
count(distinct case when timestamp >= curdate() - interval 7 day and company_id = 1 then hash end) as visitors,
sum(timestamp >= curdate() - interval 7 day and company_id = 1) as pageviews
from behaviour
group by day
If your table is big, this can be expensive so I would not recommend that.
Alternatively, you can generate a derived table of dates and left join it with your original query:
select
curdate - interval x.n day day,
count(distinct b.hash) visitors,
count(b.hash) page_views
from (
select 1 n union all select 2 union all select 3 union all select 4
union all select 5 union all select 6 union all select 7
) x
left join behavior b
on b.company_id = 1
and b.timestamp >= curdate() - interval x.n day
and b.timestamp < curdate() - interval (x.n - 1) day
group by x.n
Use a query that returns all the dates from today minus 7 days to today and left join the table behaviour:
SELECT t.timestamp AS day,
Count(DISTINCT b.hash) AS visitors,
Count(b.timestamp) AS pageviews
FROM (
SELECT Subdate(Curdate(), 7) timestamp UNION ALL SELECT Subdate(Curdate(), 6) UNION ALL
SELECT Subdate(Curdate(), 5) UNION ALL SELECT Subdate(Curdate(), 4) UNION ALL SELECT Subdate(Curdate(), 3) UNION ALL
SELECT Subdate(Curdate(), 2) UNION ALL SELECT Subdate(Curdate(), 1) UNION ALL SELECT Curdate()
) t LEFT JOIN behaviour b
ON Date(b.timestamp) = t.timestamp AND b.company_id = 1
GROUP BY day
Use IFNULL:
IFNULL(expr1, 0)
From the documentation:
If expr1 is not NULL, IFNULL() returns expr1; otherwise it returns expr2. IFNULL() returns >a numeric or string value, depending on the context in which it is used.
You can use next trick:
First, get query that return 1 dummy row: SELECT 1;
Next use LEFT JOIN to connect summary row(s) without condition. This join will return values in case data exists on NULL values in other case.
Last select from joined queries onle what we need and convert NULL's to ZERO's
using IFNULL dunction.
SELECT
IFNULL(b.day,0) AS DAY,
IFNULL(b.visitors,0) AS visitors,
IFNULL(b.pageviews,0) AS pageviews
FROM (
SELECT 1
) a
LEFT JOIN (
SELECT DATE(TIMESTAMP) AS DAY,
COUNT(DISTINCT HASH) AS visitors,
COUNT(*) AS pageviews
FROM behaviour
WHERE company_id = 1
AND TIMESTAMP >= SUBDATE(CURDATE(), 7)
GROUP BY DAY
) b ON 1 = 1;

Find number of rows for each hour where datetime columns match certain criteria

RDBMS: MySQL
The time column(s) datatype is of datetime
For every hour of the 24 hour day I need to retrieve the number of rows in which their start_time matches the hour OR the end_time is great than or equal to the hour.
Below is the current query I have which returns the data I need but only based off of one hour. I can loop through and do 24 separate queries for each hour of the day but I would love to have this in one query.
SELECT COUNT(*) as total_online
FROM broadcasts
WHERE DATE(start_time) = '2018-01-01' AND (HOUR(start_time) = '0' OR
HOUR(end_time) >= '0')
Is there a better way of querying the data I need? Perhaps by using group by somehow? Thank you.
Not exactly sure if i am following, but try something like this:
select datepart(hh, getdate()) , count(*)
from broadcasts
where datepart(hh, starttime) <=datepart(hh, endtime)
and cast(starttime as date)=cast(getdate() as date) and cast(endtime as date)=cast(getdate() as date)
group by datepart(hh, getdate())
Join with a subquery that returns all the hour numbers:
SELECT h.hour_num, COUNT(*) AS total_online
FROM (SELECT 0 AS hour_num UNION SELECT 1 UNION SELECT 2 ... UNION SELECT 23) AS h
JOIN broadcasts AS b ON HOUR(b.start_time) = h.hour_num OR HOUR(b.end_time) >= h.hour_num
WHERE DATE(b.start_time) = '2018-01-01'
GROUP BY h.hour_num

MySQL gett rows from last week with their day name

I am having trouble with a query. This is taken from a similar query where i count number of rows per month.
I want to count all rows for each day of the last week and display the day name and a count. If there are no rows for that day, display zero.
I know the UNIONS won't work but i don't know what to replace it with.
At the moment it get the last 7 days but the day name is NULL
SELECT DAYNAME(STR_TO_DATE(Days.ID, '%a')) AS `day`, COUNT(`returns`.list_date) AS `total`
FROM
(
SELECT 1 as ID UNION SELECT 2 as ID UNION SELECT 3 as ID UNION SELECT 4 as ID
UNION
SELECT 5 as ID UNION SELECT 6 as ID UNION SELECT 7 as ID
) as Days
WHERE (list_date >= DATE_SUB(NOW(), INTERVAL 1 WEEK))
GROUP BY Days.id
UPDATE:
I have created a SQL fiddle showing the code output from #Gordon Linoff answer below which doesn't get the counted rows
http://sqlfiddle.com/#!9/969463/1
One method for doing what you want is a LEFT JOIN:
SELECT DAYNAME(DATE_SUB(CURDATE(), INTERVAL Days.n DAY)) AS `day`,
COUNT(r.list_date) AS `total`
FROM (SELECT 1 as n UNION ALL SELECT 2 as n UNION ALL
SELECT 3 as n UNION ALL SELECT 4 as n UNION ALL
SELECT 5 as n UNION ALL SELECT 6 as n UNION ALL
SELECT 7 as n
) Days LEFT JOIN
returns r
ON r.list_date = DATE_SUB(CURDATE(), INTERVAL Days.n DAY))
GROUP BY Days.n
ORDER BY Days.n;
Some notes:
Use UNION ALL instead of UNION, unless you have a good reason for incurring the overhead of removing duplicates.
This assumes that returns.list_date is actually a date, because it uses = rather than >=.
The use of now() and >= is a bit confusing, because now() has a time component, which you generally want to ignore.

Return a zero for a day with no results

I have a query which returns the total of users who registered for each day. Problem is if a day had no one register it doesn't return any value, it just skips it. I would rather it returned zero
this is my query so far
SELECT count(*) total FROM users WHERE created_at < NOW() AND created_at >
DATE_SUB(NOW(), INTERVAL 7 DAY) AND owner_id = ? GROUP BY DAY(created_at)
ORDER BY created_at DESC
Edit
i grouped the data so i would get a count for each day- As for the date range, i wanted the total users registered for the previous seven days
A variation on the theme "build your on 7 day calendar inline":
SELECT D, count(created_at) AS total FROM
(SELECT DATE_SUB(NOW(), INTERVAL D DAY) AS D
FROM
(SELECT 0 as D
UNION SELECT 1
UNION SELECT 2
UNION SELECT 3
UNION SELECT 4
UNION SELECT 5
UNION SELECT 6
) AS D
) AS D
LEFT JOIN users ON date(created_at) = date(D)
WHERE owner_id = ? or owner_id is null
GROUP BY D
ORDER BY D DESC
I don't have your table structure at hand, so that would need adjustment probably. In the same order of idea, you will see I use NOW() as a reference date. But that's easily adjustable. Anyway that's the spirit...
See for a live demo http://sqlfiddle.com/#!2/ab5cf/11
If you had a table that held all of your days you could do a left join from there to your users table.
SELECT SUM(CASE WHEN U.Id IS NOT NULL THEN 1 ELSE 0 END)
FROM DimDate D
LEFT JOIN Users U ON CONVERT(DATE,U.Created_at) = D.DateValue
WHERE YourCriteria
GROUP BY YourGroupBy
The tricky bit is that you group by the date field in your data, which might have 'holes' in it, and thus miss records for that date.
A way to solve it is by filling a table with all dates for the past 10 and next 100 years or so, and to (outer)join that to your data. Then you will have one record for each day (or week or whatever) for sure.
I had to do this only for MS SqlServer, so how to fill a date table (or perhaps you can do it dynamically) is for someone else to answer.
A bit long winded, but I think this will work...
SELECT count(users.created_at) total FROM
(SELECT DATE_SUB(CURDATE(),INTERVAL 6 DAY) as cdate UNION ALL
SELECT DATE_SUB(CURDATE(),INTERVAL 5 DAY) UNION ALL
SELECT DATE_SUB(CURDATE(),INTERVAL 4 DAY) UNION ALL
SELECT DATE_SUB(CURDATE(),INTERVAL 3 DAY) UNION ALL
SELECT DATE_SUB(CURDATE(),INTERVAL 2 DAY) UNION ALL
SELECT DATE_SUB(CURDATE(),INTERVAL 1 DAY) UNION ALL
SELECT CURDATE()) t1 left join users
ON date(created_at)=t1.cdate
WHERE owner_id = ? or owner_id is null
GROUP BY t1.cdate
ORDER BY t1.cdate DESC
It differs from your query slightly in that it works on dates rather than date times which your query is doing. From your description I have assumed you mean to use whole days and therefore have used dates.