I need to do a query and join with all days of the year but in my db there isn't a calendar table.
After google-ing I found generate_series() in PostgreSQL. Does MySQL have anything similar?
My actual table has something like:
date qty
1-1-11 3
1-1-11 4
4-1-11 2
6-1-11 5
But my query has to return:
1-1-11 7
2-1-11 0
3-1-11 0
4-1-11 2
and so on ..
This is how I do it. It creates a range of dates from 2011-01-01 to 2011-12-31:
select
date_format(
adddate('2011-1-1', #num:=#num+1),
'%Y-%m-%d'
) date
from
any_table,
(select #num:=-1) num
limit
365
-- use limit 366 for leap years if you're putting this in production
The only requirement is that the number of rows in any_table should be greater or equal to the size of the needed range (>= 365 rows in this example). You will most likely use this as a subquery of your whole query, so in your case any_table can be one of the tables you use in that query.
Enhanced version of solution from #Karolis that ensures it works for any year (including leap years):
select date from (
select
date_format(
adddate('2011-1-1', #num:=#num+1),
'%Y-%m-%d'
) date
from
any_table,
(select #num:=-1) num
limit
366
) as dt
where year(date)=2011
I was looking to this solution but without the "hardcoded" date, and I came-up with this one valid for the current year(helped from this answers).
Please note the
where year(date)=2011
is not needed as the select already filter the date. Also this way, it does not matter which table(at least as stated before the table has at least 366 rows) is been used, as date is "calculated" on runtime.
select date from (
select
date_format(
adddate(MAKEDATE(year(now()),1), #num:=#num+1),
'%Y-%m-%d'
) date
from
your_table,
(select #num:=-1) num
limit
366 ) as dt
Just in case someone is looking for generate_series() to generate a series of dates or ints as a temp table in MySQL.
With MySQL8 (MySQL version 8.0.27) you can do something like this to simulate:
WITH RECURSIVE nrows(date) AS (
SELECT MAKEDATE(2021,333) UNION ALL
SELECT DATE_ADD(date,INTERVAL 1 day) FROM nrows WHERE date<=CURRENT_DATE
)
SELECT date FROM nrows;
Result:
2021-11-29
2021-11-30
2021-12-01
2021-12-02
2021-12-03
2021-12-04
2021-12-05
2021-12-06
Related
I have a temp_table with user_id and date and I want to find the DAU , WAU and MAU and I am querying this for that where:
DAU - Count of active users for that day
WAU - Count of active users in last 7 days
MAU - Count of active users in last 30 days
where the date is starting from a date that is mentioned here , so there can be no current_date comparison.
dau as (Select casted_date, count(user_id) as dau
from temp table
group by casted_date)
select casted date, dau,
sum(dau) over (order by casted_date rows between -6 preceding and current row) as wau,
sum(dau) over (order by casted_date rows between -29 preceding and current row) as mau
from dau;
but the query is giving me an error like this :
syntax error at or near "-".
PS: I am writing the query in mysql
I don't know if your query logic be completely correct, but the syntax error you are currently seeing is due to the window function calls. Consider this corrected version:
sum(dau) over (order by casted_date rows between 6 preceding and current row) as wau,
sum(dau) over (order by casted_date rows between 29 preceding and current row) as mau
There is no need to use -6 to refer to the previous 6 rows, as 6 preceding already means this.
First, you have a syntax error, you have casted date where you should have casted_date and I would not use an alias of date either which happens to be a MySQL keyword without escaping it.
You can use case-when to achieve your goal:
select
td.casted_date,
count(distinct td.id) as DAU,
count(distinct tw.id) as WAU,
count(distinct tm.id) as MAU
from temp td
left join temp tw
on tw.casted_date between date_sub(td.casted_date, interval 7 day) and td.casted_date
left join temp tm
on tm.casted_date between date_sub(td.casted_date, interval 30 day) and td.casted_date
group by td.casted_date;
Tested with this schema:
create table temp(
id int primary key auto_increment,
casted_date date
);
insert into temp(casted_date)
values
('2020-02-07'),
('2020-02-07'),
('2020-02-07'),
('2020-02-06'),
('2020-02-06'),
('2020-02-06'),
('2020-01-16'),
('2020-01-16'),
('2020-01-16');
Fiddle can be found here: http://sqlfiddle.com/#!9/441aaa/10
Xou can use biuild in date functions to parttion the window functions
create table temp(
id int primary key auto_increment,
casted_date date,
user_id int
);
insert into temp(casted_date,user_id)
values
('2020-02-07',1),
('2020-02-07',2),
('2020-02-07',3),
('2020-02-06',1),
('2020-02-06',2),
('2020-02-06',4),
('2020-01-16',1),
('2020-01-16',2),
('2020-01-16',1);
Records: 9 Duplicates: 0 Warnings: 0
WITH
dau as (Select casted_date, count(user_id) as dau
from temp
group by casted_date)
select casted_date, dau,
sum(dau) over (PARTITION BY YEAR(casted_date) ,WEEK(casted_date) order by casted_date ) as wau,
sum(dau) over (PARTITION BY YEAR(casted_date),MONTH(casted_date) order by casted_date ) as mau
from dau
ORDER BY casted_date ASC;
casted_date
dau
wau
mau
2020-01-16
3
3
3
2020-02-06
3
3
3
2020-02-07
3
6
6
fiddle
What I am trying to do is to get the record count for each day of the last 7 days,
Let's say I have 3 records today, 4 records yesterday, 2 records two days ago, etc.
I'd like to have something like that:
[12/06/2021] 1
[11/06/2021] 4
[10/06/2021] 3
[09/06/2021] 6
[08/06/2021] 7
[07/06/2021] 2
[06/06/2021] 7
(Or get only the count, it's OK too.)
I have a field - message_datetime that saves the datetime.
Is there a way to do this in one query?
What I've done:
select CAST(message_datetime AS DATE),count(message_datetime) from messages group by CAST(message_datetime AS DATE) WHERE message_datetime
It worked but I wanted the last 7 days. Thanks
I was waiting for you to post your own effort, but since somebody has already "jumped the gun" and started to post answers, so:
Assuming the name of the table is my_table, then try:
select date(message_datetime) as message_date, count(*) as cnt from my_table
where datediff(curdate(), date(message_datetime)) < 7
group by date(message_datetime)
order by date(message_datetime) desc
Update
Following the Strawberry's suggestion, here is an updated query that should be more performant if message_datetime is an indexed column:
select date(message_datetime) as message_date, count(*) as cnt from my_table
where message_date_time >= date_sub(curdate(), interval 6 day)
group by date(message_datetime)
order by date(message_datetime) desc
i have this table
desc t1;
with this datas
mysql> select* from t1;
now i do this
mysql> select day_date,count(day_date) from t1 group by (date(day_date));
I'm trying to fetch the records with half an hour time interval of the current day with concern data count for that time period.
So, my output came as expected. But, If count(no records) on the particular time period let's say 7:00 - 7:30 I'm not getting that record with zero count.
My Query as follows :
SELECT time_format( FROM_UNIXTIME(ROUND(UNIX_TIMESTAMP(start_time)/(30* 60)) * (30*60)) , '%H:%i')
thirtyHourInterval , COUNT(bot_id) AS Count FROM bot_activity
WHERE start_time BETWEEN CONCAT(CURDATE(), ' 00:00:00') AND CONCAT(CURDATE(), ' 23:59:59')
GROUP BY ROUND(UNIX_TIMESTAMP(start_time)/(30* 60))
For reference of my output :
We need a source for that 7:30 row; a row source for all the time values.
If we have a clock table that contains all of the time values we want to return, such that we can write a query that returns that first column, the thirty minute interval values we want to return,
as an example:
SELECT c.hhmm AS thirty_minute_interval
FROM clock c
WHERE c.hhmm ...
ORDER BY c.hhmm
then we can do an outer join the results of the query with the missing rows
SELECT c.hhmm AS _thirty_minute_interval
, IFNULL(r._cnt_bot,0) AS _cnt_bot
FROM clock c
LEFT
JOIN ( -- query with missing rows
SELECT time_format(...) AS thirtyMinuteInterval
, COUNT(...) AS _cnt_bot
FROM bot_activity
WHERE
GROUP BY time_format(...)
) r
ON r.thirtyMinuteInterval = c.hhmm
WHERE c.hhmm ...
ORDER BY c.hhmm
The point is that the SELECT will not generate "missing" rows from a source where they don't exist; we need a source for them. We don't necessarily have to have a separate clock table, we could have an inline view generate the rows. But we do need to be able to SELECT those value from a source.
( Note that bot_id in the original query is indeterminate; the value will be from some row in the collapsed set of rows, but no guarantee which value. (If we add ONLY_FULL_GROUP_BY to sql_mode, the query will throw an error, like most other relational databases will when non-aggregate expressions in the SELECT list don't appear in the GROUP BY are aren't functionally dependent on the GROUP BY )
EDIT
In place of a clock table, we can use an inline view. For small sets, we could something like this.
SELECT c.tmi
FROM ( -- thirty minute interval
SELECT CONVERT(0,TIME) + INTERVAL h.h+r.h HOUR + INTERVAL m.mm MINUTE AS tmi
FROM ( SELECT 0 AS h UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3
UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7
UNION ALL SELECT 8 UNION ALL SELECT 9 UNION ALL SELECT 10 UNION ALL SELECT 11
) h
CROSS JOIN ( SELECT 0 AS h UNION ALL SELECT 12 ) r
CROSS JOIN ( SELECT 0 AS mm UNION ALL SELECT 30 ) m
ORDER BY tmi
) c
ORDER
BY c.tmi
(Inline view c is a standin for a clock table, returns time values on thirty minute boundaries.)
That's kind of ugly. We can see where if we had a rowsource of just integer values, we could make this much simpler. But if we pick that apart, we can see how to extend the same pattern to generate fifteen minute intervals, or shorten it to generate two hour intervals.
I have a yearly list of sales of an item for the past 20 years.
the data is like this.
date ; sales value
2001-01-01 ; 423
2001-01-02 ; 152
2001-01-03 ; 162
2001-01-04 ; 172
.
.
.
I have a behavioral problem. I must find the five consecutive days where the sum of sales is maximum in a year, for each year, for the past 20 years. Then using the result i must analyse the spending pattern.
how can i get the 5 consecutive days whose sum is maximum in a year?
I must get it for all years with dates and sum of sales in those 5 days total value. Can anyone help me in my assignment, please?
TIA
Well, in MySQL 8+, you can use window functions. In earlier versions, a correlated subquery. That looks like:
select year(t.date),
max(next_5_days_sales),
substring_index(group_concat(date order by next_5_days_sales desc), ',', 1) as date_at_max
from (select t.*,
(select sum(t2.sales)
from t t2
where t2.date >= t.date and t2.date < t.date + interval 5 day
) as next_5_days_sales
from t
) t
group by year(t.date);
Notes:
You will need to reset the group_concat_max_len, because 1024 is probably not long enough for the intermediate result.
This allows the periods to span year boundaries.
In MySQL 8, use window functions!
select t.*
from (select t.*,
row_number() over (partition by year(date) order by next_5_days_sales) as seqnum
from (select t.*,
sum(t2.sales) over (order by date range between current row and 4 following) as as next_5_days_sales
from t
) t
) t
where seqnum = 1;
Data:
values date
14 1.1.2010
20 1.1.2010
10 2.1.2010
7 4.1.2010
...
sample query about january 2010 should get 31 rows. One for every day. And values vould be added. Right now I could do this with 31 queries but I would like this to work with one. Is it possible?
results:
1. 34
2. 10
3. 0
4. 7
...
This is actually surprisingly difficult to do in SQL. One way to do it is to have a long select statement with UNION ALLs to generate the numbers from 1 to 31. This demonstrates the principle but I stopped at 4 for clarity:
SELECT MonthDate.Date, COALESCE(SUM(`values`), 0) AS Total
FROM (
SELECT 1 AS Date UNION ALL
SELECT 2 UNION ALL
SELECT 3 UNION ALL
SELECT 4 UNION ALL
--
SELECT 28 UNION ALL
SELECT 29 UNION ALL
SELECT 30 UNION ALL
SELECT 31) AS MonthDate
LEFT JOIN Table1 AS T1
ON MonthDate.Date = DAY(T1.Date)
AND MONTH(T1.Date) = 1 AND YEAR(T1.Date) = 2010
WHERE MonthDate.Date <= DAY(LAST_DAY('2010-01-01'))
GROUP BY MonthDate.Date
It might be better to use a table to store these values and join with it instead.
Result:
1, 34
2, 10
3, 0
4, 7
Given that for some dates you have no data, you'll need to fill in the gaps. One approach to this is to have a calendar table prefilled with all dates you need, and join against that.
If you want the results to show day numbers as you have showing in your question, you could prepopulate these in your calendar too as labels.
You would join your data table date field to the date field of the calendar table, group by that field, and sum values. You might want to specify limits for the range of dates covered.
So you might have:
CREATE TABLE Calendar (
label varchar,
cal_date date,
primary key ( cal_date )
)
Query:
SELECT
c.label,
SUM( d.values )
FROM
Calendar c
JOIN
Data_table d
ON d.date_field = c.cal_date
WHERE
c.cal_date BETWEEN '2010-01-01' AND '2010-01-31'
GROUP BY
d.date_field
ORDER BY
d.date_field
Update:
I see you have datetimes rather than dates. You could just use the MySQL DATE() function in the join, but that would probably not be optimal. Another approach would be to have start and end times in the Calendar table defining a 'time bucket' for each day.
This works for me... Its a modification of a query I found on another site. The "INTERVAL 1 MONTH" clause ensures I get the current month data, including zeros for days that have no hits. Change this to "INTERVAL 2 MONTH" to get last months data, etc.
I have a table called "payload" with a column "timestamp" - Im then joining the timestamp column on to the dynamically generated dates, casting it so that the dates match in the ON clause.
SELECT `calendarday`,COUNT(P.`timestamp`) AS `cnt` FROM
(SELECT #tmpdate := DATE_ADD(#tmpdate, INTERVAL 1 DAY) `calendarday`
FROM (SELECT #tmpdate :=
LAST_DAY(DATE_SUB(CURDATE(),INTERVAL 1 MONTH)))
AS `dynamic`, `payload`) AS `calendar`
LEFT JOIN `payload` P ON DATE(P.`timestamp`) = `calendarday`
GROUP BY `calendarday`
To dynamically get the dates within a date range using SQL you can do this (example in mysql):
Create a table to hold the numbers 0 through 9.
CREATE TABLE ints ( i tinyint(4) );
insert into ints (i)
values (0),(1),(2),(3),(4),(5),(6),(7),(8),(9);
Run a query like so:
select ((curdate() - interval 2 year) + interval (t.i * 100 + u.i * 10 + v.i) day) AS Date
from
ints t
join ints u
join ints v
having Date between '2015-01-01' and '2015-05-01'
order by t.i, u.i, v.i
This will generate all dates between Jan 1, 2015 and May 1, 2015.
Output
2015-01-01
2015-01-02
2015-01-03
2015-01-04
2015-01-05
2015-01-06
...
2015-05-01
The query joins the table ints 3 times and gets an incrementing number (0 through 999). It then adds this number as a day interval starting from a certain date, in this case a date 2 years ago. Any date range from 2 years ago and 1,000 days ahead can be obtained with the example above.
To generate a query that generates dates for more than 1,000 days simply join the ints table once more to allow for up to 10,000 days of range, and so forth.
If I'm understanding the rather vague question correctly, you want to know the number of records for each date within a month. If that's true, here's how you can do it:
SELECT COUNT(value_column) FROM table WHERE date_column LIKE '2010-01-%' GROUP BY date_column