Tablename=run_detail
I have to calculate avg time of jobs for last 7 days, but in somecases
number of runs could be less than 7 days. eg abc has only 2 run_date.
(4.5+6+.....+7)/7=5.83 and (23.9+45.7)/2=34.8 and also need to
calculate based on latest 7 runs. for eg. 2020-07-04 to 2020-07-10,
not from 2020-07-01
Job_name run_date rownum count elapsed_time(sec) avg_time
xyz 2020-07-01 1 10 4.5 ?
xyz 2020-07-02 2 10 6 ?
.......
xyz 2020-07-10 10 10 7.0 ?
abc 2020-07-01 1 2 23.9 ?
abc 2020-07-02 2 2 45.7 ?
Desired Output
Job_name run_date rownum count elapsed_time(sec) avg_time
xyz 2020-07-01 1 10 4.5 5.83
xyz 2020-07-02 2 10 6 5.83
.......
xyz 2020-07-10 10 10 7.0 5.83
abc 2020-07-01 1 2 23.9 34.8
abc 2020-07-02 2 2 45.7 34.8
Could you please help how to achieve the avg time in mysql
If you want the overage over the preceding 7 days, you can use a window functions:
select t.*,
avg(elapsed_time) over (partition by job_name
order by run_date
range between interval -6 day preceding and current row
) as avg_time
from t;
Note: This assumes that you really want six preceding days plus the current date. If you really want 7 days before to 1 day before (the preceding week), then use:
range between interval -7 day preceding and interval -1 day preceding
EDIT:
In older versions of MySQL, you can use a correlated subquery:
select t.*,
(select avg(t2.elapsed_time)
from t t2
where t2.job_name = t.job_name and
t2.run_date <= t.run_date and
t2.run_date > t.run_date - interval 7 day
) as avg_time
from t;
Adjust the date comparison to get exactly the period you want.
Related
I have a MySQL requirement to select data from a table based on a start date and end date and group it by weekly also selecting the data in reverse order by date. Assume that, I have chosen the start date as 1st November and the end date as 04 December. Now, I would like to fetch the data as 04 December to 28 November, 27 November to 20 November, 19 November to 12 November and so on and sum the value count for that week.
Given an example table,
id
value
created_at
1
10
2021-10-11
2
13
2021-10-17
3
11
2021-10-25
4
8
2021-11-01
5
1
2021-11-10
6
4
2021-11-18
7
34
2021-11-25
8
17
2021-12-04
Now the result should be like 2021-12-04 to 2021-11-28 as one week, following the same in reverse order and summing the column value data for that week. I have tried in the query to add the interval of 7 days after the end date but it didn't work.
SELECT count(value) AS total, MIN(R.created_at)
FROM data_table AS D
WHERE D.created_at BETWEEN '2021-11-01' AND '2021-12-04' - INTERVAL 7 DAY ORDER BY D.created_at;
And it's also possible to have the last week may have lesser than 7 days.
Expected output:
end_interval
start_interval
total
2021-12-04
2021-11-27
17
2021-11-27
2021-11-20
34
2021-11-20
2021-11-13
4
2021-11-13
2021-11-06
1
2021-11-06
2021-10-30
8
2021-10-30
2021-10-25
11
Note that the last week is only 5 days depending upon the selected from and end dates.
One option to address this problem is to
generate a calendar of all your intervals, beginning from last date till first date, with a split of your choice, using a recursive query
joining back the calendar with the original table
capping start_interval at your start_date value
aggregating values for each interval
You can have three variables to be set, to customize your date intervals and position:
SET #start_date = DATE('2021-10-25');
SET #end_date = DATE('2021-12-04');
SET #interval_days = 7;
Then use the following query, as already described:
WITH RECURSIVE cte AS (
SELECT #end_date AS end_interval,
DATE_SUB(#end_date, INTERVAL #interval_days DAY) AS start_interval
UNION ALL
SELECT start_interval AS end_interval,
GREATEST(DATE(#start_date), DATE_SUB(start_interval, INTERVAL #interval_days DAY)) AS start_interval
FROM cte
WHERE start_interval > #start_date
)
SELECT end_interval, start_interval, SUM(_value) AS total
FROM cte
LEFT JOIN tab
ON tab.created_at BETWEEN start_interval AND end_interval
GROUP BY end_interval, start_interval
Check the demo here.
update: this can be done with python. here
i have a table like this:
event_id vendor_id start_date end_date
1 100 2021-01-01 2021-01-31
2 101 2021-01-15 2021-02-15
3 102 2021-02-01 2021-02-31
4 103 2021-02-01 2021-03-31
5 104 2021-03-01 2021-03-31
6 105 2021-03-01 2021-04-31
7 100 2021-04-01 2021-04-31
i would like an output like this: number of events based on month. but if the event between two or more months, it must be included in the count for each month. For example, The event in the second row (event_id=2) takes place in both January and February. Therefore, this event should be included in the total both in January and February.
output:
month total_event
2021-01 2 ---->> event_id=(1,2)
2021-02 3 ---->> event_id=(2,3,4)
2021-03 3 ---->> event_id=(4,5,6)
2021-04 2 ---->> event_id=(6,7)
Note: I wrote it to make the " --->> event_id= : " part better understood. i dont needed. i just need the month and the total_event.
i tried this query:
select date_format(start_date,'%Y-%m') as month,count(event_id) as total_event
group by date_format(start_date,'%Y-%m')
month total_event
2021-01 2
2021-02 2
2021-03 2
2021-04 1
but it counts only by start_date, so the numbers are missing.
Idea
To get the valid months list from the table
To calculate the event counts by event table's joining with the months
MySQL 8.0+
We can get the valid months list by Recursive.
Here is a full SQL. Assumed that your event table is c!
WITH RECURSIVE all_dates(dt) AS (
-- anchor
SELECT MIN(c.`start_date`) AS dt FROM c
UNION ALL
-- recursion with stop condition
SELECT dt + INTERVAL 1 MONTH
FROM all_dates WHERE dt + INTERVAL 1 MONTH <= (SELECT MAX(c.end_date) FROM c)
)
SELECT LEFT(dt, 7) AS `month`, COUNT(d.dt) AS total_event, GROUP_CONCAT(DISTINCT c.`event_id`) AS event_ids FROM all_dates d
INNER JOIN c ON LEFT(d.dt, 7) >= LEFT(c.start_date, 7) AND LEFT(d.dt, 7) <= LEFT(c.end_date, 7)
GROUP BY LEFT(dt, 7);
This issue is a reference for my other question
Python solution has been done based on extract from MySQL DB (5.6.34) where original data are stored.
My question is: Is it possible to make such calculation straight in MySQL?
Just to remind:
There is 'runners' table with accumulated distance per runner and reset tags
runner startdate cum_distance reset_event
0 1 2017-04-01 100 1
1 1 2018-04-20 125 0
2 1 2018-05-25 130 1
3 2 2015-04-05 10 1
4 2 2015-10-20 20 1
5 2 2016-11-29 50 0
I would like to calculate an accumulated distance per runner since the reset point (my comments in brackets ()):
runner startdate cum_distance reset_event runner_dist_since_reset
0 1 2017-04-01 100 1 100 <-(no reset since begin)
1 1 2018-04-20 125 0 25 <-(125-100)
2 1 2018-05-25 130 1 30 <-(130-100)
3 2 2015-04-05 10 1 10 <-(no reset since begin)
4 2 2015-10-20 20 1 10 <-(20-10)
5 2 2016-11-29 50 0 30 <-(50-20)
So far I was able to calculate only differences between reset events:
SET #DistSinceReset=0;
SELECT
runner,
startdate,
reset_event,
IF(cum_distance - #DistSinceReset <0, cum_distance, cum_distance - #DistSinceReset) AS 'runner_dist_since_reset',
#DistSinceReset := cum_distance AS 'cum_distance'
FROM
runners
WHERE
reset_event = 1
GROUP BY runner, startdate
This answer is for MySQL 8.
The information you want is the most recent cum_distance for each user with reset_event = 1. You are using MySQL 8, so you can use window functions.
Here is one method:
select r.*,
(cum_distance - coalesce(preceding_reset_cum_distance, 0)) as runner_dist_since_reset
from (select r.*,
min(cum_distance) over (partition by runner order by preceding_reset) as preceding_reset_cum_distance
from (select r.*,
max(case when reset_event = 1 then start_date end) over
(partition by runner
order by start_date
rows between unbounded preceding and 1 preceding
) as preceding_reset
from runners r
) r
) r;
I´m trying to do some analysis in the following data
WeekDay Date Count
5 06/09/2018 20
6 07/09/2018 Null
7 08/09/2018 19
1 09/09/2018 16
2 10/09/2018 17
3 11/09/2018 24
4 12/09/2018 25
5 13/09/2018 24
6 14/09/2018 23
7 15/09/2018 23
1 16/09/2018 9
2 17/09/2018 23
3 18/09/2018 33
4 19/09/2018 22
5 20/09/2018 31
6 21/09/2018 17
7 22/09/2018 10
1 23/09/2018 12
2 24/09/2018 26
3 25/09/2018 29
4 26/09/2018 27
5 27/09/2018 24
6 28/09/2018 29
7 29/09/2018 27
1 30/09/2018 19
2 01/10/2018 26
3 02/10/2018 39
4 03/10/2018 32
5 04/10/2018 37
6 05/10/2018 Null
7 06/10/2018 26
1 07/10/2018 11
2 08/10/2018 32
3 09/10/2018 41
4 10/10/2018 37
5 11/10/2018 25
6 12/10/2018 20
The problem that I want to solve is: I want to create a table with the average of the 3 last same weekdays related to the day. But, when there is a NULL in the weekday, I want to ignore and do the average only with the remain numbers, not count NULL as an 0. I will give you an example here:
The date in this table is day/month/year :)
Ex: On day 12/10/2018, I need the average from
the days 05/10/2018; 28/09/2018; 21/09/2018. These are the last 3 same weekday(six) as 12/10/2018.
. Their values are Null; 29; 17. Then the result of this average must be 23, because I need to ignore the NULL, and not be 15,333.
How can I do this?
The count() function ignores nulls (i.e. does NOT increment if it encounters null) so I suggest you simply count the values then may contain the nulls you wish to ignore.
dow datecol value
6 21/09/2018 17
6 28/09/2018 29
6 05/10/2018 Null
e.g. sum(value) above = 46, and the count(value) = 2 so the average is 23.0 (and avg(value) will also return 23.0 as it also ignores nulls)
select
weekday
, `date`
, `count`
, (select (sum(`count`) * 1.0) / (count(`count`) * 1.0)
from atable as t2
where t2.weekday = t1.weekday
and t2.`date` < t1.`date
order by t2.`date` DESC
limit 3
) as average
from atable as t1
You could just use avg(count) in the query above, and get the same result.
ps. I do hope you do NOT use count as a column name! I also would suggest you do NOT use date as a column name either. i.e. Avoid using SQL terms as names.
SELECT WeekDay, AVG(Count)
FROM myTable
WHERE Count IS NOT NULL
GROUP BY WeekDay
Use IsNULL(Count,0) in your Select
SELECT WeekDay, AVG(IsNULL(Count,0))
FROM myTable
GROUP BY WeekDay
First off, you need to get the number of instances of that weekday in the data since you just need the last 3 same week days
create table table2
as
select
row_number() over(partition by weekday order by date desc) as rn
,weekday
,date
,count
from table
From here, you can get what you want. With you explanation, you don't need to filter out the NULL values for count. Just doing the avg() aggregation will simply ignore it.
select
weekday
,avg(count)
from table2
where rn in (1,2,3)
group by weekday
I have a table that contains:
id date user_id duration amount
1 2014-01-01 00:00:00 1 1 £10
2 2014-01-02 00:00:00 2 2 £10
3 2014-01-03 00:00:00 3 3 £10
I'm trying to display the amount per month. Any ideas how to do this in a query?
Working on the assumptions that you can extract the month from you datetime easily, so the real question is about the aggregation logic, and that you can create a numbers table.
Here is a simple example that shows the pattern.
sqlfiddle
CREATE TABLE Num (num int);
INSERT INTO Num VALUES (0),(1),(2),(3),(4);
CREATE TABLE Tbl (start int, run int);
INSERT INTO Tbl VALUES (1,2),(2,3);
SELECT start + num active_month
,count(*) * 10 income
FROM Tbl
INNER JOIN
Num ON num < run
GROUP BY start + num
Like Karl, I'm pretty sure some kind of numbers table is necessary here. Personally I like the approach given here, which defines a view (well, several of them) to generate numbers, instead of having to actually store a table full of numbers. Whether you use a table or a view, when you SELECT from it, it just looks like this:
n
---
0
1
2
3
…
With that you can construct a query like this:
SELECT
purchases.purchase_id,
purchases.date_purchased,
purchases.duration,
-- generator_16 is our numbers table
generator_16.n,
-- Below we calculate the year and month (year_mon) in the following way:
-- (1) Get the first day of the year, e.g. if date_purchased is 2012-12-28,
-- this gives us 2012-01-01.
-- (2) Get the month number, e.g. 12 for 2012-12-28) and add that many months
-- to the first day of the year, which gives us the first day of the
-- month, 2012-12-01.
-- (3) Add "n" months, where "n" is the number we get from the numbers table,
-- starting at 0.
DATE_ADD( -- (3)
DATE_ADD( -- (2)
MAKEDATE( YEAR(purchases.date_purchased), 1 ), -- (1)
INTERVAL MONTH(purchases.date_purchased) - 1 MONTH -- (2)
),
INTERVAL generator_16.n MONTH -- (3)
) AS year_mon,
purchases.amount_income / purchases.duration AS amount
FROM purchases
-- The below JOIN means that if `purchases.duration` is 3, we get three rows
-- that have 0, 1, and 2 in the `n` column, which we use as the number of dates
-- to add in (3) above.
JOIN generator_16
ON generator_16.n BETWEEN 0 AND purchases.duration - 1
ORDER BY purchases.purchase_id, year_mon;
This gives us a result like this (SQL Fiddle):
purchase_id date_purchased duration n year_mon amount
----------- -------------- -------- - ------------ ------
1 2013-12-28 … 2 0 2013-12-01 … 7.5
1 2013-12-28 … 2 1 2014-01-01 … 7.5
2 2014-01-04 … 1 0 2014-01-01 … 10
3 2014-02-04 … 6 0 2014-02-01 … 6.6667
3 2014-02-04 … 6 1 2014-03-01 … 6.6667
3 2014-02-04 … 6 2 2014-04-01 … 6.6667
3 2014-02-04 … 6 3 2014-05-01 … 6.6667
3 2014-02-04 … 6 4 2014-06-01 … 6.6667
3 2014-02-04 … 6 5 2014-07-01 … 6.6667
I inserted blank lines to separate the purchase_id groups so you can see how n increases from 0 to duration - 1 with each item in the group. As you can see, year_mon is equal to n months after the first day of the date_purchased month plus n months, and amount is equal to amount_income / duration.
We're almost done, but as you can see year_mon has repetition: 2014-01-01 is shown twice. This is great news, because we can then use GROUP BY to group by that column and SUM(amount) to get the total for that month:
SELECT
DATE_ADD(
DATE_ADD(
MAKEDATE( YEAR(purchases.date_purchased), 1 ),
INTERVAL MONTH(purchases.date_purchased) - 1 MONTH
),
INTERVAL generator_16.n MONTH
) AS year_mon,
SUM(purchases.amount_income / purchases.duration) AS total
FROM purchases
JOIN generator_16
ON generator_16.n BETWEEN 0 AND purchases.duration - 1
GROUP BY year_mon
ORDER BY year_mon;
The only difference between this query and the previous month is that we do GROUP BY year_mon and then SUM(amount_income / duration) to get total for the month, yielding this result (SQL Fiddle):
year_mon total
------------ ------
2013-12-01 … 7.5
2014-01-01 … 17.5
2014-02-01 … 6.6667
2014-03-01 … 6.6667
2014-04-01 … 6.6667
2014-05-01 … 6.6667
2014-06-01 … 6.6667
2014-07-01 … 6.6667
...and of course you can use DATE_FORMAT and CAST or ROUND to get nicely-formatted dates and amounts, or you can do that in your front-end code.
What about :
SELECT a.my_date, a.income, IFNULL(SUM(DISTINCT(a.income)) + sum( b.income ), a.income) as roll_up
FROM (
SELECT purchase_id, DATE_FORMAT( date_purchased, '%y-%m') AS my_date, SUM( amount_income / duration ) AS "income"
FROM incomes
GROUP BY my_date
) AS a
LEFT OUTER JOIN (
SELECT purchase_id, DATE_FORMAT( date_purchased, '%y-%m') AS my_date, SUM(amount_income / duration ) AS "income"
FROM incomes
GROUP BY my_date
) AS b ON ( a.purchase_id > b.purchase_id )
GROUP BY a.purchase_id
It's a bit tricky to do that in one shot - and it might be improved - but that gives the following results :
my_date income roll_up
13-12 8.5000 8.5000
14-01 10.0000 18.5000
14-02 16.6667 35.1667
My data set is :
1 2013-12-28 00:00:00 1 2 15
2 2014-01-04 00:00:00 2 1 10
3 2014-02-04 00:00:00 3 6 40
4 2013-12-29 00:00:00 4 1 1
5 2014-02-28 00:00:00 5 2 20