I have for example the following table :
date sales
2016-02-01 2
2016-02-02 4
2016-02-03 4
2016-02-04 7
2016-02-05 3
2016-02-06 1
2016-02-07 6
2016-02-08 3
I would like to obtain a column that makes a rolling cumulative sum of the sales over x days. For example, over 3 days we would obtain :
date sales rolling_cumul_3_days
2016-02-01 2 2
2016-02-02 4 6
2016-02-03 4 10
2016-02-04 7 15
2016-02-05 3 14
2016-02-06 1 11
2016-02-07 6 10
2016-02-08 3 10
Is it possible to obtain in a single query, or do I have to run a cumulative sum for each date over the past x days and then aggregate the result ?
This is a general idea I have for this query (but definitely not correct...) :
/* Initiate variables */
SET #csum := 0;
SET #date_cursor := '2016-02-01'
/* Query for rolling result */
SELECT date, sales, MAX(
/* Query that cumulate sales on a three day interval for each date */
SELECT (#csum := #csum + sales) as cumul_3_days
FROM table
WHERE date <= #date_cursor
AND date >= DATE_SUB(#date_cursor, INTERVAL -2 DAY);
/* Reset variables for next date */
SET #csum := 0;
SET #date_cursor := DATE_ADD(#date_cursor, INTERVAL 1 DAY);
) AS rolling_cumul_3_days
FROM table
Without variables:
SELECT x.*
, SUM(y.sales)
FROM my_table x
JOIN my_table y
ON y.date BETWEEN x.date - INTERVAL 3-1 DAY
AND x.date GROUP BY x.date;
If performance is an issue, then no doubt a variables-type solution will be forthcoming.
Related
I have a MySQL requirement to select data from a table based on a start date and end date and group it by weekly also selecting the data in reverse order by date. Assume that, I have chosen the start date as 1st November and the end date as 04 December. Now, I would like to fetch the data as 04 December to 28 November, 27 November to 20 November, 19 November to 12 November and so on and sum the value count for that week.
Given an example table,
id
value
created_at
1
10
2021-10-11
2
13
2021-10-17
3
11
2021-10-25
4
8
2021-11-01
5
1
2021-11-10
6
4
2021-11-18
7
34
2021-11-25
8
17
2021-12-04
Now the result should be like 2021-12-04 to 2021-11-28 as one week, following the same in reverse order and summing the column value data for that week. I have tried in the query to add the interval of 7 days after the end date but it didn't work.
SELECT count(value) AS total, MIN(R.created_at)
FROM data_table AS D
WHERE D.created_at BETWEEN '2021-11-01' AND '2021-12-04' - INTERVAL 7 DAY ORDER BY D.created_at;
And it's also possible to have the last week may have lesser than 7 days.
Expected output:
end_interval
start_interval
total
2021-12-04
2021-11-27
17
2021-11-27
2021-11-20
34
2021-11-20
2021-11-13
4
2021-11-13
2021-11-06
1
2021-11-06
2021-10-30
8
2021-10-30
2021-10-25
11
Note that the last week is only 5 days depending upon the selected from and end dates.
One option to address this problem is to
generate a calendar of all your intervals, beginning from last date till first date, with a split of your choice, using a recursive query
joining back the calendar with the original table
capping start_interval at your start_date value
aggregating values for each interval
You can have three variables to be set, to customize your date intervals and position:
SET #start_date = DATE('2021-10-25');
SET #end_date = DATE('2021-12-04');
SET #interval_days = 7;
Then use the following query, as already described:
WITH RECURSIVE cte AS (
SELECT #end_date AS end_interval,
DATE_SUB(#end_date, INTERVAL #interval_days DAY) AS start_interval
UNION ALL
SELECT start_interval AS end_interval,
GREATEST(DATE(#start_date), DATE_SUB(start_interval, INTERVAL #interval_days DAY)) AS start_interval
FROM cte
WHERE start_interval > #start_date
)
SELECT end_interval, start_interval, SUM(_value) AS total
FROM cte
LEFT JOIN tab
ON tab.created_at BETWEEN start_interval AND end_interval
GROUP BY end_interval, start_interval
Check the demo here.
update: this can be done with python. here
i have a table like this:
event_id vendor_id start_date end_date
1 100 2021-01-01 2021-01-31
2 101 2021-01-15 2021-02-15
3 102 2021-02-01 2021-02-31
4 103 2021-02-01 2021-03-31
5 104 2021-03-01 2021-03-31
6 105 2021-03-01 2021-04-31
7 100 2021-04-01 2021-04-31
i would like an output like this: number of events based on month. but if the event between two or more months, it must be included in the count for each month. For example, The event in the second row (event_id=2) takes place in both January and February. Therefore, this event should be included in the total both in January and February.
output:
month total_event
2021-01 2 ---->> event_id=(1,2)
2021-02 3 ---->> event_id=(2,3,4)
2021-03 3 ---->> event_id=(4,5,6)
2021-04 2 ---->> event_id=(6,7)
Note: I wrote it to make the " --->> event_id= : " part better understood. i dont needed. i just need the month and the total_event.
i tried this query:
select date_format(start_date,'%Y-%m') as month,count(event_id) as total_event
group by date_format(start_date,'%Y-%m')
month total_event
2021-01 2
2021-02 2
2021-03 2
2021-04 1
but it counts only by start_date, so the numbers are missing.
Idea
To get the valid months list from the table
To calculate the event counts by event table's joining with the months
MySQL 8.0+
We can get the valid months list by Recursive.
Here is a full SQL. Assumed that your event table is c!
WITH RECURSIVE all_dates(dt) AS (
-- anchor
SELECT MIN(c.`start_date`) AS dt FROM c
UNION ALL
-- recursion with stop condition
SELECT dt + INTERVAL 1 MONTH
FROM all_dates WHERE dt + INTERVAL 1 MONTH <= (SELECT MAX(c.end_date) FROM c)
)
SELECT LEFT(dt, 7) AS `month`, COUNT(d.dt) AS total_event, GROUP_CONCAT(DISTINCT c.`event_id`) AS event_ids FROM all_dates d
INNER JOIN c ON LEFT(d.dt, 7) >= LEFT(c.start_date, 7) AND LEFT(d.dt, 7) <= LEFT(c.end_date, 7)
GROUP BY LEFT(dt, 7);
Tablename=run_detail
I have to calculate avg time of jobs for last 7 days, but in somecases
number of runs could be less than 7 days. eg abc has only 2 run_date.
(4.5+6+.....+7)/7=5.83 and (23.9+45.7)/2=34.8 and also need to
calculate based on latest 7 runs. for eg. 2020-07-04 to 2020-07-10,
not from 2020-07-01
Job_name run_date rownum count elapsed_time(sec) avg_time
xyz 2020-07-01 1 10 4.5 ?
xyz 2020-07-02 2 10 6 ?
.......
xyz 2020-07-10 10 10 7.0 ?
abc 2020-07-01 1 2 23.9 ?
abc 2020-07-02 2 2 45.7 ?
Desired Output
Job_name run_date rownum count elapsed_time(sec) avg_time
xyz 2020-07-01 1 10 4.5 5.83
xyz 2020-07-02 2 10 6 5.83
.......
xyz 2020-07-10 10 10 7.0 5.83
abc 2020-07-01 1 2 23.9 34.8
abc 2020-07-02 2 2 45.7 34.8
Could you please help how to achieve the avg time in mysql
If you want the overage over the preceding 7 days, you can use a window functions:
select t.*,
avg(elapsed_time) over (partition by job_name
order by run_date
range between interval -6 day preceding and current row
) as avg_time
from t;
Note: This assumes that you really want six preceding days plus the current date. If you really want 7 days before to 1 day before (the preceding week), then use:
range between interval -7 day preceding and interval -1 day preceding
EDIT:
In older versions of MySQL, you can use a correlated subquery:
select t.*,
(select avg(t2.elapsed_time)
from t t2
where t2.job_name = t.job_name and
t2.run_date <= t.run_date and
t2.run_date > t.run_date - interval 7 day
) as avg_time
from t;
Adjust the date comparison to get exactly the period you want.
I have a table that contains:
id date user_id duration amount
1 2014-01-01 00:00:00 1 1 £10
2 2014-01-02 00:00:00 2 2 £10
3 2014-01-03 00:00:00 3 3 £10
I'm trying to display the amount per month. Any ideas how to do this in a query?
Working on the assumptions that you can extract the month from you datetime easily, so the real question is about the aggregation logic, and that you can create a numbers table.
Here is a simple example that shows the pattern.
sqlfiddle
CREATE TABLE Num (num int);
INSERT INTO Num VALUES (0),(1),(2),(3),(4);
CREATE TABLE Tbl (start int, run int);
INSERT INTO Tbl VALUES (1,2),(2,3);
SELECT start + num active_month
,count(*) * 10 income
FROM Tbl
INNER JOIN
Num ON num < run
GROUP BY start + num
Like Karl, I'm pretty sure some kind of numbers table is necessary here. Personally I like the approach given here, which defines a view (well, several of them) to generate numbers, instead of having to actually store a table full of numbers. Whether you use a table or a view, when you SELECT from it, it just looks like this:
n
---
0
1
2
3
…
With that you can construct a query like this:
SELECT
purchases.purchase_id,
purchases.date_purchased,
purchases.duration,
-- generator_16 is our numbers table
generator_16.n,
-- Below we calculate the year and month (year_mon) in the following way:
-- (1) Get the first day of the year, e.g. if date_purchased is 2012-12-28,
-- this gives us 2012-01-01.
-- (2) Get the month number, e.g. 12 for 2012-12-28) and add that many months
-- to the first day of the year, which gives us the first day of the
-- month, 2012-12-01.
-- (3) Add "n" months, where "n" is the number we get from the numbers table,
-- starting at 0.
DATE_ADD( -- (3)
DATE_ADD( -- (2)
MAKEDATE( YEAR(purchases.date_purchased), 1 ), -- (1)
INTERVAL MONTH(purchases.date_purchased) - 1 MONTH -- (2)
),
INTERVAL generator_16.n MONTH -- (3)
) AS year_mon,
purchases.amount_income / purchases.duration AS amount
FROM purchases
-- The below JOIN means that if `purchases.duration` is 3, we get three rows
-- that have 0, 1, and 2 in the `n` column, which we use as the number of dates
-- to add in (3) above.
JOIN generator_16
ON generator_16.n BETWEEN 0 AND purchases.duration - 1
ORDER BY purchases.purchase_id, year_mon;
This gives us a result like this (SQL Fiddle):
purchase_id date_purchased duration n year_mon amount
----------- -------------- -------- - ------------ ------
1 2013-12-28 … 2 0 2013-12-01 … 7.5
1 2013-12-28 … 2 1 2014-01-01 … 7.5
2 2014-01-04 … 1 0 2014-01-01 … 10
3 2014-02-04 … 6 0 2014-02-01 … 6.6667
3 2014-02-04 … 6 1 2014-03-01 … 6.6667
3 2014-02-04 … 6 2 2014-04-01 … 6.6667
3 2014-02-04 … 6 3 2014-05-01 … 6.6667
3 2014-02-04 … 6 4 2014-06-01 … 6.6667
3 2014-02-04 … 6 5 2014-07-01 … 6.6667
I inserted blank lines to separate the purchase_id groups so you can see how n increases from 0 to duration - 1 with each item in the group. As you can see, year_mon is equal to n months after the first day of the date_purchased month plus n months, and amount is equal to amount_income / duration.
We're almost done, but as you can see year_mon has repetition: 2014-01-01 is shown twice. This is great news, because we can then use GROUP BY to group by that column and SUM(amount) to get the total for that month:
SELECT
DATE_ADD(
DATE_ADD(
MAKEDATE( YEAR(purchases.date_purchased), 1 ),
INTERVAL MONTH(purchases.date_purchased) - 1 MONTH
),
INTERVAL generator_16.n MONTH
) AS year_mon,
SUM(purchases.amount_income / purchases.duration) AS total
FROM purchases
JOIN generator_16
ON generator_16.n BETWEEN 0 AND purchases.duration - 1
GROUP BY year_mon
ORDER BY year_mon;
The only difference between this query and the previous month is that we do GROUP BY year_mon and then SUM(amount_income / duration) to get total for the month, yielding this result (SQL Fiddle):
year_mon total
------------ ------
2013-12-01 … 7.5
2014-01-01 … 17.5
2014-02-01 … 6.6667
2014-03-01 … 6.6667
2014-04-01 … 6.6667
2014-05-01 … 6.6667
2014-06-01 … 6.6667
2014-07-01 … 6.6667
...and of course you can use DATE_FORMAT and CAST or ROUND to get nicely-formatted dates and amounts, or you can do that in your front-end code.
What about :
SELECT a.my_date, a.income, IFNULL(SUM(DISTINCT(a.income)) + sum( b.income ), a.income) as roll_up
FROM (
SELECT purchase_id, DATE_FORMAT( date_purchased, '%y-%m') AS my_date, SUM( amount_income / duration ) AS "income"
FROM incomes
GROUP BY my_date
) AS a
LEFT OUTER JOIN (
SELECT purchase_id, DATE_FORMAT( date_purchased, '%y-%m') AS my_date, SUM(amount_income / duration ) AS "income"
FROM incomes
GROUP BY my_date
) AS b ON ( a.purchase_id > b.purchase_id )
GROUP BY a.purchase_id
It's a bit tricky to do that in one shot - and it might be improved - but that gives the following results :
my_date income roll_up
13-12 8.5000 8.5000
14-01 10.0000 18.5000
14-02 16.6667 35.1667
My data set is :
1 2013-12-28 00:00:00 1 2 15
2 2014-01-04 00:00:00 2 1 10
3 2014-02-04 00:00:00 3 6 40
4 2013-12-29 00:00:00 4 1 1
5 2014-02-28 00:00:00 5 2 20
I am having an issue with a SELECT command in MySQL. I have a database of securities exchanged daily with maturity from 1 to 1000 days (>1 mio rows). I would like to get the outstanding amount per day (and possibly per category). To give an example, suppose this is my initial dataset:
DATE VALUE MATURITY
1 10 3
1 15 2
2 10 1
3 5 1
I would like to get the following output
DATE OUTSTANDING_AMOUNT
1 25
2 35
3 15
Outstanding amount is calculated as the total of securities exchanged still 'alive'. That means, in day 2 there is a new exchange for 10 and two old exchanges (10 and 15) still outstanding as their maturity is longer than one day, for a total outstanding amount of 35 on day 2. In day 3 instead there is a new exchange for 5 and an old exchange from day 1 of 10. That is, 15 of outstanding amount.
Here's a more visual explanation:
Monday Tuesday Wednesday
10 10 10 (Day 1, Value 10, matures in 3 days)
15 15 (Day 1, 15, 2 days)
10 (Day 2, 10, 1 day)
5 (Day 3, 5, 3 days with remainder not shown)
-------------------------------------
25 35 15 (Outstanding amount on each day)
Is there a simple way to get this result?
First of all in the main subquery we find SUM of all Values for current date. Then add to them values from previous dates according their MATURITY (the second subquery).
SQLFiddle demo
select T1.Date,T1.SumValue+
IFNULL((select SUM(VALUE)
from T
where
T1.Date between
T.Date+1 and T.Date+Maturity-1 )
,0)
FROM
(
select Date,
sum(Value) as SumValue
from T
group by Date
) T1
order by DATE
I'm not sure if this is what you are looking for, perhaps if you give more detail
select
DATE
,sum(VALUE) as OUTSTANDING_AMOUNT
from
NameOfYourTable
group by
DATE
Order by
DATE
I hope this helps
Each date considers each row for inclusion in the summation of value
SELECT d.DATE, SUM(m.VALUE) AS OUTSTANDING_AMOUNT
FROM yourTable AS d JOIN yourtable AS m ON d.DATE >= m.MATURITY
GROUP BY d.DATE
ORDER BY d.DATE
A possible solution with a tally (numbers) table
SELECT date, SUM(value) outstanding_amount
FROM
(
SELECT date + maturity - n.n date, value, maturity
FROM table1 t JOIN
(
SELECT 1 n UNION ALL
SELECT 2 UNION ALL
SELECT 3 UNION ALL
SELECT 4 UNION ALL
SELECT 5
) n ON n.n <= maturity
) q
GROUP BY date
Output:
| DATE | OUTSTANDING_AMOUNT |
-----------------------------
| 1 | 25 |
| 2 | 35 |
| 3 | 15 |
Here is SQLFiddle demo