I'm learning about recursive functions, Since I need to extract a row for each day in a range of days. This is my current data
+----+------------+------------+
| id | from | to |
+----+------------+------------+
| 1 | 09-20-2019 | 09-25-2019 |
+----+------------+------------+
The goal is to receive my data as follows
+----+------------+
| id | date |
+----+------------+
| 1 | 09-20-2019 |
| 1 | 09-21-2019 |
| 1 | 09-22-2019 |
| 1 | 09-23-2019 |
| 1 | 09-24-2019 |
| 1 | 09-25-2019 |
+----+------------+
I'm following an example seen here: https://stackoverflow.com/a/54538866/1731057
But for some reason my recursive function is looking for the 'cte' table.
Query 1 ERROR: Table 'project.cte' doesn't exist
WITH cte AS (
SELECT date_from
FROM event_dates
UNION ALL
SELECT DATE_ADD(event_dates.date_from, INTERVAL 1 DAY)
FROM cte
WHERE DATE_ADD(event_dates.date_from, INTERVAL 1 DAY) <= event_dates.date_until
)
select * FROM cte;
The structure of your recursive CTE is off, and the upper half of the union should be a seed base case. Then, the recursive part should add one day to the previous incoming value:
WITH RECURSIVE cte (n, dt) AS (
SELECT 1, '2019-09-20'
UNION ALL
SELECT n + 1, TIMESTAMPADD(DAY, n, '2019-09-20') FROM cte WHERE n <= 5
)
SELECT * FROM cte;
Demo
Of note, we use TIMESTAMPADD() here to get around the problem of the INTERVAL expression, which can't really take a variable.
If you want to use this approach to generate a series of dates which matches the from and to values in your table, then you can try a join:
SELECT
t1.dt
FROM cte t1
INNER JOIN yourTable t2
ON t1.dt BETWEEN t2.from_date AND t2.to_date;
When used this way, the recursive CTE is acting as a calendar table.
Related
Table1 contains a column with cumulated values (all positive integers):
id ValuesCum
1 5
2 8
3 20
I would like to write a statement that returns an extra column with the incremented values for each row. The output should read something like:
id ValuesCum ValuesInc
1 5 (5)
2 8 3
3 20 12
Does anyone have a solution for this?
If you are running MySQL 8.0, you can use window function lag() for this:
select
t.*,
ValuesCum - lag(ValuesCum, 1, 0) over(order by id) ValuesInc
from mytable t
In earlier versions, an alternative is a correlated subquery:
select
t.*,
ValuesCum - (
select coalesce(max(t1.ValuesCum), 0)
from mytable t1
where t1.id < t.id
) ValuesInc
from mytable t
You can use a correlated subquery to get the value of ValuesCum of the previous id:
select t.*,
t.ValuesCum -
coalesce((select ValuesCum from tablename where id < t.id order by id desc limit 1), 0) ValuesInc
from tablename t
See the demo.
Results:
| id | ValuesCum | ValuesInc |
| --- | --------- | --------- |
| 1 | 5 | 5 |
| 2 | 8 | 3 |
| 3 | 20 | 12 |
I have a table like this:
+----+---------+------------+
| id | price | date |
+----+---------+------------+
| 1 | 340 | 2018-09-02 |
| 2 | 325 | 2018-09-05 |
| 3 | 358 | 2018-09-08 |
+----+---------+------------+
And I need to make a view which has a row for every day. Something like this:
+----+---------+------------+
| id | price | date |
+----+---------+------------+
| 1 | 340 | 2018-09-02 |
| 1 | 340 | 2018-09-03 |
| 1 | 340 | 2018-09-04 |
| 2 | 325 | 2018-09-05 |
| 2 | 325 | 2018-09-06 |
| 2 | 325 | 2018-09-07 |
| 3 | 358 | 2018-09-08 |
+----+---------+------------+
I can do that using PHP with a loop (foreach) and making a temp variable which holds the previous price til there is a new date.
But I need to make a view ... So I should do that using pure-SQL .. Any idea how can I do that?
You could use a recursive CTE to generate the records in the "gaps". To avoid that an infinite gap after the last date is "filled", first get the maximum date in the source data and make sure not to bypass that date in the recursion.
I have called your table tbl:
with recursive cte as (
select id,
price,
date,
(select max(date) date from tbl) mx
from tbl
union all
select cte.id,
cte.price,
date_add(cte.date, interval 1 day),
cte.mx
from cte
left join tbl
on tbl.date = date_add(cte.date, interval 1 day)
where tbl.id is null
and cte.date <> cte.mx
)
select id,
price,
date
from cte
order by 3;
demo with mysql 8
Here is an approach which should work without analytic functions. This answer uses a calendar table join approach. The first CTE below is the base table on which the rest of the query is based. We use a correlated subquery to find the most recent date earlier than the current date in the CTE which has a non NULL price. This is the basis for finding out what the id and price values should be for those dates coming in from the calendar table which do not appear in the original data set.
WITH cte AS (
SELECT cal.date, t.price, t.id
FROM
(
SELECT '2018-09-02' AS date UNION ALL
SELECT '2018-09-03' UNION ALL
SELECT '2018-09-04' UNION ALL
SELECT '2018-09-05' UNION ALL
SELECT '2018-09-06' UNION ALL
SELECT '2018-09-07' UNION ALL
SELECT '2018-09-08'
) cal
LEFT JOIN yourTable t
ON cal.date = t.date
),
cte2 AS (
SELECT
t1.date,
t1.price,
t1.id,
(SELECT MAX(t2.date) FROM cte t2
WHERE t2.date <= t1.date AND t2.price IS NOT NULL) AS nearest_date
FROM cte t1
)
SELECT
(SELECT t2.id FROM yourTable t2 WHERE t2.date = t1.nearest_date) id,
(SELECT t2.price FROM yourTable t2 WHERE t2.date = t1.nearest_date) price,
t1.date
FROM cte2 t1
ORDER BY
t1.date;
Demo
Note: To make this work on MySQL versions earlier than 8+, you would need to inline the CTEs above. It would result in verbose code, but, it should still work.
Since you are using MariaDB, it is rather trivial:
MariaDB [test]> SELECT '2019-01-01' + INTERVAL seq-1 DAY FROM seq_1_to_31;
+-----------------------------------+
| '2019-01-01' + INTERVAL seq-1 DAY |
+-----------------------------------+
| 2019-01-01 |
| 2019-01-02 |
| 2019-01-03 |
| 2019-01-04 |
| 2019-01-05 |
| 2019-01-06 |
(etc)
There are variations on this wherein you generate a large range of dates, but then use a WHERE to chop to what you need. And use LEFT JOIN with the sequence 'derived table' on the 'left'.
Use something like the above as a derived table in your query.
ie:
| id | num |
| a | 1 |
| b | 2 |
| c | 3 |
| d | 4 |
| e | 5 |
and this query is essentially what I'm trying to do:
select num as number, sum(case num > number then num else 0 end) as summation from table;
(I'm trying to sum up all the ints larger than the currently selected num in the column num.)
example output from above table:
| num | summation |
| 1 | 14 |
| 2 | 12 |
| 3 | 9 |
| 4 | 5 |
|5 | 0 |
The problem lies in the fact that I can't use the alias defined in the same select statement; is there another way?
Thanks!
If you're on MySQL 8.0 you can use window functions.
SELECT num,
sum(num) OVER (ORDER BY num DESC) - num summation
FROM elbat
ORDER BY num;
Prior to MySQL 8.0 you can use a correlated subquery.
SELECT t1.num,
coalesce((SELECT sum(t2.num)
FROM elbat t2
WHERE t2.num > t1.num),
0) summation
FROM elbat t1
ORDER BY t1.num;
You can write this using a correlated subquery:
select num,
(select sum(num)
from t 2
where t2.num >= t.num
) - num as summation
from t;
You can use correlated subquery :
select num,
(select sum(num)
from table t2
where t2.num > t.num
) as summation
from table t1;
this works, although a bit messy:
select num, (select sum(case when table.num > temp.num then num else 0 end)
from (select * from table) as temp
) as summation
from table;
Say I have a dataset of :
|dateid | value |
|20150101 | 1 |
|20150102 | 2 |
|20150103 | 3.1 |
|20150104 | 4.3 |
|20150105 | 3.1 |
|20150106 | 1 |
|20150107 | 1 |
|20150108 | 1 |
|.... | |
|.... | ... |
|20151001 | 10.3|
I want to query the average of every past 7 days based on a date range.
say for dateid from 20150707 and 20150730, when I select row of 20150707, I also need the average value between 20150701 and 20150707( (1+2+3.1+4.3+1+1+1+1)/7) as well as the value for 20150707(1) like:
select dateid, value , avg(value) as avg_past_7 from mytable where dateid between 20150707 and 20150730GROUP BY every past_7days.
And when the records are less than 7 rows to count, the avg remains null.
That means if I only have records from 20150707-20150730 in the table, the past_7_day avg for 20150707/8/9/10/11/12 remains null.
Correlated sub-select:
select dateid, value, (select avg(value) from mytable t2
where t2.dateid between (DATE_SUB(date(t1.dateid),INTERVAL 6 day)+0)
and t1.dateid) as avg_past_7
from mytable t1
where dateid between 20150101 and 20150201 order by dateid;
Use Date_SUB With Interval of 7 Days
I solve the problem by :
select t1.dateid, t1.value, if(count(1)>=7,avg(t2.value),null)
from mytable t1 , mytable t2
where t2.dateid between DATE_SUB(date(t1.dateid),INTERVAL 6 day)+0 and t1.dateid and
t1.dateid between 20150105 and 20150201
group by t1.dateid ,t1.value
order by dateid;
I am struggling in to get result from mysql in the following way. I have 10 records in mysql db table having date and unit fields. I need to get used units on every date.
Table structure as follows, adding today unit with past previous unit in every record:
Date Units
---------- ---------
10/10/2012 101
11/10/2012 111
12/10/2012 121
13/10/2012 140
14/10/2012 150
15/10/2012 155
16/10/2012 170
17/10/2012 180
18/10/2012 185
19/10/2012 200
Desired output will be :
Date Units
---------- ---------
10/10/2012 101
11/10/2012 10
12/10/2012 10
13/10/2012 19
14/10/2012 10
15/10/2012 5
16/10/2012 15
17/10/2012 10
18/10/2012 5
19/10/2012 15
Any help will be appreciated. Thanks
There's a couple of ways to get the resultset. If you can live with an extra column in the resultset, and the order of the columns, then something like this is a workable approach.
using user variables
SELECT d.Date
, IF(#prev_units IS NULL
,#diff := 0
,#diff := d.units - #prev_units
) AS `Units_used`
, #prev_units := d.units AS `Units`
FROM ( SELECT #prev_units := NULL ) i
JOIN (
SELECT t.Date, t.Units
FROM mytable t
ORDER BY t.Date, t.Units
) d
This returns the specified resultset, but it includes the Units column as well. It's possible to have that column filtered out, but it's more expensive, because of the way MySQL processes an inline view (MySQL calls it a "derived table")
To remove that extra column, you can wrap that in another query...
SELECT f.Date
, f.Units_used
FROM (
query from above goes here
) f
ORDER BY f.Date
but again, removing that column comes with the extra cost of materializing that result set a second time.
using a semi-join
If you are guaranteed to have a single row for each Date value, either stored as a DATE, or as a DATETIME with the timecomponent set to a constant, such as midnight, and no gaps in the Date value, and Date is defined as DATE or DATETIME datatype, then another query that will return the specifid result set:
SELECT t.Date
, t.Units - s.Units AS Units_Used
FROM mytable t
LEFT
JOIN mytable s
ON s.Date = t.Date + INTERVAL -1 DAY
ORDER BY t.Date
If there's a missing Date value (a gap) such that there is no matching previous row, then Units_used will have a NULL value.
using a correlated subquery
If you don't have a guarantee of no "missing dates", but you have a guarantee that there is no more than one row for a particular Date, then another approach (usually more expensive in terms of performance) is to use a correlated subquery:
SELECT t.Date
, ( t.Units - (SELECT s.Units
FROM mytable s
WHERE s.Date < t.Date
ORDER BY s.Date DESC
LIMIT 1)
) AS Units_used
FROM mytable t
ORDER BY t.Date, t.Units
spencer7593's solution will be faster, but you can also do something like this...
SELECT * FROM rolling;
+----+-------+
| id | units |
+----+-------+
| 1 | 101 |
| 2 | 111 |
| 3 | 121 |
| 4 | 140 |
| 5 | 150 |
| 6 | 155 |
| 7 | 170 |
| 8 | 180 |
| 9 | 185 |
| 10 | 200 |
+----+-------+
SELECT a.id,COALESCE(a.units - b.units,a.units) units
FROM
( SELECT x.*
, COUNT(*) rank
FROM rolling x
JOIN rolling y
ON y.id <= x.id
GROUP
BY x.id
) a
LEFT
JOIN
( SELECT x.*
, COUNT(*) rank
FROM rolling x
JOIN rolling y
ON y.id <= x.id
GROUP
BY x.id
) b
ON b.rank= a.rank -1;
+----+-------+
| id | units |
+----+-------+
| 1 | 101 |
| 2 | 10 |
| 3 | 10 |
| 4 | 19 |
| 5 | 10 |
| 6 | 5 |
| 7 | 15 |
| 8 | 10 |
| 9 | 5 |
| 10 | 15 |
+----+-------+
This should give the desired result. I don't know how your table is called so I named it "tbltest".
Naming a table date is generally a bad idea as it also refers to other things (functions, data types,...) so I renamed it "fdate". Using uppercase characters in field names or tablenames is also a bad idea as it makes your statements less database independent (some databases are case sensitive and some are not).
SELECT
A.fdate,
A.units - coalesce(B.units, 0) AS units
FROM
tbltest A left join tbltest B ON A.fdate = B.fdate + INTERVAL 1 DAY