Diff between values in different rows, based on timediff - mysql

Question

MariaDB version 10.4.10.
I have a stock scraper script that fetches stock data every hour and inserts it into a MySQL database. I want a way to get price difference for each stock between, for example:
stocks fetched at 2020-03-25 07:00 and 2020-03-25 19:00 (12 hours)
stocks fetched at 2020-03-25 07:00 and 2020-03-26 07:00 (24 hours)
stocks fetched at 2020-03-25 08:00 and 2020-03-25 20:00 (12 hours)
stocks fetched at 2020-03-25 08:00 and 2020-03-26 08:00 (24 hours)
etc
The database structure looks something like this:
stocks( time_fetched DATETIME, name VARCHAR, price INT )
Some sample data:
**time_fetched name price**
2020-03-25 07:00:00 stock_A 10
2020-03-25 07:00:00 stock_B 14
2020-03-25 08:00:00 stock_A 12
2020-03-25 08:00:00 stock_B 20
...
2020-03-25 19:00:00 stock_A 28
2020-03-25 19:00:00 stock_B 32
2020-03-25 20:00:00 stock_A 40
2020-03-25 20:00:00 stock_B 36
...
2020-03-26 07:00:00 stock_A 12
2020-03-26 07:00:00 stock_B 16
2020-03-26 08:00:00 stock_A 18
2020-03-26 08:00:00 stock_B 16
Expected result:
**time_fetched name current_price price_12h_ago price_24h_ago**
2020-03-25 19:00:00 stock_A 28 10 NULL
2020-03-25 19:00:00 stock_B 32 14 NULL
2020-03-25 20:00:00 stock_A 40 12 NULL
2020-03-25 20:00:00 stock_B 36 20 NULL
2020-03-26 07:00:00 stock_A 12 28 10
2020-03-26 07:00:00 stock_B 16 32 14
2020-03-26 08:00:00 stock_A 18 40 12
2020-03-26 08:00:00 stock_B 16 36 20
Currently I am using SQL similar to this:
WITH prices AS (
SELECT time_fetched, name, price,
LAG(price, 12) OVER(PARTITION BY name ORDER BY time_fetched) AS price_12h_ago,
LAG(price, 24) OVER(PARTITION BY name ORDER BY time_fetched) AS price_24h_ago
FROM stocks
)
SELECT time_fetched, name, price AS current_price, price_12h_ago, price_24h_ago
FROM prices
This works, given that all stocks have price data fetched for all hours. In reality, there are sometimes gaps beetween hours, and price data for some hours and some stocks are missing in the stocks database.
This means that the above code that fetches price on 12 rows before the current one, does not always represent 12 hours before current row.
So I would need a way to get price difference based on actual timediff.
Hope this makes any sense to anyone out there :)

Answer 1

You can use the range() clause. If your times are precise:
SELECT time_fetched, name, price,
MIN(price) OVER (PARTITION BY name
ORDER BY time_fetched
RANGE BETWEEN INTERVAL 12 hour AND INTERVAL 12 hour
) as price_12h_ago,
MIN(price) OVER (PARTITION BY name
ORDER BY time_fetched
RANGE BETWEEN INTERVAL 24 hour AND INTERVAL 24 hour
) as price_24h_ago
FROM stocks;
Unless you set the minutes and seconds to exactly 0, you may want a broader range than just an instant. For instance:
SELECT time_fetched, name, price,
MIN(price) OVER (PARTITION BY name
ORDER BY time_fetched
RANGE BETWEEN INTERVAL '12:05' HOUR_MINUTE AND INTERVAL '11:55' HOUR_MINUTE
) as price_12h_ago,
MIN(price) OVER (PARTITION BY name
ORDER BY time_fetched
RANGE BETWEEN INTERVAL '24:05' HOUR_MINUTE AND INTERVAL '23:55' HOUR_MINUTE
) as price_24h_ago
FROM stocks;