Merge two queries that use different timeframes for min/max/avg - mysql

I am using 10.1.39-MariaDB - mariadb.org binary and I have the following table:
| id | date | ticker | close |
|-------|---------------------|--------|-----------|
| 39869 | 2019-09-18 00:00:00 | AAPL | 221.96160 |
| 39870 | 2019-09-17 00:00:00 | AAPL | 220.70000 |
| 39871 | 2019-09-16 00:00:00 | AAPL | 219.90000 |
| 39872 | 2019-09-13 00:00:00 | AAPL | 218.75000 |
| 39873 | 2019-09-12 00:00:00 | AAPL | 223.09000 |
| 39874 | 2019-09-11 00:00:00 | AAPL | 223.59000 |
| 39875 | 2019-09-10 00:00:00 | AAPL | 216.70000 |
I have two queries where I am calculating metrics for 52-weeks and the second query calculates aggregation metrics for 20-days:
52-weeks:
SELECT
Y.*
FROM
(
SELECT
MAX(CLOSE) AS week_52_High,
DATE AS week_52_High_date,
MIN(CLOSE) AS week_52_Low,
DATE AS week_52_Low_date,
AVG(CLOSE) AS week_52_Avg
FROM
`prices`
WHERE
DATE >= DATE(NOW()) - INTERVAL 52 WEEK AND ticker = "AAPL") Y
LEFT JOIN prices tmax52 ON
tmax52.date = Y.week_52_High_date AND tmax52.close = week_52_High
LEFT JOIN prices tmin52 ON
tmin52.date = Y.week_52_Low_date AND tmin52.close = week_52_Low
LEFT JOIN prices tavg52 ON
tavg52.close = week_52_Avg
20-days
SELECT
Y.*
FROM
(
SELECT
MAX(CLOSE) AS day_20_High,
DATE AS day_20_High_date,
MIN(CLOSE) AS day_20_Low,
DATE AS day_20_Low_date,
AVG(CLOSE) AS day_20_Avg
FROM
`prices`
WHERE
DATE >= DATE(NOW()) - INTERVAL 20 DAY AND ticker = "AAPL") Y
LEFT JOIN prices tmax20 ON
tmax20.date = Y.day_20_High_date AND tmax20.close = day_20_High
LEFT JOIN prices tmin20 ON
tmin20.date = Y.day_20_Low_date AND tmin20.close = day_20_Low
LEFT JOIN prices tavg20 ON
tavg20.close = day_20_Avg
Both queries calculate the min/max/avg for each close price and attach the days, when this happened.
Any suggestions how to merge these two queries to get everything in 1 output?
I appreciate your replies!

Your first problem is that your query won't actually work. The correct way to get the dates for the high and low values is as below. Note that it is meaningless to try and get a date for the average close as it's highly unlikely that the stock will have closed at that price.
SELECT
Y.day_20_High,
tmax20.DATE AS day_20_High_date,
Y.day_20_Low,
tmin20.DATE AS day_20_Low_date,
Y.day_20_Avg
FROM
(
SELECT
MAX(CLOSE) AS day_20_High,
MIN(CLOSE) AS day_20_Low,
ROUND(AVG(CLOSE),2) AS day_20_Avg
FROM
`prices`
WHERE
DATE >= CURDATE() - INTERVAL 20 DAY AND ticker = "AAPL") Y
LEFT JOIN prices tmax20 ON tmax20.close = Y.day_20_High
LEFT JOIN prices tmin20 ON tmin20.close = Y.day_20_Low
Output (for my sample data)
day_20_High day_20_High_date day_20_Low day_20_Low_date day_20_Avg
107.50 2019-09-20 101.10 2019-09-10 104.05
Demo on dbfiddle
Having corrected the query, you can now just JOIN to the same query for 52-week data:
SELECT
Y20.day_20_High,
tmax20.DATE AS day_20_High_date,
Y20.day_20_Low,
tmin20.DATE AS day_20_Low_date,
Y20.day_20_Avg,
Y52.week_52_High,
tmax52.DATE AS week_52_High_date,
Y52.week_52_Low,
tmin52.DATE AS week_52_Low_date,
Y52.week_52_Avg
FROM ((
SELECT
MAX(CLOSE) AS day_20_High,
MIN(CLOSE) AS day_20_Low,
ROUND(AVG(CLOSE),2) AS day_20_Avg
FROM
`prices`
WHERE
DATE >= CURDATE() - INTERVAL 20 DAY AND ticker = "AAPL") Y20
LEFT JOIN prices tmax20 ON tmax20.close = Y20.day_20_High
LEFT JOIN prices tmin20 ON tmin20.close = Y20.day_20_Low)
JOIN ((
SELECT
MAX(CLOSE) AS week_52_High,
MIN(CLOSE) AS week_52_Low,
ROUND(AVG(CLOSE),2) AS week_52_Avg
FROM
`prices`
WHERE
DATE >= DATE(NOW()) - INTERVAL 52 WEEK AND ticker = "AAPL") Y52
LEFT JOIN prices tmax52 ON tmax52.close = Y52.week_52_High
LEFT JOIN prices tmin52 ON tmin52.close = Y52.week_52_Low)
Output (for my sample data)
day_20_High day_20_High_date day_20_Low day_20_Low_date day_20_Avg week_52_High week_52_High_date week_52_Low week_52_Low_date week_52_Avg
107.50 2019-09-20 101.10 2019-09-10 104.05 109.70 2019-08-24 100.00 2019-08-21 104.19
Demo on dbfiddle

If you want to merge your queries one after another:
with t as (
select 0 as id, 1 as v0, 2 as v1
union all select 1 as id, 1 as v0, 2 as v1
)
select *
from (
select t.*, 'weekly_report' as typ from t
union all
select t.*, 'monthly_report' as typ from t
) as t1
id | v0 | v1 | typ
-: | -: | -: | :-------------
0 | 1 | 2 | weekly_report
1 | 1 | 2 | weekly_report
0 | 1 | 2 | monthly_report
1 | 1 | 2 | monthly_report
db<>fiddle here
And to have your results column by column, do a join:
with t as (
select 0 as id, 1 as v0, 2 as v1
union all select 1 as id, 1 as v0, 2 as v1
)
select t0.id, t0.v0, t0.v1, t1.v0, t1.v1
from (
select * from t
) as t0
full join (
select * from t
) as t1 on t0.id = t1.id
id | v0 | v1 | v0 | v1
-: | -: | -: | -: | -:
0 | 1 | 2 | 1 | 2
1 | 1 | 2 | 1 | 2
db<>fiddle here

Related

Create a row for every day in a date range?

I have a table like this:
+----+---------+------------+
| id | price | date |
+----+---------+------------+
| 1 | 340 | 2018-09-02 |
| 2 | 325 | 2018-09-05 |
| 3 | 358 | 2018-09-08 |
+----+---------+------------+
And I need to make a view which has a row for every day. Something like this:
+----+---------+------------+
| id | price | date |
+----+---------+------------+
| 1 | 340 | 2018-09-02 |
| 1 | 340 | 2018-09-03 |
| 1 | 340 | 2018-09-04 |
| 2 | 325 | 2018-09-05 |
| 2 | 325 | 2018-09-06 |
| 2 | 325 | 2018-09-07 |
| 3 | 358 | 2018-09-08 |
+----+---------+------------+
I can do that using PHP with a loop (foreach) and making a temp variable which holds the previous price til there is a new date.
But I need to make a view ... So I should do that using pure-SQL .. Any idea how can I do that?
You could use a recursive CTE to generate the records in the "gaps". To avoid that an infinite gap after the last date is "filled", first get the maximum date in the source data and make sure not to bypass that date in the recursion.
I have called your table tbl:
with recursive cte as (
select id,
price,
date,
(select max(date) date from tbl) mx
from tbl
union all
select cte.id,
cte.price,
date_add(cte.date, interval 1 day),
cte.mx
from cte
left join tbl
on tbl.date = date_add(cte.date, interval 1 day)
where tbl.id is null
and cte.date <> cte.mx
)
select id,
price,
date
from cte
order by 3;
demo with mysql 8
Here is an approach which should work without analytic functions. This answer uses a calendar table join approach. The first CTE below is the base table on which the rest of the query is based. We use a correlated subquery to find the most recent date earlier than the current date in the CTE which has a non NULL price. This is the basis for finding out what the id and price values should be for those dates coming in from the calendar table which do not appear in the original data set.
WITH cte AS (
SELECT cal.date, t.price, t.id
FROM
(
SELECT '2018-09-02' AS date UNION ALL
SELECT '2018-09-03' UNION ALL
SELECT '2018-09-04' UNION ALL
SELECT '2018-09-05' UNION ALL
SELECT '2018-09-06' UNION ALL
SELECT '2018-09-07' UNION ALL
SELECT '2018-09-08'
) cal
LEFT JOIN yourTable t
ON cal.date = t.date
),
cte2 AS (
SELECT
t1.date,
t1.price,
t1.id,
(SELECT MAX(t2.date) FROM cte t2
WHERE t2.date <= t1.date AND t2.price IS NOT NULL) AS nearest_date
FROM cte t1
)
SELECT
(SELECT t2.id FROM yourTable t2 WHERE t2.date = t1.nearest_date) id,
(SELECT t2.price FROM yourTable t2 WHERE t2.date = t1.nearest_date) price,
t1.date
FROM cte2 t1
ORDER BY
t1.date;
Demo
Note: To make this work on MySQL versions earlier than 8+, you would need to inline the CTEs above. It would result in verbose code, but, it should still work.
Since you are using MariaDB, it is rather trivial:
MariaDB [test]> SELECT '2019-01-01' + INTERVAL seq-1 DAY FROM seq_1_to_31;
+-----------------------------------+
| '2019-01-01' + INTERVAL seq-1 DAY |
+-----------------------------------+
| 2019-01-01 |
| 2019-01-02 |
| 2019-01-03 |
| 2019-01-04 |
| 2019-01-05 |
| 2019-01-06 |
(etc)
There are variations on this wherein you generate a large range of dates, but then use a WHERE to chop to what you need. And use LEFT JOIN with the sequence 'derived table' on the 'left'.
Use something like the above as a derived table in your query.

Ordering issue when using SQL variable

I run this query:
SELECT stockcarddetail.id, stockcarddetail.date, stockcarddetail.quantity, stockcarddetail.pricePerItem
FROM Stockcard
LEFT JOIN staff
ON staff.branchId = stockcard.branchId
LEFT JOIN stockcarddetail
ON stockcarddetail.stockcardId = stockcard.id
WHERE staff.username = 'jemmy.h'
AND stockcarddetail.quantity > 0
AND stockcard.productId = '98924a5f-6afb-11e7-8dd4-2c56dcbcb038'
ORDER BY date ASC
and get the result below:
id | date | quantity| pricePerItem
50 | 2017-10-15 | 10.00 | 10000.00
1 | 2017-10-18 | 20.00 | 10000.00
Then, I need to calculate the cumulative of quantity based on the order above, so I run this query:
SELECT a.*, #tot:=#tot + a.quantity FROM
(SELECT #tot:= 0)b
JOIN
(SELECT stockcarddetail.id, stockcarddetail.date, stockcarddetail.quantity, stockcarddetail.pricePerItem
FROM Stockcard
LEFT JOIN staff
ON staff.branchId = stockcard.branchId
LEFT JOIN stockcarddetail
ON stockcarddetail.stockcardId = stockcard.id
WHERE staff.username = 'jemmy.h'
AND stockcarddetail.quantity > 0
AND stockcard.productId = '98924a5f-6afb-11e7-8dd4-2c56dcbcb038'
ORDER BY date ASC) a
Then I got this result:
id | date | quantity| pricePerItem | #tot
1 | 2017-10-18 | 20.00 | 10000.00 | 20
50 | 2017-10-15 | 10.00 | 10000.00 | 30
However, the result that I want is like this:
id | date | quantity| pricePerItem | #tot
50 | 2017-10-15 | 10.00 | 10000.00 | 10
1 | 2017-10-18 | 20.00 | 10000.00 | 30
How can I get the expected result?
EDIT
Simplified version of the problem can be found here: http://sqlfiddle.com/#!9/f6ad91/3
From what I understand from you, you want the cumulative total for each entry.
I suggest ditching the variable and relying on a subquery instead:
SELECT
scd.id,
scd.date,
scd.quantity,
scd.pricePerItem,
(SELECT SUM(scd1.quantity) FROM StockcardDetail AS scd1 WHERE scd1.stockcardId = scd.stockcardId AND scd1.date <= scd.date) AS total
FROM Stockcard
LEFT JOIN staff ON staff.branchId = stockcard.branchId
LEFT JOIN stockcarddetail AS scd ON scd.stockcardId = stockcard.id
WHERE staff.username = 'jemmy.h'
AND scd.quantity > 0
AND stockcard.productId = '98924a5f-6afb-11e7-8dd4-2c56dcbcb038'
ORDER BY scd.date ASC
The idea behind this is to make it select the sum of all entries prior (including the current one) for each entry.
As per my understanding, you should get the expected output from your query. But, you aren't getting your expected output, then other possible solution is (WITHOUT JOIN)
SET #tot:= 0;
SELECT
stockcarddetail.id,
stockcarddetail.date,
stockcarddetail.quantity,
stockcarddetail.pricePerItem,
#tot:=#tot + stockcarddetail.quantity as Total
FROM Stockcard
LEFT JOIN staff ON staff.branchId = stockcard.branchId
LEFT JOIN stockcarddetail ON stockcarddetail.stockcardId = stockcard.id
WHERE staff.username = 'jemmy.h' AND stockcarddetail.quantity > 0 AND stockcard.productId = '98924a5f-6afb-11e7-8dd4-2c56dcbcb038'
ORDER BY date ASC

MYSQL: How to get the current entry of multipe tables

I got three tables with data for products:
One where the products are defined.
A second where is defined which store has what products with what priceing with a timestamp.
And a third with the stock in the store of the products that are assiged.
Table 1 - products:
prdoductID|Name
----------|-------
1 |banana
2 |apple
Table 2 - prices:
storeID | productID | price | timestamp
--------|-----------|-------|------------
1 | 1 | 5,90 | 2016-03-27 16:00:00
1 | 1 | 5,90 | 2016-03-27 17:00:00
2 | 1 | 5,00 | 2016-03-27 16:00:00
2 | 2 | 5,00 | 2016-03-27 16:00:00
2 | 2 | 5,90 | 2016-03-28 19:00:00
Table 3 - stocks:
storeID | productID | stock | timestamp
--------|-----------|-------|------------
1 | 1 | 50 | 2016-03-27 08:00:00
1 | 1 | 5 | 2016-03-27 17:00:00
2 | 1 | 60 | 2016-03-27 09:00:00
2 | 2 | 0 | 2016-03-27 16:00:00
2 | 2 | 55 | 2016-03-28 19:00:00
`
Now I want to get the current state of one store for example at time 2016-03-27 14:00:00.
My current query get's for me the data but for all timestamps. I just want to get the corresponding entrys with the recent timesamps for each product of this store with the storeID = x;
Note: The user is able to plan. So there could be entries with futre timestamps so just a MAX would not work.
Query:
SELECT price.*, prod.name, stock.timestamp as 'stockTimesamp',
stock.stock FROM prices price
LEFT OUTER JOIN products prod on price.productID = prod.productID and
price.timestamp=(SELECT MAX(price.timestamp) WHERE price.timestamp <= '2016-03-27 14:00:00' and price.productID = prod.productID)
LEFT OUTER JOIN stocks stock on price.storeID = best.storeID and price.productID = stock.productID and stock.timestamp=(SELECT MAX(best.timestamp) WHERE stock.timestamp <= '2016-03-27 14:00:00' and stock.productID = prod.productID and stock.productID = price.productID)
where price.storeID=21
ORDER BY stock.timestamp DESC
Hope you can help me out...
This is a troublesome data structure for this type of query. You can get the most recent timestamp per product for each table and then join back to get additional information:
select p.*,
(select max(timestamp)
from stocks s
where s.storeid = $storeid and s.productid = p.productid and
s.timestamp <= '2016-03-27 14:00:00'
) as stock_timestamp,
(select max(timestamp)
from prices pr
where pr.storeid = $storeid and pr.productid = p.productid and
pr.timestamp <= '2016-03-27 14:00:00'
) as stock_timestamp
from products p;
Then, join back to the original tables to get additional information:
select p.*, s.*, pr.*
from (select p.*,
(select max(timestamp)
from stocks s
where s.storeid = $storeid and s.productid = p.productid and
s.timestamp <= '2016-03-27 14:00:00'
) as stock_timestamp,
(select max(timestamp)
from prices pr
where pr.storeid = $storeid and pr.productid = p.productid and
pr.timestamp <= '2016-03-27 14:00:00'
) as price_timestamp
from products p
) p left join
stocks s
on s.storeid = $storeid and s.productid = p.productid and
s.timestamp = p.stock_timestamp left join
prices pr
on pr.storeid = $storeid and pr.productid = p.productid and
pr.timestamp = p.price_timestamp ;
I should note that your problem would be relatively trivial if you had an effective and end date on each record. That is a better way to store a slowly changing dimension.

nested query & transaction

Update #1: query gives me syntax error on Left Join line (running the query within the left join independently works perfectly though)
SELECT b1.company_id, ((sum(b1.credit)-sum(b1.debit)) as 'Balance'
FROM MyTable b1
JOIN CustomerInfoTable c on c.id = b1.company_id
#Filter for Clients of particular brand, package and active status
where c.brand_id = 2 and c.status = 2 and c.package_id = 3
LEFT JOIN
(
SELECT b2.company_id, sum(b2.debit) as 'Current_Usage'
FROM MyTable b2
WHERE year(b2.timestamp) = '2012' and month(b2.timestamp) = '06'
GROUP BY b2.company_id
)
b3 on b3.company_id = b1.company_id
group by b1.company_id;
Original Post:
I keep track of debits and credits in the same table. The table has the following schema:
| company_id | timestamp | credit | debit |
| 10 | MAY-25 | 100 | 000 |
| 11 | MAY-25 | 000 | 054 |
| 10 | MAY-28 | 000 | 040 |
| 12 | JUN-01 | 100 | 000 |
| 10 | JUN-25 | 150 | 000 |
| 10 | JUN-25 | 000 | 025 |
As my result, I want to to see:
| Grouped by: company_id | Balance* | Current_Usage (in June) |
| 10 | 185 | 25 |
| 12 | 100 | 0 |
| 11 | -54 | 0 |
Balance: Calculated by (sum(credit) - sum(debits))* - timestamp does not matter
Current_Usage: Calculated by sum(debits) - but only for debits in JUN.
The problem: If I filter by JUN timestamp right away, it does not calculate the balance of all time but only the balance of any transactions in June.
How can I calculate the current usage by month but the balance on all transactions in the table. I have everything working, except that it filters only the JUN results into the current usage calculation in my code:
SELECT b.company_id, ((sum(b.credit)-sum(b.debit))/1024/1024/1024/1024) as 'BW_remaining', sum(b.debit/1024/1024/1024/1024/28*30) as 'Usage_per_month'
FROM mytable b
#How to filter this only for the current_usage calculation?
WHERE month(a.timestamp) = 'JUN' and a.credit = 0
#Group by company in order to sum all entries for balance
group by b.company_id
order by b.balance desc;
what you will need here is a join with sub query which will filter based on month.
SELECT T1.company_id,
((sum(T1.credit)-sum(T1.debit))/1024/1024/1024/1024) as 'BW_remaining',
MAX(T3.DEBIT_PER_MONTH)
FROM MYTABLE T1
LEFT JOIN
(
SELECT T2.company_id, SUM(T2.debit) T3.DEBIT_PER_MONTH
FROM MYTABLE T2
WHERE month(T2.timestamp) = 'JUN'
GROUP BY T2.company_id
)
T3 ON T1.company_id-T3.company_id
GROUP BY T1.company_id
I havn't tested the query. The point here i am trying to make is how you can join your existing query to get usage per month.
alright, thanks to #Kshitij I got it working. In case somebody else is running into the same issue, this is how I solved it:
SELECT b1.company_id, ((sum(b1.credit)-sum(b1.debit)) as 'Balance',
(
SELECT sum(b2.debit)
FROM MYTABLE b2
WHERE b2.company_id = b1.company_id and year(b2.timestamp) = '2012' and month(b2.timestamp) = '06'
GROUP BY b2.company_id
) AS 'Usage_June'
FROM MYTABLE b1
#Group by company in order to add sum of all zones the company is using
group by b1.company_id
order by Usage_June desc;

MySQL SUM in different ways

I have two tables
user_raters:
| id(int) | to_id(int) | value(int) | created_at(datetime)
|1 | 2 | 1 | 2009-03-01 00:00:00
EDIT: I changed the user_rater_id. history_user_raters.user_rater_id is related to user_raters.id
history_user_raters:
| id(int) | user_rater_id(int) | value(int) | created_at(datetime)
| 1 | 1 | 1 | 2009-03-02 00:00:00
| 2 | 1 | 1 | 2009-03-02 00:00:00
| 3 | 1 | -1 | 2009-03-02 00:00:00
| 4 | 1 | 1 | 2009-03-03 00:00:00
| 5 | 1 | -1 | 2009-03-03 00:00:00
| 6 | 1 | -1 | 2009-03-03 00:00:00
| 7 | 1 | -1 | 2009-03-03 00:00:00
I want to count the sum of the values from history_user_raters as it relates to the to_id from user_raters. The result from the query should be:
| year | month | day | total | down | up
| 2009 | 3 | 2 | 1 | 1 | 2
| 2009 | 3 | 3 | -2 | 3 | 1
I have a query that is close, but it is not counting the up and down correctly. The total is right. Can some one help me write the query or new query that calculates correct up and down?
My current query:
SELECT
YEAR(history.created_at) AS `year`,
MONTH(history.created_at) AS `month`,
DAY(history.created_at) AS `day`,
SUM(history.value) as `total`,
(SELECT
abs(SUM(historydown.value))
FROM `user_raters` as raterdown
INNER JOIN `history_user_raters` AS historydown
WHERE raterdown.to_id = 2
AND historydown.value = -1
AND date(historydown.created_at)
GROUP BY history.created_at) as down,
(SELECT SUM(historyup.value)
FROM `user_raters` as raterup
INNER JOIN `history_user_raters` AS historyup
WHERE raterup.to_id = 2
AND historyup.value = 1
AND date(history.created_at)
GROUP BY raterup.to_id) as up
FROM `user_raters`
INNER JOIN history_user_raters AS history ON user_raters.id = history.user_rater_id
WHERE (user_raters.to_id = 2)
GROUP BY DATE(history.created_at)
I might see it too simply (and sorry I can't test with data at the moment), but I'm guessing the following trick with two CASE statements would do just what is needed
SELECT
YEAR(history.created_at) AS year,
MONTH(history.created_at) AS month,
DAY(history.created_at) AS day,
SUM(history.value) as total,
SUM(CASE WHEN history.value < 0 THEN history.value ELSE 0 END) as down,
SUM(CASE WHEN history.value > 0 THEN history.value ELSE 0 END) as up
FROM `user_raters`
INNER JOIN `history_user_raters` AS history
ON user_raters.id = history.user_rater_id
WHERE (user_raters.to_id = 1) -- or some other condition...
GROUP BY DATE(history.created_at)
EDIT: #OMG Ponies deleted his answer. This response make no sense now, but I am not going to delete my answer, because I think it is silly.
#OMG ponies
Your query runs, but it returns no results. I had to adjust it a bit to add the to_id in the main queries where clause
SELECT
YEAR( t.created_at ) AS `year` ,
MONTH( t.created_at ) AS `month` ,
DAY( t.created_at ) AS `day` ,
SUM( t.value ) AS `total` ,
MAX( COALESCE( x.sum_down, 0 ) ) AS down,
MAX( COALESCE( y.sum_up, 0 ) ) AS up
FROM history_user_raters AS t
JOIN user_raters AS ur ON ur.to_id = t.user_rater_id
LEFT JOIN (
SELECT hur.user_rater_id,
SUM( hur.value ) AS sum_down
FROM history_user_raters AS hur
WHERE hur.value = -1
GROUP BY hur.user_rater_id
) AS x ON x.user_rater_id = t.user_rater_id
LEFT JOIN (
SELECT hur.user_rater_id,
SUM( hur.value ) AS sum_up
FROM history_user_raters AS hur
WHERE hur.value =1
GROUP BY hur.user_rater_id
) AS y ON y.user_rater_id = t.user_rater_id
WHERE ur.to_id =1
GROUP BY YEAR( t.created_at ) , MONTH( t.created_at ) , DAY( t.created_at )