Cumulative sum as dependant subquery - mysql

Question

I have a table called transactions which contains sellers and their transactions: sale, waste, and whenever they receive products. The structure is essentially as follows:
seller_id transaction_date quantity reason product unit_price
--------- ---------------- -------- ------ ------- ----------
1 2018-01-01 10 import 1 100.0
1 2018-01-01 -5 sale 1 100.0
1 2018-01-01 -1 waste 1 100.0
2 2018-01-01 -3 sale 4 95.5
I need a daily summary of each seller, including the value of their sales, waste and starting inventory. The problem is, the starting inventory is a cumulative sum of quantities up until the given day (the imports at the given day is also included). I have the following query:
SELECT
t.seller_id,
t.transaction_date,
t.SUM(quantity * unit_price) as amount,
t.reason as reason,
(
SELECT SUM(unit_price * quantity) FROM transactions
WHERE seller_id = t.seller_id
AND (transaction_date <= t.transaction_date)
AND (
transaction_date < t.transaction_date
OR reason = 'import'
)
) as opening_balance
FROM transactions t
GROUP BY
t.transaction_date,
t.seller_id
t.reason
The query works and I get the desired results. However, even after creating indices for both the outer and the subquery, it takes way too much time (about 30 seconds), because the opening_balance query is a dependant subquery which is calculated for each row over and over again.
How can i optimize, or rewrite this query?
Edit: the subquery had a small bug with a missing WHERE condition, i fixed it, but the essence of the question is the same. I created a fiddle with example data to play around:
https://www.db-fiddle.com/f/ma7MhufseHxEXLfxhCtGbZ/2

Answer 1

Following approach utilizing User-defined variables can be more performant than using the Correlated Subquery. In your case, a temp variable was used to account for the calculation logic, which also get outputted. You can ignore that.
You can try the following query (can add more explanation if needed):
Query
SELECT dt.reason,
dt.amount,
#bal := CASE
WHEN dt.reason = 'import'
AND #sid <> dt.seller_id THEN dt.amount
WHEN dt.reason = 'import' THEN #bal + #temp + dt.amount
WHEN #sid = 0
OR #sid = dt.seller_id THEN #bal
ELSE 0
end AS opening_balance,
#temp := CASE
WHEN dt.reason <> 'import'
AND #sid = dt.seller_id
AND #td = dt.transaction_date THEN #temp + dt.amount
ELSE 0
end AS temp,
#sid := dt.seller_id AS seller_id,
#td := dt.transaction_date AS transaction_date
FROM (SELECT seller_id,
transaction_date,
reason,
Sum(quantity * unit_price) AS amount
FROM transactions
WHERE seller_id IS NOT NULL
GROUP BY seller_id,
transaction_date,
reason
ORDER BY seller_id,
transaction_date,
Field(reason, 'import', 'sale', 'waste')) AS dt
CROSS JOIN (SELECT #sid := 0,
#td := '',
#bal := 0,
#temp := 0) AS user_vars;
Result (note that I have ordered by seller_id first and then transaction_date)
| reason | amount | opening_balance | temp | seller_id | transaction_date |
| ------ | ------ | --------------- | ----- | --------- | ---------------- |
| import | 1250 | 1250 | 0 | 1 | 2018-12-01 |
| sale | -850 | 1250 | -850 | 1 | 2018-12-01 |
| waste | -100 | 1250 | -950 | 1 | 2018-12-01 |
| import | 950 | 1250 | 0 | 1 | 2018-12-02 |
| sale | -650 | 1250 | -650 | 1 | 2018-12-02 |
| waste | -450 | 1250 | -1100 | 1 | 2018-12-02 |
| import | 2000 | 2000 | 0 | 2 | 2018-12-01 |
| sale | -1200 | 2000 | -1200 | 2 | 2018-12-01 |
| waste | -250 | 2000 | -1450 | 2 | 2018-12-01 |
| import | 750 | 1300 | 0 | 2 | 2018-12-02 |
| sale | -600 | 1300 | -600 | 2 | 2018-12-02 |
| waste | -450 | 1300 | -1050 | 2 | 2018-12-02 |
View on DB Fiddle

Answer 2

do thing something like this ?
SELECT s.* ,#balance:=#balance+(s.quantity*s.unit_price) AS opening_balance FROM (
SELECT t.* FROM transactions t
ORDER BY t.seller_id,t.transaction_date,t.reason
) s
CROSS JOIN ( SELECT #balance:=0) AS INIT
GROUP BY s.transaction_date, s.seller_id, s.reason;
SAMPLE
MariaDB [test]> select * from transactions;
+----+-----------+------------------+----------+------------+--------+
| id | seller_id | transaction_date | quantity | unit_price | reason |
+----+-----------+------------------+----------+------------+--------+
| 1 | 1 | 2018-01-01 | 10 | 100 | import |
| 2 | 1 | 2018-01-01 | -5 | 100 | sale |
| 3 | 1 | 2018-01-01 | -1 | 100 | waste |
| 4 | 2 | 2018-01-01 | -3 | 99.5 | sale |
+----+-----------+------------------+----------+------------+--------+
4 rows in set (0.000 sec)
MariaDB [test]> SELECT s.* ,#balance:=#balance+(s.quantity*s.unit_price) AS opening_balance FROM (
-> SELECT t.* FROM transactions t
-> ORDER BY t.seller_id,t.transaction_date,t.reason
-> ) s
-> CROSS JOIN ( SELECT #balance:=0) AS INIT
-> GROUP BY s.transaction_date, s.seller_id, s.reason;
+----+-----------+------------------+----------+------------+--------+-----------------+
| id | seller_id | transaction_date | quantity | unit_price | reason | opening_balance |
+----+-----------+------------------+----------+------------+--------+-----------------+
| 1 | 1 | 2018-01-01 | 10 | 100 | import | 1000 |
| 2 | 1 | 2018-01-01 | -5 | 100 | sale | 500 |
| 3 | 1 | 2018-01-01 | -1 | 100 | waste | 400 |
| 4 | 2 | 2018-01-01 | -3 | 99.5 | sale | 101.5 |
+----+-----------+------------------+----------+------------+--------+-----------------+
4 rows in set (0.001 sec)
MariaDB [test]>

Answer 3

SELECT
t.seller_id,
t.transaction_date,
SUM(quantity) as amount,
t.reason as reason,
quantityImport
FROM transaction t
inner join
(
select sum(ifnull(quantityImport,0)) quantityImport,p.transaction_date,p.seller_id from
( /* subquery get all the date and seller distinct row */
select transaction_date ,seller_id ,reason
from transaction
group by seller_id, transaction_date
)
as p
left join
( /* subquery get all the date and seller and the import quantity */
select sum(quantity) quantityImport,transaction_date ,seller_id
from transaction
where reason='Import'
group by seller_id, transaction_date
) as n
on
p.seller_id=n.seller_id
and
p.transaction_date>=n.transaction_date
group by
p.seller_id,p.transaction_date
) as q
where
t.seller_id=q.seller_id
and
t.transaction_date=q.transaction_date
GROUP BY
t.transaction_date,
t.seller_id,
t.reason;