I have this schema:
CREATE TABLE `test` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`amount` int(11) DEFAULT NULL,
`group_id` int(11) DEFAULT NULL,
PRIMARY KEY (`id`)
);
and i have populated that table with the following data:
insert into test (amount, group_id) values
(1,1), (3,1), (-4,1), (-2,1), (5,1), (10,1), (18,1), (-3,1),
(-5,1), (-7,1), (12,1), (-9,1), (6,1), (0,1), (185,2), (-150,2)
The table is:
# id, amount, group_id
1, 1, 1
2, 3, 1
3, -4, 1
4, -2, 1
5, 5, 1
6, 10, 1
7, 18, 1
8, -3, 1
9, -5, 1
10, -7, 1
11, 12, 1
12, -9, 1
13, 6, 1
14, 0, 1
15, 185, 2
16, -150, 2
This is the query i am using right now:
SELECT
t1.id,
t1.amount,
t1.cumsum,
(MAX(t1.cumsum) OVER (ORDER BY id RANGE BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW) - t1.cumsum) as drawdown
FROM
(
SELECT
id,
amount,
group_id,
SUM(amount) OVER (ORDER BY id RANGE BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW) as cumsum
FROM
test
) as t1
order by drawdown desc
the query returns this output:
id, amount, cumsum, drawdown
10 -7 16 15
12 -9 19 12
9 -5 23 8
4 -2 -2 6
14 0 25 6
13 6 25 6
3 -4 0 4
8 -3 28 3
11 12 28 3
5 5 3 1
16 150 360 0
2 3 4 0
7 18 31 0
15 185 210 0
1 1 1 0
6 10 13 0
Just a fast clarification:
With the term "drawdown" (in this case) i mean the difference from the max cumsum of the value(field) AND the simple cumsum (cumulative sum).
I am not referring to any particular trading definition, i just need to difference between the biggest pick and the lowest. (obviously the highest pick must occour before the lowest = downtrend).
OK, my problem is that i need to group by the field group_id, if i add the group by clause on the query all the windows function will mess up.
RESULT EXPECTED:
I need to get a list grouped by group_id field that show the max drawdown for each group.
Edit: (#Akina)
group_id, drawdown
1 15
2 150
Related
I have data organized by
date,time
client_id
transaction_id
transaction value
A client can have multiple transactions per day at different times. I would like to:
extract the last value of the day for each client. This is specified by the last transaction_id for the client for the day
Sum up all the values in 1 for each day
Calculate the average of all values in 2
I cannot figure out the SQL code to do this. A dummy dataset is attached. Please can someone assist? I am looking for SQL script that will yield the value 1116 in the example below. Thank you
SOURCE DATA
CALCULATIONS
****
****
Date, Time
Client_id
Transaction_id
Value
Extracted value
Sum for the day
Average of sums
DATE 1, TIME 1
A
1
123
1078
1105
DATE 1, TIME 2
A
2
230
DATE 1, TIME 3
A
3
233
233
DATE 1, TIME 4
B
4
179
DATE 1, TIME 5
B
5
263
263
DATE 1, TIME 6
C
6
136
DATE 1, TIME 7
C
7
116
DATE 1, TIME 8
C
8
296
DATE 1, TIME 9
C
9
126
126
DATE 1, TIME 10
D
10
100
DATE 1, TIME 11
D
11
198
198
DATE 1, TIME 12
F
12
258
258
DATE 2, TIME 13
A
13
224
1132
DATE 2, TIME 14
A
14
104
104
DATE 2, TIME 15
B
15
187
DATE 2, TIME 16
B
16
118
DATE 2, TIME 17
B
17
248
DATE 2, TIME 18
B
18
185
185
DATE 2, TIME 19
C
19
291
291
DATE 2, TIME 20
E
20
292
292
DATE 2, TIME 21
G
21
276
DATE 2, TIME 22
G
22
260
260
Sample data:
CREATE TABLE `source_data` (
`Date, Time` datetime DEFAULT NULL,
`Client_id` char(1) DEFAULT NULL,
`Transaction_id` int DEFAULT NULL,
`Value` int DEFAULT NULL
);
INSERT INTO `source_data` VALUES
('2022-07-01 00:01:00', 'A', 1, 123),
('2022-07-01 00:01:05', 'A', 2, 230),
('2022-07-01 00:01:10', 'A', 3, 233),
('2022-07-01 00:02:00', 'B', 4, 179),
('2022-07-01 00:02:05', 'B', 5, 263),
('2022-07-01 00:03:00', 'C', 6, 136),
('2022-07-01 00:03:05', 'C', 7, 116),
('2022-07-01 00:03:10', 'C', 8, 296),
('2022-07-01 00:03:10', 'C', 9, 126),
('2022-07-01 00:04:00', 'D', 10, 100),
('2022-07-01 00:04:05', 'D', 11, 198),
('2022-07-01 00:06:00', 'F', 12, 258),
('2022-07-02 00:01:00', 'A', 13, 224),
('2022-07-02 00:01:05', 'A', 14, 104),
('2022-07-02 00:02:00', 'B', 15, 187),
('2022-07-02 00:02:05', 'B', 16, 118),
('2022-07-02 00:02:10', 'B', 17, 248),
('2022-07-02 00:02:15', 'B', 18, 185),
('2022-07-02 00:03:00', 'C', 19, 291),
('2022-07-02 00:05:00', 'E', 20, 292),
('2022-07-02 00:07:00', 'G', 21, 276),
('2022-07-02 00:07:05', 'G', 22, 260);
Query:
WITH last_tx_values_per_day AS (
SELECT DISTINCT
date(`Date, Time`) as `Date`,
`Client_id`,
first_value(`Value`) OVER (
PARTITION BY date(`Date, Time`), `Client_id`
ORDER BY `Transaction_id` DESC) AS last_tx_value
FROM `source_data`
),
daily_sums AS (
SELECT `Date`, sum(last_tx_value) AS sum_per_day
FROM last_tx_values_per_day
GROUP BY `Date`
)
SELECT avg(sum_per_day) AS `Average of daily sums`
FROM daily_sums;
Apart from the first CTE, it should be pretty self-explanatory.
The CTE last_tx_values_per_day uses the first_value window function to get the last transaction value. Here, a window partitioned by date (the date-part of the datetime field) and the ID of the client is used.
To get the value of the last transaction, the records in the window will be ordered by transaction id, but in a descending order. Therefore, the first record in the window is the one with the highest transaction id. That is, the window function results in the value of the last transaction.
The CTE daily_sums just sums things up for each day, so that the average can be taken, finally.
Update
As the OP uses MySQL 5.7, which doesn't support CTEs and window functions, the following should work:
SELECT avg(sum_per_day) AS `Average of daily sums`
FROM (SELECT sum(`Value`) AS sum_per_day
FROM `source_data` s
WHERE `Transaction_id` = (
SELECT max(`Transaction_id`)
FROM `source_data`
WHERE date(`Date, Time`) = date(s.`Date, Time`)
AND client_id = s.client_id
)
GROUP BY date(`Date, Time`)
) daily_sums;
please help me how to create mysql query for reporting in fifo method,
table persediaan_source
id
id_barang
jumlah
harga
tanggal
id_jenis_transaksi
89
26
12
1050000
2022-07-15 05:55:23
1
90
26
8
0
2022-07-15 05:55:52
2
91
26
16
1100000
2022-07-15 05:56:22
1
95
26
10
0
2022-07-15 05:59:09
2
id_jenis_transaksi = 1 is Buy
id_jenis_transaksi = 2 is Use
i need report like this below
id
date
remarks
buy_qty
buy_price
buy_total
use_qty
use_qty_detail
use_price
use_total
bal_qty
bal_qty_detail
bal_price
bal_total
1
2022-07-15 05:55:23
Buy
12
1050000
12600000
0
0
0
0
12
12
1050000
12600000
2
2022-07-15 05:55:52
Use
0
0
0
8
8
1050000
8400000
4
4
1050000
4200000
3
2022-07-15 05:56:22
Buy
16
1100000
17600000
0
0
0
0
20
4
1050000
4200000
4
2022-07-15 05:56:22
Buy
0
0
0
0
0
0
0
0
16
1100000
17600000
5
2022-07-15 05:59:09
Use
0
0
0
10
4
1050000
4200000
10
0
1050000
0
6
2022-07-15 05:59:09
Use
0
0
0
0
6
1100000
6600000
0
10
1100000
11000000
in row #3 must be breakdown in bal_qty_detail because there is a different price and have a remaining qty from a previous price, also in row #5 must be breakdown in use_qty_detail
CREATE TABLE `persediaan_source` (
`id` int(11) NOT NULL,
`id_barang` int(11) NOT NULL,
`jumlah` double NOT NULL,
`harga` double NOT NULL,
`tanggal` datetime NOT NULL,
`id_jenis_transaksi` tinyint(4) NOT NULL COMMENT 'id = 1 -> buy, id = 2 -> use'
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
INSERT INTO `persediaan_source` (`id`, `id_barang`, `jumlah`, `harga`, `tanggal`, `id_jenis_transaksi`) VALUES
(89, 26, 12, 1050000, '2022-07-15 05:55:23', 1),
(90, 26, 8, 0, '2022-07-15 05:55:52', 2),
(91, 26, 16, 1100000, '2022-07-15 05:56:22', 1),
(95, 26, 10, 0, '2022-07-15 05:59:09', 2);
Hi using the below process you can achieve the mention reporting format. But to handle the granularity of fifo you should aviod creating these logic in mysql.
SELECT #bal_qty:=0,#old_qty:=0,#old_qty_price:=0,#new_qty:=0,#new_qty_price:=0;
CREATE TEMPORARY TABLE test1
SELECT id, tanggal as date,
#remarks:=if(id_jenis_transaksi=1,'Buy','Use') as remarks,
#buy_qty:=if(id_jenis_transaksi=1,jumlah,0) as buy_qty,
#buy_price:=if(id_jenis_transaksi=1,harga,0) as buy_price,
#buy_total:=(#buy_qty * #buy_price) as buy_total,
#use_qty:=if(id_jenis_transaksi=2,jumlah,0) as use_qty,
#use_qty_detail:=if(#remarks='Use',if(#old_qty>0,#old_qty, #use_qty),0) as use_qty_detail,
#use_price:=#old_qty_price as use_price,
#use_total:=(#use_qty * #use_price) as use_total,
#bal_qty:=if(#remarks='Buy',#bal_qty + #buy_qty, if(#bal_qty>#use_qty,#bal_qty - #use_qty, #bal_qty)) as bal_qty,
#old_qty:=if(#old_qty > 0, #old_qty, if(#remarks='Use',#bal_qty, #old_qty)) as old_qty,
#old_qty_price:=if(#old_qty_price > 0, #old_qty_price, #buy_price) as old_qty_price,
#new_qty:=if(#old_qty>0,#buy_qty, if(#new_qty>0, #new_qty,0)) as new_qty,
#new_qty_price:=if(#old_qty_price > 0 and #buy_price > 0, #buy_price, if(#new_qty_price>0,#new_qty_price,0)) as new_qty_price,
#bal_qty_detail:=if(#old_qty > 0, #old_qty, #buy_qty) as bal_qty_detail,
#bal_price:=#old_qty_price as bal_price,
#bal_total:=(#bal_qty_detail* #old_qty_price) as bal_total,
#split_flag:=if(#bal_qty != #buy_qty and #bal_qty != #old_qty,1,0) as split_flag
FROM persediaan_source;
CREATE TEMPORARY TABLE test2 select * from test1 where split_flag=1
SELECT #cnt:=0;
SELECT (#cnt:=#cnt + 1) as id, date, remarks, buy_qty, buy_price, buy_total, use_qty, use_qty_detail, use_price, use_total, bal_qty, bal_qty_detail, bal_price, bal_total
FROM (
(SELECT * FROM test1)
UNION ALL
(SELECT id, date, remarks, 0 as buy_qty, 0 as buy_price, 0 as buy_total, 0 as use_qty,
#used_qty:=(use_qty - use_qty_detail) as use_qty_detail,
if(remarks='Use',#new_qty_price,0) as use_price,
(#used_qty * new_qty_price) as use_total, 0 as bal_qty, 0 as old_qty,
0 as old_qty_price, 0 as new_qty,0 as new_qty_price,
#bal_qty_detail:=if(remarks='Buy', buy_qty, bal_qty) as bal_qty_detail,
new_qty_price as bal_price,
(#bal_qty_detail * new_qty_price) as bal_total, split_flag
FROM test2)
) as t order by 2
I am trying to get the average of a column after doing a division grouped by weeks.
Here is my original table
id
week
amount
div
1
5
50
5
2
5
40
3
3
4
35
3
4
4
60
10
5
6
70
9
First I want to SUM them with group by weeks, and divide the amount/div
week
amount
div
amount/div
5
90
8
11.25
4
95
13
7.30
6
70
9
7.77
Now to get the average of amount/div which is (11.25+7.3+7.77)/3 = 8.77
I just want to get the 8.77
Here is what I tried:
SELECT AVG(amount/ div) from mytable GROUP BY week
but I didn't get the desired result of 8.77
Sample DB made.
http://sqlfiddle.com/#!9/6c7fa7/9
Used FLOOR to match your values 11.25, 7.3, 7.77, usual ROUND(...,2) got 1 cent difference.
CREATE TABLE stats (
id INT PRIMARY KEY AUTO_INCREMENT,
week INT,
amount INT,
divs INT
);
INSERT INTO stats ( week, amount, divs ) VALUES ( 5, 50, 5 );
INSERT INTO stats ( week, amount, divs ) VALUES ( 5, 40, 3 );
INSERT INTO stats ( week, amount, divs ) VALUES ( 4, 35, 3 );
INSERT INTO stats ( week, amount, divs ) VALUES ( 4, 60, 10 );
INSERT INTO stats ( week, amount, divs ) VALUES ( 6, 70, 9 );
SELECT ROUND(AVG(amount_divs), 2)
FROM (
SELECT FLOOR(100* SUM(amount)/SUM(divs))/100 AS amount_divs
FROM stats
GROUP BY week
) x
If you run following query
SELECT AVG(amount/divs)
FROM stats
It has different meaning, it is the average of all records (5 values), not the 3 after SUM
i am trying to get the last rows where rec_p_id = 4 SORTED by the timestamp. Since i do not want to have all the results WHERE rec_p_id = 4, i am using GROUP BY to group it by send_p_id.
My SQL query looks like this:
SELECT *
FROM
( SELECT *
FROM chat
WHERE rec_p_id= "4"
ORDER
BY timestamp DESC) as sub
GROUP
BY send_p_id
My table looks like this:
Table chat
c_id
send_p_id
rec_p_id
timestamp
1
3
4
2020-05-01 14:46:00
2
3
4
2020-05-01 14:49:00
3
3
4
2020-05-01 14:50:00
4
7
4
2020-05-01 12:00:00
5
4
7
2020-05-01 12:10:00
6
7
4
2020-05-01 12:20:00
7
9
4
2020-05-01 16:50:00
8
9
4
2020-05-01 17:00:00
I want to get the last occurrences:
c_id
send_p_id
rec_p_id
timestamp
3
3
4
2020-05-01 14:50:00
6
7
4
2020-05-01 12:20:00
8
9
4
2020-05-01 17:00:00
But instead i get all the first ones:
c_id
send_p_id
rec_p_id
timestamp
1
3
4
2020-05-01 14:46:00
4
7
4
2020-05-01 12:00:00
7
9
4
2020-05-01 16:50:00
I saw the query i am using in this question: ORDER BY date and time BEFORE GROUP BY name in mysql
it seems to work for all of them. What am i doing wrong with my query?
Thanks in advance.
Looking to your expected result seems you are looking for
select max(c_id) c_id, send_p_id, min(timestamp) timestamp
from chat WHERE rec_p_id= "4"
group by send_p_id
ORDER BY c_id
Group by is for aggregated result ..
an use without aggregation function can produce unpredicatble result and in version > 5.6 can produce error
I used this answer and built this setup for you.
The code below is the copy of it, so that you can run it later yourself.
For the solution the example from the official manual.
CREATE TABLE chat
(
c_id INT PRIMARY KEY
, send_p_id INT
, rec_p_id INT
, timestamp DATETIME
);
INSERT INTO chat VALUES
(1, 3, 4, '2020-05-01 14:46:00')
, (2, 3, 4, '2020-05-01 14:49:00')
, (3, 3, 4, '2020-05-01 14:50:00')
, (4, 7, 4, '2020-05-01 12:00:00')
, (5, 4, 7, '2020-05-01 12:10:00')
, (6, 7, 4, '2020-05-01 12:20:00')
, (7, 9, 4, '2020-05-01 16:50:00')
, (8, 9, 4, '2020-05-01 17:00:00');
Solution:
SELECT c_id,
send_p_id,
rec_p_id,
timestamp
FROM chat AS c
WHERE timestamp=(SELECT MAX(c1.timestamp)
FROM chat AS c1
WHERE c.send_p_id = c1.send_p_id)
AND send_p_id != 4
ORDER BY timestamp;
I have a column that changes values.
I want to count by adding at each change up and subtracting at each change down. Assuming x[] are my values, Delta is the sign of change in x's elements, and y[] is my targeted results or counts.
We count up until the next delta -1 at which we start counting down, then we resume counting up when delta changes back to +1. In summary we add normally until we have a delta of -1 at that time we start subtracting, then resume adding up at the next +1 delta.
x: 1, 3, 4, 4, 4, 5, 5, 3, 3, 4, 5, 5, 6, 5, 4, 4, 4, 3, 4, 5, 6, 7, 8
Delta: 0, 1, 1, 0, 0, 1, 0, -1, 0, 1, 1, 0, 1, -1, -1, 0, 0, -1, 1, 1, 1, 1, 1
y: 1, 2, 3, 4, 5, 6, 7, 6, 5, 6, 7, 8, 9, 8, 7, 6, 5, 4, 5, 6, 7, 8, 9
The length of my array is in the millions of rows, and efficiency is important. Not sure if such operation should be done in SQL or whether I would be better off retrieving the data from the database and performing such calculation outside.
You could use this query in SQL-Server, presuming a PK-column for the ordering:
WITH CTE AS
(
SELECT t.ID, t.Value,
LastValue = Prev.Value,
Delta = CASE WHEN Prev.Value IS NULL
OR t.Value > Prev.Value THEN 1
WHEN t.Value = Prev.Value THEN 0
WHEN t.Value < Prev.Value THEN -1 END
FROM dbo.TableName t
OUTER APPLY (SELECT TOP 1 t2.ID, t2.Value
FROM dbo.TableName t2
WHERE t2.ID < t.ID
ORDER BY t2.ID DESC) Prev
)
, Changes AS
(
SELECT CTE.ID, CTE.Value, CTE.LastValue, CTE.Delta,
Change = CASE WHEN CTE.Delta <> 0 THEN CTE.Delta
ELSE (SELECT TOP 1 CTE2.Delta
FROM CTE CTE2
WHERE CTE2.ID < CTE.ID
AND CTE2.Delta <> 0
ORDER BY CTE2.ID DESC) END
FROM CTE
)
SELECT SUM(Change) FROM Changes c
The result is 9 as expected:
complete result set
only Sum
The OUTER APPLY links the current with the previous record, the previous record is the one with the highest ID < current.ID. It works similar to a LEFT OUTER JOIN.
The main challenge was the sub-query in the last CTE. That is necessary to find the last delta that is <> 0 to determine if the current delta is positive or negative.
You can also use LAG and SUM with OVER (Assuming you have SQL Server 2012 or above) like this.
Sample Data
DECLARE #Table1 TABLE (ID int identity(1,1), [x] int);
INSERT INTO #Table1([x])
VALUES (1),(3),(4),(4),(4),(5),(5),(3),(3),(4),(5),(5),(6),(5),(4),(4),(4),(3),(4),(5),(6),(7),(8);
Query
;WITH T1 as
(
SELECT ID,x,ISNULL(LAG(x) OVER(ORDER BY ID ASC),x - 1) as PrevVal
FROM #Table1
), T2 as
(
SELECT ID,x,PrevVal,CASE WHEN x > PrevVal THEN 1 WHEN x < PrevVal THEN -1 ELSE 0 END as delta
FROM T1
)
SELECT ID,x,SUM(COALESCE(NULLIF(T2.delta,0),TI.delta,0))OVER(ORDER BY ID) as Ordered
FROM T2 OUTER APPLY (SELECT TOP 1 delta from T2 TI WHERE TI.ID < T2.ID AND TI.x = T2.x AND TI.delta <> 0 ORDER BY ID DESC) as TI
ORDER BY ID
Output
ID x Ordered
1 1 1
2 3 2
3 4 3
4 4 4
5 4 5
6 5 6
7 5 7
8 3 6
9 3 5
10 4 6
11 5 7
12 5 8
13 6 9
14 5 8
15 4 7
16 4 6
17 4 5
18 3 4
19 4 5
20 5 6
21 6 7
22 7 8
23 8 9
You use sql-server and mysql tag. If this can be done within SQL-Server you should have a look on the OVER-clause: https://msdn.microsoft.com/en-us/library/ms189461.aspx
Assuming there's an ordering criteria it is possible to state a ROW-clause and use the value of a preceeding row. Many SQL-functions allow the usage of OVER.
You could define a computed column which does the calculation on insert...
Good luck!