Referencing a column in a sub-query's where clause - mysql

I have a table that has records for multiple session events. Each row is an event in a session, and a session can have multiples of the same event. Those are basically game sessions and each event is round start or round end. My data looks something like
Session_id | Event_type | Event_Time
1 | round_start | 12:01:00
1 | round_end | 12:02:00
1 | round_start| 12:05:00
1 | round_end | 12:7:00
2 | round_start | 14:11:00
2 | round_end | 14:12:00
3 | round_start| 15:09:00
3 | round_end | 15:13:00
I am trying to find the average round duration. I tried the following SQL
select
RS.session_id,
RS.Event_Time as StartTime,
RE.EndTime,
TIMESTAMPDIFF(MINUTE,RE.EndTime,RS.Event_Time) as duration
from amp_event_mamlaka as RS
left join
(
select session_id, min(event_time) as EndTimd from amp_event_mamlaka
where Event_Type = "Round End" and session_id = RS.session_id and event_time>RS.Event_Time
) RE
on RE.session_id = RS.session_id
The issue is that I can't reference RS.session_id and RS.event_time in the joined table.
I am using MySQL. Any suggestions on how to accomplish this?
Thanks

I would suggest that you approach this with a correlated subquery:
select RS.session_id, RS.Event_Time as StartTime,
(select smin(event_time)
from amp_event_mamlaka em
where em.session_id = RS.session_id and
em.Event_Type = 'Round End' and
em.event_time > RS.Event_Time
) as EndTime,
from amp_event_mamlaka RS;
You can do the timestamp difference using a subquery:
select RS.*, TIMESTAMPDIFF(MINUTE, EndTime, Event_Time) as duration
from (select RS.session_id, RS.Event_Time as StartTime,
(select min(event_time)
from amp_event_mamlaka em
where em.session_id = RS.session_id and
em.Event_Type = 'Round End' and
em.event_time > RS.Event_Time
) as EndTime
from amp_event_mamlaka RS
) RS

A subquery, as opposed to a nested query, should only return one value. Your requirement is an example where you want data from pairs of rows. The subquery is only used to connect the pair, not supply data. Fiddle
select e1.SessionID, e1.EventType, e1.EventTime, e2.EventType, e2.EventTime, TimeStampDiff( minute, e1.EventTime, e2.EventTime ) Duration
from Events e1
join Events e2
on e2.SessionID = e1.SessionID
and e2.EventType = 'end'
and e2.EventTime =(
select Min( EventTime )
from Events
where SessionID = e1.SessionID
and EventType = 'end'
and EventTime > e1.EventTime )
where e1.EventType = 'start';

Instead of keeping it in where clause of subquery you can keep the condition in Join On clause. Try this.
SELECT RS.session_id,
RS.Event_Time AS StartTime,
RE.EndTime,
Timestampdiff(MINUTE, RE.EndTime, RS.Event_Time) AS duration
FROM amp_event_mamlaka AS RS
LEFT JOIN (SELECT session_id,
Min(event_time) AS EndTimd
FROM amp_event_mamlaka
WHERE Event_Type = "Round End") RE
ON RE.session_id = RS.session_id
AND RE.event_time > RS.Event_Time

Related

Need to Pick Max Date when status = N otherwise No in MYSQL

I have a table which have records like this
ID DATEADD STATUS
'A0011' '04/01/2018 11:58:31' 'C'
'A0011' '31/05/2019 10:02:36' 'N'
'B0022' '04/01/2018 11:58:31' 'N'
'B0022' '31/05/2019 10:02:36' 'N'
'B0022' '30/04/2020 19:44:36' 'C'
'C0033' '04/01/2018 11:58:31' 'N'
'C0033' '30/05/2019 06:02:36' 'C'
'C0033' '29/04/2020 05:44:36' 'C'
I'm trying to get the Max Date for each ID which have STATUS = 'N'. If I get MAX DATE and STATUS = 'C' then I don't want that record.
Output :
ID DATEADD STATUS
'A0011' '31/05/2019 10:02:36' 'N'
SCRIPT :
SELECT I.* FROM INVOICE I
INNER JOIN (
Select ID,MAX(DATEADD)DATEADD,STATUS FROM INVOICE WHERE STATUS = 'N'
GROUP BY ID,STATUS) O
ON I.ID = O.ID AND O.DATEADD = I.DATEADD
But I'm not able to get desired output.
If your mysql version support the window function, we can try to use ROW_NUMBER window function to get each ID latest DATEADD then compare the STATUS
SELECT *
FROM (
SELECT *,ROW_NUMBER() OVER(PARTITION BY ID ORDER BY DATEADD DESC) rn
FROM INVOICE
) t1
WHERE rn = 1 AND STATUS = 'N'
sqlfiddle
if your MySQL version didn't support the window function we can try to use correlated subquery
SELECT *
FROM (
SELECT *, (SELECT COUNT(*)
FROM INVOICE tt
WHERE tt.ID = t1.ID AND tt.DATEADD > t1.DATEADD) rn
FROM INVOICE t1
) t1
WHERE rn = 1 AND STATUS = 'N'
sqlfiddle
You can use NOT EXISTS:
SELECT i1.*
FROM INVOICE i1
WHERE i1.STATUS = 'N'
AND NOT EXISTS (
SELECT 1
FROM INVOICE i2
WHERE i2.ID = i1.ID
AND STR_TO_DATE(i2.DATEADD, '%d/%m/%Y %H:%i:%s') > STR_TO_DATE(i1.DATEADD, '%d/%m/%Y %H:%i:%s')
);
If the column's DATEADD data type is DATETIME or TIMESTAMP the last condition would be simpler:
...AND i2.DATEADD > i1.DATEADD
See the demo.
We can use ORDER BY and LIMIT 1 to get the row that we want without using any functions, sub-queries, CTE etc.
Thank you to D-Shih for the test schema.
If we want the maximum date with status 'N' for each ID we can use the second query.
SELECT
ID,
DATEADD,
STATUS
FROM INVOICE
ORDER BY
STATUS DESC,
DATEADD DESC
LIMIT 1;
ID | DATEADD | STATUS
:---- | :--------- | :-----
A0011 | 2019-05-31 | N
SELECT
ID,
MAX(DATEADD) AS DATEADD,
STATUS
FROM INVOICE
WHERE STATUS = 'N'
GROUP BY ID
ORDER BY ID;
ID | DATEADD | STATUS
:---- | :--------- | :-----
A0011 | 2019-05-31 | N
B0022 | 2019-05-31 | N
C0033 | 2018-01-04 | N
db<>fiddle here

SQL running total from previous year

How to do running total or cumulative sum from this query? Is it possible to run by correlated subquery? The cumulative sum results will be show as 'New value'.
SELECT
sum(data2.quantity/1000) AS UnitMT
FROM
data2
INNER JOIN itmnocate ON data2.item = itmnocate.ItemNumber
and Source in ('imported','local','by product')
WHERE date1 >= DATE_FORMAT('2018-04-12', '%Y-01-01')- INTERVAL 1 YEAR
AND date1 <= DATE_FORMAT('2018-04-12', '%Y-12-31') - INTERVAL 1 YEAR
AND data2.unit = 'KG'
and data2.customeracc not in (select Customeraccount from custlist WHERE Custcat = 'bcsb')
GROUP BY month(date1)
This is a cumulative problem If your mysql version support window function you can use SUM with window function to do cumulative.
The DATE1 column can be used as the basis for order by to do cumulative.
SELECT *,
sum(UnitMT) over (order by month(date1)) 'New value'
FROM T t1
sqlfiddle
If your mysql version didn't support window function, you can try to use subquery in select to do cumulative.
CREATE TABLE T( date1 DATE,UnitMT int);
INSERT INTO T VALUES ('2017-01-01',66535);
INSERT INTO T VALUES ('2017-02-01',108337);
INSERT INTO T VALUES ('2017-03-01',132767);
INSERT INTO T VALUES ('2017-04-01',100687);
INSERT INTO T VALUES ('2017-05-01',125151);
Query 1:
SELECT *,
(SELECT SUM(UnitMT) FROM T tt WHERE month(tt.date1) <= month(t1.date1)) 'New value'
FROM T t1
Results:
| date1 | UnitMT | New value |
|------------|--------|-----------|
| 2017-01-01 | 66535 | 66535 |
| 2017-02-01 | 108337 | 174872 |
| 2017-03-01 | 132767 | 307639 |
| 2017-04-01 | 100687 | 408326 |
| 2017-05-01 | 125151 | 533477 |
Note
T symbol your current result set data.
You could try this query, it's efficient and works an all MySQL versions:
select #cumSum := 0;
select UnitMT, #cumSum := #cumSum + UnitMT
from tbl
order by date1;
Demo
For your specific problem, you can use variables and a subquery:
SELECT mon, UnitMT,
(#sum := #sum + UnitMT) as running_sum
FROM (SELECT month(date1) as mon, sum(data2.quantity/1000) AS UnitMT
FROM data2 INNER JOIN
itmnocate
ON data2.item = itmnocate.ItemNumber AND
Source IN ('imported', 'local', 'by product')
WHERE date1 >= DATE_FORMAT('2018-04-12', '%Y-01-01') - INTERVAL 1 YEAR AND
date1 <= DATE_FORMAT('2018-04-12', '%Y-12-31') - INTERVAL 1 YEAR AND
data2.unit = 'KG' AND
data2.customeracc not in (select Customeraccount from custlist WHERE Custcat = 'bcsb')
GROUP BY month(date1)
ORDER BY month(date1)
) m CROSS JOIN
(SELECT #sum := 0) params;

How to get column value from next record

I have a following result set:
POST | DATE
--------------------------------------
Senior Software Engg. | 2018-04-18
Software Engg. | 2017-04-18
Assoc. Software Engg. | 2016-04-18
SQL query:
SELECT DISTINCT designation_id as id, d.title as POST, DATE(dt_datetime) as DATE
FROM users_history_check u
INNER JOIN
designations d
ON d.id = u.designation_id
WHERE u.id = $userID
ORDER BY DATE DESC
I want to fetch next record and perform date difference calculation in months, and display records.
Expected Output :
POST | Start DATE | End DATE | MONTHS
---------------------------------------------------------------
Senior Software Engg. | 2018-04-18 | - |
Software Engg. | 2017-04-18 | 2018-04-18 | 12
Assoc. Software Engg. | 2016-04-18 | 2017-04-18 | 12
Something like :
SELECT DISTINCT designation_id as id, d.title as POST, DATE(dt_datetime) as Start DATE, NEXT_RECORD(DATE(dt_datetime)) as End DATE, DATEDIFF(Start DATE, End DATE) as MONTHS....
Any help is very much appreciated. Thanks.
SELECT `POST`,
`DATE`,
IFNULL(END_DATE,'') AS END_DATE,
IFNULL(MONTH,'') AS MONTH
FROM
(SELECT `POST`,
`DATE`,
#prev AS END_DATE,
TIMESTAMPDIFF(month,DATE,#prev) AS MONTH,
#prev := T.DATE AS VarDate
FROM Table1 T,
(SELECT #prev:=null)R
) T1
OUTPUT
POST DATE END_DATE MONTH
Senior Software Engg. 2018-04-18
Software Engg. 2017-04-18 2018-04-18 12
Assoc. Software Engg. 2016-04-18 2017-04-18 12
Demo Link
http://sqlfiddle.com/#!9/33260/15
EXPLANATION:
In Sub query, I am saving the Date value in #prev variable and in each row using that variable to calculate the END_DATE before assigning the current date value from Column Date.
Then using the sub query to present the data in a proper way.
You can get the previous date using variables:
SELECT id, post, date,
(CASE WHEN (#tmp_prevd := #prevd) = NULL THEN NULL -- never happens
WHEN (#prevd := date) = NULL THEN NULL -- never happens
ELSE #tmp_prevd
END) as prev_date
FROM (SELECT DISTINCT designation_id as id, d.title as POST, DATE(dt_datetime) as DATE
FROM users_history_check u INNER JOIN
designations d
ON d.id = u.designation_id
WHERE u.id = $userID
ORDER BY DATE DESC
) ud CROSS JOIN
(SELECT #prevd := NULL) params;
This is tricky, because all references to a variable need to be in the same expression. That is why this uses CASE in a rather arcane way.
In MySQL 8.0 and basically all other databases, you could use LEAD() instead.
Try This....
SELECT T1.POST,T1.DATE ,T2.DATE,DATEDIFF(MONTH,T1.DATE,T2.DATE)
FROM(
SELECT ROW_NUMBER()OVER(ORDER BY DATE DESC) AS SlNo,*
FROM Mytable)T1
LEFT JOIN (SELECT ROW_NUMBER()OVER(ORDER BY DATE DESC)+1 AS SlNo,*
FROM Mytable)T2
ON(T1.SlNo = T2.SlNo )
I suggest using self-join like this
select d1.post,
d1.d `start DATE`,
min(d2.d) `end DATE`,
timestampdiff(month, d1.d, min(d2.d)) `MONTHS`
from data d1
left join data d2 on d1.d < d2.d
group by d1.post, d1.d
dbfiddle demo
The data table is the result of your SQL. It can be added using WITH or you may use subquery as well.
SELECT * ,Datediff(Month,[Date],endate)
FROM
(
SELECT *,Lead( [Date], 1, Null) OVER (
ORDER BY [Date]) AS Endate --INTO SourceTable
FROM
(
SELECT 'Senior Software Engg.' POST , '2018-04-18' DATE UNION ALL
SELECT 'Software Engg.' POST , '2017-04-18' DATE UNION ALL
SELECT 'Assoc. Software Engg.' POST , '2016-04-18' DATE
)A
)B
ORDER BY [Date] desc

Using IFNULL in where clause

I have a problem with an SQL statement not returning any result from a specific result.
SELECT statementbalance AS 'BringForwardFromPreviousDay',
IFNULL((SELECT SUM(statementdebit) FROM statement WHERE merchantid = '4' AND statementdate = '2018-01-08'),0) AS 'TotalDebit',
IFNULL((SELECT SUM(statementcredit) FROM statement WHERE merchantid = '4' AND statementdate = '2018-01-08'),0) AS 'TotalCredit',
IFNULL((SELECT statementbalance FROM statement WHERE merchantid = '4' AND statementdate <= '2018-01-08' ORDER BY transactionid DESC LIMIT 1),0) AS 'TotalBalance'
FROM statement
WHERE merchantid = '4' and statementdate <= '2018-01-07'
ORDER BY transactionid DESC LIMIT 1
The SQL statement is to capture data dated from today '2018-01-08'and from previous days '2018-01-07' and grab the last recorded data hence the '<='
Because merchantid = '4' is a newly added merchant, it does not have any data to grab from <= 2018-01-07 hence i want it to return 0 instead of null and prevent other data from returning null.
I tried adding ifnull on the statementnbalance but it still returns null and i can only think of including ifnull on the where clause but i tried to no avail.
Here is the sqlfiddle of using merchantid '2' that works fine.
http://sqlfiddle.com/#!9/7cae3e0/1
I think this is what you want:
select
x.merchantid as merchantid,
ifnull(c.previous_days_balance, 0) as BringForwardFormPreviousDay,
ifnull(a.latest_total_debit, 0) as TotalDebit,
ifnull(a.latest_total_credit, 0) as TotalCredit,
ifnull(b.latest_balance, 0) as LatestBalance
from
(
select distinct merchantid from statement
) x
left outer join
(
select
merchantid,
sum(statementdebit) as latest_total_debit,
sum(statementcredit) as latest_total_credit
from
statement
where
statementdate = '2018-01-08'
group by
merchantid
) a
on x.merchantid = a.merchantid
left outer join
(
select
merchantid,
statementbalance as latest_balance
from
statement
where
(merchantid, transactionid) in
(
select
merchantid,
max(transactionid)
from
statement
where
statementdate = '2018-01-08'
group by
merchantid
)
) b
on x.merchantid = b.merchantid
left outer join
(
select
merchantid,
statementbalance as previous_days_balance
from
statement
where
(merchantid, transactionid) in
(
select
merchantid,
max(transactionid)
from
statement
where
statementdate <= '2018-01-07'
group by
merchantid
)
) c
on x.merchantid = c.merchantid;
I added another row to illustrate the extra case:
INSERT INTO statement VALUES ('99', '5', '131', 'Purchase: TopUp Cheezy', '2018-01-05', '23:35:31', '38.20', '0.00', '5000.00');
The results are:
+------------+-----------------------------+------------+-------------+----------------+
| merchantid | BringForwardFormPreviousDay | TotalDebit | TotalCredit | LatestBalance |
+------------+-----------------------------+------------+-------------+----------------+
| 1 | 35 | 15 | 0 | 5 |
| 2 | 182.33 | 4.9 | 0 | 177.43 |
| 4 | 0 | 95.48 | 200 | 104.52 |
| 5 | 5000 | 0 | 0 | 0 |
+------------+-----------------------------+------------+-------------+----------------+
4 rows in set (0.00 sec)
This assumes that transactionId keeps on increasing with time. That is not an entirely safe assumption. It would be better to use timestamps rather than dates for the transaction so you can find the latest one (or the latest one that is before today). I see that you do have statementtime but a separate column...
The query doesn't return anything because there are no data to return that fit your where clause. What you could do is using an IFNULL on your whole select statement:
SELECT ifnull((SELECT statementbalance AS 'BringForwardFromPreviousDay'
FROM statement
WHERE merchantid = '4'
AND statementdate <= '2018-01-07'
ORDER BY transactionid DESC LIMIT 1), 0)
and work from there. However, your statement can only contain one column, so you either have to work this into a stored procedure, or you have to deal with this issue at a different place (maybe in your code?)
You need to use a LEFT JOIN to allow for rows that don't exist.
SELECT IFNULL(t3.statementbalance, 0) AS BringForwardFromPreviousDay,
t1.TotalDebit, t1.TotalCredit, t2.TotalBalance
FROM (SELECT SUM(statementdebit) AS TotalDebit,
SUM(statementcredit) AS TotalCredit
FROM statement
WHERE merchantId = '4' AND statementdate = '2018-01-08') AS t1
CROSS JOIN (
SELECT statementbalance AS TotalBalance
FROM statement
WHERE merchantId = '4' AND statementdate <= '2018-01-08'
ORDER BY statementdate DESC
LIMIT 1) AS t2
LEFT JOIN (
SELECT statementbalance
FROM statement
WHERE t3.merchantID = '4' AND t3.statementdate <= '2018-01-07'
ORDER BY statementdate DESC
LIMIT 1) ON 1=1

Finding local maximum between zero-values using SQL

I have a table with minute-by-minute data from an IOT device. Every minute there is a new row with a timestamp and a value that represents a metric. The metric starts at 0 and increments for a while before it resets and starts over.
When I plot it, it looks like the picture. I want to find the local maximum value of each run, as the blue circles indicate.
Is it possible to find and group the consecutive rows where the metric is > 0 and then find the maximum of each group?
Update
Table structure:
+-------------+------------------+
| Field | Type |
+-------------+------------------+
| id | int(10) unsigned |
| timestamp | timestamp |
| metric_name | varchar(32) |
| value | int(10) |
+-------------+------------------+
This is based on the following assumptions:
Id is a perfectly sequential integer (with no gaps)
You want to get the value logged directly before the 0 value
Code:
SELECT *
FROM metrics m1
WHERE m.id IN (
SELECT m2.id - 1
FROM metrics m2
WHERE m1.value = 0)
I join everything that isnt zero before a timestamp where it is zero, then I find the ones with no values inbetween that 0 and the last one..
SELECT
value,
timestamp
FROM
metrics
LEFT JOIN metrics zeros
on metrics.time < zeros.time
and zeros.value = 0
LEFT JOIN metrics betweenZero
on metrics.time < betweenZero.time
and betweenZero.time < zeros.time
INNER JOIN metrics noBetweens
on table.id = noBetweens.id
and betweenZero.id IS NULL
If you need it for a paritulcar metric_name, WHERE metric_name = the_metric_nameon the end.
This should give you the max value per group along with start time and end time of each window with only 1 pass over the data.
select metric_name, max(value) value, max(start_group) start_time, max(end_group) end_time from(
select metric_name, value,
case when #prev_ts is not null then #prev_ts end prev_ts,
case when value = 0 then #ts := timestamp end as start_group,
#ts as grouping,
#prev_ts := timestamp end_group
from metric join (select #prev_ts := null as p) prev
order by timestamp
) q
group by metric_name, grouping;
This will create a sample data set of 1000 rows, that resets every minute.
insert into metric(timestamp, metric_name, value)
select now() - interval rn second, 'pressure', v
from(
select #rn := #rn + 1 rn, mod(1000 - #rn,60) * pow(1000 - mod(#rn,121),1) v
from table_with_at_least_1000_rows
join (select #rn := 0) rn
limit 1000
) q
;
Try this:
SELECT
T.min_id
,T.max_id
,MAX(M.value) as local_max
FROM
metrics M
JOIN (
SELECT
id as min_id
,(
SELECT MIN(id) FROM Metrics MI
WHERE
MI.id > MO.id
AND MI.value = 0) as max_id
FROM Metrics MO
WHERE
value = 0
)T ON M.id BETWEEN T.min_id AND T.max_id
GROUP BY
T.min_id, T.max_id
My solution doesn't care about gaps but I am assuming that the sequence of ids is monotonic, that is they increase along the series by time. (You could probably substitute id for timestamp in the query even.) I had made a few minor syntax-type errors that I have since corrected since my first attempt and I have tested it with a simple Fiddle. I think it works.
select t0.*
from
T t0 inner join
(
select max_z, max(id) as max_id, max(value) as local_max
from
(
select
id, value,
(
select max(t2.id) as max_id from T t2
where t2.id < t.id and t2.value = 0
) as max_z
from T t
where t.value <> 0
) p /* partitions */
group by p.max_z
) x /* extrema */
on t0.id between max_z and max_id and t0.value = x.local_max
Btw it returns all the rows when there's a tie for the local maximum.
http://sqlfiddle.com/#!9/de832/2