EDIT: To clarify there are many users and each user has many records, this is a log table of user activities,
how to find the timestamp difference every record and subsequent record that satisfies some condition ,
for example assuming the table is something like this
| id |u_id| .. | timestamp |
|----|----|----|--------------------|
| 50 | 1 | .. | 2014-04-22 15:35:44|
| 90 | 2 | .. | 2014-04-22 13:35:44|
| .. | .. | .. | ..... |
How do I find the time difference between every record and the next record for only one user id ?
Assuming that you want to do this for all users, the easiest way is to use variables:
select t.*,
if(u_id = #u_id, timediff(`timestamp`, #timestamp), NULL) as diff,
#timestamp := `timestamp`, #u_id := u_id
from table t cross join
(select #timestamp := 0, #u_id := 0) var
order by u_id, timestamp;
It is important that you explicitly order the records to be sure that the processing occurs in sequential order.
Try
select timediff(`timestamp`, #lasttime),
#lasttime := `timestamp`
from your_table
cross join (select #lasttime := 0) d
where u_id = 1
order by id
There are a couple of ways you can do this, the first would be to use a correlated subquery:
SELECT T.id,
T.u_id,
timestamp,
( SELECT T2.timestamp
FROM T AS T2
WHERE T2.u_id = T.u_id
AND T2.timestamp > T.timestamp
ORDER BY T2.Timestamp
LIMIT 1
) AS NextTimeStamp
FROM T;
Or you could do this using JOIN.
SELECT T.id,
T.u_id,
T.timestamp,
T2.timestamp AS NextTimeStamp
FROM T
LEFT JOIN T AS T2
ON T2.u_id = T.u_id
AND T2.timestamp > T.timestamp
LEFT JOIN T AS T3
ON T3.u_id = T.u_id
AND T3.timestamp > T.timestamp
AND T3.timestamp < T2.timestamp
WHERE T3.id IS NULL;
Which one is best will depend on your actual requirements, amount of data, and indexes.
Related
I like to add event duration to a previous record every time a new record gets added.
This is what I have
ID EventType EventTime EventDuration
-------------------------------------
1 TypeA 10:20 NULL
2 TypeB 09:30 NULL
3 TypeC 08:00 NULL
This is what I want to achieve:
ID EventType EventTime EventDuration
-------------------------------------
1 TypeA 10:20 00:50
2 TypeB 09:30 01:30
3 TypeC 08:00 ...
4 ... ...
When a new records gets added (with ID, EventType and EventTime), the duration of the previous record (timediff between TypeB and Type A) should be added to the previous record in column EventDuration.
What I have so far is:
SELECT
id, EventTime,
timestampdiff(minute,
(SELECT EventTime
FROM TableName t2
WHERE t2.id < t1.id ORDER BY t2.id DESC LIMIT 1),EventTime)
AS EventDuration
FROM records t1
WHERE id = ....<this is where I get stuck, this should be a query that identifies the ID of latest EventTime record>
Any suggestions?
(I am running MySQL 5.6.39)
If you are running MySQL 8.0, you can use window functions for this:
update mytable t
inner join (
select id, timediff(eventTime, lag(eventTime) over(order by eventTime)) event_duration
from mytable t
) t1 on t1.id = t.id
set t.event_duration = t1.event_duration
If you want to update only the last but 1 record, you can order by and limit in the subquery (or in the outer query):
update mytable t
inner join (
select id, timediff(eventTime, lag(eventTime) over(order by eventTime)) event_duration
from mytable t
order by id desc
limit 1, 1
) t1 on t1.id = t.id
set t.event_duration = t1.event_duration
In earlier versions, one option is to emulate lag() with a window function:
update mytable t
inner join (
select
id,
timediff(
(select min(eventTime) from mytable t1 where t1.eventTime > t.eventTime),
eventTime
) event_duration
from mytable t
) t1 on t1.id = t.id
set t.event_duration = t1.event_duration
I have a table looking like this:
id | date | related_id
1 2018-01-01
2 2018-01-01
3 2018-01-02
4 2018-01-05 2
5 2018-01-06
A query SELECT * FROM table WHERE date='2018-01-01' should produce the following result:
id | date | related_id
1 2018-01-01
2 2018-01-01
4 2018-01-05 2
How can I achieve that in one MySql query?
If you have only one "layer", you can do this:
SELECT t.*
FROM theTable AS t
LEFT JOIN theTable AS rt ON t.related_id = rt.id
WHERE t.`date` = searchValue OR rt.`date` = searchValue
;
If there are an indefinite number of layers, and you have MySQL 8.0, you can use a CTE:
WITH RECURSIVE myCte AS (
SELECT * FROM theTable WHERE `date` = searchValue
UNION
SELECT t.*
FROM theTable AS t
INNER JOIN myCTE ON t.related_id = myCTE.id
)
SELECT * FROM myCTE;
Disclaimer: I am more familiar with MS-SQL CTE's, so there could be some problems with that latter option.
Extend your WHERE condition to the related_id's date:
SELECT * FROM table t
WHERE
t.date = '2018-01-01'
OR
(SELECT date FROM table WHERE id = t.related_id) = '2018-01-01'
or with a self join:
SELECT t.*
FROM table t LEFT JOIN table tt
ON tt.id = t.related_id
WHERE
t.date = '2018-01-01'
OR
tt.date = '2018-01-01'
or with EXISTS:
SELECT t.*
FROM table t
WHERE
t.date = '2018-01-01'
OR
EXISTS (
SELECT 1 FROM table
WHERE id = t.related_id AND date = '2018-01-01'
)
You can use EXISTS :
SELECT t.*
FROM table t
WHERE t.date = '2018-01-01' OR
EXISTS (SELECT 1 FROM table t1 WHERE t.related_id = t1.id);
You can simply do this:
SELECT table.* FROM table LEFT JOIN table related
ON table.related_id = related.id
WHERE
table.date = '2018-01-01'
OR related.date = '2018-01-01';
this will work with the date function:
SELECT * FROM table WHERE date=DATE('2018-01-01');
or
SELECT * FROM table WHERE date=STR_TO_DATE(DATE, '%d/%m/%Y')
I have this MySQL table named records. Below is its contents.
id record_id Data1 Data2 Time
1 1 null 1 1/1/16
2 1 1 null 1/3/16
3 1 2 null 1/4/16
4 1 null 3 1/5/16
5 2 1 null 2/1/16
6 2 1 null 2/3/16
7 2 7 null 2/4/16
8 2 null 5 2/5/16
I would like to have a MySQL query to retrieve the last non-null record of each column for each record_id. The result would look something like;
record_id Data1 Data2 Time
1 2 3 1/5/16
2 7 5 2/5/16
The tricky part to this problem is that multiple columns are involved.
SELECT t1.*
FROM yourTable t1
INNER JOIN
(
SELECT record_id, MAX(Time) AS Time
FROM yourTable
GROUP BY record_id
) t2
ON t1.record_id = t2.record_id AND
t1.Time = t2.Time
If you simply want the greatest value for the data and time columns, then see the answer given by #Matt. But your language makes it unclear what you really want.
Update:
Something like this might give the results you want:
SELECT a.record_id,
a.Data1,
b.Data2,
c.Time
FROM
(
SELECT t1.record_id,
t1.Data1
FROM yourTable t1
INNER JOIN
(
SELECT record_id,
MAX(CASE WHEN Data1 IS NULL THEN 0 ELSE id END) AS Data1Id
FROM yourTable
GROUP BY record_id
) t2
ON t1.record_id = t2.record_id AND
t1.Id = t2.Data1Id
) a
INNER JOIN
(
SELECT t1.record_id,
t1.Data2
FROM yourTable t1
INNER JOIN
(
SELECT record_id,
MAX(CASE WHEN Data2 IS NULL THEN 0 ELSE id END) AS Data2Id
FROM yourTable
GROUP BY record_id
) t2
ON t1.record_id = t2.record_id AND
t1.Id = t2.Data2Id
) b
ON a.record_id = b.record_id
INNER JOIN
(
SELECT t1.record_id,
t1.Time
FROM yourTable t1
INNER JOIN
(
SELECT record_id,
MAX(CASE WHEN Data2 IS NULL THEN 0 ELSE id END) AS TimeId
FROM yourTable
GROUP BY record_id
) t2
ON t1.record_id = t2.record_id AND
t1.Id = t2.TimeId
) c
ON a.record_id = c.record_id
Demo Here:
SQLFiddle
This one may solve your problem:
select
record_id,
substring_index(group_concat(Data1 order by Time desc), ',', 1) Data1,
substring_index(group_concat(Data2 order by Time desc), ',', 1) Data2,
substring_index(group_concat(Time order by Time desc), ',', 1) Time
from records
group by record_id
;
It may not be as fast as other answers, but is another version... give it a try. If you have a Data3 column in your table, you can copy/paste the Data1 column and just change all references of this column to the new one.
Just to explain how this works: the group_concat function concatenates all non-null values of a column with a separator (, by default). You can order the column before the concatenation. It works a bit like a window function in Oracle, Postgre, and others... The substring_index is just getting the first concatenated value, as the list is in a descending order of time.
it looks like you are just wanting the maximum data1, max data2, and max time which would be simple aggregation:
SELECT
record_id
,MAX(Data1) as Data1
,MAX(Data2) as Data2
,MAX(Time) as Time
FROM
yourTable
GROUP BY
record_id
SQL fiddle for it http://www.sqlfiddle.com/#!9/d95bc1/2
If latest non-null value per column is desired you can use:
SELECT t.record_id, MAX(t.Data1) as Data1, MAX(t.Data2) as Data2, MAX(t.Time) as Time
FROM
yourTable t
LEFT JOIN
(
SELECT
record_id, MAX(Time) as MaxTime
FROM
yourTable t
WHERE
Data1 IS NOT NULL
GROUP BY
record_id
) d1
ON t.record_id = d1.record_id
AND t.Time = d1.MaxTime
LEFT JOIN
(
SELECT
record_id, MAX(Time) as MaxTime
FROM
yourTable t
WHERE
Data2 IS NOT NULL
GROUP BY
record_id
) d2
ON t.record_id = d2.record_id
AND t.Time = d2.MaxTime
WHERE
d1.record_id IS NOT NULL
OR d2.record_id IS NOT NULL
GROUP BY
t.record_id
Using Tim's method you can actually still get to your results looking at the Latest Data1 record and then the latest Data2 record and then aggregating so they are not purely the MAX of everything but rather representative of the latest 2 records 1 for Data1 and 1 for Data2.
SQL fiddle for this part: http://www.sqlfiddle.com/#!9/d95bc1/10
I would like to have a MySQL query to retrieve the last non-null record of each column for each record_id.
Of course, what is still somewhat unclear is how you determine that a row is the last row, since rows in a database are by definition unordered.
So, my interpretation is that you want the last non-null Data1, Data2 and Time column values for each distinct record_id value. And a row value is considered last if it has a higher id value than another row value.
Assuming my understanding is correct, the following query would work:
select t.record_id,
(select t2.Data1
from tbl t2
where t2.record_id = t.record_id
and t2.Data1 is not null
order by t2.id desc
limit 1) as Data1,
(select t2.Data2
from tbl t2
where t2.record_id = t.record_id
and t2.Data2 is not null
order by t2.id desc
limit 1) as Data2,
(select t2.Time
from tbl t2
where t2.record_id = t.record_id
and t2.Time is not null
order by t2.id desc
limit 1) as Time
from tbl t
group by t.record_id
order by t.record_id
SQLFiddle Demo
I have a table with minute-by-minute data from an IOT device. Every minute there is a new row with a timestamp and a value that represents a metric. The metric starts at 0 and increments for a while before it resets and starts over.
When I plot it, it looks like the picture. I want to find the local maximum value of each run, as the blue circles indicate.
Is it possible to find and group the consecutive rows where the metric is > 0 and then find the maximum of each group?
Update
Table structure:
+-------------+------------------+
| Field | Type |
+-------------+------------------+
| id | int(10) unsigned |
| timestamp | timestamp |
| metric_name | varchar(32) |
| value | int(10) |
+-------------+------------------+
This is based on the following assumptions:
Id is a perfectly sequential integer (with no gaps)
You want to get the value logged directly before the 0 value
Code:
SELECT *
FROM metrics m1
WHERE m.id IN (
SELECT m2.id - 1
FROM metrics m2
WHERE m1.value = 0)
I join everything that isnt zero before a timestamp where it is zero, then I find the ones with no values inbetween that 0 and the last one..
SELECT
value,
timestamp
FROM
metrics
LEFT JOIN metrics zeros
on metrics.time < zeros.time
and zeros.value = 0
LEFT JOIN metrics betweenZero
on metrics.time < betweenZero.time
and betweenZero.time < zeros.time
INNER JOIN metrics noBetweens
on table.id = noBetweens.id
and betweenZero.id IS NULL
If you need it for a paritulcar metric_name, WHERE metric_name = the_metric_nameon the end.
This should give you the max value per group along with start time and end time of each window with only 1 pass over the data.
select metric_name, max(value) value, max(start_group) start_time, max(end_group) end_time from(
select metric_name, value,
case when #prev_ts is not null then #prev_ts end prev_ts,
case when value = 0 then #ts := timestamp end as start_group,
#ts as grouping,
#prev_ts := timestamp end_group
from metric join (select #prev_ts := null as p) prev
order by timestamp
) q
group by metric_name, grouping;
This will create a sample data set of 1000 rows, that resets every minute.
insert into metric(timestamp, metric_name, value)
select now() - interval rn second, 'pressure', v
from(
select #rn := #rn + 1 rn, mod(1000 - #rn,60) * pow(1000 - mod(#rn,121),1) v
from table_with_at_least_1000_rows
join (select #rn := 0) rn
limit 1000
) q
;
Try this:
SELECT
T.min_id
,T.max_id
,MAX(M.value) as local_max
FROM
metrics M
JOIN (
SELECT
id as min_id
,(
SELECT MIN(id) FROM Metrics MI
WHERE
MI.id > MO.id
AND MI.value = 0) as max_id
FROM Metrics MO
WHERE
value = 0
)T ON M.id BETWEEN T.min_id AND T.max_id
GROUP BY
T.min_id, T.max_id
My solution doesn't care about gaps but I am assuming that the sequence of ids is monotonic, that is they increase along the series by time. (You could probably substitute id for timestamp in the query even.) I had made a few minor syntax-type errors that I have since corrected since my first attempt and I have tested it with a simple Fiddle. I think it works.
select t0.*
from
T t0 inner join
(
select max_z, max(id) as max_id, max(value) as local_max
from
(
select
id, value,
(
select max(t2.id) as max_id from T t2
where t2.id < t.id and t2.value = 0
) as max_z
from T t
where t.value <> 0
) p /* partitions */
group by p.max_z
) x /* extrema */
on t0.id between max_z and max_id and t0.value = x.local_max
Btw it returns all the rows when there's a tie for the local maximum.
http://sqlfiddle.com/#!9/de832/2
I have a MySQL table of the following form
account_id | call_date
1 2013-06-07
1 2013-06-09
1 2013-06-21
2 2012-05-01
2 2012-05-02
2 2012-05-06
I want to write a MySQL query that will get the maximum difference (in days) between successive dates in call_date for each account_id. So for the above example, the result of this query would be
account_id | max_diff
1 12
2 4
I'm not sure how to do this. Is this even possible to do in a MySQL query?
I can do datediff(max(call_date),min(call_date)) but this would ignore dates in between the first and last call dates. I need some way of getting the datediff() between each successive call_date for each account_id, then finding the maximum of those.
I'm sure fp's answer will be faster, but just for fun...
SELECT account_id
, MAX(diff) max_diff
FROM
( SELECT x.account_id
, DATEDIFF(MIN(y.call_date),x.call_date) diff
FROM my_table x
JOIN my_table y
ON y.account_id = x.account_id
AND y.call_date > x.call_date
GROUP
BY x.account_id
, x.call_date
) z
GROUP
BY account_id;
CREATE TABLE t
(`account_id` int, `call_date` date)
;
INSERT INTO t
(`account_id`, `call_date`)
VALUES
(1, '2013-06-07'),
(1, '2013-06-09'),
(1, '2013-06-21'),
(2, '2012-05-01'),
(2, '2012-05-02'),
(2, '2012-05-06')
;
select account_id, max(diff) from (
select
account_id,
timestampdiff(day, coalesce(#prev, call_date), call_date) diff,
#prev := call_date
from
t
, (select #prev:=null) v
order by account_id, call_date
) sq
group by account_id
| ACCOUNT_ID | MAX(DIFF) |
|------------|-----------|
| 1 | 12 |
| 2 | 4 |
see it working live in an sqlfiddle
If you have an index on account_id, call_date, then you can do this rather efficiently without variables:
select account_id, max(call_date - prev_call_date) as diff
from (select t.*,
(select t2.call_date
from table t2
where t2.account_id = t.account_id and t2.call_date < t.call_date
order by t2.call_date desc
limit 1
) as prev_call_date
from table t
) t
group by account_id;
Just for educational purposes, doing it with JOIN:
SELECT t1.account_id,
MAX(DATEDIFF(t2.call_date, t1.call_date)) AS max_diff
FROM t t1
LEFT JOIN t t2
ON t2.account_id = t1.account_id
AND t2.call_date > t1.call_date
LEFT JOIN t t3
ON t3.account_id = t1.account_id
AND t3.call_date > t1.call_date
AND t3.call_date < t2.call_date
WHERE t3.account_id IS NULL
GROUP BY t1.account_id
Since you didn't specify, this shows max_diff of NULL for accounts with only 1 call.
SELECT a1.account_id , max(a1.call_date - a2.call_date)
FROM account a2, account a1
WHERE a1.account_id = a2.account_id
AND a1.call_date > a2.call_date
AND NOT EXISTS
(SELECT 1 FROM account a3 WHERE a1.call_date > a3.call_date AND a2.call_date < a3.call_date)
GROUP BY a1.account_id
Which gives :
ACCOUNT_ID MAX(A1.CALL_DATE - A2.CALL_DATE)
1 12
2 4