Mysql query to skip rows and check for status changes - mysql

Question

I'm building a mysql query but I'm stuck... (I'm logging each minute)
I have 3 tables. Logs, log_field, log_value.
logs -> id, create_time
log_value -> id, log_id,log_field_id,value
log_field -> id, name (one on the entries is status and username)
The values for status can be online,offline and idle...
What I would like to see is from my query is:
When in my logs someone changes from status, I want a row with create_time, username, status.
So for a given user, I want my query to skip rows until a new status appears...
And I need to be able to put a time interval in which status changes are ignored.
Can someone please help ?

Answer 1

Although you have nothing to differentiate an actual "User" (such as by user ID) listed in your post, and what happens if you have two "John Smith" names.
First, an introduction to MySQL #variables. You can think of them as an inline program running while the query is processing rows. You create variables, then change them as each row gets processed, IN THE SAME order as the := assignment in the field selection occurs which is critical. I'll cover that shortly.
Fist an initial premise. You have a field value table of all possible fields that can/do get logged. Of which, two of them exist... one is for the user's name, another for the status you are looking a log changed. I don't know what those internal "ID" numbers are, but they would have to be fixed values per your existing table. In my scenario, I am assuming that field ID = 1 is for the User's Name, and field ID 2 = status column... Otherwise, you would need two more joins to get the field table just to confirm which field was the one you wanted. Obviously my "ID" field values will not match your production tables, so please change those accordingly.
Here's the query...
select FinalAlias.*
from (
select
PQ.*,
if( #lastUser = PQ.LogUser, 1, 0 ) as SameUser,
#lastTime := if( #lastUser = PQ.LogUser, #lastTime, #ignoreTime ) as lastChange,
if( PQ.create_time > #lastTime + interval 20 minute, 1, 0 ) as BeyondInterval,
#lastTime := PQ.create_time as chgTime,
#lastUser := PQ.LogUser as chgUser
from
( select
ByStatus.id,
l.create_time,
ByStatus.Value LogStatus,
ByUser.Value LogUser
from
log_value as ByStatus
join logs l
on ByStatus.log_id = l.id
join log_value as ByUser
on ByStatus.log_id = ByUser.log_id
AND ByUser.log_field_id = 1
where
ByStatus.log_field_id = 2
order by
ByUser.Value,
l.create_time ) PQ,
( select #lastUser := '',
#lastTime := now(),
#ignoreTime := now() ) sqlvars
) FinalAlias
where
SameUser = 1
and BeyondInterval = 1
Now, what's going on. The inner-most query (result alias PQ representing "PreQuery") is just asking for all log values where the field_id = 2 (status column) exists. From that log entry, go to the log table for it's creation time... while we're at it, join AGAIN to the log value table on the same log ID, but this time also look for field_id = 1 so we can get the user name.
Once that is done, get the log ID, Creation time, Status Value and Who it was for all pre-sorted on a per-user basis and sequentially time oriented. This is the critical step. The data must be pre-organized by user/time to compare the "last" time for a given user to the "next" time their log status changed.
Now, the MySQL #variables. Join the prequery to another select of #variables which is given an "sqlvars" query alias. This will pre-initialize the variables fo #lastUser, #lastTime and #ignoreTime. Now, look at what I'm doing in the field list via section
if( #lastUser = PQ.LogUser, 1, 0 ) as SameUser,
#lastTime := if( #lastUser = PQ.LogUser, #lastTime, #ignoreTime ) as lastChange,
if( PQ.create_time > #lastTime + interval 20 minute, 1, 0 ) as BeyondInterval,
#lastTime := PQ.create_time as chgTime,
#lastUser := PQ.LogUser as chgUser
This is like doing the following pseudo code in a loop for every record (which is already sequentially ordered by same person and their respective log time
FOR EACH ROW IN RESULT SET
Set a flag "SameUser" = 1 if the value of the #lastUser is the same
as the current person record we are looking at
if the last user is the same as the previous record
use the #lastTime field as the "lastChange" column
else
use the #ignore field as the last change column
Now, build another flag based on the current record create time
and whatever the #lastTime value is based on a 20 minute interval.
set it to 1 if AT LEAST the 20 minute interval has been meet.
Now the key to the cycling the next record.
force the #lastTime = current record create_time
force the #lastUser = current user
END FOR LOOP
So, if you have the following as a result of the prequery... (leaving date portion off)
create status user sameuser lastchange 20minFlag carry to next row compare
07:34 online Bill 0 09:05 0 07:34 Bill
07:52 idle Bill 1 07:34 0 07:52 Bill
08:16 online Bill 1 07:52 1 08:16 Bill
07:44 online Mark 0 09:05 0 07:44 Mark
07:37 idle Monica 0 09:05 0 07:37 Monica
08:03 online Monica 1 07:37 1 08:03 Monica
Notice first record for Bill. The flag same user = 0 since there was nobody before him. The last change was 9:05 (via the NOW() when creating the sqlvars variables), but then look at the "carry to next row compare". This is setting the #lastTime and #lastUser after the current row was done being compared as needed.
Next row for Bill. It sees he is same as last user previous row, so the SameUser flag is set to 1. We now know that we have a good "Last Time" to compare against the current record "Create Time". So, from 7:34 to 7:52 is 18 minutes and LESS than our 20 minute interval so the 20 minute flag is set to 0. Now, we retain the current 7:52 and Bill for third row.
Third row for Bill. Still Same User (flag=1), last change of 7:52 compared to now 8:16 and we have 24 minutes... So the 20 minute flag = 1. Retain 8:16 and Bill for next row.
First row for Mark. Same User = 0 since last user was Bill. Uses same 9:05 ignore time and don't care about 20 min flag, but now save 7:44 and Mark for next row compare.
On to Monica. Different than Mark, so SameUser = 0, etc to finish similar to Bill.
So, now we have all the pieces and rows considered. Now, take all these and wrap them up as the "FinalAlias" of the query and all we do is apply a WHERE clause for "SameUser = 1" AND "20 Minute Flag" has been reached.
You can strip down the final column list as needed, and remove the where clause to look at results, but be sure to add an outer ORDER BY clause for name/create_time to see similar pattern as I have here.