Selecting the number of last consecutive days from timestamp (excepting today) - mysql

I have a table A_DailyLogins with the columns ID (auto increment), Key (userid) and Date (timestamp). I want a query which would return the number of last consecutive days from those timestamp based on the Key, for example if he has a row for yesterday, one for two days ago and another one for three days ago, but the last one isn't from four days ago, it would return 3, because this is the number of last days the user was logged in.
My attempt was to create a query selecting the last 7 rows of the players ordered by Date DESC (this is what I wanted in the first place, but then I thought that it would be great to have all the last consecutive days), and then I retrieved the query result and compared the dates (converted to year/month/day with functions from that language [Pawn]) and increased the number of consecutive days when a date is before the other one with one day. (but this is extremely slow compared to what I think that can be done directly only with MySQL)
The closest thing I found is this: Check for x consecutive days - given timestamps in database . But it still isn't how I want it to be, it's still pretty different. I tried to modify it, but it is way too hard for me, I don't have that much experience in MySQL.

context
let consecutive login period be a period where the user is logged in on all days ( has an entry in A_DailyLogins on every day in period ) where there is no entry in A_DailyLogins immediately before or after the consecutive login period with the same user
and number of consecutive days be the difference between the maximum and minumum dates in a consecutive login period
the maximum date of a consecutive login period has no login entry immediately after ( sequentially ) to it..
the minimum date of a consecutive login period has no login entry immediately previous ( sequentially ) to it..
plan
left join A_DailyLogins to itself using same user and sequential dates where right is null to find maximums
analogous logic to find minimums
calculate row ordering over minimums and maximums with appropriate order by
join maximums and minimums on row number
filter where maximum login is yesterday/today
calculate date_diff between maximum and minimum in range
left join users to above resultset and coalesce over the case where user does not have a consecutive login period ending yesterday/today
input
+----+------+------------+
| ID | Key | Date |
+----+------+------------+
| 25 | eric | 2015-12-23 |
| 26 | eric | 2015-12-25 |
| 27 | eric | 2015-12-26 |
| 28 | eric | 2015-12-27 |
| 29 | eric | 2016-01-01 |
| 30 | eric | 2016-01-02 |
| 31 | eric | 2016-01-03 |
| 32 | nusa | 2015-12-27 |
| 33 | nusa | 2015-12-29 |
+----+------+------------+
query
select all_users.`Key`,
coalesce(nconsecutive, 0) as nconsecutive
from
(
select distinct `Key`
from A_DailyLogins
) all_users
left join
(
select
lower_login_bounds.`Key`,
lower_login_bounds.`Date` as from_login,
upper_login_bounds.`Date` as to_login,
1 + datediff(least(upper_login_bounds.`Date`, date_sub(current_date, interval 1 day))
, lower_login_bounds.`Date`) as nconsecutive
from
(
select curr_login.`Key`, curr_login.`Date`, #rn1 := #rn1 + 1 as row_number
from A_DailyLogins curr_login
left join A_DailyLogins prev_login
on curr_login.`Key` = prev_login.`Key`
and prev_login.`Date` = date_add(curr_login.`Date`, interval -1 day)
cross join ( select #rn1 := 0 ) params
where prev_login.`Date` is null
order by curr_login.`Key`, curr_login.`Date`
) lower_login_bounds
inner join
(
select curr_login.`Key`, curr_login.`Date`, #rn2 := #rn2 + 1 as row_number
from A_DailyLogins curr_login
left join A_DailyLogins next_login
on curr_login.`Key` = next_login.`Key`
and next_login.`Date` = date_add(curr_login.`Date`, interval 1 day)
cross join ( select #rn2 := 0 ) params
where next_login.`Date` is null
order by curr_login.`Key`, curr_login.`Date`
) upper_login_bounds
on lower_login_bounds.row_number = upper_login_bounds.row_number
where upper_login_bounds.`Date` >= date_sub(current_date, interval 1 day)
and lower_login_bounds.`Date` < current_date
) last_consecutive
on all_users.`Key` = last_consecutive.`Key`
;
output
+------+------------------+
| Key | last_consecutive |
+------+------------------+
| eric | 2 |
| nusa | 0 |
+------+------------------+
valid as run on 2016-01-03
sqlfiddle

Related

Get the hourly average amount based on rows with minutely timestamp

I have a table structure that looks like this:
+---------+------------+----------+---------+-----------+------------+
| id | account_id | hashrate | workers | sharerate | timestamp |
+---------+------------+----------+---------+-----------+------------+
| 1227368 | 42 | 405211 | 1 | 6183 | 1534264380 |
| 1227367 | 12 | 450077 | 1 | 6868 | 1534264380 |
+---------+------------+----------+---------+-----------+------------+
A row is created every minute for a user with an active worker.
I'm attempting to display a chart that will show the average hashrate and sharerate for each hour over a 24 hour period.
My sql query so far is:
SELECT
timestamp,
SUM(hashrate) as hashes,
SUM(sharerate) AS shares
FROM statistics_users
WHERE FROM_UNIXTIME(timestamp) >= DATE_SUB(NOW(), INTERVAL 1 DAY)
GROUP BY timestamp;
which returns:
+------------+------------+----------+
| timestamp | hashes | shares |
+------------+------------+----------+
| 1534177980 | 2282744244 | 34831913 |
Which is returning 1440 objects (one for each minute of day). I want to group the items by hour and average all sums returned from my query of minutes into an hour. Not sure where to go from here and I'm fairly new to writing sql queries, so any help would be much appreciated.
We can use an expression that returns a representation of hour, basically trimming off minutes and seconds, and then GROUP BY that expression. One possibility it to use the MySQL DATE_FORMAT function.
SELECT DATE_FORMAT(FROM_UNIXTIME(t.timestamp),'%Y-%m-%d %H') AS ts_hr
, AVG(t.hashes)
, AVG(t.shares)
FROM (
-- original query goes here as inline view
SELECT timestamp
, SUM(hashrate) as hashes
, SUM(sharerate) AS shares
FROM statistics_users
WHERE FROM_UNIXTIME(timestamp) >= NOW() + INTERVAL -1 DAY
GROUP BY timestamp
) t
GROUP
BY DATE_FORMAT(FROM_UNIXTIME(t.timestamp),'%Y-%m-%d %H')
-- ^^^^^^^^^^^^ ^^^^^^^^^^^^^^^
Note that the condition in the WHERE clause doesn't respect hour boundaries, so we're going to get a partial result for first hour, and a partial last hour.
The specification isn't clear, as to whether we want the average of the sums for each timestamp, or more simply
SELECT DATE_FORMAT(FROM_UNIXTIME(t.timestamp),'%Y-%m-%d %H') AS ts_hr
, AVG(t.hashrate)
, AVG(t.sharerate)
FROM statistics_users t
WHERE FROM_UNIXTIME(t.timestamp) >= NOW() + INTERVAL -1 DAY
GROUP
BY DATE_FORMAT(FROM_UNIXTIME(t.timestamp),'%Y-%m-%d %H')
-- ^^^^^^^^^^^^ ^^^^^^^^^^^^^^^
https://dev.mysql.com/doc/refman/8.0/en/date-and-time-functions.html#function_date-format

How can I return new IDs in accordance with a date?

I have a table with the following data (merely an example, actual table has 600,000 rows) (aid = access id [primary key] and id = user id [foreign key]):
aid | id | date
332 | 1 | 2016-12-15
331 | 4 | 2016-12-15
330 | 3 | 2016-12-15
329 | 1 | 2016-12-14
328 | 1 | 2016-12-14
327 | 2 | 2016-12-14
326 | 3 | 2016-12-13
325 | 2 | 2016-12-13
324 | 1 | 2016-12-13
323 | 1 | 2016-12-12
322 | 3 | 2016-12-12
321 | 1 | 2016-12-12
Each id is a users primary key, and every time they access something in my system I log them in this table (with the date in the format as shown, and their id). A user can be logged multiple times a day.
I'm looking to: return the total number of times the thing has been accessed in a day and return the total number of NEW users who have accessed the thing in a day, for the last 8 days (something will always be logged each day, so using "LIMIT 8" is fine for getting only the last 8 days).
My SQL currently looks like:
SELECT COUNT(id), COUNT(distinct id), date
FROM table
GROUP BY date
ORDER BY date DESC
LIMIT 8;
That SQL does the first part correctly, but I can't figure out how to get it to return the number of users who have never accessed the thing until that day.
Desired results would be, the one "newuser" represents the user with id "4" as they have never accessed the thing before:
COUNT(id) | newusers | date
3 | 1 | 2016-12-15
3 | 0 | 2016-12-14
3 | 0 | 2016-12-13
3 | 0 | 2016-12-12
Sorry if I didn't explain this clear enough.
To get new users you want the first day an id appeared:
select id, min(date)
from t
group by id;
The rest is just a join and group by:
select d.date, cnt, count(dd.id) as newusers
from (select date, count(*) as cnt
from t
group by date
) d left join
(select id, min(date) as mindate
from t
group by id
) dd
on d.date = dd.mindate
group by d.date, d.cnt
limit 8;
To get the number of new users you need to compare them to a set of ids over the past 8 days
My MySQL is a bit rusty, so you might have to correct the syntax.
SELECT COUNT(id)
FROM table
WHERE id NOT IN (
SELECT DISTINCT id
FROM table
WHERE date BETWEEN DATE(DATE_SUB(NOW(), INTERVAL 8 DAY)) AND DATE(DATE_SUB(NOW(), INTERVAL 1 DAY))
)
I'll leave it as a task for you to combine it with your other query ;)
Hi if your date column in database is datetime/date or other date representing format you can do something like this:
for getting all users who accessed something in 8 days:
Select id, date from table
where date BETWEEN DATE_ADD(NOW(), INTERVAL -9 DAY) AND NOW()
I think, you can do whatever grouping you want on that.
To get new users, you can either go with self join or with sub select
selfjoin:
select t.id, t.date from table as t
LEFT join table as t2
ON t.id = t2.id
AND t.date BETWEEN DATE_ADD(NOW(), INTERVAL -1 DAY) AND NOW()
AND t2.date NOT BETWEEN DATE_ADD(NOW(), INTERVAL -9 DAY) AND NOW()
WHERE t2.id IS NULL
i used left join to match all access from users and then in where excluded those rows. However self joins are slow, and even slower with LEFT join
subselect:
select id, date from table
where date BETWEEN DATE_ADD(NOW(), INTERVAL -1 DAY) AND NOW()
AND id NOT IN (
SELECT id FROM table
WHERE date BETWEEN DATE_ADD(NOW(), INTERVAL -2 DAY) AND DATE_ADD(NOW(), INTERVAL -1 DAY)
)
I know those betweens with date_adds are not exactly nice looking, but i hope it will help you more than grouping dates
I would suggest using date with time for more information, but its entirely up to meaning of yours data

Average time difference between rows in database

Using MySQL, I have a table that keep track of user visit:
USER_ID | TIMESTAMP
--------+----------------------
1 | 2014-08-11 14:37:36
2 | 2014-08-11 12:37:36
3 | 2014-08-07 16:37:36
1 | 2014-07-14 15:34:36
1 | 2014-07-09 14:37:36
2 | 2014-07-03 14:37:36
3 | 2014-05-23 15:37:36
3 | 2014-05-13 12:37:36
Time is not important, more concern about answer to "how many days between entries"
How do I go about figuring how the average number of days between entries through SQL queries?
For example, the output should look like something like:
(output is just a sample, not reflection of the data table above)
USER_ID | AVG TIME (days)
--------+----------------------
1 | 2
2 | 3
3 | 1
MySQL has no direct "get something from a previous row" capabilities. Easiest workaround is to use a variable to store that "previous" value:
SET last = null;
SELECT user_id, AVG(diff)
FROM (
SELECT user_id, IF(last IS NULL, 0, timestamp - last) AS diff, #last := timestamp
FROM yourtable
ORDER BY user_id, timestamp ASC
) AS foo
GROUP BY user_id
The inner query does your "difference from previous row" calculations, and the outer query does the averaging.

Counting appointments for each day using MYSQL

I'm in trouble with a mysql statement counting appointments for one day within a given time period. I've got a calendar table including starting and finishing column (type = DateTime). The following statement should count all appointments for November including overall appointments:
SELECT
COUNT('APPOINTMENTS') AS Count,
DATE(c.StartingDate) AS Datum
FROM t_calendar c
WHERE
c.GUID = 'blalblabla' AND
((DATE(c.StartingDate) <= DATE('2012-11-01 00:00:00')) AND (DATE(c.EndingDate) >= DATE('2012-11-30 23:59:59'))) OR
((DATE(c.StartingDate) >= DATE('2012-11-01 00:00:00')) AND (DATE(c.EndingDate) <= DATE('2012-11-30 23:59:59')))
GROUP BY DATE(c.StartingDate)
HAVING Count > 1
But how to include appointments that starts before a StartingDate and ends on the StartingDate?
e.g.
StartingDate = 2012-11-14 17:00:00, EndingDate = 2012-11-15 08:00:00
StartingDate = 2012-11-15 09:00:00, EndingDate = 2012-11-15 10:00:00
StartingDate = 2012-11-15 11:00:00, EndingDate = 2012-11-15 12:00:00
My statement returns a count of 2 for 15th of November. But that's wrong because the first appointment is missing. How to include these appointments? What I am missing, UNION SELECT, JOIN, sub selection?
A possible solution?
SELECT
c1.GUID, COUNT('APPOINTMENTS') + COUNT(DISTINCT c2.ANYFIELD) AS Count,
DATE(c1.StartingDate) AS Datum,
COUNT(DISTINCT c2.ANYFIELD)
FROM
t_calendar c1
LEFT JOIN
t_calendar c2
ON
c2.ResourceGUID = c1.ResourceGUID AND
(DATE(c2.EndingDate) = DATE(c1.StartingDate)) AND
(DATE(c2.StartingDate) < DATE(c1.StartingDate))
WHERE
((DATE(c1.StartingDate) <= DATE('2012-11-01 00:00:00')) AND (DATE(c1.EndingDate) >= DATE('2012-11-30 23:59:59'))) OR
((DATE(c1.StartingDate) >= DATE('2012-11-01 00:00:00')) AND (DATE(c1.EndingDate) <= DATE('2012-11-30 23:59:59')))
GROUP BY
c1.ResourceGUID,
DATE(c1.StartingDate)
First: Consolidate range checking
First of all your two range where conditions can be replaced by a single one. And it also seems that you're only counting appointments that either completely overlap target date range or are completely contained within. Partially overlapping ones aren't included. Hence your question about appointments that end right on the range starting date.
To make where clause easily understandable I'll simplify it by using:
two variables to define target range:
rangeStart (in your case 1st Nov 2012)
rangeEnd (I'll rather assume to 1st Dec 2012 00:00:00.00000)
won't be converting datetime to dates only (using date function) the way that you did, but you can easily do that.
With these in mind your where clause can be greatly simplified and covers all appointments for given range:
...
where (c.StartingDate < rangeEnd) and (c.EndingDate >= rangeStart)
...
This will search for all appointments that fall in target range and will cover all these appointment cases:
start end
target range |==============|
partial front |---------|
partial back |---------|
total overlap |---------------------|
total containment |-----|
Partial front/back may also barely touch your target range (what you've been after).
Second: Resolving the problem
Why you're missing the first record? Simply because of your having clause that only collects those groups that have more than 1 appointment starting on a given day: 15th Nov has two, but 14th has only one and is therefore excluded because Count = 1 and is not > 1.
To answer your second question what am I missing is: you're not missing anything, actually you have too much in your statement and needs to simplified.
Try this statement instead that should return exactly what you're after:
select count(c.GUID) as Count,
date(c.StartingDate) as Datum
from t_calendar c
where (c.GUID = 'blabla') and
(c.StartingDate < str_to_date('2012-12-01', '%Y-%m-%d') and
(c.EndingDate >= str_to_date('2012-11-01', '%Y-%m-%d'))
group by date(c.StartingDate)
I used str_to_date function to make string to date conversion more safe.
I'm not really sure why you included having in your statement, because it's not really needed. Unless your actual statement is more complex and you only included part that's most relevant. In that case you'll likely have to change it to:
having Count > 0
Getting appointment count per day in any given date range
There are likely other ways as well but the most common way would be using a numbers or ?calendar* table that gives you the ability to break a range into individual points - days. They you have to join your appointments to this numbers table and provide results.
I've created a SQLFiddle that does the trick. Here's what it does...
Suppose you have numbers table Num with numbers from 0 to x. And appointments table Cal with your records. Following script created these two tables and populates some data. Numbers are only up to 100 which is enough for 3 months worth of data.
-- appointments
create table Cal (
Id int not null auto_increment primary key,
StartDate datetime not null,
EndDate datetime not null
);
-- create appointments
insert Cal (StartDate, EndDate)
values
('2012-10-15 08:00:00', '2012-10-20 16:00:00'),
('2012-10-25 08:00:00', '2012-11-01 03:00:00'),
('2012-11-01 12:00:00', '2012-11-01 15:00:00'),
('2012-11-15 10:00:00', '2012-11-16 10:00:00'),
('2012-11-20 08:00:00', '2012-11-30 08:00:00'),
('2012-11-30 22:00:00', '2012-12-05 00:00:00'),
('2012-12-01 05:00:00', '2012-12-10 12:00:00');
-- numbers table
create table Nums (
Id int not null primary key
);
-- add 100 numbers
insert into Nums
select a.a + (10 * b.a)
from (select 0 as a union all
select 1 union all
select 2 union all
select 3 union all
select 4 union all
select 5 union all
select 6 union all
select 7 union all
select 8 union all
select 9) as a,
(select 0 as a union all
select 1 union all
select 2 union all
select 3 union all
select 4 union all
select 5 union all
select 6 union all
select 7 union all
select 8 union all
select 9) as b
Now what you have to do now is
Select a range of days which you do by selecting numbers from Num table and convert them to dates.
Then join your appointments to those dates so that those appointments that fall on particular day are joined to that particular day
Then just group all these appointments per each day and get results
Here's the code that does this:
-- just in case so comparisons don't trip over
set names 'latin1' collate latin1_general_ci;
-- start and end target date range
set #s := str_to_date('2012-11-01', '%Y-%m-%d');
set #e := str_to_date('2012-12-01', '%Y-%m-%d');
-- get appointment count per day within target range of days
select adddate(#s, n.Id) as Day, count(c.Id) as Appointments
from Nums n
left join Cal c
on ((date(c.StartDate) <= adddate(#s, n.Id)) and (date(c.EndDate) >= adddate(#s, n.Id)))
where adddate(#s, n.Id) < #e
group by Day;
And this is the result of this rather simple select statement:
| DAY | APPOINTMENTS |
-----------------------------
| 2012-11-01 | 2 |
| 2012-11-02 | 0 |
| 2012-11-03 | 0 |
| 2012-11-04 | 0 |
| 2012-11-05 | 0 |
| 2012-11-06 | 0 |
| 2012-11-07 | 0 |
| 2012-11-08 | 0 |
| 2012-11-09 | 0 |
| 2012-11-10 | 0 |
| 2012-11-11 | 0 |
| 2012-11-12 | 0 |
| 2012-11-13 | 0 |
| 2012-11-14 | 0 |
| 2012-11-15 | 1 |
| 2012-11-16 | 1 |
| 2012-11-17 | 0 |
| 2012-11-18 | 0 |
| 2012-11-19 | 0 |
| 2012-11-20 | 1 |
| 2012-11-21 | 1 |
| 2012-11-22 | 1 |
| 2012-11-23 | 1 |
| 2012-11-24 | 1 |
| 2012-11-25 | 1 |
| 2012-11-26 | 1 |
| 2012-11-27 | 1 |
| 2012-11-28 | 1 |
| 2012-11-29 | 1 |
| 2012-11-30 | 2 |

MySQL: group by consecutive days and count groups

I have a database table which holds each user's checkins in cities. I need to know how many days a user has been in a city, and then, how many visits a user has made to a city (a visit consists of consecutive days spent in a city).
So, consider I have the following table (simplified, containing only the DATETIMEs - same user and city):
datetime
-------------------
2011-06-30 12:11:46
2011-07-01 13:16:34
2011-07-01 15:22:45
2011-07-01 22:35:00
2011-07-02 13:45:12
2011-08-01 00:11:45
2011-08-05 17:14:34
2011-08-05 18:11:46
2011-08-06 20:22:12
The number of days this user has been to this city would be 6 (30.06, 01.07, 02.07, 01.08, 05.08, 06.08).
I thought of doing this using SELECT COUNT(id) FROM table GROUP BY DATE(datetime)
Then, for the number of visits this user has made to this city, the query should return 3 (30.06-02.07, 01.08, 05.08-06.08).
The problem is that I have no idea how shall I build this query.
Any help would be highly appreciated!
You can find the first day of each visit by finding checkins where there was no checkin the day before.
select count(distinct date(start_of_visit.datetime))
from checkin start_of_visit
left join checkin previous_day
on start_of_visit.user = previous_day.user
and start_of_visit.city = previous_day.city
and date(start_of_visit.datetime) - interval 1 day = date(previous_day.datetime)
where previous_day.id is null
There are several important parts to this query.
First, each checkin is joined to any checkin from the previous day. But since it's an outer join, if there was no checkin the previous day the right side of the join will have NULL results. The WHERE filtering happens after the join, so it keeps only those checkins from the left side where there are none from the right side. LEFT OUTER JOIN/WHERE IS NULL is really handy for finding where things aren't.
Then it counts distinct checkin dates to make sure it doesn't double-count if the user checked in multiple times on the first day of the visit. (I actually added that part on edit, when I spotted the possible error.)
Edit: I just re-read your proposed query for the first question. Your query would get you the number of checkins on a given date, instead of a count of dates. I think you want something like this instead:
select count(distinct date(datetime))
from checkin
where user='some user' and city='some city'
Try to apply this code to your task -
CREATE TABLE visits(
user_id INT(11) NOT NULL,
dt DATETIME DEFAULT NULL
);
INSERT INTO visits VALUES
(1, '2011-06-30 12:11:46'),
(1, '2011-07-01 13:16:34'),
(1, '2011-07-01 15:22:45'),
(1, '2011-07-01 22:35:00'),
(1, '2011-07-02 13:45:12'),
(1, '2011-08-01 00:11:45'),
(1, '2011-08-05 17:14:34'),
(1, '2011-08-05 18:11:46'),
(1, '2011-08-06 20:22:12'),
(2, '2011-08-30 16:13:34'),
(2, '2011-08-31 16:13:41');
SET #i = 0;
SET #last_dt = NULL;
SET #last_user = NULL;
SELECT v.user_id,
COUNT(DISTINCT(DATE(dt))) number_of_days,
MAX(days) number_of_visits
FROM
(SELECT user_id, dt
#i := IF(#last_user IS NULL OR #last_user <> user_id, 1, IF(#last_dt IS NULL OR (DATE(dt) - INTERVAL 1 DAY) > DATE(#last_dt), #i + 1, #i)) AS days,
#last_dt := DATE(dt),
#last_user := user_id
FROM
visits
ORDER BY
user_id, dt
) v
GROUP BY
v.user_id;
----------------
Output:
+---------+----------------+------------------+
| user_id | number_of_days | number_of_visits |
+---------+----------------+------------------+
| 1 | 6 | 3 |
| 2 | 2 | 1 |
+---------+----------------+------------------+
Explanation:
To understand how it works let's check the subquery, here it is.
SET #i = 0;
SET #last_dt = NULL;
SET #last_user = NULL;
SELECT user_id, dt,
#i := IF(#last_user IS NULL OR #last_user <> user_id, 1, IF(#last_dt IS NULL OR (DATE(dt) - INTERVAL 1 DAY) > DATE(#last_dt), #i + 1, #i)) AS
days,
#last_dt := DATE(dt) lt,
#last_user := user_id lu
FROM
visits
ORDER BY
user_id, dt;
As you see the query returns all rows and performs ranking for the number of visits. This is known ranking method based on variables, note that rows are ordered by user and date fields. This query calculates user visits, and outputs next data set where days column provides rank for the number of visits -
+---------+---------------------+------+------------+----+
| user_id | dt | days | lt | lu |
+---------+---------------------+------+------------+----+
| 1 | 2011-06-30 12:11:46 | 1 | 2011-06-30 | 1 |
| 1 | 2011-07-01 13:16:34 | 1 | 2011-07-01 | 1 |
| 1 | 2011-07-01 15:22:45 | 1 | 2011-07-01 | 1 |
| 1 | 2011-07-01 22:35:00 | 1 | 2011-07-01 | 1 |
| 1 | 2011-07-02 13:45:12 | 1 | 2011-07-02 | 1 |
| 1 | 2011-08-01 00:11:45 | 2 | 2011-08-01 | 1 |
| 1 | 2011-08-05 17:14:34 | 3 | 2011-08-05 | 1 |
| 1 | 2011-08-05 18:11:46 | 3 | 2011-08-05 | 1 |
| 1 | 2011-08-06 20:22:12 | 3 | 2011-08-06 | 1 |
| 2 | 2011-08-30 16:13:34 | 1 | 2011-08-30 | 2 |
| 2 | 2011-08-31 16:13:41 | 1 | 2011-08-31 | 2 |
+---------+---------------------+------+------------+----+
Then we group this data set by user and use aggregate functions:
'COUNT(DISTINCT(DATE(dt)))' - counts the number of days
'MAX(days)' - the number of visits, it is a maximum value for the days field from our subquery.
That is all;)
As data sample provided by Devart, the inner "PreQuery" works with sql variables. By defaulting the #LUser to a -1 (probable non-existent user ID), the IF() test checks for any difference between last user and current. As soon as a new user, it gets a value of 1... Additionally, if the last date is more than 1 day from the new date of check-in, it gets a value of 1. Then, the subsequent columns reset the #LUser and #LDate to the value of the incoming record just tested against for the next cycle. Then, the outer query just sums them up and counts them for the final correct results per the Devart data set of
User ID Distinct Visits Total Days
1 3 9
2 1 2
select PreQuery.User_ID,
sum( PreQuery.NextVisit ) as DistinctVisits,
count(*) as TotalDays
from
( select v.user_id,
if( #LUser <> v.User_ID OR #LDate < ( date( v.dt ) - Interval 1 day ), 1, 0 ) as NextVisit,
#LUser := v.user_id,
#LDate := date( v.dt )
from
Visits v,
( select #LUser := -1, #LDate := date(now()) ) AtVars
order by
v.user_id,
v.dt ) PreQuery
group by
PreQuery.User_ID
for a first sub-task:
select count(*)
from (
select TO_DAYS(p.d)
from p
group by TO_DAYS(p.d)
) t
I think you should consider changing database structure. You could add table visits and visit_id into your checkins table. Each time you want to register new checkin you check if there is any checkin a day back. If yes then you add a new checkin with visit_id from yesterday's checkin. If not then you add new visit to visits and new checkin with new visit_id.
Then you could get you data in one query with something like that:
SELECT COUNT(id) AS number_of_days, COUNT(DISTINCT visit_id) number_of_visits FROM checkin GROUP BY user, city
It's not very optimal but still better than doing anything with current structure and it will work. Also if results can be separate queries it will work very fast.
But of course drawbacks are you will need to change database structure, do some more scripting and convert current data to new structure (i.e. you will need to add visit_id to current data).