Get the hourly average amount based on rows with minutely timestamp - mysql

I have a table structure that looks like this:
+---------+------------+----------+---------+-----------+------------+
| id | account_id | hashrate | workers | sharerate | timestamp |
+---------+------------+----------+---------+-----------+------------+
| 1227368 | 42 | 405211 | 1 | 6183 | 1534264380 |
| 1227367 | 12 | 450077 | 1 | 6868 | 1534264380 |
+---------+------------+----------+---------+-----------+------------+
A row is created every minute for a user with an active worker.
I'm attempting to display a chart that will show the average hashrate and sharerate for each hour over a 24 hour period.
My sql query so far is:
SELECT
timestamp,
SUM(hashrate) as hashes,
SUM(sharerate) AS shares
FROM statistics_users
WHERE FROM_UNIXTIME(timestamp) >= DATE_SUB(NOW(), INTERVAL 1 DAY)
GROUP BY timestamp;
which returns:
+------------+------------+----------+
| timestamp | hashes | shares |
+------------+------------+----------+
| 1534177980 | 2282744244 | 34831913 |
Which is returning 1440 objects (one for each minute of day). I want to group the items by hour and average all sums returned from my query of minutes into an hour. Not sure where to go from here and I'm fairly new to writing sql queries, so any help would be much appreciated.

We can use an expression that returns a representation of hour, basically trimming off minutes and seconds, and then GROUP BY that expression. One possibility it to use the MySQL DATE_FORMAT function.
SELECT DATE_FORMAT(FROM_UNIXTIME(t.timestamp),'%Y-%m-%d %H') AS ts_hr
, AVG(t.hashes)
, AVG(t.shares)
FROM (
-- original query goes here as inline view
SELECT timestamp
, SUM(hashrate) as hashes
, SUM(sharerate) AS shares
FROM statistics_users
WHERE FROM_UNIXTIME(timestamp) >= NOW() + INTERVAL -1 DAY
GROUP BY timestamp
) t
GROUP
BY DATE_FORMAT(FROM_UNIXTIME(t.timestamp),'%Y-%m-%d %H')
-- ^^^^^^^^^^^^ ^^^^^^^^^^^^^^^
Note that the condition in the WHERE clause doesn't respect hour boundaries, so we're going to get a partial result for first hour, and a partial last hour.
The specification isn't clear, as to whether we want the average of the sums for each timestamp, or more simply
SELECT DATE_FORMAT(FROM_UNIXTIME(t.timestamp),'%Y-%m-%d %H') AS ts_hr
, AVG(t.hashrate)
, AVG(t.sharerate)
FROM statistics_users t
WHERE FROM_UNIXTIME(t.timestamp) >= NOW() + INTERVAL -1 DAY
GROUP
BY DATE_FORMAT(FROM_UNIXTIME(t.timestamp),'%Y-%m-%d %H')
-- ^^^^^^^^^^^^ ^^^^^^^^^^^^^^^
https://dev.mysql.com/doc/refman/8.0/en/date-and-time-functions.html#function_date-format

Related

Mysql Subtract 'N' days to date value and compare to current date. N and Date are two column values

My table contains remind_me_befor_days column and expiry_date column.
I have to select records with condition (expiry_date - remind_me_befor_days) = current_date.
How this possible by query. Anyone can help me ?
Thanks in advance
You can do date arithemtics:
select t.*
from mytable t
where t.expiry_date - interval t.remind_me_befor_days day = current_date
Demo on DB Fiddle:
-- sample data
select * from mytable;
id | expiry_date | remind_me_befor_days
-: | :---------- | -------------------:
1 | 2019-12-25 | 10
2 | 2019-12-25 | 5
-- query
select t.*, current_date
from mytable t
where t.expiry_date - interval t.remind_me_befor_days day = current_date
id | expiry_date | remind_me_befor_days | current_date
-: | :---------- | -------------------: | :-----------
1 | 2019-12-25 | 10 | 2019-12-15
To split this into a few subproblems, we have:
Find the current date.
Subtract your remind_me_befor_days column value from the expiry_date column.
Use that as a filter for the records you want from the table.
The first can be done using the MySQL CURDATE() function. Read more here. Combining that with the second and third steps, you get something that looks like
SELECT *
FROM my_table
WHERE expiry_date - remind_me_befor_days = CURDATE()
I would strongly recommend that you write the condition as:
where expiry_date = curdate() + interval remind_me_before_days day
This is index friendly, so it can make use of an index on expiry_date.

SQL count row equal to column

Here is my table:
+--------+---------------------+
| roomNo | date |
+--------+---------------------+
| 1 | 2017-05-17 16:05:00 |
| 1 | 2017-05-17 15:05:00 |
| 2 | 2019-05-20 12:30:00 |
| 2 | 2019-05-15 10:30:00 |
| 2 | 2019-05-14 08:00:00 |
+--------+---------------------+
I want to get the day where the room is used at least once and which day(s) had the most operations in it and how many times, in the current year. I don't know how to compare the dates.
The expected result would be something like :
+--------+------------+------------+
| roomNo | date | operations |
+--------+------------+------------+
| 2 | 2019-05-20 | 3 |
+--------+------------+------------+
We can use MySQL DATE function to lop off times from DATETIME and TIMESTAMP columns. Or we could use MySQL DATE_FORMAT function, to return just year, month day.
We can use an aggregate function like COUNT or SUM in a query with GROUP BY to get counts by room and day.
If "current year" means from Jan 1 thru Dec 31, we can use expression to derive date values of '2019-01-01' and '2020-01-01', and do a comparison of the date column to those values in the WHERE clause.
As a start, consider this:
SELECT t.roomno
, DATE(t.date) AS date_
, COUNT(*) AS cnt_
FROM mytable t
WHERE t.date >= DATE_FORMAT(NOW(),'%Y-01-01') + INTERVAL 0 YEAR
AND t.date < DATE_FORMAT(NOW(),'%Y-01-01') + INTERVAL 1 YEAR
GROUP
BY t.roomno
, DATE(t.date)
ORDER
BY t.roomno
, cnt_ DESC
If the goal is to just return one of the rooms that has the highest number of uses, we could use a LIMIT clause, and order by the highest count to lowest,
ORDER
BY cnt_ DESC
, t.roomno
LIMIT 1
If the results are more complex than that, we can omit the LIMIT clause, and use the result from that query as an inline view in an outer query.
With MySQL 8.0, we can use common table expression (CTE) and window/analytic functions, to get more elaborate results.

How can I return new IDs in accordance with a date?

I have a table with the following data (merely an example, actual table has 600,000 rows) (aid = access id [primary key] and id = user id [foreign key]):
aid | id | date
332 | 1 | 2016-12-15
331 | 4 | 2016-12-15
330 | 3 | 2016-12-15
329 | 1 | 2016-12-14
328 | 1 | 2016-12-14
327 | 2 | 2016-12-14
326 | 3 | 2016-12-13
325 | 2 | 2016-12-13
324 | 1 | 2016-12-13
323 | 1 | 2016-12-12
322 | 3 | 2016-12-12
321 | 1 | 2016-12-12
Each id is a users primary key, and every time they access something in my system I log them in this table (with the date in the format as shown, and their id). A user can be logged multiple times a day.
I'm looking to: return the total number of times the thing has been accessed in a day and return the total number of NEW users who have accessed the thing in a day, for the last 8 days (something will always be logged each day, so using "LIMIT 8" is fine for getting only the last 8 days).
My SQL currently looks like:
SELECT COUNT(id), COUNT(distinct id), date
FROM table
GROUP BY date
ORDER BY date DESC
LIMIT 8;
That SQL does the first part correctly, but I can't figure out how to get it to return the number of users who have never accessed the thing until that day.
Desired results would be, the one "newuser" represents the user with id "4" as they have never accessed the thing before:
COUNT(id) | newusers | date
3 | 1 | 2016-12-15
3 | 0 | 2016-12-14
3 | 0 | 2016-12-13
3 | 0 | 2016-12-12
Sorry if I didn't explain this clear enough.
To get new users you want the first day an id appeared:
select id, min(date)
from t
group by id;
The rest is just a join and group by:
select d.date, cnt, count(dd.id) as newusers
from (select date, count(*) as cnt
from t
group by date
) d left join
(select id, min(date) as mindate
from t
group by id
) dd
on d.date = dd.mindate
group by d.date, d.cnt
limit 8;
To get the number of new users you need to compare them to a set of ids over the past 8 days
My MySQL is a bit rusty, so you might have to correct the syntax.
SELECT COUNT(id)
FROM table
WHERE id NOT IN (
SELECT DISTINCT id
FROM table
WHERE date BETWEEN DATE(DATE_SUB(NOW(), INTERVAL 8 DAY)) AND DATE(DATE_SUB(NOW(), INTERVAL 1 DAY))
)
I'll leave it as a task for you to combine it with your other query ;)
Hi if your date column in database is datetime/date or other date representing format you can do something like this:
for getting all users who accessed something in 8 days:
Select id, date from table
where date BETWEEN DATE_ADD(NOW(), INTERVAL -9 DAY) AND NOW()
I think, you can do whatever grouping you want on that.
To get new users, you can either go with self join or with sub select
selfjoin:
select t.id, t.date from table as t
LEFT join table as t2
ON t.id = t2.id
AND t.date BETWEEN DATE_ADD(NOW(), INTERVAL -1 DAY) AND NOW()
AND t2.date NOT BETWEEN DATE_ADD(NOW(), INTERVAL -9 DAY) AND NOW()
WHERE t2.id IS NULL
i used left join to match all access from users and then in where excluded those rows. However self joins are slow, and even slower with LEFT join
subselect:
select id, date from table
where date BETWEEN DATE_ADD(NOW(), INTERVAL -1 DAY) AND NOW()
AND id NOT IN (
SELECT id FROM table
WHERE date BETWEEN DATE_ADD(NOW(), INTERVAL -2 DAY) AND DATE_ADD(NOW(), INTERVAL -1 DAY)
)
I know those betweens with date_adds are not exactly nice looking, but i hope it will help you more than grouping dates
I would suggest using date with time for more information, but its entirely up to meaning of yours data

Selecting the number of last consecutive days from timestamp (excepting today)

I have a table A_DailyLogins with the columns ID (auto increment), Key (userid) and Date (timestamp). I want a query which would return the number of last consecutive days from those timestamp based on the Key, for example if he has a row for yesterday, one for two days ago and another one for three days ago, but the last one isn't from four days ago, it would return 3, because this is the number of last days the user was logged in.
My attempt was to create a query selecting the last 7 rows of the players ordered by Date DESC (this is what I wanted in the first place, but then I thought that it would be great to have all the last consecutive days), and then I retrieved the query result and compared the dates (converted to year/month/day with functions from that language [Pawn]) and increased the number of consecutive days when a date is before the other one with one day. (but this is extremely slow compared to what I think that can be done directly only with MySQL)
The closest thing I found is this: Check for x consecutive days - given timestamps in database . But it still isn't how I want it to be, it's still pretty different. I tried to modify it, but it is way too hard for me, I don't have that much experience in MySQL.
context
let consecutive login period be a period where the user is logged in on all days ( has an entry in A_DailyLogins on every day in period ) where there is no entry in A_DailyLogins immediately before or after the consecutive login period with the same user
and number of consecutive days be the difference between the maximum and minumum dates in a consecutive login period
the maximum date of a consecutive login period has no login entry immediately after ( sequentially ) to it..
the minimum date of a consecutive login period has no login entry immediately previous ( sequentially ) to it..
plan
left join A_DailyLogins to itself using same user and sequential dates where right is null to find maximums
analogous logic to find minimums
calculate row ordering over minimums and maximums with appropriate order by
join maximums and minimums on row number
filter where maximum login is yesterday/today
calculate date_diff between maximum and minimum in range
left join users to above resultset and coalesce over the case where user does not have a consecutive login period ending yesterday/today
input
+----+------+------------+
| ID | Key | Date |
+----+------+------------+
| 25 | eric | 2015-12-23 |
| 26 | eric | 2015-12-25 |
| 27 | eric | 2015-12-26 |
| 28 | eric | 2015-12-27 |
| 29 | eric | 2016-01-01 |
| 30 | eric | 2016-01-02 |
| 31 | eric | 2016-01-03 |
| 32 | nusa | 2015-12-27 |
| 33 | nusa | 2015-12-29 |
+----+------+------------+
query
select all_users.`Key`,
coalesce(nconsecutive, 0) as nconsecutive
from
(
select distinct `Key`
from A_DailyLogins
) all_users
left join
(
select
lower_login_bounds.`Key`,
lower_login_bounds.`Date` as from_login,
upper_login_bounds.`Date` as to_login,
1 + datediff(least(upper_login_bounds.`Date`, date_sub(current_date, interval 1 day))
, lower_login_bounds.`Date`) as nconsecutive
from
(
select curr_login.`Key`, curr_login.`Date`, #rn1 := #rn1 + 1 as row_number
from A_DailyLogins curr_login
left join A_DailyLogins prev_login
on curr_login.`Key` = prev_login.`Key`
and prev_login.`Date` = date_add(curr_login.`Date`, interval -1 day)
cross join ( select #rn1 := 0 ) params
where prev_login.`Date` is null
order by curr_login.`Key`, curr_login.`Date`
) lower_login_bounds
inner join
(
select curr_login.`Key`, curr_login.`Date`, #rn2 := #rn2 + 1 as row_number
from A_DailyLogins curr_login
left join A_DailyLogins next_login
on curr_login.`Key` = next_login.`Key`
and next_login.`Date` = date_add(curr_login.`Date`, interval 1 day)
cross join ( select #rn2 := 0 ) params
where next_login.`Date` is null
order by curr_login.`Key`, curr_login.`Date`
) upper_login_bounds
on lower_login_bounds.row_number = upper_login_bounds.row_number
where upper_login_bounds.`Date` >= date_sub(current_date, interval 1 day)
and lower_login_bounds.`Date` < current_date
) last_consecutive
on all_users.`Key` = last_consecutive.`Key`
;
output
+------+------------------+
| Key | last_consecutive |
+------+------------------+
| eric | 2 |
| nusa | 0 |
+------+------------------+
valid as run on 2016-01-03
sqlfiddle

Change DATE_ADD's unit based on column's value

I have a table which stores the datetime an operation was last_run, I also have a frequency column, which is the frequency, in months, that the operation should run.
+----+---------------------+-----------+
| id | last_run | frequency |
+----+---------------------+-----------+
| 1 | 2014-05-22 00:00:00 | 12 |
|----|---------------------|-----------|
| 2 | 2015-05-15 00:00:00 | 0.25 |
+----+---------------------+-----------+
I'm using the following query to obtain the date that the operation should next be run:
SELECT DATE_ADD(last_run, INTERVAL frequency MONTH) AS next_run FROM table;
This fails to return the correct next_run date on the second record because of the frequency value.
Is it possible to add a condition so that when frequency = 0.25
frequency becomes 7
MONTH becomes DAY
This is part of an old system which I am currently refactoring. Would it just be better to store the interval type (DAY/MONTH/YEAR), and pass that to the query, or is there a way of adding the above conditions to the query?
I think you have to use a big case statement. Something like this:
select (case when frequency >= 1
then DATE_ADD(last_run, INTERVAL frequency MONTH)
when frequency < 1
then DATE_ADD(last_run, INTERVAL frequency*28 DAY)
ELSE DATE_ADD(last_run, INTERVAL 1 MONTH) --Adding for may be null values
end)