Aggregate SQL functions with irregular time stamps - mysql

Question

I have a table which includes a time stamp and river flow. Some days I have multiple records, yet other days I have no records.
How can I calculate the average flow and total flow between two dates?
Assuming a linear value between two points is acceptable. Maybe some sort of weighted average. If there is some least squares regression algorithm or something similar which provides more accurate results, that would also be great.
EDIT. For a given day I have the following fictional data for illustration purposes. If possible, I would like to do better than assuming an average of 146 as the flows are high for longer duration's, and the real average is likely over 200.
10/12/15 12:00 AM 100
10/12/15 12:01 AM 102
10/12/15 12:02 AM 104
10/12/15 12:03 AM 106
10/12/15 12:04 AM 200
10/12/15 10:00 PM 204
10/12/15 11:00 PM 208
Average 146

Answer 1

You need to do a weighted average. For this, you need the next time stamp:
select rf.*,
(select rf2.timestamp
from riverflow rf2
where rf2.timestamp > rf.timestamp
order by rf.timestamp asc
limit 1
) as nextTimestamp
from riverflow rf;
Next, comes the weighted average. I have no idea how you want to handle the issue that the period of measurement may not align with given days. Instead, we'll just take values and report on the beginning and ending observation times:
select min(timestamp) as start, max(timestamp) as end,
(sum(riverflow * timestampdiff(second, timestamp, nexttimestamp) / (24*60*60)) /
(timestampdiff(second, min(timestamp), max(timestamp)) / (24*60*60)
) as avgRiverflow
from (select rf.*,
(select rf2.timestamp
from riverflow rf2
where rf2.timestamp > rf.timestamp
order by rf2.timestamp asc
limit 1
) as nextTimestamp
from riverflow rf
where timestamp >= $date1 and timestamp < $date2
) t;

Answer 2

Something like this should be generally the right direction:
SELECT AVG(dayflowRate) AS avgFlowRate, SUM(dayFlow) AS totalFlow
FROM (
SELECT DATE(theTS) AS theDate, AVG(flowRate) AS dayFlowRate
, AVG(flowRate) * (24*60*60) AS dayFlow
FROM theTable
WHERE theTS BETWEEN [beginTS] AND [endTS]
GROUP BY theDate
) AS dayQ
;
However, it assumes full days with that 24 * 60 * 60 multiplier (expanded only for clarity by the way). If you need greater precision, you will want to look into MIN and MAX aggregations, and TIME_TO_SEC function.
I think this (below) might be a little more accurate:
SELECT AVG(dayflowRate) AS avgFlowRate, SUM(dayFlow) AS totalFlow
FROM (
SELECT DATE(theTS) AS theDate, AVG(flowRate) AS dayFlowRate
, AVG(flowRate)
* ( TIME_TO_SEC(LEAST(MAX(theTS), [endTS])
- TIME_TO_SEC(GREATEST(MIN(theTS), [beginTS]))
)
AS dayFlow
FROM theTable
WHERE theTS BETWEEN [beginTS] AND [endTS]
GROUP BY theDate
) AS dayQ
;
Edit: or might not, if that day's measurements are at 11am and 1pm, it's dayFlow will only actually be for two hours of flow, even if it is in the middle of multiple days.
This should be best:
SELECT AVG(dayflowRate) AS avgFlowRate, SUM(dayFlow) AS totalFlow
FROM (
SELECT DATE(theTS) AS theDate, AVG(flowRate) AS dayFlowRate
, AVG(flowRate)
* ( IF(DATE(theTS)=DATE([endTS]), TIME_TO_SEC([endTS]), (24*60*60))
- IF(DATE(theTS)=DATE([beginTS]), TIME_TO_SEC([beginTS]), 0)
)
AS dayFlow
FROM theTable
WHERE theTS BETWEEN [beginTS] AND [endTS]
GROUP BY theDate
) AS dayQ
;