How to best get daily SUM for minute-level data? - mysql

Question

I have a data set consisting of minute-by-minute data. My goal is to return minute-by-minute records, and add calculations that create sums of a certain field for the past 24 hours, counting back from each minute record.
The query I have is the following:
SELECT main.recorded_at AS x, (SELECT SUM(precipitation) FROM data AS sub WHERE sub.host = main.host sub.recorded_at BETWEEN SUBTIME(main.recorded_at, '24:00:00') AND main.recorded_at) AS y FROM data AS main WHERE host = 'xxxx' ORDER BY x ASC;
Is there a more efficient way to write this query? I have tried, but failed, so far, using LEFT JOINS and different GROUP BYs.
When I explain this query, I get the following:
1 PRIMARY main ref host host 767 const 4038 100.00 Using where; Using filesort
2 DEPENDENT SUBQUERY sub ref host,recorded_at host 767 const 4038 100.00 Using where
In total, the query takes about 200 seconds to run with 8000 records, getting slower all the time. My goal is to get the aggregate 24-hour precipitation for each result, and somehow in under 2 seconds.
Maybe I'm going about this the wrong way? I'm open to suggestions for other avenues to get the same result. :)
Thanks!
~Mike

Answer 1

Assuming I'm understanding your question correctly, it looks like you can use SUM with CASE to achieve the same result without using the correlated subquery.
SELECT recorded_at AS x,
SUM(CASE WHEN recorded_at BETWEEN SUBTIME(recorded_at, '24:00:00') AND recorded_at
THEN precipitation END) As y
FROM data
WHERE host = 'xxxx'
GROUP BY recorded_at
ORDER BY x ASC;
While I'm not sure this would yield a better performance, I do think it would solve your issue using an OUTER JOIN with GROUP BY:
SELECT main.recorded_at AS x,
SUM(sub.precipitation) As y
FROM data main LEFT JOIN data sub ON
main.host = sub.host AND
sub.recorded_at BETWEEN SUBTIME(main.recorded_at, '24:00:00') AND main.recorded_at
WHERE main.host = 'xxxx'
GROUP BY main.recorded_at
ORDER BY x ASC;