mysql db for time-temperature values - mysql

Question

I need your help to build my db the right way.
I need to store time-temperature values for different rooms of my house
and I want to use DyGraph to graph the data sets.
I want to implement different time windows: 1 hour, 24 hours, 48 hours,
1 week, ....
I will be detecting the temperature with a 15 minutes interval, so I will have 4 time-temperature values per hour.
Each room has an ID so the time-temperature values will be associated
to the proper room.
The table I built i very simple:
----------------------------------
| ID | DATE | TEMP |
----------------------------------
| 1 |2014-04-30 00:00:00 | 18.6 |
----------------------------------
| 2 |2014-04-30 00:00:00 | 18.3 |
----------------------------------
| 3 |2014-04-30 00:00:00 | 18.3 |
----------------------------------
| 1 |2014-04-30 00:15:00 | 18.5 |
----------------------------------
For some strange reason, when the number of rows gets to 500 or so,the
server becomes very slow.
Also, I have a web page were I can read the different temperatures of
the rooms: this page polls the server through AJAX every 5 seconds (because it needs
to be frequently updated!), but when the number of rows of the table
gets around 500, it hangs.
I tried to split the table and I created a table for each room, then a
table for each time-window and now everything seems to be working fine.
Since I do not think this is the best/most efficient way to organize
this thing, I need your help to give it a better structure.
I use a php script to retrieve the temperature data for all the rooms of my house:
$query = "SELECT * FROM temperature t1
WHERE (id, date) IN
(SELECT id,MAX(date) FROM
temperature t2 GROUP BY id)";
this query allows me to collect the temperature values in an array called $options:
$result_set = mysql_query($query, $connection);
while($rows = mysql_fetch_array($result_set)){
$options [] = $rows;
}
then, I json-encode the array:
$j = json_encode($options);
and send it to the ajax script, which shows the data on the web page:
echo $j;
In the ajax script, I save the data in a variable and then parse it:
var return_data = xhr.responseText;
var temperature = JSON.parse(return_data);
next I loop through the array to extract the temperature values and put it in the right place on the web page:
for(var j=0; j<temperature.length; j++){
document.getElementById("TEMPArea" + j).innerHTML = temperature[j].temp + "°C";
}
This works fine as long as the rows in the 'temperature' table are less than 600 or so: polling every 5 seconds is not a problem.
Above 600, the page refresh gets slow and eventually it hangs and stops refreshing.
EDIT: Right now, I am working on a virtual machine with Windows 7 64bit, Apache, PHP and MySQL, 4GB RAM. Do you think this could be an issue?

Answer 1

It seems like I was poor in details, so here's something more to what I said.
I use a php script to retrieve the temperature data for all the rooms of my house:
$query = "SELECT * FROM temperature t1
WHERE (id, date) IN
(SELECT id,MAX(date) FROM
temperature t2 GROUP BY id)";
this query allows me to collect the temperature values in an array called $options:
$result_set = mysql_query($query, $connection);
while($rows = mysql_fetch_array($result_set)){
$options [] = $rows;
}
then, I json-encode the array:
$j = json_encode($options);
and send it to the ajax script, which shows the data on the web page:
echo $j;
In the ajax script, I save the data in a variable and then parse it:
var return_data = xhr.responseText;
var temperature = JSON.parse(return_data);
next I loop through the array to extract the temperature values and put it in the right place on the web page:
for(var j=0; j<temperature.length; j++){
document.getElementById("TEMPArea" + j).innerHTML = temperature[j].temp + "°C";
}
As I said in the first message, this works fine as long as the rows in the 'temperature' table are less than 600 or so: polling every 5 seconds is not a problem.
Above 600, the page refresh gets slow and eventually it hangs and stops refreshing.
I am not an expert, the code is pretty simple and straight forward, so I am having problems detecing the cause.
Thanks again.

Answer 2

I think the query is the main source of problems:
it's a slow way of getting the answer you want (you can always run it in Workbench and study the output of EXPLAIN - see the manual for more details
it implicitly supposes that all sensors with transmit at the same time, and as soon as that's not the case your output dataset won't be complete. Normally you'll want the latest data from each individual sensor
so I propose a somewhat different approach:
add an index on date and one on id to speed up queries. The lack of a PK is an issue, but let's first focus on solving the current issues...
obtain the list of available sensors - minimal solution
select distinct id from temperature;
but it would be better to store a list of available sensors in some other table - this query will also get slower as the number of records in temperature grows.
iterate over the results of that list to fetch the latest value for each of the sensors
select * from temperature
where id = (value obtained in previous step)
order by date desc
limit 1;
with this query you'll only get the most recent record associated with each sensor. Thanks to the indexes, the speed impact of a growing table should be minimal.
reassemble these results in a data structure to send to your client web page.
Also, as stated in the documentation, the mysql_* extension is deprecated and should not be used in new programs. Use mysqli_ or preferably PDO. Both these extensions will also allow you to use parameterized queries, the only real protection against SQL Injection issues. See here for a quick introduction on how to use them