My dataset is a table with 3 rows : ID of a hard-drive, percentage of empty space, timestamp. The table is appended with the new state of each HDD (1200 of them) every 20 minutes.
If I want to pick the last state of my HDD pool, I go for a MAX(timestamp), and a MIN(timestamp) if I want the oldest.
But say I have a given timestamp, how can I ask MySQL to retrieve data from more or less X seconds around this timestamp ?
WHERE yourTimeStamp
between TIMESTAMPADD(SECOND,-3,yourtimestamp)
and TIMESTAMPADD(SECOND, 3,yourtimestamp)
where -3 and + 3 was substituted for X
http://dev.mysql.com/doc/refman/5.5/en/date-and-time-functions.html#function_timestampadd for more help.
Like this:
WHERE timestamp BETWEEN DATE_SUB('<given_timestamp>', INTERVAL 5 SECOND)
AND DATE_ADD('<given_timestamp>', INTERVAL 5 SECOND);
As mentioned in the other answer, your query is slow when selection is based on the timestamp field.
You can add an INDEX on that column to speed it up:
ALTER TABLE <table_name> ADD INDEX(`timestamp`)
Note that, depending on the size of your table, the first time you add an index it takes a while. Secondly, it slows down INSERT queries and adds to the size of your database. This is different for everybody so you just have to find out by testing.
Related
SELECT *
FROM LOGS
WHERE datetime > DATE_SUB(NOW(), INTERVAL 1 MONTH)
I have a big table LOGS (InnoDB). When I try to get last month's data, the query waits too long.
I created an index for column datetime but it seems not helping. How to speed up this query?
Since the database records are inserted in oldest to newest, you could create 2 calls. The first call requesting the ID of the oldest record:
int oldestRecordID = SELECT TOP 1 MIN(id)
FROM LOGS
WHERE datetime > DATE_SUB(NOW(), INTERVAL 1 MONTH)
Then with that ID just request all records where ID > oldestRecordID:
SELECT *
FROM LOGS
WHERE ID > oldestRecordID
It's multiple calls, but it could be faster however I am sure you could combine those 2 calls too.
Probably the only thing you can do is create a clustered index on datetime. This will ensure that the values are co-located.
However, I don't think this will solve your real problem. Why are you bringing back all records from a month. This is a lot of data.
In all likelihood, you could summarize the data in the database and only bring back the information you need rather than all the data.
database table screenshot
I have a table as shown in pic with 10+ columns with number of poeple. I want the duration to be in 15 mins.i.e splitting 10:00-10:30 row to 10:00-10:15 and 10:15-10:30 and so on.
The column value should be divided by 2. Any pointers?
I would store the duration as a datetime for just the beginning of the duration. I am assuming your duration is a varchar or something now.
alter table your_table add duration_time datetime
update your_table set duration_time = cast(concat(date_field,' ',left(duration_field,locate('-',duration_field))) as datetime)
and then always assume 15 minutes with the code that reads the table. You can then create a mysql query
insert into your_table
select (every field..., date_add(duration_time, interval 15 minute) as duration_time from your_table
will duplicate every entry and add 15 minutes.
Then when you query your data you can format the time part as time-time+15 minutes in your python code
I have a 'timestamp' type column in my table called updated_date. When adding a column to the table, all rows got updated to the same updated_date. Not a disaster as we're still in testing, but it kind of broke the functionality of our site (which shows things in order of updated_date).
Is there a way I can change all the updated_date values in the column (but where id is lower than x) to some random date (or an incremental date)?
Thanks in advance!
This might solve your problem:
UPDATE updated_table SET timestamp = FROM_UNIXTIME(1e9 + id) WHERE id < x;
Basically it sets dates to Unix timestamps corresponding to 1 billion + id (1,000,000,000 unix timestamp is 2001-09-08 21:46:40). That way you get unique timestamps in order of id.
Well, you could do this
UPDATE table SET updated_time = NOW() WHERE id < x
Given id belongs to table
in case you want some random data from the past
UPDATE test2 SET update_time = NOW() - interval rand()*120 day - interval rand()*36000 second WHERE id < x
Tweak it to your needs
Timestamps are just the number of seconds since the epoch (1970-01-01 00:00:01). If you start with a base timestamp, you can just add a random number of seconds since that number and you have random dates.
I have Rails application which using MYSQL as database. For some condition, I have to delete all the records from table which was stored exactly 2 hours before the current time.
My query is :
DELETE FROM TABLE_NAME WHERE (NOW() - created_at) > 7200;
Here create_at is datetime column type. Storing the value in the format "2012-12-04 06:39:44"
My problem is, the above query fetch the records even though the record created time is just 40 to 50 minutes and got deleted. The only problem is the record got delete after it reach 40 to 50 minx from it create time.
Can any one please correct my query. I want the MySQL solution. Please help me
You probably need this if you want to delete records created exactly 2 hours ago:
DELETE FROM TABLE_NAME WHERE created_at = NOW() - INTERVAL 2 HOUR
or this, that will delete all records created more than 2 hours ago:
DELETE FROM TABLE_NAME WHERE created_at < NOW() - INTERVAL 2 HOUR
Try this ::
DELETE FROM TABLE_NAME WHERE TIMEDIFF(NOW(),created_at) < '02:00:00';
Try:
DELETE FROM TABLE_NAME WHERE created_at<DATE_SUB(NOW(),INTERVAL 2 HOUR)
This query will delete everything created MORE THAN 2 hours ago. Putting an equal sign would mean EXACTLY 2 hours ago (in second). Of course you can format date to consider only minutes, but that would slow down the query.
If created_at is indexed (and I think it should be) don't perform any functions on it so it can use index to perform delete faster.
I understand you want to delete all records created within a time lapse. So, you shouldn't apply a "greater than" operator to the subtract operation. Instead you should try to specify an appropriated time frame.
You could also take a look to the timediff function http://dev.mysql.com/doc/refman/5.5/en/date-and-time-functions.html#function_timediff
Sorry I'm not able to post the right statement for you, since I don't have a mysql server at hand.
I have the following table in MySQL that records event counts of stuff happening each day
event_date event_count
2011-05-03 21
2011-05-04 12
2011-05-05 12
I want to be able to query this efficiently by date range AND by day of week. For example - "What is the event_count on Tuesdays in May?"
Currently the event_date field is a date type. Are there any functions in MySQL that let me query this column by day of week, or should I add another column to the table to store the day of week?
The table will hold hundreds of thousands of rows, so given a choice I'll choose the most efficient solution (as opposed to most simple).
Use DAYOFWEEK in your query, something like:
SELECT * FROM mytable WHERE MONTH(event_date) = 5 AND DAYOFWEEK(event_date) = 7;
This will find all info for Saturdays in May.
To get the fastest reads store a denormalized field that is the day of the week (and whatever else you need). That way you can index columns and avoid full table scans.
Just try the above first to see if it suits your needs and if it doesn't, add some extra columns and store the data on write. Just watch out for update anomalies (make sure you update the day_of_week column if you change event_date).
Note that the denormalized fields will increase the time taken to do writes, increase calculations on write, and take up more space. Make sure you really need the benefit and can measure that it helps you.
Check DAYOFWEEK() function
If you want textual representation of day of week - use DAYNAME() function.