I'm going to try to explain this best I can I will provide more information if needed quickly.
I'm storing data for each hour in military time. I only need to store a days worth of data. My table structure is below
CREATE TABLE `onlinechart` (
`id` int(255) NOT NULL AUTO_INCREMENT,
`user` varchar(100) DEFAULT NULL,
`daytime` varchar(10) DEFAULT NULL,
`maxcount` smallint(20) DEFAULT NULL,
`lastupdate` varchar(100) DEFAULT NULL,
PRIMARY KEY (`id`)
) ENGINE=innodb AUTO_INCREMENT=2 DEFAULT CHARSET=latin1
The "user" column is unique to each user. So I will have list for each user.
The "daytime" column I'm having it store the day and hour together. So as for today and hour it would be "2116" so the day is 21 and the current hour is 16.
The "maxcount" column is what data for each hour. I'm tracking just one total number each hour.
The "lastupdate" column is just a timestamp im using to delete data that is 24 hours+ old.
I have the script running in PHP fine for the tracking. It keeps a total of 24 rows of data for each user and deletes anything older then 24hours. My problem is how would I go about a query that would start from the current hour/day and pull that past 24 hours maxcount and display them in order.
Thanks
You will run into an issue of handling this near the end of the year. It's advisable you switch to using the native timestamp type of MySQL (described here: http://dev.mysql.com/doc/refman/5.0/en/datetime.html). Then you can grab max count by doing something such as:
SELECT * FROM onlinechart WHERE daytime >= ? ORDER BY maxcount
The question mark should be replaced by the timestamp - 86400 (number of seconds in a day).
Related
I have a weather observation table with frequent entries - for simplicity let's consider just the temperature observations. The observations can be somewhat sporadic, but sometimes up to half a dozen occur in each clock hour interval. My goal is to run a procedure at, say, hourly intervals to find those historical hours that contain multiple temperature observations and find the average temperature and time and then to replace all those observations with the single averaged observation.
I have managed to compose a mysql query which creates an averaged temperature value for the hour interval (shown below) but am needing assistance to know how to take this one step further by actually replacing each hours observation entries with the single new average entry.
SELECT stationcode, AVG(temperature) as t_avg, COUNT(temperature) as t_count, FROM_UNIXTIME
(AVG(UNIX_TIMESTAMP(obs_datetime))) as datetime_avg, MINUTE(obs_datetime) as minute, HOUR(obs_datetime) as hour, DAY(obs_datetime) as day, MONTH(obs_datetime) as month, YEAR(obs_datetime) as year
FROM obs_table
WHERE stationcode='AT301'
GROUP BY hour, day, month, year
HAVING count(*) > 1
ORDER BY datetime_avg DESC
I am imagining that the solution might involve a join or a temporary table. Can anyone provide any sample code or hints as to how I can go about this?
Adding following due to a request for the table structure:
--
-- Table structure for table `obs_table`
--
CREATE TABLE `obs_table` (
`rec_id` bigint(12) UNSIGNED NOT NULL,
`stationcode` varchar(8) NOT NULL,
`obs_datetime` datetime NOT NULL,
`temperature` float DEFAULT NULL,
`temp_dewpt` float DEFAULT NULL,
`rel_humidity` float DEFAULT NULL,
`wind_dir_degs` float DEFAULT NULL,
`wind_avg_kmh` float DEFAULT NULL,
`wind_gust_kmh` float DEFAULT NULL,
`pressure_hpa` float DEFAULT NULL,
`visibility_m` float DEFAULT NULL,
`description` varchar(255) DEFAULT NULL,
`icon` varchar(255) DEFAULT NULL
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
--
-- Indexes for table `obs_table`
--
ALTER TABLE `obs_table`
ADD PRIMARY KEY (`rec_id`),
ADD UNIQUE KEY `stationcode` (`stationcode`,`obs_datetime`);
I am working on an e-commerce website where user can show interest on available products and we store it as a lead in mysql table. This Leads table consists of millions of records and grows by 8 records per seconds. Table structure is as follows:
LeadId | ProductId | UserId | RequestDate(DateTime)
Table Schema:
`id` int(11) NOT NULL AUTO_INCREMENT,
`ProductId` int(11) DEFAULT NULL,
`UserID` int(11) NOT NULL,
`RequestDateTime` datetime(3) NOT NULL,
PRIMARY KEY (`id`),
KEY `ix_leads_requestdatetime` (`RequestDateTime`) USING BTREE,
KEY `ix_leads_productid` (`ProductId`) USING BTREE,
KEY `ix_leads_userid` (`UserID`) USING BTREE
Now, the requirement is to allow one user to give maximum 10 leads in a day. I have following approaches to implement this:
Select query to count number of records for that day in Leads table and check if < 20 before insertion.
Maintain a DailyLeadCount table which contains count of leads for each userId for particular date. Table Structure:
UserId | Date | Count
Table Schema:
`RequestDate` date NOT NULL,
`UserId` int(11) NOT NULL,
`LeadCount` smallint(6) NOT NULL,
PRIMARY KEY (`RequestDate`,`UserId`)
I will check count in this table before inserting in Leads table and update this count after insertion accordingly. Also, as only one day data will be useful in this table, I will create a job to archive it daily.
Which approach is better? Is running select query on Leads table to get count more heavy than insert/update and select query on DailyLeadCount table?
Is it worth maintaining and archiving a table daily for it?
Is there any other way to handle this?
Change
KEY `ix_leads_userid` (`UserID`) USING BTREE
to
INDEX(UserID, RequestDateTime)
Then spit at the user when
( SELECT COUNT(*) FROM Leads WHERE UserID = 1234
AND RequestDateTime > NOW() - INTERVAL 24 HOUR
) >= 10
The query will be fast enough to do in real time.
The count is between this time yesterday and now -- this may not be exactly what you want. If, instead, you want the clock to start at midnight this morning:
AND RequestDateTime > CURDATE()
If "since midnight yesterday":
AND RequestDateTime > CURDATE() - INTERVAL 1 DAY
If you want to use his timezone for midnight, it gets messier.
Potential issue: If he can somehow batch his leads, he could insert multiple leads in the same millisecond. (I am noticing DATETIME(3).)
Your idea of a Summary Table works best if you need to check against "yesterday", not so well for "the last 86400000 milliseconds".
I want to update an column after every 20 minutes but it wont not work as I will. I use this SQL:
UPDATE visitors SET
is_online = '0'
WHERE is_online = '1'
AND DATE_ADD(date_lastactive, INTERVAL 20 MINUTE) < NOW()
The database looks like this:
CREATE TABLE IF NOT EXISTS `visitors` (
`id` int(10) NOT NULL AUTO_INCREMENT,
`ipaddress` text NOT NULL,
`page` text NOT NULL,
`page_get` text NOT NULL,
`date_visited` datetime NOT NULL,
`date_lastactive` datetime NOT NULL,
`date_revisited` datetime NOT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `id` (`id`)
)
I have tried to change the < to > but it updates after every page refreshes with that arrow.
How can I fix my problem?
Thanks in advance.
If you need to run this query every 20 minutes independently on your site visitors and page loads you have to use a system scheduler: cron on Unix, and Task Scheduler on Windows.
Just code a simple shell script.
You can not make an sql query that repeats itself in every 20 minutes. There is no such of combination of mysql statements.
For reference, this is my current table:
`impression` (
`impressionid` bigint(19) unsigned NOT NULL AUTO_INCREMENT,
`creationdate` datetime NOT NULL,
`ip` int(4) unsigned DEFAULT NULL,
`canvas2d` tinyint(1) DEFAULT '0',
`canvas3d` tinyint(1) DEFAULT '0',
`websockets` tinyint(1) DEFAULT '0',
`useragentid` int(10) unsigned NOT NULL,
PRIMARY KEY (`impressionid`),
UNIQUE KEY `impressionsid_UNIQUE` (`impressionid`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1 AUTO_INCREMENT=447267 ;
It keeps a record of all the impressions on a certain page. After one day of running, it has gathered 447266 views. Those are a lot of records.
Now I want the amount of visitors per minute. I can easily get them like this:
SELECT COUNT( impressionid ) AS visits, DATE_FORMAT( creationdate, '%m-%d %H%i' ) AS DATE
FROM `impression`
GROUP BY DATE
This query takes a long time, of course. Right now around 56 seconds.
So I'm wondering what to do next. Do I:
Create an index on creationdate (I don't know if that'll help since I'm using a function to alter this data by which to group)
Create new fields that stores hours and minutes separately.
The last one would cause there to be duplicate data, and I hate that. But maybe it's the only way in this case?
Or should I go about it in some different way?
If you run this query often, you could denormaize the calculated value into a separate column (perhaps by a trigger on insert/update) then grouping by that.
Your idea of hours and minutes is a good one too, since it lets you group a few different ways other than just minutes. It's still denormalization, but it's more versatile.
Denormalization is fine, as long as it's justified and understood.
I am trying to create a table in MySQL that has a timestamp field that is 7 days ahead of now() is there a way to do this?
This is my current code
CREATE TABLE tbl_reg
(
reg_id int(7) NOT NULL auto_increment KEY,
user_id int(7) NOT NULL,
registration_key char(50) NOT NULL,
registration_expires timestamp default now()+7days,
stamp_created timestamp default now()
);
Can anyone help as i cant find anything like this on the mysql site and wondered if there was a way to do it?
There are a number of date/time functions in MySQL that will do the trick here.
http://dev.mysql.com/doc/refman/5.5/en/date-and-time-functions.html#function_date-add
registration_expires=DATE_ADD(NOW(), INTERVAL 7 DAY)
You can't set an expression as a default, though - you'll need to do it in your INSERT queries. Notice that even if your expr value is > 1 there is no plural used for the unit value.
Or you could create a view from a query where you add the interval, or when you query the db always add the 7 days interval.