I'm developing an application that
stores an e-mail address (to a user) in a table.
stores the number of days the user would like to stay in the table.
takes the user off the table when the number of days is up.
I don't really know how to approach this, so here are my questions:
Each second, do I have the application check through every table entry for the time that's currently stored in, let's say, the time_left column?
Wouldn't (1) be inefficient if I'm expecting a significant number (10,000+) users?
If not (2), what's the best algorithm to implement for such a task?
What's the name of what I'm trying to do here? I'd like to do some more research on it before and while I'm writing the script, so I need a good search query to start with.
I plan on writing this script in Perl, although I'm open to suggestions with regards to language choice, frameworks, etc... I'm actually new to web development (both on the back-end and front-end), so I'd appreciate it if you could advise me precisely.
Thank you!
*after posting, Topener asked a valid question:
Why would you store users if they won't get requested?
Assume the user is just sitting in the database.
Let's say I'm using the user's e-mail address every 5 minutes from the time the user was added to the database (so if the user's entry was born at 2:00PM-October 18, the user would be accessed at 2:05, 2:10, etc...).
If the user decides that they want out of the database in 10 days, that means their entry is being accessed normally (every 5 minutes from 2:00PM-October 18) until 2:00PM-October 28.
So to clarify, based on this situation:
The system would have to constantly compare the current time with the user's expiration date, wouldn't it?
you should not store the time_left variable, bt you should store vaildTo. This way, whenever the user is requested from the database, you can check if it is valid.
If not, then do whatever you want with it.
This approach wont let you make any cronjobs, or will cost you extramload.
Hey Mr_spock I like the above answer from Topener. Instead of storing a number of days the user would like to be valid, store the day the user would like to be be removed.
Adding a field like validToDate, which would be a DATETIME field type, you can do a query like
delete from tablename where validToDate <= NOW()
where
the italicized text is a SQL query
tablename is the name of the table in question
NOW() is a valid sql function that returns the current DATETIME
validToDate is a field of type DATETIME
This has what ever efficiency SQL server promises, I think it is fairly good.
You could write a separate program/script which makes the delete query on a set interval. If you are on a Linux machine you can create a cron job to do it. Doing it every second may become very resource intensive for slower machines and larger tables, but I don't believe that will become an issue for a simple delete query.
Related
Let's say that I have a website and I want to know all the users that logged in during a certain time interval.
Would it be a good idea to create a new table in the database for this purpose and add a new entry whenever a users logs in?
The table would contain 2 columns: the id of the user and the login date.
My main concern is that the number of entries from the table will become extremely large.
Can this be considered a good idea? Do you know if this method is being applied for other websites?
Thanks in advance!
The number of records in a table can be controlled via external script, which is put on cron/scheduler. If it becomes too big, old records can be removed
if it is not possible, as a workaround there could be a check of the number of records on each insert
just do not forget to set an index on the date field...
Yes, you can create a table that logs all the login time of each user. If there are millions of users you might want to store the recent login time instead. If space is not a problem then it will be good to store the login time each time a user is authenticated or authorized. Like this you can archive the data in this table periodically.
The general answer to this question is 'depends'.
You can:
Add user to the table on login. You hit the disk for each user, so be careful with a big amount of users.
You store a bunch of users in memory and write all the group at a certain size or time. This way you hit the disk fewer times.
Depending on how many users you expect you can think of a no-SQL solution.
Depending on your system, I advise the 2nd o 3rd approach
Read this for more info: Fast write performance, even if reads are very slow
I am using a MySql table to store a session record for the current logged in user. Once the user logs off, I update few fields in the same record and flags(revoked) it that it should not be used again. So for every LogIn a new record is created. This serves my purpose, but it turns out that the table is going to grow huge.
What should be the standard approach for storing Sessions? Should the ones, which are revoked be stored in a separate table, or should they be deleted or left in the same table?
I consider leaving the data in the same session table. While querying for a particular record, I query with two fields : (idPeople (not unique) and revoked (0 or 1)), for example SELECT * FROM session WHERE idPeople = "someValue" AND revoked = 0. and then update the record if needed while the user is, logged in or kogging out. Will the huge size of table affect this? or MySql will handle this? And what are other ramifications for this which I am unable to see?
First, it may be a good idea to add a unique field to your table (e.g. SESSION_ID, which could be a running auto-increment number), define this field as a unique ID, and use it to quickly find the record to be updated (i.e. revoke=1).
Second, this type of table always triggers the question you are asking, and the best answer can only be given after you assess and answer some preliminary questions, for instance:
When you wish to check the activities of a user, how far into the past does it make sense to go? One month? One year?
What is the longest period that you may wish to keep this information available (even using non routine queries to retrieve?
What type of questions (queries) I expect to be asked on this table?
One you answer those questions, you can consider the following options:
Have a routine process that would run once a day (at midnight or any other time your system can afford it) which would delete rows whose timestamp is older than, say, one month (or any other period suiting your needs), OR
Same as above but would first copy those records to an "history" table,
Change the structure of your table to a more efficient one, by adding some fields (as suggested above) and indices that would provide good answers for your "SELECT" needs.
Currently we're working on a solution where we want to track (for analysis) the articles a user clicks on/opens and 'likes' from a given list of articles. Subsequently, the user needs to be able to see and re-click/open the article (searching is not needed) in a section on his/her personal user profile. Somewhere around the 100 new articles are posted every day. The increasing(!) amount of daily visitor (users) lies around the 2000 a day. The articles are currently stored and maintained within a MySQL Db.
We could create a new record in the MySQL Db for every article read / 'liked'. 'Worst case' would this create (2500 * 100 = ) 250000 records a day. That won’t hold long of course… So how would you store (process) this within XPages, given the scenario?
My thoughts after reading “the article” :) about MIME/Bean’s: what about keeping 'read articleObjects' in a scope and (periodically) store/save them as MIME on the user profile document? This only creates 100 articleObjects a day (or 36500 a year). In addition, one could come up with a mechanism where articleObjects are shifted from one field to another as time passes by, so the active scope would only contain the 'read articleObjects' from last month or so.
I would say that this exactly what a relational database is for. My first approach would be to have a managed bean (session scope) to read/access user's data in MySQL (JDBC). If you want, you can build an internal cache inside the bean.
For the presented use case, I would not bother with the JDBC datasources in ExtLib. Perhaps even the #Jdbc functions would suffice.
Also, you did not say how you are doing the analysis? If you store the information in Domino, you'll probably have to write an export tool.
Okay..
I am making a web base application,that will be connected to a sms gateway.
It is basically an attendance app for colleges.
The Attendance will be updated using a sms by the teacher.
Now,the main part-
What I want to do is,the teacher should not be able to correct the attendance after 10minutes of sending the 1st message.i.e,the database should accept a correction or new message for the same class and the same teacher only for 10 minutes after the 1st attendance is recieved in the database.
So only recieving from a particular number should be blocked and also only if it is for the same class...
I hope the question is clear :o
Thankyou
This is not the sort of thing that you should be enforcing at the DB level, it belongs in your application code. If you can't connect time, number & class together in your DB, it's time to change your schema.
As Sean McSomething mentioned, this is not done at database level, this is business logic that should be checked just before interacting with a database. The best practice actually is to simply add a column time_created and before updating simply check if NOW() and time_created interval is less than 10 minutes. It's a pretty trivial job, but don't bother trying to do this in database with some stored procedures or other stuff, as it will make your application almost un-debuggable and very sloppy.
Check if there is a row with active number and active class, if there are no - insert, if there are any - check if this row's time_created is greater than 10 minutes ago, if it's not - update, ignore otherwise.
I have a resource that has a availability field that lists what hours of a day its available for use?
eg. res1 available between 0-8,19-23 hours on a day, the range here can be comma separated values of hour ranges. e.g are 0-23 for 24 hour access, 0-5,19-23 or 0-5,12-15,19-23
What's the best way to store this one? Is char a good option? When the resource is being accessed, my php needs to check the current hour with the hour defined here and then decide whether to allow this access or not. Can I ask mysql to tell me if the current hour is in the range specified here?
I'd store item availability in a separate table, where for each row I'd have (given your example):
id, startHour, endHour, resourceId
And I'd just use integers for the start and end times. You can then do queries against a join to see availability given a certain hour of the day using HOUR(NOW()) or what have you.
(On the other hand, I would've preferred a non-relational database like MongoDb for this kind of data)
1) create a table for resource availability, normalized.
CREATE TABLE res_avail
{
ra_resource_id int,
ra_start TIME,
ra_end TIME
# add appropriate keys for optimization here
)
2) populate with ($resource_id, '$start_time', '$end_time') for each range in your list (use explode())
3) then, you can query: (for example, PHP)
sql = "SELECT ra_resource_id FROM res_avail where ('$time' BETWEEN ra_start AND ra_end)";
....
I know this is an old question, but since v5.7 MySQL supports storing values in JSON format. This means you can store all ranges in one JSON field. This is great if you want to display opening times in your front-end using JavaScript. But it's not the best solution when you want to show all places that are currently open, because querying on a JSON field means a full table scan. But it would be okay if you only need to check on for one place at the time. For example, you load a page showing the details of one place and display whether it's open or closed.