Let's say that I have a website and I want to know all the users that logged in during a certain time interval.
Would it be a good idea to create a new table in the database for this purpose and add a new entry whenever a users logs in?
The table would contain 2 columns: the id of the user and the login date.
My main concern is that the number of entries from the table will become extremely large.
Can this be considered a good idea? Do you know if this method is being applied for other websites?
Thanks in advance!
The number of records in a table can be controlled via external script, which is put on cron/scheduler. If it becomes too big, old records can be removed
if it is not possible, as a workaround there could be a check of the number of records on each insert
just do not forget to set an index on the date field...
Yes, you can create a table that logs all the login time of each user. If there are millions of users you might want to store the recent login time instead. If space is not a problem then it will be good to store the login time each time a user is authenticated or authorized. Like this you can archive the data in this table periodically.
The general answer to this question is 'depends'.
You can:
Add user to the table on login. You hit the disk for each user, so be careful with a big amount of users.
You store a bunch of users in memory and write all the group at a certain size or time. This way you hit the disk fewer times.
Depending on how many users you expect you can think of a no-SQL solution.
Depending on your system, I advise the 2nd o 3rd approach
Read this for more info: Fast write performance, even if reads are very slow
Related
So, I came up with an idea to store my user information and the updates they make to their own profiles in a way that it is always possible to rollback (as an option to give to the user, for auditing and support purposes, etc.) while at the same time improving (?) the security and prevent malicious activity.
My idea is to store the user's info in rows but never allow the API backend to delete or update those rows, only to insert new ones that should be marked as the "current" data row. I created a graphical explanation:
Schema image
The potential issues that I come up with this model is the fact that users may update the information too frequently, bloating up the database (1 million users and an average of 5 updates per user are 5 million entries). However, for this I came up with the idea of putting apart the rows with "false" in the "current" column through partitioning, where they should not harm the performance and will await to be cleaned up every certain time.
Am I right to choose this model? Is there any other way to do such a thing?
I'd also use a second table user_settings_history.
When a setting is created, INSERT it in the user_settings_history table, along with a timestamp of when it was created. Then also UPDATE the same settings in the user_settings table. There will be one row per user in user_settings, and it will always be the current settings.
So the user_settings would always have the current settings, and the history table would have all prior sets of settings, associated with the date they were created.
This simplifies your queries against the user_settings table. You don't have to modify your queries to filter for the current flag column you described. You just know that the way your app works, the values in user_settings are defined as current.
If you're concerned about the user_settings_history table getting too large, the timestamp column makes it fairly easy to periodically DELETE rows over 180 days old, or whatever number of days seems appropriate to you.
By the way, 5 million rows isn't so large for a MySQL database. You'd want your queries to use an index where appropriate, but the size alone isn't disadvantage.
I am using a MySql table to store a session record for the current logged in user. Once the user logs off, I update few fields in the same record and flags(revoked) it that it should not be used again. So for every LogIn a new record is created. This serves my purpose, but it turns out that the table is going to grow huge.
What should be the standard approach for storing Sessions? Should the ones, which are revoked be stored in a separate table, or should they be deleted or left in the same table?
I consider leaving the data in the same session table. While querying for a particular record, I query with two fields : (idPeople (not unique) and revoked (0 or 1)), for example SELECT * FROM session WHERE idPeople = "someValue" AND revoked = 0. and then update the record if needed while the user is, logged in or kogging out. Will the huge size of table affect this? or MySql will handle this? And what are other ramifications for this which I am unable to see?
First, it may be a good idea to add a unique field to your table (e.g. SESSION_ID, which could be a running auto-increment number), define this field as a unique ID, and use it to quickly find the record to be updated (i.e. revoke=1).
Second, this type of table always triggers the question you are asking, and the best answer can only be given after you assess and answer some preliminary questions, for instance:
When you wish to check the activities of a user, how far into the past does it make sense to go? One month? One year?
What is the longest period that you may wish to keep this information available (even using non routine queries to retrieve?
What type of questions (queries) I expect to be asked on this table?
One you answer those questions, you can consider the following options:
Have a routine process that would run once a day (at midnight or any other time your system can afford it) which would delete rows whose timestamp is older than, say, one month (or any other period suiting your needs), OR
Same as above but would first copy those records to an "history" table,
Change the structure of your table to a more efficient one, by adding some fields (as suggested above) and indices that would provide good answers for your "SELECT" needs.
I want design a inside system message for about 10000 user!
Steps seem easy:
1- create a DB table. for example: Message
2- create an user table. for example : user
3- the search Message DB and return all messages to user.
but you know for all 10000 user is too much time with one table to access messages.
My plan is create a independent table for each user for keep his messages. (not one table for all users)
but after that,Mybe I have 10000 tables in my DB. it is too much and administration is too heavy.
Is there a better way to save messages?
First , 10000 users is not so much data for mysql. In my system, I have more than 10,000,000 data in one table, and it works well.
Second, whether user and message should be stored in independent tables, this depends on the query. If you make a lot of join, I suggest You store them in one table, But if the join action is not that frequent, you can seperate them.
I checked one of my Joomla! websites this evening and to my horror found that I had thousands of spam registrations. I can't bring up all the users on one page on the website because it crashes, it's obviously too much for the server.
Even if I display 100 users per page, I've got 500 pages, it will take me until next week to delete them. So I thought maybe I can do from the database. The same thing happened, if I have 30 users showing, there are over 1000 pages. So I change the setting to show 1000 users, I wasn't able to delete the 1000 user because the page just crashed again.
So I'm thinking that maybe I can backup my own account from the user table. However, do I have to create another user table in order to reinstall my account? I hope you understand my dilemma
What I might do is go to phpmyadmin and export any data you want to keep even one at a time.
Then empty the table (i.e. delete all the rows).
Then import all of the data you exported back into the empty table.
Of it's just the one record you want to keep #Sparkup's answer will be quicker though.
Were you using a user profile plugin? If so you'll want to delete any records there also.
Then at minimum enable recaptcha, but also if you don't really want user registration, turn that off in the global configuration.
If you want to delete every user except your own you could do :
DELETE FROM users WHERE email != 'your_email';
Please note this will delete every other account
Be sure to make a backup of your database first.
If you want to remove emails with a certain extension :
DELETE FROM users WHERE email LIKE '%.co.uk';
DELETE FROM users WHERE email LIKE '%gmail.com';
Mysql Delete would be a good choice. Access Joomla database from terminal(linux) or cmd(windows) that would be fast .using captcha might be useful to stop spamming at certain extent.
I'm developing an application that
stores an e-mail address (to a user) in a table.
stores the number of days the user would like to stay in the table.
takes the user off the table when the number of days is up.
I don't really know how to approach this, so here are my questions:
Each second, do I have the application check through every table entry for the time that's currently stored in, let's say, the time_left column?
Wouldn't (1) be inefficient if I'm expecting a significant number (10,000+) users?
If not (2), what's the best algorithm to implement for such a task?
What's the name of what I'm trying to do here? I'd like to do some more research on it before and while I'm writing the script, so I need a good search query to start with.
I plan on writing this script in Perl, although I'm open to suggestions with regards to language choice, frameworks, etc... I'm actually new to web development (both on the back-end and front-end), so I'd appreciate it if you could advise me precisely.
Thank you!
*after posting, Topener asked a valid question:
Why would you store users if they won't get requested?
Assume the user is just sitting in the database.
Let's say I'm using the user's e-mail address every 5 minutes from the time the user was added to the database (so if the user's entry was born at 2:00PM-October 18, the user would be accessed at 2:05, 2:10, etc...).
If the user decides that they want out of the database in 10 days, that means their entry is being accessed normally (every 5 minutes from 2:00PM-October 18) until 2:00PM-October 28.
So to clarify, based on this situation:
The system would have to constantly compare the current time with the user's expiration date, wouldn't it?
you should not store the time_left variable, bt you should store vaildTo. This way, whenever the user is requested from the database, you can check if it is valid.
If not, then do whatever you want with it.
This approach wont let you make any cronjobs, or will cost you extramload.
Hey Mr_spock I like the above answer from Topener. Instead of storing a number of days the user would like to be valid, store the day the user would like to be be removed.
Adding a field like validToDate, which would be a DATETIME field type, you can do a query like
delete from tablename where validToDate <= NOW()
where
the italicized text is a SQL query
tablename is the name of the table in question
NOW() is a valid sql function that returns the current DATETIME
validToDate is a field of type DATETIME
This has what ever efficiency SQL server promises, I think it is fairly good.
You could write a separate program/script which makes the delete query on a set interval. If you are on a Linux machine you can create a cron job to do it. Doing it every second may become very resource intensive for slower machines and larger tables, but I don't believe that will become an issue for a simple delete query.