Trying to delete several thousand users from database - mysql

I checked one of my Joomla! websites this evening and to my horror found that I had thousands of spam registrations. I can't bring up all the users on one page on the website because it crashes, it's obviously too much for the server.
Even if I display 100 users per page, I've got 500 pages, it will take me until next week to delete them. So I thought maybe I can do from the database. The same thing happened, if I have 30 users showing, there are over 1000 pages. So I change the setting to show 1000 users, I wasn't able to delete the 1000 user because the page just crashed again.
So I'm thinking that maybe I can backup my own account from the user table. However, do I have to create another user table in order to reinstall my account? I hope you understand my dilemma

What I might do is go to phpmyadmin and export any data you want to keep even one at a time.
Then empty the table (i.e. delete all the rows).
Then import all of the data you exported back into the empty table.
Of it's just the one record you want to keep #Sparkup's answer will be quicker though.
Were you using a user profile plugin? If so you'll want to delete any records there also.
Then at minimum enable recaptcha, but also if you don't really want user registration, turn that off in the global configuration.

If you want to delete every user except your own you could do :
DELETE FROM users WHERE email != 'your_email';
Please note this will delete every other account
Be sure to make a backup of your database first.
If you want to remove emails with a certain extension :
DELETE FROM users WHERE email LIKE '%.co.uk';
DELETE FROM users WHERE email LIKE '%gmail.com';

Mysql Delete would be a good choice. Access Joomla database from terminal(linux) or cmd(windows) that would be fast .using captcha might be useful to stop spamming at certain extent.

Related

Safely Drop User Table in Ruby on Rails Application

I'm running a Ruby on Rails application. In this system, Users can login and logout as necessary. I used gem devise as well. I've realized now that I have a lot of Users in my system, but most of them are no longer users. I can delete them from the system, but I want to start afresh.
My question is, is it safe to drop the Users table, User.delete_all? Then insert myself using seed.rb as admin user. Then add all Users (new and old) from the system (Add New User feature I have on the system).
What I believe you're looking for is the delete_all-action, if you have no callbacks etc in the User-model. Dropping the table, is not what you'll want, as you would have to rebuild and recreate the table. By deleting the rows in it, you're simply removing the content.
Depending on your use of Users, you could argue, that it's safe/unsafe to delete content. If the users are persisted, and you're storing password etc, you should not empty it.
If, on the other hand, a User is in fact a "session", or similar, you should be safe to empty it, as that would simply "log out" your users.
Another consideration is, if you need to cleanup the users, maybe add a field last_logged_in or similar. Leave it for a while (And of course update it, when a user logs in), and then delete anyone that hasn't logged in for, say 1 month; User.where("last_logged_in < ?", 1.month.ago).delete_all.
I cannot recommend, just deleting content, without testing first, and ensuring that Users data are stored, or 'easily recoverable', by a new login. Or atleast that they're actually able to login without being baffled. Also, ensure that any relations from User, is handled, or you'll end up with orphaned records in other tables.

Store all user's login dates

Let's say that I have a website and I want to know all the users that logged in during a certain time interval.
Would it be a good idea to create a new table in the database for this purpose and add a new entry whenever a users logs in?
The table would contain 2 columns: the id of the user and the login date.
My main concern is that the number of entries from the table will become extremely large.
Can this be considered a good idea? Do you know if this method is being applied for other websites?
Thanks in advance!
The number of records in a table can be controlled via external script, which is put on cron/scheduler. If it becomes too big, old records can be removed
if it is not possible, as a workaround there could be a check of the number of records on each insert
just do not forget to set an index on the date field...
Yes, you can create a table that logs all the login time of each user. If there are millions of users you might want to store the recent login time instead. If space is not a problem then it will be good to store the login time each time a user is authenticated or authorized. Like this you can archive the data in this table periodically.
The general answer to this question is 'depends'.
You can:
Add user to the table on login. You hit the disk for each user, so be careful with a big amount of users.
You store a bunch of users in memory and write all the group at a certain size or time. This way you hit the disk fewer times.
Depending on how many users you expect you can think of a no-SQL solution.
Depending on your system, I advise the 2nd o 3rd approach
Read this for more info: Fast write performance, even if reads are very slow

How to correctly handle multiple privileged MySQL connection accounts

I have read many posts and articles that state quite clearly that for the best in worst-case damage limitation, MySQL accounts should only be able to do what they're intended to do, and nothing else. This makes perfect sense and so there would be a connection account to my Database that does all the SELECTing of data, and another account that does UPDATE and INSERT activities, so that if someone does get the compromise by SELECTing data they shouldn't, then they can't quite as easily then UPDATE that data or INSERT etc.
You get the idea.
But, I have various Databases and use accounts to read/SELECT data and the this is output to the client page, but often these accounts will need to be updated, small things such as updating when a (website) user logs into their account, or updating some sort of hitcounter or other minor feature. Given the ringfencing of concerns outlined above, I feel it's a bit like using a flood to put out a campfire, to allow UPDATEing (etc.) to a privileged MySQL connection simply to say that user Bob logged in last at 4:10pm.
I have been digging on the web for suitable guides, blog posts and articles about how to best structure using multiple MySQL privileged accounts to complete the nessecary work with as minimum a risk of excess privilege as possible, but I have found nothing that has been much use, (mostly because of my wording seems to be attracting articles about setting up website users, or other topics associated with these keywords :-/ )
I have a few ideas on current approach and wanted a bit of feedback on the best method for doing activities as described in paragraph 2, typically 95% SELECTing, and a few specific instances of UPDATEing, or if any of the following are possible (or on flipside, are very bad ideas)? :
I currently have seperate PHP connection objects for each connection privilege user. Is this the best approach?
Could I somehow giving a privilege user access to only update a certain table (or even a certain table column?), as well as SELECT from any table? This would be perfect.
Are using TRIGGERs a common approach and would this have any down sides if I created a Trigger (with a privileged user) and then let a SELECT user account access triggers?
Could I set certain users can only use certain triggers?
Is there another way of doing this?
MySQL allows for users to have different privileges set both at database and individual table levels. The documentation on the GRANT (http://dev.mysql.com/doc/refman/5.7/en/grant.html) syntax gives an example of setting all privileges to a user on one database while only select access to a table in another database.
Privileges can even be set for specific columns (http://dev.mysql.com/doc/refman/5.7/en/grant.html#grant-column-privileges) in a table & also for stored procedures (http://dev.mysql.com/doc/refman/5.7/en/grant.html#grant-routine-privileges).

MySql : how can I implement a sort of "read/not read yet" topics?

I'm about to implement a list of topic/argument in my forum, and I'd like to insert a sort of flag like "read/not read yet" for each message, regard each user in my website.
I think at somethings like this : a table watched_topics with id(INT), user(VARCHAR) and topic_id(INT). When a user watch the page, I'll insert (if the data doesn't exist) these information.
When another user will insert a new message in a topic, I'll delete from the table watched_topics all line with that topic_id.
That could provide a trouble : Think about to 9000 topics and 9000 users that have watched all topics : the table will be so big (9000x9000=81000000).
So, I think is not the best strategy to implement this kind of stuff! Any suggestion would be appreciated :)
Cheers
May I suggest a different approach?
Make use of web browser history mechanism.
Every topic can get a new, unique URL every time a new message is added there. It could include the number of messages, last modified time or a combination of both.
If the user did see the topic, he must have visited it, so a properly set up CSS can help identifying the read ones. You can even use some client-side scripts to modify the behaviour of the page based on that.
Another way to do that would be to keep the watched topics table the way you want to do it, but also store last visit time in user's profile and show all topics as read that haven't changed since that time.
However it's pretty safe to assume that all users reading all topics is very unlikely.
Your suggestion sounds good. I would make user-field also a foreign key - it gives you a bit more flexibility.
Are you sure all 9000 topics are read by all 9000 users? I mean is this reality? Like you said, topic-entries are deleted when new message is added. And when that happens, another 9000 entries are deleted :)
I would index the table and go with your suggestion (with user_id change). If the table size gets in your way, you can always change the implementation later. Most likely it will never be the issue anyway.
For the deletion: you could save what the latest msg-ID was the user saw. This way you do not have to perform a lot of delete actions every time a msg is posted in a much-viewed topic.

MySQL - Saving and loading

I'm currently working on a game, and just a while ago i started getting start on loading and saving.
I've been thinking, but i really can't decide, since I'm not sure which would be more efficient.
My first option:
When a user registers, only the one record is inserted (into 'characters' table). When the user tries to login, and after he/she has done so successfully, the server will try loading all information from the user (which is separate across multiple tables, and combines via mysql 'LEFT JOIN'), it'll run though all the information it has and apply them to the entity instance, if it runs into a NULL (which means the information isn't in the database yet) it'll automatically use a default value.
At saving, it'll insert or update, so that any defaults that have been generated at loading will be saved now.
My second option:
Simply insert all the required rows at registration (rows are inserted when from website when the registration is finished).
Downsides to first option: useless checks if the user has logged in once already, since all the tables will be generated after first login.
Upsides to first option: if any records from tables are deleted, it would insert default data instead of kicking player off saying it's character information is damaged/lost.
Downsides to second option: it could waste a bit of memory, since all tables are inserted at registration, and there could be spamming bots, and people who don't even manage to get online.
Upsides to first option: We don't have to check for anything in the server.
I also noted that the first option may screw up any search systems (via admincp, if we try looking a specific users).
I would go with the second option, add default rows to your user account, and flag the main user table as incomplete. This will maintain data integrity across your database, whereas every user record is complete in it's entirety. If you need to remove the record, you can simply add a cascading delete script to clean house.
Also, I wouldn't develop your data schema based off of malacious bots creating accounts. If you are concerned about the integrity of your user accounts, add some sort of data validation into your solution or an automated clean-house script to clear out incomplete accounts once the meet a certain criteria, i.e. the date created meeting a certain threshold.
You mention that there's multiple tables of data for each user, with some that can have a default value if none exist in the table. I'm guessing this is set up something like a main "characters" table, with username, password, and email, and a separate table for something like "favorite shortcuts around the site", and if they haven't specified personal preferences, it defaults to a basic list of "profile, games list, games by category" etc.
Then the question becomes when registering, should an explicit copy of the favorite shortcuts default be added for that user, or have the null value default to a default list?
I'd suggest that it depends on the nature of the auxiliary data tables; specifically the default value for those tables. How often would the defaults change? If the default changes often, a setup like your first option would result in users with only a 'basic' entry would frequently get new auxiliary data, while those that did specify their own entries would keep their preferences. Using your second option, if the default changed, in order to keep users updated, a search/replace would have to be done to change entries that were the old default to the new default.
The other suggestion is to take another look at your database structure. You don't mention that your current table layout is set in stone; is there a way to not have all the LEFT JOIN tables, and have just one 'characters' table?