I have been searching for this for a while and unable to find something useful.
Is it a good practice or even important to create 2 MySQL users, one for reading and then use that whenever I'm initiating a MySQL SELECT.
And on the other side, create another user for writing and use it whenever I'm doing an INSERT, UPDATE, DELETE, ...?
Would this help at anything for example if I'm writing and reading to the database at the same time?
Assume we're using InnoDB tables.
"good practice" is very hard to define - you've got a whole bunch of different things to trade off against each other.
I'm assuming that the database is being used as a back-end for some other system, and that your users don't have direct access to a SQL prompt. In that case, there are no real benefits to creating different MySQL users - it simply makes the front-end more complex, and an attacker who can reach the database and knows the "read-only" credentials almost certainly also knows the "read/write" credentials. From a security point of view, you should invest your time in network security of the database server, and secure storage of connection details.
From a concurrency point of view - two or more users reading and writing at the same time - you won't really gain anything either. This particular requirement is one of the things relational databases do very well, and I don't think it's affected at all by the permissions of the users - it's far more to do with whether you're using transactions, and how quickly your SQL executes.
Related
I have an interface that I want to allow an arbitrary SQL select statement (as an input string) to be input that will select data from a given table for use in an operation. I want to make sure that this statement does not make changes to the database.
string query = GetStringFromForm(...);
DatabaseStatement statement(query);
statement.execute();
while (statement.fetch(...))
...
One way to implement this would be to create a new database user with the appropriate permissions and then execute the statement under that user. This would be a hassle as it would require setting up this new user and creating a new database connection for it and so on.
Is there a way to isolate the permissions for a single statement MySQL 5.5? Or some other way to do this?
With MySQL 5.6 you can do:
START TRANSACTION READ ONLY;
https://dev.mysql.com/doc/refman/5.6/en/commit.html
I think it's what you're looking for, but you have to upgrade to 5.6 to use it.
Don't connect to the database with the same login for everything.
At the very least you should use thee logins for this achitecture:
A development level login for creating tables, etc
The login used by your application to make the application run
The login used to execute user specified queries
This means that your application login only has the permissions it needs - to read or write to the tables necessary, not to do everything to every table; application logins shouldn't need to be able to CREATE or DROP tables, for example.
This limits the impact of mistakes in code, but also the scope to which someone could hack your system (such as with SQL Injection attacks).
It also means that the login for running user specific queries needs only to be granted SELECT permissions, and only to the tables/views/function that it should be able to use. If they try to run an INSERT or a DELETE that they don't have permissions for, you can catch the error and tell the user that they're a very naughty boy - secure in the mind the the RDBMS simply won't let the user damage anything that you haven't already given them permission to do.
In short, RDBMS already have login permission architectures. Use those to limit the permissions and functionallity of different aspects of your code.
I would not try to re-invent this wheel. It is extremely likely that there is a trick or hack that you missed that exposes a vulnerability in your application. I appreciate that you say this is a hassle, but it really is the right way of doing things, and the only reliable way of doing things. There's a reason that it's the standard approach to data security, sorry.
(And trust me, even if no-one is trying to hack your system, eventually someone will type some screwball query in - accidentally bypassing your security and making a pigs ear of your database.)
I have used sql (mostly mysql) for years but not to a professional standard, so I'm looking for a shove in the right direction.
I am currently designing a web app that will collect user's names/addresses/emails etc in one set of tables, as well as other personal information in another set of tables. These would most naturally reside in one database, but I've been considering splitting the user contact information in one database on a separate server and all the other information into another database/server, the theory being that a hacker would have to break both systems to get anything very useful.
I've done searches off and on for a few weeks and haven't found this type of design discussed much so far. Is this generally done? Is it overkill? Is there a design method to approach it, or will I have to roll it all on my own?
I did find Is splitting databases a legitimate security measure? which I guess is saying that this approach is likely overkill.
I tend to think this is overkill.
Please check my answer on this question: Sharing users between 2 databases
Keep in mind to address separately database design and data access
security issues. Data access security should not lead you to illogical
choices in database design.
IMHO that seems to be wrong. By splitting data across 2 DB you will only increase complexity without reasonable security profits.
I think this is where data encryption can be used. Generate encryption key based on user credentials and encrypt/decrypt sensible data by user requests. Since private data must be shown only to that user, everything should be ok.
Here's an approach I used before:
Server1: DB
Server2: SC
DB is in a network domain that is accessible by the public, but cannot access SC
SC is in a network domain that is not accessible by the public, but can access SC
DB is where you stored all pertinent information, including the 'really important stuff'.
At a specified interval (I used 5 seconds) SC checks DB for any new records in any table it may want to monitor (there is a job or scheduled task) and encrypts the important information.
Although I was utilizing SQL Server 2005 and was able to work in two domains (a private(intern al) and public(for client access) and that what I just shared was a stripped down (removed as much MSSQL-exclusive parts), simplified version, with some effort I think it would be possible to recreate something similar in mysql, especially if you can host your two databases in separate, physical machines.
While many will also think this is overkill, this idea had been implemented. It costs more, and requires more work when it's data reporting time but the clients were pleased.
When a user registers on my site, they have their own table created in one of my databases. This table stores all of the posts that the user makes.
What I would also like to do is also generate them their own MySql user - which ONLY has permission to read, write and delete from their table.
Creating that shouldn't be a problem - I've got Google for that.
What I'm wondering is, let's imagine that I clock myself 10,000,000 or more users at one point in the future, would having that many MySql users affect my database performance?
For the sake of answering your question, a quick points... before I explain why you are doing it wrong...
The performance hit will come from having massive amount of tables. (The limit is massive so should you ever reach that high, I would for gods sake hope that you recruit someone who can slap your database silly and explain why you have mutilated it so much). Excuse the harshness :)
Okay, now onto how you should actually be doing it.
Multi-Tenancy
First, you need to learn about how to design a database that is designed for multi tenant application. This is exactly what you are creating by the sounds of it, but you are doing it COMPLETELY wrong. I cannot stress that enough.
Here are some resources which you should read immediately.
Quick overview of what multi tenancy actually is (You can skim read this one).
Read this Multi-Tenant Data Archictecture article several times! Then repeat.
Then read this question:
- How to design a multi tenant mysql database
After you have done that. You should learn about ACL (Access Control Lists).
If you explain what sort of data you are trying to model, I will be happy to update this post with a simple table schema to match what you might require.
Yes it will drop your performance. Usually a server application uses a database connection pool with several connections (say app_user is connected 5 times). Every SQL request is handled by one of these connections. That way the overhead of creating a new connection, handling the query and dropping the connection is reduced to a minimum.
Now in your scenario every user would have his own table with its own user. That means if a user logs into your application he has to open his own connection, as he has to use his specific user account. Now instead of just 5 connections 10,000 connections have to be opened. That would not scale as each connection has its own thread and uses some ram space. Furthermore there are only about 64k ports available for your connections.
So your application would not scale for that many users.
I'm working on an application that's similar to Wufoo in that it allows our users to create their own databases and collect/present records with auto generated forms and views.
Since every user is creating a different schema (one user might have a database of their baseball card collection, another might have a database of their recipes) our current approach is using MySQL to create separate databases for every user with its own tables. So in other words, the databases our MySQL server contains look like:
main-web-app-db (our web app containing tables for users account info, billing, etc)
user_1_db (baseball_cards_table)
user_2_db (recipes_table)
....
And so on. If a user wants to set up a new database to keep track of their DVD collection, we'd do a "create database ..." with "create table ...". If they enter some data in and then decide they want to change a column we'd do an "alter table ....".
Now, the further along I get with building this out the more it seems like MySQL is poorly suited to handling this.
1) My first concern is that switching databases every request, first to our main app's database for authentication etc, and then to the user's personal database, is going to be inefficient.
2) The second concern I have is that there's going to be a limit to the number of databases a single MySQL server can host. Pretending for a moment this application had 500,000 user databases, is MySQL designed to operate this way? What if it were a million, or more?
3) Lastly, is this method going to be a nightmare to support and scale? I've never heard of MySQL being used in this way so I do worry about how this affects things like replication and other methods of scaling.
To me, it seems like MySQL wasn't built to be used in this way but what do I know. I've been looking at document-based databases like MongoDB, CouchDB, and Redis as alternatives because it seems like a schema-less approach to this particular problem makes a lot of sense.
Can anyone offer some advice on this?
Since you are leaving the schema up to your users to decide, it doesn't make sense using a relational database that forces you to define a schema.
Use a NoSQL database. Do some more reading on stack overflow.
What is NoSQL, how does it work, and what benefits does it provide?
Pros/Cons of document based database vs relational database
What is the best Document-oriented database?
Creating tables on the fly like you describe is a very bad idea. Supporting schema changes would be a nightmare. Each time someone added or removed a field you would have to run an ALTER TABLE ... command, and if there's data i the table, that's not a quick operation since it basically creates a new table with the new scehma and moves all the data over to the new one. Don't go down that route.
You could implement some kind of key/value-store on top of MySQL without too much work, or use something like Friendly, but going for a proper document database is probably a much simpler way.
MongoDB would be my choice, but there's a lot of things to consider, and others may say that Cassandra would be better. It's very easy to get going with MongoDB, and using it feels quite familiar to using a SQL database. It does indexing more or less identically, and querying is not too different either. The best thing though, is probably that you don't need an ORM, your objects are stored more or less as-is in the database. Reading and writing can be done very close to the metal without requiring a lot of mapping to and from objects.
I'm building a web app. This app will use MySQL to store all the information associated with each user. However, it will also use MySQL to store sys admin type stuff like error logs, event logs, various temporary tokens, etc. This second set of information will probably be larger than the first set, and it's not as important. If I lost all my error logs, the site would go on without a hiccup.
I am torn on whether to have multiple databases for these different types of information, or stuff it all into a single database, in multiple tables.
The reason to keep it all in one, is that I only have to open up one connection. I've noticed a measurable time penalty for connection opening, particularly using remote mysql servers.
What do you guys do?
Fisrt,i must say, i think storing all your event logs, error logs in db is a very bad idea, instead you may want to store them on the filesystem.
You will only need error logs or event logs if something in your web app goes unexpected. Then you download the file, and examine it, thats all. No need to store it on the db. It will slow down your db and your web app.
As an answer to your question, if you really want to do that, you should seperate them, and you should find a way to keep your page running even your event og and error log databases are loaded and responding slowly.
Going with two distinct database (one for your application's "core" data, and another one for "technical" data) might not be a bad idea, at least if you expect your application to have a lot of users :
it'll allow you to put one DB on one server, and the other DB on a second server
and you can think about scaling a bit more, later : more servers for the "core" data, and still only one for the "technical" data -- or the opposite
if the "technical" data is not as important, you can (more easily) have two distinct backup processes / policies
having two distinct databases, and two distinct servers, also means you can have heavy calculations on the technical data, without impacting the DB server that hosts the "core" data -- and those calculations can be useful, on logs, or stuff like that.
as a sidenote : if you don't need that kind of "reporting" calculations, maybe storing those data to a DB is not useful, and files would do perfectly ?
Maybe opening two connections means a bit more time -- but that difference is probably rather negligible, is it not ?
I've worked a couple of times on applications that would use two database :
One "master" / "write" database, that would be used only for writes
and one "slave" database (a replication of the first one, to several slave servers), that would be used for reads
This way, yes, we sometimes open two connections -- bu one server alone would not have been able to handle the load...
Use connection pooling anyway. So the time to get a connection is not a problem. But if you have 2 connections, transaction handling become more complicated. On the other hand, sometimes it's handy to have 2 connections: if something goes wrong on the business transaction, you can rollback transaction and still log the failure on the admin transaction. But I would still stick to one database.
I would only use one databse - mostly for the reason you supply: You only need one connection to reach both logging and user stored data.
Depending on your programming language, some frameworks (J2EE as an example) provide connection pooling. With two databases you would need two pools. In PHP on the other hand, the performance come in to perspective when setting up a connection (or two).
I see no reason for two databases. It'd be perfectly acceptable to have tables that are devoted to "technical" and "business"data, but the logical separation should be sufficient.
Physical separation doesn't seem necessary to me, unless you mean an application and data warehouse star schema. In that case, it's either real-time updates or, more typically, a nightly batch ETL.
It makes no difference to mysql in any way whether you use separate "datbases", they are simply catalogues.
It may make setting permissions easier, this is a legitimate reason to do it. Other than that, it is exactly the same as keeping the tables in the same db (except you can have several tables with the same name ... but please don't)
Putting them on separate servers might be a good idea however, as you probably don't want your core critical (user info, for example) data mixed in with your high-volume, unimportant data. This is particularly true for old audit data, debug logs etc.
Also short-lived data, such as search results, sessions etc, could be placed on a different server - it presumably has no high availability[1] requirement.
Having said that, if you don't need to do this, dump it all on one server where it's easier to manage (backup, provide high availibilty, manage security etc).
It is not generally possible to take a consistent snapshot of data on >1 server. This is a good reason to only have one (or one that you care about for backup purposes)
[1] Of the data, not the database.
In MySQL, InnoDB has an option of storing all tables of a certain database in one file, or having one file per table.
Having one file per table is somewhat recommended anyway, and if you do that, it makes difference on the database storage level if you have one database or several.
With connection pooling, one database or several is probably not going to matter either.
So, in my opinion, the question is if you'd ever consider separating the "other half" of the database to a separate server - with the separate server having perhaps a very different hardware configuration, such as no RAID. If so, consider using separate databases. If not, use a single database.