Does it matter how many databases I use in my web system? I am planning to have:
User information and related tables
Admin tables and all system tables
Reporting system
Audit logs of tables
User object tables like Photos, Videos, Comments
User API applications to read/write data.
Questions:
I am using MySQL and MagnoDB with cakephp. So if i implement above then i will use 6 databases in the system. Add backups so 2 of each then 12 databases total. Any advantage / disadvantage this way vs dumping all tables into 1 database? I assume thesedays with sites like yahoo, amazon, facebook, etc having hundreds or thousands of databases is the norm OR are these all powered by 1 database but having multiple instances?
For lookup tables: Do i duplicate them in each database or 1 copy in the admin database is good enough?
Also if i have multiple instance of the same DB do i need to name them like DB1, DB2, DB3 or can i call them anything?
We are developing a local reviews website so expect lots of users eventually.
Everything that has relative information should be in the same DB.
So if all the things you have mentioned have relative info you would need probably 3 DBs: dev, prod, backup.
If some of that info is not related to anything else, that it should be in the separate DB.
As a developer, I always create a new DB for each new unrelated project. Otherwise, you create / add new features to the existing DB.
Use a single database. The problem with using multiple DBs include distributed queries (as pointed out already), plus overhead associated with each db server / instance, and general maintenance complexity.
What you want are tablespaces http://dev.mysql.com/doc/refman/5.1/en/create-tablespace.html
Consider with dbs like Oracle, the overhead per instance is 300-500mb+. Not to mention a new set of processes and separate buffer caches. You want a single, unified buffer cache to make the most of your RAM.
Partitioning using a database as the partition unit isn't saving you much, but will make a giant headache. MySQL can handle huge amounts of data (terabytes), as long as you design your schemas well and tune the storage. And use separate tablespaces.
Moving your app, and backup / restore should be simple too.
The only reason I create separate DBs is if there are multiple customers involved, or the project requires it. But it is usually not required from a technical point of view.
disadvantages: trying to do joins across multiple database
Any area that you might need to relate to another area should all be in the same db.
Related
I am developing a site that has many subdomains in it.
It has blogging module, management system, and many more. I have shared this question in various sites but couldn't get a proper reply.
Question is should I use one database for all the modules, this means my database would have nearly 100 tables. Is this approach be appropriate or should I create separate database for every module?
Well, it does not really matter.
If you use innodb with single data file (innodb_file_per_table setting is not enabled), then all data will be stored in a single file anyway.
With innodb separate file per table mode or with myisam table engine, the only difference between one or multiple databases is really the directory where the database files are stored. Unless the directories (databases) are located in different storage devices with different speeds, their performance will be the same.
There can be 2 reasons to keep some tables in a different database:
Security: mysql does not support role based access control. Therefore if there is a group of tables that should be accessible by a certain group of users only, then the access control is more manageable if those tables are in a different database.
If some of the modules you mentioned happen to use the same table name, then you will have to move them to a separate database or you need to modify the code and table names to avoid errors.
There is no right or wrong way to design a system. Just advantages and disadvantages to the various techniques. I normally work in Oracle and SQL Server so I had to look up some terms for MySQL. According to my research, in MySQL a database is synonymous with a schema which changes things. I'd consider these things when planning the physical design for any vendor:
Security - Do all subdomains need read/write to each other? How are the users secured? Choosing one or many schemas can impact how easy schema and user security is to manage and control.
Growth - Do some subdomains grow at a faster rate than others? If yes, I'd consider separating them to allow for the different growth rates.
Organization - Is it easier to identify the different subdomains in practice if they're separated? If you don't separate them, use a strong naming convention so you can easily identify objects within one subdomain.
Linking - How easy is it to access one schema/database from another?
Hope this helps.
Little question, I'm developing a saas software (erp).
I designed it with 1 database per account for these reasons :
I make a lot of personalisation, and need to add specific table columns for each account.
Easier to manage db backup (and reload data !)
Less risky : sometimes I need to run SQL queries on a table, in case of an error with bad query (update / delete...), only one customer is affected instead of all of them.
Bas point : I'm turning to have hundreds of databases...
I'm hiring a company to manage my servers, and they said that it's better to have only one database, with a few tables, and put all data in the same tables with column as id_account. I'm very very surprised by these words, so I'm wondering... what are your ideas ?
Thanks !
Frederic
The current environment I am working in, we handle millions of records from numerous clients. Our solution is to use Schema to segregate each individual client. A schema allows you to partition your clients into separate virtual databases while inside a single db. Each schema will have an exact copy of the tables from your application.
The upside:
Segregated client data
data from a single client can be easily backed up, exported or deleted
Programming is still the same, but you have to select the schema before db calls
Moving clients to another db or standalone server is a lot easier
adding specific tables per client is easier (see below)
single instance of the database running
tuning the db affects all tenants
The downside:
Unless you manage your shared schema properly, you may duplicate data
Migrations are repeated for every schema
You have to remember to select the schema before db calls
hard pressed to add many negatives... I guess I may be biased.
Adding Specific Tables: Why would you add client specific tables if this is SAAS and not custom software? Better to use a Postgres DB with a Hstore field and store as much searchable data as you like.
Schemas are ideal for multi-tenant databases Link Link
A lot of what I am telling you depends on your software stack, the capabilities of your developers and the backend db you selected (all of which you neglected to mention)
Your hardware guys should not decide your software architecture. If they do, you are likely shooting yourself in the leg before you even get out of the gate. Get a good senior software architect, the grief they will save you, will likely save your business.
I hope this helps...
Bonne Chance
I plan to create a websystem where organisations (customers) can setup a website with particular functions and their information stored in a (mysql) database. I already started with a database design which includes one master database plus a customer database for each new organisation (which is created after filling in a webform).
Now I start to question my database design decision and wonder whether just single database for all organisations would not be a better choice ? The reason being that there will be various communication (==information exchange) between some of these organisations and these communicating organisations would have unnecessary copies of some tables (e.g., they both have 2-3 tables which are almost copies of each other and therefore they could also share).
Furthermore, implementing the information exchange seems a bit more complex with various databases than with one database. On the other hand, I assume(d) that database queries by the various customers within a single very large database may require much more time than with a system with multiple databases.
Bottomline, as a non-database expert I'm not sure which of the two options would be best to proceed with, and therefore would appreciate your advice.
Go with one database. If you are successful and you have thousands of users on the system you are going to need a army of database administrators to look after the various systems. For that reason alone I would avoid having multiple databases.
I have a 2GB Digital Ocean VPS with 2 CPUs, to host a social network app written in Java. Right now my app stores data to Cassandra, but Cassandra is a new technology & not as reliable as MySQL that has been for years, also my experience in managing Cassandra as a DBA is not much. So I wanted to change my primary datasource back to MySQL but since some of the data is stored just schemaless, for e.g. there are lists specific to each user that are easily stored in Cassandra. For this type of data, I would use Cassandra as primary database.
So, to sum up, I would replicate my entire data in both the databases. Data will be written to both databases but read from where I can get it most performantly. This will help me in case when the entire Cassandra cluster goes down I can serve from mysql or vice versa. Is this usually done & recommended to do ?
(Right now I have a single 2 GB VPS that would host my app as well as the databases)
Normally we never see that people managing two separate database system just for purpose of data lost and recovery, always better to rely upon replication or mirroring of any of database system. Both are good and providing enough solution for replication so better you will choose any single one of them.
Is it because of size?
By the way, what is the limit of the size?
There are many reasons to use two databases, some that come to mind:
Size (the limit of which is controlled by the operating system, filesystem, and database server)
Separation of types of data. Think of a database like a book -- you wouldn't write a book that spans multiple subjects, and you shouldn't (necessarily) have a database with multiple subjects. Just so all of the data is somehow related, you could keep it together (i.e. all the tables have something to do with one website or application).
Import / Export - it might be easier to import data into your application if you can drop and restore a whole database, rather than import individual rows into a database table.
Separate applications, or services. I can't see any reason to use separate databases for a single app/service.
(note: replication, even multimaster, isn't a separate database. Neither is Sharding.)
I believe some on here are confusing Database with a Database Instance.
Example:
A phone book is a prime example of a Database.
Replication:
having 2 copies of the same phone book does not mean you have 2 databases. It means you have 2 copies of 1 database, and that you can hand 1 to someone else so you can both look up different things at the same time thereby accomplishing more work at once.
Sharding:
You could tear those phonebooks apart at the end of the white pages and the beginning of the yellow and hand them to 2 more people. You could further tear them at each letter and when you need susan summers ask the person with that section of the book to look for her.
suppose you wanted to publish or reuse some external database, and keep it separate from your primary database. This would be a good reason to use 2 databases... You can drop and reimport the external database at any time without affecting your database, and vice versa...
You can use two databases the same reason most banks have two ATMs, for reliability. You can swap one in if the other fails, but to do it quickly requires setup, such as a cname and controlling your own DNS server.
You can also do writes on one database, if the writes have complex triggers on them, and use some synching between databases to keep the second one updates, which is used for selects.
You can use two databases for load sharing, for example, use round-robin to split up the load so one isn't overloaded.
I sometimes have separate database because they handle different concerns. I.E. a Reporting database or an Authentication Database.
Replication
Making your system scalable by devide your database system to different physical location
Provide redundancy/replication as backup and seamless uptime.a
As Ben mentioned, Replication is one reason. Another is load balancing.
For example, Hotmail uses many database servers and customer data is broken up across the databases.
To have all of their customers' data on one server would not only require massive storage requirements, but the response times would be horrible.
In other cases, the data may be separated by function. You may well have two sets of data which are either not connected, or at least very loosely so, and in such cases, it may make sense to separate that data from the rest.
Also consider IO needs. Writing to one, reading from the other. One with immediate transactional needs, others where "transactions" can be queued, one instance at high priority, the other at "idle" priority, &c. It is very obvious however with the correct hardware and tablespace/filesystem layouts most of these situations can be achieved in a singular DB.
I think SQLite databases on the iPhone is limited to a size of 50 megabytes, but you can open several databases.