We are US-based and have a SaaS web app. One of our clients is in Canada and requires any personal data collected be stored only in Canada.
Is there a best-practice or standard way to handle this type of situation?
Thoughts so far (none of which area ideal):
Move the 'orders' table to a different Canada-based database server, and set it as the database for the 'Order' model. (But would that restrict us from doing JOINS...etc?)
Keep an entire second database server with a complete copy of the database in Canada, and use that for their app. (but this would be a pain when we want to update the database, make table changes, manage servers in general...etc)
I'm not incredibly knowledgeable about servers, so pardon if i used the wrong terminology anywhere in this question.
Related
I have an application in which we want to provide the functionality using which user can add/update/delete the columns of different tables. My approach is to create a different database for each client so that their changes specific to tables will remain in their database.
Since each client will have their own database, I wonder how can I manage authentication and authorization? Do I need to create a different database for that as well? Will it affect the performance of the application?
Edit: The approach that I am planning to use for authentication and authorization is to create an additional field called "Account" on the login page. This account name will guide the program to connect it to correct database. And each database will have it's own users to authenticate.
The answer to your question is of course (and unfortunately) Yes and No. :)
This is known as multi-tenant data architecture.
Having separate databases can definitely be a great design option however so can using one database shared with all of your clients/customers and you will need to consider many factors before choosing.
Each design has pluses and minuses.
Here are your 3 essential choices
1) Each customer shares the same database and database tables.
2) Each customer shares the same database but they get their own schema inside the database so they each get their own set of tables.
3)Each customer gets their own database.
One major benefit (that I really like) to the separate database approach is data security. What I mean by this is that every customer gets their own database and because of this they will edit/update/delete just their database. Because of this, there is no risk in end users overriding other users data either due to programmatic error on your part or due to a security breach in your application.
When all users are in the same database you could accidentally pull and expose another customers data. Or, worse, you could expose a primary key to a record on screen and forget to secure it appropriately and a power user could override this key very easily to a key that belongs to another customer thus exposing another clients data.
However, lets say that all of your customers are actually subsidieries of 1 large company and you need to roll up financials every day/week/month/year etc.
If this is the case, then having a database for every client could be a reporting nightmare and having everyone in a single database sharing tables would just make life so much easier. When it comes time to report on your daily sales for instance, its easier to just sum up a column then go to 10,000 databases and sum them up. :)
So the answer definitely depends on your applicaton and what it will be doing.
I work on a large enterprise system where we have tens of thousands of clients in the same database and in order to support this we took very great care to secure all of our data very carefully.
I also work on a side project in my spare time which supports a database per customer multi-tenant architecture.
So, consider what your application will do, how you will backup your data, do you need to roll up data etc and this will help you decide.
Heres a grea article on MSDN for this:
https://msdn.microsoft.com/en-us/library/aa479086.aspx
Regarding your question about authentication.
Yes, having a separate database for authentication is a great design. When a customer authenticates, you will authenticate them off of your authentication database and they will receive the connectionstring to their database as part of this authentication. Then all data from that point comes from that clients database.
Hope this was helpful.
Good luck!
Little question, I'm developing a saas software (erp).
I designed it with 1 database per account for these reasons :
I make a lot of personalisation, and need to add specific table columns for each account.
Easier to manage db backup (and reload data !)
Less risky : sometimes I need to run SQL queries on a table, in case of an error with bad query (update / delete...), only one customer is affected instead of all of them.
Bas point : I'm turning to have hundreds of databases...
I'm hiring a company to manage my servers, and they said that it's better to have only one database, with a few tables, and put all data in the same tables with column as id_account. I'm very very surprised by these words, so I'm wondering... what are your ideas ?
Thanks !
Frederic
The current environment I am working in, we handle millions of records from numerous clients. Our solution is to use Schema to segregate each individual client. A schema allows you to partition your clients into separate virtual databases while inside a single db. Each schema will have an exact copy of the tables from your application.
The upside:
Segregated client data
data from a single client can be easily backed up, exported or deleted
Programming is still the same, but you have to select the schema before db calls
Moving clients to another db or standalone server is a lot easier
adding specific tables per client is easier (see below)
single instance of the database running
tuning the db affects all tenants
The downside:
Unless you manage your shared schema properly, you may duplicate data
Migrations are repeated for every schema
You have to remember to select the schema before db calls
hard pressed to add many negatives... I guess I may be biased.
Adding Specific Tables: Why would you add client specific tables if this is SAAS and not custom software? Better to use a Postgres DB with a Hstore field and store as much searchable data as you like.
Schemas are ideal for multi-tenant databases Link Link
A lot of what I am telling you depends on your software stack, the capabilities of your developers and the backend db you selected (all of which you neglected to mention)
Your hardware guys should not decide your software architecture. If they do, you are likely shooting yourself in the leg before you even get out of the gate. Get a good senior software architect, the grief they will save you, will likely save your business.
I hope this helps...
Bonne Chance
I'll be soon developing a big cms where users can configure their website managing news, products, services and much more about their company.
Think about a shopify without the ecommerce part (at least for now).
The rdbms is MySQL and the user base will be about 150 (maybe bigger).
I'm trying to figure out which one of these two approaches would fit better.
DEDICATED DATABASE FOR EACH USER
PROS:
performance (and possible future sharding?): is querying smaller database with just your data better than querying a giant database with every user data?
easy "export my data" for users: I can simply dump their own db without fetching everything and putting it in some big encoded logical datastruct
SINGLE DATABASE FOR EVERY USER
PROS:
less general overhead
statistic: just one db to query to get and aggregate whatever I need
backup: one dump (not sure about this one because I've no experience in cluster dumping)
Which way would you go for? I don't think shopify created a dedicated database for any user registered... or maybe they did?
I'd like more experienced people than me to help me figuring out the best way and all the variables I can not guess right now because of my ignorance.
It sounds like you're developing a software-as-a-service hosted system, rather than a software package to distribute to customers for them to run on their own servers. In that case, in general, you will have an easier time developing and administering your service if you design it for a single database handling multiple users.
You'll be able to add new users to your system with data manipulation language (DML) rather than data definition language (DDL). That is, you'll insert rows for new users rather than create tables. That will make your life a LOT easier when you go live.
You are absolutely right that stuff like backups and aggregate reporting will be far easier if you have a single shared database.
Don't worry too much about the user data export functions. You'll have to develop software for those functions anyway; it won't be that hard to filter by user when you do the export.
But there's a downside you should consider to the single-database approach: if part of your requirement is to conceal various users' existence or data from each other, you'll have to be very careful to do this in your development. Will your users be competitors with each other? That could be tricky. You'll need to trust your in-house admin and support teams to refrain from disclosing one user's data to another by mistake (or deliberately). With a separate database per user, you'll have a smaller risk in that area.
150 users aren't many. Don't worry about scalability until you have a workload of paying customers. When that happens you can add MySQL server RAM, partitions, solid-state disks, replication, memcached, sharding, and all that other expensive and high-workload stuff. If you add those things before you go live, you'll just take longer and blow more money before you go live. Not good.
I am working on a web application that will require users to have their own set of private data. My original plan was to create a stores table, a users table, and a user_stores intersecting table. Then I would, in the stores table, save the database name for that store (and create each store-specific database with an application user and password so the web application could always login).
Each store would have similar data (users, products, shipping methods, etc), and I know I can use foreign key references to tie everything together in one giant database. However, being that the data is very specific and potentially proprietary, would it be better to use my original design, or make a single database with everyone's data in there?
I am thinking for scaling concerns, separate databases would be better because we could put the more active accounts on their own (or more powerful) database servers and simply add a server location field in the stores table if we needed to. Additionally, it may be more secure because we could make add the user login information to the database and only give them access to their data (preventing one user from editing another user's stuff). My question is, are there concerns that I am missing though? Just about every post I have read about this says not to use the method I am thinking of, and I am no DBA. Any input would be helpful.
Additional Information:
This will be hosted on a Dedicated Server that I will have root access to. I can create as many MySQL databases as I need to.
I would use a single database for sure. Use the following to get started. There are several reasons to go with a single db, however the biggest reason of all is to save you from a maintenance nightmare. If you have to change the schema, you will have a mess on your hands.
http://msdn.microsoft.com/en-us/library/aa479086.aspx
In a multi-tenant database, database designers think about querying, cost, data isolation and protection, maintenance, and disaster recovery.
Multi-tenant solutions range from one database per tenant ("shared nothing") to one row per tenant ("shared everything"). This SO answer summarizes the tradeoffs. If you're designing a database that falls under some kind of regulatory environment (HIPAA, FERPA, etc.), that regulatory environment might trump all other considerations.
One database per tenant is a defensible decision in some cases. It's not clear whether that's the best answer in your case, though.
I'm building up an online (paid) service used for business administration purposes. The database is structured like so:
I have a contacts table filled with persons, contact info and the like. Then I have a few other tables holding information about payments, agreements and appointments. Also statistics like how much money was transferred this month, how many hours worth of appointments this month and the like.
I'm using MySQL (but could also go for MSSQL or some other service if necessary) and I had no formal training in any programming language whatsoever (yet).
I'm building a WPF application for acces to this database. Also planning on building an app so users can access their data and plan new appointments and register payments on the go.
I'm going to go for a login system to verify their right to login and use my service.
My question is about how to structure this. I'm not an SQL expert nor have I had any formal training in SQL or any other programming language. What I do know though is that my client-side app is almost out of the alpha stage.
So far I have come up with two ways to structure this.
1. Users get a seperate database.
My original idea was to give each user a seperate database, this makes it easier to provide people with statistics. Also it makes it easier to spread the workload through multiple, seperate servers. People would login to a master/main server, where their login information is stored, fetch their server info and programatically be 'redirected' to their own database. Spreading these databases also make it easier to provide individual back-ups to users.
The down-side of this is the sheer quantity of databases I'd have to manage. I'm planning on ending up with hundreds of thousands of users. Let's just say I want the system to be able to provide to an infinite amount of users.
2. Everything is stored in one database.
It's also possible to store everything in one database. This would make the database structure somewhat more complicated (while it also makes the whole a lot simpler). I'd have to add 'AND consumer_ID='" + MyID + "' to every query. (Which ofcourse is possible) and add a few tables to handle statistics per user.
It would be simpler to provide every user with the same database updates. Maintenance would be easier.
The down-side of this is that it makes it harder to spread the workload to seperate servers, I'd have to build something to make it possible that seperate servers mirror each other. Also I'd have to make sure that the workload is automatically divided between the servers, instead of simply going for: Fill server with X databases, then new server, fill, new etc.
I'm not in the luxury of hiring someone with any SQL training.
The most important thing for me now is that the system can be easily maintained while still being safe and reliable. I'm an amateur developer, going to college next year. I don't want to spend 50% of my time maintaining the database.
I think I got the major part of the details you might need, if you need anymore please ask for them.
I thank you in advance :)
Just go with solution 2. The downside of spreading the workload to many servers is fullfilled by "partitioning", look here for a starting point: http://dev.mysql.com/doc/refman/5.1/en/partitioning-overview.html
Partitioning would allow you for example to put all information of a table containing even IDs for consumers on the one, all other on the second server. Or whatever you want...
But i wouldn't start that complicated: do you need that now? It burdens you (either way) with such a big additional overhead! You can also look into the NoSQL database world for solutions that can be spread to as many servers as you want with low effort. You loose SQL and it's ACID features in the most cases; if you need those NoSQL is not an option.