While building my app, I came across a problem. I have some database tables with information, I want to reuse for different applications. Mainly for authentication and user privileges.
That is why i decided to split my database into two, one for user data (data I will need for other applications) and another for application related data (data I will need only for this).
In some cases, I need to reference a foreign key from one database on another database. I had no problem doing so while databases are in the same connection. I did it like so:
CREATE TABLE `database1`.`table1` (
`id` int(10) unsigned NOT NULL AUTO_INCREMENT,
`foreign_key` int(10) unsigned DEFAULT NULL,
PRIMARY KEY (`id`),
KEY `table1_foreign_key_foreign` (`foreign_key`),
CONSTRAINT `table1_foreign_key_foreign` FOREIGN KEY (`foreign_key`) REFERENCES `database2`.`table2` (`id`)
);
Now here is my problem. I am getting to know Docker and I would like to create a container for each database. If my understanding is correct, each container acts as a different connection.
Is it even possible to reference a foreign key on different database connection?
Is there another way of referencing a foreign key from one Docker container on another?
Any suggestions or comments would be much appreciated.
Having a foreign key cross database boundaries is a bad idea for multiple reasons.
Scaling out: You are tying the databases to the same instance. Moving a database to a new instance becomes much more complicated, and you definitely do not want to end up with a FK constraint running over a linked server. Please, no. Don't.
Disaster Recovery: Your DR process has a significant risk. Are your backups capturing the data at the exact same point in time? If not, there is the risk that the related data will not match after a restore. Even a difference of a few seconds can invalidate the integrity of the relationship.
Different subsystems: Each database requires resources. Some are explicit, others are shared, but there is overhead for each database running in your instance.
Security: Each database has its own security implementation. Different logins and access permissions. If a user in your DATA database needs to lookup a value against the USER database, you'll need to manage permissions in both. Segregating the data by database doesn't solve or enhance your security, it just makes it more complicated. The overhead to manage the security for the sensitive data doesn't change, you'll still need to review and manage users and permissions based on the data (not the location of the data). You should be able to implement exactly the same security controls within the single database.
No, that is not possible. You can not create FK to different instance of DB (or other Docker container in your case).
You may try to make this check on application level.
Related
Say I have two tables, Address and Person, which resides on two totally separate servers. I want to refer to Address in Person by a foreign key.
I want to put them in separate servers, because they are with the databases that perform jobs closer to each of them. e.g. the DB person resides in may define a lot of tables that relate to Person, but none of them except Person relate to Address, and vice versa. Failing of the server Person resides on will have no impact on the Address's server, unless the foreign key is involved in the action.
If this makes sense, how can I do that in mysql? And how can I do that in java using hibernate?
No, but you can have trigger to ensure foreign key constraint.
A ActiveRecord::UnknownPrimaryKey occurred in survey_response#create:
Unknown primary key for table question_responses in model QuestionResponse.
activerecord (3.2.8) lib/active_record/reflection.rb:366:in `primary_key'
Our application has been raising these exceptions and we do not know what is causing them. The exception happens in both production and test environments, but it is not reproducible in either. It seems to have some relation to server load, but even in times of peak loads some of the requests still complete successfully. The app (both production and test environments) is Rails 3.2.8, ruby 1.9.3-p194 using MySQL with the mysql2 gem. Production is Ubuntu and dev/test is OS X. The app is running under Phusion Passenger in production.
Here is a sample stack trace: https://gist.github.com/4068400
Here are the two models in question, the controller and the output of "desc question_responses;": https://gist.github.com/4b3667a6896b60383dc3
It most definitely has a primary key, which is a standard rails 'id' column.
Restarting the app server temporarily stops the exceptions from occurring, otherwise they happen over a period of time 30 minutes - 6 hours in length, starting as suddenly as they stop.
It always occurs on the same controller action, table and model.
Has anyone else run into this exception?
FWIW, I was getting this same intermittent error and after a heck of a lot of head-scratching I found the cause.
We have separate DBs per client, and some how one of the client's DBs had a missing primary key on the users table. This meant that when that client accessed our site, Rails updated it's in-memory schema to that of the database it had connected to, with the missing primary key. Any future requests served by that Passenger app process (or any others that had been 'infected' by this client) which tried to access the users table borked out with the primary key error, regardless of whether that particular database had a primary key.
In the end a fairly self-explanatory error, but difficult to pin down when you've got 500+ databases and only one of them was causing the problem, and it was intermittent.
Got this problem because my workers used shared connection to database. But I was on unicorn.
I know that Passenger reconnects by default, but maybe you have some complicated logic. Connections to number of databases, for example. So you need to reconnect all connections.
This same thing happened to me. I had a composite primary key in one of my table definitions which caused the error. It was further compounded because annotate models did not (but will shortly / does now) support annotation of composite primary keys.
My solution was to make the id column the only primary key and add a constraint (not shown) for the composition. To do this you need to drop auto_increment on id if this is set, drop your composite primary key, then re-add both the primary status and autoincrement.
ALTER TABLE indices MODIFY id INT NOT NULL;
ALTER TABLE indices DROP PRIMARY KEY;
ALTER TABLE indices MODIFY id INT NOT NULL PRIMARY KEY AUTO_INCREMENT;
on postgres database
ALTER TABLE indices ALTER COLUMN id SET DATA TYPE INT;
ALTER TABLE indices ADD PRIMARY KEY (id)
How can I add a constraint that references a foreign column from another database?
Some time ago I read that it can be done with linked server and others say with triggers. What's the preferred way of doing this if that's possible at all?
Thanks!
Linked servers will not work.
FK's must point to local tables.
The preferred way is not to do this, though you can pull a few hacks to make it happen.
For example you could have triggers cause the reference to be created/checked but I wouldn't consider that equivalent to a FK constraint.
Related question:
Can you have a Foreign Key onto a View of a Linked Server table in SQLServer 2k5?
Use an instead of trigger (you may or may not need linked servers depending on whether the other database is on a differnt server). Make sure the trigger can handle multiple row inserts/updates/deletes. I'd also suggest moving the records that fail the check to an exception table.
Foreign keys cannot go across database boundaries. If you try to do this, you'll get:
Msg 1763, Level 16, State 0, Line 1
Cross-database foreign key
references are not supported. Foreign key ***
Msg 1750, Level 16, State 0, Line 1
Could not create constraint. See previous errors.
If you need to enforce some kind of relationship between two separate databases, then yes - you might need a linked server (if that second database is on a second server), and possibly triggers - but all of those things will be very hard to get right, very inefficient and very error prone.
One way you might be able to do this would be data replication - replicate the table you want to reference into your source database, and then establish a foreign key relationship with that replicated table. But that will never be quite "live" and "real-time" - there will also be a bit of a lag in the data replication.
I'm designing my database schema using MySQL Workbench, which is pretty cool because you can do diagrams and it converts them :P
Anyways, I've decided to use InnoDB because of it's Foreign Key support. One thing I noticed though is that it allows you to set On Update and on Delete options for foreign keys. Can someone explain where "Restrict", "Cascade" and set null could be used in a simple example?
For example, say I have a user table which includes a userID. And say I have a message table message which is a many-to-many which has 2 foreign keys (which reference the same primary key, userID in the user table). Is setting the On Update and On Delete options any useful in this case? If so, which one do I choose? If this isn't a good example, could you please come up with a good example to illustrate how these could be useful?
Thanks
Do not hesitate to put constraints on the database. You'll be sure to have a consistent database, and that's one of the good reasons to use a database. Especially if you have several applications requesting it (or just one application but with a direct mode and a batch mode using different sources).
With MySQL you do not have advanced constraints like you would have in postgreSQL but at least the foreign key constraints are quite advanced.
We'll take an example, a company table with a user table containing people from theses company
CREATE TABLE COMPANY (
company_id INT NOT NULL,
company_name VARCHAR(50),
PRIMARY KEY (company_id)
) ENGINE=INNODB;
CREATE TABLE USER (
user_id INT,
user_name VARCHAR(50),
company_id INT,
INDEX company_id_idx (company_id),
FOREIGN KEY (company_id) REFERENCES COMPANY (company_id) ON...
) ENGINE=INNODB;
Let's look at the ON UPDATE clause:
ON UPDATE RESTRICT : the default : if you try to update a company_id in table COMPANY the engine will reject the operation if one USER at least links on this company.
ON UPDATE NO ACTION : same as RESTRICT.
ON UPDATE CASCADE : the best one usually : if you update a company_id in a row of table COMPANY the engine will update it accordingly on all USER rows referencing this COMPANY (but no triggers activated on USER table, warning). The engine will track the changes for you, it's good.
ON UPDATE SET NULL : if you update a company_id in a row of table COMPANY the engine will set related USERs company_id to NULL (should be available in USER company_id field). I cannot see any interesting thing to do with that on an update, but I may be wrong.
And now on the ON DELETE side:
ON DELETE RESTRICT : the default : if you try to delete a company_id Id in table COMPANY the engine will reject the operation if one USER at least links on this company, can save your life.
ON DELETE NO ACTION : same as RESTRICT
ON DELETE CASCADE : dangerous : if you delete a company row in table COMPANY the engine will delete as well the related USERs. This is dangerous but can be used to make automatic cleanups on secondary tables (so it can be something you want, but quite certainly not for a COMPANY<->USER example)
ON DELETE SET NULL : handful : if you delete a COMPANY row the related USERs will automatically have the relationship to NULL. If Null is your value for users with no company this can be a good behavior, for example maybe you need to keep the users in your application, as authors of some content, but removing the company is not a problem for you.
usually my default is: ON DELETE RESTRICT ON UPDATE CASCADE. with some ON DELETE CASCADE for track tables (logs--not all logs--, things like that) and ON DELETE SET NULL when the master table is a 'simple attribute' for the table containing the foreign key, like a JOB table for the USER table.
Edit
It's been a long time since I wrote that. Now I think I should add one important warning. MySQL has one big documented limitation with cascades. Cascades are not firing triggers. So if you were over confident enough in that engine to use triggers you should avoid cascades constraints.
http://dev.mysql.com/doc/refman/5.6/en/triggers.html
MySQL triggers activate only for changes made to tables by SQL statements. They do not activate for changes in views, nor by changes to tables made by APIs that do not transmit SQL statements to the MySQL Server
http://dev.mysql.com/doc/refman/5.6/en/stored-program-restrictions.html#stored-routines-trigger-restrictions
==> See below the last edit, things are moving on this domain
Triggers are not activated by foreign key actions.
And I do not think this will get fixed one day. Foreign key constraints are managed by the InnoDb storage and Triggers are managed by the MySQL SQL engine. Both are separated. Innodb is the only storage with constraint management, maybe they'll add triggers directly in the storage engine one day, maybe not.
But I have my own opinion on which element you should choose between the poor trigger implementation and the very useful foreign keys constraints support. And once you'll get used to database consistency you'll love PostgreSQL.
12/2017-Updating this Edit about MySQL:
as stated by #IstiaqueAhmed in the comments, the situation has changed on this subject. So follow the link and check the real up-to-date situation (which may change again in the future).
You'll need to consider this in context of the application. In general, you should design an application, not a database (the database simply being part of the application).
Consider how your application should respond to various cases.
The default action is to restrict (i.e. not permit) the operation, which is normally what you want as it prevents stupid programming errors. However, on DELETE CASCADE can also be useful. It really depends on your application and how you intend to delete particular objects.
Personally, I'd use InnoDB because it doesn't trash your data (c.f. MyISAM, which does), rather than because it has FK constraints.
Addition to #MarkR answer - one thing to note would be that many PHP frameworks with ORMs would not recognize or use advanced DB setup (foreign keys, cascading delete, unique constraints), and this may result in unexpected behaviour.
For example if you delete a record using ORM, and your DELETE CASCADE will delete records in related tables, ORM's attempt to delete these related records (often automatic) will result in error.
three servers
server 1 is central point which links the other two servers.
the other two servers have a table which has a field which should act like a foreign key
problem
well i do not know how to do this, using vs08 or sql server 08
diagram view
table on server 1
sv1pg1 id -- primary key
details
tables same schema on srvr2 and srvr3
linkedSRVid -- pk
linkedto -- fk constrain should be between (server 1 primary key and this field)
note that
"this is just a simplest! way i can think of to represent the prob, the real dbs contain
many fields and tables and the following applies"
it's not always that the network connection between the server remains up.
the link only need for "writing purpose" not viewing, as viewing done by direct
connection to the respective server.
A foreign key constraint can reference other tables only within the same database. This means that even if those databases were on the same server it still would not work.
Considering that the schemas are the same, you may want to look into replication.