We have two instances of a rails application which each talk to their own database; we're in the process of converting them to a single application with a single database. We've already made the parts of it which need to be specific to a particular domain work correctly; now we just need to merge the databases. We're going to copy the data from one instance into the other's database, and fix up the IDs so they don't overlap. There are a lot of tables with a lot of foreign keys. What's a good way of doing this, such that the foreign keys still point to the correct row in the new database?
If this is unclear, I'm happy to complicate matters with bad ascii art.
Most relational databases let you annotate a foreign key to be constrained to watch for when the primary key in the pointed-to table changes. You can set the foreign key to be "auto-updated" when this happens using ON UPDATE CASCADE. Do this for all foreign keys in both databases, then update all primary keys in both databases, and all foreign keys will be automatically converted.
How about updating every id column (including the foreign keys) to be its original value times 10, then add 1 for the first database and 2 for the second database.
That way id 1 becomes 11 on db 1 and 12 on db 2. Since both the primary and foreign keys go through the same change, you don't have to worry about how the records relate, you just make the updates with the same formula.
So it would go something like
On db 1:
UPDATE user SET id = id * 10 + 1;
UPDATE privilege SET id = id * 10 + 1, user_id = user_id * 10 + 1;
On db 2:
UPDATE user SET id = id * 10 + 2;
UPDATE privilege SET id = id * 10 + 2, user_id = user_id * 10 + 2;
Related
Good day
I create database at localhost for website. and put some info, than i delete and re-enter info from database. and now for 'id' primary key i have more than 200 rows. I want to re-arrange primary key.
for example
id |name
1 |Samuel
2 |Smith
4 |Gorge
15 |Adam
19 |David
i want to have
id |name
1 |Samuel
2 |Smith
3 |Gorge
4 |Adam
5 |David
Is it possible to do with any command?
You could drop the primary key column and re-create it. All the ids will then be reassigned, I assume in the order in which the rows were inserted.
alter table your_table drop column id;
then to create it
ALTER TABLE `your_table_name` ADD `id` INT NOT NULL AUTO_INCREMENT PRIMARY KEY FIRST;
The purpose of a primary key is to uniquely identify each row, so rows in one table can be related to rows in another table. Remember, this is a relational database and part of the meaning of "relational" is that entities are related to each other.
In other words, you don't want to change the primary key of rows, because that will break links from other tables. MySQL does not guarantee that auto incremented values are inserted without holes. In fact, as you have discovered, deletions and re-inserts cause problems.
Your interpretation of the "primary key" as a sequential number with no gaps assigned to each row maintained by the database is simply not correct.
Even though you don't want to do this, you can. I advise against it, but you can:
declare #rn := 0;
update t
set id = (#rn := #rn + 1)
order by id;
If you want to enforce this over time, you will need to learn about triggers.
Consider this scenario: Gorge sends some offensive emails, and people complain and his account (#4) is denylisted.
Then you reorder your primary key values, and Adam is now assigned id 4. Suddenly, he finds himself banned! And lots of people mistrust him without cause.
Primary keys are not required to be consecutive -- they're only required to be unique. It's normal for there to be gaps, if you sometimes ROLLBACK transactions, or DELETE rows.
Most likely the primary key is being auto generated from some sort of auto increment sequence. In that case you can take the following steps:
1) update all the primary keys to the next value of the sequence: this will collapse all of the values into a contiguous range. In your case those ids will be 20, 21, 22, 23, 24. Postgres example:
UPDATE my_table SET id = nextval(my_table_id_sequence)
2) reset the sequence to start at 1: In Postgres this would look like the following:
ALTER SEQUENCE my_table_id_sequence RESTART WITH 1
3) update the values to the next value of the sequence again: Now can move all the rows back "down" to start at 1, and in your case they will be 1, 2, 3, 4, 5. It is important to first consolidate all the values at the "top" of the sequence before resetting, because that way we guarantee that there wont be any primary key collisions at the "bottom"
UPDATE my_table SET id = nextval(my_table_id_sequence)
NOTE: this approach only works if there are no foriegn keys which are referring to the primary key of the table. If there are foreign keys you can still take the same approach, but first do these 3 steps:
1) find all of the related tables/columns that are referencing this primary key column
2) create a function that will cascade updates to the pk out to all fks
3) create a trigger that will execute the above function whenever the pk is updated: at this point, when we update the primary key column, all of the related foreign keys will also be updated. Depending on the database, you might need to explicitly defer constraint validation, or do the whole thing in one transaction.
For an example of what the above might look like in Postgres you can take a look at my answer here How Do I Deep Copy a Set of Data, and Change FK References to Point to All the Copies?
I am having my application deployed on two separate regions say US-WEST and EU, both application has its own DB. And now I want to move the EU region DB to US-WEST.
This will lead to primary key collision since both the db has the tables with same primary auto increment id, can anybody give me suggestion to solve this.
Scenario:
User Table from DB1(say from US-WEST) has the following entries
ID Name
1 Rob
2 San
3 Tulip
User Table from DB2(say from EU) has the following entries
ID Name
1 John
2 Michael
3 Natasha
For every one of the two original databases (say db0 and db1):
Back up the db.
Lock database for use by this script only.
For all the tables in the database that have foreign keys defined without ON UPDATE CASCADE, change all these foreign keys constraints with this option.
For every table with an auto_increment (or a simple integer) Primary Key, run this (the cascading updates will make the rest):
.
UPDATE TableX
SET Pk = 2 * Pk - 0 --- for db0
ORDER BY Pk DESC
UPDATE TableX
SET Pk = 2 * Pk - 1 --- for db1
ORDER BY Pk DESC
Export the tables from each database.
Now merge the two databases by simply merging the corresponding tables. All data from db0 will have even ids and all from db1 will have odd ids. No collisions.
For tables without auto-incrementing Primary Keys or for tables which may have common rows, the merging should be different, off course.
Unlock.
You can read about auto_increment_increment and related system variables that you can change so from this point, the two databases produce different auto incremented ids (one odd ids, the other even ones).
Turn off auto-increment in your destination DB. Then first import data from DB1 and the from DB2. In your importing from DB2 add a constant value that is higher than your hightest id in the first DB. Like this:
insert into destination_table
select id + 10000, othercolumns from source_table
After importing the data you can turn on auto-increment again.
EDIT :
If your id column references to other tables then this method will break the relation to these tables.
I think you have to extend your destination DB with a column for example regionID and edit the primary key settings for this table. Use a Primary key with the two columns ID and regionID. Then import the data from the two tables like this:
Insert into destination_table values(regionID, ID, Name)
select 1,ID, Name from DB1
Insert into destination_table values(regionID, ID, Name)
select 2,ID, Name from DB2
Now, the tricky part. You have to do this for all tables, where you use the ID as a relation. After transferring all data you only have to edit your SQL statements to use regionID and ID combined as key.
Remove primery key and Turn off auto-increment from id field your destination DB table.
Then first import data from from both DB.
Delete id column from destination table.
Create again id column make that column auto increament primary key.
So I want to delete a row from a table e.g
1
2
3
4
5
7
8
How do I rename the tables rows primary keys so they are equal to
1
2
3
4
5
6
7
This question comes frequently but it's a false problem. You don't have to care about id. It's just an identifier. Leave it as it is.
You can add a progressive number programmatically, using your favourite programming language or via sql.
select *,#row:=#row+1 as progressive_number
from table, (select #row:=0) as t
You shouldn't rename primary keys. Their function is to provide a unique internal identifier in your database. Even it has a sequence like 1, 10, 43, 88 .. it doesn't matter as it should have no cosmetic meaning whatsoever.
You can delete the row by its primary key...
delete from my_table where id = 4
And then decrement all subsequent keys:
update my_table set id = id - 1 where id > 4;
Assuming all the foreign keys referencing the primary key are set to on update cascade, the internal referential integrity will be maintained. If you're using the numeric id to build URLs, your URLs are all now broken.
But you absolutely should not do this. This is not how databases work. It is not a case of aesthetics, it is outright wrong to do this with your data. This is literally no good reason to do this, and a ton of reasons not to.
Hello is it possible to save the deleted auto incremented primary key in my database. For example
I have
Name_ID
1
2
3
4
If I delete primary key 4 and I insert again the primary key of I inserted should be four.
so. Name 1 2 3 4 5
I deleted primary key 5 (Name 1 2 3 4)
I added a data primary key should be 5 again not 6. THANKS!
Auto generated fields always have gaps in these cases.
What if you have an audit or history table that stored the rows with ID = 4, ID = 5? Then delete them again? How do you differentiate rows?
In your example, you've only deleted the last row? What is you delete ID = 1? Then what?
That is, they are just internal numbers unique to that table (and any associated tables like audit ones): no external meaning should be attached
As with other comments and answers here, I would not recommend this, especially if the data in the auto increment column is referenced externally, but you can set the next auto increment number to a specific value via an ALTER TABLE query
ALTER TABLE T_YourTable AUTO_INCREMENT=4
You could also drop the column and then re-add the column with the same attributes (this could be expensive if you have a lot of rows).
Why?
It's only intended to be a unique identifier.
You'll also get gaps with database clusters and whenever you rollback an insert transaction which overlaps a commited transaction - not just when you delete data.
A mechanism to fill-in-the-gaps would be complex, slow and difficult to maintain - and it's not needed.
I'm developing a helpdesk-like system, and I want to employ foreign keys, to make sure the DB structure is decent, but I don't know if I should use them at all, and how to employ them properly.
Are there any good tutorials on how (and when) to use Foreign keys ?
edit The part where I'm the most confused at is the ON DELETE .. ON UPDATE .. part, let's say I have the following tables
table 'users'
id int PK auto_increment
department_id int FK (departments.department_id) NULL
name varchar
table 'departments'
id int PK auto_increment
name
users.department_id is a foreign key from departments.department_id, how does the ON UPDATE and ON DELETE functions work here when i want to delete the department or the user?
ON DELETE and ON UPDATE refer to how changes you make in the key table propagate to the dependent table. UPDATE means that the key values get changed in the dependent table to maintain the relation, and DELETE means that dependent records get deleted to maintain the integrity.
Example: Say you have
Users: Name = Bob, Department = 1
Users: Name = Jim, Department = 1
Users: Name = Roy, Department = 2
and
Departments: id = 1, Name = Sales
Departments: id = 2, Name = Bales
Now if you change the deparments table to modify the first record to read id = 5, Name = Sales, then with "UPDATE" you would also change the first two records to read Department = 5 -- and without "UPDATE" you wouldn't be allowed to make the change!
Similarly, if you deleted Department 2, then with "DELETE" you would also delete the record for Roy! And without "DELETE" you wouldn't be allowed to remove the department without first removing Roy.
You will need foreign keys if you are splitting your database into tables and you are working with a DBMS (e.g. MySQL, Oracle and others). I assume from your tags you are using MySQL.
If you don't use foreign keys your database will become hard to manage and maintain. The process of normalisation ensures data consistency, which uses foreign keys.
See here for foreign keys. See here for why foreign keys are important in a relational database here.
Although denormalization is often used when efficiency is the main factor in the design. If this is the case you may want to move away from what I have told you.
Hope this helps.