It is popular to save all versions of posts when editing (like in stackexchange projects), as we can restore old versions. I wonder what is the best way to save all versions.
Method 1: Store all versions in the same table, and adding a column for order or active version. This will makes the table too long.
Method 2: Create an archive table to store older versions.
In both methods, I wonder how deals with the row ID which is the main identifier of the article.
The "best" way to save revision history depends on what your specific goals/constraints are -- and you haven't mentioned these.
But here some thoughts about your two suggested methods:
create one table for posts, and one for post history, for example:
create table posts (
id int primary key,
userid int
);
create table posthistory (
postid int,
revisionid int,
content varchar(1000),
foreign key (postid) references posts(id),
primary key (postid, revisionid)
);
(Obviously there would be more columns, foreign keys, etc.) This is straightforward to implement and easy to understand (and easy to let the RDBMS maintain referential integrity), but as you mentioned may result in posthistory have too many rows to be searched quickly enough.
Note that postid is a foreign key in posthistory (and the PK of posts).
Use a denormalized schema where all of the latest revisions are in one table, and previous revisions are in a separate table. This requires more logic on the part of the program, i.e. when I add a new version, replace the post with the same id in the post table, and also add this to the revision table.
(This may be what SE sites use, based on the data dump in the SE Data Explorer. Or maybe not, I can't tell.)
For this approach, postid is also a foreign key in the posthistory table, and the primary key in the posts table.
In my opinion, a interesting approach is
to define another table, for example posts_archive (it will contain all columns of posts table + an auto-incremented primary key + optionally a date...)
to feed this table through after-insert and after-updates triggers defined on posts table.
If the size of the table is an issue, then the second option would be the better choice. That way the active version can be returned quickly from a smaller table, and restoring an older version from the larger archive table is accepted to take longer. That said, the size of the table should not be an issue with a sensible database and indexing.
Either way, you need a primary key that consists of multiple table columns instead of just row ID. The trivial answer would be to include a timestamp containing the time each revision was created into the key, so that ID continues to identify a specific article, and ID and revision time together identify a specific revision of the article.
Dealing with temporal data is a known problem.
The method 1 simply changes your table identifier: you will end up with a table containing messageID, version, description, ... with a primary key messageID, version.
Modifying the data is done by simply adding a row with an incremented version. Querying is a little bit more complicated.
The method 2 is more tedious, you will end up with a table with a rowID and a second table that is exactly the same as in the method 1. Then, on every update, you will have to remember to copy the data into the "backup table".
The method 3: answser given by Matt
In my opinion, method 1 and 3 are better. The schema is simplier in 1, but you can have unversionned data for your posts using the method 3.
Related
State description
I have two databases, DB1 and DB2, that have the same table, Author, with the fields Author.ID and Author.AuthorName.
The DB1.Author has the AUTO_INCREMENT on its Author.ID field, while the DB2.Author does not have the AUTO_INCREMENT since it relies on the correctness of DB1 data.
Both tables have the PRIMARY index on Author.ID and a UNIQUE index on Author.AuthorName.
DB2.Author has rows copied from the DB1.Author.
Both databases use MariaDB version 10.6.7.
The problem
DB1 manager deleted some entries in the DB1.Author table, and then reordered indexes to have no gaps in index numbers. This means they might have had:
ID
AuthorName
1
A
2
B
3
C
Then they deleted the row where the AuthorName was 'B':
ID
AuthorName
1
A
3
C
And they finally updated the indexes to have no gaps (3-C changed to 2-C):
ID
AuthorName
1
A
2
C
Now I need to find a way to copy such updated state of the rows from the DB1.Author to the DB2.Author without deleting everything from the DB2.Author table, so that I don't lose the data on CASCADE effects.
What is the best approach for this?
My shot
This is what I did, but it obviously cannot work, since in the case of duplicate key, it would attempt to create another duplicate key (duplicate ID 2 would try to INSERT duplicate value of 'C', since it already exists on ID 3):
INSERT INTO DB2.Author (ID, AuthorName)
SELECT DB1.Author.ID, DB1.Author.AuthorName FROM DB1.Author
ON DUPLICATE KEY UPDATE
ID = DB1.Author.ID,
AuthorName = DB1.Author.AuthorName;
Additional ways?
Other than the possible SQL query solution, are there any other ways to automatically update the table data in one database when the other database changes its data? Would need to replicate only some tables, while other, linked tables are different.
tl;dr your problem is your DB manager. The solution is to get him/her to undo the damage they caused by restoring the data to how it was before. Deleting rows is fine. Updating primary keys is never OK.
Do not create a work around or validate it by accommodating his/her mistake, because doing so will make it more likely that it will happen again.
Full answer.
Your actual problem is your "DB manager", who violated a fundamental rule of databases: Never update surrogate key values!
In your case it's even more tragic, because gaps in the ID column values don't matter in any way. If gaps do matter, you're in even worse shape. Allow me to explain...
The author's name is your actual identifier. We know this because there a unique constraint on it.
The ID column is a surrogate key, which are most conveniently implemented as an auto incrementing integer, but surrogate keys would work just as well if they were random (unique) numbers. Gaps, and even the choice of values themselves, are irrelevant to the effectiveness of surrogate keys.
You need to treat the DB2 table as completely wrong as the update of primary keys on the source table would have completely spoilt it.
Delete everything in DB2 table
Insert into DB2 table everything from DB1 table
Going forwards, without being condescending, the users with access to DB1 need training (or perhaps you need to reconsider the security against the DB). Updating a primary key value is a wrong thing to do. Gapless sequences is a silly thing to want, especially when you have known dependencies. In fact, gapless sequences is often listed as poor database security (as it makes it easy to just cycle through all data).
You probably want to consider commercial solutions for logical data replication. If they don’t support updates of primary keys then you can use that as a good enough reason not to.
I would invest time in making sure there’s no other logical corruptions of data like this.
During the creation of tables using mysql on phpmyadmin, I always find an issue when it comes to primary keys and their auto-increments. When I insert lines into my table. The auto_increment works perfectly adding a value of 1 to each primary key on each new line. But when I delete a line for example a line where the primary key is 'id = 4' and I add a new line to the table. The primary key in the new line gets a value of 'id = 5' instead of 'id = 4'. It acts like the old line was never deleted.
Here is an example of the SQL statement:
CREATE TABLE employe(
id INT UNSIGNED PRIMARY KEY AUTO_INCREMENT,
name VARCHAR(30) NOT NULL
)
ENGINE = INNODB;
How do I find a solution to this problem ?
Thank you.
I'm pretty sure this is by design. If you had IDs up to 6 in your table and you deleted ID 2, would you want the next input to be an ID of 2? That doesn't seem to follow the ACID properties. Also, if there was a dependence on that data, for example, if it was user data, and the ID determined user IDs, it would invalidate pre-existing information, since if user X was deleted and the same ID was assigned to user Y, that could cause integrity issues in dependent systems.
Also, imagine a table with 50 billion rows. Should the table run an O(n) search for the smallest missing ID every time you're trying to insert a new record? I can see that getting out of hand really quickly.
Some links you might like to read:
Principles of Transaction-Oriented Database Recovery (1983)
How can we re-use the deleted id from any MySQL-DB table?
Why do you care?
Primary keys are internal row identifiers that are not supposed to be sexy or good looking. As long as they are able identify each row uniquely, they serve their purpose.
Now, if you care about its value, then you probably want to expose the primary key value somewhere, and that's a big red flag. If you need an external, visible identifier, you can create a secondary column with any formatting sequence and values you want.
As a side note, the term AUTO_INCREMENT is a bit misleading. It doesn't really mean they increase one by one all the time. It just mean it will try to produce sequential numbers, as long as it is possible. In multi-threaded apps that's usually not possible since batches or numbers are reserved per thread so the row insertion sequence may end actually not following the natural numbering. Row deletions have a similar effect, as well as INSERT with roll backs.
Primary keys are meant to be used for joining tables together and
indexing, they are not meant to be used for human usage. Reordering
primary key columns could orphan data and wreck havoc to your queries.
Tips: Add another column to your table and reorder that column to your will if needed (show that column to your user instead of the primary key).
As my title states, I'm curious about the best practices for modifying an existing table in a (mysql) database. In my scenario, I have a table that is already full of data and has a column named product_id that is currently the primary key for the table. I'm working on a feature where I'm finding product_id doesn't necessarily need to be unique or the primary key, since I want to allow multiple records for the same product. Database design isn't a strength of mine yet, but in my head I feel like what I would want to do is run the command DROP PRIMARY KEY for the product_id column, then add a column called id and making this the new primary key. Then I would need to update the id column for each record with a unique id for it to be a valid primary key. As far as database design is concerned, is this the best practice for doing this or is it better to create a new table with the updated structure and copying the current records into the new table?
EDIT:
More about the feature I'm working on. The products are books and I'm trying to allow multiple sections of these books to be previewed. In order to do this, I'm storing page ranges that can be previewed. Right now, only one page range is allowed, which is why the product id doesn't need to be unique anymore.
A primary key is ALWAYS unique.
Why do you don't want it to be unique? It sounds like you are exposing the key outside the database, that the PK is visible somehow and some user(s) think it should behave differently. If this is the case then this is a really bad practice.
This is the typical case of the notorious "natural keys". They are a disaster waiting to happen; I don't like big time bombs. I've been strongly opposed to them for some time now. It's good they teach them in schools so you know what not to use in the real world.
Now for the solution. If product_id is exposed, then it shouldn't be the PK at all. Solution?
Create a new column (id maybe?) that is internal, that is unique, and not exposed to the user, while keeping product_id. This new column could have the exact same value as product_id at first.
Change all FK references from other tables to the new id column.
Then, remove the PK constraint from product_id and do whatever you want to do with it.
Add the PK contraint to the new id column.
There is a table. No PK, 2 FK, with some arbitrary number of columns.
Unfortunately FK are not unique in any way.
Adding new data is easy.
Deleting data (finding a row) is ok if I put unique constraint to some other col.
(DELETE ... WHERE fk1=:fk1 AND fk2=:fk2 AND ucol=:ucol)
What to do with UPDATE?
I cant use that ucol because that same ucol might be subject of change. I have several solutions, but none of them seem ok.
Solution1:
Put PK in table. Use it for DELETE and UPDATE. Deleting will make lot of holes in it but that's no problem. In theory, it can run out of PK numbers (int, unsigned int) if there's some heavy deleting going on.
Solution1a
Make CK of (fk1, fk2, some new col) use that to locate the row. It's the same as just using the PK.
Solution2
Use timestamp with microtime/ hash/ unique key generator/ something to populate new unique col. That col is used as PK to locate the row for UPDATE and DELETE. Excellent only if unique algo does it's job perfectly.
My question:
Is there something better? That doesn't require fancy algorithms and have no risk of overflowing auto-incremented PK...
----------------- edit----------------
Solution2a
Use mysql UUID! It's far better (and easier to use) than, creating custom timestamp / hash / something_unique.
As per my suggestion , it will be better to add a PK to the table because of following reasons:
1. It will give unique id to each row , which will help in DELETE and UPDATE script.
2. PK will create a cluster index on the column which will improve performace of the table while retriving data.
3. Its always adviced to provide a PK in each table.
4. In future you can use the PK as a FK in any table if required.
I have a database which will store millions of Post ID#s. I need to associate with each post ID # a number of User ID #s (on the order of about 20-50 for each post ID). I was thinking of constructing a semicolon delimited list in PHP and just inserting that into a DB field on the post ID row.
Is this a relativly efficient and good way to go about doing this?
Thanks!
The long answer to this is you need to create a one-to-many association table. Proper database normalization principles dictate this.
The problem with your approach, serializing the list into the database as a semicolon-concatenated list, is the data itself is virtually useless unless you can deserialize it.
Fields of this sort:
Cannot be indexed effectively.
Can grow to exceed the storage capacity of the column.
Require context to properly utilize.
Cannot work with foreign key integrity checking.
Cannot be easily amended.
Removing entires requires re-writing the entire field.
Cannot be queried directly.
Cannot be used in JOIN operations.
You're talking about creating a simple association table:
CREATE TABLE user_posts (
id INT AUTO_INCREMENT PRIMARY KEY,
user_id INT,
post_id INT
)
You'd have a UNIQUE index on user_id,post_id to ensure that you don't have duplicates. The inclusion of an id column is mostly so you can remove particular rows without having to specify user+post pairs.
No, this is a very bad idea.
A foreign key is what you want here. Basically, for every post_ID you also store the USER ID as a foreign key.
So, if you have a POSTS table, you add a column User_ID (or Poster_ID) and reference the USER ID in the USER table.
I think you should review some of the basics - please see links:
http://www.functionx.com/sql/Lesson11.htm
http://creately.com/blog/diagrams/er-diagrams-tutorial/
https://cs.uwaterloo.ca/~gweddell/cs348/errelational-handout.pdf