I have some products, which has it's own ids and i'm designing MySQL DB and then I will import this data, there is much more than product table, but it doesn't matter now.
It's good idea to reuse existing product ids as primary key? So into the autoincerement ID column will be imported existing product ids, I never did that like I'm describing.
It is also worth to mention, that IDs are normal unsigned integer values and also that products are now only some rows in xls sheet.
I think it would be great to keep the IDs as they are if you have any relationship build up upon those IDs, and for the new IDs that will be added just let them increment with the identity property.
To insert defined IDs on an identity column (auto-increment) use the following:
Set Identity_Insert [TableName] On
-- --------------------------------------------
youre insert query goes here
-- --------------------------------------------
Set Identity_Insert [TableName] Off
Related
I have two tables : Shop and Product
Table Shop
(id INT AUTO_INCREMENT,
shop_id INT,
PRIMARY KEY(id)
);
Table Product
(
product_id INT AUTO_INCREMENT,
p_name VARCHAR(100),
p_price INT,
shop_id INT,
PRIMARY KEY(product_id),
FOREIGN KEY(shop_id) REFERENCES Shop(id)
);
On a server using Node and mysql2 package for queries.
On a client side, I'm displaying all Products that are related to specific Shop in a table.
User can change Products, and when he is pressing Save, requests are being made, sending new data, and storing her.
User can either change existing Products, or add new ones.
But i have concerns, how it will behave with a relatively big amount of products per one shop. Let's say there are 1000 of them.
The data that was inserted - marked with the flag saved_in_db=false.
Existing data, that was changed - changed=true.
Considered a few approaches :
On a server, filtering array of records received from a client, INSERT into db newly created, that are not stored yet.
But to UPDATE existing Products, i need to create a bunch of UPDATE Products SET p_name=val_1 WHERE id = ? queries, and execute them at once.
To take all Products with the specified Shop_id, DELETE them, and INSERT a new bulk of data. Not making separation between already existing records, or changed.
In this approach, i see two cons.
First - sending constant amount of data from client to server.
Second - running out of ids in DB. Because if there are 10 shops, with 1000 Products in each, and every user frequently updates records, every update, even if one new record was added, or changed, will increment id by around 1000.
Is it the only way, to update a certain amount of records in DB, executing a bunch of UPDATE queries one after another?
You could INSERT...ON DUPLICATE KEY UPDATE.
INSERT INTO Products (product_id, p_name)
VALUES (123, 'newname1'), (456, 'newname2'), (789, 'newname3'), ...more...
ON DUPLICATE KEY UPDATE p_name = VALUES(p_name);
This does not change the primary key values, it only updates the columns you tell it to.
You must include the product id's in the INSERT VALUES, because that's how it detects that you're inserting a row that already exists in the table.
I have a table with name as listings and inside there I have a COLUMN namely as when some rows are deleted so the AUTO Incrmementation columns namely as "ID" goes into soemthing very bad values..Like missing values in between which I don't like and don't suit like a professional way..so therefore I want please if you people can guide me to how reset all ID columns values in rows on each INSERT or DELETE Query Exeution please..!
If you really want to find the lowest unused key value, don't use AUTO_INCREMENT at all, and manage your keys manually. However, this is NOT a recommended practice.
AS explained at Auto Increment after delete in MySQL
Primary autoincrement keys in database are used to uniquely identify a
given row and shouldn't be given any business meaning. So leave the
primary key as is and add another column called for example
courseOrder. Then when you delete a record from the database you may
want to send an additional UPDATE statement in order to decrement the
courseOrder column of all rows that have courseOrder greater than the
one you are currently deleting.
As a side note you should never modify the value of a primary key in a
relational database because there could be other tables that reference
it as a foreign key and modifying it might violate referential
constraints.
Well it is not recommended but you insisted , so use this to re order
By using something like:
ALTER TABLE table_name AUTO_INCREMENT = 1;
We have a shopping cart as pictured below, The setup works well, except for one fatal flaw. If you place an order the order is linked to a product, so If I update the product after you have purchased the product there is no way for me to show you want the product looked like when you bought it (including price). This means we need versioning.
My plan at present is to, when a new product, or variant is created, or an existing one is edited, create a duplicate of the product or variant in the database. When a purchase is made, link the order to the version, not the product.
This seems rather simple, except from what I can see the only things we don't need to version are the categories (as no one cares what categories it was in.). So we need to version:
Products
Variants
The key -> value pairs of attributes for each version
The images
My current thinking is,
note: When a product is created a default variant is created as well, this cannot be removed.
When a product is created
Insert the product into the products table.
Create the default variant
Duplicate the product into the products_versions table
Replace current id column with a product_id column
Add id column
Duplicate the variant into the variants_versions table
Replace current id column with variant_id column
Add id column
Replace product_id column with product_version_id column
When a product is edited
Update the product into the products table.
Duplicate the product into the products_versions table
Replace current id column with a product_id column
Add id column
Duplicate all product variants into the variants_versions table
Replace current id column with variant_id column
Add id column
Replace product_id column with product_version_id column
Duplicate all variant_image_links into the variant_Image_link_version table
Replace current variant_id column with variant_version_id column
When a variant is added
Add the variant into the variants table.
Duplicate the product into the products_versions table
Replace current id column with a product_id column
Add id column
Duplicate all product variants into the variants_versions table
Replace current id column with variant_id column
Add id column
Replace product_id column with product_version_id column
When a variant is edited
Update the variant in the variants table.
Duplicate the product into the products_versions table
Replace current id column with a product_id column
Add id column
Duplicate all product variants into the variants_versions table
Replace current id column with variant_id column
Add id column
Replace product_id column with product_version_id column
Duplicate all variant_image_links into the variant_Image_link_version table
Replace current variant_id column with variant_version_id column
So the final structure looks like Full Size
Now this all seems great, except it seems like a heck of a lot of duplicated data, e.g. if we update a product we duplicate the variants even though they would not have been updated since they were inserted. Also, this seems like a lot of work.
Is there a better way of doing this?
You can do what ERP (and also possibly Payroll) systems do: Add a Start and End Date/Time. So...
the variant and prices match with their product based on the common dates.
all queries default to running on current date and the joins between each table need to also take into account the overlapping/intersecting date ranges. parent_start_date <= child_start_date AND parent_end_date >= child_end_date
You would end up with duplicated rows for each price change or variant but you then don't need to keep update as many records (like variant ids) when the product price changes.
Need to ensure valid dates are used. PS: Use your system's max date for the End datetime of the most current/recent record.
Btw, some related questions along the same line:
Ways to implement data versioning in MongoDB
Ways to implement data versioning in PostreSQL
Ways to implement data versioning in Cassandra
Row versioning in MySQL
Another approach to this would be to never edit or remove your data, only create new data. In SQL terms, the only operations you ever run on your tables are INSERTs and SELECTs.
To accomplish what you want, each table would need the following colums:
version_id - this would be your primary key
id - this would be the thing that holds versions of your object together (e.g. to find all versions of a product, SELECT * FROM products WHERE id = ?)
creation_date
is_active - you're not deleting anything, so you need to flag to (logically) get rid of data
With this, here's what your products table would look like:
CREATE TABLE products (
version_id CHAR(8) NOT NULL PRIMARY KEY,
id INTEGER NOT NULL,
creation_date TIMESTAMP NOT NULL DEFAULT NOW(),
is_active BOOLEAN DEFAULT true,
name VARCHAR(1024) NOT NULL,
price INTEGER NOT NULL
);
CREATE TABLE variants (
version_id CHAR(8) NOT NULL PRIMARY KEY,
id INTEGER NOT NULL,
creation_date TIMESTAMP NOT NULL DEFAULT NOW(),
is_active BOOLEAN DEFAULT true,
product_version_id CHAR(8) NOT NULL,
price INTEGER NOT NULL,
override_price INTEGER NOT NULL,
FOREIGN KEY (product_version_id) REFERENCES products(version_id)
);
Now, to insert into either table
Generate a unique version_id (there are several strategies for this, one is to use a database sequence, or for MySQL use ant AUTO_INCREMENT).
Generate an id. This id is consistent for all versions of a product.
To update a row in a table, one must insert the entire graph e.g. to update a product, one must insert a new product, and new variants. (There is a lot of room for optimization here, but it's easiest to start with the un-optimized solution.)
For example, to update a product
Generate a unique version_id
Use the same id
Insert new product variants. The variants will be the same as the ones linked to the previous version of the product that you're "updating", except the product_version_id will be different.
This principal can extend to all your tables.
To find the most recent version of a product, you need to use the creation_date column to get the product that was most recently created.
This model will use more space, but I think this may be a fair trade-off given it's simplicity: there are only INSERTs and SELECTs and data is never mutated.
Two tables share a unique identifier 'id'. Both tables are meant to be joined by using 'id'.
Defining 'id' as an auto incrementing primary key in both tables may risk update inconsistencies.
Is there some general pattern to avoid such a situation or do I have to deal with updating table1 first and table2 by utilizing the last inserted id after (therefore not declaring id as auto inc in table2)?
First, if you use InnoDB table engine in MySQL you could use both transactions and foreign keys for data consistency.
Second, after the insert in the first table, you could get the last insert id (depending on the way you access the db) and use it as foreign key.
Eg
Table 1: Users: user_id, username
Table 2: User_Profiles: user_id, name, phone
In User_Profiles you don't need to define user_id as auto increment, but first insert a record in Users table and use the user_id for the User_Profiles record. If you do this in transaction, the Users record won't be seen outside of the transaction connection until it's completed, this way you guarantee that even if something bad happens after you insert the user, but before you have inserted the profile - there won't be messed up data.
You could also define that the user_id column in User_Profiles table is foreign key of Users table thus if someone deletes a record from the Users table, the database would automatically delete the one in User_Profiles. There are many other options - read more about that.
There is no problem with same column name 'id' in any number of tables.
Several persistence layer frameworks do it same way.
Just use aliases in your SQL to distinct your tables accordingly.
do I have to deal with updating table1 first and table2 by utilizing the last inserted id after (therefore not declaring id as auto inc in table2)?
Yes. And make id a foreign key so it can only exist in table2 if it already exists in table1.
Yes you do, and remember to wrap the operation in a transaction.
I am having my application deployed on two separate regions say US-WEST and EU, both application has its own DB. And now I want to move the EU region DB to US-WEST.
This will lead to primary key collision since both the db has the tables with same primary auto increment id, can anybody give me suggestion to solve this.
Scenario:
User Table from DB1(say from US-WEST) has the following entries
ID Name
1 Rob
2 San
3 Tulip
User Table from DB2(say from EU) has the following entries
ID Name
1 John
2 Michael
3 Natasha
For every one of the two original databases (say db0 and db1):
Back up the db.
Lock database for use by this script only.
For all the tables in the database that have foreign keys defined without ON UPDATE CASCADE, change all these foreign keys constraints with this option.
For every table with an auto_increment (or a simple integer) Primary Key, run this (the cascading updates will make the rest):
.
UPDATE TableX
SET Pk = 2 * Pk - 0 --- for db0
ORDER BY Pk DESC
UPDATE TableX
SET Pk = 2 * Pk - 1 --- for db1
ORDER BY Pk DESC
Export the tables from each database.
Now merge the two databases by simply merging the corresponding tables. All data from db0 will have even ids and all from db1 will have odd ids. No collisions.
For tables without auto-incrementing Primary Keys or for tables which may have common rows, the merging should be different, off course.
Unlock.
You can read about auto_increment_increment and related system variables that you can change so from this point, the two databases produce different auto incremented ids (one odd ids, the other even ones).
Turn off auto-increment in your destination DB. Then first import data from DB1 and the from DB2. In your importing from DB2 add a constant value that is higher than your hightest id in the first DB. Like this:
insert into destination_table
select id + 10000, othercolumns from source_table
After importing the data you can turn on auto-increment again.
EDIT :
If your id column references to other tables then this method will break the relation to these tables.
I think you have to extend your destination DB with a column for example regionID and edit the primary key settings for this table. Use a Primary key with the two columns ID and regionID. Then import the data from the two tables like this:
Insert into destination_table values(regionID, ID, Name)
select 1,ID, Name from DB1
Insert into destination_table values(regionID, ID, Name)
select 2,ID, Name from DB2
Now, the tricky part. You have to do this for all tables, where you use the ID as a relation. After transferring all data you only have to edit your SQL statements to use regionID and ID combined as key.
Remove primery key and Turn off auto-increment from id field your destination DB table.
Then first import data from from both DB.
Delete id column from destination table.
Create again id column make that column auto increament primary key.