I am creating a Django app where the primary keys are AutoFields. i.e. I am not manually assigning any field as primary key in my models.
I need to use mySQL.
I will need to export all the data to excel or perhaps another django app from time to time. Therefore the primary keys must be unique to be able to identify new records or records to be deleted in excel/other app.
However, I have read that mySQL autoincrement counter resets to the max key when database restarts. This will result in reassignment of keys if the latest records were deleted.
I need to avoid this. No key should be reassigned.
How can this be done?
MySQL 8.0 now keeps the last auto-increment per table persistently. So it remembers between restarts, and does not reset the auto-increment.
https://www.percona.com/blog/2018/10/08/persistence-of-autoinc-fixed-in-mysql-8-0/
Related
I Use Delphi 10.2, MySQL. I have a table that has about 50,000 records and has an Auto_Increment primary key. It has suddenly, and on it's own with no help from me, started trying to re-insert old key values. As a matter of fact, it started over with the value 1. I have no idea how to fix this and I hope you might be able to help.
Thanks,
Jim Sawyer
If the MySQL table is defined with an auto increment primary key then you should never specify the key value. MySQL should not re-use old key values, but you may want to check if there is any table corruption. You can also reset the table's auto-increment value using an ALTER TABLE command. (There's a tutorial on this here: https://www.mysqltutorial.org/mysql-reset-auto-increment)
You can use the Firedac monitoring to confirm whether or not you are sending the primary key to MySQL - set you connection to be monitored using the FireDAC component - they supply a monitoring tool that you can setup to see all of the SQL being transferred. Normally the Firedac layer would do an insert with no primary key and then use LAST_INSERT_ID to update the TField to have the actual value inserted.
If you are sending the wrong key then alter your logic so you don;t send the primary key on an insert.
you can reset the autoincrement value to any value you want with the following command
ALTER TABLE <table_name> AUTO_INCREMENT = <new value>;
so if new value is 100, the next inserted record receives a value of 100.
Bit of a strange one - I haven't changed the config, database, PHP code for this site for about 8 years. The whole time the guy I made it for has been adding, removing, changing stock with no issue, and suddenly today gets this error:
Duplicate entry '2541' for key 'PRIMARY'
executing (inserted generic values for the texts):
INSERT INTO stock (id,title,category,description,price,last_update,sold) VALUES(NULL,'Item name','72','Item description','0',1613723525,'no')
Searching around seemed to suggest this was a common problem when the primary key is not set to auto increment - makes sense. However, checking it out through phpMyAdmin, it definitely is.
Is there something in the index being set to primary rather than unique?
There are 5 other tables on the database but none are linked (as in hard links through the SQL, PHP handles all the cross-table stuff).
I checked and indeed there IS an item in the stock table with ID 2541 already. So why is that NULL AUTO_INCREMENT value converting to an existing id?
EDIT
I noticed that a table I created more recently (via MySQL Workbench probably) has a different setup for the indexes, with the addition of an id_UNIQUE index - do I need one of these on the stock table that is causing issues?
Based on your description and your comment "Interestingly, each time I refresh the ID it is attempted to insert (and failing on) increments by 1", I suspect that somehow the seed for the autoincrement for that table got changed to some value that was inserted at some time before.
How exactly that could happen I don't know.
Now, each time you attempt to insert a record this internal counter increments, so you see in the error message that the number increases (2541, 2542, ...) When you attempt to insert a row the internal counter increments regardless of whether the transaction is committed to the database or not. In your case the insert operation is rolled back, because the generated value violates the unique constraint and the internal counter keeps growing.
To change the seed you can run the following statement:
ALTER TABLE stock AUTO_INCREMENT=1234567;
You'll need to set it to the current MAX value that exists in the table, so that new entries that the system attempts to insert do not conflict.
See also How to set initial value and auto increment in MySQL?
This answer shows how to change the autoincrement seed in MySQL Workbench and in PhpMyAdmin. Maybe somebody did this accidentally and didn't notice.
I'm developing an Android application in which the data is stored in a SQLite database.
I have made sync with a MySQL database, in the web, to where I'm sending the data stored in the SQLite in the device.
The problem is that I don't know how to maintain the relations between tables, because the primary keys are going to be updated with AUTO_INCREMENT, and the foreign keys remain the same, breaking the relations between tables.
If this is a full migration, don't use auto increment during migration - create tables with normal columns. Use ALTER TABLE to change the model after import.
For incremental sync, the easiest way I see is additional column in each MySQL table called sqlite_id and filled with original id. Then you can update references using UPDATE (with joins).
Alternatives involve temporary tables for storing data and an auxiliary table used for pairing. Tedious for bigger data model.
The approach I tend to use, if possible, is to avoid auto increment in such situations. I have usaully an auxiliary table with four columns like this: t_import(tablename, operationid, sqlite_id, mysqlid).
Process is the following:
Import the primary keys into t_import. Use operationid to separate parallel imports if needed.
Generate new keys for data tables and store them into t_import table. This can be combined with step one.
Import the actual data and use t_import for setting new primary keys and restore relations.
That should work for most scenarios I know about.
Thanks or the help, you have given me some ideas.
I will try to add a id2 field to the tables that will store the same value as the primary key (_id)
When I send the information from SQLite to MySQL and the primary key is incremented I will have the id2 field with the original value of the primary key so that I can compare it with the foreign key of the other tables and update it.
Let’s see if it works.
Thanks
I am using Cayenne to add records to a MySQL database, and I am seeing some strange behavior.
When I run my application, I create a DataContext, perform a series of adds, then close the application. This works out well, because I am using an integer for a primary key, and when I add a record to the database, the key automatically increments. For some reason, it starts at 200 for the first record, then goes to 201 for the second record, etc.
If, however, I stop the application, then run it again, the primary key starts at 200 again! This, of course, causes an exception to be thrown because a new record ends up having a duplicate primary key. It is looking like when I create a new object using the DataContext's newObject() after starting my application, Cayenne does not "remember" how far the primary key was incremented when the application was previously run.
Does anyone know what is causing this reset of the primary key values, and (more importantly) how to stop it from happening??? Or have I found a bug in the current version of Cayenne? I am using Version 3.0.2.
Someone please advise...
The last used PK for a given table is stored in a special table called AUTO_PK_SUPPORT. Please check the contents of this table between the restarts of your app. Also check you application Cayenne logs for reads and writes to AUTO_PK_SUPPORT. This should give you an idea of what's happening.
Aside from that you might switch to auto-increment PK (see "Primary Key Provided by Database" section here). MySQL supports auto-incremented PK columns and if you have an option of altering the schema, this IMO is the cleanest PK generation strategy out of all available. (And it doesn't require AUTO_PK_SUPPORT).
I am using cfwheels (coldfusion orm framework).
I recently moved some data from my previous host to a new one. Now I am trying to insert into a table, but am getting an error message: "Error Executing Database Query.
Duplicate entry '13651' for key 'PRIMARY'"
I looked into the database and it appears a record with id 13651 already exists. So I think the problem is with mysql generating the right auto increment value.
It seems Auto_Increment value is damaged or not set to max value in that column. It's possible due to bulk insert.
So as per solution, set the maximum PK value + 1 as new AUTO_INCREMENT value. Now when you insert the records in this table, they will automatically pick the next incremented correctly.
ALTER.TABLE tablename AUTO_INCREMENT = value
Is the rest of the data for that record, and the one you are trying to insert, the same? If you you might just need to tell the ORM to replace that value?
If primary key has auto increment attribute turned on, do not insert it manually. remove that primary key part from your insert query (whatever the syntax according to the taste of your ORM framework).