I have an entity with a strategy to auto generate an id based on an integer column in MySQL. Things work, but while testing exceptions and related rollbacks, I noticed that MySQL does not reset last incremented value.
So a successful save produces entity id 1
An attempted save gets entity id 2 but is rolled back.
Then a successful save of a new entity gets entity id 3.
Consequently, in the table we have two records. One with id 1 and the other with id 3.
Are there any ways to control this? Basically, in the scenario I have just described, I would like to see two entities: one with id set to 1 and the other with id set to 2.
No, you can't change that. That is how it is supposed to be.
An auto-increment id has to be unique. That's all.
Auto-increment numbers have to be unique, but they don't have to be consecutive. They are monotonically increasing only as a coincidence of their implementation.
You can always insert a specific value and bypass the auto-increment mechanism. But you'd have to know what value is the "next" value. To avoid race conditions, you'd have to lock the table, query the MAX(id)+1 and then insert that value.
And that's exactly what MySQL would have to do, too, if it were to do this automatically.
The way auto-increment works now allows maximum concurrency without race conditions. So it is by design that it "loses" some values from time to time, when you rollback an INSERT, or else if you subsequently DELETE a value.
You can handle it using your own auto increment logic.
Have a Max+1 idgenerator or have a table that maintains PK auto generated IDs of such tables.
A table like this
LastKey TableName
1 TableX
5 TableY
Everytime, you will have to query from this table to get the incremented id.
Related
I have a table1 with an id field, type AutoIncrement. I need to copy the entire record from table1 into table2 if there is no record with the same id in table2. Then I delete the record from table1.
I need to know that if table1 gets new records, the id field will never be a number that was ever used before. Does this happen automatically, or do I need to do something to ensure this?
I tried deleting some records and adding new ones, and it really didn't use the same id, but I'm not sure that this is what always happens.
It is possible to duplicate numbers in autoincremet field quite easy, but normally applications don't work this way.
Access remembers last inserted value in autoincrement field and uses it for calculating next value. You cannot insert particular value into autoincrement field using table designer or recordset in VBA, but it's possible if you use INSERT SQL statement. So, if autoincrement field has no unique index, you can insert any value. Also if you insert value less than maximum existing number, Access will generate duplicates automatically.
So I would not recommend rely on unique autoincrement numbers without unique index.
INSERT SQL can be used for resetting numeration without dropping field/table, just run query like this in query builder or using VBA:
INSERT INTO Table1 ( id ) SELECT 1;
This is table with autoincrement field ID I just created:
it is really so, Auto-increment fields in MS Access are always incremental, even if records are deleted, database compacted, etc.
The proposed number can be reset deleting the auto-increment field, perform the copy of the table and then adding the auto-increment field again.
Auto increment never uses the same # even though it's deleted from the table.
It requires complete reset so that it will start from the base and create new #.
I receive batches of, say, 100 items that I need to insert into three related MySQL tables: say current, recent, and historical. I want to insert each batch in each table as a group in a single insert statement for speed. The current table has an auto-increment primary key id that I need to obtain for each inserted row and use as the primary key to insert the same row in the recent and historical tables. My idea is to get the current auto_increment value for current, increment it by 100 using alter table current AUTO_INCREMENT=, then insert the 100 rows into current with programmatically set ids from the block that I just "reserved". Then I can use the same 100 reserved id values for the inserts into the recent and historical tables without having to query them again from the current table.
My question: Is there some reason that this is a bad idea? I have seen nothing about it on the web. The closest I have seen on stack overflow is Insert into an auto increment field but that is not quite the same thing. I can see possible threading issues if I tried to do this from more than one thread at a time.
I'm also open to other suggestions on how to accomplish this.
There might be concurrency issues: If another connection inserts values between the time you get the current value and you set the new value, you would get duplicate keys.
I am not aware if that can happen in your situation, however, or if the inserts happen only from your batch described above, and there is never another instance of it running in parallel.
Methinks you shoud decouple the IDs from the 3 tables and using ALTER TABLE sounds very fishy too.
The most proper way I can think of:
in recent and historical, add a colum that references to current ID; don't try to force the primary IDs to be the same.
Acquire a WRITE table lock on current.
Get the auto_increment value X for current.
Insert your 100 records; their IDs should now run from X+1 to X+100.
Release the table lock.
Insert records in recent and historical with the know IDs in the extra column.
Note: I'm not sure if the auto_increment value points to the next ID, or the current highest value. If you use MAX(id) then you should use the code above.
This is [a bit] late, but in case someone else has this same question (as I did):
As Ethan pointed out in his comment, auto_increment is an internal MySQL utility to produce unique keys. Since you have the ability to generate your own id values external to MySQL, I suggest removing the auto_increment overhead from the table (but keep id as PK, for transport to the other tables). You can then insert your own id values along with the data.
Obviously once you do this you'll have to program your own incrementing id values. To retrieve a "starting point" for each batch and maintain the speed of a single batch INSERT call, create another table (I'll call in management) with just a single record of last_id, which is equivalent to, but independent of, max(id) of your three primary tables. Then, each time a new batch is ready to be processed, start a transaction on management with a write lock, read management.last_id, UPDATE management.last_id to (last_id+1)+number in batch, then close the transaction. You now have sequential id values to insert that are reserved for that batch because any future calls to management.last_id will the next-larger set of id values.
The write-locked transaction removes any concurrency issues (as stated in FrankPI's answer) because any other processes attempting to read management must wait for the lock to be removed and will return the value after the UPDATE. This also removes the id ambiguity in JvO's answer: "...IDs should now run from X+1 to X+100", which can be a dangerous assumption.
I have a table in MySQL using InnoDB and a column is there with the name "id".
So my problem is that whenever I delete the last row from the table and then insert a new value, the new value gets inserted after the deleted id.
I mean suppose my id is 32, and I want to delete it and then if I insert a new row after delete, then the column id auto-increments to 33. So the serial format is broken ie,id =30,31,33 and no 32.
So please help me out to assign the id 32 instead of 33 when ever I insert after deleting the last column.
Short answer: No.
Why?
It's unnecessary work. It doesn't matter, if there are gaps in the serial number.
If you don't want that, don't use auto_increment.
Don't worry, you won't run out of numbers if your column is of type int or even bigint, I promise.
There are reasons why MySQL doesn't automatically decrease the autoincrement value when you delete a row. Those reasons are
danger of broken data integrity (imagine multiple users perform deletes or inserts...doubled entries may occur or worse)
errors may occur when you use master slave replication or transactions
and so on ...
I highly recommend you don't waste time on this! It's really, really error prone.
You have two major misunderstandings about how a relational database works:
there is no such thing as the "last row" in a relational database.
The ID (assuming that is your primary key) has no meaning whatsoever. It doesn't matter if the new row is assigned the 33, 35354 or 236532652632. It's just a value to uniquely identify that row.
Do not rely on consecutive values in your primary key column.
And do not try the max(id)+1 approach. It will simply not work in a system with more than one transaction.
You should stop fighting this, even using SELECT max(id) will not fix this properly when using transactional database engine like Innodb.
Why you might ask? Imagine that you have 2 transactions, A and B, that started almost at the same time, both doing INSERT. First transaction A needs new row id, and it will use it from invisible sequence associated with this table (known as AUTOINCREMENT value), say 21. Another transaction B will use another successive value (say 22) - so far so good.
But, what if transaction A rolls back? Value 21 cannot be reused, and 22 is already committed. And what if there were 10 such transactions?
And max(id) can assign the same value to both A and B, so this is not valid as well.
I suppose you mean "Whenever I delete the last row from the table", isn't it?
Anyway this is how autoincrement works. It's made to keep correct data relations. If in another table you use an id of a record that has been deleted it's more correct to get an error instead of get another record when querying that id.
Anyway here you can see how to get the first free id in a field.
In order to determine how often some object has been used, I use a table with the following fields;
id - objectID - timestamp
Every time an object is used, it's ID and time() are added in. This allows me to determine how often an object has been used in the last hour/minute/second etc.
After one hour, the row is useless (I'm not checking above one hour). However, it is my understanding that it is unwise to simply delete the row, because it may mess up the primary key (auto_increment ID).
So I added a field called "active". Prior to checking how often an object has been used I loop over all WHERE active=1 and set it to 0 if more than 1 hour has passed. I don't think this would give any concurrency problems between multiple users, but this leaves me with alot of unused data.
Now I'm thinking that maybe it's best to, prior to inserting new usage data, check if there is a field with active=0 and then rather than inserting a new row, update that one with the new data, and set active to 1 again. However, this would require table locking to prevent multiple clients from updating the same row.
Can anyone shed some more light on this, please?
I've never heard anywhere that deleting rows messes up primary keys.
Are you perhaps attempting to ensure that the id values automatically assigned by auto_increment match those of another table? This is not necessary - you can simply use an INTEGER PRIMARY KEY as the id column and assign the values explicitly.
You could execute an update query that match all rows older than 1 hour.
UPDATE table SET active=0 WHERE timestamp < now() - interval 1 hour
I was just creating a new table using MySQL Query Browser, and noticed there's a tick under Auto Increment Column. How does that work?
When adding to the database programatically, do I just add a number, and then the database automatically increments that number?
Everytime a NEW user registers on my site, I want their Customer ID (integer only) to auto increment, so I don't have to try and randomly generate a unique number.
Can this be done simply?
Thank you!
When adding to the database programatically, do I just add a number, and then the database automatically increments that number?
Yes, that's the way auto_increment works.
The value will be incremented for each new row
The value is unique, duplicates are not possible
If a row is deleted, the auto_increment column of that row will not be re-assigned.
The auto_increment value of the last inserted row can be accessed using the mySQL function LAST_INSERT_ID() but it must be called right after the insert query, in the same database connection
mySQL Reference
1 more,
You can insert your own value also (ie your random value).
Yes. Auto_Increment columns work like they say on the tin. Tips
when INSERT - ing, use NULL or omit the column
Use LAST_INSERT_ID() (or API equivalents) to obtain the last generated value.
for security and business logic reasons, it's usually better form to not directly use a key value for a customer identifier. Consider using Hashed / randomised surrogate customer keys instead.
Ta
Yes, that's the exact purpose of AUTO_INCREMENT. It looks at whatever is the current increment value for that table, and stores that value plus 1 for the new row that comes in, automatically. You can omit that field from your INSERT statements and MySQL will handle it for you for every new row that comes in, giving each row its own unique ID.
When you enable Auto Increment an ID will always get automatically added whenever a new record is made.. Example:
If you have 1 record with ID 1 in your table and you add a new record, the ID will automatically be 2.