so i have same problem with this question Auto Increment after delete in MySQL
and according to that answer, and considering for each condition that provided on that answer, i've to say that i must do the auto increment after delete on mysql.
i have table lets say
table : product
fields : id, name
so if i have ids of 16,17,18 and then i deleted id 16, and i make the new one, i want to make the new id was on "16" not on 19. according to that answer
Take a step back and ask "why you need to recycle key values?" Do unsigned INT (or BIGINT) not provide a large enough key space?
i must say that yes, it's possible the fields cant provide large enough key space
what should i do
note: the id on that table its unique and didnt connected to any other table, i just want to avoid if the ids cant provide large enough key space again
How to achieve it?
To be honest, this is achieveable.Insert record's corresponding id value into an ordered queue every time it is deleted, and take the smallest value from the queue as the id to be insered(if the queue is empty, use id generated by database).
Is it worth doing?
I will assume that your primary key is the id, whcih is a clustered index in mysql. So this way may cause more page splits, which will slow insertion performance.
And you need to ensure that the id in the ordered queue can only be consumed by one inserting thread at the same time.If you inserted fail, you need to re-enqueue the id.If the size of the ordered queue is too large, the sorting operation will also slow performance.
At last, you can't rely on id for analysis. eg. The record with max id is the latest record.
Related
I have an existing schema with a non-auto-incrementing primary key. The key is used as a foreign key in a dozen other tables.
I have inherited a program with major performance problems. Currently, when a new row is added to this table, this is how a new unique id is created:
1) a query for all existing primary key values is retrieved
2) a random number is generated
3) if the number does not exist in the retrieved values, use it, otherwise goto (2)
The app is multi-threaded and multi-server, so simply grabbing the existing ids once at startup isn't an option. I do not have unique information from the initiating request to grab and convert into a pseudo-unique value (like a member id).
I understand it is theoretically possible to perform surgery on the internals to add autoincrementing to an existing primary key. I understand it would also be possible to systematically drop all foreign keys pointing to this table, then create-rename-insert a new version of the table, then add back foreign keys, but this table format is dictated by a third-party app and if I mess this up then Bad Things happen.
Is there a way to leverage sql/mysql to come up with unique row values?
The closest I have come up with is choosing a number randomly from a large space and hoping it is unique in the database, then retrying when the odd collision occurs.
Ideas?
If the table has a primary key that isn't being used for foreign key references, then drop that primary key. The goal is to make your column an auto-incremented primary key.
So, look for the maximum value and then the following should do what you want:
alter table t modify id int not null auto_increment primary key;
alter table t auto_increment = <maximum value> + 1;
I don't think it is necessary to explicitly set the auto_increment value, but I like to be sure.
I think you can SELECT MAX('strange-id-column')+1. That value will be unique and you can put that sql code inside a transaction with the INSERT code in order to prevent errors.
It seems really expensive to pull back a list of all primary key values (for large sets), and then to generate psuedo-random value and verify it's unique, by checking it against the list.
One of the big problems I see with this approach is that a pseudo-random number generator will generate the same sequence of values, when the sequence is started with the same seed value.
If that ever happens, then there will be collision after collision after collision until the sequence reaches a value that hasn't yet been used. And the next time it happens, you'd spin through that whole list again, to add one more value.
I don't understand why the value has to be random.
If there's not a requirement for pseudo-randomness, and an ascending value would be okay, here's what I would do if I didn't want to make any changes to the existing table:
I'd create another "id-generator" table that has an auto_increment column. I perform inserts to that table to generate id values.
Instead of running a query to pull back all existing id values from the existing table, I'd instead perform an INSERT into the "id-generator" table, and then a SELECT LAST_INSERT_ID() to retrieve the id of the row I just inserted, and that would use that as "generated" id value.
Basically, emulating an Oracle SEQUENCE object. It wouldn't be necessary to keep all of the rows in "id-generator" table. So, I could perform a DELETE of all rows that have an id value less than the maximum id value.
If there is a requirement for pseudo-randomness (shudder) I'd probably just attempt the INSERT as a way to find out if the key exists or not. If the insert fails due to a duplicate key, I'd try again with a different id value.
The repeated sequence from a pseudo-random generator scares me... if I got several collisions in a row... are these from a previously used sequence, or are they values from a different sequence. I don't have any way of knowing. Abandoning the sequence and restarting with a new seed, if that seed has been used before, I'm off chasing another series of previously generated values.
For low levels of concurrency (average concurrent ongoing inserts < 1) You can use optimistic locking to achieve a unique id without autoincrement:
set up a one-row table for this function, eg:
create table last_id (last_id bigint not null default 0);
To get your next id, retrieve this value in your app code, apply your newId function, and then attempt to update the value, eg:
select last_id from last_id; // In DB
newId = lastId + 1 // In app code
update last_id set last_id=$newId where last_id=$lastId // In DB
Check the number of rows that were updated. If it was 0, another server beat you to it and you should return to step 1.
Quick question:
I have a sports league database with a list of games (let's say 40 or so). Each game is auto-assigned an ID number as the primary key when importing the entire schedule from a spreadsheet. The games are then displayed on the web page in descending order thanks to this invisible (to the user) primary key. Here's an example: League Schedule
Works great. The only problem is that sometimes the games are rescheduled and moved to a later date or a new game is added and has to be inserted into an already existing schedule. To this point, I've had to manually edit each affected row's ID (using PhpMyAdmin) to account for the changes and this can be quite tedious and time consuming.
What I'd really like to do is set the table to readjust primary key values on the fly. Meaning, if I inserted a brand new game into the fifth row of the table, all games thereafter would automatically be readjusted (ID 5 would become 6, ID 6 would become 7, and so on).
Is there a way to set-up the table to do this, or a particular SQL command I can use to accomplish it just the same? Apologies if this has already been asked many times in different ways. Any and all feedback is appreciated.
You should not use your PRIMARY KEY for that. Add a special column like sort with a regular INDEX, not UNIQUE. It does not have to be INT either, you can use real numbers. This way you will always be able to insert new row between any two rows of your schedule.
No, auto-increment is required to be unique, but it is not required to be in any particular order or even contiguous. The fact that auto-increment is monotonically increasing is only by coincidence of its implementation. Don't rely on the values being in chronological order.
Trying to adjust the values is not only manual and awkward, but it risks race conditions, or else would require locking a lot of rows. What if you insert a row with id 5, but your table has 1 billion rows greater than id 5?
There's also a risk of renumbering primary key columns, because any user who got an email telling them that they need to go to game 42 may end up going to the wrong game.
If you have need to view the rows in a particular order (e.g. chronological), then use a DATE column for that, not an auto-increment column.
I receive batches of, say, 100 items that I need to insert into three related MySQL tables: say current, recent, and historical. I want to insert each batch in each table as a group in a single insert statement for speed. The current table has an auto-increment primary key id that I need to obtain for each inserted row and use as the primary key to insert the same row in the recent and historical tables. My idea is to get the current auto_increment value for current, increment it by 100 using alter table current AUTO_INCREMENT=, then insert the 100 rows into current with programmatically set ids from the block that I just "reserved". Then I can use the same 100 reserved id values for the inserts into the recent and historical tables without having to query them again from the current table.
My question: Is there some reason that this is a bad idea? I have seen nothing about it on the web. The closest I have seen on stack overflow is Insert into an auto increment field but that is not quite the same thing. I can see possible threading issues if I tried to do this from more than one thread at a time.
I'm also open to other suggestions on how to accomplish this.
There might be concurrency issues: If another connection inserts values between the time you get the current value and you set the new value, you would get duplicate keys.
I am not aware if that can happen in your situation, however, or if the inserts happen only from your batch described above, and there is never another instance of it running in parallel.
Methinks you shoud decouple the IDs from the 3 tables and using ALTER TABLE sounds very fishy too.
The most proper way I can think of:
in recent and historical, add a colum that references to current ID; don't try to force the primary IDs to be the same.
Acquire a WRITE table lock on current.
Get the auto_increment value X for current.
Insert your 100 records; their IDs should now run from X+1 to X+100.
Release the table lock.
Insert records in recent and historical with the know IDs in the extra column.
Note: I'm not sure if the auto_increment value points to the next ID, or the current highest value. If you use MAX(id) then you should use the code above.
This is [a bit] late, but in case someone else has this same question (as I did):
As Ethan pointed out in his comment, auto_increment is an internal MySQL utility to produce unique keys. Since you have the ability to generate your own id values external to MySQL, I suggest removing the auto_increment overhead from the table (but keep id as PK, for transport to the other tables). You can then insert your own id values along with the data.
Obviously once you do this you'll have to program your own incrementing id values. To retrieve a "starting point" for each batch and maintain the speed of a single batch INSERT call, create another table (I'll call in management) with just a single record of last_id, which is equivalent to, but independent of, max(id) of your three primary tables. Then, each time a new batch is ready to be processed, start a transaction on management with a write lock, read management.last_id, UPDATE management.last_id to (last_id+1)+number in batch, then close the transaction. You now have sequential id values to insert that are reserved for that batch because any future calls to management.last_id will the next-larger set of id values.
The write-locked transaction removes any concurrency issues (as stated in FrankPI's answer) because any other processes attempting to read management must wait for the lock to be removed and will return the value after the UPDATE. This also removes the id ambiguity in JvO's answer: "...IDs should now run from X+1 to X+100", which can be a dangerous assumption.
I have a table in MySQL using InnoDB and a column is there with the name "id".
So my problem is that whenever I delete the last row from the table and then insert a new value, the new value gets inserted after the deleted id.
I mean suppose my id is 32, and I want to delete it and then if I insert a new row after delete, then the column id auto-increments to 33. So the serial format is broken ie,id =30,31,33 and no 32.
So please help me out to assign the id 32 instead of 33 when ever I insert after deleting the last column.
Short answer: No.
Why?
It's unnecessary work. It doesn't matter, if there are gaps in the serial number.
If you don't want that, don't use auto_increment.
Don't worry, you won't run out of numbers if your column is of type int or even bigint, I promise.
There are reasons why MySQL doesn't automatically decrease the autoincrement value when you delete a row. Those reasons are
danger of broken data integrity (imagine multiple users perform deletes or inserts...doubled entries may occur or worse)
errors may occur when you use master slave replication or transactions
and so on ...
I highly recommend you don't waste time on this! It's really, really error prone.
You have two major misunderstandings about how a relational database works:
there is no such thing as the "last row" in a relational database.
The ID (assuming that is your primary key) has no meaning whatsoever. It doesn't matter if the new row is assigned the 33, 35354 or 236532652632. It's just a value to uniquely identify that row.
Do not rely on consecutive values in your primary key column.
And do not try the max(id)+1 approach. It will simply not work in a system with more than one transaction.
You should stop fighting this, even using SELECT max(id) will not fix this properly when using transactional database engine like Innodb.
Why you might ask? Imagine that you have 2 transactions, A and B, that started almost at the same time, both doing INSERT. First transaction A needs new row id, and it will use it from invisible sequence associated with this table (known as AUTOINCREMENT value), say 21. Another transaction B will use another successive value (say 22) - so far so good.
But, what if transaction A rolls back? Value 21 cannot be reused, and 22 is already committed. And what if there were 10 such transactions?
And max(id) can assign the same value to both A and B, so this is not valid as well.
I suppose you mean "Whenever I delete the last row from the table", isn't it?
Anyway this is how autoincrement works. It's made to keep correct data relations. If in another table you use an id of a record that has been deleted it's more correct to get an error instead of get another record when querying that id.
Anyway here you can see how to get the first free id in a field.
I have one of my primary key column in my table to auto-increment. However when I delete a row from the table that has the highest primary key id (lets say 11). Then the next time I do an insertion it inserts the key as 12 not 11 (though logically it can use 11 as there is no entry associated with the key 11). How can I make this happen?
Are you really sure you want this? An autoincrement column will guarantee a unique number, and that's enough. You could update the next autoincrement value I guess (i'll have to look it up how that works), but I don't think you should want that.
If you need to control the numbers in a column, you should do so manually.
nevertheless, you can change the autoincrement number like so:
ALTER TABLE tbl AUTO_INCREMENT = 100;
(from: http://dev.mysql.com/doc/refman/5.0/en/example-auto-increment.html )
Another remark: If you have numbers one to ten, and you remove 5, you cannot easily do this. You can hardly make the next auto_increment 5 because 6 is already there.
So again, while you can do something dirty for your example, it's really hard to do this in a real environment. Maybe start a new question with description of your situation, and ask for advice how to approach that problem without the auto_increment tricks :)
Mysql doesn't have that feature out of the box, you'll need to code it in your application. One problem you'll have is that if 2 transactions want to get and id, one of the them will get a duplicate id error. Of couse, this is better to avoid.
All the DB engines lack this "feature", as it not good for concurrency.