I use transactions in MySQL to store orders. Each order has OrderID (BIGINT), which looks like this: XXXXXX0001, with last four digits incrementing (1620200001, 1620200002, 1620200003, ...).
The transaction works as follows:
start transaction
get new OrderID (increment by 1)
do some stuff
commit/rollback
Saving the transaction can take up to several seconds and if multiple orders are created in very short timespan, duplicate OrderID's can be inserted into database. Before first order is commited, second is assigned the same OrderID, which, at the moment is next in line.
What is best way to prevent this? Having UNIQUE OrderID does not solve it (there would be rollback in second order). I could get rid of transaction and save OrderID quicker, but this leads to other potential problems and does not entirely solve this (just reduces chances of problem happening).
Any help would be appreciated.
Read about AUTO_INCREMENT. Search for it in the manual on CREATE TABLE. It's a long page, but AUTO_INCREMENT is documented about 1/4 of the way down the page.
Briefly, you just declare the primary key with a column option:
CREATE TABLE mytable (
id BIGINT NOT NULL AUTO_INCREMENT PRIMARY KEY,
...other columns...
);
The initial value is 1, or you can make it start at a higher value:
ALTER TABLE mytable AUTO_INCREMENT=1620200001;
A table with an auto-increment column ensures that each concurrent transaction gets a unique, increasing value. There is no race condition, because the INSERT acquires a brief table-lock during which it increments the value. Unlike transaction-based locks, the auto-increment table lock is released immediately. So concurrent sessions don't have to wait for your transaction to finish.
Auto-increment is guaranteed to be unique. That is, the same value will not be allocated to multiple sessions. However, it's not guaranteed to allocate consecutive values. Also, it may allocate a value to one session, but that session decides to rollback its transaction. The value it had allocated is NOT returned to any kind of queue of values, because there has probably other sessions that have allocated the next few values in the meantime. So it's possible to "lose" values and then your table has "gaps" or non-consecutive values.
Do not worry about gaps. These could also happen even if the values were consecutive, because you might delete a row later.
I need to setup a table that will have two auto increment fields. 1 field will be a standard primary key for each record added. The other field will be used to link multiple records together.
Here is an example.
field 1 | field 2
1 1
2 1
3 1
4 2
5 2
6 3
Notice that each value in field 1 has the auto increment. Field 2 has an auto increment that increases slightly differently. records 1,2 and 3 were made at the same time. records 4 and 5 were made at the same time. record 6 was made individually.
Would it be best to read the last entry for field 2 and then increment it by one in my php program? Just looking for the best solution.
You should have two separate tables.
ItemsToBeInserted
id, batch_id, field, field, field
BatchesOfInserts
id, created_time, field, field field
You would then create a batch record, and add the insert id for that batch to all of the items that are going to be part of the batch.
You get bonus points if you add a batch_hash field to the batches table and then check that each batch is unique so that you don't accidentally submit the same batch twice.
If you are looking for a more awful way to do it that only uses one table, you could do something like:
$batch = //Code to run and get 'SELECT MAX(BATCH_ID) + 1 AS NEW_BATCH_ID FROM myTable'
and add that id to all of the inserted records. I wouldn't recommend that though. You will run into trouble down the line.
MySQL only offers one auto-increment column per table. You can't define two, nor does it make sense to do that.
Your question doesn't say what logic you want to use to control the incrementing of the second field you've called auto-increment. Presumably your PHP program will drive that logic.
Don't use PHP to query the largest ID number, then increment it and use it. If you do your system is vulnerable to race conditions. That is, if more than one instance of your PHP program tries that simultaneously, they will occasionally get the same number by mistake.
The Oracle DBMS has an object called a sequence which gives back guaranteed-unique numbers. But you're using MySQL. You can obtain unique numbers with a programming pattern like the following.
First create a table for the sequence. It has an auto-increment field and nothing else.
CREATE TABLE sequence (
sequence_id INT NOT NULL AUTO_INCREMENT,
PRIMARY KEY (`sequence_id`)
)
Then when you need a unique number in your program, issue these three queries one after the other:
INSERT INTO sequence () VALUES ();
DELETE FROM sequence WHERE sequence_id < LAST_INSERT_ID();
SELECT LAST_INSERT_ID() AS sequence;
The third query is guaranteed to return a unique sequence number. This guarantee holds even if you have dozens of different client programs connected to your database. That's the beauty of AUTO_INCREMENT.
The second query (DELETE) keeps the table from getting big and wasting space. We don't care about any rows in the table except for the most recent one.
I'm having problem implementing auto-increment logic in my app. Says I inserts a 'group', and in mysql it has value 10 for its Id, next one would be 11, 12 and so forth.
But once the record (assume it's Id 12) got deleted, the next new item is 12 again. So it may have conflict.
Could possibly makes the auto increment don't repeat the same Int? I want every Id be unique, once it's delete means it never come back.
InnoDB really has this "Feature" or Bug, that the recent auto_increment is NOT stored in the table space. As soon as you restart the MySQL Server, the "auto_increment" value is taken from the highest recent value of the table, thus conflicting with possibly deleted values.
The solution to this is really ugly. You could create a table with the highest unused values per table, in the form
tablename maxvalue
tableA 375
tableB 12
and you could write a Post-Startup-Script, if you manage the MySQL-Server. So after every delete of a row of such a table you would check per AFTER DELETE, if that row was the max-value. That is a bit easier with newer versions of MySQL, since table informations are stored in INFORMATION_SCHEMA, and not only calculated with every select (which means reading INFORMATION_SCHEMA does not fire heavy and blocking queries so often).
You only have to update maxvalue if the deleted row was that max value.
It is a bit easier to update the maxalue on every insert on a row, if that does not slow down the system.
In some cases you have just one table with critical references, and that table has an index, so you can retrieve maxvalue from that table.
All in all that is a big problem with InnoDB, and writing a lot of Triggers just for this single unsaved number auto_increment is really not nice.
I think you not set id is primary key and auto increment
I have a table with some rows, each row has a unique key. When a row is deleted from the table, all rows that are below this row should be 'moved up'. Is there some built in function in MySQL that does this or should I just do it with PHP or perhaps UPDATE table SET id=id-1 WHERE id > deletedid?
Using the last one seems a bit messy.
What would be the best way to do this?
Why do you want to do this? I know it's ugly to have holes in your unique ID sequence, but the downside of invalidating any references to IDs from outside the database is normally very much greater. The normal thing is to just accept the sequence won't be contiguous. If these represent a sequence, consider just sorting by the order rather than expecting the N'th value to have value N (any sort of iteration should provide its own index somewhere for this use).
If the value is one you set yourself, and you definitely want to keep it as having values from 1 to N (N="number of rows"), and you want to keep the sequence of values even if they're not in the order the rows were inserted, then "UPDATE table SET id=id-1 WHERE id > deletedid" is probably the best answer.
If the value is an auto_increment field, and you don't care which numbers go with with rows as long as each row has a number from 1 to N, you can alternatively do ALTER TABLE DROP COLUMN 'columnname' and then ALTER TABLE again to add the column again, and the database will regenerate the ids from 0. (Not necessarily in the same order, though it often is.)
There may be a way to renumber only the rows after that point, but (according to a quick google) it doesn't look like there's anything easier than what you're already planning.
First you have to ensure that the column is not a foreign-key for any other table.
Then you can try this (I am not 100% positive it will work):
DELETE FROM
MyTable
WHERE
id = deletedid;
UPDATE
table
SET
id=id-1
WHERE
id > deletedid
ORDER BY
id
As stated in mysql docs:
If the ORDER BY clause is specified,
the rows are updated in the order that
is specified.
and in this way you ensure uniqueness of the field.
I'm using MySQL's AUTO_INCREMENT field and InnoDB to support transactions. I noticed when I rollback the transaction, the AUTO_INCREMENT field is not rollbacked? I found out that it was designed this way but are there any workarounds to this?
It can't work that way. Consider:
program one, you open a transaction and insert into a table FOO which has an autoinc primary key (arbitrarily, we say it gets 557 for its key value).
Program two starts, it opens a transaction and inserts into table FOO getting 558.
Program two inserts into table BAR which has a column which is a foreign key to FOO. So now the 558 is located in both FOO and BAR.
Program two now commits.
Program three starts and generates a report from table FOO. The 558 record is printed.
After that, program one rolls back.
How does the database reclaim the 557 value? Does it go into FOO and decrement all the other primary keys greater than 557? How does it fix BAR? How does it erase the 558 printed on the report program three output?
Oracle's sequence numbers are also independent of transactions for the same reason.
If you can solve this problem in constant time, I'm sure you can make a lot of money in the database field.
Now, if you have a requirement that your auto increment field never have gaps (for auditing purposes, say). Then you cannot rollback your transactions. Instead you need to have a status flag on your records. On first insert, the record's status is "Incomplete" then you start the transaction, do your work and update the status to "compete" (or whatever you need). Then when you commit, the record is live. If the transaction rollsback, the incomplete record is still there for auditing. This will cause you many other headaches but is one way to deal with audit trails.
Let me point out something very important:
You should never depend on the numeric features of autogenerated keys.
That is, other than comparing them for equality (=) or unequality (<>), you should not do anything else. No relational operators (<, >), no sorting by indexes, etc. If you need to sort by "date added", have a "date added" column.
Treat them as apples and oranges: Does it make sense to ask if an apple is the same as an orange? Yes. Does it make sense to ask if an apple is larger than an orange? No. (Actually, it does, but you get my point.)
If you stick to this rule, gaps in the continuity of autogenerated indexes will not cause problems.
I had a client needed the ID to rollback on a table of invoices, where the order must be consecutive
My solution in MySQL was to remove the AUTO-INCREMENT and pull the latest Id from the table, add one (+1) and then insert it manually.
If the table is named "TableA" and the Auto-increment column is "Id"
INSERT INTO TableA (Id, Col2, Col3, Col4, ...)
VALUES (
(SELECT Id FROM TableA t ORDER BY t.Id DESC LIMIT 1)+1,
Col2_Val, Col3_Val, Col4_Val, ...)
Why do you care if it is rolled back? AUTO_INCREMENT key fields are not supposed to have any meaning so you really shouldn't care what value is used.
If you have information you're trying to preserve, perhaps another non-key column is needed.
I do not know of any way to do that. According to the MySQL Documentation, this is expected behavior and will happen with all innodb_autoinc_lock_mode lock modes. The specific text is:
In all lock modes (0, 1, and 2), if a
transaction that generated
auto-increment values rolls back,
those auto-increment values are
“lost.” Once a value is generated for
an auto-increment column, it cannot be
rolled back, whether or not the
“INSERT-like” statement is completed,
and whether or not the containing
transaction is rolled back. Such lost
values are not reused. Thus, there may
be gaps in the values stored in an
AUTO_INCREMENT column of a table.
If you set auto_increment to 1 after a rollback or deletion, on the next insert, MySQL will see that 1 is already used and will instead get the MAX() value and add 1 to it.
This will ensure that if the row with the last value is deleted (or the insert is rolled back), it will be reused.
To set the auto_increment to 1, do something like this:
ALTER TABLE tbl auto_increment = 1
This is not as efficient as simply continuing on with the next number because MAX() can be expensive, but if you delete/rollback infrequently and are obsessed with reusing the highest value, then this is a realistic approach.
Be aware that this does not prevent gaps from records deleted in the middle or if another insert should occur prior to you setting auto_increment back to 1.
INSERT INTO prueba(id)
VALUES (
(SELECT IFNULL( MAX( id ) , 0 )+1 FROM prueba target))
If the table doesn't contain values or zero rows
add target for error mysql type update FROM on SELECT
If you need to have the ids assigned in numerical order with no gaps, then you can't use an autoincrement column. You'll need to define a standard integer column and use a stored procedure that calculates the next number in the insert sequence and inserts the record within a transaction. If the insert fails, then the next time the procedure is called it will recalculate the next id.
Having said that, it is a bad idea to rely on ids being in some particular order with no gaps. If you need to preserve ordering, you should probably timestamp the row on insert (and potentially on update).
Concrete answer to this specific dilemma (which I also had) is the following:
1) Create a table that holds different counters for different documents (invoices, receipts, RMA's, etc..); Insert a record for each of your documents and add the initial counter to 0.
2) Before creating a new document, do the following (for invoices, for example):
UPDATE document_counters SET counter = LAST_INSERT_ID(counter + 1) where type = 'invoice'
3) Get the last value that you just updated to, like so:
SELECT LAST_INSERT_ID()
or just use your PHP (or whatever) mysql_insert_id() function to get the same thing
4) Insert your new record along with the primary ID that you just got back from the DB. This will override the current auto increment index, and make sure you have no ID gaps between you records.
This whole thing needs to be wrapped inside a transaction, of course. The beauty of this method is that, when you rollback a transaction, your UPDATE statement from Step 2 will be rolled back, and the counter will not change anymore. Other concurrent transactions will block until the first transaction is either committed or rolled back so they will not have access to either the old counter OR a new one, until all other transactions are finished first.
SOLUTION:
Let's use 'tbl_test' as an example table, and suppose the field 'Id' has AUTO_INCREMENT attribute
CREATE TABLE tbl_test (
Id int NOT NULL AUTO_INCREMENT ,
Name varchar(255) NULL ,
PRIMARY KEY (`Id`)
)
;
Let's suppose that table has houndred or thousand rows already inserted and you don't want to use AUTO_INCREMENT anymore; because when you rollback a transaction the field 'Id' is always adding +1 to AUTO_INCREMENT value.
So to avoid that you might make this:
Let's remove AUTO_INCREMENT value from column 'Id' (this won't delete your inserted rows):
ALTER TABLE tbl_test MODIFY COLUMN Id int(11) NOT NULL FIRST;
Finally, we create a BEFORE INSERT Trigger to generate an 'Id' value automatically. But using this way won't affect your Id value even if you rollback any transaction.
CREATE TRIGGER trg_tbl_test_1
BEFORE INSERT ON tbl_test
FOR EACH ROW
BEGIN
SET NEW.Id= COALESCE((SELECT MAX(Id) FROM tbl_test),0) + 1;
END;
That's it! You're done!
You're welcome.
$masterConn = mysql_connect("localhost", "root", '');
mysql_select_db("sample", $masterConn);
for($i=1; $i<=10; $i++) {
mysql_query("START TRANSACTION",$masterConn);
$qry_insert = "INSERT INTO `customer` (id, `a`, `b`) VALUES (NULL, '$i', 'a')";
mysql_query($qry_insert,$masterConn);
if($i%2==1) mysql_query("COMMIT",$masterConn);
else mysql_query("ROLLBACK",$masterConn);
mysql_query("ALTER TABLE customer auto_increment = 1",$masterConn);
}
echo "Done";