I use one table withe some casual columns such as id, name, email, etc...also I'm inserting a variable numbers of records in each transaction, to be much efficient I need to have one unique id lets call it transaction id, that would be the same for each group of data which are inserted in one transaction, should be increment. What is the best approach for doing that?
I was thought to use
select max(transaction_id) from users
and increment that value on server side, but that seams like old fashion solution.
You could have another table usergroups with an auto-incrementing primary key, you first insert a record there (maybe including some other useful information about the group). Then get the group's unique id generated during this last insert using mysql_insert_id(), and use that as the groupid for your inserts into the first table.
This way you're still using MySQL's auto-numbering which guarantees you a unique groupid. Doing select max(transaction_id) from users and incrementing this isn't safe, since it's non-atomic (another thread may have read the same max(transaction_id) before you've had a change to increment it, and will start inserting records with a conflicting groupid).
Add new table with auto_increment column
You can create new table with auto_increment column. So you'll be able to generate unique integers in thread safe way. It'll work like this:
DB::insert_into_transaction_table()
transaction_id = DB::mysql_last_insert_id() ## this is integer value
for each record:
DB::insert_into_table(transaction_id, ...other parameters...)
And you don't require mysql transactions for this.
Generate unique string on server side before inserting
You can generate unique id (for example GUID) on server side and use it for all records inserting. But your transaction_id field should be long enough to store values generated this way (some char(...) type). It'll work like this:
transaction_id = new_GUID() ## this is usually a string value
for each record:
DB::insert_into_table(transaction_id, ...other parameters...)
Related
I have a table that looks something like:
name: posts
columns:
- id
- sequence_id
- text
- like_count
The ID is a standard auto-incremented unique integer index.
The sequence ID should be similar — it is also a unique integer index.
The difference is that I want to increment it to the new maximum value in the table on update or insert not just insert.
Currently I accomplish this with a Redis counter that I increment before inserting into the database.
I’d like to drop the Redis dependency, though, and do this with purely MySQL if possible.
One option I thought of was creating a post_updates table which just has an auto-incrementing ID which I use the same way but this feels worse.
Another option is doing a full column scan to do max(sequence_id) + 1 but that isn’t really scalable and it would have race conditions.
Are there some better options I’m not aware of?
There's a solution in the manual to simulate a sequence object in MySQL:
CREATE TABLE sequence (id INT NOT NULL);
INSERT INTO sequence VALUES (0);
The sequence table doesn't need an auto-increment itself, and it stores only one row.
When you are ready to increment your sequence_id in the table you describe, you first update the sequence value in the following manner:
UPDATE sequence SET id = LAST_INSERT_ID(id+1);
Now you can use that value when inserting/updating your table:
INSERT INTO posts SET sequence_id = LAST_INSERT_ID(), text = '...';
or
UPDATE posts SET sequence_id = LAST_INSERT_ID(), like_count = like_count+1;
Note that this is a little bit more heavy than an auto-increment, because updating the sequence table creates a row lock, not just an auto-increment lock. This could put an upper limit on the rate of traffic, because many concurrent clients trying to access this table would queue up on each other.
If you want a very high-throughput solution, I'd recommend to continue using Redis.
If I delete the max IDs (for example 9, 10,11) from the table, the sql will not begin from the max existing ID ! it begins with 12!
is it possible to let the sql add new id according to existing max id?
how to get the max primary key autoincrement in the table (the deleted one too)?
This code finds the existing only and not the deleted ids:
select max(id) from table1
If I delete the max IDs (for example 9, 10,11) from the table, the sql will not begin from the max existing ID ! it begins with 12 !
That is correct. There is nothing surprising or exclamation-worthy about this. That's how RDBMSes work.
1- is it possible to let the sql add new id according to existing max id ?
Not easily at all. As a matter of fact, it is so difficult, and any attempt to do it would be so tied to the particular RDBMS that you are using, (that is, so not portable,) that you are advised to not even try. That's because RDBMSes are built for highly concurrent use, so they have to be able to prevent the possibility of different clients inserting inconsistent primary key values.
2- how to get the max primary key autoincrement in the table
There really is no such thing as a "max primary key autoincrement in the table", because it may be changing at a very fast rate as someone is inserting rows into the table, so by the time your select max(id) from table1 would return a value to you, the actual max value in the database may already be different.
I need to setup a table that will have two auto increment fields. 1 field will be a standard primary key for each record added. The other field will be used to link multiple records together.
Here is an example.
field 1 | field 2
1 1
2 1
3 1
4 2
5 2
6 3
Notice that each value in field 1 has the auto increment. Field 2 has an auto increment that increases slightly differently. records 1,2 and 3 were made at the same time. records 4 and 5 were made at the same time. record 6 was made individually.
Would it be best to read the last entry for field 2 and then increment it by one in my php program? Just looking for the best solution.
You should have two separate tables.
ItemsToBeInserted
id, batch_id, field, field, field
BatchesOfInserts
id, created_time, field, field field
You would then create a batch record, and add the insert id for that batch to all of the items that are going to be part of the batch.
You get bonus points if you add a batch_hash field to the batches table and then check that each batch is unique so that you don't accidentally submit the same batch twice.
If you are looking for a more awful way to do it that only uses one table, you could do something like:
$batch = //Code to run and get 'SELECT MAX(BATCH_ID) + 1 AS NEW_BATCH_ID FROM myTable'
and add that id to all of the inserted records. I wouldn't recommend that though. You will run into trouble down the line.
MySQL only offers one auto-increment column per table. You can't define two, nor does it make sense to do that.
Your question doesn't say what logic you want to use to control the incrementing of the second field you've called auto-increment. Presumably your PHP program will drive that logic.
Don't use PHP to query the largest ID number, then increment it and use it. If you do your system is vulnerable to race conditions. That is, if more than one instance of your PHP program tries that simultaneously, they will occasionally get the same number by mistake.
The Oracle DBMS has an object called a sequence which gives back guaranteed-unique numbers. But you're using MySQL. You can obtain unique numbers with a programming pattern like the following.
First create a table for the sequence. It has an auto-increment field and nothing else.
CREATE TABLE sequence (
sequence_id INT NOT NULL AUTO_INCREMENT,
PRIMARY KEY (`sequence_id`)
)
Then when you need a unique number in your program, issue these three queries one after the other:
INSERT INTO sequence () VALUES ();
DELETE FROM sequence WHERE sequence_id < LAST_INSERT_ID();
SELECT LAST_INSERT_ID() AS sequence;
The third query is guaranteed to return a unique sequence number. This guarantee holds even if you have dozens of different client programs connected to your database. That's the beauty of AUTO_INCREMENT.
The second query (DELETE) keeps the table from getting big and wasting space. We don't care about any rows in the table except for the most recent one.
I am facing a problem in my application. I have a table that one field name is registration_no. Before inserting a new record i increment registration_no field by 1 and then insert that incremented registration_no in that table. the problem is when some user concurrently insert data some registration_no value has been same. how can i prevent this.
You want to use a sequence.
Two caveats:
The AUTO_INCREMENT feature described in the article is non-standard and may give portability issues when moving to a different database.
If an INSERT is aborted, a number from the sequence is consumed still, so you may end up with holes in the sequence. If that is unacceptable, use an autogenerated sequence for the primary (surrogate) key, and add a separate map from that key to the "official" sequence number, enforcing uniqueness in the index of that table.
The alternative is to enforce UNIQUEness in the database, use an appropriate TRANSACTION ISOLATION LEVEL and add application logic to handle failure to INSERT.
You could have the database set the registration_no for you, and not do this in code. You can get the registration_no in the result of an insert statement and this will solve your concurrency problem.
alter table myTable modify column registration_no int auto_increment
The result of your query will be the index of the record. IF registration_no is not the index your will need to query the auto generated registration_no based on the returned index id.
I want to generate customer ids for invoices and thus don't want to start counting from 1 for obvious reasons. In MySQL can you generate a random number that is unique?
I know about the RAND() function, but it does not guarantee uniqueness. What's the right approach for this?
Doesn't work:
INSERT INTO test (number) VALUES (FLOOR(0 + (RAND() * 9999999999)));
PS: Server side I'm using PHP and generate invoices using FPDF.
I suggest an AUTO_INCREMENT column and seed the value at 10 digits. You could have it be the only column in the table, like below, or more practically seed your invoice table id.
CREATE TABLE tablename (
id bigint unsigned not null auto_increment,
primary key(id),
auto_increment=1000000000
);
Actually, the reasons aren't that obvious unless it's simply that you don't want your customers to know there are so few that they all have one-digit IDs :-)
As an aside, the customer ID is usually generated when adding a customer rather than doing an invoice. If you meant an invoice number, that's different, but the reasoning will be the same.
Either populate the table with a dummy entry with a suitable lower ID (314159 for example) then use something like:
insert into test (number) select max(number)+1 from test
or use an auto incrementing key with a suitable starting value (again, this value is up to you but you should choose something relatively "non-round").
I've sometimes used the former for situations where I want total control over what values are assigned (including the ability to easily change them) but a more robust solution would probably be the auto-increment one.
ALTER TABLE my_invoice_table auto_increment=1000000001
Incrementing invoice numbers are fine - they only have to be unique, not random.