I'm setting up a table that needs two auto-incrementing fields, 'id' and 'member#'.
I'll use AUTO_INCREMENT = 1001 on the latter for new data, as there is old data with member numbers less than 1000.
I'll use 'MAX(id)+1' on the 'id' field to auto-increment it.
But I'm not sure if this will do the job whenever there's an INSERT, or even where to put that bit of code. All I'm trying to do here is auto-increment the field, not SELECTing anything.
And out of curiosity, why is there only one AUTO_INCREMENTing field per table?
Surely, it can't be difficult to code AUTO_INCREMENT_2, AUTO_INCREMENT_3 etc.
All answers and assistance appreciated.
================================
ADDITIONAL INFORMATION AND LINKS
Sorry for the delay in my response, I've been doing additional research.
Ok so to explain further, we have people joining our group via the net. As such we need to assign a unique membership number to each person. Two John Does? Two different membership numbers. For this I've set the member# column as AUTO_INCREMENT, and then AUTO_INCREMENT = 1001 as a table option. Old membership numbers have three digits, new memberships have four. So each time someone registers as a new member on the web, there's an insert command that automatically assigns the next four digit membership number in the series to the new member.
member# INT(6) UNSIGNED NOT NULL UNIQUE KEY AUTO_INCREMENT
And as a table option AUTO_INCREMENT = 1001
I hope this is clear. Other situations where someone might want to use a similar strategy could be assigning consecutive invoice numbers, receipt numbers, account numbers, etc. So how does one guarantee a +1 result, ie consecutive numbers?
Now we also need a table id column. Lots of tables need a table id. It too needs to be assigned an AUTO_INCREMENT value, in our case, beginning with 1, and incrementing by 1 (the default), to identify and distinguish one row from another. But unfortunately there can be only one AUTO_INCREMENT column per table in MySQL. :-/
So this situation belongs to a class of problems known as MAX+1 problems. (It may also be related to ROW_COUNT and LAST_INSERT_ID solutions.) The limit of a single AUTO_INCREMENT field per table requires a MAX+1 workaround and I am looking for advice on the best way to implement this. For example, is there a way to set this up inside the CREATE TABLE command itself, when defining the id field? Or something else of an equally simple nature, such as writing a function. It is indeed preferable to optimize for efficiency and use only needed features rather than implement a series of commands. Typically a suggested work around might be:
Lock tables membership write;
$max = SELECT MAX(id) FROM membership;
INSERT INTO membership ( id, firstname, lastname )
VALUES ($max+1 , 'jane', 'smith')
unlock tables;
Is there something better?
As whether AUTO_INCREMENT_2 /_3... features should exist. Well, I'd have to point out that there are a lot of features in MySQL that I'll never use, but obviously someone needs them. Nevertheless, it would be convenient to have this for those (rare) occasions when you might need it. Perhaps there is a distinction to be drawn between having a feature available and using it on any given table. I doubt an unused feature requires much in the way of additional memory or clicks (which are pretty cheap these days anyways).
Some links that may prove useful in understanding this situation:
https://duckduckgo.com/?q=mysql+max%2B1+problems&t=ffab&atb=v1-1&ia=web
Insert and set value with max()+1 problems
Problem with MySql INSERT MAX()+1
https://bugs.mysql.com/bug.php?id=3575
All answers, advice and assistance appreciated.
Each InnoDB table has at most one counter for its auto-increment. This is part of the implementation. If you could define N auto-increment columns, in the same table, it would need more storage space to store N counters. It would require the auto-increment lock to last longer while you incremented N counters.
As for why is there only one per table, sure, it is possible that they could implement it to support more than one, but why?
It would make the implementation a lot more complex, and hinder performance, for cases that 99.99% of apps don't need.
They were trying to solve the needs for the majority of cases. In nearly every case of a table with an auto-increment, one per table is sufficient.
In nearly every case where someone like you thinks they need more than one per table, you'd be wise to step back and reconsider your design.
In MySQL the table structure cannot contain more than one auto_increment field. When you try to create a table with 2 autoincremented fields or alter the table in attempt to create second autoincrement, the query fails.
Autoincrement guarantees that each next value generated in the field will be greater than previous one in current connection. But it do NOT guarantee, that each next value generated in the field will be greater than previous value by 1. The "delta" may be 2 or even 1000... it cannot be negative or zero only.
Related
I have got a table which has an id (primary key with auto increment), uid (key refering to users id for example) and something else which for my question won’t matter.
I want to make, lets call it, different auto-increment keys on id for each uid entry.
So, I will add an entry with uid 10, and the id field for this entry will have a 1 because there were no previous entries with a value of 10 in uid. I will add a new one with uid 4 and its id will be 3 because I there were already two entried with uid 4.
...Very obvious explanation, but I am trying to be as explainative an clear as I can to demonstrate the idea... clearly.
What SQL engine can provide such a functionality natively? (non Microsoft/Oracle based)
If there is none, how could I best replicate it? Triggers perhaps?
Does this functionality have a more suitable name?
In case you know about a non SQL database engine providing such a functioality, name it anyway, I am curious.
Thanks.
MySQL's MyISAM engine can do this. See their manual, in section Using AUTO_INCREMENT:
For MyISAM tables you can specify AUTO_INCREMENT on a secondary column in a multiple-column index. In this case, the generated value for the AUTO_INCREMENT column is calculated as MAX(auto_increment_column) + 1 WHERE prefix=given-prefix. This is useful when you want to put data into ordered groups.
The docs go on after that paragraph, showing an example.
The InnoDB engine in MySQL does not support this feature, which is unfortunate because it's better to use InnoDB in almost all cases.
You can't emulate this behavior using triggers (or any SQL statements limited to transaction scope) without locking tables on INSERT. Consider this sequence of actions:
Mario starts transaction and inserts a new row for user 4.
Bill starts transaction and inserts a new row for user 4.
Mario's session fires a trigger to computes MAX(id)+1 for user 4. You get 3.
Bill's session fires a trigger to compute MAX(id). I get 3.
Bill's session finishes his INSERT and commits.
Mario's session tries to finish his INSERT, but the row with (userid=4, id=3) now exists, so Mario gets a primary key conflict.
In general, you can't control the order of execution of these steps without some kind of synchronization.
The solutions to this are either:
Get an exclusive table lock. Before trying an INSERT, lock the table. This is necessary to prevent concurrent INSERTs from creating a race condition like in the example above. It's necessary to lock the whole table, since you're trying to restrict INSERT there's no specific row to lock (if you were trying to govern access to a given row with UPDATE, you could lock just the specific row). But locking the table causes access to the table to become serial, which limits your throughput.
Do it outside transaction scope. Generate the id number in a way that won't be hidden from two concurrent transactions. By the way, this is what AUTO_INCREMENT does. Two concurrent sessions will each get a unique id value, regardless of their order of execution or order of commit. But tracking the last generated id per userid requires access to the database, or a duplicate data store. For example, a memcached key per userid, which can be incremented atomically.
It's relatively easy to ensure that inserts get unique values. But it's hard to ensure they will get consecutive ordinal values. Also consider:
What happens if you INSERT in a transaction but then roll back? You've allocated id value 3 in that transaction, and then I allocated value 4, so if you roll back and I commit, now there's a gap.
What happens if an INSERT fails because of other constraints on the table (e.g. another column is NOT NULL)? You could get gaps this way too.
If you ever DELETE a row, do you need to renumber all the following rows for the same userid? What does that do to your memcached entries if you use that solution?
SQL Server should allow you to do this. If you can't implement this using a computed column (probably not - there are some restrictions), surely you can implement it in a trigger.
MySQL also would allow you to implement this via triggers.
In a comment you ask the question about efficiency. Unless you are dealing with extreme volumes, storing an 8 byte DATETIME isn't much of an overhead compared to using, for example, a 4 byte INT.
It also massively simplifies your data inserts, as well as being able to cope with records being deleted without creating 'holes' in your sequence.
If you DO need this, be careful with the field names. If you have uid and id in a table, I'd expect id to be unique in that table, and uid to refer to something else. Perhaps, instead, use the field names property_id and amendment_id.
In terms of implementation, there are generally two options.
1). A trigger
Implementations vary, but the logic remains the same. As you don't specify an RDBMS (other than NOT MS/Oracle) the general logic is simple...
Start a transaction (often this is Implicitly already started inside triggers)
Find the MAX(amendment_id) for the property_id being inserted
Update the newly inserted value with MAX(amendment_id) + 1
Commit the transaction
Things to be aware of are...
- multiple records being inserted at the same time
- records being inserted with amendment_id being already populated
- updates altering existing records
2). A Stored Procedure
If you use a stored procedure to control writes to the table, you gain a lot more control.
Implicitly, you know you're only dealing with one record.
You simply don't provide a parameter for DEFAULT fields.
You know what updates / deletes can and can't happen.
You can implement all the business logic you like without hidden triggers
I personally recommend the Stored Procedure route, but triggers do work.
It is important to get your data types right.
What you are describing is a multi-part key. So use a multi-part key. Don't try to encode everything into a magic integer, you will poison the rest of your code.
If a record is identified by (entity_id,version_number) then embrace that description and use it directly instead of mangling the meaning of your keys. You will have to write queries which constrain the version number but that's OK. Databases are good at this sort of thing.
version_number could be a timestamp, as a_horse_with_no_name suggests. This is quite a good idea. There is no meaningful performance disadvantage to using timestamps instead of plain integers. What you gain is meaning, which is more important.
You could maintain a "latest version" table which contains, for each entity_id, only the record with the most-recent version_number. This will be more work for you, so only do it if you really need the performance.
I'm looking for an insight as I'm a bit stuck.
Right now I'm using auto increment ID's in all of my MySQL innoDB tables. Even for transaction tables (large volume tables).
In my application, someone enters a financial journal which contains of a few transaction lines which are saved in a transaction table.
Now I get requirement requests that the users want to be able to change those transactions rather then insert a total correction for that journal and insert new lines.
No problem for my procedures, but one question arises for me: what if the number of transaction lines for that journal particular changes? If they are the same I could just overwrite the lines with the current auto increment IDs that they already have.
But what if there are more or less journal lines than the first time they were inserted?
Sure, I can delete all previously inserted lines and insert new ones which the auto increment ID field handles fine. But it leaves a gap in the IDs.
I've read before about peoples opinions about the gap I'm talking about, where things go from 'don't worry about it, you'll never run out' to 'oh my, a problem'.
Is it a bad thing if i leave out the auto increment ID as I do not even need it in this trans-table? I always find the rows in the trans-table through the journal number, which is unique.
Are there any drawbacks with leaving out an auto increment ID field on this table?
All I can think if that it maybe may slow down queries on that table (and am not sure of that), but can't think of anything else.
For reference, my trans table looks somewhat like this:
ID --> auto increment bigint
Journal --> int(11)
JournalLineNumber --> int(3)
Description --> varchar (50)
Amount --> decimal
etc.
etc.
By the waym I still do not know how to insert things like tables for displaying mysql table info in my questions in stackoverflow, does anyone know? Googled it again just now but all kinds of confusing tools seems to be needed to simple enter a table of information....but hopefully I'm wrong.
The important thing is that you have a primary key, and that it is reasonably compact. UUID isn't too great because it's big; bigint is fine. int is great. If your have an int or bigint journal number, you can use that as the PK, and you don't need an auto_increment. As long as you have a PK and it isn't huge, performance will not be negatively impacted.
Note that signed (default) int(11) is only good for up to 2^31 (~2bn). Given that your auto_increment was bigint, will you need more than -2bn to +2bn range for this? Should it be unsigned for 0 to 4bn instead? Is 4bn going to be enough for the foreseeable future? ALTER TABLE on 4bn rows is going to take a while.
Paste the output of SHOW CREATE TABLE. To show it as a block, make sure each line is prefixed with at least 4 spaces. Or just use the formatting toolbar.
If I have a database with the following information, how can I setup my next INSERT query so that the ID is filled in? (so that it is 5 in this instance.)
Basically, once it gets to 24, it will continue inserting in order (ex: 30,31,32)
You don't. Not with an auto-incrementing integer anyway.
You could change the column to not be an auto-incrementing integer, but then you'll need to determine the next ID before performing each insert which would make all of your INSERT queries unnecessarily complex and the code more difficult to maintain. Not to mention introducing a significant point of failure if multiple threads try to insert and the operation to find the next ID and insert a record isn't fully atomic.
Why do you even need this? There's no reason for a database-generated primary key integer to be contiguous like that. Its purpose is to be unique, and as long as it serves that purpose it's working. There's no need to "fill in the holes" left by previously deleted records.
You could add a different column to the database and perform the logic for finding the next contiguous number when inserting records on that column. But you'd still run into the same aforementioned problems of race conditions and unnecessary complexity.
Change your filename to something more meaningful than the id.
I think something like files/uploads/20130515_170349.wv (for the first row) makes a lot of sense (assuming you don't have more than one file per second.
This also has the advantage that ordering the file names alphabetically is chronological order, making it easier to see the newer and older files.
You can just give it the I'd field and value
Insert into table (I'd, etc, etc) values (5, etc, etc);
However I don't think you can do it dynamically. If I'd is auto increment then it'll keep on oncrementinf whether or not previous tuples have been deleted etc.
A bit of introduction: After pondering over what kind of unique ids which will be exposed in URLs and elsewhere to use I've chosen Linear Congruential Generators (http://en.wikipedia.org/wiki/Linear_congruential_generator).
Why not UUIDs or auto-increment?
UUIDs are too long and harder to store in db (the recommended way is to convert them to VARBINARY(16)).
Auto_increment exposes sequence of registrations and additions of new entities and gives an ability to predict next ids. For example, if a service gets popular, users can make multiple registrations to get hold of a nice id and then try to sell such an account, ids will give some kind of status: the earlier the registration the cooler. I prefer to avoid such things.
With LCG the sequence is randomized and I can choose parameters so that possible values will fit nicely into a data type for a particular purpose. For example, use INT UNSIGNED for userids and choose parameters to give a period of 2^32.
The problem is that to generate the next id I need to get the value of the last id:
nextId = (a * lastId + c) % m
As I understand I have to set the very first id myself? Is it important which number I choose?
What is a neat way of generating new ids? Perhaps create a table with a list of last generated ids for each table? Or add an auto_increment column to each table to keep track of last generated id? And how to avoid problems when there are lots of registrations in a short amount of time?
Update1:
I've found one approach which is multi-user safe using info from here: http://dev.mysql.com/doc/refman/5.5/en/information-functions.html#function_last-insert-id
CREATE TABLE sequences (users INT UNSIGNED NOT NULL, posts BIGINT UNSIGNED NOT NULL);
INSERT INTO sequences VALUES(123456,123456789);
And then to get a new ID:
UPDATE sequences SET users=LAST_INSERT_ID((a * users + c) % m);
SELECT LAST_INSERT_ID();
To do this reliably in MySQL you're going to need to write a stored procedure, and use a one-row table with the latest ID in it.
Your stored procedure needs to lock the table, read the latest ID, generate the new ID, update it into the table, unlock the table, and return the new ID to the caller.
You could also keep a multi-row table with the list of IDs you have generated. In that case your stored procedure needs to lock, read the most recently generated ID, generate a new one, insert it into the table, unlock, and return. Obviously in this case you're going to need a reliable way to find the most recently generated ID. Perhaps using an autoincrement column and an ID column would do the trick.
Another way to accomplish what you want is to write a stored procedure that generates a multi-digit random number (I'd go with at least 48 binary digits), then attempts to insert it as the primary key of a table. As long as the insert fails due to key collision, try another random number. These long random numbers are even harder to predict than your LCG sequence.
You must test your stored procedure rigorously under a heavy multi-client load once you've developed it, before you put it into production. If you don't test adequately, you will be sorry. I know by experience this stuff is hard to get right.
The UUID does have the size disadvantage you mention. But it has a very strong advantage: it's been thoroughly tested. You don't need to try to reinvent the wheel if you choose it. (In my experience reinventing wheels, I've come up with some flat tires.)
I am using MySql in phpMyadmin. I have a table which contains a primary key. This primary key is the 'userid' and it is also an "auto increment" field. The application also has a functionality of deleting a particular user with a 'userid'. So after deleting a user when i again create a new user, the 'userid' gets a value of the next integer. i want the table to consider the deletion and assign primary key value, numbers which have been deleted
..
example:
the 'userid' values in the table are - 1,2,3,4,5,6,7....
i deleted userid with value 3.
so now when i create a next record of user, the table should use the userid value '3' as it is no longer in use. how can i do that in phpmyadmin?
i want to do this to keep the no of values of userid minimum. the count may go upto a 5 digit value of the userid. hence if a 2 digit is available to use since its been deleted before, using this 2 digit value will save memory usage of the database
It is entirely possible to assign the ID that is no longer used by explicitely providing it in the next insert you make. AUTO_INCREMENT only assigns an id if you do not supply it yourself.
Be certain though that the ID is really not being used, otherwise the insertion will fail.
That being said, I would discourage doing this. I am not 100% certain, but I think that when you declare an integer in MySQL, it requires integer space, regardless of how many digits the integer has, but I am open to clarification on this point. In any case, I believe the minor benefit of potentially using a little less space is not worth risking failure by tinkering with your IDs.
In my experience, such little things have a tendency to haunt you later on, and I do not see the real benefit.
I suggest looking for other ways to improve memory usage if necessary.