I am trying to make a column in a mysql database that auto increments by one but goes from 0-Z and then rolls.
For example
000, 001, 002, ..., 009, 00A, 00B, ..., 00Z, 010, ..., 0ZZ, ..., 100.
I would like to have the database create the column through an auto incrementing field.
The ideas I have are:
Create a column for each character that goes from 0-36, then auto increment row N (where N is the least significant digit) by 1. Then add a trigger on each column to add 1 to column N-1 when column N reaches 36.
Create a table with 36 rows where each row contains a character 0-Z and pull the appropriate character from the table with similar carry logic from the above
Create a stored procedure to do the appropriate logic from item 1
Have the actual program generate a value and insert it into the table
have a regular auto incrementing value and calculate the next value in the sequence (this is the least optimal as it makes it difficult to parse by a person just looking in the database)
I was hoping that there was something elegant which would allow for this like a built in mechanism to do this that I just do not know. I have no knowledge on stored procedures / triggers so help with it would be greatly appreciated. I think the easiest way would be to have a lookup table for the characters and when row 36 is reached it is reset to 0 and then there is a carry to row N-1.
Based on your comments, my recommendation is to do the following:
Use a regular integer auto_increment column as the primary key for the row, and then have a column of type varchar or one of the *text types (depending on your mysql server version and data storage requirements) to store your "identifier" that the customer uses.
The identifier can be auto-generated using a trigger.
If you're going to do lookups based on the identifier (i.e. perhaps the user enters an identifier to "jump to" a record) you will want an index on that column.
Related
This is a problem that bothers me whenever there is a need to add a new field to a table. Here the table has got about 1.5 Billion records (partitioned and sharded so it is physically separated files). Now I need to add a nullable field which is varchar(1024), which is going to accept some JSON strings. It is possible that the field length has to be increased in future to accommodate longer strings.
Here are the arguments
All existing rows will have null values for this field. (fav. new table)
Only 5% of the newly inserted records will have value for this. (fav. new table )
Most of the current queries on the table will need to access this field. (fav. alter)
I'm not sure if query memory allocation has a role to play in this, based on where I store.
Now should I add to current table, or define another table with same primary keys to store this data.
Your comments would help a decision.
Well if your older records wont need to have that varchar field , you should put it in another table and while pulling data give a join with primary key of other
Its not a big deal you can simply add a column in that table and for that just set null for that new column.
I think that, regardless of the 3 situations you have posited, you should alter the existing table, rather than creating a new one.
My reasoning is as follows:
1) Your table is very large (1.5 billion rows). If you create a new table, you would replicate the PK for 1.5 billion rows in the new table.
This will cause the following problems:
a) Wastage of DB space.
b) Time-intensive. Populating a new table with 1.5 billion rows and updating their PKs is a non-trivial exercise.
c) Rollback-segment exhaustion. If the rollback segments have insufficient space during the insertion of the new rows, the insert will fail. This will increase the DB fragmentation.
On the other hand, all these problems are avoided by altering the table:
1) There is no space wastage.
2) The operation won't be time-consuming.
3) There is no risk of rollback segment failure or DB fragmentation.
So alter the table.
Both these approaches have merits and demerits. I think I found a compromise between these two options., which has benefits of both approaches
create a new table to hold the JSON string. This table has same primary key as first table. Say the first table is Customer, and second table is Customer_json_attributes
alter the current table(customer) to add a flag indicating the presence of value in the JSON field. say json_present_indicator char(1).
Application to set the json_present_indicator='Y' in the fist table if there is a value for the JSON field in the second table, if not set to 'N'
Select queries will have a left join having json_present_indicator = ‘Y’ as a join condition. This will be efficient join as the query will search the second table only when the indicator is ‘Y’. Remember only 5% of the records will have a value on the JSON field
I have an existing schema with a non-auto-incrementing primary key. The key is used as a foreign key in a dozen other tables.
I have inherited a program with major performance problems. Currently, when a new row is added to this table, this is how a new unique id is created:
1) a query for all existing primary key values is retrieved
2) a random number is generated
3) if the number does not exist in the retrieved values, use it, otherwise goto (2)
The app is multi-threaded and multi-server, so simply grabbing the existing ids once at startup isn't an option. I do not have unique information from the initiating request to grab and convert into a pseudo-unique value (like a member id).
I understand it is theoretically possible to perform surgery on the internals to add autoincrementing to an existing primary key. I understand it would also be possible to systematically drop all foreign keys pointing to this table, then create-rename-insert a new version of the table, then add back foreign keys, but this table format is dictated by a third-party app and if I mess this up then Bad Things happen.
Is there a way to leverage sql/mysql to come up with unique row values?
The closest I have come up with is choosing a number randomly from a large space and hoping it is unique in the database, then retrying when the odd collision occurs.
Ideas?
If the table has a primary key that isn't being used for foreign key references, then drop that primary key. The goal is to make your column an auto-incremented primary key.
So, look for the maximum value and then the following should do what you want:
alter table t modify id int not null auto_increment primary key;
alter table t auto_increment = <maximum value> + 1;
I don't think it is necessary to explicitly set the auto_increment value, but I like to be sure.
I think you can SELECT MAX('strange-id-column')+1. That value will be unique and you can put that sql code inside a transaction with the INSERT code in order to prevent errors.
It seems really expensive to pull back a list of all primary key values (for large sets), and then to generate psuedo-random value and verify it's unique, by checking it against the list.
One of the big problems I see with this approach is that a pseudo-random number generator will generate the same sequence of values, when the sequence is started with the same seed value.
If that ever happens, then there will be collision after collision after collision until the sequence reaches a value that hasn't yet been used. And the next time it happens, you'd spin through that whole list again, to add one more value.
I don't understand why the value has to be random.
If there's not a requirement for pseudo-randomness, and an ascending value would be okay, here's what I would do if I didn't want to make any changes to the existing table:
I'd create another "id-generator" table that has an auto_increment column. I perform inserts to that table to generate id values.
Instead of running a query to pull back all existing id values from the existing table, I'd instead perform an INSERT into the "id-generator" table, and then a SELECT LAST_INSERT_ID() to retrieve the id of the row I just inserted, and that would use that as "generated" id value.
Basically, emulating an Oracle SEQUENCE object. It wouldn't be necessary to keep all of the rows in "id-generator" table. So, I could perform a DELETE of all rows that have an id value less than the maximum id value.
If there is a requirement for pseudo-randomness (shudder) I'd probably just attempt the INSERT as a way to find out if the key exists or not. If the insert fails due to a duplicate key, I'd try again with a different id value.
The repeated sequence from a pseudo-random generator scares me... if I got several collisions in a row... are these from a previously used sequence, or are they values from a different sequence. I don't have any way of knowing. Abandoning the sequence and restarting with a new seed, if that seed has been used before, I'm off chasing another series of previously generated values.
For low levels of concurrency (average concurrent ongoing inserts < 1) You can use optimistic locking to achieve a unique id without autoincrement:
set up a one-row table for this function, eg:
create table last_id (last_id bigint not null default 0);
To get your next id, retrieve this value in your app code, apply your newId function, and then attempt to update the value, eg:
select last_id from last_id; // In DB
newId = lastId + 1 // In app code
update last_id set last_id=$newId where last_id=$lastId // In DB
Check the number of rows that were updated. If it was 0, another server beat you to it and you should return to step 1.
I want to create a table, where the fields are labeled with an auto-incremented number at the end, such as:
Comp1
Comp2
Comp3
Is that possible? then if yes, how?
I have a question for you first. Will the word "Comp" ever change?
If it will never change just create a column called id that's auto increment and prefix it as "Comp" in your code.
If the word "Comp" may change you have the option to split into two column. One id and another prefix. You will be querying with the id and prefix in your where clause.
Select * from yourtable where id=2232 and prefix ="Comp";
Another option is what you exactly desire, create a column of type binary(16) and use the hex() and unhex() functions to store and retrieve the id. However you will still need to maintain a separate column for auto incrementing ID. If you don't want to do that then before you insert get the last inserted record and then increment it yourself then insert. But this may have the chance for collision. Be sure to index this field you plan to query it. Get a buy in from your DBAs as they won't be happy as your index will grow larger :)
I have a table in MySQL using InnoDB and a column is there with the name "id".
So my problem is that whenever I delete the last row from the table and then insert a new value, the new value gets inserted after the deleted id.
I mean suppose my id is 32, and I want to delete it and then if I insert a new row after delete, then the column id auto-increments to 33. So the serial format is broken ie,id =30,31,33 and no 32.
So please help me out to assign the id 32 instead of 33 when ever I insert after deleting the last column.
Short answer: No.
Why?
It's unnecessary work. It doesn't matter, if there are gaps in the serial number.
If you don't want that, don't use auto_increment.
Don't worry, you won't run out of numbers if your column is of type int or even bigint, I promise.
There are reasons why MySQL doesn't automatically decrease the autoincrement value when you delete a row. Those reasons are
danger of broken data integrity (imagine multiple users perform deletes or inserts...doubled entries may occur or worse)
errors may occur when you use master slave replication or transactions
and so on ...
I highly recommend you don't waste time on this! It's really, really error prone.
You have two major misunderstandings about how a relational database works:
there is no such thing as the "last row" in a relational database.
The ID (assuming that is your primary key) has no meaning whatsoever. It doesn't matter if the new row is assigned the 33, 35354 or 236532652632. It's just a value to uniquely identify that row.
Do not rely on consecutive values in your primary key column.
And do not try the max(id)+1 approach. It will simply not work in a system with more than one transaction.
You should stop fighting this, even using SELECT max(id) will not fix this properly when using transactional database engine like Innodb.
Why you might ask? Imagine that you have 2 transactions, A and B, that started almost at the same time, both doing INSERT. First transaction A needs new row id, and it will use it from invisible sequence associated with this table (known as AUTOINCREMENT value), say 21. Another transaction B will use another successive value (say 22) - so far so good.
But, what if transaction A rolls back? Value 21 cannot be reused, and 22 is already committed. And what if there were 10 such transactions?
And max(id) can assign the same value to both A and B, so this is not valid as well.
I suppose you mean "Whenever I delete the last row from the table", isn't it?
Anyway this is how autoincrement works. It's made to keep correct data relations. If in another table you use an id of a record that has been deleted it's more correct to get an error instead of get another record when querying that id.
Anyway here you can see how to get the first free id in a field.
I was just creating a new table using MySQL Query Browser, and noticed there's a tick under Auto Increment Column. How does that work?
When adding to the database programatically, do I just add a number, and then the database automatically increments that number?
Everytime a NEW user registers on my site, I want their Customer ID (integer only) to auto increment, so I don't have to try and randomly generate a unique number.
Can this be done simply?
Thank you!
When adding to the database programatically, do I just add a number, and then the database automatically increments that number?
Yes, that's the way auto_increment works.
The value will be incremented for each new row
The value is unique, duplicates are not possible
If a row is deleted, the auto_increment column of that row will not be re-assigned.
The auto_increment value of the last inserted row can be accessed using the mySQL function LAST_INSERT_ID() but it must be called right after the insert query, in the same database connection
mySQL Reference
1 more,
You can insert your own value also (ie your random value).
Yes. Auto_Increment columns work like they say on the tin. Tips
when INSERT - ing, use NULL or omit the column
Use LAST_INSERT_ID() (or API equivalents) to obtain the last generated value.
for security and business logic reasons, it's usually better form to not directly use a key value for a customer identifier. Consider using Hashed / randomised surrogate customer keys instead.
Ta
Yes, that's the exact purpose of AUTO_INCREMENT. It looks at whatever is the current increment value for that table, and stores that value plus 1 for the new row that comes in, automatically. You can omit that field from your INSERT statements and MySQL will handle it for you for every new row that comes in, giving each row its own unique ID.
When you enable Auto Increment an ID will always get automatically added whenever a new record is made.. Example:
If you have 1 record with ID 1 in your table and you add a new record, the ID will automatically be 2.