I have an existing sql table with 3 columns and 100+ entries/rows. There is an id column with autoincrement.
Now, I want to add 10 new rows at the beginning of the table with id from 1 to 10. But I cannot lose any existing row. So, how do I do it?
One idea that just came to my mind is perhaps I can increase the existing id by adding 10, like 1+10 becomes 11, 25+10 becomes 35, and then I can add rows at the beginning. What will be the script for this IF this is possible?
All you need to do for this is to set the auto_increment for that table to whatever number you need to create space for the new records you want to insert.
For example, if you inserted rows with id's 1-100, you might:
Check the next auto_increment value by running:
select auto_increment as val from information_schema.tables where table_schema='myschema' and table_name='mytable';
Let's assume that value would be 101 (the value that would be used if you inserted a new row). You can "advance" the auto_increment value by running:
alter table myschema.mytable auto_increment = 111;
If you insert a new row like this:
insert into mytable (not_the_id_column) values ('test');
It will get the "next" id of 111. But if you specify id values manually, you are ok in this case as long as you use any value less than 111, so you could insert your desired records like this:
insert into mytable (id, not_the_id_column) values (101, 'test101');
insert into mytable (id, not_the_id_column) values (102, 'test102');
... -- more inserts as needed
Now, you still must take proper precautions when updating PK values, or any value that has dependencies on it (Foreign Key or otherwise), but it is completely legitimate to forcibly advance and/or backfill the id values, as long as the resulting auto_increment value doesn't duplicate one that's already in the table.
I agree with juergen d's comment that you should not do this, but I realize there are situations where this kind of thing must be done.
SELECT MAX(id)-MIN(id)+1 INTO #x FROM theTable;
UPDATE theTable SET id = id + #x;
SELECT MIN(id) INTO #x FROM theTable;
UPDATE theTable SET id = 10 + id - #x;
If the id is the primary key, value collisions within an update can cause MySQL to reject the update. (Hence the pair of updates to avoid such a possibility.)
Edit: Factoring N.B.'s strong objection into this, it would also probably be good to verify the table's next auto-increment value is not going to collide with the updated records after the update is completed. I don't have an appropriate database on hand to verify whether UPDATE statements affect it; and even if they do affect it, you may end up wanting to reduce it so as to not create an unnecessary gap (gaps should ideally not be a problem, but if they are or you are just mildly OCD, it is worth looking into).
What is the best solution for inserting boolean values on database if you want more query performance and minimum losing of memory on select statement.
For example:
I have a table with 36 fields that 30 of them has boolean values (zero or one) and i need to search records using the boolean fields that just have true values.
SELECT * FROM `myTable`
WHERE
`field_5th` = 1
AND `field_12th` = 1
AND `field_20` = 1
AND `field_8` = 1
Is there any solution?
If you want to store boolean values or flags there are basically three options:
Individual columns
This is reflected in your example above. The advantage is that you will be able to put indexes on the flags you intend to use most often for lookups. The disadvantage is that this will take up more space (since the minimum column size that can be allocated is 1 byte.)
However, if you're column names are really going to be field_20, field_21, etc. Then this is absolutely NOT the way to go. Numbered columns are a sign you should use either of the other two methods.
Bitmasks
As was suggested above you can store multiple values in a single integer column. A BIGINT column would give you up to 64 possible flags.
Values would be something like:
UPDATE table SET flags=b'100';
UPDATE table SET flags=b'10000';
Then the field would look something like: 10100
That would represent having two flag values set. To query for any particular flag value set, you would do
SELECT flags FROM table WHERE flags & b'100';
The advantage of this is that your flags are very compact space-wise. The disadvantage is that you can't place indexes on the field which would help improve the performance of searching for specific flags.
One-to-many relationship
This is where you create another table, and each row there would have the id of the row it's linked to, and the flag:
CREATE TABLE main (
main_id INT UNSIGNED NOT NULL AUTO_INCREMENT PRIMARY KEY,
);
CREATE TABLE flag (
main_id INT UNSIGNED NOT NULL,
name VARCHAR(16)
);
Then you would insert multiple rows into the flag table.
The advantage is that you can use indexes for lookups, and you can have any number of flags per row without changing your schema. This works best for sparse values, where most rows do not have a value set. If every row needs all flags defined, then this isn't very efficient.
For performance comparisson you can read a blog post I wrote on the topic:
Set Performance Compare
Also when you ask which is "Best" that's a very subjective question. Best at what? It all really depends on what your data looks like and what your requirements are and how you want to query it.
Keep in mind that if you want to do a query like:
SELECT * FROM table WHERE some_flag=true
Indexes will only help you if few rows have that value set. If most of the rows in the table have some_flag=true, then mysql will ignore indexes and do a full table scan instead.
How many rows of data are you querying over? You can store the boolean values in an integer value and use bit operations to test for them them. It's not indexable, but storage is very well packed. Using TINYINT fields with indexes would pick one index to use and scan from there.
I have a table: test
id int(10) auto-increment
name char(36)
Now let us say my WHOLE table it filled from ID 1000 => max unique id number.
Id 1 - 1000 = deleted previously.
Question 1; WILL mysql re-use these deleted id's?
Question 2; If not, how to I go about having auto-increment or whatever to re-use unique identifier that does not exist in table?
The reason I am asking, is that my table will consist of alot of entries, and that alot of entries will be deleted all the time. What happens when I "run-out-of-id" when using auto-increment?
Thanks for any enlightment on this :)
-Tom
WILL mysql re-use these deleted id's?
When mysqld starts, it determines the next value for every AUTO_INCREMENT column by finding the maximum of the incumbent records (and adding 1). Therefore, if you delete the record with the highest value and restart the server, the deleted id will indeed be reused.
Otherwise, values will only be reused if you manually alter the next AUTO_INCREMENT value (this is not recommended as it is not concurrency-safe):
ALTER TABLE foo AUTO_INCREMENT = 12345;
If not, how to I go about having auto-increment or whatever to re-use unique identifier that does not exist in table?
Generally speaking, you don't: consider redesigning your data structure so that inserts/deletes do not happen in this fashion, or else use a larger integer type (BIGINT UNSIGNED is 8 bytes, so can go up to 2^64 or ~10^19).
What happens when I "run-out-of-id" when using auto-increment?
As stated in the manual:
Use the smallest integer data type for the AUTO_INCREMENT column that is large enough to hold the maximum sequence value you will need. When the column reaches the upper limit of the data type, the next attempt to generate a sequence number fails.
No, MySQL won't reuse the IDs from deleted records
Do you really need to? If the type of your autoincrement column is BIGINT, you've got 18446744073709551615 possible IDs
you need to reset the autoincrement, Autoincrement just keep incrementing and won't go back at least if you don't set it.
alter table tablename auto_increment=value
like this
mysql> alter table t1 auto_increment=200;
Query OK, 202 rows affected (0.04 sec)
Records: 202 Duplicates: 0 Warnings: 0
EDIT:
if you delete some records, the auto_increment will be "last value+1", it doesn't matter what you do, only if you delete the full table you'll be able to 'start over'.
The best would be with a trigger, but triggers can't alter tables (http://dev.mysql.com/doc/refman/5.0/en/create-trigger.html)
So your last option is a store procedure take a look here: Is it possible to alter a table in mySql via stored procedures?
Which is not recommended.
I have a fairly large table with about 250k rows. It has an auto incremented ID column that is really sort of useless. I can't just get rid of the column without rewriting too much of the app, but the ID is never used as a foreign key or anything else (except simply as an identifier when you want to delete a row, I guess).
The majority of the data gets deleted and rewritten at least a few times a day (don't ask! it's not important, though I realize it's poor design!), though the total count of the rows stays fairly uniform. What this means is that each day to AI # increases by a quarter million or so.
My question is this: in several years' time, the ID column will get too large for the INT value. Is there a way to "reset" the ID, like an OPTIMIZE or something, or should I just plan on doing a SELECT INTO a temp table and truncating the original table, resetting the ID to 0?
Thanks
If you have the id as integer you can have 2^32 / 2 (2.147.483.647) rows, if is unsigned integer duplicate to 4.294.967.295, no worry 250.000 in nothing, if you want more, use unsigned bigint (18.446.744.073.709.551.615) :P
For reset the auto_numeric position:
ALTER TABLE table AUTO_INCREMENT = 1
Either change the datatype of ID to BIGINT and adjust your program accordingly, or if you're clearing everything out when you delete data you can use TRUNCATE TABLE TABLENAME which will reset the sequence.
Easiest and fastest :) Just drop the index, set autoincrement=1, and add it back :)
ALTER TABLE yourtable DROP id_field;
ALTER TABLE yourtable AUTO_INCREMENT=1;
ALTER TABLE yourtable ADD id_field INT NOT NULL AUTO_INCREMENT FIRST, ADD PRIMARY KEY (id_field);
I'm using MySQL's AUTO_INCREMENT field and InnoDB to support transactions. I noticed when I rollback the transaction, the AUTO_INCREMENT field is not rollbacked? I found out that it was designed this way but are there any workarounds to this?
It can't work that way. Consider:
program one, you open a transaction and insert into a table FOO which has an autoinc primary key (arbitrarily, we say it gets 557 for its key value).
Program two starts, it opens a transaction and inserts into table FOO getting 558.
Program two inserts into table BAR which has a column which is a foreign key to FOO. So now the 558 is located in both FOO and BAR.
Program two now commits.
Program three starts and generates a report from table FOO. The 558 record is printed.
After that, program one rolls back.
How does the database reclaim the 557 value? Does it go into FOO and decrement all the other primary keys greater than 557? How does it fix BAR? How does it erase the 558 printed on the report program three output?
Oracle's sequence numbers are also independent of transactions for the same reason.
If you can solve this problem in constant time, I'm sure you can make a lot of money in the database field.
Now, if you have a requirement that your auto increment field never have gaps (for auditing purposes, say). Then you cannot rollback your transactions. Instead you need to have a status flag on your records. On first insert, the record's status is "Incomplete" then you start the transaction, do your work and update the status to "compete" (or whatever you need). Then when you commit, the record is live. If the transaction rollsback, the incomplete record is still there for auditing. This will cause you many other headaches but is one way to deal with audit trails.
Let me point out something very important:
You should never depend on the numeric features of autogenerated keys.
That is, other than comparing them for equality (=) or unequality (<>), you should not do anything else. No relational operators (<, >), no sorting by indexes, etc. If you need to sort by "date added", have a "date added" column.
Treat them as apples and oranges: Does it make sense to ask if an apple is the same as an orange? Yes. Does it make sense to ask if an apple is larger than an orange? No. (Actually, it does, but you get my point.)
If you stick to this rule, gaps in the continuity of autogenerated indexes will not cause problems.
I had a client needed the ID to rollback on a table of invoices, where the order must be consecutive
My solution in MySQL was to remove the AUTO-INCREMENT and pull the latest Id from the table, add one (+1) and then insert it manually.
If the table is named "TableA" and the Auto-increment column is "Id"
INSERT INTO TableA (Id, Col2, Col3, Col4, ...)
VALUES (
(SELECT Id FROM TableA t ORDER BY t.Id DESC LIMIT 1)+1,
Col2_Val, Col3_Val, Col4_Val, ...)
Why do you care if it is rolled back? AUTO_INCREMENT key fields are not supposed to have any meaning so you really shouldn't care what value is used.
If you have information you're trying to preserve, perhaps another non-key column is needed.
I do not know of any way to do that. According to the MySQL Documentation, this is expected behavior and will happen with all innodb_autoinc_lock_mode lock modes. The specific text is:
In all lock modes (0, 1, and 2), if a
transaction that generated
auto-increment values rolls back,
those auto-increment values are
“lost.” Once a value is generated for
an auto-increment column, it cannot be
rolled back, whether or not the
“INSERT-like” statement is completed,
and whether or not the containing
transaction is rolled back. Such lost
values are not reused. Thus, there may
be gaps in the values stored in an
AUTO_INCREMENT column of a table.
If you set auto_increment to 1 after a rollback or deletion, on the next insert, MySQL will see that 1 is already used and will instead get the MAX() value and add 1 to it.
This will ensure that if the row with the last value is deleted (or the insert is rolled back), it will be reused.
To set the auto_increment to 1, do something like this:
ALTER TABLE tbl auto_increment = 1
This is not as efficient as simply continuing on with the next number because MAX() can be expensive, but if you delete/rollback infrequently and are obsessed with reusing the highest value, then this is a realistic approach.
Be aware that this does not prevent gaps from records deleted in the middle or if another insert should occur prior to you setting auto_increment back to 1.
INSERT INTO prueba(id)
VALUES (
(SELECT IFNULL( MAX( id ) , 0 )+1 FROM prueba target))
If the table doesn't contain values or zero rows
add target for error mysql type update FROM on SELECT
If you need to have the ids assigned in numerical order with no gaps, then you can't use an autoincrement column. You'll need to define a standard integer column and use a stored procedure that calculates the next number in the insert sequence and inserts the record within a transaction. If the insert fails, then the next time the procedure is called it will recalculate the next id.
Having said that, it is a bad idea to rely on ids being in some particular order with no gaps. If you need to preserve ordering, you should probably timestamp the row on insert (and potentially on update).
Concrete answer to this specific dilemma (which I also had) is the following:
1) Create a table that holds different counters for different documents (invoices, receipts, RMA's, etc..); Insert a record for each of your documents and add the initial counter to 0.
2) Before creating a new document, do the following (for invoices, for example):
UPDATE document_counters SET counter = LAST_INSERT_ID(counter + 1) where type = 'invoice'
3) Get the last value that you just updated to, like so:
SELECT LAST_INSERT_ID()
or just use your PHP (or whatever) mysql_insert_id() function to get the same thing
4) Insert your new record along with the primary ID that you just got back from the DB. This will override the current auto increment index, and make sure you have no ID gaps between you records.
This whole thing needs to be wrapped inside a transaction, of course. The beauty of this method is that, when you rollback a transaction, your UPDATE statement from Step 2 will be rolled back, and the counter will not change anymore. Other concurrent transactions will block until the first transaction is either committed or rolled back so they will not have access to either the old counter OR a new one, until all other transactions are finished first.
SOLUTION:
Let's use 'tbl_test' as an example table, and suppose the field 'Id' has AUTO_INCREMENT attribute
CREATE TABLE tbl_test (
Id int NOT NULL AUTO_INCREMENT ,
Name varchar(255) NULL ,
PRIMARY KEY (`Id`)
)
;
Let's suppose that table has houndred or thousand rows already inserted and you don't want to use AUTO_INCREMENT anymore; because when you rollback a transaction the field 'Id' is always adding +1 to AUTO_INCREMENT value.
So to avoid that you might make this:
Let's remove AUTO_INCREMENT value from column 'Id' (this won't delete your inserted rows):
ALTER TABLE tbl_test MODIFY COLUMN Id int(11) NOT NULL FIRST;
Finally, we create a BEFORE INSERT Trigger to generate an 'Id' value automatically. But using this way won't affect your Id value even if you rollback any transaction.
CREATE TRIGGER trg_tbl_test_1
BEFORE INSERT ON tbl_test
FOR EACH ROW
BEGIN
SET NEW.Id= COALESCE((SELECT MAX(Id) FROM tbl_test),0) + 1;
END;
That's it! You're done!
You're welcome.
$masterConn = mysql_connect("localhost", "root", '');
mysql_select_db("sample", $masterConn);
for($i=1; $i<=10; $i++) {
mysql_query("START TRANSACTION",$masterConn);
$qry_insert = "INSERT INTO `customer` (id, `a`, `b`) VALUES (NULL, '$i', 'a')";
mysql_query($qry_insert,$masterConn);
if($i%2==1) mysql_query("COMMIT",$masterConn);
else mysql_query("ROLLBACK",$masterConn);
mysql_query("ALTER TABLE customer auto_increment = 1",$masterConn);
}
echo "Done";