I have the following sql code to create a table
CREATE TABLE db.object (
`objid` bigint(20) NOT NULL AUTO_INCREMENT,
`object_type` varchar(32) NOT NULL,
PRIMARY KEY (`objid`)
) ENGINE=InnoDB AUTO_INCREMENT=2 DEFAULT CHARSET=utf8;
However, the values in the objid are coming out as 1,2,3... (The insert statement is not adding the ids)
Shouldn't AUTO_INCREMENT=2 make the objid start from 2 instead of 1
With InnoDB tables, the AUTO_INCREMENT value will be reset to the maximum value (plus 1) when the table is opened. The auto increment value exists only in memory, it is not persisted on disk.
A table open would happen, for example, when the MySQL instance was shutdown and then restarted, and a reference is made to the table.
A table can also be closed at other times. For example, when open_table_cache is exceeded (that is, when a large number of other tables is opened), MySQL will close some of the open tables, to make room in the cache for newly opened tables.
I believe this behavior is documented somewhere in the MySQL Reference Manual.
I used your SQL, created the object table and entered two values for object_type and objid started at 2. Can't see anything wrong here...
It might. There are enough exceptions and gotchas with auto-inc on InnoDB tables that it bears urging a full review of the documentation.
That said, there is one scenario I can think of where MySQL ignores the initializer value. I'll quote the documentation:
InnoDB uses the in-memory auto-increment counter as long as the server runs. When the server is stopped and restarted, InnoDB reinitializes the counter for each table for the first INSERT to the table, as described [here]:
InnoDB executes the equivalent of the following statement on the first insert into a table containing an AUTO_INCREMENT column after a restart:
SELECT MAX(ai_col) FROM table_name FOR UPDATE;
A server restart also cancels the effect of the AUTO_INCREMENT = N table option in CREATE TABLE and ALTER TABLE statements, which you can use with InnoDB tables to set the initial counter value or alter the current counter value.
So if you create that table, then do a server restart (like as part of a deployment process), you'll get a nice value of 1 for the initial row. If you want to countermand this, you need to create the table, then insert a dummy row with the auto-inc value you want, then restart, then delete the dummy row.
Related
http://dev.mysql.com/doc/refman/5.0/en/example-auto-increment.html
That document I'm reading seems to say something like:
"In this case (when the AUTO_INCREMENT column is part of a multiple-column index), AUTO_INCREMENT values are reused if you delete the row with the biggest AUTO_INCREMENT value in any group."
I don't really understand what's being said there. Aren't the values supposed to be reused automatically?
Thanks in advance...
InnoDB resets the auto_increment field when you restart the database.
When InnoDB restarts, it finds the highest value in the column and then starts from there.
This won't happen in MyISAM because it caches the last incremented id.
Update
This feature/bug has been around since 2003 and can lead to serious issues. Take the example below,
Table t1 has an auto-inc primary key.
Table t2 has a column for the primary key in t1 without a foreign key "constraint". In other words, when a row is deleted in t1 the corresponding rows in t2 are orphaned.
As we know with InnoDB restart, an id can be re-issued. Therefore orphaned rows in t2 can be falsely linked to new rows in t1.
This bug has been finally fixed in MySQL 8.0.0 WL#6204 - InnoDB
persistent max value for autoinc columns.
InnoDB will keep track of the maximum value and on restart preserve
that max value and start from there.
When you have a primary key field that is also auto_increment, no matter how many rows (or in what order) are removed, the ids that no longer exist are not used again, and the field is incremented continuously for each row.
However, when your primary key consists of multiple fields (E.G. an auto_increment field and another field), the auto_increment field is reused, and only when the deleted id is the last id. What this means is if you have values like 1, 2, 3, 4, and 5, and you remove the row with the field value of 5, the next row will have an id of 5. However, if you remove the row with the id of 2, this will, again, not be used, and the next row will have an id of 6.
As described with the InnoDB engine the last PK utilised will be reused if the row is deleted prior to a reboot as it is not cached. If this PK is a Foreign Key (FK) in another table problems can arise (FK hell). This is how I discovered the problem when a little old lady shot up to 195 cm(!) as the deleted person's additional data was picked up. The work around for this is either to implement 'ON DELETE CASCADE' for the child tables or (not my preferred option) to code around this issue.
As mentioned in answer :
InnoDB resets the auto_increment field when you restart the database.
When InnoDB restarts, it finds the highest value in the column and then start from there.
and this behavior could lead to Foreign key problem.
Fortunately from MySQL 8.0.0 it will be fixed.
More info:
Bug #199 Innodb autoincrement stats los on restart
WL#6204: InnoDB persistent max value for autoinc columns
See BUG#199 on MySQL bugs.
Currently InnoDB does the following when a table
is opened: SELECT MAX(c) FROM t; where c is the AUTOINC column name.
This step is used to initialise the column's next autoinc value and
allocation of autoinc values starts from this point. InnoDB also does this
when it executes 'ALTER TABLE AUTO_INCREMENT=N;'.
...
InnoDB should keep track of the maximum value and on restart preserve
that max value and start from there.
If the table is of InnoDB type, then the deleted ids are reused in certain scenarios. Here is what I did to test it:
Table user has id column as primary key, with auto_increment set
CREATE TABLE user (
id bigint(20) NOT NULL AUTO_INCREMENT,
PRIMARY KEY (id),
) ENGINE=InnoDB AUTO_INCREMENT=2108 DEFAULT CHARSET=latin1;
My createUser API call, creates a entry in user table. Lets say user with id = 1 was created.
Call deleteUser API which just deletes row from user table.
Stop and start the database
Call createUserAPI call again. It creates user with same id (id = 1).
So, it does uses the deleted id, if the server is restarted.
I got this issue because I had another table called user_rules which stored user_id in a column, but it did not have FK reference to user table (which was wrong). Another issue was that when user was getting deleted, entry from user_rules table was not getting deleted and since it had no FK set, db also didn't complain.
And when server got restarted, user rules were messed up!
I've been using InnoDB for a project, and relying on auto_increment. This is not a problem for most of the tables, but for tables with deletion, this might be an issue:
AUTO_INCREMENT Handling in InnoDB
particularly this part:
AUTO_INCREMENT column named ai_col: After a server startup, for the first insert into a table t, InnoDB executes the equivalent of this statement:
SELECT MAX(ai_col) FROM t FOR UPDATE;
InnoDB increments by one the value retrieved by the statement and assigns it to the column and to the auto-increment counter for the table.
This is a problem because while it ensures that within the table, the key is unique, there are foreign keys to this table where those keys are no longer unique.
The mysql server does/should not restart often, but this is breaking. Are there any easy ways around this?
If you have a foreign key constraint, how can you delete a row from table A when table B references that row? That seems like an error to me.
Regardless, you can avoid the reuse of auto-increment values by resetting the offset when your application starts back up. Query for the maximum in all the tables that reference table A, then alter the table above that maximum, e.g. if the max is 989, use this:
alter table TableA auto_increment=999;
Also beware that different MySQL engines have different auto-increment behavior. This trick works for InnoDB.
So you have two tables:
TableA
A_ID [PK]
and
TableB
B_ID [PK]
A_ID [FK, TableA.A_ID]
And in TableB, the value of A_ID is not unique? Or is there a value in TableB.A_ID that is not in TableA.A_ID?
If you need the value of TableB.A_ID to be unique, then you need to add a UNIQUE constraint to that column.
Or am I still missing something?
Use a foreign key constraint with 'SET NULL' for updates and deletes.
Create another table with a column that remembers the last created Id. This way you don't have to take care of the max values in new tables that have this as foreign key.
I checked.
alter table TableA auto_increment=1;
does NOT work.
And the reason I found in two documents
http://docs.oracle.com/cd/E17952_01/refman-5.1-en/innodb-auto-increment-handling.html
InnoDB uses the following algorithm to initialize the auto-increment counter for a table t that contains an AUTO_INCREMENT column named ai_col: After a server startup, for the first insert into a table t, InnoDB executes the equivalent of this statement:
SELECT MAX(ai_col) FROM t FOR UPDATE;
InnoDB increments the value retrieved by the statement and assigns it to the column and to the auto-increment counter for the table. By default, the value is incremented by one. This default can be overridden by the auto_increment_increment configuration setting.
and
http://docs.oracle.com/cd/E17952_01/refman-5.1-en/alter-table.html
You cannot reset the counter to a value less than or equal to any that have already been used.
This is the reason why alter table will not work. I think that only option is to wipe out data and rewrite it in a new table with new id.
In my case table was logfile , so I just did:
RENAME TABLE SystemEvents To SystemEvents_old;
CREATE TABLE SystemEvents LIKE SystemEvents_old;
I am using MariaDB 10.1.9. Short version: What I really want to know for certain is if I can modify an indexed auto_increment field on an innodb table from int to bigint without locking the table?
Long version: Is there a detailed explanation of which ALTER TABLE operations require which lock level? The documentation just says "Depending on the required operation and the used storage engine, different lock strategies can be used for ALTER TABLE.". It doesn't provide a link to any details and each operation on the ALTER TABLE page does not specify it's required level.
From experimentation, I know ADD COLUMN does not require a lock. MODIFY COLUMN allows reads, but can it be manually set to allow writes? The MariaDB documentation says you can set the lock level, but if you don't set it restrictive enough, it will give an error - but it doesn't say what that error is. The current table column definition looks like
`Id` int(10) NOT NULL AUTO_INCREMENT
KEY `Id` (`Id`)
When I try
ALTER TABLE MyTable MODIFY MyField bigint AUTO_INCREMENT LOCK=NONE;
I just get a generic SQL syntax error. Even if I specify DEFAULT, I get an error, so I'm not sure how to use the LOCK - which I would expect the proper error to tell me when I have chosen an improper lock level.
The syntax...
alter_specification [, alter_specification] ...
... requires a comma
ALTER TABLE MyTable
MODIFY COLUMN MyField BIGINT AUTO_INCREMENT, -- comma here
LOCK=NONE;
I'm guessing the error was not all that "generic" -- it should have said something about the right syntax to use near 'LOCK... which is your hint not that the quoted term is the beginning of the error, but rather that that the parser/lexer expected something other than the quoted value to occur at that position (because it was looking for the comma).
If the column you are altering is the primary key, a lock seems inevitable -- because the entire table should need rebuilding, including all the indexes, since the primary key "rides free" in all indexes, as it is what's used after a non-covering index lookup to actually find the rows matched by the index.
I'm converting an existing database from MyISAM to InnoDB and implementing various foreign keys, I'm having an issue with running the convert script on my database though:-
I'm running all queries as below
DELETE FROM example WHERE user NOT IN (select id FROM users);
ALTER TABLE `example` CHANGE `user` `user` INT( 11 ) UNSIGNED NOT NULL ;
ALTER TABLE example ADD FOREIGN KEY (user) REFERENCES users(ID);
ALTER TABLE example ADD FOREIGN KEY (car) REFERENCES cars(ID);
When I run all queries it fails due to a foreign key constraint, due to the fact the DELETE statement hasn't run - if I run them individually, it's fine - is it an issue with commit on the innodb database or is it due to speed of the delete not completing before the next query?
Is it also ok to have two foreignkeys of ID? (two different tables users.id and cars.id).
Thanks!
No idea of what the error message might say or what you're trying to accomplish but ALTER TABLE is a DDL statement and those cannot be rollbacked in MySQL. The Statements That Cause an Implicit Commi manual chapter explains:
The statements listed in this section (and any synonyms for them)
implicitly end any transaction active in the current session, as if you had done a COMMIT before executing the statement. As of MySQL
5.5.3, most of these statements also cause an implicit commit after executing;
[...]
Data definition language (DDL) statements that define or modify database objects
[...]
ALTER TABLE, CREATE TABLE, and DROP TABLE do not commit a transaction
if the TEMPORARY keyword is used. (This does not apply to other
operations on temporary tables such as CREATE INDEX, which do cause a
commit.) However, although no implicit commit occurs, neither can the
statement be rolled back. Therefore, use of such statements will
violate transaction atomicity: For example, if you use CREATE
TEMPORARY TABLE and then roll back the transaction, the table remains
in existence.
I wrote a java program that accesses a MySQL innodb database.
Whenever an INSERT IGNORE statement encounters a duplicate entry the Auto Increment primary key is incremented.
Is this behaviour the expected? I think it shouldn't happen with IGNORE. That means that IGNORE actually incurs an extra overhead for writing the new primary key value.
The table is the following:
CREATE TABLE `tablename` (
`id` bigint(20) unsigned NOT NULL AUTO_INCREMENT,
`rowname` varchar(50) NOT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `rowname` (`rowname`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
Thank you!
This has been the default behaviour since MySQL 5.1.22.
You can set the configuration variable innodb_autoinc_lock_mode to 0 (a.k.a “traditional” lock mode) If you'd like to avoid gaps in your auto-increment columns. It may incur a performance penalty, though, as this mode has the effect of holding a table lock until the INSERT completes.
From the docs on InnoDB AUTO_INCREMENT Lock Modes:
innodb_autoinc_lock_mode = 0 (“traditional” lock mode)
The traditional lock mode provides the same behavior that existed
before the innodb_autoinc_lock_mode configuration parameter was
introduced in MySQL 5.1. The traditional lock mode option is provided
for backward compatibility, performance testing, and working around
issues with “mixed-mode inserts”, due to possible differences in
semantics.
In this lock mode, all “INSERT-like” statements obtain a special
table-level AUTO-INC lock for inserts into tables with AUTO_INCREMENT
columns. This lock is normally held to the end of the statement (not
to the end of the transaction) to ensure that auto-increment values
are assigned in a predictable and repeatable order for a given
sequence of INSERT statements, and to ensure that auto-increment
values assigned by any given statement are consecutive.
I believe this is a configurable setting in InnoDB. See: AUTO_INCREMENT Handling in InnoDB
You'd want to go with
innodb_autoinc_lock_mode = 0
INSERT INTO `tablename` (id, rowname) SELECT '1', 'abc' FROM dual WHERE NOT EXISTS(SELECT NULL FROM `tablename` WHERE `rowname`='abc');
or short (because the id field has an increment in the table )
INSERT INTO `tablename` (rowname) SELECT 'abc' FROM dual WHERE NOT EXISTS(SELECT NULL FROM `tablename` WHERE `rowname`='abc');
The solution may look cumbersome, but it works as the author needs.
I think this behaviour is reasonable. The auto-increment should not be relied upon to give sequences that don't have gaps.
For example, a rolled back transaction still consumes IDs:
INSERT INTO t (normalcol, uniquecol) VALUES
('hello','uni1'),
('hello','uni2'),
('hello','uni1');
Generates a unique key violation obviously, and inserts no rows into the database (assuming transactional engine here). However, it may consume up to 3 auto-inc values without inserting anything.
Not sure if it's expected, though I would recommend switching to:
INSERT ... ON DUPLICATE KEY UPDATE