I am using MariaDB 10.1.9. Short version: What I really want to know for certain is if I can modify an indexed auto_increment field on an innodb table from int to bigint without locking the table?
Long version: Is there a detailed explanation of which ALTER TABLE operations require which lock level? The documentation just says "Depending on the required operation and the used storage engine, different lock strategies can be used for ALTER TABLE.". It doesn't provide a link to any details and each operation on the ALTER TABLE page does not specify it's required level.
From experimentation, I know ADD COLUMN does not require a lock. MODIFY COLUMN allows reads, but can it be manually set to allow writes? The MariaDB documentation says you can set the lock level, but if you don't set it restrictive enough, it will give an error - but it doesn't say what that error is. The current table column definition looks like
`Id` int(10) NOT NULL AUTO_INCREMENT
KEY `Id` (`Id`)
When I try
ALTER TABLE MyTable MODIFY MyField bigint AUTO_INCREMENT LOCK=NONE;
I just get a generic SQL syntax error. Even if I specify DEFAULT, I get an error, so I'm not sure how to use the LOCK - which I would expect the proper error to tell me when I have chosen an improper lock level.
The syntax...
alter_specification [, alter_specification] ...
... requires a comma
ALTER TABLE MyTable
MODIFY COLUMN MyField BIGINT AUTO_INCREMENT, -- comma here
LOCK=NONE;
I'm guessing the error was not all that "generic" -- it should have said something about the right syntax to use near 'LOCK... which is your hint not that the quoted term is the beginning of the error, but rather that that the parser/lexer expected something other than the quoted value to occur at that position (because it was looking for the comma).
If the column you are altering is the primary key, a lock seems inevitable -- because the entire table should need rebuilding, including all the indexes, since the primary key "rides free" in all indexes, as it is what's used after a non-covering index lookup to actually find the rows matched by the index.
Related
Disclaimer: I have only novice knowledge of and experience with databases.
I'm following a Laravel course on Laracasts, and in the database video, the instructor sets the ID column to a type of SERIAL. This is different to how I've seen this done in all other database tutorials, where they will usually check the A_I (auto-increment) checkbox, and this automatically makes the column primary, and leaves the type to be something like INT.
Hovering over the SERIAL type in PHPMyAdmin tells me that it's an alias for BIGINT UNSIGNED NOT NULL AUTO_INCREMENT UNIQUE, but is there ever a particular reason to prefer it over the settings that checking the A_I checkbox sets up? Does either way offer any advantages or disadvantages?
I did find this for PostgreSQL, indicating SERIAL is old and outdated, but I couldn't find an equivalent for MySQL and I'm unsure if the same applies to it.
I'm sure MySQL's SERIAL type was implemented to make it easy for folks who were accustomed to PostgreSQL to have one set of CREATE TABLE statements that would work on both brands of database, and do more or less the same thing.
In an old version of the MySQL manual, it was stated that SERIAL is a compatibility feature (without naming the brand it was intended to be compatible with). The language about compatibility was removed (see https://bugs.mysql.com/bug.php?id=7978).
Now that even PostgreSQL has changed its recommended practice and they use IDENTITY columns instead of SERIAL, the MySQL feature is really unnecessary.
There is no advantage to using SERIAL in MySQL. On the contrary, if you do use it in a CREATE TABLE statement, you will see that the syntax isn't saved. It is just an alias for the BIGINT UNSIGNED AUTO_INCREMENT UNIQUE, as documented.
I find that it's actually wasteful to do this, because I typically declare the auto-increment column as a PRIMARY KEY anyway, and this makes the UNIQUE redundant. So you end up with two unique indexes for no reason.
mysql> create table mytable (id serial primary key);
mysql> show create table mytable\G
*************************** 1. row ***************************
Table: mytable
Create Table: CREATE TABLE `mytable` (
`id` bigint(20) unsigned NOT NULL AUTO_INCREMENT,
PRIMARY KEY (`id`),
UNIQUE KEY `id` (`id`) -- this one is superfluous
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4
P.S. This question is almost but not quite a duplicate of What is the difference between SERIAL and AUTO_INCREMENT in mysql
I have created a table with the following query.
CREATE table IF NOT EXISTS result(id varchar(255) not null, product varchar(255), prodcount int, searched_at datetime default CURRENT_TIMESTAMP, primary key (id, product, searched_at)) partition by hash(yearweek(searched_at))
Now I need to change the column name of searched_at to created_at. While using AlterTable command, I am getting the following error:
"Error: ER_BAD_FIELD_ERROR: Unknown column 'searched_at' in 'partition function'"
It seems that there is already a bug raised in MySQL: https://bugs.mysql.com/bug.php?id=85413
Does this issue have any workarounds?
Well, you probably have to start with ALTER TABLE table DROP PARTITIONING.
However, I would recommend stopping there, except for fixing the indexes.
PARITITION BY HASH() probably has no viable use case.
It sounds like you may be searching on a date range? If so, BY HASH will touch all the partitions because, for all it knows, the data is wrapped every year.
Pleas provide the important queries; I will help you redo the indexes.
I have the following sql code to create a table
CREATE TABLE db.object (
`objid` bigint(20) NOT NULL AUTO_INCREMENT,
`object_type` varchar(32) NOT NULL,
PRIMARY KEY (`objid`)
) ENGINE=InnoDB AUTO_INCREMENT=2 DEFAULT CHARSET=utf8;
However, the values in the objid are coming out as 1,2,3... (The insert statement is not adding the ids)
Shouldn't AUTO_INCREMENT=2 make the objid start from 2 instead of 1
With InnoDB tables, the AUTO_INCREMENT value will be reset to the maximum value (plus 1) when the table is opened. The auto increment value exists only in memory, it is not persisted on disk.
A table open would happen, for example, when the MySQL instance was shutdown and then restarted, and a reference is made to the table.
A table can also be closed at other times. For example, when open_table_cache is exceeded (that is, when a large number of other tables is opened), MySQL will close some of the open tables, to make room in the cache for newly opened tables.
I believe this behavior is documented somewhere in the MySQL Reference Manual.
I used your SQL, created the object table and entered two values for object_type and objid started at 2. Can't see anything wrong here...
It might. There are enough exceptions and gotchas with auto-inc on InnoDB tables that it bears urging a full review of the documentation.
That said, there is one scenario I can think of where MySQL ignores the initializer value. I'll quote the documentation:
InnoDB uses the in-memory auto-increment counter as long as the server runs. When the server is stopped and restarted, InnoDB reinitializes the counter for each table for the first INSERT to the table, as described [here]:
InnoDB executes the equivalent of the following statement on the first insert into a table containing an AUTO_INCREMENT column after a restart:
SELECT MAX(ai_col) FROM table_name FOR UPDATE;
A server restart also cancels the effect of the AUTO_INCREMENT = N table option in CREATE TABLE and ALTER TABLE statements, which you can use with InnoDB tables to set the initial counter value or alter the current counter value.
So if you create that table, then do a server restart (like as part of a deployment process), you'll get a nice value of 1 for the initial row. If you want to countermand this, you need to create the table, then insert a dummy row with the auto-inc value you want, then restart, then delete the dummy row.
We have loaded several tables (hundreds of millions of rows each) but the following query requires the entire table to be rewritten (the type is exactly the same apart from the AUTO_INCREMENT):
ALTER TABLE sales MODIFY id BIGINT(20) NOT NULL AUTO_INCREMENT;
Is this just a huge oversight by MySQL or does it fundamentally change the format the data on disk?
Changing the AUTO_INCREMENT characteristic of a column cannot be done as an online DDL change. Even when I try to request it explicitly:
mysql> alter table sales modify column id bigint not null auto_increment,
ALGORITHM=INPLACE;
ERROR 1846 (0A000): ALGORITHM=INPLACE is not supported.
Reason: Cannot change column type INPLACE. Try ALGORITHM=COPY.
This case of changing the column definition seems to count as a change to the column type (based on the error message).
https://dev.mysql.com/doc/refman/5.6/en/innodb-create-index-overview.html#innodb-online-ddl-summary-grid says:
Change column data type... Only supports ALGORITHM=COPY
In MySQL 5.7, changing the type of a column is still required to rebuild the table, except in one case: increasing VARCHAR length is allowed as an INPLACE change.
You aren't the only one to think that it should be allowed to change the AUTO_INCREMENT property without doing a table restructure: https://bugs.mysql.com/bug.php?id=72109
I wrote a java program that accesses a MySQL innodb database.
Whenever an INSERT IGNORE statement encounters a duplicate entry the Auto Increment primary key is incremented.
Is this behaviour the expected? I think it shouldn't happen with IGNORE. That means that IGNORE actually incurs an extra overhead for writing the new primary key value.
The table is the following:
CREATE TABLE `tablename` (
`id` bigint(20) unsigned NOT NULL AUTO_INCREMENT,
`rowname` varchar(50) NOT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `rowname` (`rowname`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
Thank you!
This has been the default behaviour since MySQL 5.1.22.
You can set the configuration variable innodb_autoinc_lock_mode to 0 (a.k.a “traditional” lock mode) If you'd like to avoid gaps in your auto-increment columns. It may incur a performance penalty, though, as this mode has the effect of holding a table lock until the INSERT completes.
From the docs on InnoDB AUTO_INCREMENT Lock Modes:
innodb_autoinc_lock_mode = 0 (“traditional” lock mode)
The traditional lock mode provides the same behavior that existed
before the innodb_autoinc_lock_mode configuration parameter was
introduced in MySQL 5.1. The traditional lock mode option is provided
for backward compatibility, performance testing, and working around
issues with “mixed-mode inserts”, due to possible differences in
semantics.
In this lock mode, all “INSERT-like” statements obtain a special
table-level AUTO-INC lock for inserts into tables with AUTO_INCREMENT
columns. This lock is normally held to the end of the statement (not
to the end of the transaction) to ensure that auto-increment values
are assigned in a predictable and repeatable order for a given
sequence of INSERT statements, and to ensure that auto-increment
values assigned by any given statement are consecutive.
I believe this is a configurable setting in InnoDB. See: AUTO_INCREMENT Handling in InnoDB
You'd want to go with
innodb_autoinc_lock_mode = 0
INSERT INTO `tablename` (id, rowname) SELECT '1', 'abc' FROM dual WHERE NOT EXISTS(SELECT NULL FROM `tablename` WHERE `rowname`='abc');
or short (because the id field has an increment in the table )
INSERT INTO `tablename` (rowname) SELECT 'abc' FROM dual WHERE NOT EXISTS(SELECT NULL FROM `tablename` WHERE `rowname`='abc');
The solution may look cumbersome, but it works as the author needs.
I think this behaviour is reasonable. The auto-increment should not be relied upon to give sequences that don't have gaps.
For example, a rolled back transaction still consumes IDs:
INSERT INTO t (normalcol, uniquecol) VALUES
('hello','uni1'),
('hello','uni2'),
('hello','uni1');
Generates a unique key violation obviously, and inserts no rows into the database (assuming transactional engine here). However, it may consume up to 3 auto-inc values without inserting anything.
Not sure if it's expected, though I would recommend switching to:
INSERT ... ON DUPLICATE KEY UPDATE