Does
ALTER TABLE sample ADD COLUMN `hasItem` tinyint(1) DEFAULT NULL
lock the entire table?
Short answer: For MySQL < 5.6 locks are required. From 5.6 on, and using InnoDB, locks are not required for many ALTER TABLE operations including adding a column.
If you're using MySQL 5.5 or older, it will get a read lock for the whole operation and then a brief write lock at the end.
From the MySQL documentation for ALTER TABLE...
In most cases, ALTER TABLE makes a temporary copy of the original table... While ALTER TABLE is executing, the original table is readable by other sessions (with the exception noted shortly). Updates and writes to the table that begin after the ALTER TABLE operation begins are stalled until the new table is ready...
The exception referred to earlier is that ALTER TABLE blocks reads (not just writes) at the point where it is ready to install a new version of the table .frm file, discard the old file, and clear outdated table structures from the table and table definition caches. At this point, it must acquire an exclusive lock.
Which is to say, when adding a column it read locks the table for most of the operation, then gets a write lock at the end.
MySQL 5.6 added the Online DDL to InnoDB which speeds up and improves many things such as altering tables and indexes. Adding a column to a table will no longer require table locks except possibly brief exclusive locks at the start and end of the operation.
It should happen automatically, but to be sure set ALGORITHM=inplace and LOCK=none to your ALTER TABLE statement.
There is one exception...
InnoDB tables created before MySQL 5.6 do not support ALTER TABLE ... ALGORITHM=INPLACE for tables that include temporal columns (DATE, DATETIME or TIMESTAMP) and have not been rebuilt using ALTER TABLE ... ALGORITHM=COPY.
Related
We've got an InnoDB table with 70 million rows, and we have been trying to run an alter table statement to modify and add a couple of columns. The query seems to have altered the table, and is now in the state of 'committing alter table to storage engine'.
START TRANSACTION;
ALTER TABLE table
MODIFY COLUMN column1 int(11) NOT NULL DEFAULT 0,
MODIFY COLUMN column2 tinyint(1) NOT NULL DEFAULT 1,
ADD COLUMN column3 int(11),
ADD COLUMN column4 int(11) NOT NULL DEFAULT 1,
ADD COLUMN column5 varchar(255);
COMMIT;
This has been running overnight, and is at 19 hours at the current time. We do not have the performance schema enabled so cannot look at an estimated time of completion. My concern lies as to what the query is doing and whether the query will rollback if killed. I have seen other questions relate to queries that are stuck in copying to tmp tables, or awaiting a table lock. However I cannot find anything about being stuck while the alter table is committing.
Is it safe to kill a query in this state, and if the query is killed, will it rollback successfully?
The server is running MariaDB 10.2
From the documentation:
Some statements cannot be rolled back. In general, these include data definition language (DDL) statements, such as those that create or drop databases, those that create, drop, or alter tables or stored routines.
You should design your transactions not to include such statements. If you issue a statement early in a transaction that cannot be rolled back, and then another statement later fails, the full effect of the transaction cannot be rolled back in such cases by issuing a ROLLBACK statement.
I implemented the ALGORITHM=INPLACE and LOCK=NONE for InnoDB in MySQL 5.6.
Depending on the previous table definition, this operation could imply ALGORITHM=INPLACE, or it could fall back to ALGORITHM=COPY.
Starting with MariaDB 10.3 (MDEV-11369), ADD COLUMN would be instantaneous; before that, the table would have to be rebuilt. (The syntax ALGORITHM=INPLACE is very misleading.)
Starting with MariaDB 10.2.13 and 10.3.5 (MDEV-11415), ALGORITHM=COPY will no longer write undo log records for the individual rows, and the rollback (in case of client disconnect or killed server) should be much faster.
As the ALTER TABLE is a DDL statement, it causes an implicit commit when it is executed. This means that it cannot be rolled back but interrupting the DDL (by killing the connection) will cause the already applied changes to be rolled back in a controlled manner.
Given that you are using the default ALTER TABLE operation (no ALGORITHM defined), cancelling it should be relatively fast as all it has to do, at least according to my knowledge, is to discard the new copy of the table.
I need to create an index on a large InnoDB production table and want to do this without locking the table in any way. I am using MySQL 5.6 (.38-83.90).
I tried
create index my_index on my_table(col1, col2);
Neither columns are primary keys. col1 is a foreign key.
Well, this totally locked the table. Other queries were stalled with "Waiting for table metadata lock" bringing my website to its knees. I had to kill the create index query.
From this https://dev.mysql.com/doc/refman/5.6/en/innodb-create-index-overview.html I thought that it would not lock the table: "... no syntax changes are required... The table remains available for read and write operations while the index is being created or dropped."
I see that I can set LOCK=NONE or LOCK=SHARED, but I don't see that it should be necessary or, if it is, which one I need to use.
"You can specify LOCK=NONE to assert that concurrent DML is permitted during the DDL operation. MySQL automatically permits concurrent DML when possible."
"You can specify LOCK=SHARED to assert that concurrent queries are permitted during a DDL operation. MySQL automatically permits concurrent queries when possible."
None of the limitations https://dev.mysql.com/doc/refman/5.6/en/innodb-create-index-limitations.html seem to apply to my case.
What am I missing?
My guess (just a guess) is that you are missing the ALGORITHM=INPLACE clause on the CREATE INDEX statement.
CREATE INDEX my_index ON my_table(col1, col2) ALGORITHM=INPLACE ;
^^^^^^^^^^^^^^^^^
Also be aware of transactions acquiring and holding metadata locks.
https://dev.mysql.com/doc/refman/5.6/en/metadata-locking.html
Any transaction that has referenced my_table will continue to hold a metadata lock on that table until the transaction is committed or rolled back. I suggest checking the TRANSACTIONS section of SHOW ENGINE INNODB STATUS output.
I am using MySQL Server-5.1.58 Community log. The problem is after deleting the data the allocated space of MySQL database is not getting free and as a result day by day the backup size of my using database is increasing.
Please kindly let me know any tool which can resolve the issue.
Remember that MySQL locks the table during the time OPTIMIZE TABLE is running
For your MySQL version from the official documentation:
OPTIMIZE TABLE should be used if you have deleted a large part of a
table or if you have made many changes to a table with variable-length
rows (tables that have VARCHAR, VARBINARY, BLOB, or TEXT columns).
Deleted rows are maintained in a linked list and subsequent INSERT
operations reuse old row positions. You can use OPTIMIZE TABLE to
reclaim the unused space and to defragment the data file
Additional notes for InnoDB:
For InnoDB tables, OPTIMIZE TABLE is mapped to ALTER TABLE, which
rebuilds the table to update index statistics and free unused space in
the clustered index. Beginning with MySQL 5.1.27, this is displayed in
the output of OPTIMIZE TABLE when you run it on an InnoDB table, as
shown here:
mysql> OPTIMIZE TABLE foo;
Table does not support optimize, doing recreate + analyze instead
So:
OPTIMIZE [NO_WRITE_TO_BINLOG | LOCAL] TABLE
tbl_name [, tbl_name] ...
By default, OPTIMIZE TABLE statements are written to the binary log so
that they will be replicated to replication slaves. Logging can be
suppressed with the optional NO_WRITE_TO_BINLOG keyword or its alias
LOCAL.
I have a table with the primary key of max signed INT hit, 2147483647
Imagine I want to switch it to unsigned, and there are no negative values in the table since it is a primary key, because Im under the current belief that it is the fastest way to get the table going again.
Should the ALTER TABLE statement to switch it to an unsigned INT be a relatively quick process, since the values of the ids shouldn't change? What about locking?
Mysql documentation on ALTER TABLE command describes quite in a detailed manner under "Storage, Performance, and Concurrency Considerations" section which changes can be done quickly, without table copy and index rebuild, and what locks mysql will apply during the course of the command. Changing the column type is unfortunately not listed as something that can be done in place (of course, read the documentation corresponding to your mysql version, I just linked the newest one).
For some operations, an in-place ALTER TABLE is possible that does not
require a temporary table:
For ALTER TABLE tbl_name RENAME TO new_tbl_name without any other options, MySQL simply renames any files that correspond to the table
tbl_name without making a copy. (You can also use the RENAME TABLE
statement to rename tables. See Section 13.1.28, “RENAME TABLE
Syntax”.) Any privileges granted specifically for the renamed table
are not migrated to the new name. They must be changed manually.
Alterations that modify only table metadata and not table data are immediate because the server only needs to alter the table .frm file,
not touch table contents. The following changes are fast alterations
that can be made this way:
Renaming a column.
Changing the default value of a column.
Changing the definition of an ENUM or SET column by adding new enumeration or set members to the end of the list of valid member
values, as long as the storage size of the data type does not change.
For example, adding a member to a SET column that has 8 members
changes the required storage per value from 1 byte to 2 bytes; this
will require a table copy. Adding members in the middle of the list
causes renumbering of existing members, which requires a table copy.
ALTER TABLE with DISCARD ... PARTITION ... TABLESPACE or IMPORT ... PARTITION ... TABLESPACE do not create any temporary tables or
temporary partition files.
ALTER TABLE with ADD PARTITION, DROP PARTITION, COALESCE PARTITION, REBUILD PARTITION, or REORGANIZE PARTITION does not create
any temporary tables (except when used with NDB tables); however,
these operations can and do create temporary partition files.
ADD or DROP operations for RANGE or LIST partitions are immediate operations or nearly so. ADD or COALESCE operations for HASH or KEY
partitions copy data between all partitions, unless LINEAR HASH or
LINEAR KEY was used; this is effectively the same as creating a new
table, although the ADD or COALESCE operation is performed partition
by partition. REORGANIZE operations copy only changed partitions and
do not touch unchanged ones.
Renaming an index.
Adding or dropping an index, for InnoDB.
Locking:
While ALTER TABLE is executing, the original table is readable by
other sessions (with the exception noted shortly). Updates and writes
to the table that begin after the ALTER TABLE operation begins are
stalled until the new table is ready, then are automatically
redirected to the new table without any failed updates. The temporary
copy of the original table is created in the database directory of the
new table. This can differ from the database directory of the original
table for ALTER TABLE operations that rename the table to a different
database.
The exception referred to earlier is that ALTER TABLE blocks reads
(not just writes) at the point where it is ready to install a new
version of the table .frm file, discard the old file, and clear
outdated table structures from the table and table definition caches.
At this point, it must acquire an exclusive lock. To do so, it waits
for current readers to finish, and blocks new reads (and writes).
In a MySQL MASTER MASTER scenario using InnoDB
When dropping an index on one instance will the same table on the other instance be available?
What is the sequence of activities?
I assume the following sequence:
DROP INDEX on 1st instance
Added to the binary log
DROP INDEX on 2nd instance
Can anyone confirm?
I believe the following will happen:
Your DROP INDEX (which really runs an ALTER TABLE ... DROP INDEX) runs on the master
If the ALTER completes successfully the statement will then be added to the binlog and will be run on the slave
This means that the ALTER TABLE on the other machine won't start until the ALTER TABLE has successfully completed on the first (master) machine.
While the ALTER TABLE is running on either machine the table will be readable for a while and then neither readable/writeable as MySQL first makes a copy of the table internally then applies the changes.
From http://dev.mysql.com/doc/refman/5.0/en/alter-table.html
In most cases, ALTER TABLE works by
making a temporary copy of the
original table. The alteration is
performed on the copy, and then the
original table is deleted and the new
one is renamed. While ALTER TABLE is
executing, the original table is
readable by other sessions. Updates
and writes to the table are stalled
until the new table is ready, and then
are automatically redirected to the
new table without any failed updates.
The temporary table is created in the
database directory of the new table.
This can be different from the
database directory of the original
table if ALTER TABLE is renaming the
table to a different database.
In a MySQL MASTER MASTER scenario using InnoDB
In so far as I'm aware, such a thing is not possible. You'd need to use NDB, or be in a multi-master-slave environment with auto-incrementing fields configured to increment by 1/2/3/more. So assuming the latter. (Note: if you're aware of an InnoDB based solution, please share.)
When dropping an index on one instance will the same table on the other instance be available?
Dropping an index only means your index won't be available. Not the table. It'll be written (and propagated) to the binary log and begone with.