So I have been asked to change the engine of a few tables in a production database from MyISAM to InnoDB. I am trying to figure out how that will affect usage in production (as the server can afford no downtime).
I have read some conflicting information. Some information I have read state that the tables are locked and will not receive updates until after the conversion completes (IE, updates are not queued, just discarded until it completes).
In other places, I have read that while the table is locked, the inserts and updates will be queued until the operation is complete, and THEN the write actions are performed.
So what exactly is the story here?
This is directly from the manual:
In most cases, ALTER TABLE makes a temporary copy of the original
table. MySQL waits for other operations that are modifying the table,
then proceeds. It incorporates the alteration into the copy, deletes
the original table, and renames the new one. While ALTER TABLE is
executing, the original table is readable by other sessions. Updates
and writes to the table that begin after the ALTER TABLE operation
begins are stalled until the new table is ready, then are
automatically redirected to the new table without any failed updates.
So, number two wins. They're not "failed", they're "stalled".
The latter is correct. All queries against a table that's being altered are blocked until the alter completes, and are processed once the alter finishes. Note that this includes read queries (SELECT) as well as write queries (INSERT, UPDATE, DELETE).
Related
I want to change column datatype from text to varchar(256) and the table has many data is there a chance of data loss or error i can encounter ? or a downtime?
Assuming you are using Innodb tables on MySQL 8, changing the type of a column needs to rewrite the table.
alter table will wait until all ongoing writes are done (insert, update, delete) and then read lock the table while it does the copy. Only selects will be allowed. Inserts, updates, and deletes will all wait until the copy is finished.
ALTER TABLE operations that use the COPY algorithm wait for other operations that are modifying the table to complete. After alterations are applied to the table copy, data is copied over, the original table is deleted, and the table copy is renamed to the name of the original table. While the ALTER TABLE operation executes, the original table is readable by other sessions (with the exception noted shortly). Updates and writes to the table started after the ALTER TABLE operation begins are stalled until the new table is ready, then are automatically redirected to the new table.
Once the copy is done, it will briefly lock the table to complete the copy.
The exception referred to earlier is that ALTER TABLE blocks reads (not just writes) at the point where it is ready to clear outdated table structures from the table and table definition caches. At this point, it must acquire an exclusive lock. To do so, it waits for current readers to finish, and blocks new reads and writes.
You will lose data if any of your text values are longer than 255 characters; the values will be truncated.
There isn't much practical difference between text and varchar so the value of such a change on a large table is questionable.
I have a script that runs 24-7 on a table to perform necessary functions on it. However, when it is running, it is almost impossible to do an ALTER TABLE ADD INDEX statement, as it seems like it just hangs indefinitely. Is there any way around this? How should I go about adding this index?
The Alter table statement is getting a metadata lockout. You cannot perform your alter statement while another transaction is in process on the same table. Since your script runs 24-7, it is not possible to do what you are asking.
To ensure transaction serializability, the server must not permit one session to perform a data definition language (DDL) statement on a table that is used in an uncompleted explicitly or implicitly started transaction in another session. The server achieves this by acquiring metadata locks on tables used within a transaction and deferring release of those locks until the transaction ends. A metadata lock on a table prevents changes to the table's structure. This locking approach has the implication that a table that is being used by a transaction within one session cannot be used in DDL statements by other sessions until the transaction ends.
You can read more about this Here at dev.mysql.
Version 5.6 has ALTER TABLE ... ALGORITHM=INLACE ... to do ADD INDEX and several other ALTERs without blocking everything.
pt-online-table-alter (from Percona.com) can do it in older versions of MySQL. It uses a TRIGGER.
How should I convert MyISAM table to InnoDB on live working site in real time?
Should I just change in phpmyadmin in operations menu "Storage Engine" to InnoDB?
Would it lock all table during conversion?
You will indeed get a write lock on the table. You will still be able to read from the table while the new table is being created and copied over.
From the documentation on ALTER TABLE:
While ALTER TABLE is executing, the original table is readable by
other sessions (with the exception noted shortly). Updates and writes
to the table that begin after the ALTER TABLE operation begins are
stalled until the new table is ready, then are automatically
redirected to the new table without any failed updates.
...
The exception referred to earlier is that ALTER TABLE blocks reads
(not just writes) at the point where it is ready to install a new
version of the table .frm file, discard the old file, and clear
outdated table structures from the table and table definition caches.
At this point, it must acquire an exclusive lock. To do so, it waits
for current readers to finish, and blocks new reads (and writes).
Check out pt-online-schema-change. See example in this answer https://dba.stackexchange.com/questions/60570/best-way-to-add-a-new-column-to-a-large-table-mysql-myisam/60576#60576
It will block a table for a second or so to rename two tables.
While it might work you'd better make a copy of your table and try the storage engine change operation on the copy.
That would also give you an estimate of time needed for the operation.
Even better would be if you do that on a copy of your application and make extensive checks whether it would work at all with the new storage engine.
I am using Engine InnoDB on my MySQL server.
I have a patch script to upgrade my tables like add new columns and fill in default data.
I want to make sure there is no other session using the database. So I need a way to lock the database:
The lock shouldn't kick out an existing session. If their is any other existing session just fail the lock and report error
The lock need to prevent other sessions to read/write/change the database.
Thanks a lot everyone!
You don't need to worry about locking tables yourself. As the MySQL documentation (http://dev.mysql.com/doc/refman/5.1/en/alter-table.html) says:
In most cases, ALTER TABLE makes a temporary copy of the original
table. MySQL waits for other operations that are modifying the table,
then proceeds. It incorporates the alteration into the copy, deletes
the original table, and renames the new one. While ALTER TABLE is
executing, the original table is readable by other sessions. Updates
and writes to the table that begin after the ALTER TABLE operation
begins are stalled until the new table is ready, then are
automatically redirected to the new table without any failed updates.
To select information related to a list of hundreds of IDs... rather than make a huge select statement, I create temp table, insert the ids into it, join it with a table to select the rows matching the IDs, then delete the temp table. So this is essentially a read operation, with no permanent changes made to any persistent database tables.
I do this in a transaction, to ensure the temp table is deleted when I'm finished. My question is... what happens when I commit such a transaction vs. let it roll it back?
Performance-wise... does the DB engine have to do more work to roll back the transaction vs committing it? Is there even a difference since the only modifications are done to a temp table?
Related question here, but doesn't answer my specific case involving temp tables: Should I commit or rollback a read transaction?
EDIT (Clarification of Question):
Not looking for advice up to point of commit/rollback. Transaction is absolutely necessary. Assume no errors occur. Assume I have created a temp table, assume I know real "work" writing to tempdb has occurred, assume I perform read-only (select) operations in the transaction, and assume I issue a delete statement on the temp table. After all that... which is cheaper, commit or rollback, and why? What OTHER work might the db engine do at THAT POINT for a commit vs a rollback, based on this specific scenario involving temp-tables and otherwise read-only operations?
If we are talking about local temporary table (i.e. the name is prefixed with a single #), the moment you close your connection, SQL Server will kill the table. Thus, assuming your data layer is well designed to keep connections open as short a time as possible, I would not worry about wrapping the creation of temp tables in a transaction.
I suppose there could be a slight performance difference of wrapping the table in a transaction but I would bet it is so small as to be inconsequential compared to the cost of keeping a transaction open longer due to the time to create and populate the temp table.
A simpler way to insure that the temp table is deleted is to create it using the # sign.
CREATE TABLE #mytable (
rowID int,
rowName char(30) )
The # tells SQL Server that this table is a local temporary table. This table is only visible to this session of SQL Server. When the session is closed, the table will be automatically dropped. You can treat this table just like any other table with a few exceptions. The only real major one is that you can't have foreign key constraints on a temporary table. The others are covered in Books Online.
Temporary tables are created in tempdb.
If you do this, you won't have to wrap it in a transaction.