I have a script that runs 24-7 on a table to perform necessary functions on it. However, when it is running, it is almost impossible to do an ALTER TABLE ADD INDEX statement, as it seems like it just hangs indefinitely. Is there any way around this? How should I go about adding this index?
The Alter table statement is getting a metadata lockout. You cannot perform your alter statement while another transaction is in process on the same table. Since your script runs 24-7, it is not possible to do what you are asking.
To ensure transaction serializability, the server must not permit one session to perform a data definition language (DDL) statement on a table that is used in an uncompleted explicitly or implicitly started transaction in another session. The server achieves this by acquiring metadata locks on tables used within a transaction and deferring release of those locks until the transaction ends. A metadata lock on a table prevents changes to the table's structure. This locking approach has the implication that a table that is being used by a transaction within one session cannot be used in DDL statements by other sessions until the transaction ends.
You can read more about this Here at dev.mysql.
Version 5.6 has ALTER TABLE ... ALGORITHM=INLACE ... to do ADD INDEX and several other ALTERs without blocking everything.
pt-online-table-alter (from Percona.com) can do it in older versions of MySQL. It uses a TRIGGER.
Related
i want to prohibit the deletion of fournisseur that he have at least an achats
I use a MySQL database
If you want to limit other threads (or processes) from modifying the fournisseur table during your action, then you should lock the table (you can lock it only for write actions or for all actions).
Do note that you will need to implement some mutual-exclusion mechanism to sync all clients that modify this table (you can lock the table and still allow others to SELECT and JOIN on this table, the others still need to lock the table - but with a READ lock).
If another client tries to acquire a WRITE lock (because it wants to modify a locked table), and the locking thread does not release it in a short time, it will generate an error (timeout).
Such error can be handled in the script (return a specific response so your code knows it reached timeout on the lock statement block).
If it tries to modify a locked table without trying to acquire the lock first, an exception is thrown.
Please see the MySQL documentation here
I need to create an index on a large InnoDB production table and want to do this without locking the table in any way. I am using MySQL 5.6 (.38-83.90).
I tried
create index my_index on my_table(col1, col2);
Neither columns are primary keys. col1 is a foreign key.
Well, this totally locked the table. Other queries were stalled with "Waiting for table metadata lock" bringing my website to its knees. I had to kill the create index query.
From this https://dev.mysql.com/doc/refman/5.6/en/innodb-create-index-overview.html I thought that it would not lock the table: "... no syntax changes are required... The table remains available for read and write operations while the index is being created or dropped."
I see that I can set LOCK=NONE or LOCK=SHARED, but I don't see that it should be necessary or, if it is, which one I need to use.
"You can specify LOCK=NONE to assert that concurrent DML is permitted during the DDL operation. MySQL automatically permits concurrent DML when possible."
"You can specify LOCK=SHARED to assert that concurrent queries are permitted during a DDL operation. MySQL automatically permits concurrent queries when possible."
None of the limitations https://dev.mysql.com/doc/refman/5.6/en/innodb-create-index-limitations.html seem to apply to my case.
What am I missing?
My guess (just a guess) is that you are missing the ALGORITHM=INPLACE clause on the CREATE INDEX statement.
CREATE INDEX my_index ON my_table(col1, col2) ALGORITHM=INPLACE ;
^^^^^^^^^^^^^^^^^
Also be aware of transactions acquiring and holding metadata locks.
https://dev.mysql.com/doc/refman/5.6/en/metadata-locking.html
Any transaction that has referenced my_table will continue to hold a metadata lock on that table until the transaction is committed or rolled back. I suggest checking the TRANSACTIONS section of SHOW ENGINE INNODB STATUS output.
I am using Engine InnoDB on my MySQL server.
I have a patch script to upgrade my tables like add new columns and fill in default data.
I want to make sure there is no other session using the database. So I need a way to lock the database:
The lock shouldn't kick out an existing session. If their is any other existing session just fail the lock and report error
The lock need to prevent other sessions to read/write/change the database.
Thanks a lot everyone!
You don't need to worry about locking tables yourself. As the MySQL documentation (http://dev.mysql.com/doc/refman/5.1/en/alter-table.html) says:
In most cases, ALTER TABLE makes a temporary copy of the original
table. MySQL waits for other operations that are modifying the table,
then proceeds. It incorporates the alteration into the copy, deletes
the original table, and renames the new one. While ALTER TABLE is
executing, the original table is readable by other sessions. Updates
and writes to the table that begin after the ALTER TABLE operation
begins are stalled until the new table is ready, then are
automatically redirected to the new table without any failed updates.
So I have been asked to change the engine of a few tables in a production database from MyISAM to InnoDB. I am trying to figure out how that will affect usage in production (as the server can afford no downtime).
I have read some conflicting information. Some information I have read state that the tables are locked and will not receive updates until after the conversion completes (IE, updates are not queued, just discarded until it completes).
In other places, I have read that while the table is locked, the inserts and updates will be queued until the operation is complete, and THEN the write actions are performed.
So what exactly is the story here?
This is directly from the manual:
In most cases, ALTER TABLE makes a temporary copy of the original
table. MySQL waits for other operations that are modifying the table,
then proceeds. It incorporates the alteration into the copy, deletes
the original table, and renames the new one. While ALTER TABLE is
executing, the original table is readable by other sessions. Updates
and writes to the table that begin after the ALTER TABLE operation
begins are stalled until the new table is ready, then are
automatically redirected to the new table without any failed updates.
So, number two wins. They're not "failed", they're "stalled".
The latter is correct. All queries against a table that's being altered are blocked until the alter completes, and are processed once the alter finishes. Note that this includes read queries (SELECT) as well as write queries (INSERT, UPDATE, DELETE).
To select information related to a list of hundreds of IDs... rather than make a huge select statement, I create temp table, insert the ids into it, join it with a table to select the rows matching the IDs, then delete the temp table. So this is essentially a read operation, with no permanent changes made to any persistent database tables.
I do this in a transaction, to ensure the temp table is deleted when I'm finished. My question is... what happens when I commit such a transaction vs. let it roll it back?
Performance-wise... does the DB engine have to do more work to roll back the transaction vs committing it? Is there even a difference since the only modifications are done to a temp table?
Related question here, but doesn't answer my specific case involving temp tables: Should I commit or rollback a read transaction?
EDIT (Clarification of Question):
Not looking for advice up to point of commit/rollback. Transaction is absolutely necessary. Assume no errors occur. Assume I have created a temp table, assume I know real "work" writing to tempdb has occurred, assume I perform read-only (select) operations in the transaction, and assume I issue a delete statement on the temp table. After all that... which is cheaper, commit or rollback, and why? What OTHER work might the db engine do at THAT POINT for a commit vs a rollback, based on this specific scenario involving temp-tables and otherwise read-only operations?
If we are talking about local temporary table (i.e. the name is prefixed with a single #), the moment you close your connection, SQL Server will kill the table. Thus, assuming your data layer is well designed to keep connections open as short a time as possible, I would not worry about wrapping the creation of temp tables in a transaction.
I suppose there could be a slight performance difference of wrapping the table in a transaction but I would bet it is so small as to be inconsequential compared to the cost of keeping a transaction open longer due to the time to create and populate the temp table.
A simpler way to insure that the temp table is deleted is to create it using the # sign.
CREATE TABLE #mytable (
rowID int,
rowName char(30) )
The # tells SQL Server that this table is a local temporary table. This table is only visible to this session of SQL Server. When the session is closed, the table will be automatically dropped. You can treat this table just like any other table with a few exceptions. The only real major one is that you can't have foreign key constraints on a temporary table. The others are covered in Books Online.
Temporary tables are created in tempdb.
If you do this, you won't have to wrap it in a transaction.