I'm seeing a pending insert lock up a MyISAM table. The query itself isn't a standout, but in the FULL PROCESSLIST it looks like there's an INSERT with a lock and a bunch of SELECTs waiting on it.
The query itself isn't a standout. Reading the MySQL docs, I see something that is a standout: "If there are holes, concurrent inserts are disabled."
I presume this means holes in the index, and in my index here there is definitely "holes" in the sense that there are not consecutive ID numbers in the primary key. So I'm wondering: if concurrency is disabled and some INSERT hangs, how long is it going to hold that lock and will it hold that lock on the entire table (and not just the row)?
MyISAM does not support row level locking, so the lock is on entire table. I didn't find any information about lock timeout, it seems that insert can theoretically hang forever.
Related
I have requirement where we need to update the row without holding the lock for the while updating.
Here is the details of the requirements, we will be running a batch processing on a table every 5 mins update blogs set is_visible=1 where some conditions this query as to run on millions of records so we don't want to block all the rows for write during updates.
I totally understand the implications of not having write locks which is fine for us because is_visible column will be updated only by this batch process no other thread wil update this column. On the other hand there will be lot of updates to other columns of the same table which we don't want to block
First of all, if you default on the InnoDB storage engine of MySQL, then there is no way you can update data without row locks except setting the transaction isolation level down to READ UNCOMMITTED by running
SET TRANSACTION ISOLATION LEVEL READ UNCOMMITTED;
However, I don't think the database behavior is what you expect since the dirty read is allowed in this case. READ UNCOMMITTED is rarely useful in practice.
To complement the answer from #Tim, it is indeed a good idea to have a unique index on the column used in the where clause. However, please note as well that there is no absolute guarantee that the optimizer will eventually choose such execution plan using the index created. It may work or not work, depending on the case.
For your case, what you could do is to split the long transaction into multiple short transactions. Instead of updating millions of rows in one shot, scanning only thousands of rows each time would be better. The X locks are released when each short transaction commits or rollbacks, giving the concurrent updates the opportunity to go ahead.
By the way, I assume that your batch has lower priority than the other online processes, thus it could be scheduled out of peak hours to further minimize the impact.
P.S. The IX lock is not on the record itself, but attached to the higher-granularity table object. And even with REPEATABLE READ transaction isolation level, there is no gap lock when the query uses a unique index.
Best practice is to always acquire a specific lock when there is a chance that an update could happen concurrently with other transactions. If your storage engine be MyISAM, then MySQL will lock the entire table during an update, and there isn't much you can do about that. If the storage engine be InnoDB, then it is possible that MySQL would only put an exclusive IX lock on the records targeted by the update, but there are caveats to this being the case. The first thing you would do to try to achieve this would be a SELECT ... FOR UPDATE:
SELECT * FROM blogs WHERE <some conditions> FOR UPDATE;
In order to ensure that InnoDB only locks the records being updated, there needs to be a unique index on the column which appears in the WHERE clause. In the case of your query, assuming id were the column involved, it would have to be a primary key, or else you would need to create a unique index:
CREATE UNIQUE INDEX idx ON blogs (id);
Even with such an index, InnoDB may still apply gap locks on the records in between index values, to ensure that the REPEATABLE READ contract is enforced.
So, you may add an index on the column(s) involved in your WHERE clause to optimize the update on InnoDB.
I have a MYISAM table on which long-running SELECT statements are conducted. Do these statements cause locks on the tables that are being read? If so, how do you stop the locks from occurring?
According to https://dev.mysql.com/doc/refman/5.7/en/internal-locking.html:
MySQL uses table-level locking for MyISAM, MEMORY, and MERGE tables,
permitting only one session to update those tables at a time. This
locking level makes these storage engines more suitable for read-only,
read-mostly, or single-user applications.
Does this also mean that SELECT statements create locks?
Do these [long-running SELECT] statements cause locks on the tables that are being read?
Yes. A MyISAM table cannot be updated while it is being read from. If there is a long-running read (i.e, SELECT) query on the table, any UPDATE or DELETE queries will have to wait until the read query is complete.
(INSERT queries are sometimes an exception. So long as there are no internal holes in the table structure, a writer may be allowed to insert a row at the end of a table without obtaining a lock. However, if any rows are ever updated or deleted, this may create holes, which will require INSERT queries to obtain a write lock as well.)
For more information on the internals, read up on table-level locks. But the short and the long of matters is, you probably shouldn't use MyISAM if concurrency is a concern.
I have an issue with table locking in InnoDB on delete operation.
I have a table queue with for example one column and a lot of transactions which can insert rows into this queue or delete them.
There isn't any two transactions working with the same rows at the same time. So, all row locks must be distinct.
But sometimes when delete operation deletes the most part of rows in the table, InnoDB prefers to use table lock instead of row lock and that causes deadlocks.
I can't reproduce this deadlock exactly, but I found that lock problem.
i.e. I have table queue:id with values(1,3,4,5,6,7)
Transaction 1:
insert into queue value(2);
Transaction 2:
delete from queue where id in (1,3,4,5,6,7); -- here the lock comes
First of all assuming id is a primary key or at least indexed column.
Insert should not lock the table, so chances are any other update/delete query is executing at same time of deletion the records.
If it is not the case then it can be due to "gap locking" as mentioned #a_horse_with_no_name.
So at which time you get this issue again then you need to store all processes "show full processlist" at your end and also check "show engine innodb status" where it will show you processids related with deadlock, this will help you to get exact problem.
Further You can avoid this locking to delete all rows one by one based on primary key.
My question is a follow up to this answer. I want to find out how to perform a select statement without locking a table with MyISAM engine.
The answer states the following if you have InnoDB but not MyISAM. What is the equivalent for MyISAM engine?
SET TRANSACTION ISOLATION LEVEL READ UNCOMMITTED ;
SELECT * FROM TABLE_NAME ;
COMMIT ;
This is the default behaviour with MyISAM tables. If one actually wants to lock a MyISAM table, one must manually acquire a table-level lock. Transaction isolation level, START TRANSACTION, COMMIT, ROLLBACK have no effect on MyISAM tables behaviour since MyISAM does not support transactions.
More about internal locking mechanisms
A READ lock is implicitely acquired before, and released after execution of a SELECT statement. Notice that several concurrent, simultaneous, SELECT statements could be running at the same time, because several sessions may hold a READ lock on the same table.
Conversely, a WRITE lock is implicitely acquired before executing an INSERT or UPDATE or DELETE statement. This means that no read (let alone a concurrent write) can take place as long as a write is in progress*.
The above applies to MyISAM, MEMORY, and MERGE tables only.
You might want to read more about this here:
Internal locking methods
Read vs Write locks
* However, these locks are not always required thanks to this clever trick:
The MyISAM storage engine supports concurrent inserts to reduce contention between readers and writers for a given table: If a MyISAM table has no free blocks in the middle of the data file, rows are always inserted at the end of the data file. In this case, you can freely mix concurrent INSERT and SELECT statements for a MyISAM table without locks.
MyISAM does indeed use a read lock during SELECT. An INSERT at the end of the table can get around that.
But try doing an UPDATE, DELETE, or ALTER TABLE while a long-running SELECT is in progress. Or vice-versa, reading from a table while a change to that table is running. It's first-come, first-serve, and the later thread blocks until the first thread is done.
MyISAM doesn't have any support for transactions, so it must work this way. If a SELECT were reading rows from a table, and a concurrent thread changes some of those rows, you would get a race condition. The SELECT may read some of the rows before the change, and some of the rows after the change, resulting in a completely mixed-up view of the data.
Anything you do with SET TRANSACTION ISOLATION LEVEL has no effect with MyISAM.
For these reasons, it's recommended to use InnoDB instead.
i am reading the book High Performance MySQL, it mentions:
performing one query per table uses table locks more efficiently: the queries
will lock the tables invididually and relatively briefly, instead of locking
them all for a longer time.
MyISAM places table-lock even when selecting something? can someone explain a little bit?
MyISAM has different kinds of locks. A SELECT operation places a READ LOCK on the table. There can be multiple active read locks at any given time, as long as there are no active WRITE LOCKS. Operations that modify the table, eg. INSERT, UPDATE, DELETE or ALTER TABLE place a WRITE LOCK on the table. Write lock can only be placed on a table when there are no active read locks; If there are active read locks, MyISAM queues the write lock to be activated as soon as all active read locks are expired.
Likewise when there's an active write lock, attempting to place a read lock on a table will queue the lock (and the associated query) until write locks have expired on the table.
Ultimately this all means that:
You can have any number of active read locks (also called shared locks)
You can only have one active write lock (also called an exclusive lock)
For more information see: http://dev.mysql.com/doc/refman/5.5/en/internal-locking.html
reko_t provided a good answer, I will try to elaborate on it:
Yes.
You can have EITHER one writer or several readers
Except there is a special case, called concurrent inserts. This means that you can have one thread doing an insert, while one or more threads are doing select (read) queries.
there are a lot of caveats doing this:
it has to be "at the end" of the table - not in a "hole" in the middle
Only inserts can be done concurrently (no updates, deletes)
There is still contention on the single MyISAM key buffer. There is a single key buffer, protected by a single mutex, for the whole server. Everything which uses an index needs to take it (typically several times).
Essentially, MyISAM has poor concurrency. You can try to fake it, but it's bad whichever way you look at it. MySQL / Oracle has made no attempts to improve it recently (looking at the source code, I'm not surprised - they'd only introduce bugs).
If you have a workload with lots of "big" SELECTs which retrieve lots of rows, or are hard in some way, they may often overlap, this may seem ok. But a single row update or delete will block the whole lot of them.