According to the official docs from here:
the lock compatibility matrix:
X IX S IS
X Conflict Conflict Conflict Conflict
IX Conflict Compatible Conflict Compatible
S Conflict Conflict Compatible Compatible
IS Conflict Compatible Compatible Compatible
The docs also say:
Thus, intention locks do not block anything except full table requests
(for example, LOCK TABLES ... WRITE). The main purpose of IX and IS
locks is to show that someone is locking a row, or going to lock a row
in the table.
If the intention locks only block full table requests, then how to explain the IX conflicts with S lock in the above lock compatibility matrix? To my understanding, the S and X in the lock compatibility matrix are both record locks, it's that right?
To my understanding, the S and X in the lock compatibility matrix are both record locks, it's that right?
That is correct. Incorrect is the assumption that you can directly compare table and record locks, which the documentation probably does not make completely clear, and the part "These rules can be conveniently summarized by means of the following lock type compatibility matrix" might be a bit misleading, as it does not cover everything (namely any conflict information about S/X-table locks with record locks).
Technically, that matrix defines the result when checking locks on some object, e.g. whenever MySQL tries to add a lock to something. If you try to get an S lock on a table, it would conflict with an IX lock on that table.
If a record could have an intention lock, it would conflict there too. Just because a lockable object doesn't use intention locks doesn't change the (general) compatibility matrix.
Technically, the internal datatype for locks is the same for records and tables, the intention locks are just never set for records. A record lock will actually never be compared to a table lock (as these are two different objects), and the only reason a record lock interferes with a table lock is the locking protocol (which requires a lock on both the table and the record to lock a record).
So to lock a record, you will typically require a different table lock. The compatibity matrix is the same, but the value for the table can and usually will differ from the value of the record.
So
Thus, intention locks do not block anything except full table requests (for example, LOCK TABLES ... WRITE).
is correct because only full table requests require a lock that conflicts with an existing intention lock, and records do not use intention locks. But to repeat it: you still have to compare tow different locks. An S-lock on a record cannot conflict with the lock on a table, as these two object will never be compared.
The manual is mixing table and record locks a bit. It actually defines IS and IX as:
Intention shared (IS): Transaction T intends to set S locks on individual rows in table t.
Intention exclusive (IX): Transaction T intends to set X locks on those rows.
so if you want, IS and IX in the matrix can somewhat be interpreted as properties of rows (which they technically are not), while you read them as the lock on the table (as it can only be set for a table, but which is a different lock). But the matrix still only describes the situation to compare records (which the manual probably does not make clear enough), and it does not include any compatibility information with an S or X table lock.
So to summarize: you do not need "to explain the IX conflicts with S lock in the above lock compatibility matrix", as it simply does not cover that situation.
Related
I'm using MySQL InnoDB tables and trying to understand the reasons for some row-level locking in the case of an index range scan. I found that an extra index record (out of range) may be locked depending on the uniqueness of the index used. See the example below (verified in version 8.0.18).
CREATE TABLE foo (
a INT NOT NULL,
b INT NOT NULL,
c CHAR(1) NOT NULL,
PRIMARY KEY (a),
UNIQUE KEY (b)
) ENGINE=InnoDB;
INSERT INTO foo VALUES (1,1,'A'), (3,3,'B'), (5,5,'C'), (7,7,'D'), (9,9,'E');
Test case 1
Session 1:
START TRANSACTION;
SELECT * FROM foo WHERE a < 2 FOR UPDATE;
Session 2:
DELETE FROM foo WHERE a = 3; -- Success
Test case 2
This uses the original rows of the table with the deleted record returned.
Session 1:
START TRANSACTION;
SELECT * FROM foo WHERE b < 2 FOR UPDATE;
Session 2:
DELETE FROM foo WHERE b = 3; -- Blocks
Locking the secondary index record with b = 3 in the second test case looks unnecessary.
Why does InnoDB block the next index entry to the right of the scanned range in case of a secondary index? Is there any practical reason for this?
Can someone give an example of a problem that could happen if the record with b = 3 is not blocked in the second test case?
Finally I found the answer. In short, there are no significant reasons for such additional locking in the second test case. When a locking read of a secondary index range is performed, sufficient locks are set, but not a necessary minimum. Thus, extra locks are set only because it was easier for some InnoDB programmers to write code. Who cares about extra locks if everything works for the most part?
I posted a bug report about this issue: https://bugs.mysql.com/bug.php?id=98639
Unfortunately, their employee does not want to register this bug. He does not understand my arguments and comes up with erroneous explanations. He made my last arguments private and stopped responding.
I also asked about this issue in the official forum and received the following answer: https://forums.mysql.com/read.php?22,684356,684482
In short, significant efforts are required to fix this bug. But since this is a very small and insignificant bug (more precisely, a performance issue), they do not want to fix it yet. However, in version 8.0.18, they fixed a similar problem for the clustered index, and it took them more than a month.
I'm very surprised that optimizing such a simple single-level scanning algorithm takes so much time and is so difficult for the MySQL team.
Like #Barmar already mentioned it is so because MySQL is setting a GAP or a Next-key lock. I assume that you are using the default innodb isolation level REPEATABLE READ
The MySQL documentation says:
For locking reads (SELECT with FOR UPDATE or LOCK IN SHARE MODE), UPDATE, and DELETE statements, locking depends on whether the statement uses a unique index with a unique search condition, or a range-type search condition.
For a unique index with a unique search condition, InnoDB locks only the index record found, not the gap before it.
For other search conditions, InnoDB locks the index range scanned, using gap locks or next-key locks to block insertions by other sessions into the gaps covered by the range. For information about gap locks and next-key locks, see Section 14.7.1, “InnoDB Locking”.
See:https://dev.mysql.com/doc/refman/5.6/en/innodb-transaction-isolation-levels.html#isolevel_repeatable-read
At the moment i'm looking for info on how the gap lock is set, but I also got this info, and I guess it is helpfull information:
Keep in mind that the lock is based on a internal index because the column that you are using in your criteria is not uniquely indexed.
A gap lock is a lock on a gap between index records, or a lock on the gap before the first or after the last index record
Gap locks in InnoDB are “purely inhibitive”, which means that their only purpose is to prevent other transactions from inserting to the gap. Gap locks can co-exist. A gap lock taken by one transaction does not prevent another transaction from taking a gap lock on the same gap. There is no difference between shared and exclusive gap locks. They do not conflict with each other, and they perform the same function.
Gap locking can be disabled explicitly. This occurs if you change the transaction isolation level to READ COMMITTED or enable the innodb_locks_unsafe_for_binlog system variable (which is now deprecated). Under these circumstances, gap locking is disabled for searches and index scans and is used only for foreign-key constraint checking and duplicate-key checking.
There are also other effects of using the READ COMMITTED isolation level or enabling innodb_locks_unsafe_for_binlog. Record locks for nonmatching rows are released after MySQL has evaluated the WHERE condition. For UPDATE statements, InnoDB does a “semi-consistent” read, such that it returns the latest committed version to MySQL so that MySQL can determine whether the row matches the WHERE condition of the UPDATE.
see https://dev.mysql.com/doc/refman/5.6/en/innodb-locking.html#innodb-gap-locks
According to All about locking in SQL Server, a update lock can be transformed into a exclusive lock when there is something to write. Meanwhile, the compatibility of the three locks(X,S and U) can be refer to the following table.
X S U
X ✗ ✗ ✗
S ✗ ✓ ✓
U ✗ ✓ ✗
However, It is mentioned in some blogs that there is a SX lock since MySQL 5.7 which implements an idea from the paper Concurrency of operations on B-trees(1977). From these blogs I find the SX lock is quite similar with the update lock. For example, they have the same compatibility table.
Since I can't find more "official" introduction about the SX lock in MySQL, I want to ask what's the difference between both of the locks?
Since the compatibility matrix defines a lock, having the same matrix makes the update lock and the sx lock equivalent.
Actually, in database theory, an update lock should behave asymmetrical, disallowing a new shared lock when there is already an update lock, see e.g. these lecture notes, penultimate page. Your reference also mentions this:
It is important to understand that update lock is asymmetrical in regards of shared locks. While the update lock can be imposed on a record that has the shared lock, the shared lock cannot be imposed on the record that already has the update lock
Practically though, both the MySQL sx-lock and the SQL Server (and others) update lock are symmetrical, like in your matrix.
The purpose of an update lock is to allow other transactions to still read the data (using a shared lock) while preventing a deadlock when two transactions want to escalate to a exclusive lock (as neither can do that when the other has a shared lock).
So, to conclude, the lock types are, in principle, the same, although I could imagine that they are named differently to emphasize that they are not intended to be asymmetric, and maybe to leave an option to later add the "real" update lock, so you may or may not want to treat them as unequal in that regard.
But there are major differences in how they are used. For example, in contrast to SQL Server, MySQL does not use update locks on rows: MySQL uses exclusive locks for updates, and to allow for concurrent reads, usually does not lock while reading, unless you use locking reads.
After acid requirements, the detailled behaviour (when which lock is applied on what object for what operation) is mostly relevant for performance and concurrency. While you could just lock the whole database for every query, this would only allow one query at a time, so the more fine-grained you lock specific objects, the more concurrency you allow (with the tradeoff to increase the risk of deadlocks).
Now to the purpose of the sx-locks in MySQL:
They were introduced to allow a more fine-tuned locking of indexes, see MySQL-5.7 improves DML oriented workloads, as some operations on indexes required the complete index to be exclusively locked, hindering concurrency. So the reason they are not more present int the documentation is that, although there might be other use cases for this type in the future, they are currently limited to very specific internal locking.
I have a query such as
Select count(*) from table log where num = ?;
If I set the isolation level to serializable, then the range lock will be acquired for the where clause.
My question is: Can other transaction also acquire the range lock in share mode to read the count as the above OR the range lock is exclusive and all other transactions have to wait until the current transaction commits before executing the above read query.
Background: I am trying to implement a view counter for heavy traffic website. To reduce IO to the database, I create a log table so that every time there is a view, I only write a new row in the log table. Once a while, I (randomly) decide if I want to clear the log table and add the number of rows in the log table into a column of a view count table. This means I have to be careful with interleaving transaction.
The statements below are relevant only to SQL Server and were made before the OP made clear this was really about MySQL, about which I know nothing. I'm leaving it here since it (and the resulting discussion) might be of some use nevertheless, but it is not a complete, relevant answer to the question.
SELECT statements only ever acquire shared locks, on all isolation levels (unless overridden with a table hint). And shared locks are always compatible with each other (see Lock Compatibility), so there's no problem if other transactions want to acquire shared (range) locks as well. So yes, you can have any number of queries performing SELECT COUNT(*) in parallel and they will never block each other.
This doesn't mean other transactions don't have to wait. In particular, a DELETE query must eventually acquire an exclusive lock, and it will have to wait if the SELECT is holding a shared lock. Normally this is not an issue since the engine releases locks as soon as possible. When it does become an issue, you'll want to look at solutions like snapshot isolation, which uses optimistic concurrency and conflict detection rather than locking. Under that model, a SELECT will never block any other query (save those that want table locks). Of course, this isn't free; the row versioning is uses takes up disk space and I/O.
MySQL documentation says that SELECT FOR UPDATE sets an IX lock. IX lock is intention exclusive lock and when issued it means "Transaction T intends to set X (exclusive) locks on scanned rows". This means that before SELECT FOR UPDATE succeeds it must first get IX and then X. MySQL glossary says this about intention exclusive lock:
intention lock
A kind of lock that applies to the table level, used to indicate what kind of lock the transaction intends to acquire on rows in the table. Different transactions can acquire different kinds of intention locks on the same table, but the first transaction to acquire an intention exclusive (IX) lock on a table prevents other transactions from acquiring any S or X locks on the table. Conversely, the first transaction to acquire an intention shared (IS) lock on a table prevents other transactions from acquiring any X locks on the table. The two-phase process allows the lock requests to be resolved in order, without blocking locks and corresponding operations that are compatible. For more details on this locking mechanism, see Section 14.3.5.3, “InnoDB Lock Modes”.
Also IX and IX are compatible (lock type compatibility matrix) which means that if transaction 1 issues IX and right after another concurrent transaction issues IX it will succeed.
Is there a possibility that two concurrent IX are issued at the exact same moment and MySQL grants/acquires IX for both transactions for the same table. Or MySQL grants at any point only one IX if concurrent IX are issued. I'm thinking that MySQL grants only one of them, even if call is made and triggered at MySQL side at the exact same time.
EDIT: basically if I generalize my question: If two (concurrent) sql statements that lock rows (e.g. update, select for update, select lock in share mode, insert, delete) come to MySQL at the exact same time, I suppose MySQL treats them sequentially. Just want to make sure, that I am thinking right how MySQL works internally.
Locking operations necessarily are serialized. At the nanosecond level in the server's logical workflow there's no such thing as "multiple lock requests at precisely the same moment." Even if there were, the server's logic would arbitrarily place them into some order and grant them one after the other.
A really smart next-generation massively parallel server might be able to figure out that different lock requests were guaranteed never to interfere with each other, and handle them truly in parallel. But for now, there's no such thing as simultaneous.
i am reading the book High Performance MySQL, it mentions:
performing one query per table uses table locks more efficiently: the queries
will lock the tables invididually and relatively briefly, instead of locking
them all for a longer time.
MyISAM places table-lock even when selecting something? can someone explain a little bit?
MyISAM has different kinds of locks. A SELECT operation places a READ LOCK on the table. There can be multiple active read locks at any given time, as long as there are no active WRITE LOCKS. Operations that modify the table, eg. INSERT, UPDATE, DELETE or ALTER TABLE place a WRITE LOCK on the table. Write lock can only be placed on a table when there are no active read locks; If there are active read locks, MyISAM queues the write lock to be activated as soon as all active read locks are expired.
Likewise when there's an active write lock, attempting to place a read lock on a table will queue the lock (and the associated query) until write locks have expired on the table.
Ultimately this all means that:
You can have any number of active read locks (also called shared locks)
You can only have one active write lock (also called an exclusive lock)
For more information see: http://dev.mysql.com/doc/refman/5.5/en/internal-locking.html
reko_t provided a good answer, I will try to elaborate on it:
Yes.
You can have EITHER one writer or several readers
Except there is a special case, called concurrent inserts. This means that you can have one thread doing an insert, while one or more threads are doing select (read) queries.
there are a lot of caveats doing this:
it has to be "at the end" of the table - not in a "hole" in the middle
Only inserts can be done concurrently (no updates, deletes)
There is still contention on the single MyISAM key buffer. There is a single key buffer, protected by a single mutex, for the whole server. Everything which uses an index needs to take it (typically several times).
Essentially, MyISAM has poor concurrency. You can try to fake it, but it's bad whichever way you look at it. MySQL / Oracle has made no attempts to improve it recently (looking at the source code, I'm not surprised - they'd only introduce bugs).
If you have a workload with lots of "big" SELECTs which retrieve lots of rows, or are hard in some way, they may often overlap, this may seem ok. But a single row update or delete will block the whole lot of them.