deadlock found on MySql5.0 but not found after MySql5.6 - mysql

CREATE TABLE 'test'.'t1' (
'id' INT NULL);
CREATE TABLE 'test'.'t2' (
'id' INT NULL);
INSERT INTO test.t1 VALUES(1);
INSERT INTO test.t2 VALUES(1);
example1:
sqlConnection1:
SET autocommit = 0;
START TRANSACTION;
UPDATE test.t1 set id = 1 WHERE id = 2;
sqlConnection2:
SET autocommit = 0;
START TRANSACTION;
LOCK TABLES test.t2 WRITE,test.t1 WRITE;
COMMIT;
UNLOCK TABLES;
sqlConnection1:
UPDATE test.t2 set id = 1 where id = 2;
COMMIT;
sqlConnection2:
ERROR 1213 (40001): Deadlock found when trying to get lock; try restart transaction
example2:
sqlConnection1:
SET autocommit = 0;
START TRANSACTION;
UPDATE test.t1 set id = 1 WHERE id = 1;
sqlConnection2:
SET autocommit = 0;
START TRANSACTION;
LOCK TABLES test.t2 WRITE,test.t1 WRITE;
COMMIT;
UNLOCK TABLES;
sqlConnection1:
UPDATE test.t2 set id = 1 where id = 1;
COMMIT;
sqlConnection1:
ERROR 1213 (40001): Deadlock found when trying to get lock; try restart transaction
example3:
deadlock not found after mysql5.6
question:
What causes the results to differ in three examples

There are several points in this problem.
When InnoDB tries to update a record it will put a RECORD LOCK or a GAP LOCK onto the range it is going to update. For records that are existent, InnoDB will put a record lock, otherwise a gap lock. For example, in example 1, InnoDB will add a gap lock, and in this case, it adds a [2,∞) gap lock onto the table.
MySQL's LOCK TABLES command actually implements in MySQL itself but not in InnoDB. Although MySQL's LOCK TABLE has a "deadlock-free design" in other table-lock-only storage engines like MyISAM, it cannot avoid the deadlocks in InnoDB since InnoDB has row-level locks. Fortunately, the InnoDB has a deadlock detection mechanism that can detect when an InnoDB row lock and a MySQL table lock occur together on the same table and do what you see -- rollback a "lighter" transaction to avoid deadlock, which is so-called "victim".
The InnoDB engine will generally decide the transaction with fewer rows affected to be the victim. In example 1, connection 1 puts two GAP LOCKS on rows [2,∞) for two tables, while connection 2 only acquires two WRITE LOCKS and does not get anything done. Thus, connection 2 is "lighter" and being selected as the "victim". In example 2, connection 1 only has two RECORD LOCKS while connection 2 still has two WRITE LOCKS, and connection 1 basically do not do anything. Thus InnoDB decides that connection 1 uses fewer resources (i.e. locks in this case), and chooses connection 1 as the victim.
Since MySQL 5.6, a new lock wait mechanism has been introduced to MySQL. When MySQL cannot acquire a table lock, it will wait as long as the interval lock-wait-timeout specified. quote from the MySQL document:
This variable[lock-wait-timeout] specifies the timeout in seconds for attempts to acquire metadata locks. The permissible values range from 1 to 31536000 (1 year). The default is 31536000.
This timeout applies to all statements that use metadata locks. These include DML and DDL operations on tables, views, stored procedures, and stored functions, as well as LOCK TABLES, FLUSH TABLES WITH READ LOCK, and HANDLER statements.
Thus, MySQL will not acquire a table lock when it sees InnoDB already has a lock on the table it is going to manipulate and wait until InnoDB gets its work done. For further information, you can refer to MySQL document(The following documents are for 8.0 version, but the basic principles are the same):
https://dev.mysql.com/doc/refman/8.0/en/innodb-locks-set.html
https://dev.mysql.com/doc/refman/8.0/en/innodb-deadlock-detection.html

Related

Why set a shared lock when duplicate-key error occurs during insert

As Locks Set by Different SQL Statements in InnoDB, INSERT section said,
INSERT sets an exclusive lock on the inserted row.
Prior to inserting the row, a type of gap lock called an insert intention gap lock is set.
If a duplicate-key error occurs, a shared lock on the duplicate index record is set.
I wonder why set a shared lock, as the subsequent doc said, this might lead to a deadlock. Why not simply request an exclusive lock?
This use of a shared lock can result in deadlock should there be multiple sessions trying to insert the same row if another session already has an exclusive lock.
The below example copied from the MySQL document which demonstrate how to trigger the deadlock.
CREATE TABLE t1 (i INT, PRIMARY KEY (i)) ENGINE = InnoDB;
Now suppose that three sessions perform the following operations in order:
Session 1:
START TRANSACTION;
INSERT INTO t1 VALUES(1);
Session 2:
START TRANSACTION;
INSERT INTO t1 VALUES(1);
Session 3:
START TRANSACTION;
INSERT INTO t1 VALUES(1);
Session 1:
ROLLBACK;

Why does switching from a shared lock to an exclusive lock cause a deadlock and rolls back all other blocked transactions?

I was testing some queries on MySQL 5.7.26 and noticed something strange (?). Why does a deadlock occur when the below commands are entered in the following order:
session 1:
start transaction;
SELECT * FROM test_table WHERE id = 1 LOCK IN SHARE MODE;
session 2:
UPDATE test_table set col_1 = "foobar" WHERE id = 1;
# blocks
session 1:
SELECT * FROM test_table WHERE id = 1 FOR UPDATE;
session 2:
# displays ERROR 1213 (40001): Deadlock found when trying to get lock; try restarting transaction
I ask this because I feel like this is related to something I encountered in a web app I am working on which deals with a child table containing a foreign key to a certain parent table. It goes something like the following and also leads to a deadlock (hopefully it's simple enough to follow):
session 1:
start transaction;
INSERT INTO child_table (parent_id, col1) VALUES (1, "a");
session 2:
UPDATE parent_table SET col2 = "foo" WHERE id = 1;
# blocks, because the insert query in session 1 creates a shared lock on the related parent_table row
session 1:
UPDATE parent_table SET col2 = "bar" WHERE id = 1;
session 2:
# displays ERROR 1213 (40001): Deadlock found when trying to get lock; try restarting transaction
Guess I should read the docs more. An example exists in the link below that is exactly the same as my question
https://dev.mysql.com/doc/refman/8.0/en/innodb-deadlock-example.html
Basically: "Deadlock occurs here because client A needs an X lock to update the row. However, that lock request cannot be granted because client B already has a request for an X lock and is waiting for client A to release its S lock. Nor can the S lock held by A be upgraded to an X lock because of the prior request by B for an X lock. As a result, InnoDB generates an error for one of the clients and releases its locks"

Why MySQL InnoDB set an S or X Next-Key lock on the duplicate index record when a duplicate-key error occurs?

MySQL document (https://dev.mysql.com/doc/refman/8.0/en/innodb-locks-set.html) mentioned,
If a duplicate-key error occurs, a shared lock on the duplicate index record is set. This use of a shared lock can result in deadlock should there be multiple sessions trying to insert the same row if another session already has an exclusive lock. ...
...
INSERT ... ON DUPLICATE KEY UPDATE differs from a simple INSERT in that an exclusive lock rather than a shared lock is placed on the row to be updated when a duplicate-key error occurs.
and I've read the source code(https://github.com/mysql/mysql-server/blob/f8cdce86448a211511e8a039c62580ae16cb96f5/storage/innobase/row/row0ins.cc#L1930) that corresponding this situation, InnoDB indeed set the S or X lock when a duplicate-key error occurs.
if (flags & BTR_NO_LOCKING_FLAG) {
/* Set no locks when applying log
in online table rebuild. */
} else if (allow_duplicates) {
... ...
/* If the SQL-query will update or replace duplicate key we will take
X-lock for duplicates ( REPLACE, LOAD DATAFILE REPLACE, INSERT ON
DUPLICATE KEY UPDATE). */
err = row_ins_set_rec_lock(LOCK_X, lock_type, block, rec, index, offsets, thr);
} else {
... ...
err = row_ins_set_rec_lock(LOCK_S, lock_type, block, rec, index, offsets, thr);
}
But I wonder why InnoDB has to set such locks, it seems that these locks will bring more problems than they solve(they solved this problem: MySQL duplicate key error causes a shared lock set on the duplicate index record?).
Firstly, it can result in deadlock easily, the same MySQL document showed 2 examples about the deadlock.
Worse, the S or X lock is not single index-record lock, it is Next Key lock and may refuse many values to be inserted rather than just one duplicated value.
e.g.
CREATE TABLE `t` (
`id` int NOT NULL AUTO_INCREMENT,
`c` int DEFAULT NULL,
`d` int DEFAULT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `uniq_idx_c` (`c`)
) ENGINE=InnoDB AUTO_INCREMENT=48 DEFAULT CHARSET=utf8mb4
mysql> select * from t;
+----+------+------+
| id | c | d |
+----+------+------+
| 30 | 10 | 10 |
| 36 | 100 | 100 |
+----+------+------+
mysql> show variables like '%iso%';
+-----------------------+-----------------+
| Variable_name | Value |
+-----------------------+-----------------+
| transaction_isolation | REPEATABLE-READ |
+-----------------------+-----------------+
1 row in set (0.41 sec)
# Transaction 1
mysql> begin;
Query OK, 0 rows affected (0.00 sec)
mysql> insert into t values (null, 100, 100);
ERROR 1062 (23000): Duplicate entry '100' for key 't.uniq_idx_c'
# not commit
# Transcation 2
mysql> insert into t values(null, 95, 95);
ERROR 1205 (HY000): Lock wait timeout exceeded; try restarting transaction
mysql> insert into t values(null, 20, 20);
ERROR 1205 (HY000): Lock wait timeout exceeded; try restarting transaction
mysql> insert into t values(null, 50, 50);
ERROR 1205 (HY000): Lock wait timeout exceeded; try restarting transaction
# All c in [10, 100] can not be inserted
The goal in an ACID database is that queries in your session have the same result if you try to run them again.
Example: You run an INSERT query that results in a duplicate key error. You would expect if you retry that INSERT query, it would again fail with the same error.
But what if another session updates the row that caused the conflict, and changes the unique value? Then if you retry your INSERT, it would succeed, which is unexpected.
InnoDB has no way to implement true REPEATABLE-READ transactions when your statements are locking. E.g. INSERT/UPDATE/DELETE, or even SELECT with the locking options FOR UPDATE, FOR SHARE, or LOCK IN SHARE MODE. Locking SQL statements in InnoDB always act on the latest committed version of a row, not the version of that row that is visible to your session.
So how can InnoDB simulate REPEATABLE-READ, ensuring that the row affected by a locking statement is the same as the latest committed row?
By locking rows that are indirectly referenced by your locking statement, preventing them from being changed by other concurrent sessions.
Another possible explain I found in MySQL source code is row0ins.cc Line 2141
We set a lock on the possible duplicate: this
is needed in logical logging of MySQL to make
sure that in roll-forward we get the same duplicate
errors as in original execution

'metadatalock' occured when I did 'lock tables [table_name] write'

I did it like below
-- connection 1
START TRANSACTION;
LOCK TABLES [table_name] WRITE;
-- connection 2
START TRANSACTION;
SELECT * FROM [table_name]; -- waiting for table metadata lock
I don't understand above situation.
I thought WRITE LOCK doesn't prevent SELECT query.
When I did like below,
-- connection 1
START TRANSACTION;
UPDATE [table_name] SET [column = value] WHERE id = 1;
-- connection 2
START TRANSACTION;
SELECT * FROM [table_name] WHERE id = 1; -- doesn't wait
SELECT doesn't wait as you saw.
And I thought WRITE LOCK also work like UPDATE clause.
But it seems like not..
Now I know something wrong?
Write lock does not work like row-level locking.
https://dev.mysql.com/doc/refman/8.0/en/lock-tables.html says:
WRITE lock:
Only the session that holds the lock can access the table. No other session can access it until the lock is released.
"Access" in this context means read or write.

How locks (S,X,IS,IX) work in Mysql with queries like FOR UPDATE/LOCK IN SHARE MODE?

1:
I was trying this and it was working fine:
start transaction;
select * from orders where id = 21548 LOCK IN SHARE MODE;
update orders set amount = 1500 where id = 21548;
commit;
According to the definition of LOCK IN SHARE MODE , it locks the table with IS lock and lock the selected rows with S lock.
When a row is locked with S lock.How can it be modified without releasing lock?
It needs X lock to modify it.Right? Or is it valid only for different connection transaction?
2:
//session1
start transaction;
select * from orders where id = 21548 FOR UPDATE;
Keep this session1 same and try this in the different session:
//session2
select * from orders where id = 21548; //working
update orders set amount = 2000 where id = 21548; //waiting
FOR UPDATE locks the entire table into IX mode and selected row into X mode.
As X mode is incompatible with S mode then how come select query in second session is getting executed?
One answer might be that select query is not asking for S lock that's why it's running successfully.But update query in the second session is also not asking for X lock , but as you execute it , it starts waiting for the lock held by session1.
I have read a lot of stuff regarding this but not able to clear my doubts.Please help.
Bill Karwin answered this question through email.He said:
The same transaction that holds an S lock can promote the lock to an X lock. This is not a conflict.
The SELECT in session 1 with FOR UPDATE acquires an X lock. A simple SELECT query with no locking clause specified does not need to acquire an S lock.
Any UPDATE or DELETE needs to acquire an X lock. That's implicit. Those statements don't have any special locking clause for that.
For more details on IS/IX locks and FOR UPDATE/LOCK IN SHARE MODE please visit
shared-and-exclusive-locks .