MySQL deadlock on one table with multithreading code - mysql
I've created an application which gathers prices and stores it in a database.
The application runs with two threads. But now and then a deadlock occurs. I think it is because the treads insert a bulk of prices to the same table.
The table
[tripid | date | duration | price | garant | updatetime | persons | accommodationid]
The indexes
unique (tripid, date, duration, price, persons)
foreign key 'accommodationid' to table 'accommodation.id'
The query
insert into prices (tripid, date, duration, price, persons, accomodationid)
values ( 1 , 2016-6-4, 8, 200,2,32),
... a whole lot more ...
( 1 , 2016-7-4, 8, 200,2,32)
on duplicate key update price = values(price);
Becouse there can be 1000 values to insert in 1 query it takes some time. I think the deadlock occurs when the second thread wants to insert another 1000 values but the first one is not completed.
The innodb status
------------------------
LATEST DETECTED DEADLOCK
------------------------
2016-03-16 23:13:13 0xef8
*** (1) TRANSACTION:
TRANSACTION 3732195928, ACTIVE 0 sec inserting
mysql tables in use 1, locked 1
LOCK WAIT 5 lock struct(s), heap size 1136, 3 row lock(s), undo log entries 1
MySQL thread id 415, OS thread handle 7328, query id 10956855 ---(ip)-- --name-- update
insert into `prices` (tripid,`date`,`duration`,`price`,`garant`,`updatetime`,`persons`,`accommodationid`) values
(179881,'2016-03-18',2,206,'0','2016-03-16','1','119'),(179881,'2016-03-18',3,260,'0','2016-03-16','1','119'),(179881,'2016-03-18',4,313,'0','2016-03-16','1','119'),(179881,'2016-03-18',5,366,'0','2016-03-16','1','119'),(179881,'2016-03-19',2,206,'0','2016-03-16','1','119'),(179881,'2016-03-19',3,260,'0','2016-03-16','1','119'),(179881,'2016-03-19',4,313,'0','2016-03-16','1','119'),(179881,'2016-03-19',5,366,'0','2016-03-16','1','119'),(179881,'2016-03-20',2,206,'0','2016-03-16','1','119'),(179881,'2016-03-20',3,260,'0','2016-03-16','1','119'),(179881,'2016-03-20',4,313,'0','2016-03-16','1','119'),(179881,'2016-03-20',5,366,'0','2016-03-16','1','119'),(179881,'2016-03-21',2,206,'0','2016-03-16','1','119'),(179881,'2016-03-21',3,260,'0','2016-03-16','1','119'),(179881,'2016-03-21',4,313,
*** (1) WAITING FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 247 page no 1457 n bits 136 index keyGroup of table `travel`.`prices` trx id 3732195928 lock_mode X insert intention waiting
Record lock, heap no 1 PHYSICAL RECORD: n_fields 1; compact format; info bits 0
0: len 8; hex 73757072656d756d; asc supremum;;
*** (2) TRANSACTION:
TRANSACTION 3732195927, ACTIVE 0 sec inserting, thread declared inside InnoDB 4529
mysql tables in use 1, locked 1
15 lock struct(s), heap size 1136, 476 row lock(s), undo log entries 472
MySQL thread id 414, OS thread handle 3832, query id 10956852 ---(ip)-- --name-- update
insert into `prices` (tripid,`date`,`duration`,`price`,`garant`,`updatetime`,`persons`,`accommodationid`) values
(179880,'2016-03-18',2,185,'0','2016-03-16','1','1438'),(179880,'2016-03-18',3,217,'0','2016-03-16','1','1438'),(179880,'2016-03-18',4,249,'0','2016-03-16','1','1438'),(179880,'2016-03-18',5,279,'0','2016-03-16','1','1438'),(179880,'2016-03-18',6,312,'0','2016-03-16','1','1438'),(179880,'2016-03-18',7,343,'0','2016-03-16','1','1438'),(179880,'2016-03-18',8,375,'0','2016-03-16','1','1438'),(179880,'2016-03-19',2,185,'0','2016-03-16','1','1438'),(179880,'2016-03-19',3,217,'0','2016-03-16','1','1438'),(179880,'2016-03-19',4,249,'0','2016-03-16','1','1438'),(179880,'2016-03-19',5,279,'0','2016-03-16','1','1438'),(179880,'2016-03-19',6,312,'0','2016-03-16','1','1438'),(179880,'2016-03-19',7,343,'0','2016-03-16','1','1438'),(179880,'2016-03-20',2,185,'0','2016-03-16','1','1438'),(179880,'2016
*** (2) HOLDS THE LOCK(S):
RECORD LOCKS space id 247 page no 1457 n bits 72 index keyGroup of table `travel`.`prices` trx id 3732195927 lock_mode X
Record lock, heap no 1 PHYSICAL RECORD: n_fields 1; compact format; info bits 0
0: len 8; hex 73757072656d756d; asc supremum;;
*** (2) WAITING FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 247 page no 1457 n bits 136 index keyGroup of table `travel`.`prices` trx id 3732195927 lock_mode X insert intention waiting
Record lock, heap no 1 PHYSICAL RECORD: n_fields 1; compact format; info bits 0
0: len 8; hex 73757072656d756d; asc supremum;;
*** WE ROLL BACK TRANSACTION (1)
------------
The Question
How do I avoid the deadlock?
I seem to have fixed it by first locking the table, then do the huge query and unlock it so the second thread gets acces.
LOCK TABLES prices WRITE;
insert into prices (tripid, date, duration, price, persons, accomodationid)
values ( 1 , 2016-6-4, 8, 200,2,32),
... a whole lot more ...
( 1 , 2016-7-4, 8, 200,2,32)
on duplicate key update price = values(price);
UNLOCK TABLES;
Related
Why is there a S-lock?
Isolation Level is: Repeat Read(RR)
MySQL 5.6 DeadLock log:
------------------------
LATEST DETECTED DEADLOCK
------------------------
2022-09-21 10:01:58 2b0d1fb0b700
*** (1) TRANSACTION:
TRANSACTION 19414864283, ACTIVE 0.448 sec inserting
mysql tables in use 1, locked 1
LOCK WAIT 9 lock struct(s), heap size 2936, 4 row lock(s), undo log entries 4
LOCK BLOCKING MySQL thread id: 8219895 block 8219858
MySQL thread id 8219858, OS thread handle 0x2b0d3978d700, query id 194299614451 10.111.76.151 test_trade update
insert into merchandise (merchandise_no, serial_no, `status`,
expand_status, quantity, title, describes)
values ('TR20220111100058055986', '20000666665555620435', 20,
10, 1, '',
'')
*** (1) WAITING FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 9057 page no 5298 n bits 600 index `uk_serial_no` of table `test_trade`.`merchandise` trx id 19414864283 lock_mode X locks gap before rec insert intention waiting
Record lock, heap no 144 PHYSICAL RECORD: n_fields 2; compact format; info bits 32
0: len 20; hex 3230323230363236313635343139373736383435; asc 20000666665555776845;;
1: len 4; hex 00f06178; asc ax;;
*** (2) TRANSACTION:
TRANSACTION 19414864275, ACTIVE 0.412 sec inserting
mysql tables in use 1, locked 1
8 lock struct(s), heap size 1184, 4 row lock(s), undo log entries 4
MySQL thread id 8219895, OS thread handle 0x2b0d1fb0b700, query id 194299614132 10.111.76.156 test_trade update
insert into merchandise (merchandise_no, serial_no, `status`,
expand_status, quantity, title, describes)
values ('TR20220111100058135388', '20000666665555630510', 20,
10, 1, '',
'')
*** (2) HOLDS THE LOCK(S):
RECORD LOCKS space id 9057 page no 5298 n bits 600 index `uk_serial_no` of table `test_trade`.`merchandise` trx id 19414864275 lock mode S
Record lock, heap no 144 PHYSICAL RECORD: n_fields 2; compact format; info bits 32
0: len 20; hex 3230323230363236313635343139373736383435; asc 20000666665555776845;;
1: len 4; hex 00f06178; asc ax;;
*** (2) WAITING FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 9057 page no 5298 n bits 600 index `uk_serial_no` of table `test_trade`.`merchandise` trx id 19414864275 lock_mode X locks gap before rec insert intention waiting
Record lock, heap no 144 PHYSICAL RECORD: n_fields 2; compact format; info bits 32
0: len 20; hex 3230323230363236313635343139373736383435; asc 20000666665555776845;;
1: len 4; hex 00f06178; asc ax;;
*** WE ROLL BACK TRANSACTION (2)
Table merchandise has a primary key and auto-incre column "id", and a unique key column "serial_no". From the deadlock log, I know then reason of DeadLock: the insert intention X-lock of transaction(2) waiting the insert intention X-lock of transaction(1),and then insert intention X-lock of transaction(1) waiting then S-lock of transaction(2).
I do not know 'Why is there a S-lock'? Then column serial_no is unique key, and there is not select sql in my server.
MySql dead lock root cause - Hibernate
I have a code snippet in hibernate, structured like below.
This method is a common method and when I use two different locks (In parallel stream) I end up having dead lock scenario. I am trying to understand the root cause of the dead lock.
[Spring/JPA/Hibernate/MySql]
#Transactional(isolation = Isolation.SERIALIZABLE)
public void someMethod(String lockName) { // Lock name can be of two types
jpaRepository.lock(lockName, TIMEOUT_10_SECONDS);
List<Object> values = jpaRepository.findByDate();
Integer sequence = jpaRepository.getNextSequenceValue(); // Fetches next value from sequence generator
//...... set sequence numbers to object
jpaRepository.updateSequence(); // A named query here to update sequence with latest value
jpaRepository.saveAll(); // JPA's persist call
}
Innodb status
------------------------
LATEST DETECTED DEADLOCK
------------------------
2022-03-24 00:05:03 0x16c113000
*** (1) TRANSACTION:
TRANSACTION 2000, ACTIVE 0 sec inserting
mysql tables in use 1, locked 1
LOCK WAIT 6 lock struct(s), heap size 1128, 3 row lock(s), undo log entries 2
MySQL thread id 54, OS thread handle 6131019776, query id 976 localhost 127.0.0.1 root update
/* mysql-connector-java-8.0.27 (Revision: e920b979015ae7117d60d72bcc8f077a839cd791 */ insert into SequenceId (invoice_id, post_processed_sequence_value, sequence_id, sequence_value) values (1648076703544, '79', '2000', 79)
*** (1) HOLDS THE LOCK(S):
RECORD LOCKS space id 3 page no 6 n bits 192 index invoice_id of table `sequenceid` trx id 2000 lock mode S
Record lock, heap no 1 PHYSICAL RECORD: n_fields 1; compact format; info bits 0
0: len 8; hex 73757072656d756d; asc supremum;;
*** (1) WAITING FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 3 page no 6 n bits 192 index invoice_id of table `sequenceid` trx id 2000 lock_mode X insert intention waiting
Record lock, heap no 1 PHYSICAL RECORD: n_fields 1; compact format; info bits 0
0: len 8; hex 73757072656d756d; asc supremum;;
*** (2) TRANSACTION:
TRANSACTION 1999, ACTIVE 0 sec inserting
mysql tables in use 1, locked 1
LOCK WAIT 6 lock struct(s), heap size 1128, 3 row lock(s), undo log entries 2
MySQL thread id 53, OS thread handle 6132133888, query id 977 localhost 127.0.0.1 root update
/* mysql-connector-java-8.0.27 (Revision: e920b979015ae7117d60d72bcc8f077a839cd791) */ insert into SequenceId (ref_id, post_processed_sequence_value, sequence_id, sequence_value) values (1648076703512, '50046', '1000', 50046)
*** (2) HOLDS THE LOCK(S):
RECORD LOCKS space id 3 page no 6 n bits 192 index invoice_id of table `sequenceid` trx id 1999 lock mode S
Record lock, heap no 1 PHYSICAL RECORD: n_fields 1; compact format; info bits 0
0: len 8; hex 73757072656d756d; asc supremum;;
*** (2) WAITING FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 3 page no 6 n bits 192 index invoice_id of table `sequenceid` trx id 1999 lock_mode X insert intention waiting
Record lock, heap no 1 PHYSICAL RECORD: n_fields 1; compact format; info bits 0
0: len 8; hex 73757072656d756d; asc supremum;;
*** WE ROLL BACK TRANSACTION (2)
When a transaction is executed in serializable isolation read statements acquire shared lock. Please find the details below Thread 1 tries to insert record by acquiring key. For insert there is no way we can hold a key. So mysql acquires a shared lock on the latest record before inserting. Thread 2 tries to update the latest record and acquires the shared lock before inserting. Now Thread 1 and Thread 2 are competing for the same lock and it causes deadlock.
MySQL Deadlock: Transaction holds and waits for the same lock [duplicate]
I have very simple table: CREATE TABLE `d` ( `id` int(11) DEFAULT NULL, UNIQUE KEY `id` (`id`) ) ENGINE=InnoDB DEFAULT CHARSET=utf8 without records: select * from d; Empty set (0,01 sec) Then I try to open two transactions in different sessions: Session #1: begin; Query OK, 0 rows affected (0,00 sec) select * from d where id = 100 for update; Empty set (0,00 sec) Session #2: begin; Query OK, 0 rows affected (0,00 sec) select * from d where id = 700 for update; Empty set (0,00 sec) Now I try to insert new record in Session #2 and session "freezes": insert into d values (700); And when I try to do the same (with another id field) in Session #1 it crashes: insert into d values (100); --> ERROR 1213 (40001): Deadlock found when trying to get lock; try restarting transaction in Session #1 insert into d values (700); --> Query OK, 1 row affected (4,08 sec) in Session #2 How can I to fix the deadlock? InnoDB status is: ------------------------ LATEST DETECTED DEADLOCK ------------------------ 2017-07-06 15:59:25 0x70000350d000 *** (1) TRANSACTION: TRANSACTION 43567, ACTIVE 15 sec inserting mysql tables in use 1, locked 1 LOCK WAIT 3 lock struct(s), heap size 1136, 2 row lock(s), undo log entries 1 MySQL thread id 4, OS thread handle 123145358217216, query id 89 localhost root update insert into d values (700) *** (1) WAITING FOR THIS LOCK TO BE GRANTED: RECORD LOCKS space id 126 page no 4 n bits 72 index id of table `trx`.`d` trx id 43567 lock_mode X insert intention waiting Record lock, heap no 1 PHYSICAL RECORD: n_fields 1; compact format; info bits 0 0: len 8; hex 73757072656d756d; asc supremum;; *** (2) TRANSACTION: TRANSACTION 43568, ACTIVE 7 sec inserting mysql tables in use 1, locked 1 3 lock struct(s), heap size 1136, 2 row lock(s), undo log entries 1 MySQL thread id 3, OS thread handle 123145357938688, query id 90 localhost root update insert into d values (100) *** (2) HOLDS THE LOCK(S): RECORD LOCKS space id 126 page no 4 n bits 72 index id of table `trx`.`d` trx id 43568 lock_mode X Record lock, heap no 1 PHYSICAL RECORD: n_fields 1; compact format; info bits 0 0: len 8; hex 73757072656d756d; asc supremum;; *** (2) WAITING FOR THIS LOCK TO BE GRANTED: RECORD LOCKS space id 126 page no 4 n bits 72 index id of table `trx`.`d` trx id 43568 lock_mode X insert intention waiting Record lock, heap no 1 PHYSICAL RECORD: n_fields 1; compact format; info bits 0 0: len 8; hex 73757072656d756d; asc supremum;; *** WE ROLL BACK TRANSACTION (2)
This deadlock error is a bug in the MySQL InnoDB engine that has not been fixed for 12 years. (Bug #25847: https://bugs.mysql.com/bug.php?id=25847, workaround: How do I lock on an InnoDB row that doesn't exist yet?) It is not related to Unique Key. Running this query will also result in the same Deadlock error. Session #1: CREATE TABLE t (id int) ENGINE=InnoDB; Session #1: SET AUTOCOMMIT = 0; Session #1: SELECT id FROM t WHERE id = 1 FOR UPDATE; Session #2: SET AUTOCOMMIT = 0; Session #2: SELECT id FROM t WHERE id = 2 FOR UPDATE; Session #1: INSERT INTO t (id) VALUES (1); -- Hang Session #2: INSERT INTO t (id) VALUES (2); -- Session #1: OK, Session #2: Deadlock found when trying to get lock; try restarting transaction InnoDB Status is the same: ------------------------ LATEST DETECTED DEADLOCK ------------------------ 2019-10-24 00:25:31 0x1638 *** (1) TRANSACTION: TRANSACTION 1287, ACTIVE 62 sec inserting mysql tables in use 1, locked 1 LOCK WAIT 3 lock struct(s), heap size 1136, 2 row lock(s) MySQL thread id 7, OS thread handle 9444, query id 143 localhost ::1 root update INSERT INTO t (id) VALUES (1) *** (1) WAITING FOR THIS LOCK TO BE GRANTED: RECORD LOCKS space id 23 page no 3 n bits 72 index GEN_CLUST_INDEX of table `test`.`t` trx id 1287 lock_mode X insert intention waiting Record lock, heap no 1 PHYSICAL RECORD: n_fields 1; compact format; info bits 0 0: len 8; hex 73757072656d756d; asc supremum;; *** (2) TRANSACTION: TRANSACTION 1288, ACTIVE 19 sec inserting, thread declared inside InnoDB 5000 mysql tables in use 1, locked 1 3 lock struct(s), heap size 1136, 2 row lock(s) MySQL thread id 9, OS thread handle 5688, query id 145 localhost ::1 root update INSERT INTO t (id) VALUES (2) *** (2) HOLDS THE LOCK(S): RECORD LOCKS space id 23 page no 3 n bits 72 index GEN_CLUST_INDEX of table `test`.`t` trx id 1288 lock_mode X Record lock, heap no 1 PHYSICAL RECORD: n_fields 1; compact format; info bits 0 0: len 8; hex 73757072656d756d; asc supremum;; *** (2) WAITING FOR THIS LOCK TO BE GRANTED: RECORD LOCKS space id 23 page no 3 n bits 72 index GEN_CLUST_INDEX of table `test`.`t` trx id 1288 lock_mode X insert intention waiting Record lock, heap no 1 PHYSICAL RECORD: n_fields 1; compact format; info bits 0 0: len 8; hex 73757072656d756d; asc supremum;; *** WE ROLL BACK TRANSACTION (2)
Note that starting with MySQL 8.0.1, the performance schema exposes innodb data locks. See https://dev.mysql.com/doc/refman/8.0/en/data-locks-table.html See https://dev.mysql.com/doc/refman/8.0/en/data-lock-waits-table.html In this example, after the first select in session 1 alone the locks are: mysql> select * from performance_schema.data_locks \G *************************** 1. row *************************** ENGINE: INNODB ENGINE_LOCK_ID: 1808:76 ENGINE_TRANSACTION_ID: 1808 THREAD_ID: 35 EVENT_ID: 13081 OBJECT_SCHEMA: test OBJECT_NAME: d PARTITION_NAME: NULL SUBPARTITION_NAME: NULL INDEX_NAME: NULL OBJECT_INSTANCE_BEGIN: 139756088373592 LOCK_TYPE: TABLE LOCK_MODE: IX LOCK_STATUS: GRANTED LOCK_DATA: NULL *************************** 2. row *************************** ENGINE: INNODB ENGINE_LOCK_ID: 1808:2:5:1 ENGINE_TRANSACTION_ID: 1808 THREAD_ID: 35 EVENT_ID: 13111 OBJECT_SCHEMA: test OBJECT_NAME: d PARTITION_NAME: NULL SUBPARTITION_NAME: NULL INDEX_NAME: id OBJECT_INSTANCE_BEGIN: 139756088370552 LOCK_TYPE: RECORD LOCK_MODE: X LOCK_STATUS: GRANTED LOCK_DATA: supremum pseudo-record <--- HERE 2 rows in set (0.00 sec) This is not a solution to the deadlock itself, but having visibility on the locks taken goes a long way to understand the problem. Here, both SELECT FOR UPDATE lock the "suprenum" record, because id 100 and 700 are greater than the biggest ID in the table (it's empty). Once more records are present (say at ID = 500), both queries should execute concurrently, as a different gap in IDs will be locked.
I suspect the deadlock happens because InnoDB is conservative on dealing with "gaps". Note that 100 and 700 are both in the same nebulous area of untouched land. InnoDB can't (or at least does not) deal with the fact that "100" and "700" are different. InnoDB would like to tag individual rows but there are no rows already in the table with those ids. Transaction 2 was probably going to timeout (see innodb_lock_wait_timeout). When you poked Transaction 1 a second time with #2 still wanting a lock, InnoDB punted and gave up. Bottom Line: Live with deadlocks. When they happen, start back at the BEGIN. You have found yet-another obscure case where an unnecessary deadlock happens. Furthermore, I suspect that fixing the code for this case would slow down most other cases, and lead to a series of errors that would take several releases to discover and fix.
Why do two concurrent delete + insert statements deadlock on an empty table?
I'm curious to know why two concurrent DELETE followed by INSERT statements that use primary keys causes a deadlock in MySQL when the primary keys don't exist. The example is contrived to illustrate the issue in it's simplest form. Here is the setup. > SELECT ##GLOBAL.tx_isolation, ##tx_isolation; +-------------------------+------------------+ | ##GLOBAL.tx_isolation | ##tx_isolation | |-------------------------+------------------| | REPEATABLE-READ | REPEATABLE-READ | +-------------------------+------------------+ 1 row in set Time: 0.002s > select version(); +-------------+ | version() | |-------------| | 5.7.12 | +-------------+ 1 row in set Time: 0.002s create table lock_test ( id int(11) not null, primary key (`id`) ); Below, 1> represents one mysql terminal and 2> represents another. 1> begin; 1> delete from lock_test where id = 1; 2> begin; 2> delete from lock_test where id = 2; 1> insert into lock_test values (1); -- hangs 2> insert into lock_test values (2); *** deadlock *** Here's the show engine innodb status output: ------------------------ LATEST DETECTED DEADLOCK ------------------------ 2018-06-06 16:15:18 0x70000ba52000 *** (1) TRANSACTION: TRANSACTION 807765, ACTIVE 46 sec inserting mysql tables in use 1, locked 1 LOCK WAIT 3 lock struct(s), heap size 1136, 2 row lock(s) MySQL thread id 620, OS thread handle 123145496289280, query id 43097 localhost ::1 root update insert into lock_test values (1) *** (1) WAITING FOR THIS LOCK TO BE GRANTED: RECORD LOCKS space id 6819 page no 3 n bits 72 index PRIMARY of table `content_graph`.`lock_test` trx id 807765 lock_mode X insert intention waiting Record lock, heap no 1 PHYSICAL RECORD: n_fields 1; compact format; info bits 0 0: len 8; hex 73757072656d756d; asc supremum;; *** (2) TRANSACTION: TRANSACTION 807766, ACTIVE 37 sec inserting mysql tables in use 1, locked 1 3 lock struct(s), heap size 1136, 2 row lock(s) MySQL thread id 617, OS thread handle 123145497681920, query id 43099 localhost ::1 root update insert into lock_test values (2) *** (2) HOLDS THE LOCK(S): RECORD LOCKS space id 6819 page no 3 n bits 72 index PRIMARY of table `content_graph`.`lock_test` trx id 807766 lock_mode X Record lock, heap no 1 PHYSICAL RECORD: n_fields 1; compact format; info bits 0 0: len 8; hex 73757072656d756d; asc supremum;; *** (2) WAITING FOR THIS LOCK TO BE GRANTED: RECORD LOCKS space id 6819 page no 3 n bits 72 index PRIMARY of table `content_graph`.`lock_test` trx id 807766 lock_mode X insert intention waiting Record lock, heap no 1 PHYSICAL RECORD: n_fields 1; compact format; info bits 0 0: len 8; hex 73757072656d756d; asc supremum;; *** WE ROLL BACK TRANSACTION (2) Note that if you first insert records with ids 1, and 2 then repeat the sequence above there is no deadlock. My sense is that because the key is not in the index (both are appending) the delete has to lock more of the index (more of the page the primary would land in) but I want to make sure I have my understanding correct.
The "gap" is locked in anticipation that someone may try to insert the row I am trying to delete. Or, to look at it another way... It would be too slow to perfectly handle every odd case. So, InnoDB chooses to handle most cases efficiently and punt on the rare oddball case. Bottom line: Live with it. You will get deadlocks. You won't necessarily be able to understand them. But your code needs to recover -- by simply rolling back and going back to the BEGIN.
Solution for Insert Intention Locks in MySQL
I have very simple table: CREATE TABLE `d` ( `id` int(11) DEFAULT NULL, UNIQUE KEY `id` (`id`) ) ENGINE=InnoDB DEFAULT CHARSET=utf8 without records: select * from d; Empty set (0,01 sec) Then I try to open two transactions in different sessions: Session #1: begin; Query OK, 0 rows affected (0,00 sec) select * from d where id = 100 for update; Empty set (0,00 sec) Session #2: begin; Query OK, 0 rows affected (0,00 sec) select * from d where id = 700 for update; Empty set (0,00 sec) Now I try to insert new record in Session #2 and session "freezes": insert into d values (700); And when I try to do the same (with another id field) in Session #1 it crashes: insert into d values (100); --> ERROR 1213 (40001): Deadlock found when trying to get lock; try restarting transaction in Session #1 insert into d values (700); --> Query OK, 1 row affected (4,08 sec) in Session #2 How can I to fix the deadlock? InnoDB status is: ------------------------ LATEST DETECTED DEADLOCK ------------------------ 2017-07-06 15:59:25 0x70000350d000 *** (1) TRANSACTION: TRANSACTION 43567, ACTIVE 15 sec inserting mysql tables in use 1, locked 1 LOCK WAIT 3 lock struct(s), heap size 1136, 2 row lock(s), undo log entries 1 MySQL thread id 4, OS thread handle 123145358217216, query id 89 localhost root update insert into d values (700) *** (1) WAITING FOR THIS LOCK TO BE GRANTED: RECORD LOCKS space id 126 page no 4 n bits 72 index id of table `trx`.`d` trx id 43567 lock_mode X insert intention waiting Record lock, heap no 1 PHYSICAL RECORD: n_fields 1; compact format; info bits 0 0: len 8; hex 73757072656d756d; asc supremum;; *** (2) TRANSACTION: TRANSACTION 43568, ACTIVE 7 sec inserting mysql tables in use 1, locked 1 3 lock struct(s), heap size 1136, 2 row lock(s), undo log entries 1 MySQL thread id 3, OS thread handle 123145357938688, query id 90 localhost root update insert into d values (100) *** (2) HOLDS THE LOCK(S): RECORD LOCKS space id 126 page no 4 n bits 72 index id of table `trx`.`d` trx id 43568 lock_mode X Record lock, heap no 1 PHYSICAL RECORD: n_fields 1; compact format; info bits 0 0: len 8; hex 73757072656d756d; asc supremum;; *** (2) WAITING FOR THIS LOCK TO BE GRANTED: RECORD LOCKS space id 126 page no 4 n bits 72 index id of table `trx`.`d` trx id 43568 lock_mode X insert intention waiting Record lock, heap no 1 PHYSICAL RECORD: n_fields 1; compact format; info bits 0 0: len 8; hex 73757072656d756d; asc supremum;; *** WE ROLL BACK TRANSACTION (2)
This deadlock error is a bug in the MySQL InnoDB engine that has not been fixed for 12 years. (Bug #25847: https://bugs.mysql.com/bug.php?id=25847, workaround: How do I lock on an InnoDB row that doesn't exist yet?) It is not related to Unique Key. Running this query will also result in the same Deadlock error. Session #1: CREATE TABLE t (id int) ENGINE=InnoDB; Session #1: SET AUTOCOMMIT = 0; Session #1: SELECT id FROM t WHERE id = 1 FOR UPDATE; Session #2: SET AUTOCOMMIT = 0; Session #2: SELECT id FROM t WHERE id = 2 FOR UPDATE; Session #1: INSERT INTO t (id) VALUES (1); -- Hang Session #2: INSERT INTO t (id) VALUES (2); -- Session #1: OK, Session #2: Deadlock found when trying to get lock; try restarting transaction InnoDB Status is the same: ------------------------ LATEST DETECTED DEADLOCK ------------------------ 2019-10-24 00:25:31 0x1638 *** (1) TRANSACTION: TRANSACTION 1287, ACTIVE 62 sec inserting mysql tables in use 1, locked 1 LOCK WAIT 3 lock struct(s), heap size 1136, 2 row lock(s) MySQL thread id 7, OS thread handle 9444, query id 143 localhost ::1 root update INSERT INTO t (id) VALUES (1) *** (1) WAITING FOR THIS LOCK TO BE GRANTED: RECORD LOCKS space id 23 page no 3 n bits 72 index GEN_CLUST_INDEX of table `test`.`t` trx id 1287 lock_mode X insert intention waiting Record lock, heap no 1 PHYSICAL RECORD: n_fields 1; compact format; info bits 0 0: len 8; hex 73757072656d756d; asc supremum;; *** (2) TRANSACTION: TRANSACTION 1288, ACTIVE 19 sec inserting, thread declared inside InnoDB 5000 mysql tables in use 1, locked 1 3 lock struct(s), heap size 1136, 2 row lock(s) MySQL thread id 9, OS thread handle 5688, query id 145 localhost ::1 root update INSERT INTO t (id) VALUES (2) *** (2) HOLDS THE LOCK(S): RECORD LOCKS space id 23 page no 3 n bits 72 index GEN_CLUST_INDEX of table `test`.`t` trx id 1288 lock_mode X Record lock, heap no 1 PHYSICAL RECORD: n_fields 1; compact format; info bits 0 0: len 8; hex 73757072656d756d; asc supremum;; *** (2) WAITING FOR THIS LOCK TO BE GRANTED: RECORD LOCKS space id 23 page no 3 n bits 72 index GEN_CLUST_INDEX of table `test`.`t` trx id 1288 lock_mode X insert intention waiting Record lock, heap no 1 PHYSICAL RECORD: n_fields 1; compact format; info bits 0 0: len 8; hex 73757072656d756d; asc supremum;; *** WE ROLL BACK TRANSACTION (2)
Note that starting with MySQL 8.0.1, the performance schema exposes innodb data locks. See https://dev.mysql.com/doc/refman/8.0/en/data-locks-table.html See https://dev.mysql.com/doc/refman/8.0/en/data-lock-waits-table.html In this example, after the first select in session 1 alone the locks are: mysql> select * from performance_schema.data_locks \G *************************** 1. row *************************** ENGINE: INNODB ENGINE_LOCK_ID: 1808:76 ENGINE_TRANSACTION_ID: 1808 THREAD_ID: 35 EVENT_ID: 13081 OBJECT_SCHEMA: test OBJECT_NAME: d PARTITION_NAME: NULL SUBPARTITION_NAME: NULL INDEX_NAME: NULL OBJECT_INSTANCE_BEGIN: 139756088373592 LOCK_TYPE: TABLE LOCK_MODE: IX LOCK_STATUS: GRANTED LOCK_DATA: NULL *************************** 2. row *************************** ENGINE: INNODB ENGINE_LOCK_ID: 1808:2:5:1 ENGINE_TRANSACTION_ID: 1808 THREAD_ID: 35 EVENT_ID: 13111 OBJECT_SCHEMA: test OBJECT_NAME: d PARTITION_NAME: NULL SUBPARTITION_NAME: NULL INDEX_NAME: id OBJECT_INSTANCE_BEGIN: 139756088370552 LOCK_TYPE: RECORD LOCK_MODE: X LOCK_STATUS: GRANTED LOCK_DATA: supremum pseudo-record <--- HERE 2 rows in set (0.00 sec) This is not a solution to the deadlock itself, but having visibility on the locks taken goes a long way to understand the problem. Here, both SELECT FOR UPDATE lock the "suprenum" record, because id 100 and 700 are greater than the biggest ID in the table (it's empty). Once more records are present (say at ID = 500), both queries should execute concurrently, as a different gap in IDs will be locked.
I suspect the deadlock happens because InnoDB is conservative on dealing with "gaps". Note that 100 and 700 are both in the same nebulous area of untouched land. InnoDB can't (or at least does not) deal with the fact that "100" and "700" are different. InnoDB would like to tag individual rows but there are no rows already in the table with those ids. Transaction 2 was probably going to timeout (see innodb_lock_wait_timeout). When you poked Transaction 1 a second time with #2 still wanting a lock, InnoDB punted and gave up. Bottom Line: Live with deadlocks. When they happen, start back at the BEGIN. You have found yet-another obscure case where an unnecessary deadlock happens. Furthermore, I suspect that fixing the code for this case would slow down most other cases, and lead to a series of errors that would take several releases to discover and fix.