Is MySQL allows multiple next-key locks on same record/range from multiple sessions? As following example shows, the same delete statement won't block each other.
My understanding is MySQL support MVCC, which means it has snapshot for each sessions below. Am I right?
MySQL: 5.7
Isolation level: Repeatable Read
Database: innodb
CREATE TABLE `game_summaries` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`game_id` int(11) DEFAULT NULL,
`value` int(11) DEFAULT NULL,
PRIMARY KEY (`id`),
KEY `index_game_summaries_on_game_id` (`game_id`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
session 1
start transaction;
delete from game_summaries where game_id = 2;
session 2
start transaction;
delete from game_summaries where game_id = 2;
run show engine innodb status \G to get all locks
------------
TRANSACTIONS
------------
Trx id counter 2849
Purge done for trx's n:o < 2841 undo n:o < 0 state: running but idle
History list length 0
LIST OF TRANSACTIONS FOR EACH SESSION:
---TRANSACTION 2848, ACTIVE 8 sec
2 lock struct(s), heap size 1136, 1 row lock(s)
MySQL thread id 3, OS thread handle 139644825081600, query id 232 172.18.0.1 root starting
show engine innodb status
TABLE LOCK table `test`.`game_summaries` trx id 2848 lock mode IX
RECORD LOCKS space id 27 page no 4 n bits 72 index index_game_summaries_on_game_id of table `test`.`game_summaries` trx id 2848 lock_mode X
Record lock, heap no 1 PHYSICAL RECORD: n_fields 1; compact format; info bits 0
0: len 8; hex 73757072656d756d; asc supremum;;
---TRANSACTION 2847, ACTIVE 18 sec
2 lock struct(s), heap size 1136, 1 row lock(s)
MySQL thread id 2, OS thread handle 139644825351936, query id 229 172.18.0.1 root
TABLE LOCK table `test`.`game_summaries` trx id 2847 lock mode IX
RECORD LOCKS space id 27 page no 4 n bits 72 index index_game_summaries_on_game_id of table `test`.`game_summaries` trx id 2847 lock_mode X
Record lock, heap no 1 PHYSICAL RECORD: n_fields 1; compact format; info bits 0
0: len 8; hex 73757072656d756d; asc supremum;;
According to https://dev.mysql.com/doc/refman/8.0/en/innodb-locking.html, the gap lock is allowed to be hold on a gap by different transactions, even you saw lock_mode X there.
It is also worth noting here that conflicting locks can be held on a gap by different transactions. For example, transaction A can hold a shared gap lock (gap S-lock) on a gap while transaction B holds an exclusive gap lock (gap X-lock) on the same gap. The reason conflicting gap locks are allowed is that if a record is purged from an index, the gap locks held on the record by different transactions must be merged.
Related
In my application, a deadlock occurs when I run queries that are almost the same.
When I ran a SHOW ENGINE INNODB STATUS, result is same as follows. (InnoDB, mysql5.7)
*** (1) TRANSACTION:
TRANSACTION 41516159, ACTIVE 0 sec starting index read
mysql tables in use 1, locked 1
LOCK WAIT 7 lock struct(s), heap size 1136, 3 row lock(s), undo log entries 25
MySQL thread id 266983, OS thread handle 47538359256832, query id 156835755 172.31.5.225 Test_Table updating
Update Test_Table SET updatedAt = '2021-11-01 23:59:59' WHERE PK = 1
*** (1) WAITING FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 142 page no 542 n bits 0 index PRIMARY of table `Test_Table` trx id 41516159 lock_mode X locks rec but not gap waiting
Record lock, heap no 68 PHYSICAL RECORD: n_fields 23; compact format; info bits 0
*** (2) TRANSACTION:
TRANSACTION 41516160, ACTIVE 0 sec starting index read
mysql tables in use 1, locked 1
LOCK WAIT 7 lock struct(s), heap size 1136, 3 row lock(s), undo log entries 25
MySQL thread id 266996, OS thread handle 47549115574016, query id 156835795 XXX.XX.XX.XX Test_Table updating Update Test_Table SET updatedAt = '2021-11-01 23:59:58' WHERE PK = 1
*** (2) HOLDS THE LOCK(S):
RECORD LOCKS space id 142 page no 542 n bits 0 index PRIMARY of table `Test_Table` trx id 41516160 lock mode S locks rec but not gap Record lock, heap no 68 PHYSICAL RECORD: n_fields 23; compact format; info bits 0
I can't understand why update query make a s-lock.
I have 2 tables EXPERIMENT and ENTITIES. ENTITIES has an id field which references primary Id of table EXPERIMENT.
I inserted multiple experiments along with the child entities concurrently and got deadlocks.
show engine innodb status shows the debugging information.
I am unable to find why the deadlock occured. I guess that it happened because the child entities is verifying the Foreign_Key in the experiment table, but that doesn't look like it should generate deadlock.
I am using AUTO INCREMENT for the id's and SERIALIZABLE transaction ISOLATION.
Here is the relevant section from the innodb status :
------------------------
LATEST DETECTED DEADLOCK
------------------------
2019-05-14 12:34:45 0x7000060f3000
*** (1) TRANSACTION:
TRANSACTION 162916, ACTIVE 1 sec inserting
mysql tables in use 1, locked 1
LOCK WAIT 10 lock struct(s), heap size 1136, 6 row lock(s), undo log entries 2
MySQL thread id 183, OS thread handle 123145405919232, query id 2464 localhost 127.0.0.1 root update
INSERT INTO _ENTITIES (TYPE, FK_EXPERIMENT_ID, CONTENT, CREATED_USER_ID) VALUES ('VARIATION', 42, '{"variantName":"Variant 1","actions":[{"blockId":0,"type":"SendEmail","criteria":{"and":[{"operator":"EQ","attr":"_id","val":"Test","ruleId":1,"category":"default"},{"operator":"EQ","attr":"productLanguage","val":"CS_CZ","ruleId":1,"category":"contextual"}]},"order":1,"surfaceActionName":"EMAIL","params":{"verified":true,"selectedTemplate":"Design-Paid-Portfolio-A"},"name":"Action Block 1","treatmentId":"","default":true},{"blockId":1,"type":"wait","criteria":{"and":[{"operator":"EQ","attr":"_id","val":"Test","ruleId":1,"category":"default"},{"operator":"EQ","attr":"productLanguage","val":"CS_CZ","ruleId":1,"category":"contextual"}]},"order":1,"surfaceActionName":"wait","params":{"unit":"hour","data":10,"verified":true},"name":"Action Block 1","treatmentId":"","default":true}],"variantPercentage":80}', 'uk
*** (1) WAITING FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 6728 page no 4 n bits 96 index ENTITIES_EXPERIMENT_ID of table `test_database`.`_entities` trx id 162916 lock_mode X insert intention waiting
Record lock, heap no 1 PHYSICAL RECORD: n_fields 1; compact format; info bits 0
0: len 8; hex 73757072656d756d; asc supremum;;
*** (2) TRANSACTION:
TRANSACTION 162906, ACTIVE 1 sec inserting
mysql tables in use 1, locked 1
8 lock struct(s), heap size 1136, 4 row lock(s), undo log entries 2
MySQL thread id 164, OS thread handle 123145403969536, query id 2549 localhost 127.0.0.1 root update
INSERT INTO _ENTITIES (TYPE, FK_EXPERIMENT_ID, CONTENT, CREATED_USER_ID) VALUES ('VARIATION', 33, '{"variantName":"Variant 1","actions":[{"blockId":0,"type":"SendEmail","criteria":{"and":[{"operator":"EQ","attr":"_id","val":"Test","ruleId":1,"category":"default"},{"operator":"EQ","attr":"productLanguage","val":"CS_CZ","ruleId":1,"category":"contextual"}]},"order":1,"surfaceActionName":"EMAIL","params":{"verified":true,"selectedTemplate":"Design-Paid-Portfolio-A"},"name":"Action Block 1","treatmentId":"","default":true},{"blockId":1,"type":"wait","criteria":{"and":[{"operator":"EQ","attr":"_id","val":"Test","ruleId":1,"category":"default"},{"operator":"EQ","attr":"productLanguage","val":"CS_CZ","ruleId":1,"category":"contextual"}]},"order":1,"surfaceActionName":"wait","params":{"unit":"hour","data":10,"verified":true},"name":"Action Block 1","treatmentId":"","default":true}],"variantPercentage":80}', 'uk
*** (2) HOLDS THE LOCK(S):
RECORD LOCKS space id 6728 page no 4 n bits 96 index ENTITIES_EXPERIMENT_ID of table `test_database`.`_entities` trx id 162906 lock mode S
Record lock, heap no 1 PHYSICAL RECORD: n_fields 1; compact format; info bits 0
0: len 8; hex 73757072656d756d; asc supremum;;
*** (2) WAITING FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 6728 page no 4 n bits 96 index ENTITIES_EXPERIMENT_ID of table `test_database`.`_entities` trx id 162906 lock_mode X insert intention waiting
Record lock, heap no 1 PHYSICAL RECORD: n_fields 1; compact format; info bits 0
0: len 8; hex 73757072656d756d; asc supremum;;
*** WE ROLL BACK TRANSACTION (2)
------------
How should i interpret this and why did deadlock occur?
The code used is :
Integer generatedId = experimentDAO.add(experimentQO);
......
for..
entitiesDAO.add(entitiesQO);
....
ExperimentQO experimentQO = experimentDAO.get(generatedId);
Exception occurs while adding the entity.
What I see is:
Trx #1 is waiting for an X mode insert intention lock (a kind of gap lock) on the ENTITIES_EXPERIMENT_ID index.
Trx #2 holds an S lock on the ENTITIES_EXPERIMENT_ID index, which is blocking Trx #1
Trx #2 is also waiting for an insert intention lock on the ENTITIES_EXPERIMENT_ID index.
We might assume Trx #1 is also holding an S lock on the same index. S locks are shared, so multiple transactions can acquire S locks on the same row (or gap) concurrently.
If both transactions acquired S locks first, and then both tried to request X locks, then they would enter into a situation where both were waiting on the other one, with no way to break the deadlock.
It's possible that both INSERT statements acquired the S locks as a first step. Or it's possible that you did some other queries that acquire S locks in the same transaction prior to the INSERTs, so both transactions are still holding their respective S locks.
You haven't shown the table definition, so there might be some foreign key constraints, that would cause S locks to be acquired for rows referenced indirectly.
I found below deadlock in "Show engine innodb status" on myql 8.0.11.
As per my understanding both queries should lock only one row (the primary key user_id which is different for both queries). However the second query "*** (2) HOLDS THE LOCK(S): "RECORD LOCKS space id 5461 page no 76054 n bits 96 index PRIMARY of tableuser"`
and First query is waiting to acquire same lock. I did not understand why? even when both lock statement say "locks rec but not gap" which means both are locking only individual records (which should have been their primary key user_id )
Would like to know is my understanding (as explained above) correct? If yes then how to explain the below logs?
*** (1) TRANSACTION:
mysql tables in use 1, locked 1
LOCK WAIT 3 lock struct(s), heap size 1136, 2 row lock(s), undo log entries 1
update `user` set `user`.`xxxx` = 1 where ( user_id = 4939334)
*** (1) WAITING FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 5461 page no 76054 n bits 96 index PRIMARY of table `user` trx id 5385693398 lock_mode X locks rec but not gap waiting
Record lock, heap no 24 PHYSICAL RECORD: n_fields 20; compact format; info bits 0
*** (2) TRANSACTION:
mysql tables in use 1, locked 1
3 lock struct(s), heap size 1136, 2 row lock(s), undo log entries 1
update `user` set `user`.`xxxx` = 1 where ( user_id = 4917613)
*** (2) HOLDS THE LOCK(S):
RECORD LOCKS space id 5461 page no 76054 n bits 96 index PRIMARY of table `user` trx id 5385693399 lock_mode X locks rec but not gap
Record lock, heap no 24 PHYSICAL RECORD: n_fields 20; compact format; info bits 0
*** (2) WAITING FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 5461 page no 63136 n bits 88 index PRIMARY of table `user` trx id 5385693399 lock_mode X locks rec but not gap waiting
Record lock, heap no 19 PHYSICAL RECORD: n_fields 20; compact format; info bits 0
I use JMeter to test my program, somehow the total response number is stopping increasing, then I find out there is a deadlock in MySQL. I don't understand what's meaning of below log. Seems that transaction(2) owned a S lock and trying to own a X lock of the same table. Does this cause the deadlock? If so, why will this happen?
ATEST DETECTED DEADLOCK
------------------------
2019-01-02 14:38:27 0x70000f30a000
*** (1) TRANSACTION:
TRANSACTION 24004, ACTIVE 0 sec inserting
mysql tables in use 2, locked 2
LOCK WAIT 5 lock struct(s), heap size 1136, 2 row lock(s), undo log entries 2
MySQL thread id 11953, OS thread handle 123145549275136, query id 418447 localhost 127.0.0.1 username executing
INSERT INTO MRBS_SCHEDULE (ID,START_TIME,END_TIME,ROOM_ID,CREATE_BY,PRESIDE,REPEAT_ID,DESCRIPTION,NUM,TITLE,PRESIDE_EMAIL,PROJECTOR,CONFERENCE_CALL,CREATE_ID,BOOK_TIME,END_TYPE,EXPECTED_END_TIME) select null,'2019-01-03 19:53:00','2019-01-03 19:53:00',10113558,'d','d',12245755,'fdsfds',10,null,'d#sh.ff.com',0,0,10227622,'2019-01-02 14:38:27.358',0,'2019-01-03 19:53:00' from dual WHERE NOT EXISTS (SELECT * FROM MRBS_SCHEDULE ms where ms.START_TIME<'2019-01-03 19:53:00' and ms.END_TIME>'2019-01-03 19:53:00' and ms.ROOM_ID=10113558)
*** (1) WAITING FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 55 page no 228 n bits 872 index IND_MRBS_SCHEDULE_END_TIME of table `meeting`.`mrbs_schedule` trx id 24004 lock_mode X insert intention waiting
Record lock, heap no 1 PHYSICAL RECORD: n_fields 1; compact format; info bits 0
0: len 8; hex 73757072656d756d; asc supremum;;
*** (2) TRANSACTION:
TRANSACTION 24005, ACTIVE 0 sec inserting
mysql tables in use 2, locked 2
5 lock struct(s), heap size 1136, 2 row lock(s), undo log entries 2
MySQL thread id 11940, OS thread handle 123145557155840, query id 418448 localhost 127.0.0.1 username executing
INSERT INTO MRBS_SCHEDULE (ID,START_TIME,END_TIME,ROOM_ID,CREATE_BY,PRESIDE,REPEAT_ID,DESCRIPTION,NUM,TITLE,PRESIDE_EMAIL,PROJECTOR,CONFERENCE_CALL,CREATE_ID,BOOK_TIME,END_TYPE,EXPECTED_END_TIME) select null,'2019-01-03 19:54:00','2019-01-03 19:54:00',10113685,'z','z',12245756,'fdsfds',10,null,'z#sz.ff.com',0,0,10227544,'2019-01-02 14:38:27.397',0,'2019-01-03 19:54:00' from dual WHERE NOT EXISTS (SELECT * FROM MRBS_SCHEDULE ms where ms.START_TIME<'2019-01-03 19:54:00' and ms.END_TIME>'2019-01-03 19:54:00' and ms.ROOM_ID=10113685)
*** (2) HOLDS THE LOCK(S):
RECORD LOCKS space id 55 page no 228 n bits 872 index IND_MRBS_SCHEDULE_END_TIME of table `meeting`.`mrbs_schedule` trx id 24005 lock mode S
Record lock, heap no 1 PHYSICAL RECORD: n_fields 1; compact format; info bits 0
0: len 8; hex 73757072656d756d; asc supremum;;
*** (2) WAITING FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 55 page no 228 n bits 872 index IND_MRBS_SCHEDULE_END_TIME of table `meeting`.`mrbs_schedule` trx id 24005 lock_mode X insert intention waiting
Record lock, heap no 1 PHYSICAL RECORD: n_fields 1; compact format; info bits 0
0: len 8; hex 73757072656d756d; asc supremum;;
*** WE ROLL BACK TRANSACTION (2)
Edit: Table MRBS_SCHEDULE primary key use Auto_increment
Deadlock can occur anytime and it is normal behavior. It happens when transaction 1 tries to update Table B using Table A but at the same time Transaction 2 tries to update Table A using Table B.
basicly Transactions holds theirselves and since nobody wants to give up, deadlock occurs.
you can set deadlock threshold for your db using innodb_lock_wait_timeout
From the documentation,
When deadlock detection is enabled (the default), InnoDB automatically detects transaction deadlocks and rolls back a transaction or transactions to break the deadlock. InnoDB tries to pick small transactions to roll back, where the size of a transaction is determined by the number of rows inserted, updated, or deleted.
InnoDB is aware of table locks if innodb_table_locks = 1 (the default) and autocommit = 0, and the MySQL layer above it knows about row-level locks. Otherwise, InnoDB cannot detect deadlocks where a table lock set by a MySQL LOCK TABLES statement or a lock set by a storage engine other than InnoDB is involved. Resolve these situations by setting the value of the innodb_lock_wait_timeout system variable.
Reference: https://dev.mysql.com/doc/refman/8.0/en/innodb-deadlock-detection.html
I'm curious to know why two concurrent DELETE followed by INSERT statements that use primary keys causes a deadlock in MySQL when the primary keys don't exist. The example is contrived to illustrate the issue in it's simplest form.
Here is the setup.
> SELECT ##GLOBAL.tx_isolation, ##tx_isolation;
+-------------------------+------------------+
| ##GLOBAL.tx_isolation | ##tx_isolation |
|-------------------------+------------------|
| REPEATABLE-READ | REPEATABLE-READ |
+-------------------------+------------------+
1 row in set
Time: 0.002s
> select version();
+-------------+
| version() |
|-------------|
| 5.7.12 |
+-------------+
1 row in set
Time: 0.002s
create table lock_test ( id int(11) not null, primary key (`id`) );
Below, 1> represents one mysql terminal and 2> represents another.
1> begin;
1> delete from lock_test where id = 1;
2> begin;
2> delete from lock_test where id = 2;
1> insert into lock_test values (1); -- hangs
2> insert into lock_test values (2);
*** deadlock ***
Here's the show engine innodb status output:
------------------------
LATEST DETECTED DEADLOCK
------------------------
2018-06-06 16:15:18 0x70000ba52000
*** (1) TRANSACTION:
TRANSACTION 807765, ACTIVE 46 sec inserting
mysql tables in use 1, locked 1
LOCK WAIT 3 lock struct(s), heap size 1136, 2 row lock(s)
MySQL thread id 620, OS thread handle 123145496289280, query id 43097 localhost ::1 root update
insert into lock_test values (1)
*** (1) WAITING FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 6819 page no 3 n bits 72 index PRIMARY of table `content_graph`.`lock_test` trx id 807765 lock_mode X insert intention waiting
Record lock, heap no 1 PHYSICAL RECORD: n_fields 1; compact format; info bits 0
0: len 8; hex 73757072656d756d; asc supremum;;
*** (2) TRANSACTION:
TRANSACTION 807766, ACTIVE 37 sec inserting
mysql tables in use 1, locked 1
3 lock struct(s), heap size 1136, 2 row lock(s)
MySQL thread id 617, OS thread handle 123145497681920, query id 43099 localhost ::1 root update
insert into lock_test values (2)
*** (2) HOLDS THE LOCK(S):
RECORD LOCKS space id 6819 page no 3 n bits 72 index PRIMARY of table `content_graph`.`lock_test` trx id 807766 lock_mode X
Record lock, heap no 1 PHYSICAL RECORD: n_fields 1; compact format; info bits 0
0: len 8; hex 73757072656d756d; asc supremum;;
*** (2) WAITING FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 6819 page no 3 n bits 72 index PRIMARY of table `content_graph`.`lock_test` trx id 807766 lock_mode X insert intention waiting
Record lock, heap no 1 PHYSICAL RECORD: n_fields 1; compact format; info bits 0
0: len 8; hex 73757072656d756d; asc supremum;;
*** WE ROLL BACK TRANSACTION (2)
Note that if you first insert records with ids 1, and 2 then repeat the sequence above there is no deadlock.
My sense is that because the key is not in the index (both are appending) the delete has to lock more of the index (more of the page the primary would land in) but I want to make sure I have my understanding correct.
The "gap" is locked in anticipation that someone may try to insert the row I am trying to delete.
Or, to look at it another way... It would be too slow to perfectly handle every odd case. So, InnoDB chooses to handle most cases efficiently and punt on the rare oddball case.
Bottom line: Live with it. You will get deadlocks. You won't necessarily be able to understand them. But your code needs to recover -- by simply rolling back and going back to the BEGIN.