Why do I get a MySQL database deadlock in this situation? - mysql

Question

I find a similar question asked here Deadlock using SELECT ... FOR UPDATE in MySQL. But the answer didn't really explain why this happens.
The situation is quite easy to reproduce # Mysql 5.7.17 (or other versions in 5.5 or 5.6):
CREATE TABLE `test` (
`id` bigint(11) NOT NULL AUTO_INCREMENT,
`val` varchar(255) COLLATE utf8_unicode_ci NOT NULL,
PRIMARY KEY (`id`),
KEY `search` (`val`)
) ENGINE=InnoDB AUTO_INCREMENT=4 DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci;
insert into test set val='pre-lock';
==session1==
start transaction;
select * from test where val='pre-lock' for update;
==session2==
start transaction;
select * from test where val='pre-lock' for update;
==session1==
insert into test set val='/a/b/c'; //note that if the set val='pre-lock111', there will be no deadlock at session 2
==session2==
ERROR 1213 (40001): Deadlock found when trying to get lock; try restarting transaction
The result of show engine innodb status:
LATEST DETECTED DEADLOCK
------------------------
2017-04-06 23:54:03 0x7000057db000
*** (1) TRANSACTION:
TRANSACTION 1333, ACTIVE 18 sec starting index read
mysql tables in use 1, locked 1
LOCK WAIT 2 lock struct(s), heap size 1136, 1 row lock(s)
MySQL thread id 5, OS thread handle 123145394155520, query id 62 localhost root Sending data
select * from test where val='pre-lock' for update
*** (1) WAITING FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 24 page no 4 n bits 72 index search of table `test_tnx`.`test` trx id 1333 lock_mode X waiting
Record lock, heap no 2 PHYSICAL RECORD: n_fields 2; compact format; info bits 0
0: len 8; hex 7072652d6c6f636b; asc pre-lock;;
1: len 8; hex 8000000000000001; asc ;;
*** (2) TRANSACTION:
TRANSACTION 1332, ACTIVE 29 sec inserting
mysql tables in use 1, locked 1
4 lock struct(s), heap size 1136, 4 row lock(s), undo log entries 1
MySQL thread id 62, OS thread handle 123145394434048, query id 63 localhost root update
insert into test set val='/a/b/c'
*** (2) HOLDS THE LOCK(S):
RECORD LOCKS space id 24 page no 4 n bits 72 index search of table `test_tnx`.`test` trx id 1332 lock_mode X
Record lock, heap no 1 PHYSICAL RECORD: n_fields 1; compact format; info bits 0
0: len 8; hex 73757072656d756d; asc supremum;;
Record lock, heap no 2 PHYSICAL RECORD: n_fields 2; compact format; info bits 0
0: len 8; hex 7072652d6c6f636b; asc pre-lock;;
1: len 8; hex 8000000000000001; asc ;;
*** (2) WAITING FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 24 page no 4 n bits 72 index search of table `test_tnx`.`test` trx id 1332 lock_mode X locks gap before rec insert intention waiting
Record lock, heap no 2 PHYSICAL RECORD: n_fields 2; compact format; info bits 0
0: len 8; hex 7072652d6c6f636b; asc pre-lock;;
1: len 8; hex 8000000000000001; asc ;;
*** WE ROLL BACK TRANSACTION (1)
My perception is that the insert clause in session 1 acquires a gap lock which is somehow overlapping with the gap lock by the select for update. And therefore, this create a deadlock. However, I couldn't find any document supporting my idea. Please help to explain.
Updated 1 on 7th. Apr. 2017:
My full use case is like follows. Each session starts a transaction. It first locks the "pre-lock" row for uniqueness, after which performs a set of insertions. Then, it commits all insertions and release the lock. To my understand, if I remove the "pre-lock", two transactions may result in deadlock, and both fail (mysql kills one and let the other go as pointed out by #Michael - sqlbot). Meanwhile, it is allowed for some other sessions to read the content of the table already been committed before， and therefore, exclusively locking the table is not the cue.
Updated 2 on 7th. Apr. 2017:
It it quite a long story to to fully explain my objective. In short, I want to use mysql as a locking service for an external system. These insertions I mentioned is a set of external locks. It can not be made unique since I have read/write locks, and want to support kind of wildcard. Before these insertions, I actually need to do a conflict analysis with a set of selections. Setting up a global pre-lock sounds a good solution for me to accomplish this task.