A puzzled MySQL deadlock caused by two update..in - mysql

Question

We got a puzzled deadlock on MySQL 5.7 (Engine: InnoDB, Isolation Level: RR). The report result of show engine innodb status as below shown
*** (1) TRANSACTION:
TRANSACTION 1739954050, ACTIVE 0 sec starting index read
mysql tables in use 1, locked 1
LOCK WAIT 5 lock struct(s), heap size 1136, 4 row lock(s)
MySQL thread id 4253877, OS thread handle 47904135608064, query id 4259685238 jacky Searching rows for update
UPDATE fruit_setting set
value = CASE
WHEN eid = 'L1XSHY' and `key` = 'priority' THEN '14343'
WHEN eid = 'Rtb95t' and `key` = 'priority' THEN '14344'
WHEN eid = 'wdsNwr' and `key` = 'priority' THEN '14345'
WHEN eid = 'K1Ikqy' and `key` = 'priority' THEN '14345'
WHEN eid =
*** (1) WAITING FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 533 page no 65378 n bits 0 index PRIMARY of table `jacky`.`fruit_setting` trx id 1739954050 lock_mode X locks rec but not gap waiting
...
*** (2) TRANSACTION:
TRANSACTION 1739954049, ACTIVE 0 sec fetching rows
mysql tables in use 1, locked 1
LOCK WAIT 94 lock struct(s), heap size 1136, 184 row lock(s)
MySQL thread id 4257460, OS thread handle 47904340621056, query id 4259685231 jacky Searching rows for update
UPDATE fruit_setting set value = CASE
WHEN eid = 'M5ecaA' and `key` = 'priority' THEN '16171'
WHEN eid = '1o0bdT' and `key` = 'priority' THEN '16172'
WHEN eid = 'XNxx5S' and `key` = 'priority' THEN '16173'
WHEN eid = '1o0bdT' and `key` = 'priority' THEN '16174'
WHEN eid =
*** (2) HOLDS THE LOCK(S):
RECORD LOCKS space id 533 page no 65378 n bits 0 index PRIMARY of table `jacky`.`fruit_setting` trx id 1739954049 lock_mode X locks rec but not gap
Record lock, heap no 105 PHYSICAL RECORD: n_fields 10; compact format; info bits 0
...
*** (2) WAITING FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 533 page no 46944 n bits 0 index PRIMARY of table `jacky`.`fruit_setting` trx id 1739954049 lock_mode X locks rec but not gap waiting
Record lock, heap no 58 PHYSICAL RECORD: n_fields 10; compact format; info bits 0
...
The sqls in above report has been truncated (might due to size limit of MySQL), one of the whole sql looks like (we only record the prepared statement)
UPDATE fruit_setting set value = CASE
WHEN eid = ? and `key` = ? THEN ?
WHEN eid = ? and `key` = ? THEN ?
WHEN eid = ? and `key` = ? THEN ?
WHEN eid = ? and `key` = ? THEN ?
WHEN eid = ? and `key` = ? THEN ?
WHEN eid = ? and `key` = ? THEN ?
WHEN eid = ? and `key` = ? THEN ?
WHEN eid = ? and `key` = ? THEN ?
WHEN eid = ? and `key` = ? THEN ?
END
WHERE aid = ? and eid in (?, ?, ?, ?, ?, ?, ?, ?, ?) and `key` = ?
The table DDL
CREATE TABLE `fruit_setting` (
`id` bigint(20) NOT NULL AUTO_INCREMENT,
`aid` varchar(32) NOT NULL,
`eid` varchar(32) NOT NULL,
`key` varchar(32) NOT NULL,
`value` varchar(32) DEFAULT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `i_eid_key` (`eid`, `key`),
KEY `i_aid_key` (`aid`, `key`),
KEY `i_aid_eid` (`aid`, `eid`)
);
Note that the aid and key in two sqls are the same and the eid in in clause could be overlapped; And thus we guess that the deadlock occurred since the locks are acquired in reverse order on index i_eid_key or i_aid_eid.
Question 1: why the lock is not waiting/hold on secondary index i_eid_key or i_aid_eid, but on primary index? AFAIK, the secondary index would be locked before primary index if the searching for update used the secondary index.
If a secondary index is used in a search and index record locks to be set are exclusive, InnoDB also retrieves the corresponding clustered index records and sets locks on them.
https://dev.mysql.com/doc/refman/8.0/en/innodb-locks-set.html
Question 2: Does MySQL acquire lock on index record one by one based on the order of eid appeared in in clause during execute update ... where eid in (...)?
By the way, the deadlock is hard to reproduce, I have tried to locked on eid in different order, however, the new deadlock report showed that the lock was on the secondary index.

Answer 1

For Question 1:
After some discussion and thought, we think that the deadlock might occurred like this: There are at least two same extension_ids in two sqls.
InnoDB first locks on two different secondary indexes, such as i_eid_key i_aid_key
and then lock the PK record, which is in the reverse order, and finally result in the deadlock.
For Question 2:
The MySQL optimization would sort the (constant) values in in clause