General error: 1205 Lock wait timeout exceeded; try restarting transaction - mysql

Question

I am developing an app with Laravel + MySQL for online exam assessment. A lot of users submit their response at the same time (within the span for few mins) so there is heavy load on the database server.
I am getting several General error: 1205 Lock wait timeout exceeded; try restarting transaction during many concurrent submissions.
I enabled slow query log and saw following entries in my log
update `form_responses` set `photo_url` = 'https://example.com/ae252b371effc7cb11dbcbbb18602026.jpg', `form_responses`.`updated_at` = '2020-09-26 11:39:17' where `id` = 32407;
As per my understanding, this is just a simple row update with a primary column being queried (means indexing should work).
But what I can't wrap my head around is - why Innodb is not using row level locking and provide a lock on this particular row. Is there some specific Innodb value I should increase this in case?

Answer 1

InnoDB is using row level locking.
Problem is, the application or the ORM are using it the wrong way, not giving MySQL sufficient information on what is to be done, resulting in a deadlock.
The easiest - not, I hasten to say, the most efficient! - way to overcome this problem is probably to resort to advisory locking if you find out that the updates all happen in the same small set of methods.
Note that it is not at all a given that your deadlock is in that table; you should run SHOW ENGINE INNODB STATUS to see what the last detected deadlock was. For instance you'll find:
LATEST DETECTED DEADLOCK
------------------------
200915 11:31:03
*** (1) TRANSACTION:
TRANSACTION 22B8A76B9, ACTIVE 0 sec starting index read
mysql tables in use 3, locked 3
LOCK WAIT 4 lock struct(s), heap size 1248, 4 row lock(s)
MySQL thread id 11161737, OS thread handle 0x7fd06d708700, query id ... UPDATE table SET ... WHERE Dat_UsrId = '11254' && Dat_CreUte='...' && Dat_Tip = 'C'
*** (2) TRANSACTION:
TRANSACTION 22B8A76B4, ACTIVE 0 sec fetching rows
mysql tables in use 3, locked 3
2616 lock struct(s), heap size 244152, 7262 row lock(s)
MySQL thread id 11161676, OS thread handle 0x7fd06ca97700, query id ... UPDATE table SET ... WHERE ...
*** (2) HOLDS THE LOCK(S):
RECORD LOCKS space id 866959 page no 46 n bits 1616 index `NewIndex2` of table `schema`.`table` trx id 22B8A76B4 lock_mode X
Record lock, heap no 1 PHYSICAL RECORD: n_fields 1; compact format; info bits 0
0: len 8; hex 73757072656d756d; asc supremum;;
*** (2) WAITING FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 866959 page no 68831 n bits 72 index `PRIMARY` of table `schema`.`table` trx id 22B8A76B4 lock_mode X locks rec but not gap waiting
Record lock, heap no 5 PHYSICAL RECORD: n_fields 18; compact format; info bits 0
*** WE ROLL BACK TRANSACTION (1)
------------
Notice that here the logical error is in transaction TWO's query. This is a very strong hint:
2616 lock struct(s), heap size 244152, 7262 row lock(s)
...the WHERE needs to lock 7262 rows (!), which from the application logic's point of view is completely nonsensical (see also this for an explanation of "gapping").
Otherwise, you can resort to explicit transactions plus table level locking with LOCK TABLE form_responses WRITE.
The most efficient way to do it is to have all involved methods to enter an explicit transaction:
SET autocommit = 0;
UPDATE form_responses ... WHERE id = ...;
COMMIT WORK;