How to сomprehend the reason of the deadlock - namely, how to find out which transactions have captured which locks?
I have the engine.log file with the following deadlock:
------------------------
LATEST DETECTED DEADLOCK
------------------------
170327 11:09:53
*** (1) TRANSACTION:
TRANSACTION 4 2719072253, ACTIVE 5 sec, OS thread id 26215 starting index read
...
INSERT INTO... (the first transaction)
*** (1) WAITING FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 0 page no 36025889 n bits 96 index `PRIMARY` of table `mydb`.`mytable` trx id 4 2719072253 lock mode S locks rec but not gap waiting
...
*** (2) TRANSACTION:
TRANSACTION 4 2719072205, ACTIVE 35 sec, OS thread id 25564 starting index read, thread declared inside InnoDB 485
UPDATE ... (the second transaction)
*** (2) HOLDS THE LOCK(S):
RECORD LOCKS space id 0 page no 36025889 n bits 96 index `PRIMARY` of table `mydb`.`mytable` trx id 4 2719072205 lock_mode X locks rec but not gap
Record lock, heap no 27 PHYSICAL RECORD: n_fields 72; compact format; info bits 0
...
*** (2) WAITING FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 0 page no 42767646 n bits 120 index `PRIMARY` of table `mydb`.`mytable` trx id 4 2719072205 lock_mode X locks rec but not gap waiting
...
*** WE ROLL BACK TRANSACTION (1)
And my vision of what is described in the logs is the following:
1. Transaction №2 initially has one lock (and the type of the lock isn't clear from logs):
*** (2) HOLDS THE LOCK(S)
RECORD LOCKS space id 0 page no 36025889 n bits 96 index PRIMARY of
table mydb.mytable trx id 4 2719072205 lock_mode X locks rec but not
gap
Record lock, heap no 27 PHYSICAL RECORD: n_fields 72; compact format;
info bits 0
2. Transaction №1 is trying to get the lock of S type:
*** (1) WAITING FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS ... trx id 4 2719072253 lock mode S locks rec but not gap
waiting
and after unsuccessful attempt it begins to wait for releasing of transaction №2 lock;
3. Then transaction №2 is trying to get the lock of X type:
*** (2) WAITING FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS ... trx id 4 2719072205 lock_mode X locks rec but not gap
waiting
and after unsuccessful attempt it begins to wait until transaction №1 gets the S lock and release it.
Do I correctly understand logs, or is my interpretation wrong?
Your interpretation is almost correct. A few thoughts to add:
In innodb there are two basic types of locks: shared (S) and exclusive (X). If a transaction holds an X lock on a record, then neither S or X locks can be granted on the same records until the transaction completes. If a transaction holds an S lock on a record, then another S lock can be granted immediately, but an X lock request must wait until the 1st transaction completes.
The 2nd transaction being an update holds an exclusive (X) lock on the index records that are being updated. Therefore the 1st transaction cannot get hold of an S lock on the same records. However, the 2nd transaction is waiting for X lock to be granted on a different set of records (page no is different from the other locks').
The excerpt does not tell us what transaction locked the records that the 2nd transaction is waiting for. We can only presume that it was the 1st transaction - otherwise it would not be a deadlock.
Related
We are struggling with one deadlock which is occuring multiple times a day in our production environment.
------------------------
LATEST DETECTED DEADLOCK
------------------------
2018-12-27 19:07:34 7fcef1959700
*** (1) TRANSACTION:
TRANSACTION 2125001468, ACTIVE 2 sec starting index read
mysql tables in use 1, locked 1
LOCK WAIT 3 lock struct(s), heap size 1184, 2 row lock(s), undo log entries 1
MySQL thread id 42190185, OS thread handle 0x7fcffc0b1700, query id 918842488 --- updating
UPDATE synchronization SET service_synchronized_at = NULL WHERE id = 116212
*** (1) WAITING FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 615 page no 288 n bits 528 index `PRIMARY` of table `app`.`synchronization` trx table locks 1 total table locks 2 trx id 2125001468 lock_mode X locks rec but not gap waiting lock hold time 2 wait time before grant 0
*** (2) TRANSACTION:
TRANSACTION 2125001355, ACTIVE 5 sec fetching rows
mysql tables in use 2, locked 2
25216 lock struct(s), heap size 3683880, 5297668 row lock(s), undo log entries 94
MySQL thread id 42189517, OS thread handle 0x7fcef1959700, query id 918842042 --- updating
UPDATE synchronization s SET s.service_synchronized_at = now() WHERE s.service_synchronized_at IS NULL AND s.user_id IN (* time consuming select to determine which users should be updated *)
*** (2) HOLDS THE LOCK(S):
RECORD LOCKS space id 615 page no 288 n bits 528 index `PRIMARY` of table `app`.`synchronization` trx table locks 2 total table locks 2 trx id 2125001355 lock_mode X lock hold time 3 wait time before grant 0
*** (2) WAITING FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 615 page no 2313 n bits 472 index `PRIMARY` of table `app`.`synchronization` trx table locks 2 total table locks 2 trx id 2125001355 lock_mode X waiting lock hold time 0 wait time before grant 0
*** WE ROLL BACK TRANSACTION (1)
In one query I want to set service_synchronized_at = NULL. service_synchronized_at is always not null in first query before update.
In second query I add a where condition s.service_synchronized_at IS NULL thinking it would result in non-locking rows with not null values. Guess I was wrong.
The table has only primary index on id and unique constraint on user_id (and of course foreign key on user_id).
Any help is welcome.
I have been getting deadlocks while creating orders in a transaction.
Spree Adjustments table acquires a lock and keeps on updating Promotions or TaxOns. In the meanwhile, other request or order needs to update the same records.
------------------------
LATEST DETECTED DEADLOCK
------------------------
2016-09-17 20:40:15 7fee358d0b00
*** (1) TRANSACTION:
TRANSACTION 3880159183, ACTIVE 2 sec starting index read
mysql tables in use 1, locked 1
LOCK WAIT 10 lock struct(s), heap size 2936, 7 row lock(s), undo log entries 18
MySQL thread id 128414, OS thread handle 0x7fee6fd6ab00, query id 53800114 172.31.34.254 connect_api_user updating
UPDATE `my_variants` SET `my_variants`.`quantity` = 2, `my_variants`.`updated_at` = '2016-09-17 20:40:43' WHERE `my_variants`.`id` = 108430
*** (1) WAITING FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 2414 page no 1809 n bits 88 index `PRIMARY` of table `marketplace`.`my_variants` trx table locks 5 total table locks 2 trx id 3880159183 lock_mode X locks rec but not gap waiting lock hold time 2 wait time before grant 0
*** (2) TRANSACTION:
TRANSACTION 3880159065, ACTIVE 3 sec starting index read
mysql tables in use 1, locked 1
22 lock struct(s), heap size 2936, 15 row lock(s), undo log entries 36
MySQL thread id 125879, OS thread handle 0x7fee358d0b00, query id 53804455 172.31.34.254 connect_api_user updating
UPDATE `spree_adjustments` SET `spree_adjustments`.`eligible` = 0 WHERE `spree_adjustments`.`adjustable_id` = 2298885 AND `spree_adjustments`.`adjustable_type` = 'Spree::LineItem' AND `spree_adjustments`.`source_type` = 'Spree::PromotionAction' AND (`spree_adjustments`.`id` != 665173)
*** (2) HOLDS THE LOCK(S):
RECORD LOCKS space id 2414 page no 1809 n bits 88 index `PRIMARY` of table `marketplace`.`teni_variants` trx table locks 13 total table locks 2 trx id 3880159065 lock_mode X locks rec but not gap lock hold time 2 wait time before grant 0
*** (2) WAITING FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 1544 page no 2024 n bits 432 index `index_spree_adjustments_on_adjustable_id_and_adjustable_type` of table `marketplace`.`spree_adjustments` trx table locks 13 total table locks 2 trx id 3880159065 lock_mode X locks rec but not gap waiting lock hold time 0 wait time before grant 0
*** WE ROLL BACK TRANSACTION (1)
I have a running system with a MySQL database engine.
Running the SHOW ENGINE INNODB STATUS command shows the:
------------------------
LATEST DETECTED DEADLOCK
------------------------
*** (1) TRANSACTION:
UPDATE db.alarm
LEFT JOIN (db.event, db.alarm1)
ON db.event.idevent = db.alarm.idevent
AND db.alarm1.idevent = db.alarm.idevent
SET
idalarmseverity = NAME_CONST('alarmseveritycleared',1),
lastmodifieddate = NAME_CONST('moddate',_binary'2015-01-07 09:02:00' COLLATE 'binary')
WHERE db.event.ideventsource = NAME_CONST('sourceid',3)
AND (db.alarm1.idorder = NAME_CONST('orderid',18894) OR NAME_CONST('orderid',18894) = 0)
AND db.alarm.idalarmseverity > 1
*** (1) WAITING FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 0 page no 32942 n bits 440 index `GEN_CLUST_INDEX` of table `db`.`alarm` trx id 17 2553624600 lock_mode X locks rec but not gap waiting
*** (2) TRANSACTION:
UPDATE db.alarm
LEFT JOIN (db.event, db.alarm2)
ON db.event.idevent=db.alarm.idevent
AND db.alarm2.idevent=db.alarm.idevent
SET
idalarmseverity= NAME_CONST('alarmseveritycleared',1),
lastmodifieddate= NAME_CONST('moddate',_binary'2015-01-07 09:02:00' COLLATE 'binary')
WHERE db.event.ideventsource= NAME_CONST('sourceid',3)
AND db.alarm.idalarmseverity > 1
*** (2) HOLDS THE LOCK(S):
RECORD LOCKS space id 0 page no 32942 n bits 440 index `GEN_CLUST_INDEX` of table `db`.`alarm` trx id 17 2553624599 lock_mode X locks rec but not gap
*** (2) WAITING FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 0 page no 32941 n bits 440 index `GEN_CLUST_INDEX` of table `db`.`alarm` trx id 17 2553624599 lock_mode X locks rec but not gap waiting
At this point I don't know what the 2nd TRANSACTION is waiting for.
The 1st transaction is waiting for a lock (page no 32942) which is hold by the 2nd transaction. That's clear.
But the 2nd transaction is waiting for a lock (page no 32941) which is not hold by anyone.
Or if the 2 locks are the same (or overlaps) than it is the same transaction that currently holds the lock. I don't understand it well enough so it would be great if someone could explain to me how this deadlock could happen and how the UPDATE should be modified to prevent future deadlocks like this.
Thanks
This InnoDB deadlock is really making me pull my hair. As far as I can see:
the transaction (1) is waiting for the PRIMARY on "applications"
The latter has been acquired by (2) for some rather long running updates (SELECT * FROM applications WHERE ID = xxxx FOR UPDATE)
So far, so good - one would expect (1) to wait for the lock and then get on with its work.
However, once (2) gets ready to save it's work (and commit the transaction), it fails with a deadlock since for some reason (1) has managed to get a lock on some secondary index. How the hell did (1) manage to get any locks on the row if the PRIMARY is being held by (2).
One would expect that if (2) originally acquired the PRIMARY lock (SELECT * FROM applications WHERE ID = xxxx FOR UPDATE) it would also have set locks on all the secondary indexes. Is it possible that it will not lock the "tasked" index if tasked==NULL thus allowing (1) acquire a lock on "tasked" before even getting a lock on PRIMARY?
I have had no luck replication this scenario..
Thank you!
Lauri
------------------------
LATEST DETECTED DEADLOCK
------------------------
130428 17:04:06
*** (1) TRANSACTION:
TRANSACTION A369A8C, ACTIVE 1 sec fetching rows
mysql tables in use 3, locked 3
LOCK WAIT 217 lock struct(s), heap size 31160, 636 row lock(s)
MySQL thread id 13310554, OS thread handle 0x7f06cc2d7700, query id 177699568 217.146.78.151 shard67 Sending data
SELECT `applications`.* FROM `applications`
LEFT JOIN `applicants` ON applicants.ID = applications.applicant_ID
LEFT JOIN `regions` ON regions.ID = applicants.region_ID WHERE (status <> 'Blank') AND (status <> 'Closed') AND (revised < 1367154245) AND (tasked IS NULL OR tasked < 1367147045) AND (commence_year >= '2013') AND (regions.instance_ID = '1') ORDER BY `tasked` ASC, `ID` ASC LIMIT 20 FOR UPDATE
*** (1) WAITING FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 20021 page no 1192 n bits 80 index `PRIMARY` of table `dream-shard67`.`applications` trx id A369A8C lock_mode X locks rec but not gap waiting
*** (2) TRANSACTION:
TRANSACTION A369A87, ACTIVE 1 sec updating or deleting
mysql tables in use 1, locked 1
16 lock struct(s), heap size 3112, 22 row lock(s), undo log entries 5
MySQL thread id 13310563, OS thread handle 0x7f06cc151700, query id 177699599 217.146.76.127 shard67
UPDATE `applications` SET `revised` = '1367157846', `tasked` = '1367157846', `revision_ID` = '140649', `xml` = 'Zms6\noMmI$%[v....snipped binary data
*** (2) HOLDS THE LOCK(S):
RECORD LOCKS space id 20021 page no 1192 n bits 72 index `PRIMARY` of table `dream-shard67`.`applications` trx id A369A87 lock_mode X locks rec but not gap
*** (2) WAITING FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 20021 page no 292 n bits 1280 index `tasked` of table `dream-shard67`.`applications` trx id A369A87 lock_mode X locks rec but not gap waiting
*** WE ROLL BACK TRANSACTION (2)
Make sure your doing most of these:
http://dev.mysql.com/doc/refman/5.0/en/innodb-deadlocks.html
I got information from SHOW ENGINE INNODB STATUS
*** (1) TRANSACTION:
TRANSACTION 0 2799914, ACTIVE 1 sec, process no 4106, OS thread id 139808903796480 inserting
mysql tables in use 1, locked 1
LOCK WAIT 10 lock struct(s), heap size 1216, 7 row lock(s), undo log entries 3
MySQL thread id 4284, query id 2889649 localhost 127.0.0.1 test update
INSERT INTO shipping .....
*** (1) WAITING FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 0 page no 436366 n bits 88 index `PRIMARY` of table `testdatabase`.`order` trx id 0 2799914 lock mode S locks rec but not gap waiting
Record lock, heap no 14 PHYSICAL RECORD: n_fields 213; compact format; info bits 0
..........;
*** (2) TRANSACTION:
TRANSACTION 0 2799913, ACTIVE 1 sec, process no 4106, OS thread id 139808905824000 starting index read, thread declared inside InnoDB 500
mysql tables in use 1, locked 1
5 lock struct(s), heap size 1216, 5 row lock(s), undo log entries 4
MySQL thread id 4290, query id 2889711 localhost 127.0.0.1 test Updating
UPDATE order
........
*** (2) HOLDS THE LOCK(S):
RECORD LOCKS space id 0 page no 436366 n bits 88 index `PRIMARY` of table `testdatabase`.`order` trx id 0 2799913 lock_mode X locks rec but not gap
Record lock, heap no 14 PHYSICAL RECORD: n_fields 213; compact format; info bits 0
..........
*** (2) WAITING FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 0 page no 153737 n bits 88 index `PRIMARY` of table `testdatabase`.`order` trx id 0 2799913 lock_mode X locks rec but not gap waiting
Record lock, heap no 10 PHYSICAL RECORD: n_fields 213; compact format; info bits 0
......
*** WE ROLL BACK TRANSACTION (2)
------------
TRANSACTIONS
there is a FK in ship referring to order primary key.
I think T2 has hold x-lock, why it still need wait for a x-lock.
Can some one help me to reproduce such deadlock in mysql?
Thanks.
I don't know your queries, but seems like you're inserting row into child table, and then doing an update of row in parent table.
If that's true, you've hit this issue in MySQL: http://bugs.mysql.com/bug.php?id=48652
If a FOREIGN KEY constraint is defined on a table, any insert, update, or delete that requires the constraint condition to be checked sets shared record-level locks on the records that it looks at to check the constraint. InnoDB also sets these locks in the case where the constraint fails.
On a single record of the first table you have:
S lock from transaction 1 set,
S lock from transaction 2 set,
X lock from transaction 1 requested, blocked by S lock from transaction 2,
X lock from transaction 2 requested, blocked by S lock from transaction 1
Possible solution is to first update parent table, then insert row into child table. Suppose we need to increment some counter upon child row insertion, then queries would be:
UPDATE <parent row> SET count = count + 1;
INSERT <child row>; /* if the INSERT fails, roll back the trx */
If you want to update parent row only after inserting child row, you can use FOR UPDATE statement to set a lock on parent row:
SELECT <parent row> FOR UPDATE;
INSERT <child row>; /* if the INSERT fails, roll back the trx */
UPDATE <parent row> SET count = count + 1;