I am performing SELECT ... FOR UPDATE or row level locking with InnoDB tables.
My intention is to only one request can read the same row. So if two users make request for the same data as the same time. Only one of them get data, who fires the query first.
But How can i test that locking is placed or not. as I am testing it by retrieving the same data at same time and both users getting the data.
Note: My tables are InnoDB, My query executes in transaction, my query as below:
SELECT * FROM table_name WHERE cond FOR UPDATE;
Any other thing I have to check for this to make work?
open 2 mysql client session.
on session 1:
mysql> start transaction;
mysql> SELECT * FROM table_name WHERE cond FOR UPDATE;
... (result here) ...
1 row in set (0.00 sec)
on session 2:
mysql> start transaction;
mysql> SELECT * FROM table_name WHERE cond FOR UPDATE;
... (no result yet, will wait for the lock to be released) ...
back to session 1, to update selected record (and release the lock):
mysql> UPDATE table_name SET something WHERE cond;
mysql> commit;
back to session 2:
1) either showing lock timeout error
ERROR 1205 (HY000): Lock wait timeout exceeded; try restarting transaction
2) or showing result
... (result here) ...
1 row in set (0.00 sec)
3) or showing no result (because corresponding record has been modified, so specified condition was not met)
Empty set (0.00 sec)
You can use own lock mechanizm with lock_by column.
UPDATE table_name SET locked_by=#{proccess_id} WHERE cond and locked_by IS NULL
Now in your program you will get count of affected rows:
if(affected_rows==0)
return 'rows locked'
else
//do your staff with locked_by=#{process_id} rows
With this mechanism you can control locked rows and locking processes. You can also add in UPDATE statement locked_at=NOW() to get more info about locked row.
Don't forget to add some index on locked_by column.
Here is MySQL docs about working with locks.
Before update you can put lock, releasing it after. In another transaction you can check lock using it unique name. Strategy for naming you can choose yourself.
Related
To simulate a lock in mysql I can grab the row with the following:
BEGIN;
SELECT * FROM table WHERE id=1 FOR UPDATE;
Now, if I try and update that row (from another connection) it will raise the following error after innodb_lock_wait_timeout seconds (default: 50):
(1205, 'Lock wait timeout exceeded; try restarting transaction')
How would I simulate a deadlock then, so I get an error that looks like:
Deadlock found when trying to get lock; try restarting transaction”
When I try and query or update the row?
Update: even when trying to simulate the mysql deadlock example, I get Lock wait timeout exceeded; try restarting transaction rather than a deadlock message.
First of all, refering to your last edit, the example in the manual should work. If it doesn't, there is either a fundamental problem, or you are missing some detail, so I would start there and make sure that you get it working.
The deadlock example has 3 steps, and I suspect you may have missed the last one:
T1: select
T2: delete. T2 has to wait for T1 now. Waiting means, that MySQL currently still sees a possible way that both T1 and T2 can finish successfully! For example, T1 can just commit now. Noone knows, so T2 waits for what happens. If you wait too long in this step, you will get a timeout (which is what I suspect happened).
T1: delete. This will result in a deadlock in T2. You need this last step to create a non-resolvable conflict.
You should try that example first, and carefully, as the devil is in the details. Leading to a detail in your own example:
You are using SELECT ... FOR UPDATE. FOR UPDATE is actually a way to reduce the number of deadlocks (which is the opposite of what you want), at the price of locking more restrictively. E.g. you have more situation where MySQL waits just to be safe, instead of going on and hoping it will work out eventually (or not, hence deadlock). Note that the example in the manual uses LOCK IN SHARE MODE for that reason.
So to modify and expand your own example to get a deadlock, you can do
T1: START TRANSACTION;
SELECT * FROM table WHERE id=1 LOCK IN SHARE MODE;
T2: START TRANSACTION;
UPDATE table SET id=2 WHERE id=1
-- wait
T1: UPDATE table SET id=2 WHERE id=1
-- deadlock in T2
For completeness (and to exclude a potential misunderstanding): the row has to exists, if your table is e.g. empty, you won't get a deadlock.
If you use FOR UPDATE instead, you don't get a deadlock, but T2 keeps waiting until you commit/rollback T1. It has to do with the way locking works, but you can maybe get an idea of that if you add a select to T2:
T1: START TRANSACTION;
SELECT * FROM table WHERE id=1 LOCK IN SHARE MODE;
T2: START TRANSACTION;
SELECT * FROM table WHERE id=1 LOCK IN SHARE MODE;
-- fine in shared mode. Waits here if you use `for update`!
T1: UPDATE table SET id=2 WHERE id=1
-- wait
T2: UPDATE table SET id=2 WHERE id=1
-- deadlock
If you replace both LOCK IN SHARE MODE with FOR UPDATE, T2 will wait at/before the select, until T1 commits, without a deadlock.
Is Deadlock Detection enabled?
You can read more here: https://dev.mysql.com/doc/refman/8.0/en/innodb-deadlock-detection.html
A mechanism that automatically detects when a deadlock occurs, and automatically rolls back one of the transactions involved (the victim). Deadlock detection can be disabled using the innodb_deadlock_detect configuration option.
Lock another table in another transaction and then try to access other transactions table.
For example:
In transaction A lock table 1
In transaction B lock table 2
In transaction A to update table 2
In transaction B to update table 1.
Also, you can increase your timeout to 5 minutes so that while you are creating your deadlock it doesn't timeout.
UPDATE:
An example
In session A:
START TRANSACTION;
UPDATE tbl1 SET b=1 WHERE id=1;
in session B:
START TRANSACTION;
UPDATE tbl2 SET b=1 WHERE id=1;
Then
In session A:
UPDATE tbl2 SET b=1 WHERE id=1;
in session B:
UPDATE tbl1 SET b=1 WHERE id=1;
I did it like below
-- connection 1
START TRANSACTION;
LOCK TABLES [table_name] WRITE;
-- connection 2
START TRANSACTION;
SELECT * FROM [table_name]; -- waiting for table metadata lock
I don't understand above situation.
I thought WRITE LOCK doesn't prevent SELECT query.
When I did like below,
-- connection 1
START TRANSACTION;
UPDATE [table_name] SET [column = value] WHERE id = 1;
-- connection 2
START TRANSACTION;
SELECT * FROM [table_name] WHERE id = 1; -- doesn't wait
SELECT doesn't wait as you saw.
And I thought WRITE LOCK also work like UPDATE clause.
But it seems like not..
Now I know something wrong?
Write lock does not work like row-level locking.
https://dev.mysql.com/doc/refman/8.0/en/lock-tables.html says:
WRITE lock:
Only the session that holds the lock can access the table. No other session can access it until the lock is released.
"Access" in this context means read or write.
I have 5 rows in a table (1 to 5). I want row 2 lock for some update and in the meanwhile if someone tries to update row 4, then he should able to update.
I am trying this with code below, but I feel its placing lock on table level rather than row level.
------ session 1
START TRANSACTION;
SELECT * FROM test WHERE t=1 FOR UPDATE;
UPDATE test SET NAME='irfandd' WHERE t=2;
COMMIT;
----- session 2 (which is being blocked)
START TRANSACTION;
UPDATE test SET NAME='irfandd' WHERE t=4;
COMMIT;
Instead of FOR UPDATE use LOCK IN SHARE MODE. FOR UPDATE prevents other transactions to read the row as well. LOCK IN SHARE MODE allows read, but prevents updating.
Reference: MySQL Manual
------ session 1
START TRANSACTION;
SELECT * FROM test WHERE t=1 LOCK IN SHARE MODE;
UPDATE test SET NAME='irfandd' WHERE t=2;
COMMIT;
----- session 2 (which is not being blocked anymore :) )
START TRANSACTION;
UPDATE test SET NAME='irfandd' WHERE t=4;
COMMIT;
Update:
Realizing that the table has no index on t, I have the following explanation:
First, transaction T1 locks the row 1 in SELECT * FROM test WHERE t=1 FOR UPDATE
Next, transaction T2 tries to execute UPDATE test SET NAME='irfandd' WHERE t=4. To find out which row(s) are affected, it needs to scan all rows, including row 1. But that is locked, so T2 must wait until T1 finishes.
If there is any kind of index, the WHERE t=4 can use the index to decide if row 1 contains t=4 or not, so no need to wait.
Option 1: add an index on test.t so your update can use it.
Option 2: use LOCK IN SHARE MODE, which is intended for putting a read lock only.
Unfortunately this option creates a deadlock. Interestingly, T2 transaction executes (updating row 4), and T1 fails (updating row 2). It seems that T1 read-locks row 4 also, and since T2 modifies it, T1 fails because of the transaction isolation level (REPEATABLE READ by default). The final solution would be playing with Transaction Isolation Levels, using READ UNCOMMITTED or READ COMMITTED transaction levels.
The simplest is Option 1, IMHO, but it's up to your possibilities.
I found below option is more appropriate i generate 40000 numbers from concurrent session on the same time. I didnt found any duplicate number. Without below command i generate 10000 numbers and found 5 duplicate numbers.
START TRANSACTION
SELECT * FROM test WHERE t=1 FOR UPDATE;
UPDATE test SET NAME='irfandd' WHERE t=2;
COMMIT;
I am getting following exception when I try to do a batch update. There are multiple threads running at same time which might be accessing a row in database. I am doing multiple batch updates. Can anyone please comment on relation between size of batch and deadlock ? By decreasing the batch size (currently batch size = 1000), will the probability of deadlock decrease ?
The exception I am getting is
com.mysql.jdbc.exceptions.MySQLTransactionRollbackException: Deadlock found when trying to get lock; try restarting transaction
Short answer:
yes, the probability would decrease
Long answer:
Lets figure out why the deadlocks are occurring. When you update a row an exclusive lock is set on this particular row and it will be held until your transaction is commited/rolled back.
That means, no other transaction may update it — it would just block until the transaction is finished. A deadlock would occur when tran1 is willing to lock rows being held by tran2, and tran2, in turn, is already waiting for some rows locked by tran1
Here's an example:
MariaDB [test]> create table a (id int primary key, value int);
Query OK, 0 rows affected (0.14 sec)
MariaDB [test]> insert into a values (1, 0), (2, 0), (3, 0), (4, 0);
Query OK, 4 rows affected (0.00 sec)
Records: 4 Duplicates: 0 Warnings: 0
mysql console 1:
step 1> start transaction;
step 3> update a set value = 1 where id = 2;
step 5> update a set value = 1 where id = 1;
mysql console 2:
step 2> start transaction;
step 4> update a set value = 1 where id = 1;
step 6> update a set value = 1 where id = 2;
ERROR 1213 (40001): Deadlock found when trying to get lock; try restarting transaction
The more rows are being touched(=updated) during every batch update, the higher the probability of such kind of conflicts is.
You might lower this probability by traversing the rows in a well-defined order. In this case the simple example I've provided wouldn't be feasible.
More details on avoiding deadlocks are in this awesome article:
http://www.xaprb.com/blog/2006/08/03/a-little-known-way-to-cause-a-database-deadlock/
Edit:
I found a solution here http://mysql.bigresource.com/Track/mysql-8TvKWIvE/
assuming select takes a long time to execute, will this lock the table for a long time?
SET SESSION TRANSACTION ISOLATION LEVEL REPEATABLE READ;
START TRANSACTION;
SELECT foo FROM bar WHERE wee = 'yahoo!';
DELETE FROM bar WHERE wee = 'yahoo!';
COMMIT;
I wish to use a criteria to select the rows in mysql, return them to my app as resultset, and then delete these rows. How can this be done? I know I can do the following but it's too inefficient:
select * from MyTable t where _critera_.
//get the resultset and then
delete from MyTable t where t.id in(...result...)
Do I need to use a transaction? Is there a single query solution?
I needed to SELECT some rows by some criteria, do something with the data, and then DELETE those same rows atomically, that is, without deleting any rows that meet the criteria but were inserted after the SELECT.
Contrary to other answers, REPEATABLE READ is not sufficient. Refer to Consistent Nonlocking Reads. In particular note this callout:
The snapshot of the database state applies to SELECT statements within a transaction, not necessarily to DML statements. If you insert or modify some rows and then commit that transaction, a DELETE or UPDATE statement issued from another concurrent REPEATABLE READ transaction could affect those just-committed rows, even though the session could not query them.
You can try it yourself:
First create a table:
CREATE TABLE x (i INT NOT NULL, PRIMARY KEY (i)) ENGINE = InnoDB;
Start a transaction and examine the table (this will be called session 1 now):
SET TRANSACTION ISOLATION LEVEL REPEATABLE READ;
START TRANSACTION;
SELECT * FROM x;
Start another session (session 2) and insert a row. Note this session is in auto commit mode.
INSERT INTO x VALUES (1);
SELECT * FROM x;
You will see your newly inserted row. Then back in session 1 again:
SELECT * FROM x;
DELETE FROM x;
COMMIT;
In session 2:
SELECT * FROM x;
You'll see that even though you get nothing from the SELECT in session 1, you delete one row. In session 2 you will see the table is empty at the end. Note the following output from session 1 in particular:
mysql> SET TRANSACTION ISOLATION LEVEL REPEATABLE READ;
Query OK, 0 rows affected (0.00 sec)
mysql> START TRANSACTION;
Query OK, 0 rows affected (0.00 sec)
mysql> SELECT * FROM x;
Empty set (0.00 sec)
/* --- insert in session 2 happened here --- */
mysql> SELECT * FROM x;
Empty set (0.00 sec)
mysql> DELETE FROM x;
Query OK, 1 row affected (0.00 sec)
mysql> COMMIT;
Query OK, 0 rows affected (0.06 sec)
mysql> SELECT * FROM x;
Empty set (0.00 sec)
This testing was done with MySQL 5.5.12.
For a correct solution
Use SERIALIZABLE transaction isolation level. However note that session 2 will block on the INSERT.
It seems that SELECT...FOR UPDATE will also do the trick. I have not studied the manual 100% in depth to understand this but it worked when I tried it. The advantage is you don't have to change the transaction isolation level. Again, session 2 will block on the INSERT.
Delete the rows individually after the SELECT. Basically you'd have to include a unique column (the primary key would be good) in the SELECT and then use DELETE FROM x WHERE i IN (...), or something similar, where IN contains a list of keys from the SELECT's result set. The advantage is you don't need to use a transaction at all and session 2 will not be blocked at any time. The disadvantage is that you have more data to send back and forth to the SQL server. Also I don't know if deleting the rows individually is as efficient as using the same WHERE clause as the original SELECT, but if the original SELECT's WHERE clause was complicated or slow the individual deletion may well be faster, so that could be another advantage.
To editorialize, this is one of those things that is so dangerous that even though it's documented it could almost be considered a "bug." But hey, the MySQL designers didn't ask me (or anyone else, apparently).
Do I need to use a transaction? Is there a single query solution?
Yes, you need to use a transaction. You cannot delete and select rows in a single query (i.e., there is no way to "return" or "select" the rows you have deleted).
You don't necessarily need to do the REPEATABLE READ option - I believe you could also select the rows FOR UPDATE, although this is a higher level of locking. REPEATABLE READ does seem to be the lowest level of locking you could use to execute this transaction safely. It happens to be the default for InnoDB.
How much this affects your table depends on whether you have an index on the wee column or not. Without it, I believe MySQL would have to lock writes the entire table.
Further reading:
Wikipedia - Isolation (database systems)
http://dev.mysql.com/doc/refman/5.0/en/set-transaction.html
http://dev.mysql.com/doc/refman/5.0/en/innodb-locking-reads.html
Do a select statement. While looping through it, create a list string of unique IDs. Then pass this list back to mySQL using IN.
You could select your rows into a temporary table, then delete using the same criteria as your select. Since SELECT FROM WHERE FOR UPDATE also returns a result set, you could alter the SELECT FOR UPDATE to a SELECT INTO tmp_table FOR UPDATE. Then delete your selected rows, either using your original criteria, or by using the data in the temporary table as the criteria.
Something like this (but haven't checked it for syntax)
START TRANSACTION;
SELECT a,b into TMP_TABLE FROM table_a WHERE a=1 FOR UPDATE;
DELETE FROM table_a
USING table_a JOIN TMP_TABLE ON (table_a.a=TMP_TABLE.a, table_a.b=TMP_TABLE.b)
WHERE 1=1;
COMMIT;
Now your records are gone from the original table, but you also have a copy in your temporary table, which you can keep, or delete.
There is no single query solution. Use
select * from MyTable t where _critera_
//get the resultset and then
delete from MyTable where _critera_
Execute the SELECT statement with the WHERE clause and then use the same WHERE clause in the DELETE statement as well. Assuming there was no interim changes to the data, the same rows should be deleted.
EDIT: Yes, you could set this up as a single transaction so there's no modification to the tables while you're doing this.