I know that using the locks or MVCC in Mysql can achieve concurrency control, such as repeatable-reading. But I don't know how MVCC avoids phantom-reading. In other places, I learned that it is generally implemented through MVCC and Gap-Lock, but what I currently understand is that MVCC does not need locks, that is, both updates and deletions are implemented using undo-logs. If so, how do MVCC and the lock mechanism work together?
For example, to avoid phantom-reading, would MVCC add a gap-lock on some rows in T1? If so, how MVCC does when updates occurred in T2, just appends a update undo-log generally? or blocks it?
MySQL (specifically, InnoDB) does not support REPEATABLE-READ for locking statements. For example, UPDATE, DELETE or SELECT...FOR UPDATE. These statements always take locks on the most recently committed row version, as if the transaction isolation level were READ-COMMITTED.
You can observe this happening:
mysql> create table mytable (id int primary key, x int);
Query OK, 0 rows affected (0.05 sec)
mysql> insert into mytable values (1, 42);
Query OK, 1 row affected (0.02 sec)
mysql> start transaction;
Query OK, 0 rows affected (0.00 sec)
mysql> select * from mytable;
+----+------+
| id | x |
+----+------+
| 1 | 42 |
+----+------+
So far, so good. Now open a second window and update the value:
mysql> update mytable set x = 84;
Query OK, 1 row affected (0.03 sec)
Rows matched: 1 Changed: 1 Warnings: 0
Now back in the first window, a non-locking read still views the original value because of REPEATABLE-READ, but a locking read views the most recently committed version:
mysql> select * from mytable;
+----+------+
| id | x |
+----+------+
| 1 | 42 |
+----+------+
1 row in set (0.00 sec)
mysql> select * from mytable for update;
+----+------+
| id | x |
+----+------+
| 1 | 84 |
+----+------+
1 row in set (0.00 sec)
mysql> select * from mytable;
+----+------+
| id | x |
+----+------+
| 1 | 42 |
+----+------+
1 row in set (0.00 sec)
You can go back and forth as many times as you want, and the same transaction can return both values, depending on doing a locking read vs. non-locking read.
This is a strange behavior of InnoDB, but it allows reads to not be blocked. I have used other MVCC implementations such as InterBase/Firebird, which solve this differently. It would block the read until the transaction in the second window commits or rolls back. If it rolls back, then the locking read can read the original value. If the other transaction commits, then the locking read gets an error.
InnoDB makes a different choice on how to implement MVCC, to avoid blocking the read. But it causes the strange behavior where a locking read must view the latest committed row version.
As the song says, "you can't always get what you want."
Related
animals table
+----------+-------+
| name | value |
+----------+-------+
| Aardvark | 10 |
+----------+-------+
birds table
+---------+-------+
| name | value |
+---------+-------+
| Buzzard | 20 |
+---------+-------+
Session 1:
mysql> START TRANSACTION;
Query OK, 0 rows affected (0.00 sec)
mysql> SELECT value FROM Animals WHERE name='Aardvark' FOR SHARE;
+-------+
| value |
+-------+
| 10 |
+-------+
1 row in set (0.00 sec)
Session 2:
mysql> START TRANSACTION;
Query OK, 0 rows affected (0.00 sec)
mysql> SELECT value FROM Birds WHERE name='Buzzard' FOR SHARE;
+-------+
| value |
+-------+
| 20 |
+-------+
1 row in set (0.00 sec)
--waits to lock
mysql> UPDATE Animals SET value=30 WHERE name='Aardvark';
Session 1:
mysql> UPDATE Birds SET value=40 WHERE name='Buzzard';
ERROR 1213 (40001): Deadlock found when trying to get lock; try restarting transaction
Can we say that the lock is acquired UNTIL THE SQL SENTENCE IS REACHED or does a transaction obtain all locks before?
A lot of people say that we have to acquire all the (anticipated) locks that we are going to use throughout the transaction to avoid deadlocks, so the question arose in my mind.
Yes, row locks are acquired on demand, when you execute the SQL statement that requires those locks.
This means that if two sessions are running concurrently, they may acquire their respective locks in an interleaved fashion, not an atomic fashion. Therefore they might both want to lock a resource that is already locked by the other session, and in that case they create a deadlock.
One fix to prevent deadlocks is to acquire all the locks you need during the transaction — and more to the point, acquire the locks atomically. That is, all the locks must be acquired at once, and if that isn't successful, then release all the locks. Locks cannot be acquired in an interleaved fashion.
InnoDB doesn't really have a feature to do this. You may resort to table locking with MySQL's LOCK TABLES statement.
A different strategy is not to avoid deadlocks, but just recover from deadlocks when they occur.
I am at the REPEATABLE-READ level.
Why does it make me wait?
I understand that all reads (SELECTs) at any level are non-blocking.
what am I missing?
Session 1:
mysql> lock tables users write;
Query OK, 0 rows affected (0.00 sec)
Session 2:
mysql> begin;
Query OK, 0 rows affected (0.00 sec)
mysql> select * from users where id = 1; // wait
Session 1:
mysql> unlock tables;
Query OK, 0 rows affected (0.00 sec)
Session 2:
mysql> select * from users where id = 1;
+----+-----------------+--------------------+------+---------------------+--------------------------------------------------------------+----------------+---------------------+---------------------+------------+
| id | name | email | rol | email_verified_at | password | remember_token | created_at | updated_at | deleted_at |
+----+-----------------+--------------------+------+---------------------+--------------------------------------------------------------+----------------+---------------------+---------------------+------------+
| 1 | Bella Lueilwitz | orlo19#example.com | NULL | 2022-08-01 17:22:29 | $2y$10$92IXUNpkjO0rOQ5byMi.Ye4oKoEa3Ro9llC/.og/at2.uheWG/igi | MvMlaX9TQj | 2022-08-01 17:22:29 | 2022-08-01 17:22:29 | NULL |
+----+-----------------+--------------------+------+---------------------+--------------------------------------------------------------+----------------+---------------------+---------------------+------------+
1 row in set (10.51 sec)
In this question the opposite is true
Why doesn't LOCK TABLES [table] WRITE prevent table reads?
You reference a question about MySQL 5.0 posted in 2013. The answer from that time suggests that the client was allowed to get a result that had been cached in the query cache. Since then, MySQL 5.6 and 5.7 disabled the query cache by default, and MySQL 8.0 removed the feature altogether. This is a good thing.
The documentation says:
WRITE lock:
Only the session that holds the lock can access the table. No other session can access it until the lock is released.
This was true in the MySQL 5.0 days too, but the query cache allowed some clients to get around it. But I guess it wasn't reliable even then, because if the client ran a query that happened not to be cached, I suppose it would revert to the documented behavior. Anyway, it's moot, because all currently supported versions of MySQL should have the query cache disabled or removed.
On MySQL, first, I ran BEGIN; or START TRANSACTION; to start a transaction as shown below:
mysql> BEGIN;
Query OK, 0 rows affected (0.00 sec)
Or:
mysql> START TRANSACTION;
Query OK, 0 rows affected (0.00 sec)
Then, I ran the query below to check if the transaction is running but I got nothing as shown below:
mysql> SELECT trx_id FROM information_schema.innodb_trx;
Empty set (0.00 sec)
So just after this, I ran the query below to show all the rows of person table:
mysql> SELECT * FROM person;
+----+------+
| id | name |
+----+------+
| 1 | John |
| 2 | Tom |
+----+------+
2 rows in set (0.00 sec)
Then again, I ran the query below, then now, I could check the transaction is actually running:
mysql> SELECT trx_id FROM information_schema.innodb_trx;
+-----------------+
| trx_id |
+-----------------+
| 284321631771648 |
+-----------------+
1 row in set (0.00 sec)
So, does only running BEGIN; or START TRANSACTION; query really start a transaction?
Yes. start transaction starts a transaction. Otherwise, that statement would have a very misleading name.
You can check this by trying to do things you cannot do inside a transaction, e.g. change the isolation level:
START TRANSACTION;
SET TRANSACTION ISOLATION LEVEL SERIALIZABLE;
ERROR 1568 (25001): Transaction characteristics can't be changed while a transaction is in progress
So why don't you see it in the innodb_trx table?
Because innodb_trx only lists transactions that involve InnoDB. Until you e.g. query an InnoDB-table, the InnoDB engine doesn't know about that transaction and doesn't list it, just as shown in your tests. Try changing the table to a MyISAM-table. It will not show up in innodb_trx even after you ran a query on it.
A transaction is started on the mysql level, not the storage engine level.
For example, the NDB engine also supports transactions and has its own transaction table where it only shows transactions that involve NDB tables.
I'm implementing a custom table-based sequence generator for MySQL database v5.7.16 with InnoDB engine.
The sequence_table looks as follows:
+-------------+-----------+
|sequence_name|next_value |
+-------------+-----------+
| first_seq | 1 |
+-------------+-----------+
| second_seq | 1 |
+-------------+-----------+
sequence_name column is a primary key.
This sequence table contains multiple sequences for different consumers.
I use the following strategy for the sequence updates:
Select current sequence value: select next_val from sequence_table where sequence_name=?.
Add the allocation size to current sequence value.
Update the sequence value if it's current value matches the value selected in the first step: update sequence_table set next_val=? where sequence_name=? and next_val=?.
If the update is successful return the increased sequence value, otherwise repeat the process from step 1.
The documentation contains the following information:
UPDATE ... WHERE ... sets an exclusive next-key lock on every record
the search encounters. However, only an index record lock is required
for statements that lock rows using a unique index to search for a
unique row. 14.5.3 Locks Set by Different SQL Statements in InnoDB
The part in bold is a bit confusing.
As you can see, I match the primary key in the WHERE clause of the UPDATE statement.
Is it possible that the search may encounter more than one record and therefore lock multiple rows in this sequence table?
In other words, will the update in the 3rd step of the algorithm block just one or multiple rows?
You didn't mention what transaction isolation level you're planning to use.
Lets assume you're using repeatable read (in read committed no such a problem should exist)
From here:
For locking reads (SELECT with FOR UPDATE or LOCK IN SHARE MODE),
UPDATE, and DELETE statements, locking depends on whether the
statement uses a unique index with a unique search condition, or a
range-type search condition
and
For a unique index with a unique search condition, InnoDB locks only
the index record found, not the gap before it
So at least in theory it should lock only a single record and no next-key lock will be used.
More quotes from other docs pages to back my thoughts:
innodb-next-key-locks
link
A next-key lock is a combination of a record lock on the index record
and a gap lock on the gap before the index record.
gap locks
link
Gap locking is not needed for statements that lock rows using a unique
index to search for a unique row
Don't grab the sequence numbers inside the main transaction; do it before the START TRANSCTION.
Do the task in a single statement with autocommit=ON.
Both of those lead to it being much faster, less likely to block.
(You code was missing BEGIN/COMMIT and FOR UPDATE. I got rid of those rather than explaining the issues.)
Set up test:
mysql> CREATE TABLE so49197964 (
-> name VARCHAR(22) NOT NULL,
-> next_value INT UNSIGNED NOT NULL,
-> PRIMARY KEY (name)
-> ) ENGINE=InnoDB;
Query OK, 0 rows affected (0.02 sec)
mysql> INSERT INTO so49197964 (name, next_value)
-> VALUES
-> ('first', 1), ('second', 1);
Query OK, 2 rows affected (0.00 sec)
Records: 2 Duplicates: 0 Warnings: 0
mysql> SELECT * FROM so49197964;
+--------+------------+
| name | next_value |
+--------+------------+
| first | 1 |
| second | 1 |
+--------+------------+
2 rows in set (0.00 sec)
Grab 20 nums from 'first' and fetch the starting number:
mysql> UPDATE so49197964
-> SET next_value = LAST_INSERT_ID(next_value) + 20
-> WHERE name = 'first';
Query OK, 1 row affected (0.00 sec)
Rows matched: 1 Changed: 1 Warnings: 0
mysql> SELECT LAST_INSERT_ID();
+------------------+
| LAST_INSERT_ID() |
+------------------+
| 1 |
+------------------+
1 row in set (0.00 sec)
mysql> SELECT * FROM so49197964;
+--------+------------+
| name | next_value |
+--------+------------+
| first | 21 |
| second | 1 |
+--------+------------+
2 rows in set (0.00 sec)
Grab another 20:
mysql> UPDATE so49197964
-> SET next_value = LAST_INSERT_ID(next_value) + 20
-> WHERE name = 'first';
Query OK, 1 row affected (0.00 sec)
Rows matched: 1 Changed: 1 Warnings: 0
mysql> SELECT LAST_INSERT_ID();
+------------------+
| LAST_INSERT_ID() |
+------------------+
| 21 |
+------------------+
1 row in set (0.00 sec)
mysql> SELECT * FROM so49197964;
+--------+------------+
| name | next_value |
+--------+------------+
| first | 41 |
| second | 1 |
+--------+------------+
2 rows in set (0.00 sec)
As per the MySql documentation, MySql supports Multiple granularity locking(MGL).
case-1
Opened terminal-1:
// connected to mysql
mysql> start transaction;
Query OK, 0 rows affected (0.00 sec)
mysql> select id, status from tracking_number limit 5 for update;
+----+--------+
| id | status |
+----+--------+
| 1 | 0 |
| 2 | 0 |
| 3 | 0 |
| 4 | 0 |
| 5 | 0 |
+----+--------+
5 rows in set (0.00 sec)
mysql>
left it opened and opened terminal-2:
// connected to mysql
mysql> start transaction;
Query OK, 0 rows affected (0.00 sec)
mysql> select id, status from tracking_number limit 5 for update;
<!-- Hangs here. and after some time it says-->
ERROR 1205 (HY000): Lock wait timeout exceeded; try restarting transaction
Though there are plenty of rows to retrieve, T2 waits until t1 completes.
case-2
Left terminal-1 as is.Now in terminal-2:
mysql> start transaction;
Query OK, 0 rows affected (0.00 sec)
<!-- case 2.1 -->
mysql> select id, status from tracking_number where id=1;
+----+--------+
| id | status |
+----+--------+
| 1 | 0 |
+----+--------+
1 row in set (0.00 sec)
mysql> select id, status from tracking_number where id=2;
+----+--------+
| id | status |
+----+--------+
| 2 | 0 |
+----+--------+
1 row in set (0.00 sec)
<!-- case 2.2 -->
mysql> select * from tracking_number where id=2 for update;
<!-- Hangs here. and after some time -->
ERROR 1205 (HY000): Lock wait timeout exceeded; try restarting transaction
But why in case 1, T2 waits for the same set of rows that T1 has locked?
Does it mean the unbounded select query (even with limint parameter. I have tried with different range also) blocks the entire table?
Is there any way to let transactions to lock independently without specifying the field of the record(i.e., without using where field=value)?
Generally (or as per Java concurrent locking), write lock is exclusive and read is not. In case 2.1, though the records are in write lock mode, how T2 can read the same records? Since this is allowed what is the point in locking it?
Case 2.2 is understood.
Opened a terminal and a transaction:
mysql> update tracking_number set status=4 where status=0 limit 5;
Query OK, 5 rows affected (0.00 sec)
Rows matched: 5 Changed: 5 Warnings: 0
Left it there and opened another terminal and transaction:
mysql> update tracking_number set status=5 where status=0 limit 5;
T2 did not succeed until i committed (or rollback) T1.
Why is this behavior?
Let me go through your cases and explain how these locks work:
1 case
T1 wants to update some rows in your test table. This transaction puts IX lock on all table and X lock on the first 5 rows.
T2 wants to update some rows in your test table. This transaction puts IX (because IX compatible with IX) lock on all table and tries to first 5 rows but it can't do it because X is not compatible with X
So we are fine.
2.1 case
T1 wants to update some rows in your test table. This transaction put IX lock on all table and X lock on the first 5 rows.
T2 wants to select some rows from your test table. And it does not place any locks (because InnoDB provides non-locking reads)
2.1 case
T1 wants to update some rows in your test table. This transaction put IX lock on all table and X lock on the first 5 rows.
T2 wants to update (select for update)some rows from your test table. Place IS on the whole table and tries to get S lock on the row and fails because X and S are uncompatible.
Also always be aware of isolation level: different level cause different mechanism to free/acquire locks
Hope it helps