Is it possible to lock a table such that the holder can read and write, and other sessions can only read?
The documentation seems to suggestion that a read lock allows everyone to only read, and a write lock allows only the holder to read and write and other sessions have no access. Seems like having the holder able to read and write and other sessions only able to read would be a pretty frequently needed behavior -- perhaps the most frequently needed behavior.
Maybe the performance hit in implementing this scenario would be too high?
Take a look at LOCK IN SHARE MODE.
This will let you set non blocking read locks.
But remember, this can lead to deadlocks! Make sure you are okay with processes having out of date information.
There are many correct words in existing answers, but no one seems to have given a clear answer. I will try.
As you have already seen in documentation on LOCK TABLES, it can not be used for the purpose, since for the READ lock:
The session that holds the lock can read the table (but not write it).
and for the WRITE lock:
Only the session that holds the lock can access the table. No other session can access it until the lock is released.
That is the effect can hardly be achievable with an arbitrary engine table, but it can be achived with a transactional engine, that is InnoDB.
Let's think about what means that a single session keeps a constant write lock on a table and other tables can read data from the table in terms of transactions. That means that we have an open long living transaction (let it be W transaction) which locks a table for modifications and other transactions (in other sessions) can read data that is already modified, but not yet committed. In terms of isolation levels, that means that we should set up the default isolation level to READ-UNCOMMITTED, so that we would not have to change the isolation level for each new session:
SET GLOBAL TRANSACTION ISOLATION LEVEL READ UNCOMMITTED;
But our transaction W, should use a stronger isolation level, otherwise we can not apply any locking to our table. READ-COMMITTED is not strong enough, but REPEATABLE-READ is exactly what we want. That is befor starting a W transaction we should set the transaction level for the current session:
SET SESSION TRANSACTION ISOLATION LEVEL REPEATABLE READ;
Now, how to lock the whole table. Let's create a table:
CREATE TABLE t (
id INTEGER UNSIGNED NOT NULL AUTO_INCREMENT,
val VARCHAR(45) NOT NULL,
PRIMARY KEY (id)
) ENGINE = InnoDB;
LOCK IN SHARE MODE is not what we want:
If any of these rows [that are read] were changed by another transaction that has not yet committed, your query waits until that transaction ends and then uses the latest values.
LOCK FOR UPDATE seems to do what we need:
SELECT ... FOR UPDATE locks the rows and any associated index entries.
Now all we need is to lock the rows. The simplest thing we can to is to lock the primary key. COUNT(*) does a full index scan for InnoDB (since InnoDB does not know that exact row count).
SET GLOBAL TRANSACTION ISOLATION LEVEL READ UNCOMMITTED;
SET SESSION TRANSACTION ISOLATION LEVEL REPEATABLE READ;
START TRANSACTION;
SELECT COUNT(*) FROM t FOR UPDATE;
INSERT INTO t VALUES (NULL, '');
Now you can open other sessions and try to read the data from the table and try to add or modify the existing data from those sessions.
The problem is though, that you should commit the modifications in W, and as soon as you commit the transaction, the lock is released and all waiting inserts or updates are applied as well, even if you commit it with:
COMMIT AND CHAIN; SELECT COUNT(*) FROM ti FOR UPDATE;
The moral of the story is that it is much easier to have two MySQL accounts: a) writing account which has INSERT, UPDATE and DELETE GRANT permissions, and b) reading account which has not.
There is SELECT ... FOR UPDATE, which will lock the rows for other callers that do SELECT ... FOR UPDATE, but will not lock it for anyone doing just SELECT. UPDATEs will wait for the lock, as well.
This is useful when you want to fetch a value and then push an update back without anyone changing the value and you not noticing. Be careful, adding too much of those will get you into a deadlock.
You may find that the InnoDB engine does what you need by default: writes do not block reads. You need to be careful with the transaction isolation level so that writes are available when you want them.
Related
I use InnoDB engine for all my tables. I know that by default INSERT creates lock for rows that will be inserted, and UPDATE creates lock for rows that it uses (no matter if in set or where clausules). SELECT doesn't lock anything. And nothing locks whole tables.
But what if I did something like that:
SELECT * FROM table INTO OUTFILE '/tmp/file.txt'
If it would last 5min, anything could happen in some other thread. I've read I could use:
SELECT * FROM table INTO OUTFILE '/tmp/file.txt' LOCK IN SHARE MODE;
But then again I couldn't do any SELECT operations on this table, and it sucks.
What's the best approach to do this? Also, I've read that the last query should be used inside a transaction with a rollback instead of a commit - why is that so?
If you want a consistent view of an InnoDB table for a long running SELECT, the best approach is to just ensure that the transaction isolation level for the session is set to REPEATABLE READ when the SELECT is run.
That won't block other threads that attempt to read the same rows. But it might block some threads from obtaining exclusive locks or write intent locks.
https://dev.mysql.com/doc/refman/5.6/en/set-transaction.html
As an addendum, to clarify some of the points OP raises.
"SELECT doesn't lock anything."
It's true that a non-locking SELECT won't obtain row locks. But some special SELECT statements (as pointed out later) that can obtain row locks:
SELECT ... FOR UPDATE
SELECT ... LOCK IN SHARE MODE
And there are meta-data locks, which will block DDL operations on the table (e.g. ALTER TABLE) while the SELECT statement is executing.
"And nothing locks whole tables."
That's not strictly true. A LOCK TABLE statement can obtain a lock on the entire table. And a SELECT ... FOR UPDATE (with no predicates) could (potentially) obtain locks on every row in the table.
"SELECT ... LOCK IN SHARE MODE will block other SELECT statements"
This isn't true. Shared locks will block exclusive locks from other threads. But they won't block other threads from obtaining share locks, and won't block non-locking SELECT statements.
What's the best approach to do this?
To re-iterate the first part of my answer again... just run a non-locking SELECT statement. As long as the transaction isolation level isn't set to READ UNCOMMITED, the SELECT statement will get a consistent view of the rows in the table, from the timepoint at the time the SELECT begins executing.
Also, I've read that the last query should be used inside a transaction with a rollback instead of a commit - why is that so?
This is a curious notion. It has me puzzled. Why would a ROLLBACK preferred over a COMMIT?
As long as no DML changes have been applied, I think the COMMIT and the ROLLBACK would be equivalent. In both cases, all of the locks obtained by the transaction would be released. In terms of the database, I don't think it makes a difference.
This has me thinking this recommendation comes from a preferred pattern on the client side. Maybe there's a notion of following a rule such as "don't commit unless you've applied DML changes". But that's just a guess.
My personal recommendation would be to follow the normative pattern of using a COMMIT to end the transaction. I don't favor using an implicit ROLLBACK. In my personal opinion, a ROLLBACK should be issued when we want to explicitly discard DML changes that have been applied in a transaction. And that's typically due to an exception or error condition.
I am running these queries on MySQL 5.6.13.
I using repeatable read isolation level. The table looks like below:
In Session A terminal I have issued below statement
UPDATE manufacurer
SET lead_time = 2
WHERE mname = 'Hayleys';
In Session B terminal I tried to update the value lead_time of ACL Cables to 2. But since the previous UPDATE command from Session A is not yet committed (and Session A has an exclusive lock on manufacturer table), this update waits. This I can understand.
But when I try to execute a SELECT statement on Session B as below,
SELECT * FROM manufacturer
WHERE mcode = 'ACL';
it correctly query the manufacturer table and give out the row. How can this happen? Because Session A still hold the exclusive lock on manufacturer table and as I understand when an exclusive lock is held on a table no other transactions can read from or write to it till the previous transaction is committed.
Found below information on this page
http://dev.mysql.com/doc/refman/5.0/en/set-transaction.html#isolevel_repeatable-read
Scope of Transaction Characteristics
You can set transaction characteristics globally, for the current
session, or for the next transaction:
With the GLOBAL keyword, the statement applies globally for all
subsequent sessions. Existing sessions are unaffected.
With the SESSION keyword, the statement applies to all subsequent
transactions performed within the current session.
Without any SESSION or GLOBAL keyword, the statement applies to the
next (not started) transaction performed within the current session.
Have this been taken into consideration?
REPEATABLE READ
This is the default isolation level for InnoDB. For consistent reads,
there is an important difference from the READ COMMITTED isolation
level: All consistent reads within the same transaction read the
snapshot established by the first read. This convention means that if
you issue several plain (nonlocking) SELECT statements within the same
transaction, these SELECT statements are consistent also with respect
to each other.
In this article its decribes very well.
http://www.mysqlperformanceblog.com/2012/08/28/differences-between-read-committed-and-repeatable-read-transaction-isolation-levels/
It is important to remember that InnoDB actually locks index entries,
not rows. During the execution of a statement InnoDB must lock every
entry in the index that it traverses to find the rows it is modifying.
It must do this to prevent deadlocks and maintain the isolation level.
Are the tables well indexed? Can you run a SHOW ENGINE innodb STATUS to confirm that the lock is held?
There are kinds of lock in mysql: row-level lock and table-level lock.
What you need is row-level lock,which allows read the lines beyond the ones updating.
And to implement the row-level lock,you have to define the engine type of your table to 'InnoDB':
alter table TABLE_NAME engine=innodb;
In my application, I want any insert to the database to be executed as soon as a request comes for writing some data.
I am using InnoDB engine.
Since insertion requires an exclusive lock, it is possible that while current read query has a shared lock, some other reads might come which again have a shared lock and the write operation might have to wait for a long time.
I want that when there is a write operation in queue, no read operation gets a shared lock. As soon as the reads which were initiated before the current write request are completed, the write operation should be executed. After that all other read operations should take place.
How can this be implemented?
Edit
Since I am using InnoDB tables and I am not implementing table lock, there should not be a conflict between select and insert. It would be select and update. (Please correct me if there can be a conflict between select and insert as well)
In MySQL, update has higher priority than select. But if there are some read queries being executed, then update query comes in followed by some read queries. In that case, will the read queries coming after update wait for the update to finish as mentioned here http://dev.mysql.com/doc/refman/5.0/en//table-locking.html OR they will get shared lock along with the read queries which were there before the update query was fired?
You don't need to acquire shared locks when reading from the database. In fact, with the default transaction isolation level REPEATABLE READ ordinary SELECT queries within one transaction read from a consistent snapshot. No locks are acquired and required, within this transaction you will simply not see any changes committed in other transactions.
Since no shared locks are acquired, exclusive locks for updating queries are immediately granted to other sessions in the order they are filed.
MySQL doc says the following
Consistent read is the default mode in which InnoDB processes SELECT statements in READ COMMITTED and REPEATABLE READ isolation levels. A consistent read does not set any locks on the tables it accesses, and therefore other sessions are free to modify those tables at the same time a consistent read is being performed on the table.
Suppose that you are running in the default REPEATABLE READ isolation level. When you issue a consistent read (that is, an ordinary SELECT statement), InnoDB gives your transaction a timepoint according to which your query sees the database. If another transaction deletes a row and commits after your timepoint was assigned, you do not see the row as having been deleted. Inserts and updates are treated similarly.
You can try looking at INSERT DELAYED Mysql Insert Delayed
Unfortunatelly it is not available on innodb.
PS: A exclusive lock cannot be acquired if there is already a shared lock on the table. So basicly, in your situation: 3 reads obtain a shared lock, one insert needs exclusive lock. The insert will only be able to obtain the lock after the selects have finished.
Suppose I do (note: the syntax below is probably not correct, but don't worry about it...it's just there to make a point)
Start Transaction
INSERT INTO table (id, data) VALUES (100,20), (100,30);
SELECT * FROM table WHERE id = 100;
End Transaction
Hence the goal of the select is to get ALL info from the table that just got inserted by the preceding insert and ONLY by the preceding INSERT...
Now suppose that during the execution, after the INSERT got executed, some other user also performs an INSERT with id = 100...
Will the SELECT statement in the next step of the transaction also get the row inserted by the executed INSERT by the other user or will it just get the two rows inserted by the preceding INSERT within the transaction?
Btw, I'm using MySQL so please tailor your answer to MySQL
This depends entirely on the Transaction Isolation that is used by the DB Connection.
According to MySQL 5.0 Certification Study Guide
Page 420 describes three transactional conditions handled by Isolation Levels
A dirty read is a read by one transaction of uncommitted changes made by another. Suppose the transaction T1 modifies a row. If transaction T2 reads the row and sees the modification neven though T1 has not committed it, that is a dirty read. One reason this is a problem is that if T1 rollbacks, the change is undone but T2 does not know that.
A non-repeatable read occurs when a transaction performs the same retrieval twice but gets a different result each time. Suppose that T1 reads some rows, and that T2 then changes some of those rows and commits the changes. If T1 sees the changes when it reads the rows again, it gets a different result; the initial read is non-repeatable. This is a problem because T1 does not get a consistent result from the same query.
A phantom is a row that appears where it was not visible before. Suppose that T1 and T2 begin, and T1 reads some rows. If T2 inserts a new and T1 sees that row when it reads again, the row is a phantom.
Page 421 describes the four(4) Transaction Isolation Levels:
READ-UNCOMMITTED : allows a transaction to see uncommitted changes made by other transactions. This isolation level allows dirty reads, non-repeatable reads, and phantoms to occur.
READ-COMMITTED : allows a transaction to see changes made by other transactions only if they've been committed. Uncommitted changes remains invisible. This isolation level allows non-repeatable reads, and phantoms to occur.
REPEATABLE READ (default) : ensure that is a transaction issues the same SELECT twice, it gets the same result both times, regardless of committed or uncommitted changesmade by other transactions. In other words, it gets a consistent result from different executions of the same query. In some database systems, REPEATABLE READ isolation level allows phantoms, such that if another transaction inserts new rows,in the inerbal between the SELECT statements, the second SELECT will see them. This is not true for InnoDB; phantoms do not occur for the REPEATABLE READ level.
SERIALIZABLE : completely isolates the effects of one transaction from others. It is similar to REPEATABLE READ with the additional restriction that rows selected by one transaction cannot be changed by another until the first transaction finishes.
Isolation level can be set for your DB Session globally, within your session, or for a specific transaction:
SET GLOBAL TRANSACTION ISOLATION LEVEL isolation_level;
SET SESSION TRANSACTION ISOLATION LEVEL isolation_level;
SET TRANSACTION ISOLATION LEVEL isolation_level;
where isolation_level is one of the following values:
'READ UNCOMMITTED'
'READ COMMITTED'
'REPEATABLE READ'
'SERIALIZABLE'
In my.cnf you can set the default as well:
[mysqld]
transaction-isolation = READ-COMMITTED
As other user is updating the same row, row level lock will be applied. So he is able to make change only after your transaction ends. So you will be seeing the result set that you inserted. Hope this helps.
Interfere is a fuzzy word when it comes to SQL database transactions. What rows a transaction can see is determined in part by its isolation level.
Hence the goal of the select is to get ALL info from the table that
just got inserted by the preceding insert and ONLY by the preceding
INSERT...
Preceding insert is a little fuzzy, too.
You probably ought to COMMIT the insert in question before you try to read it. Otherwise, under certain conditions not under your control, that transaction could be rolled back, and the row with id=100 might not actually exist.
Of course, after it's committed, other transactions are free to change the value of "id", of "value", or both. (If they have sufficient permissions, that is.)
The transaction will make it seem like that the statements in the transaction run without any interference from other transactions. Most DBMSs (including MySQL) maintain ACID properties for transactions. In your case, you are interested in the A for Atomic, which means that the DBMS will make it seem like all the statements in your transactions run atomically without interruption.
The only users that get effect is those that require access to the same rows in a table. Otherwise the user will not be affected.
However is is slightly more complicated as the row locking can be a read lock or a write lock.
Here is an explanation for the InnoDB storage engine.
For efficiency reasons, developers do not set transactions to totally isolated for each other.
Databases support multiples isolation levels namely Serializable, Repeatable reads, Read committed and Read uncommitted. They are list from the most strict to least strict.
I am a developer and have only fair knowledge about databases. I need to understand the transaction level locking mechanism in InnoDB.
I read that InnoDB uses row level locking? As far as I understand, it locks down a particular row within a transaction. What will happen to a select statement when a table update is going on ?
For Example, assume there is transaction and a select statement both triggered from two different processes and assume Transaction1 starts before the select statement is issued.
Transaction1 : Start
Update table_x set x = y where 1=1
Transaction1 : End
Select Query
Select x from table_x
What will happen to the select statement. Will it return values "during" Transaction1 takes place or "after" it completes? And if it can begin only after Transaction1 ends, where is Row level locking in this picture?
Am I making sense or my fundamental understanding itself is wrong? Please advise.
It depends on the Isolation level.
SERIALIZABLE
REPEATABLE READS
READ COMMITTED
READ UNCOMMITTED
Good explained on wikipedia
And the mySQL docu
It does not depend only on the locking involved, but on the isolation level, which uses locking to provide the transaction isolation as defined by ACID standards. InnoDB uses not only locking, but also multiversioning of the rows to speed up transactions.
In serializable isolation level it would use read-lock with the update, so the select will have to wait for first transaction to be completed. On lower isolation levels however the lock will be write, and selects won't be blocked. In repeatable read and read committed it will scan the rollback log to get the previous value of the record, if it is updated, and in read uncommitted in will return the current value.
The difference between table-level locking and row-level locking is when you have 2 transactions that run update query. In table-level locking, the 2nd will have to wait the first one, as the whole table is locked. In row-level locking, only the rows that match the where clause* (as well as some gaps between them, but this is another topic) will be locked, which means that different transactions can update different parts of the table without need to wait for each other.
*assuming that there is index covering the where clause
The select will not wait for the transaction to complete, instead it will return the current value of the rows (aka, before the transaction started).
If you want the select to wait for the transaction to finish you can use "LOCK IN SHARE MODE":
Select x from table_x LOCK IN SHARE MODE;
This will cause the select to wait for any row(s) that are currently lock by a transaction holding an exclusive (update/delete) lock on them.
A read performed with LOCK IN SHARE MODE reads the latest available
data and sets a shared mode lock on the rows read. A shared mode lock
prevents others from updating or deleting the row read. Also, if the
latest data belongs to a yet uncommitted transaction of another
session, we wait until that transaction ends.
http://dev.mysql.com/doc/refman/5.0/en/innodb-lock-modes.html
SELECT started from outside of a transaction will see the table as it was before transaction started. It will see updated values only after transsaction is commited.