I have a MYISAM table on which long-running SELECT statements are conducted. Do these statements cause locks on the tables that are being read? If so, how do you stop the locks from occurring?
According to https://dev.mysql.com/doc/refman/5.7/en/internal-locking.html:
MySQL uses table-level locking for MyISAM, MEMORY, and MERGE tables,
permitting only one session to update those tables at a time. This
locking level makes these storage engines more suitable for read-only,
read-mostly, or single-user applications.
Does this also mean that SELECT statements create locks?
Do these [long-running SELECT] statements cause locks on the tables that are being read?
Yes. A MyISAM table cannot be updated while it is being read from. If there is a long-running read (i.e, SELECT) query on the table, any UPDATE or DELETE queries will have to wait until the read query is complete.
(INSERT queries are sometimes an exception. So long as there are no internal holes in the table structure, a writer may be allowed to insert a row at the end of a table without obtaining a lock. However, if any rows are ever updated or deleted, this may create holes, which will require INSERT queries to obtain a write lock as well.)
For more information on the internals, read up on table-level locks. But the short and the long of matters is, you probably shouldn't use MyISAM if concurrency is a concern.
Related
My question is a follow up to this answer. I want to find out how to perform a select statement without locking a table with MyISAM engine.
The answer states the following if you have InnoDB but not MyISAM. What is the equivalent for MyISAM engine?
SET TRANSACTION ISOLATION LEVEL READ UNCOMMITTED ;
SELECT * FROM TABLE_NAME ;
COMMIT ;
This is the default behaviour with MyISAM tables. If one actually wants to lock a MyISAM table, one must manually acquire a table-level lock. Transaction isolation level, START TRANSACTION, COMMIT, ROLLBACK have no effect on MyISAM tables behaviour since MyISAM does not support transactions.
More about internal locking mechanisms
A READ lock is implicitely acquired before, and released after execution of a SELECT statement. Notice that several concurrent, simultaneous, SELECT statements could be running at the same time, because several sessions may hold a READ lock on the same table.
Conversely, a WRITE lock is implicitely acquired before executing an INSERT or UPDATE or DELETE statement. This means that no read (let alone a concurrent write) can take place as long as a write is in progress*.
The above applies to MyISAM, MEMORY, and MERGE tables only.
You might want to read more about this here:
Internal locking methods
Read vs Write locks
* However, these locks are not always required thanks to this clever trick:
The MyISAM storage engine supports concurrent inserts to reduce contention between readers and writers for a given table: If a MyISAM table has no free blocks in the middle of the data file, rows are always inserted at the end of the data file. In this case, you can freely mix concurrent INSERT and SELECT statements for a MyISAM table without locks.
MyISAM does indeed use a read lock during SELECT. An INSERT at the end of the table can get around that.
But try doing an UPDATE, DELETE, or ALTER TABLE while a long-running SELECT is in progress. Or vice-versa, reading from a table while a change to that table is running. It's first-come, first-serve, and the later thread blocks until the first thread is done.
MyISAM doesn't have any support for transactions, so it must work this way. If a SELECT were reading rows from a table, and a concurrent thread changes some of those rows, you would get a race condition. The SELECT may read some of the rows before the change, and some of the rows after the change, resulting in a completely mixed-up view of the data.
Anything you do with SET TRANSACTION ISOLATION LEVEL has no effect with MyISAM.
For these reasons, it's recommended to use InnoDB instead.
Does MySQL automatically peform read / write table locks on MyIsam tables, or do I have to explicitly lock the tables?
Beyond making reasonable efforts to make certain statements atomic, MyISAM doesn't have the concept of transactions and its related row level locking.
Therefore, you should use LOCK TABLES to avoid race conditions or data inconsistencies when you use multiple statements (e.g. a SELECT statement followed by multiple related UPDATE statements).
See here about Pros and Cos Of MyISAM
MyISAM's About Internal Locking
MyISAM uses table-level locking. When a row is inserted or updated, all other changes to that table are held up until that request has been completed.
Correct me if 'm wrong
I am using mysql with InnoDB databases.
If all my transactions are Inserts and Selects (no updates), I assume I would not have to worry about SQL deadlocking.
I can't see a scenario where deadlocking would occur. Am I correct to assume deadlocking cannot occur if I only do Inserts and Selects?
May not be relevant but everything transaction is done with PDO
No. You still have to worry about SQL deadlocking.
You can get deadlocks even in the case of a transaction that inserts a single row. This is because the insert operation is not really atomic and locks are set automatically on the (possibly several) index records of the inserted row.
InnoDB MySQL storage engine has row level locks while the MyISAM MySQL storage engine has table level locks. MyISAM simply locks entire tables, and doesn't support transactions, so it's not possible to have database-level deadlocks. Note that an app can lock up another app by sitting on a table lock on the table they are both trying to access, but this is a code error, not a db-level "deadlock".
InnoDB supports transactions and has row-level locks, so db-level deadlocks are possible (and can happen occasionally in a busy system so you do need to code around them). Many of what MySQL will call "deadlocks" aren't "true deadlocks" as much as they're the result of slow UPDATEs causing other queries to time out on row locks.
So after reading Performance in PDO / PHP / MySQL: transaction versus direct execution in regards to performance issues I was thinking about I did some research on locking tables in MySQL.
On http://dev.mysql.com/doc/refman/5.0/en/table-locking.html
Table locking enables many sessions to
read from a table at the same time,
but if a session wants to write to a
table, it must first get exclusive
access. During the update, all other
sessions that want to access this
particular table must wait until the
update is done.
This part struck me particularly because most of our queries will be updates rather than inserts. I was wondering if one created a table called foo on which all updates/inserts were carried out and then a view called foo_view (A copy of foo, or perhaps foo and a linkage of several other tables plus foo) on which all selects occurred, would this locking issue still occur?
That is, would SELECT queries on foo_view still have to wait for an update to finish on foo?
Another brief question my colleague asked. Does this affect caching? I.e. if the SELECT is cached will it hit the cache and return results, or will it wait for the lock to finish first?
Your view will experience the same locking as the underlying tables.
From the MySQL Reference page on locking:
MySQL grants table write locks as
follows:
If there are no locks on the table, put a write lock on it.
Otherwise, put the lock request in the write lock queue.
MySQL grants table read locks as
follows:
If there are no write locks on the table, put a read lock on it.
Otherwise, put the lock request in the read lock queue.
It's worth mentioning that this depends on the database engine you are using. MyISAM will follow the steps above and lock the entire table (even if it is split into multiple partitions) where an engine like InnoDB will do row level locking instead.
If you're not reaching the necessary performance benchmarks with MyISAM and you have shown your bottleneck is waiting on table locks via updates, I would suggest changing the storage engine of your table to InnoDB.
I was wondering if anyone has a suggestion for what kind of storage engine to use. The programs needs to perform a lot of writes to database but very few reads.
[edit] No foreign keys necessary. The data is simple, but it needs to preform the writes very fast.
From jpipes:
MyISAM and Table-Level Locks
Unlike InnoDB, which employs row-level
locking, MyISAM uses a much
coarser-grained locking system to
ensure that data is written to the
data file in a protected manner.
Table-level locking is the only level
of lock for MyISAM, and this has a
couple consequences:
Any connection issuing an UPDATE or DELETE against a MyISAM table will
request an exclusive write lock on the
MyISAM table. If no other locks (read
or write) are currently placed on the
table, the exclusive write lock is
granted and all other connections
issuing requests of any kind (DDL,
SELECT, UPDATE, INSERT, DELETE) must
wait until the thread with the
exclusive write lock updates the
record(s) it needs to and then
releases the write lock.
Since there is only table-level locks, there is no ability (like there
is with InnoDB) to only lock one or a
small set of records, allowing other
threads to SELECT from other parts of
the table data.
The point is, for writing, InnoDB is better as it will lock less of the resource and enable more parallel actions/requests to occur.
"It needs to perform the writes very fast" is a vague requirement. Whatever you do, writes may be delayed by contention in the database. If your application needs to not block when it's writing audit records to the database, you should make the audit writing asynchronous and keep your own queue of audit data on disc or in memory (so you don't block the main worker thread/process)
InnoDB may allow concurrent inserts, but that doesn't mean they won't be blocked by contention for resources or internal locks for things like index pages.
MyISAM allows one inserter and several readers ("Concurrent inserts") under the following circumstances:
The table has no "holes in it"
There are no threads trying to do an UPDATE or DELETE
If you have an append-only table, which you recreate each day (or create a new partition every day if you use 5.1 partitioning), you may get away with this.
MyISAM concurrent inserts are mostly very good, IF you can use them.
When writing audit records, do several at a time if possible - this applies whichever storage engine you use. It is a good idea for the audit process to "batch up" records and do an insert of several at once.
You've not really given us enough information to make a considered suggestion - are you wanting to use foreign keys? Row-level locking? Page-level locking? Transactions?
As a general rule, if you want to use transactions, InnoDB/BerkeleyDB. If you don't, MyISAM.
In my experience, MyISAM is great for fast writes as long as, after insertion, it's read-only. It'll keep happily appending faster than any other option I'm familiar with (including supporting indexes).
But as soon as you start deleting records or updating index keys, and it needs to refill emptied holes (in tables or indexes) the discussion gets a lot more complicated.
For classic log-type or journal-type tables, though, it's very happy.