This is an extremely simple question, but i really can't find any useful answer on Google.
I've been reading that there's a READ LOCK on SELECT, and WRITE LOCK on UPDATE statements. However, i'm trying to find out if there's any kind of LOCK when we INSERT into a table?
The assumption is that the table is using InnoDB as the engine.
When inserting, InnoDB creates a so called "gap lock".
The manual describes this quite well:
A type of gap lock called an insertion intention gap lock is set by INSERT operations prior to row insertion. This lock signals the intent to insert in such a way that multiple transactions inserting into the same index gap need not wait for each other if they are not inserting at the same position within the gap. Suppose that there are index records with values of 4 and 7. Separate transactions that attempt to insert values of 5 and 6 each lock the gap between 4 and 7 with insert intention locks prior to obtaining the exclusive lock on the inserted row, but do not block each other because the rows are nonconflicting
In addition to that, any unique index will be locked for the provided values to make sure that two different transactions don't insert the same value (that's slightly different than the gap lock if I'm not mistaken).
For innoDB mysql uses row level locking.
When inserting there is no row to lock because You are creating it.
Adding to the gap lock description, Mysql INNODB does a ROW-LOCK on the row that is being inserted.
Related
As far as I know the gap lock is used to prevent phantom read, and I found gap lock is set by locking read in most articles via Google search.
A gap lock is a lock on a gap between index records, or a lock on the gap before the first or after the last index record. For example, SELECT c1 FROM t WHERE c1 BETWEEN 10 and 20 FOR UPDATE; prevents other transactions from inserting a value of 15 into column t.c1, whether or not there was already any such value in the column, because the gaps between all existing values in the range are locked.
https://dev.mysql.com/doc/refman/8.0/en/innodb-locking.html#innodb-gap-locks
I guess this (set gap lock on locking read) is sufficient. Why update, delete also set gap lock.
UPDATE ... WHERE ... sets an exclusive next-key lock on every record the search encounters. However, only an index record lock is required for statements that lock rows using a unique index to search for a unique row.
https://dev.mysql.com/doc/refman/8.0/en/innodb-locks-set.html
And another issue is what happened if there is no suitable index where gap lock can be attached?
Does fall back to lock on the entire table?
Here we assumed that using the default transaction isolation level Repeatable Read.
It depends on the conditions in your SELECT, UPDATE, or DELETE. They set gap locks to prevent other concurrent sessions from adding rows to the set that would be matched by the conditions.
In InnoDB, locking statements always lock the most recent committed row versions. So they don't really obey the REPEATABLE READ snapshot. They act more like READ-COMMITTED.
Therefore, if you do a statement like this:
UPDATE FROM MyTable SET ... WHERE created_at > '2020-03-22';
It must lock the gap following the highest value of created_at, which will prevent other sessions from adding new rows.
This is to simulate REPEATABLE READ, to make sure that if you run the same UPDATE again, it will affect the same rows, and it won't accidentally affect new rows.
I have a PHP DAEMON on my Ubuntu server doing huge data inserts into InnoDB. The very same tables are also being used by people using the platform.
The DAEMON when not running in TRANSACTION mode uses about 60-70 secs for 100.000 inserts. When running in TRANSACTION mode, BEGIN .... COMMIT it uses 15-20 seconds.
However will TRANSACTION mode lock the tables, and prevent users using the platform to do inserts while the DAEMON TRANSACTION is beeing preformed?
Locking the tables the users are manipulating for over 20 seconds is, of course, not desirable :)
Well I'm doing inserts in batches of 500 and 500 insie a FOR loop INSERT INTO (col1, col2) VALUES (a,b) etc. This is fine, and runs smooth, however I'm able to speed up the process significantly if i issue a BEGIN before the loop, and COMMIT after to loop, but this means the time between the BEGIN/COMMIT is over 60 seconds. But while the system is doing a few hundred thousand inserts, people using the platform can do inserts to the very same table. Will the system generated Inserts account for the user insets, or will the users have to wait XX seconds before their insert is processed?
Based on your description, you use innodb with the default autocommit mode enabled and you insert records one by one in a loop. Autocommit mode means that each insert is encapsulated into its own transaction, which is fine, but very slow, since each record is persisted separately into the disk.
If you wrap your loop that inserts the records within begin - commit statements, all inserts are run within a single transaction and are persisted to the disk only once, when the commit is issued - this is why you experience the speed gain.
Regardless of which way you insert the records, innodb will use locks. However, innodb only locks the record being inserted:
INSERT sets an exclusive lock on the inserted row. This lock is an
index-record lock, not a next-key lock (that is, there is no gap lock)
and does not prevent other sessions from inserting into the gap before
the inserted row.
Prior to inserting the row, a type of gap lock called an insert
intention gap lock is set. This lock signals the intent to insert in
such a way that multiple transactions inserting into the same index
gap need not wait for each other if they are not inserting at the same
position within the gap. Suppose that there are index records with
values of 4 and 7. Separate transactions that attempt to insert values
of 5 and 6 each lock the gap between 4 and 7 with insert intention
locks prior to obtaining the exclusive lock on the inserted row, but
do not block each other because the rows are nonconflicting.
This means, that having a transaction open for a longer period of time that only inserts records will not interfere with other users inserting records into the same table.
Pls note, that issuing single insert statements in a loop is the least efficient way of inserting larger amount of data into MySQL.
Either use bulk insert (build a single insert statement in the loop and execute it after the loop, paying attention to max_allowed_packet setting :
INSERT statements that use VALUES syntax can insert multiple rows. To
do this, include multiple lists of column values, each enclosed within
parentheses and separated by commas. Example:
INSERT INTO tbl_name (a,b,c) VALUES(1,2,3),(4,5,6),(7,8,9);
Or use load data infile statement.
These two solutions can significantly speed up the data insertion and will not cause table lock either.
Plan A: LOAD DATA. Drawback: This requires writing the data to a file. If it is already in a file, then this is the best approach.
Plan B: "Batched INSERTs" -- Build INSERT INTO t (a,b) VALUES (1,2), (3,4), ... and execute them. Do it in batches of 100-1000. This will be even faster than BEGIN..COMMIT around lots of 1-row INSERTs. Have autocommit=ON. Locking/blocking will be minimal since each 'transaction' will be only 100-1000 row's worth.
Let's see SHOW CREATE TABLE. INDEXes, especially UNIQUE indexes have an impact on the performance. We can advise further.
If this is a "Data Warehouse" application, then we should talk about "Summary Tables". These would lighten the load crated the 'readers' significantly and cut back on the need for indexes on the Fact table and prevent locking/blocking because they would be reading a different table.
Also, UUIDs are terrible for performance.
How big is the table? How much RAM do you have? What is the value of innodb_buffer_pool_size?
I have an issue with table locking in InnoDB on delete operation.
I have a table queue with for example one column and a lot of transactions which can insert rows into this queue or delete them.
There isn't any two transactions working with the same rows at the same time. So, all row locks must be distinct.
But sometimes when delete operation deletes the most part of rows in the table, InnoDB prefers to use table lock instead of row lock and that causes deadlocks.
I can't reproduce this deadlock exactly, but I found that lock problem.
i.e. I have table queue:id with values(1,3,4,5,6,7)
Transaction 1:
insert into queue value(2);
Transaction 2:
delete from queue where id in (1,3,4,5,6,7); -- here the lock comes
First of all assuming id is a primary key or at least indexed column.
Insert should not lock the table, so chances are any other update/delete query is executing at same time of deletion the records.
If it is not the case then it can be due to "gap locking" as mentioned #a_horse_with_no_name.
So at which time you get this issue again then you need to store all processes "show full processlist" at your end and also check "show engine innodb status" where it will show you processids related with deadlock, this will help you to get exact problem.
Further You can avoid this locking to delete all rows one by one based on primary key.
I would like to be able to lock a table to prevent other users doing Inserts. I don't want to lock the whole table, because this would prevent other users from updating rows. I do have another reasonably elegant solution, however if I could lock the table solely to prevent another user inserting rows, that would be a better solution. IE. Any user before attempting an INSERT would attempt to acquire this lock, and wait if already in use.
I'm not exactly sure why your trying to do this but I believe you can accomplish what your doing by locking on a dummy table.
That is for all inserts you would not actually lock on the table that you want to insert but on a different table that you use only for locks:
BEGIN WORK;
LOCK TABLE insert_locks IN EXCLUSIVE;
INSERT INTO real_table VALUES
(_id_, 'GREAT! I was waiting for it for so long!');
COMMIT WORK;
See Postgres' doc on LOCK.
Unfortunately you will have to go change any code that is doing inserts with out locking. The other option is to use some sort of Message Queue which I have done many times with great success.
I'm seeing a pending insert lock up a MyISAM table. The query itself isn't a standout, but in the FULL PROCESSLIST it looks like there's an INSERT with a lock and a bunch of SELECTs waiting on it.
The query itself isn't a standout. Reading the MySQL docs, I see something that is a standout: "If there are holes, concurrent inserts are disabled."
I presume this means holes in the index, and in my index here there is definitely "holes" in the sense that there are not consecutive ID numbers in the primary key. So I'm wondering: if concurrency is disabled and some INSERT hangs, how long is it going to hold that lock and will it hold that lock on the entire table (and not just the row)?
MyISAM does not support row level locking, so the lock is on entire table. I didn't find any information about lock timeout, it seems that insert can theoretically hang forever.