When I add a row that references another table (in a transaction), MySQL seems to lock the whole row that's being referenced. This prevents updates of other columns in the destination table that should be able to run concurrently without any problem.
Simplified example:
CREATE TABLE `t1` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`num` int(11) DEFAULT NULL,
UNIQUE KEY `id` (`id`)
);
CREATE TABLE `bar` (
`foo` int(11) NOT NULL,
KEY `foo` (`foo`),
CONSTRAINT `bar_ibfk_1` FOREIGN KEY (`foo`) REFERENCES `t1` (`id`)
);
INSERT INTO `t1` VALUES (1,1),(2,0),(3,4);
Task A:
BEGIN;
insert into bar(foo) values(2);
-- now we ask task B to do some work for us
Task B:
-- when triggered by Task A, tries to do this:
update t1 set num=num+1 where id=2;
-- does not complete because it waits for the lock
Any ideas how to avoid this deadlock? Task A should only read-lock the single value which it actually refers to, so Task B couldn't renumber or delete t1[id=2].id but would otherwise be free to update that row. Is it possible to convince MySQL to do this?
Splitting t1 into two linked tables (one for Task A to refer to and one for task B to update) would result in a heap of fairly intrusive refactoring.
Joining the tasks is not an option because B's work changes global state, thus must be able to commit even if A fails.
Switching to Postgres (which supports this; I checked) is not an easily-executed option.
This is a behavior of MySQL foreign keys that frankly convinces many projects to avoid using foreign key constraints, even though their database logically has foreign key references.
You can't lock just one column of a row. InnoDB effectively locks the whole row against update or delete if an exclusive lock exists on a child row that references it. The idea is that while a child row is depending on that parent row and is in progress of an insert/update/delete, the parent row shouldn't be deleted or its key modified. But you can't lock only the key column that is referenced by the child row.
The best solution is for the transaction against the child table to be finished and committed promptly. The fact that you tried to update the parent row and it timed out (a lock wait timeout is 50 seconds by default) indicates that you have left the transaction running too long.
P.S. What you described is simply a lock-wait. That's not a deadlock. A deadlock is when both transactions end up blocked, waiting for each other to release locks but neither can proceed because they are both waiting. A lock-wait is unidirectional. A deadlock is a cycle of mutual lock-waits.
Related
I have the following table.
CREATE TABLE `FooBar` (
`id` int(10) AUTO_INCREMENT,
`foobar_id` int(10),
PRIMARY KEY (`id`),
UNIQUE KEY `foobar_id` (`foobar_id`)
) ENGINE=InnoDB;
Based on my testing, whenever foobar_id is updated, any other update involving foobar_id will be locked until the first one ends.
What I want ideally is when someone is updating a not colliding value the table won't be locked
begin;
update `FooBar` set foobar_id=2 where id=1;
commit;
begin;
update `FooBar` set foobar_id=3 where id=2;
commit;
These two queries obviously will not collide with each other and the second one ideally should go through before the first one commit if the first update is not setting it to 3. However my testing result shows that the second query won't go through until the first commits.
When the key is not unique this won't happen.
Is there a way to get around the table locking?
I'm getting locks in update operations that doesn't seem to be related to each other.
This is the DB Context:
MySQL 5.7
InnoDB engine
Read Committed Isolation Level
Optimistic Locking concurrency control in the application
The table structure is something like this:
CREATE TABLE `external_user` (
`id` bigint(20) NOT NULL AUTO_INCREMENT,
`user_id` bigint(20) NOT NULL,
`status` varchar(30) NOT NULL,
PRIMARY KEY (`id`),
KEY `idx_user_status` (`status`),
KEY `idx_user_id` (`user_id`) USING BTREE,
);
The structure's been simplified. The real one has more attributes and some FKs to other tables.
The process is something like this:
Process 1
BEGIN;
update external_user
set user_id=33333
where (id in (400000, 400002, 400028............., 420000))
and user_id = 22222;
This is a long running query that modifies around 20k rows. Using between is not an option because we don't update all the consecutive records.
At the same time a second process starts.
Process 2
BEGIN;
update external_user
set status='disabled', user_id = 44444
where id = 10000;
It turns out that this second update is waiting for the first one to complete. So there's a lock held in the first query.
I've been reading a lot about locking in MySQL, but I couldn't find anything about updates that in where clause have a PK filter with in operator and another filter by an attribute that has an non-unique index (that is also being changed in the set clause).
Is the first query obtaining a gap lock because of the non-unique index filter? Is it possible? Even though the PK is provided as a filter?
Note: I don't have access to the engine in order to obtain more detailed information.
I never used FOREIGN KEYS before. I always placed the id it a key column but it is not necessary a foreign key. so it acts like a foreign key but it is not.
so if I have the following 2 tables
CREATE TABLE `accounts` (
`account_id` int(11) unsigned NOT NULL AUTO_INCREMENT,
`name` varchar(60) NOT NULL,
`owner_id` int(11) unsigned NOT NULL,
PRIMARY KEY (`account_id`),
KEY `owner_id` (`owner_id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8
CREATE TABLE `users` (
`user_id` int(11) unsigned NOT NULL AUTO_INCREMENT,
`name` varchar(60) NOT NULL
PRIMARY KEY (`user_id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8
the column account.owner_id is linked to users.user_id but without setting a foreign key relationship.
so I can do something like this
SELECT
a.name AS account_name,
u.name AS user_name FROM
accounts AS a INNER JOIN users AS u ON u.user_id = a.owner_id
So after learning what FOREIGN KEYS are and what they do I have created a foreign key like so.
ALTER TABLE accounts
ADD CONSTRAINT fk_owner_id FOREIGN KEY(owner_id)
REFERENCES users(user_id) ON DELETE NO ACTION ON UPDATE CASCADE;
However, I notices a performance change as they system slowed down a lot. I am not sure if adding FOREIGN KEYs will reuse performance or no? or is there a performance reduction when adding a foreign keys to a table?
Please note that that there are lots of tables and columns in my database. so I do many INNER/LEFT/RIGHT JOINS and all the columns where i identified them as FOREIGN KEYS are indexed. Notices that I did not add any indexes to this columns as all of they already exists prior adding the FOREIGN keys to my database.
my question, will FOREIGN KEYs reduce performance on UPDATE/INSERT/DELETE/SELECT?
Also is there a benefit of adding a FOREIGN KEY constraint when we specified ON DELETE NO ACTION ON UPDATE NO ACTION?
Thanks
The foreign key constraint means that any insert to the FK column has to check if the value exists in the referenced column of users. There could be some overhead to this, but it's an index lookup by definition (probably a PK lookup) so the cost shouldn't be high.
Foreign keys also create a shared lock on the parent table during some updates on the child table. This can get in the way of concurrent updates against that table, and make it seem like the system has slower performance. See http://www.mysqlperformanceblog.com/2006/12/12/innodb-locking-and-foreign-keys/
The foreign key also implicitly created an index on the FK column, if no index already exists. Every insert, update, delete has to modify all the indexes of a table at the time of the change, so there is a bit of overhead. For this reason, some people say indexes "hurt" performance of insert, update, delete. But it's not that simple -- an index that supports conditions in a WHERE clause can make an update or delete run faster by finding the affected rows more efficiently.
Re your comment:
Yes, any update to the child tables creates an exclusive lock on the modified row in the child table. You probably expected this. But this action also creates a shared lock on the referenced row in clients. Any number of sessions may independently create shared locks on the same row (hence the name shared). But an exclusive lock requires that there be no lock of any type. So if there are frequently shared locks outstanding on a row in clients, then direct updates to that row in clients can't get the required exclusive lock.
The purpose of these shared locks is basically so that the row in clients doesn't get removed or modified while someone is updating a row that depends on it. In other words, "don't delete my parent." But depending on the frequency of updates and the duration of transactions, it could make it hard to perform updates to a parent row.
One way to mitigate this is to try to make locks live for shorter periods of time, by finishing the work for a transactions promptly, and then commit.
I'm converting an existing database from MyISAM to InnoDB and implementing various foreign keys, I'm having an issue with running the convert script on my database though:-
I'm running all queries as below
DELETE FROM example WHERE user NOT IN (select id FROM users);
ALTER TABLE `example` CHANGE `user` `user` INT( 11 ) UNSIGNED NOT NULL ;
ALTER TABLE example ADD FOREIGN KEY (user) REFERENCES users(ID);
ALTER TABLE example ADD FOREIGN KEY (car) REFERENCES cars(ID);
When I run all queries it fails due to a foreign key constraint, due to the fact the DELETE statement hasn't run - if I run them individually, it's fine - is it an issue with commit on the innodb database or is it due to speed of the delete not completing before the next query?
Is it also ok to have two foreignkeys of ID? (two different tables users.id and cars.id).
Thanks!
No idea of what the error message might say or what you're trying to accomplish but ALTER TABLE is a DDL statement and those cannot be rollbacked in MySQL. The Statements That Cause an Implicit Commi manual chapter explains:
The statements listed in this section (and any synonyms for them)
implicitly end any transaction active in the current session, as if you had done a COMMIT before executing the statement. As of MySQL
5.5.3, most of these statements also cause an implicit commit after executing;
[...]
Data definition language (DDL) statements that define or modify database objects
[...]
ALTER TABLE, CREATE TABLE, and DROP TABLE do not commit a transaction
if the TEMPORARY keyword is used. (This does not apply to other
operations on temporary tables such as CREATE INDEX, which do cause a
commit.) However, although no implicit commit occurs, neither can the
statement be rolled back. Therefore, use of such statements will
violate transaction atomicity: For example, if you use CREATE
TEMPORARY TABLE and then roll back the transaction, the table remains
in existence.
I have a table
CREATE TABLE `uli` (
`id` int(10) NOT NULL,
PRIMARY KEY (`id`)
) ENGINE=MyISAM DEFAULT CHARSET=utf8
The table shall always contain one value in one row. But it could be changed often. I would like to make a select, lock the table for other connections, update a value and the unlock.
For example, the following code works excellent:
LOCK TABLES uli WRITE;
SELECT * FROM uli;
UPDATE uli SET id=id+1;
UNLOCK TABLES;
While first connection do not unlock, all other connection will wait, and only after unlock could see new value. That is exactly what I want. Is it exists some more elegant solution? Does it matter MyIsam or Innodb table to use?
You are locking for write, which means selects will still see the old value before the update. Also you are not doing anything with your select. If you use InnoDB and SERIALIZABLE isolation level, all you need is the update statement.