I am reading a sql.txt file and this is one section. What it does is create a table "Page":
DROP TABLE IF EXISTS `Page`;
CREATE TABLE `Page` (
`id` bigint(20) NOT NULL auto_increment,
`pageId` int(11) default NULL,
`name` varchar(255) default NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `pageId` (`pageId`)
)
--
-- Dumping data for table `Page`
--
LOCK TABLES `Page` WRITE;
/*!40000 ALTER TABLE `Page` DISABLE KEYS */;
/*!40000 ALTER TABLE `Page` ENABLE KEYS */;
UNLOCK TABLES;
I have a few questions:
Is the TABLES a database name?
Does the WRITE load data to the table 'page'? If yes, where is the source of the data?
Does the lock mean first locking the database?
TABLES is not the database name. It means which table(s) to lock (in this case table 'Page').
WRITE means, the table is locked for writing into it.
It means locking the table, not the database.
LOCK TABLES is a SQL command (see link for the documentation) that will apply a lock to the set of tables specified. In this case, it is only one table, Page:
LOCK TABLES `Page` WRITE;
In this case it would be two tables:
LOCK TABLES `Page`, `Book` WRITE;
The lock can have a 'type', which in this case is WRITE. That means no data can be written to the table while this lock is in place. This ensures that no updates overwrite what is being written before everything has been completed. This keeps the database table in a known state.
Note that you can still read from a table under a write lock. This means that reads will be nondeterministic as the data is being written.
SQL doesn't support locking the entire database (as far as I know). The above command will only lock the specified tables.
Related
When I add a row that references another table (in a transaction), MySQL seems to lock the whole row that's being referenced. This prevents updates of other columns in the destination table that should be able to run concurrently without any problem.
Simplified example:
CREATE TABLE `t1` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`num` int(11) DEFAULT NULL,
UNIQUE KEY `id` (`id`)
);
CREATE TABLE `bar` (
`foo` int(11) NOT NULL,
KEY `foo` (`foo`),
CONSTRAINT `bar_ibfk_1` FOREIGN KEY (`foo`) REFERENCES `t1` (`id`)
);
INSERT INTO `t1` VALUES (1,1),(2,0),(3,4);
Task A:
BEGIN;
insert into bar(foo) values(2);
-- now we ask task B to do some work for us
Task B:
-- when triggered by Task A, tries to do this:
update t1 set num=num+1 where id=2;
-- does not complete because it waits for the lock
Any ideas how to avoid this deadlock? Task A should only read-lock the single value which it actually refers to, so Task B couldn't renumber or delete t1[id=2].id but would otherwise be free to update that row. Is it possible to convince MySQL to do this?
Splitting t1 into two linked tables (one for Task A to refer to and one for task B to update) would result in a heap of fairly intrusive refactoring.
Joining the tasks is not an option because B's work changes global state, thus must be able to commit even if A fails.
Switching to Postgres (which supports this; I checked) is not an easily-executed option.
This is a behavior of MySQL foreign keys that frankly convinces many projects to avoid using foreign key constraints, even though their database logically has foreign key references.
You can't lock just one column of a row. InnoDB effectively locks the whole row against update or delete if an exclusive lock exists on a child row that references it. The idea is that while a child row is depending on that parent row and is in progress of an insert/update/delete, the parent row shouldn't be deleted or its key modified. But you can't lock only the key column that is referenced by the child row.
The best solution is for the transaction against the child table to be finished and committed promptly. The fact that you tried to update the parent row and it timed out (a lock wait timeout is 50 seconds by default) indicates that you have left the transaction running too long.
P.S. What you described is simply a lock-wait. That's not a deadlock. A deadlock is when both transactions end up blocked, waiting for each other to release locks but neither can proceed because they are both waiting. A lock-wait is unidirectional. A deadlock is a cycle of mutual lock-waits.
I'm getting locks in update operations that doesn't seem to be related to each other.
This is the DB Context:
MySQL 5.7
InnoDB engine
Read Committed Isolation Level
Optimistic Locking concurrency control in the application
The table structure is something like this:
CREATE TABLE `external_user` (
`id` bigint(20) NOT NULL AUTO_INCREMENT,
`user_id` bigint(20) NOT NULL,
`status` varchar(30) NOT NULL,
PRIMARY KEY (`id`),
KEY `idx_user_status` (`status`),
KEY `idx_user_id` (`user_id`) USING BTREE,
);
The structure's been simplified. The real one has more attributes and some FKs to other tables.
The process is something like this:
Process 1
BEGIN;
update external_user
set user_id=33333
where (id in (400000, 400002, 400028............., 420000))
and user_id = 22222;
This is a long running query that modifies around 20k rows. Using between is not an option because we don't update all the consecutive records.
At the same time a second process starts.
Process 2
BEGIN;
update external_user
set status='disabled', user_id = 44444
where id = 10000;
It turns out that this second update is waiting for the first one to complete. So there's a lock held in the first query.
I've been reading a lot about locking in MySQL, but I couldn't find anything about updates that in where clause have a PK filter with in operator and another filter by an attribute that has an non-unique index (that is also being changed in the set clause).
Is the first query obtaining a gap lock because of the non-unique index filter? Is it possible? Even though the PK is provided as a filter?
Note: I don't have access to the engine in order to obtain more detailed information.
I have exported a database (structure only) from phpMyAdmin which I want to use as the bases for a script that can update a previous version of the same database structure and add columns and tables if they don't exist.
For example, if the CREATE TABLE statement looks like this:
--
-- Table structure for table `users`
--
CREATE TABLE IF NOT EXISTS `users` (
`ID` int(11) NOT NULL AUTO_INCREMENT,
`email` varchar(52) NOT NULL,
`password` varchar(40) NOT NULL,
`dev` tinyint(1) NOT NULL DEFAULT '0',
PRIMARY KEY (`ID`)
) ENGINE=MyISAM DEFAULT CHARSET=latin1 AUTO_INCREMENT=21;
How can I adjust this so if a field in the table does not exist it will be created even if the table already exists in the database.
The aim of this script is to allow updating of already existing databases, adding columns that may be missing, but it should not remove any tables or fields.
According to this, it seems the easiest way to accomplish this is to create a temporary table (by which I don't necessarily mean of type TEMPORARY, just that the use will be short and limited; although in most cases type TEMPORARY may be a reasonable choice here), copy the data from your old table to the temporary one, create the new table with the updated structure, then bring everything back over from the temporary table. Finally, your script may rename the old table as a backup or drop it and the temporary table entirely.
I have a MySQL database with a table that has 2 million rows using innodb engine. I want to add another column, but I keep getting the following error:
Error 1062: Duplicate entry '' for key 'PRIMARY' SQL Statement: ALTER TABLE `mydb`.`table` ADD COLUMN `country` VARCHAR(35) NULL DEFAULT NULL AFTER `email`
How can I add the column without getting this error?
EDIT: Table definition
id int(11) NOT NULL AUTO_INCREMENT,
user_id varchar(45) NOT NULL,
first_name varchar(150) DEFAULT NULL,
last_name varchar(150) DEFAULT NULL,
gender varchar(10) DEFAULT NULL,
email varchar(100) DEFAULT NULL,
created_at bigint(20) DEFAULT NULL,
updated_at bigint(20) DEFAULT NULL,
PRIMARY KEY (`id`,`user_id`),
UNIQUE KEY `user_id_UNIQUE` (`user_id`),
KEY `first_name` (`first_name`),
KEY `last_name` (`last_name`)
EDIT #2: SHOW INDEXES output
Table Non_unique Key_name Seq_in_index Column_name Collation Cardinality Index_type
table 0 PRIMARY 1 id A 3516446 BTREE
table 0 PRIMARY 2 user_id A 3516446 BTREE
table 0 user_id_UNIQUE 1 user_id A 3516446 BTREE
table 1 first_name 1 first_name A 390716 BTREE
table 1 last_name 1 last_name A 439555 BTREE
it solution will lock table on write, but often suitable for solving the problem if table is not very big
LOCK TABLES my_table WRITE;
ALTER TABLE my_table
ADD COLUMN `ts` DATETIME NULL AFTER `id`;
UNLOCK TABLES;
As described in the documentation, When running an online ALTER TABLE operation:
... the thread that runs the
ALTER TABLE operation will apply an “online log” of DML operations
that were run concurrently on the same table from other connection
threads. When the DML operations are applied, it is possible to
encounter a duplicate key entry error (ERROR 1062 (23000): Duplicate
entry), even if the duplicate entry is only temporary and would be
reverted by a later entry in the “online log”. This is similar to the
idea of a foreign key constraint check in InnoDB in which constraints
must hold during a transaction.
If it's not something a program needs to do (altering tables dynamically), then just wait a moment and then try again! It worked for me. :) I guess there are some InnoDB-specific processes/states (maybe still processing another ALTER queried just a moment ago?), during which the ALTER command would fail, and you just need to catch a moment when it succeeds.
I've runned the same code and it works fine.
http://sqlfiddle.com/#!2/1937e
As a solution, I would try to recreate the table , copy the data into it, and then switch tables using rename.
If that doesn't work then it's clear that it's a bug with your current mysql configuration, and we'll need more details to figure it out or at least reproduce it (mysql version, mysql config, database settings, the actual data, etc.).
If it works then it probably was a problem with tables or indexes and here are a few things you can check (you can also start with these if you don't wish to recreate the table):
Check that you don't have any triggers that are causing other inserts
Check that you are just creating a column and not adding other keys/indexes
Check that the auto_increment value is not overflowing (for int it's over 2,000,000,000)
If none of the above, then you probably have a some corrupt data, or you missed to share some details.
I guess there are some other ALTERs or INSERTs which are still processing.
First, to check the processing triggers with this following queries:
SHOW FULL PROCESSLIST;
SELECT * FROM information_schema.INNODB_TRX\G
Then, kill the locked query by trx_mysql_thread_id: 132092 for example:
KILL 132092;
Finally, here‘s a solution - OnlineSchemaChange (OSC). It built by Facebook supports online alter MySQL table schema with minimal impact.
use change column
ALTER TABLE database.table_name
CHANGE COLUMN id id INT(11) NOT NULL AUTO_INCREMENT ,
ADD PRIMARY KEY (id);
Use percona toolkit to change db scheme without needing to lock tables: https://www.percona.com/doc/percona-toolkit/3.0/pt-online-schema-change.html
This allows to continue updating db while scheme change is being applied. This especially useful for large dbs where updating scheme can take a while.
I have a table
CREATE TABLE `uli` (
`id` int(10) NOT NULL,
PRIMARY KEY (`id`)
) ENGINE=MyISAM DEFAULT CHARSET=utf8
The table shall always contain one value in one row. But it could be changed often. I would like to make a select, lock the table for other connections, update a value and the unlock.
For example, the following code works excellent:
LOCK TABLES uli WRITE;
SELECT * FROM uli;
UPDATE uli SET id=id+1;
UNLOCK TABLES;
While first connection do not unlock, all other connection will wait, and only after unlock could see new value. That is exactly what I want. Is it exists some more elegant solution? Does it matter MyIsam or Innodb table to use?
You are locking for write, which means selects will still see the old value before the update. Also you are not doing anything with your select. If you use InnoDB and SERIALIZABLE isolation level, all you need is the update statement.