Can someone clarify the duration of the lock used to create the initial snapshot when using mysqldump with the --single-transaction and --quick options?
I have a large table (16GB, 101M rows) in a database (InnoDB) with binary logging enabled. I do not use any FK constrains on this table.
In order to keep the BIN LOG file count manageable I need to periodically update the mysqldump seed. I want to be able to run mysqldump whilst my service continues to add new records (approx 35/sec).
According to the MySQL documentation: A lock occurs when creating the snapshot and you can continue writing to the table. So is that instant or does it depend on the size of the table? i.e. needs to read the entire contents before the lock is released.
I am concerned that whilst the generating the snapshot I'll be unable to write to the table.
Can someone please clarify what happens as the table dump begins? Happy for a link that describes the process.
When you use --single-transaction and --master-data, mysqldump does the following at the beginning.
FLUSH TABLES;
FLUSH TABLES WITH READ LOCK;
SET SESSION TRANSACTION ISOLATION LEVEL REPEATABLE READ;
START TRANSACTION /*!40100 WITH CONSISTENT SNAPSHOT */;
SHOW MASTER STATUS;
UNLOCK TABLES;
After all of these are done, the actual backup starts. The lock is only necessary so that SHOW MASTER STATUS returns exactly the correct binlog coordinates of the beginning of the transaction.
The backup should not block writes, and writes should not block the backup; however, existing transactions need to be committed or rolled back before the FLUSH statements will finish, and these can interact with established transactions in such a way that new transactions stall waiting for any old transactions that are still open. But the issue will clear itself when those old transactions finish. If you aren't leaving long-running transactions (as you shouldn't be), you should have no issue.
Related
We have production mysql DB around 35G (innoDB) and we have notice when mysqldump start application get unstable. we use following command to run dump
mysqldump --password=XXXX --add-drop-table foo | gzip -c > foo.dmp.gz
after googling people said mysqldump lock table before dumping data so people suggested using --single-transaction flag for innoDB
so for experiment i started mysqldump manually and run some query on tables read/write and it allowed me to perform all operation while mysqldump was running so how do i reproduce this behavior that mysqldump really locking my table which causing application accessibility?
Does mysqldump lock Read Operation or Just Write on table?
We have few DB using MyISAM in that case what we should do to avoid locks
Use --single-transaction to avoid table locks on InnoDB tables.
There's nothing you can really do about MyISAM, though you really shouldn't be using MyISAM. The best workaround is to create a read replica and make backups from the replica so that the locks don't impact the application.
What you should find is that while a backup is running, a READ LOCAL lock is held on the tables in the single database that is currently being backed up, meaning that you can read from the tables but writes (insert/update/delete) will block except for certain inserts on MyISAM that can be achieved without disturbing the lock. Those may be allowed. The easiest way to see this happening is to repeatedly query SHOW FULL PROCESSLIST; to find threads that are blocking.
My website is created using Drupal 7 on one instance of amazon & database is on amazon RDS . whenever i am taking backup of database using Mysql with root user & simultaneously when a developer try to clear the cache on the website that time website goes offline untill backup gets over say for around 30 min this happens everytime . what is the solution to avoid this problem.
Try to use this parameter when taking backup with mysqldump . --lock-tables for MyISAM storage engine. --single-transaction for Innodb storage engine.
--lock-tables Lock all tables before dumping them. The tables are locked with READ LOCAL to allow concurrent inserts in the case of MyISAM tables.
For transactional tables such as InnoDB and BDB, --single-transaction is a much better option, because it does not need to lock the tables at all.--single-transaction produces a checkpoint that allows the dump to capture all data prior to the checkpoint while receiving incoming changes. Those incoming changes do not become part of the dump. That ensures the same point-in-time for all tables.
You could create a read replica and perform the mysqldump against the replica, to keep the load off the main server. https://aws.amazon.com/rds/details/read-replicas/
I need to make working mysql replication from master to slave. (tried it once already)
The database is quite large (over 100GB) and it will take some hours to make it ready for new slave.
The database has MyIsam and innoDB engine and both are being written
I think my only choice is to copy the data files from master to a new slave? (or make a database dump which im referring later in the topic of ROUND 2)
Before that I have to run down all the services which uses the database and
make writelock for tables or should i shut down the whole database?
After data directory sync to the new replication server I started it up and the database with the tables was there. First error that I got rid off by changing bin.log to 007324 and position to 0.
Error 1:
140213 4:52:07 [ERROR] Got fatal error 1236: 'Could not find first log file name in binary log index file' from master when reading data from binary log
140213 4:52:07 [Note] Slave I/O thread exiting, read up to log 'bin-log.007323', position 46774422
After that I got new problems from database and this error came out from every table.
Error 2:
Error 'Incorrect information in file: './database/table.frm'' on query. Default database: 'database'.
Seems that something went wrong.
ROUND 2!
After this scene I started to think that can this be done without long service break.
Master database has been already configured and it works ok to another slave.
So i did some googling and this is what i came up with.
Making read lock to tables:
FLUSH TABLES WITH READ LOCK;
Taking dump:
mysqldump --skip-lock-tables --single-transaction --flush-logs --master-data=2 -A > dbdump.sql
Packaging and moving:
gzip (pigz) the the dbdump and moving it to slave server after that finding the MASTER_LOG_FILE and MASTER_LOG_POS from the dump.
After that i don't think that i want to import the dbdump.sql because its over 100GB and
will take time. So i think SOURCE would be ok option for it.
On SLAVE server:
CREATE DATABASE dbdump;
USE dbdump;
SOURCE dbdump.db;
CHANGE MASTER TO MASTER_HOST='x.x.x.x',MASTER_USER='replication',MASTER_PASSWORD='slavepass',
MASTER_LOG_FILE='mysql-bin.000001',MASTER_LOG_POS=X;
start slave;
SHOW SLAVE STATUS \G
I haven't tested this yet, am I on to something?
--bp
Realize that issuing a SOURCE command is the same as running an import of the dumped SQL from shell. Either way, it is going to take a long time. Outside of that, you have the steps correct - flush table with read lock on master, make a database dump of master, make sure you note master binlog coordinates, import dump on slave, set binlog coordinates, start replication. Do not work with the raw binaries unless you REALLY know what you are doing (especially for INNODB tables).
If you have a number of large tables (i.e. not just one big one), you could consider parallelizing your dumps/imports by table (or groups of tables) to speed things along. There are actually tools out there to help you do this.
You CAN work with the raw binaries, but it is not for the faint of heart. In the past, I have used rsync to differentially update the raw binaries between master and slave (you still must use flush table with read lock and gather master binlog coordinates before doing this). For MyISAM tables this works pretty well actually. For InnoDB, it can be more tricky. I prefer to use the option to set InnoDB to write index and data files per table. You would need to rsync the ibdata* files. You would delete ib_logfile* files from slave.
This whole thing is a bit of a high wire act, so I would not resort to doing this unless you have no other viable options. Absolutely take a traditional SQL dump before even thinking about attempting a binary file sync, and each time until you are VERY comfortable that you actually know what you are doing.
I've read the answers to similar questions, but I don't think they answer my specific question, sorry if I am repeating here.
I am setting up replication with existing data between a master and a slave, both MyISAM. I have a master database that gets written to during the day but not overnight (ie, not now). As explained on the dev.mysql.com site, I first ran FLUSH TABLES WITH READ LOCK on the master and obtained the binary log position using SHOW MASTER STATUS.
In another session, I then ran mysqldump on the master in order to copy this data to the slave. I ran mysqldump with the --lock-all-tables option.
However, after running mysqldump, I checked the master status again and the binary log position had increased by about 30. It has not moved up since the mysqldump finished.
Is this increase due to the mysqldump? Or did the lock not take affect and I need to re-dumo the master data?
Again, apologies if I'm repeating a question! Thanks.
Mysqldump should not cause the binary log position to change.
You need to investigate why it changed. Look inside the binary logs to get an idea of what was written to it. Use mysqlbinlog command for this.
For example if you recorded the initial position as 1234 in binlog.0000003, then execute:
mysqlbinlog --start-position=1234 binlog.0000003
This should show you the changes that were applied after certain position in a binary log.
I have a RHEL 5 system with a fresh new hard drive I just dedicated to the MySQL server. To get things started, I used "mysqldump --host otherhost -A | mysql", even though I noticed the manpage never explicitly recommends trying this (mysqldump into a file is a no-go. We're talking 500G of database).
This process fails at random intervals, complaining that too many files are open (at which point mysqld gets the relevant signal, and dies and respawns).
I tried upping it at sysctl and ulimit, but the problem persists. What do I do about it?
mysqldump by default performs a per-table lock of all involved tables. If you have many tables that can exceed the amount of file descriptors of the mysql server process.
Try --skip-lock-tables or if locking is imperative --lock-all-tables.
http://dev.mysql.com/doc/refman/5.1/en/mysqldump.html--lock-all-tables, -x
Lock all tables across all databases. This is achieved by acquiring a global read lock for the duration of the whole dump. This option automatically turns off --single-transaction and --lock-tables.
mysqldump has been reported to yeld that error for larger databases (1, 2, 3). Explanation and workaround from MySQL Bugs:
[3 Feb 2007 22:00] Sergei Golubchik
This is not really a bug.
mysqldump by default has --lock-tables enabled, which means it tries to lock all tables to
be dumped before starting the dump. And doing LOCK TABLES t1, t2, ... for really big
number of tables will inevitably exhaust all available file descriptors, as LOCK needs all
tables to be opened.
Workarounds: --skip-lock-tables will disable such a locking completely. Alternatively,
--lock-all-tables will make mysqldump to use FLUSH TABLES WITH READ LOCK which locks all
tables in all databases (without opening them). In this case mysqldump will automatically
disable --lock-tables because it makes no sense when --lock-all-tables is used.
Edit: Please check Dave's workaround for InnoDB in the comment below.
If your database is that large you've got a few issues.
You have to lock the tables to dump the data.
mysqldump will take a very very long time and your tables will need to locked during this time.
importing the data on the new server will also take a long time.
Since your database is going to be essentially unusable while #1 and #2 are happening I would actually recommend stopping the database and using rsync to copy the files to the other server. It's faster than using mysqldump and much faster than importing because you don't have the added IO and CPU of generating indexes.
In production environments on Linux many people put Mysql data on an LVM partition. Then they stop the database, do an LVM snapshot, start the database, and copy off the state of the stopped database at their leisure.
I just restarted the "MySql" Server and then I could use the mysqldump command flawlessly.
Thought this might be helpful tip here.