The master waits for slave acknowledgment of transaction receipt after the sync. Upon receiving acknowledgment, the master commits the transaction to the storage engine and returns a result to the client.I want know when the slaves commit ?
The slaves commit whenever they get around to it, which can be within a matter of milliseconds, but can also take a substantial amount of time if the slave is lagging, because it's overloaded or underpowered, or because a process has obtained locks on the slave that cause the updates to wait for the locks.
Semi-synchronous replication only guarantees that no transaction is lost by making sure that at least one slave has saved -- not executed -- a copy of the the transaction.
The data is safe from any single-failure loss once the replication IO_THREAD on any slave has receive and written the replication event to its relay log on disk, and confirmed this fact back to the master server.
The slave acknowledges receipt of a transaction's events only after the events have been written to its relay log and flushed to disk.
https://dev.mysql.com/doc/refman/5.7/en/replication-semisync.html
The slave SQL_THREAD reads the events that the IO_THREAD has written to the relay log and applies them to the slave's data set, but this part is a fully-asynchronous process that provides no feedback towards the semi-sync logic.
If you need the slaves to be immediately in sync for read queries of data that has just been written to the master, native replication -- neither asynchronous (conventional) nor semi-sync provides that. If that is what you require, you need Galera Replication Provider, which makes replication truly synchronous (and makes all the servers in the cluster into writable masters).
Related
I know Mysql slaves read Master's bin-log sequentially with single thread and dump the data as relay log locally. My question is that will slave use multiple threads to read the relay logs for sync up the operations(Insert,Update,Delete)?
If it's not multiple threads, I guess the following scenario Mysql slave may not be able to catch up the Master forever.
Let's say Master has 100 tables and there is no any relationship between them. So It could be 100 writing requests to the 100 table simultaneous. If we suppose in 1 second the 100 tables finish writing in Master, then Slave may need 100 seconds to finish the sync up if it's single thread operation. Right please? Thanks!
Unless you are using MySQL Server 5.6 or later, then yes, the replication events are executed sequentially by a single thread (the "SQL" thread) after having been received from the master by the "IO" thread and written to the relay log.
However, MySQL 5.6 introduced slave_parallel_workers, which, if configured (along with other prerequisites for multithreaded replication) allows multiple (theoretically up to 1024) parallel replication worker threads, controlled and coordinated by the slave SQL thread, so that queries among independent databases can be replicated in parallel.
It uses one reading thread and one updating thread.
Your scenario doesn't make sense. The 100 updates will take as long at the master as at the slave, maybe longer, unless there is a remarkable discrepancy between the CPUs and disks of the master and slave hosts.
I have a replication setup here where data get replicated from a stationary host to a notebook.
Replication happens in two steps: the copying of the relay files, which is quite fast, and the application of the relay log events to the database, which tends to be slow.
Now my question: Suppose the slave has gotten all data from the master, but the "import process" still runs. Can I safely shut down the slave host and resume the still pending part of the replication without disturbing the process in any way?
So I am connected to the host, say "stop slave", shut down the notebook, go home and then "start slave" again without having a connection to the host. Can I expect the slave instance to resume the import process again?
Your laptop is permanently a Slave to the other machine, correct? You are just breaking the network connection to the Master every night?
There are two threads on the Slave. The I/O thread is responsible for pulling data from the binlog on the Master and putting the stuff into the "relay-log" on the Slave. If (when) the network goes away, this thread repeatedly retries. There are settings that say how frequently and when to eventually give up. Consider tuning them.
The SQL thread is responsible for applying whatever is in the relay-log. Effectively, the SQL thread can run all the time. It's quite happy to "do nothing" when there is nothing to do.
The I/O thread creates new relay-log files as needed; the SQL thread deletes a log as it finishes with it.
I have dealt with dozens of slaves over the years; I don't recall any issue with network or power failures. You are essentially causing at least a network failure every night. If you are also powering down the laptop, do it gracefully. InnoDB (but not MyISAM) recovers nicely from power failures, but don't push your luck.
STOP/START SLAVE seems unnecessary, but won't hurt. Things should "resume" and eventually "catch up".
Your quote talks about the Master purging binlogs. Well, there is an issue here. The Master does not keep track of what Slaves exist, so it can't tell if your Slave is un-connected for longer than the Master is keeping the binlogs.
See expires_logs_days. Suggest you set that to higher than the number of vacation days you might ever take.
My experience with Slaves predates GTIDs, Galera, etc.; will you be using such?
I partially have found the answer to my question:
The MySQL documentation says:
If the slave stops before the SQL thread has executed all the fetched statements, the I/O thread has at least fetched everything so that a safe copy of the statements is stored locally in the slave's relay logs, ready for execution the next time that the slave starts. This enables the master server to purge its binary logs sooner because it no longer needs to wait for the slave to fetch their contents.
This indicates that it is perfectly possible to resume the import process (execution of the statements), however, it still remains unclear
if I need to start slave before the described things happen and
what happens if the slave doesn't find its master if I do start slave.
MySQL slave server was showing Waiting for the slave SQL thread to free enough relay log space status for about a week because the relay logs were full.
It happened because the SQL Thread was stuck on an error. Once I cleared the error, it started processing relay logs again and freeing space.
Does this mean that the Slave will be out of sync with master, even after reading through all all the relay logs?
My reasoning is that since the slave couldn't download any relay logs for a week, that data would be lost.
Please clarify to me what's going on, so I can determine whether or not to resync servers. Re-syncing slave with master is not my favorite option since the database is huge.
Try running some select queries on the slave, which should return some data recently inserted in the master. If replication has stopped, the new data will not be in the slave.
We use Percona mysql on EC2 and have a master / slave setup for HA. What we observe is that the replication to the slave always falls behind in hours or even days as we continuously write in data in the master as it is the nature of our application.
What could be a problem here ?
First, think about how MySQL Replication is organized
Major Components for MySQL Replication
Slave IO Thread: It's responsible for maintaining a constant connection to a Master. It is ready to receives binlogs events from the Master's binlogs and collect them FIFO style in the Slave's relay logs.
Slave SQL Thread: It's responsible for reading binlog events stored in the relay logs. The event (DML, DDL, etc) is executed on this internal thread.
Seconds_Behind_Master: Each binlog event has the timestamp of the event (DML, DDL, etc). Seconds_Behind_Master is simply the NOW() of the Slave server minus timestamp of the event. Seconds_Behind_Master is displayed in SHOW SLAVE STATUS\G.
What is the Problem ?
If Seconds_Behind_Master is ever increasing, consider the following: The single-threaded execution path for binlog events of MySQL Replication is nothing more than the Serialization of all the SQL commands that were executed on the Master in parallel. Picture this: if 10 UPDATE commands were executed on the Master in parallel and take 1 second each to process, they get placed in the relay logs and executed FIFO style. All the UPDATEs have the same timestamp. Subtracting the timestamp for each UPDATE processed on the Slave yields a 1-second increase in Seconds_Behind_Master. Multiplied by 10, and you get 10 additional seconds of Replication Lag. This reveals the SQL Thread as a Bottleneck.
Suggestions
Master and Slave may be underspec'd. Perhaps more Memory and/or Cores so that the Slave can process binlog events faster. (Scaling Up, Slightly Linear Improvements at Best)
Try configuring InnoDB for accessing more cores
Switch to MySQL 5.6 and implement parallel slave threads (if you have multiple databases)
Wait for Percona 5.6, then upgrade and implement parallel slave threads (if you have multiple databases)
I wanted to set up a new replication slave for MySQL 5.6. Just after a CHANGE MASTER and starting the slave, I saw this line in the error log:
[Warning] Slave SQL: If a crash happens this configuration does not guarantee that the relay log info will be consistent, Error_code: 0
If it matters, these settings are included in my.ini:
skip-name-resolve
skip-host-cache
server-id = 111
report-host = myPC
relay-log-recovery
sync_master_info=1
sync_relay_log=1
sync_relay_log_info=1
replicate-do-db = myDB
skip-slave-start
Replication seems working, but this warning is scary. Any idea what makes MySQL issue this warning?
The warning is about crash-safe replication, a feature added to MySQL 5.6. It does not mean anything scary is going on (basically, replication works as in earlier releases) but it is notifying you that you could do better with the new feature set in MySQL 5.6.
A common request is to have replication crash-safe in the sense that the replication progress information always is in sync with what has actually been applied to the database, even in the event of a crash. Although transactions are not lost if the server crashes, it could require some tweaking to bring the slaves up again.
[...]
Crash-safe masters
If the master crashed when a binary log was rotated, it was possible that some orphan binlog files ended up in the binary log index file. This was fixed in 5.1 but is also a piece in the puzzle of having crash-safe replication.
Writing to the binary log is not an atomic operation, and if a crash occured while writing to the binary log, there were a possibility of a partial event at the end of the binary log.
Crash-safe slaves
If the replication information and data is stored in the same storage engine, it will allow both the data and the replication position to be updated as a single transaction, which means that it is crash-safe.
If the replication information and data is stored in different storage engines, but both support XA, they can still be committed as a single transaction.
The replication information is flushed to disk together with the transaction data. Hence writing the replication information directly to the InnoDB redo log does not offer a speed advantage, but does not prevent the user from reading the replication progress information easily.
The tables can be read from a normal session using SQL commands, which also means that it can be incorporated into such things as stored procedures and stored functions.
See the referenced blog post and the MySQL documentation for details of operation and set up instructions.