What is the difference between binlog-do-db and replicate-do-db? - mysql

I'm a beginner for MySQL Master-Slave .
and I have read two tutorials .
How to Setup MariaDB (Master-Slave) Replication
Setup MariaDB Master-Slave Replication
In first tutorial. It make me that
[mysqld] Master section
log-bin
server_id=1
replicate-do-db=employees
bind-address=192.168.0.18
[mysqld] SLAVE Section
server_id=2
replicate-do-db=employees
But in the second tutorial, it show me that
[mysqld] Master
server_id=1
log-basename=master
log-bin
binlog-format=row
binlog-do-db=unixmen
[mysqld] Slave
server-id = 2
replicate-do-db=unixmen
And why I should LOCK TABLES, and mysqldump sql, then import it ?
FLUSH TABLES WITH READ LOCK;

DISCLAIMER: To keep things easy and non-confusing here, I talk about a simple 1 Master - 1 Slave setup. No chains, no master-master, or whatever...
Your first tutorial is wrong, it should be binlog-do-db there.
Replication works like this.
The master writes all transactions into its binary log.
The slave reads the transactions from masters binary log and writes them to its relay log.
Only after that the slave executes the statements from its relay log.
binlog-do-db makes the master write only statements for the specified DB into its binary log.
replicate-do-db makes the slave just read statements from the relay log, that are for the specified DB.
replicate-do-db has no effect on the master, since there is no relay log to read from.
The LOCK TABLES part is there, so that the data is consistent. It prevents that the data on the master is modified while backing up the data is still in process.
You restore the database from this backup on the slave, because when you set up a slave, you don't always start from fresh. Therefore it's made so, that you just provide the same data basis on both servers, then you tell the slave at which transaction coordinates the master is and voila, you can start your replication. You can also unlock the master after having dumped the data. Just make sure, that you get the slave up in time before statements in the binary log get lost due to log rotation.

Related

Syncing remote MySQL database with localhost from the first inserted records with MySQL Replication

I have a database myDB in the remote server (master) which has a table myTBL with some records. I want to sync the remote database with my localhost (slave) using MySQL Replication. So I have created the replication user in master and also a database with the name of myDB and the myTBL table in the localhost.
I want to use GTIDs method to syncing my databases because there is a description in MySQL website as follows:
MySQL 8.0 supports different methods of replication. The traditional
method is based on replicating events from the master's binary log,
and requires the log files and positions in them to be synchronized
between master and slave. The newer method based on global transaction
identifiers (GTIDs) is transactional and therefore does not require
working with log files or positions within these files, which greatly
simplifies many common replication tasks.
My master MySQL mysqld.cnf has:
server_id=1
enforce_gtid_consistency = on
gtid_mode = on
log_bin
log_slave_updates
and the status is:
File: ***-bin.000001
Position: 314969
Binlog_Do_DB:
Binlog_Ignore_DB:
Executed_Gtid_Set: 9a670a45-be0a-11e9-be56-52540055d8e4:1-368
and my salve mysqld.cnf has:
server-id = 2
read_only = 1
gtid_mode=ON
enforce-gtid-consistency=ON
skip-slave-start=ON
and I use the following commands to start syncing in slave:
CHANGE MASTER TO MASTER_HOST='***', MASTER_USER='***', MASTER_PASSWORD='***', MASTER_AUTO_POSITION=1;
start slave;
And every thing is OK with this status in slave:
...
Slave_IO_Running: Yes
Slave_SQL_Running: Yes
...
So, every new record that inserted to master database is synced with my slave (localhost) database. But the records which are inserted before syncing do not placed in localhost. How can I sync them? Or in the other words, How do I start syncing from the first records? (I saw in some websites which proposed using mysqldump to dump the first data, is there a better way?)

MySQL replication Slave_SQL_Running fails after inserting data

For school I have to use master slave replication with MySQL on the same computer.
Since you can't run multiple instances of the same MySQL version on your computer, I'm using MySQL 5.6 for the master (port 3306) and MySQL 5.5 for the slave (port 3307).
After performing the following query:
stop slave;
CHANGE MASTER TO
MASTER_HOST='localhost',
MASTER_PORT=3306,
MASTER_USER='MySQL_SLAVE',
MASTER_PASSWORD='mypasswordgoeshere',
MASTER_LOG_FILE='mysql-bin.000007',
MASTER_LOG_POS=1571;
start slave;
show slave status
I see both Slave_IO_Running and Slave_SQL_Running is successful.
However, after inserting data in the master database, the Slave_SQL_Running value switches from 'Yes' to 'No'.
The Last_Error column gives this:
1594 - Relay log read failure: Could not parse relay log event entry. The possible reasons are: the master's binary log is corrupted (you can check this by running 'mysqlbinlog' on the binary log), the slave's relay log is corrupted (you can check this by running 'mysqlbinlog' on the relay log), a network problem, or a bug in the master's or slave's MySQL code. If you want to check the master's binary log or slave's relay log, you will be able to know their names by issuing 'SHOW SLAVE STATUS' on this slave.
Using the mysqlbinlog command on the binary logs of my master and slave I see no errors.
Since I run these two instances on one computer I'm pretty sure my problem isn't caused by a network problem. Since I just imported the master's data to the slave's data, I'm pretty sure this also isn't caused by the MySQL code.
Any thoughts?
Thanks for your time!
Solved the problem by changing binlog_format from 'ROW' to 'MIXED' on the master.

MySQL Master-Slave-Slave Configuration

Quick questions about MySQL Master-Slave-Slave set-ups:
I currently have a Master-Slave set up right now and I would like to add another slave. Would it be possible to clone the server running the slave, and then spin up a new server with the image from the slave, and have it pick up right where it left off? So whatever the binlog was at the time of the copy it would just run until it catches up with the master?
Ideally - I'm trying to start another slave the connects to the master without shutting down the Master for a backup. Any advice or guidance would be great. Thanks!
Yes, you can shutdown slave instance, and copy all it's data to another slave (including logs).
Don't forget to edit my.cnf on second slave (you should change server-id)
Then start both slave servers
Yes this is possible. The best way would probably be to temporarily pause the replication on the slave, determine the master binary log position information, then make your dump from the replica while replication is still paused (and no other data is changing on the replica). After the dump is complete you can restart the replica.
On the new server, just install the dump, set the binlog coordinates and start up the replication. A word of caution though. Make sure your settings for purging the binary logs on the master will allow for retention of the binary logs for long enough for you to do this set up process and get the new slave caught up before the bin logs are purged.
Here's a good tutorial on how to setup multiple replication slaves for a master server:
http://arcib.dowling.edu/cgi-bin/info2html?%28mysql%29replication-howto
It doesn't explain your scenario, but gives important hints: you must assign a unique server-id to your second slave.
Regarding your problem: If your masters binary log is kept long enough, you should not get into trouble. Just shutdown your slave for a moment, clone it and write down: MASTER_LOG_FILE and MASTER_LOG_POS of the slave; then restart the original slave and setup the second slave correctly: that means with that given MASTER_LOG_POS and *_FILE set and a unique server-id in my.cnf;
Then start up your second slave. Use "START SLAVE" to start the replication and then have a look at "SHOW SLAVE STATUS;"
Regards,
Stefan
PS: Cannot promise this to work, but I'm quit sure it should do.
You can use existing mysql slave to make a new one just do the following steps,
Stop replication on slave.
execute show slave status; and note these values Master_Log_File: master-bin.000002 &
Read_Master_Log_Pos: 1307
Take mysqldump and restore it on new mysql slave server, you can copy my.cnf file from existing mysql slave server and just change server-id.
execute change master to command on new slave server providing details of mysql master server and log file name and log position which we obtained from existing mysql slave.
execute start slave; on existing mysql slave.
to verify slave status run show slave status.
that's it you have a new mysql slave server!!
Good luck !

Error: "could not initailize master info structure" while doing Master Slave Replication in MySQL

I am trying to do Master Slave Replication for MySQL. When i am typing the following command:
CHANGE MASTER TO MASTER_HOST='10.1.100.1', MASTER_USER='slave_user', MASTER_PASSWORD='slave_password', MASTER_LOG_FILE='mysql-bin.000001', MASTER_LOG_POS=451228;
mysql> START SLAVE;
it throws the following error:
ERROR 1201 (HY000): Could not
initialize master info structure; more
error messages can be found in the
MySQL error log
Any help would be greatly appreciated.
TRY TO RESET IT, IT DOES MAGIC! ON SLAVE THE SLAVE MYSQL COMMAND TYPE:
RESET SLAVE;
THEN TRY AGAIN:
CHANGE MASTER TO MASTER_HOST='10.1.100.1', MASTER_USER='slave_user', MASTER_PASSWORD='slave_password', MASTER_LOG_FILE='mysql-bin.000001', MASTER_LOG_POS=451228;
mysql> START SLAVE;
Please check several things:
1) Make sure the Master's /etc/my.cnf has server_id actually set
Here is why: Replication relies on the server_id. Whenever a query is executed and is recorded in the master's binary log, the server_id of the master is recorded with it. By default, if a server_id is not defined in /etc/my.cnf, the server_id is defaulted to 1. However, the rules MySQL Replication demand that a server_id be explicitly defined in the master's /etc/my.cnf. In addition, for any given slave, mysqld checks the server_id of the SQL statement as it reads it from the relay log and makes sure it is different from the slave's server_id. That is how MySQL Replication knows it is safe to execute that SQL statement. This rule is necessary in the event Circular (Master-Master,MultiMaster) Replication is implemented.
use select ##server_id; in sql command line to check config really on server.
2) Make sure the Slave's /etc/my.cnf has server_id actually set
Here is why: Same reason as in #1
3) Make sure the server_id in the Master's /etc/my.cnf is different from the server_id in the Slave's /etc/my.cnf
Here is why: Same reason as in #1
As a side note : If you setup multiple slaves, please make sure each slave has a different server_id from its master and its sibling slaves.
Here is why : Example
A master with 2 slaves
MASTER has server_id 1
SLAVE1 has server_id 2
SLAVE2 has server_id 2
Replication will become agressively sluggish on SLAVE2 because a sibling slave has the same server_id. In fact, it will steadily fall behind, catch a break, process a few SQL statements. This is the master's fault for having one or more slaves with identical server_ids. This is a gotcha that is not really documented anywhere.
I've seen this dozens of times in my life time.
I had something very close to that and got same error messages.
Replication run fine, mariadb restart -> "cannot open relay log"
Solution from Neo helped in the first place.
But the root cause it seems were to small open file limits.
Try a lsof | wc and increase DefaultLimitNOFILE to 65535 in /etc/systemd/system.conf and /etc/systemd/user.conf
If nothing else helps and you are convinced everything is set correctly you will have to remove this file:
/var/lib/mysql/<relay_logname>-<connection>.info
after that perform the 'CHANGE MASTER' command as stated above

Master Log Position not moving in MySQL Replication

I have two MySQL servers configured in a Master-Slave relationship.
Master -> my.cnf:
server-id=1283522287
log-bin = /var/log/mysql/binary.log // There are write permissions here.
binlog_do_db= foo,bar
Slave -> my.cnf:
server-id = 1283706035
master-host = {master internal IP}
master-port = 3306
master-user = {master slave user}
master-password = {master slave user password}
However, although MySQL reports that replication is up and running, that is:
"show master status;" gives a valid position and file
"show slave status\G" reports that Slave_IO_Running and Slave_SQL_Running are both "Yes".
The slave is shown in the list of connected slaves, as viewed from the master.
Replication doesn't work. Whenever a change is made, the master file log position doesn't change. It stays at the starting size of 106 (bytes?).
Anybody have any idea what I'm missing?
... and I seem to have answered my own question, proving furthermore that one should NEVER ask questions on SO on Monday morning, after you'd had a weekend's sleep.
I had replication working great without the binlog-do-db statement, but it looked as though that would synchronize the mysql and information_schema, so I added that line:
binlog-do-db = foo,bar
Apparently, you can't comma delimit this (from the MySQL Manual):
Important
To log multiple databases, use this option multiple times, specifying the option once for each database to be logged. Because database names can contain commas, the list will be treated as the name of a single database if you supply a comma-separated list.
Great.