I need to replicate some databases as shown below.
-DB1---replicate to---> SLAVE_A
/
MASTER <--DB2---replicate to---> SLAVE_B
\
-DB3---replicate to---> SLAVE_C
The process described here would work well if I controlled both the master and all slave servers, but unfortunately I only control the master in this case. The slaves will potentially be in different countries, run by other admins.
My concern is that the configuration on SLAVE_A could easily be changed to replicate not only the intended DB1, but DB2 and DB3 as well, which is not good.
The only hope I have of limiting which slave replicates which database is if I can somehow control it from the master, which, from what I've been able to figure out, can't be done. A user has replication privileges either to all databases or none, which is unfortunate.
Am I missing something here, or is this just a limitation in MySQL?
Any thoughts on how this could be accomplished?
Thanks.
On the slave server there is possibility to filter transactions to apply by replicate-do-db or replicate-ignore-db parameters, included in my.cnf file or in command line as option.
The same can be done on master with binlog-do-db or binlog-ignore-db but then - it limits replication do some certain databases on the master. So for you better solution is to filter transactions on the slave.
Create a different MySQL user for each replication slave, and give that user access only to the database you want it to have replication access to.
You can read more in the MySQL Documentation: 16.1.1.3. Creating a User for Replication
CREATE USER 'slave_a'#'some_slave_server' IDENTIFIED BY 'slavepass';
GRANT REPLICATION SLAVE ON `DB1`.* TO 'slave_a'#'some_slave_server';
FLUSH PRIVILEGES;
And then repeat for Slaves B and C on DB 2 and 3 respectively.
Related
I have 2 MYSQL server.
MySQL#1
and
MySQL#2
MySQL#1 hosts a database which has been replicated thanks to this tutorial https://www.digitalocean.com/community/tutorials/how-to-set-up-master-slave-replication-in-mysql to MySQL#2. Let's name this first database DATABASE1
MySQL#2 hosts another database DATABASE2 which has nothing to do with DATABASE1.
Is it possible to replicate as master-slave without creating conflict with the first replication, to let MySQL#1 becoming the slave for MySQL#2 ?
Thanks for any tips.
It is possible. I used to do that myself. There are several ways to do so.
At slave server configuration file, add replicate-ignore-db = DATABASE_YOU_WANT_TO_IGNORE
At master server configuration file, only log the database you want replicate to slave. binlog_do_db = DATABASE_YOU_WANT_TO_REPLICATE
Not only you can specified what database to repliace, you can even specified only what table in particular database that you want to replicate. See also replicate_wild_do_table
Further Reading
Replication Slave Options and Variables
How To Set Up Selective Master Slave Replication in MySQL
Why MySQL’s binlog-do-db option is dangerous
I'm setting up MySQL replication via binlog. It's a master / slave setup.
I have a few questions on how to do this the best way.
Should information_schema be replicated, if yes/no, why?
Should mysql db be replicated, if yes/no, why?
If it all should be replicated, no binlog-do-db needs to be set?
Cheers.
No, Information Schema should NEVER be replicated. This is meta data about your tables, dbs, routines, triggers, etc. Let the server populate it's own information_schema.
mysql db is fine to replicate, especially in the instance where you will be setting up a number of users. If your master fails, you will need the same user information available on your slave server in order to fail over to it. If that should occur, you need to have the same privileges available.
List only the database you want replicated with binlog-do-db. If you don't list any, everything will be replicated by default.
No. need to replicate information schema read this for detail https://dev.mysql.com/doc/refman/5.0/en/information-schema.html
Yes you can because mysql database contains all user credentials so in order to have a backup of users you can replicate it.
Yes use binlog-do-db to replicate selected databases otherwise everything will be replicated if nothing mentioned.
I'm looking for a way to have 2 databases running; 1 at the office and 1 at a data center. When at the office, employees would connect locally but when outside the office they would connect remotely to the data center. Both databases would be fully synchronized. That means that an employee could log in and update a record on the data center and that change would replicate instantly to the office server (or vice-versa). So, either user could edit the same record.
So, the typical scenario of auto_increment_offset etc. won't work because, in this case, each server has to be able to update the same record.
Am I missing something obvious? I can't think of a viable way to handle this. 2 users on 1 db can modify the same records so there HAS to be a way to do the same in this type of setup. I just can't think of one.
You can not do this reliably with asynchronous replication. Consider this simple example:
site A: update foo set bar = 1
site B: update foo set bar = 2
Some time later (milliseconds or hours depending) the transactions are replicated to each site:
site A: update foo set bar = 2
site B: update foo set bar = 1
foo.bar will now be 2 on site A and 1 on site B. There is nothing in MySQL that will detect this or prevent it from happening.
Either partition the problem so the masters "own" different parts of the data or elect a primary master for all updates. Or implement some global locking mechanism.
With one database things are completely different since there is synchronous locking going on internally.
Here's how we did it:
In master-master configuration there are 2 masters and 2 slaves. Both are masters and slaves at the same time:
master1 => slave1[master2]
[slave1]master1 <= slave2
enable binlogging on master server - my.cnf file under [mysqld] section:
log-bin=mysql-bin.log
relay_log=relay-bin.log
server-id = 1
log-slave-updates
auto_increment_increment = 10
auto_increment_offset = 1
You may need to add:
skip-slave-start option too, to prevent from start slave at database startup.
Note:
Each server should use different server-id
auto_increment_offset should differs
Auto increment options:
auto_increment_increment controls the increment between successive AUTO_INCREMENT values.
auto_increment_offset determines the starting point for AUTO_INCREMENT column values.
With auto_increment_increment = 10 and auto_increment_offset=5, auto_inc values would be 5, 15, 25..
Dedicated user:
GRANT REPLICATION SLAVE ON . TO 'replication'#'%' IDENTIFIED BY 'some_password';
Create consistent, binary backup of MySQL database (needed to setup a replication) using LVM but the same steps will apply to ZFS/UFS snapshot and other popular techniques:
connect to MySQL database on master (server1) and run "FLUSH TABLES WITH READ LOCK". It might take few second to complete flush operation and it will lock all tables! It is imporant to keep it in mind, because such operation may impact in production load in some cases. It's better to schedule replication setup to peak-off hours.
Create LVM snapshot (server1): lvcreate -L10G -s -n mysqlbackup /dev/vg/lvm_mysql_partition
Get information about master position. Using previously created connection to database run : SHOW MASTER STATUS; to get all information about current binlog, position and so on.
Copy-paste output somewhere - for future usage.
Uunlock all tables: UNLOCK TABLES;
Mount previously created filesystem: mount /dev/vg/mysqlbackup (this will allow you to access shapshot of data created).
Now you can just copy this data to the second server directly to data dir. Before you start up your database add previously mentioned parameters to my.cnf file, changing value of server-id.
Now, with master information copied somewhere and database files propagated to slave you can remove snapshot: lvremove -f /dev/vg/mysqlbackup on master (server1).
Now, you can log into server2 to check for permissions (depends on which user was used to copy files you'll need to correct it) and after that, startup MySQL instance. Database should start, perform InnoDB recovery (if you're signed it) and after a bit, you will be able to log into using command line client. Because of skip-slave-start your slave will not start by default.
Now, on both servers you need to configure replication thread by setting up master hostname, master port, password, user and info about position:
CHANGE MASTER TO master_host='IP_addr_of_server1', master_port=3306, master_user='replication', master_password='some_password', master_log_file='info_from_MASTER_STATUS', master_log_pos='info_from_MASTER_STATUS';
Now you can start replication: START SLAVE;
Check to make sure: SHOW SLAVE STATUS\G (on new machine)
Seconds_Behind_Master might not be equal to 0 because of changes done to database after you release all tables but with replication working slave should catch master pretty fast.
After your slave server catches up with master, start master-master (server1-server2) setup. Lock all tables on server2 ( FLUSH TABLES WITH READ LOCK;) and run SHOW MASTER STATUS;. On server1 set-up replication by CHANGE MASTER TO..
of course you need to change master_host to valid IP address, master_log_file and master_log_pos values. After that, UNLOCK TABLES on server2 and START SLAVE on server1.
Now you should have master-master-slave configuration with master-master replication between server1 and server2.
I have an existing mysql replication set up (Windows 2008 to Ubuntu 9.04) and created several new tables in the master database. These are not showing up in the slave database.
Do new tables automatically get copied to the slave DB, or do I need to set up replication again?
Thanks!
I'm going to assume that other data is successfully replicating.
Replication in mysql is per-server, so the most likely problems are that either you aren't binloging the events, or that the slave is ignoring them.
For binglogs, verify you aren't turning sql_log_bin off for the connection (which would require SUPER) and that the various options binary-log options are set correctly. You can verify this by running mysqlbinlog on the server's binlogs.
On the slave side, check the replication options.
I want to make a dedicated SLAVE machine for data replication of three database on three different servers. In other words, I want to do Multiple Master => SIngle Slave replication.
Is there any way to do this, as simple as it can be ?
Thanks !
Multi-master replication (a slave with more than one master) is not supported by MySQL (besides MySQL Cluster). You can do a master-master replication of a circular (ring) replication (described here or here).
In High performance MySQL 2nd edition the authors describe a way to emulate multi-master replication using a clever combination of master-master replication and the Blackhole storage engine (Chapter 8 Replication > Replication Topologies > Custom Replication Solutions > Emulating multimaster replication p. 373 - 375).
They show two possibly topologies:
Using two co-masters (allowing to switch the master of the slave from Master 1 to Master 2)
Master 1: hosts DB1 and replicates DB2 from Master 2; the storage engine for all tables in DB2 is changed to Blackhole so that the data is not effectively stored on Master 1.
Master 2: hosts DB2 and replicates DB1 from Master 1; the storage engine for all tables in DB1 is changed to Blackhole so that the data is not effectively stored on Master 2
Slave 1: replicates DB1 and DB2 from either Master 1 or Master 2 (allowing to switch masters); the result is that Slave 1 replicates both databases that are effectively hosted on two different masters.
Using a master-chain
Master 1: only hosts DB1
Master 2: hosts DB2 and replicates DB1 from Master 1; the storage engine for all tables in DB1 is changed to Blackhole so that the data is not effectively stored on Master 2
Slave 1: replicates DB1 and DB2 from Master 2; the result is that Slave 1 replicates both databases that are effectively hosted on two different masters.
Please note that this setup only allows you to send updates to DB1 through Master 1 and updates to DB2 to Master 2. You cannot send updates to either table to arbitrary masters.
Pehaps it's possible to combine the described solution with the hack for a true master-master replication (allowing updates to both masters) that uses some sort of autoincrement-mangling and is described here or here.
No way that I'm aware of.
However, if the requirement here is simply to have a single replication-based backup machine, you can easily enough run three MySQL servers (on different addresses and/or ports) - we do that here, with two replication rings that each include our in-house staging server as a node.
An off-the-wall idea, if you really do want all the data into a single server and the table schemas are either fixed, or pretty much static and under your control: set up one server with the three databases on and link all the tables using the federated engine. In theory (huge caveat: I've never tried it!), you can then replicate off those federated tables on to a second server (again, possibly on the same machine), giving you genuine live copies of the data on a single MySQL instance. You could even try replicating back again, but that way possibly lies madness :)
I do not know a lot about MySQL, but don't you have the possibility to set an 'upload only' replication configuration, where the role of the master/publisher is only to collect updates made at the slave/subscriber level.
Might be worth a look at maatkit's table sync -- it's not "real" replication but it might be good enough.