We have some (10-) MySQL server nodes and we want them to be replicated in a way that if one goes down the rest never get affected.
I know MySQL provides Master/Slave (not good for us) and Multi-Master replication.
Does MySQL Multi-Master replication work like a mesh? I mean all nodes can get updated from all other nodes?
Is it true that even in Multi-Master replication (MySQL) each node can only get the data from one master???
So if the native MySQL replication features can not fulfill our requirement, do you know any tool that can do this? Our preference is the built-in MySql replication.
The answer to your problem is simple yet very technical aswell. Traditional MYSQL tools cannot help so you need help of Galera cache from codership. Some features are synchronous replication which MYSQL do not offer, and it works as a cluster if you have a galera MYSQL cluster setup, one goes down rest keeps on going.
I personally use Percona xtradb cluster for this, percona xtradb cluster is patched version of MYSQL and galera embeded together. All servers can take reads and writes.
I have written an article on installing this but cannot post here as stack overflow may consider as link building.
DO some research on Galera, as you use normal MYSQL in a cluster environemnt, FULL HA and HS.
If you need some more info feel free to discuss it with me.
Thanks...
I've been researching this as well. MySQL seems to offer multi-master replication. It's explained here: http://www.onlamp.com/pub/a/onlamp/2006/04/20/advanced-mysql-replication.html
and a more complex example with masters and slaves here: http://capttofu.livejournal.com/1752.html.
There are 3rd party tools as well and the one that seems to come up often is Tungsten: http://code.google.com/p/tungsten-replicator/
And here's an excellent article on multi-master replication with MySQL: http://scale-out-blog.blogspot.com/2012/04/if-you-must-deploy-multi-master.html
Related
I have two kubernetes cluster and I want to make master-master replication for Mysql database. As I read vitess documentation, that is convenient, but I don't know how to do that?
the data is more than 500G in mysql and maybe need sharding. how can I use vitess as Mysql cluster to have zero down time on database layer?
Vitess does not support master-master replication: https://vitess.io/docs/overview/scalability-philosophy/#no-active-active-replication
If zero downtime is your primary concern, then I recommend looking to Percona Xtradb Cluster or Mysql Group Replication, but these require very reliable network between cluster nodes and can easily cause more issues then solve if used incorrectly.
I want to do synchronous and asynchronous replication, synchronize for some databases, and asynchrony for others.
Asynchronous I am doing it through MariaDB, through the traditional system that has replication.
But I want to implement synchronous replication also with Mysql / MariaDB.
The problem is that I do not know if Mysql Cluster also does that work, or if it is not necessary to have Mysql installed only or MariaDb and only use Mysql Cluster for both.
Thank you.
Disclosure: I am working for the MySQL Cluster team - "MySQL Cluster" as in NDB Cluster.
MySQL NDB Cluster always uses synchronous replication between its nodes. You can still use asynchronous replication to other MySQL instances or MySQL Clusters.
AFAIK only MySQL offers NDB and as open source.
Due to the usual network limitations synchronous replication is better suited for high availability in the local data center. It gives you an always consistent view of your data, two or more active instances and makes programing against it much easier.
Asynchronous is more for replication between data centers or availability zones where you can live with temporary inconsistencies in the data and have your programming model set up accordingly.
"MySQL Cluster" has more than one meaning, so I will avoid it.
"Galera" is the underlying cluster technology in MariaDB, PXC, and (if you do the installation yourself), MySQL.
Galera provides essentially-synchronous among (typically) 3 nodes. Meanwhile, each node can have any number of asynchronous Slaves hanging off it.
Also, one Galera cluster can asynchronously replicate to another such cluster. This is sometimes done with a cluster in each of two datacenters.
Mixing sync and async at the database level is quite unusual, and seems strange. The general principle of Replication is that all servers will have exactly the same (barring delays) data. Please elaborate on what you want to do. Also, think out of the box when it comes to topologies.
Please forgive my ignorance but I do not have a ton of experience with MySQL. I am a web developer but overall I have pretty much just installed MySQL or spun up an RDS instance, connected to it and everything just works. However I am currently being tasked with creating a complete backup of our primary server (essentially high availability with a load balancer etc). So I thought now might be a good time to change how I haver our MySQL setup to take advantage of having 2 servers. Currently our MySQL lives on our primary servers and that will probably stay that way for now, however my goal with this new setup is to have 2 identical servers, both running the application and both having the exact same MySQL data structure / data. I want to be able to essentially open up the web application on one server, make a change, see it on the other or vica versa.
From my research it looks like what I want is called "Master-Master Replication" however it seems as if this is an older / more outdated approach? Every article I read is from ~2010-2012 and most people say it is depreciated and I should use Percona XtraDB Cluster, MySQL Group Replication, or Galera Cluster etc. I basically have no idea what any of that means and I am looking for the easiest way to get Master-Master type setup working. On top of that even if I wanted to use regular Master-Master replication it seems as if this is something specific to MySQL and not Percona which is what I actually use.
I know plenty about dev ops and can get everything but the MySQL up and running in no time but I would love some insight on the best way to do this. Currently servers are running Centos 7, Percona 5.7, Apache 2.4.
The easiest way to start is to setup a percona xtradb cluster with proxysql.
Percona xtradb cluster handles all the problems that are present in basic MySQL Master-Master Replication.
Both Galera and Percona Xtradb CLuster are same, as they use the same galera plugin, Only that Percona has customized it and added some extra features like in built proxysql( kind of load balancer for database).
Let's clear up one thing first. Percona is MySQL. If a feature is available on Oracle MySQL (the community version is what I am referring to here), then it is also available on Percona. Think of the two like a cake: MySQL is the base "cake" and Percona is the frosting/sprinkles/etc that go on top.
That being said you can absolutely do Master/Master using Percona MySQL. The standard/best practice is to configure this in active/hot-standby. Meaning, do not write to both nodes at the same time. Your application, or your middleware/proxy, must isolate writes to only one node. This will save you insane headaches when/if replication breaks.
ProxySQL is not a Percona product, but we do support it 100%. You can place ProxySQL in front of your M/M and setup the correct query rules to only route to one MySQL.
You are correct that the most modern solution here, and in my opinion, an easier solution, is to set up a 3-node Percona XtraDB Cluster (PXC). Again, Galera MySQL is the "cake" and PXC is all the extras on top. Group Replication is pie (just to continue the analogy.)
We consider a 3-node PXC fronted with ProxySQL to be one of the best HA solutions out there for MySQL. It is still best practice to only write to one node, and utilize the other 2 for read-only queries. If the primary node goes down, each of the other two have an up-to-the-transaction copy of the data. Again, ProxySQL helps route the traffic if primary goes down.
Backups can be taken from any PXC node using Percona XtraBackup in a non-blocking/hot fashion.
We use mysql multi-source replication, analysts and developers works with the databases, it's a main task of server(cross-bases queries, etc). Mysql slave replicates data from about 10-15 servers, some of them is realy big (400gb, 500gb, 1.5tb). Slave host - ec2 r4.2xlarge. But have some problems, main of them - stability. Often there is an errors 1236, 1594. We fix one channel - another fails, sometimes it's critical.
Backups of most master databases are performed through ebs snapshot, where datadir located. But here another problem of mysql multi-source - work with backups, unlike multi instance replication, I can not just change datadir, where backup located.
I was trying to find a solution that at least partially solved the problems of stability and work with snapshots, but I found absolutely nothing.
Did I understand correctly that there is no support for multi-source replication in the aws-RDS?
Maybe there is a similar solution, or there is another approach to solving the problem (FEDERATED is bad variant). Thanks for any help and advices.
As Michael said, you shouldn't be encountering these errors. Multi source replication has quite stable now. I personally been using it for more than 2 years now. Yes I have faced some of issues like "slave trying to access binlogs that master has purged", but most of them were easy to sort out. I would recommend you to look at these replication topics:
Errant transactions
GTID hole : this was there because of bug
max allowed packet error
We fix one channel - another fails, sometimes it's critical.
make sure you're not touching GTID set for other channels.
Did I understand correctly that there is no support for multi-source
replication in the aws-RDS?
The feature still not available in any of AWS cloud services. https://forums.aws.amazon.com/thread.jspa?messageID=781416&tstart=0
Multi source replication is not supported in RDS , you can use AWS DMS for same purpose .
Is there any product that make mysql master/slave failover process easier? something that can happen automatically, rather than manually fix it.
[... a bit later ... ;)
What are you calling "easier" ?
There is pretty much solutions with MySQL :
MySQL Cluster, with NDB Engine (High availability solution but some limitations : no foreign keys)
MySQL Master / Slave with their own replication system
MySQL replicated via DRBD and heartbeat.
There is a lot of material on this subject :
Failover support for a DB and even more here : Scaling solutions for MySQL (Replication, Clustering)
MySQL cluster in an out of the box solution, but replication is not very difficult to set up.
Regards.
The actual failover bit can be managed with heartbeat (part of the Linux HA project; http://linux-ha.org/), and I'm told that there's some tools in the MySQL Enterprise grab bag that helps with managing replication, although I've never used it myself.