Can com.mysql.jdbc.ReplicationConnection and com.mysql.jdbc.ReplicationDriver be used directly to split a read-only connection to one of the replication data sources when using a master-slaves topology?
As the javadoc of Connector/J states, the ReplicationConnection is a
Connection that opens two connections, one to a replication master, and
another to one or more slaves, and decides to use master when the connection
is not read-only, and use slave(s) when the connection is read-only.
So, I just wonder if it really works as expected, since we cannot benefit from the master-slaves topology which releases the burden of many read-only connections to the master node.
When I looked inside the source code, I found that the connections to the master and the slave have been both established in the constructor, which means that each read-only operation will not only just connect to the slave, but also connect to the master without any communication, and to be short, it doesn't release the burden of the master.
So, is it correct to use the ReplicationConnection in this way? Or maybe it is just used in other scenario?
Why do you say that it doesn't release the burden of the master? It's true that it will do a connection but it will not issue any queries against the master while the connection is in read-only mode Connection.setReadOnly(true).
So if your application puts your connection in and out of read-only mode for all calls it will in face relieve the master of all those reads while still doing all the writes there.
You can make sure it does the right thing by turning on the general log (ie query log) on both machines and see where each query goes.
Related
RDS Snapshots don't seem to work as I would expect when set up with replication. I'd like to get some guidance on if I'm making incorrect assumptions, or just doing something wrong.
Here's what happened:
I set up an RDS instance as a slave to an external mysql instance (outside of AWS)
I let the instance catch up, replication was running successfully for a few days, taking nightly snapshots of the slave on RDS.
Some queries were run on the slave accidentally, causing errors for the replication, and causing the databases to get completely out of sync.
I restored the slave from a snapshot.
What I expected:
After the snapshot restored, replication on the new slave database would be able to catch back up to the position of the master.
What actually happened:
After the snapshot restored, data was restored, but replication settings were not. show slave status returned null.
TLDR; The AWS documentation states that RDS snapshots back up the entire database instance, so I would expect all of its settings to be backed up as well, including settings for an external master, but that doesn't seem to be the case. What are the limitations of RDS's snapshot capabilities, and how should replication with an external master be handled if the slave gets too far out of sync?
Thanks!
If the replication errors that you mentioned in your question stopped replication for extended period, Amazon AWS RDS stops replication. This is done to prevent excessive storage requirements in the source side. When the RDS replica is restored using a snap shot, the new replica will never catch-up in that case because the binary logs are also deleted from the source in this case. This is mentioned in the AWS documentation but it also states that for this to happen the replication error should continue for a month.
I recently had a RDS read replica crash, so needed to spin up a new read replica.
Now I have both read replicas there.
I want to delete the one that failed.
Is it just a case of terminating it? Does it have any affect on the master or the other read replica?
Have not ever had to do this before, so want to be sure I am doing this the right way
This should be a perfectly safe operation, but taking a last-minute snapshot of your master (or even the other replica, if it's on-line, replicating, and synched) wouldn't hurt.
In MySQL replication, the connection between master and replica is initiated from the replica ("slave") server side -- if a replica goes offline, the master server just closes the connection and goes about its business; there is essentially no risk to the master from a replica disappearing, even at random.
Specifically, in RDS, replicas seem fairly clearly to be designed to be disposable, and RDS also provides the ability to "promote" a replica -- which is not a promotion to being "the" master (replacing the existing one), but rather a promotion to being "a" master, an independent standalone system of its own, fully disconnected from its former relationship with its former master.
If it made you more comfortable, you could "promote" the replica first, and then delete it once you see that process complete, though simply deleting the replica should be harmless.
http://aws.amazon.com/about-aws/whats-new/2012/10/11/amazon-rds-mysql-rr-promotion/
I'm kind of lost there, I want to setup a common MASTER/SLAVE replication on a MariaDB database. I choose MASTER/SLAVE over MASTER/MASTER to avoid complexifying things. The SLAVE will be used only if the MASTER server is down.
I've setup MariaDB 10.0.x, but when I start reading on how to achieve this replication, they introduce Galera, which, if I understand correctly, replaces MariaDB.
What do you use to tell the SLAVE server to take the relay if the MASTER server is down ? Is it handled automatically via the Galera Cluster ?
If possible, I don't want my application to be aware of the slave server : I just want to configure it with the IP of MariaDB MASTER, and if it can't be reached, to use the SLAVE instead. (But I do not want to specify this fallback in the application level)
Thanks
What you are looking for can be achieved. I just completed a setup of MariaDB 10 using asynchronous replication (not Galera). To ensure maximum uptime I setup master / master replication and used mysql-mmm to monitor the setup. This tool will manage a virtual IP and point it at one of the two masters for writing purposes. This ensures consistent writes against a single master as to avoid corruption of the data. If one master fails the virtual IP will be mapped to the other master. This provides the high availability aspect. The instructions stated below were very clear and easy to follow.
http://mysql-mmm.org/mmm2:guide
Good luck!
Using simple replication settings with one MASTER and one SLAVE, how can one ensure that the SLAVE and MASTER are fully synchronized?
Now yes, they both started from the exact same image and replication is working and reporting that everything is okay BUT:
* It has happened that there were errors stopping the replication and then the replication had to be stopped and later resumed.
* Perhaps a change accidentally occurred on the SLAVE and then it's not the same as the MASTER anymore.
* Other whichever scenarios that might break sync.
While it's possible to do a big mysqldump of both database and compare the files I would be interested in a method that can be implemented more easily and also can be checked automatically to ensure all is in sync.
Thanks
Have you tried Percona Toolkit (formerly known as Maatkit)? You can use one of their tools which is pt-table-checksum for your case. You can check other tools too at their website.
pt-table-checksum performs an online replication consistency check by
executing checksum queries on the master, which produces different
results on replicas that are inconsistent with the master. The
optional DSN specifies the master host. The tool’s exit status is
nonzero if any differences are found, or if any warnings or errors
occur.
The following command will connect to the replication master on
localhost, checksum every table, and report the results on every
detected replica:
If you have MySQL Server versions 5.6.14 or higher, you can use the MySQL Replication Synchronization Checker. It's included in the MySql server package. Is designed to work exclusively for servers that support global transaction identifiers (GTIDs) and have gtid_mode=ON.
This utility permits you to check replication servers for synchronization. It checks data consistency between a master and slaves or between two slaves. The utility reports missing objects as well as missing data.
The utility can operate on an active replication topology, applying a synchronization process to check the data. Those servers where replication is not active can still be checked but the synchronization process will be skipped. In that case, it is up to the user to manually synchronize the servers.
See MySQL Documentation for more information
You are right to be suspicious of a seemingly healthy master/slave replication setup! We were running fine when suddenly we got alerts from check_mk concerning a database that existed on our master that did not exist on our slave... but the master and slave status outputs were good! How unnerving is that? The way to prove integrity of the process is to use checksums to verify the data.
I have seen a lot of chatter on the Internet recommending pt-table-checksum . However, its limitations proved to be too onerous for us to be comfortable with. Most importantly, it requires and even sets statement-based replication (see the pt-table-checksum link). As it says in the mysql 5.6 online documentation, (for row-based replication...) "all changes can be replicated. This is the safest form of replication." There are other disadvantages to statement-based replication that make our developers nervous because some functions cannot be replicated properly; see the doc for a list.
We have already experienced issues with a master and slave using statement-based replication so we're specifically trying to avoid it.
We are going to try mysqlrplsync which specifically mentions that it "works independently of the binary log format (row, statement, or mixed)". It also mentions, however, that gtid-mode must be on and it requires MySQL 5.6.14 and higher... which means, I believe, that the MySQL delivered with RHEL7/CentOS 7 at least is out. You'll need to get the MySQL Community Edition, which is left as an exercise for the reader but you can go here for the packages or here for the repos, including RHEL derivatives and Debian.
I have a MySQL database running on our server at this location.
However, the internet connection at this location is slow (Especially when several users are connected remotely).
We also have a remote web server on a very fast internet connection.
Can I run another MySQL server on the remote server and still be able to run queries and updates on it?
I want to have two servers because
- Users at this location can connect via lan (fast)
- Users working remotely can connect to synced remote server (fast)
Is this possible? From what I understand replication does not work this way. What is replication used for then? Backups?
Thanks for your help!
[Edit]
After doing some more reading, I am a little worried about setting up multi-master replication due to the fact that I had not considered multi-master when designing the database and conflicts could be an issue.
The good news though is that most time consuming operations are queries not updates.
And, I found out that there is a driver that handles master-slave connections.
http://dev.mysql.com/doc/refman/5.1/en/connector-j-reference-replication-connection.html
That way writes will be sent to the master and reads can come from the faster connection.
Has anyone tried doing this before? My one concern is that if I update to the master, then run a query expecting to see the update on the slave, will it be there right away? Or will the slow connection make this solution just as slow as using the master for both read and write?
What you're asking, I believe, is called Multi-Master Replication, by which both servers serve as replication masters to each other. Changes on either server become replicated back to the other as soon as possible. MySQL can be configured to do it, however I'm not sure how the differences in speed would affect your performance and data integrity.
http://dev.mysql.com/doc/refman/5.1/en/mysql-cluster-replication-multi-master.html