I followed Google's guide Configuring Replication with Google Cloud SQL to have a replication between these 2 elements:
Master. The Google Cloud SQL instance. Which is supposed to be a modified, but still compatible, MySQL version by Google.
Slave. Standard MySQL version 5.5.35-0+wheezy1 running on Debian 7 Wheezy. This is on an instance of Google Compute Engine.
All the setup when smoothly and the slave is working, but all the users on the slave are not working with passwords. Every user is functional but they can be accessed without passwords which of course is an undesired behavior.
Any idea on how to get the slave working with the same user/passwords as the master?
Note: This would be more appropriate for Server Fault, but I've seen this tag way more active on SO, even with platform-only questions, so I hope this can stay here.
Related
The new Google Cloud MySQL 2nd Generation spins up its own VM instance to run the MySQL server. Please see the following picture:
What is the difference between using the 2nd Generation instance, or using my own Compute VM instance with a manually installed version of MySQL on it? Are there any advantages when it comes to high availability, security, or performance?
Adding to the answer that Terry posted, and answering your question in the comment:
You can create a highly available Cloud SQL Second Generation by doing the following:
Set up your master instance correctly including sizing it appropriately and setting up binary logging. The master instance must have one backup after binary logging has been enabled. You should place your master instance in a zone that's close to your other services. See preparing the master instance.
Create one failover replica in a different zone than the master. See creating a failover replica.
Optionally, create one or more read replicas. Note that a master instance with a failover replica is sufficient for creating a highly available configuration.
Optionally, test failover. Keep in mind that testing the failover moves the master to a new zone.
To answer your question "So what happens if the VM instance they create fails?"
A master instance falls out of high availability mode when the failover replica becomes unavailable. This can happen, for example, if the network connection between the master instance and failover replica is interrupted, or if the failover replica is down due to its own zone failure. During this time, the master instance is not in high availability mode, and you will not be able to failover to the replica because it is not safe to do so. The failover replica resumes replication on reconnection, and high availability mode is re-enabled when the failover replica finishes catching up.
The major difference is that Cloud SQL v2 does not have to be managed. Google Cloud handles management, replication, and snapshots. Additionally Cloud SQL v2 using Cloud SQL Proxy works with App Engine standard and flexible runtimes to allow for flexible, but secure connections to SQL from other clients.
In return you do not have any access to any of the underlying system.
I decided to play around with Google Could SQL and I setup a test sql instance, loaded it with some data and then setup replication on it in the google dev console. I did my testing and found out it all works great, the master/slave setup works as it should and my little POC was a success. So now I want to delete the POC sql instances but that's not going so well.
I deleted the replica instance fine (aka the 'slave') but for some reason the master instance still thinks there is a slave and therefore will not let me delete it. For example I run the following command in the gclound shell:
gcloud sql instances delete MY-INSTANCE-NAME
I get the following message:
ERROR: (gcloud.sql.instances.delete) The requested operation is not valid for a replication master instance.
This screenshot also shows that in the google dev console it clearly thinks there are no replicas attached to this instance (because I deleted them) but when I run:
gcloud sql instances describe MY-INSTANCE-NAME
It shows that there is a replica name still attached to the instance.
Any ideas on how to delete this for good? Kinda lame to keep on paying for this when it was just a POC that I want to delete (glad I didn't pick a high memory machine!)
Issue was on Google's side and they fixed it. Here were the sequence of events that led to the issue happening:
1) Change master's tier
2) Promote replica to master while the master tier change is in progress
Just had the same problem using GCloud. Deleting the failover replica first and then the master instance worked for me.
I am struggling to setup replica with Amazon RDS. I want to make RDS as my master but for slave I want to setup my server outside of RDS. For now I am setting my slave at EC2. I have checked AWS doc for the information but could not find the exact answer to my question.
However I did found a blog post which have POC for doing it http://www.ruempler.eu/2013/07/07/replicating-aws-rds-mysql-databases-to-external-slaves/
But the problem that I facing is I am getting error
ERROR: Got error reading packet from server: Slave can not handle replication events with the checksum that master is configured to log
I tried to set binlog_checksum = none at RDS but RDS is not letting me change its config file.
Please suggest me folks!!
You can use AWS DMS (Data migration service) for this purpose. It will support replication between any 2 MySQL DBs.
Please refer our blog:
https://medium.com/tensult/cross-account-and-cross-region-rds-mysql-db-replication-part-1-55d307c7ae65
The question is quite old, but I bumped into it while searching for similar functionality. Amazon supports both replication from an external master and replicating to an external slave these days:
Replication from external master:
http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/MySQL.Procedural.Importing.External.Repl.html
Replication to external slave:
http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/MySQL.Procedural.Exporting.NonRDSRepl.html
Hope this helps future searchers.
It does not appear to be possible as of now (December, 2017).
Looking at #walter-heck's second link above, it is actually (another) explanation of "... replication between an external master instance and a DB instance on Amazon RDS".
It does not appear you can setup an external replica which connects to a RDS running as master.
AWS does offer this service, of course, however each replica must be within RDS.
With all the service offerings AWS has, one would think this would be possible. However, it may not work well with the AWS business model, because you don't pay based on external replicas - you pay based on RDS size.
I have 3 mysql servers that i need to backup daily.. each server uses just 1 database w/ multiple tables..
I've scripted a mysql dump script on each server.. but this time i want each mysql server backing up to a 4th server (MASTER SERVER) (w/c is remote location) ..
The Master server will serve as a MIRROR for all 3 servers, so that we can view the data of the other servers even if one of them goes down, because the Master server will be on a more reliable internet connection .
NOTES and LIMITATIONS:
1) EACH SERVER needs to "send" their backups to the MASTER SERVER, because the master server can not do "incoming" connection to each slave servers (port forwarding not supported on the slaves)
2) Prefer that only the "changes" are backed up to make things lighter on the network. (synchronization? incremental?)
3) All are running windows 7 at the moment, because for now i'm using Navicat MySQL's synchronization features.. I would prefer to use a PHP script based solution so i can migrate things to *nix.. i've read about replication and all that stuff, but I kinda wanted a ready solution, perhaps a software i could download or buy or something.. I've no time to code my own sync/replication scripts/software. just wana get over this remote sync hurdle and move on w/ the project.
regards to all
i've read about replication and all that stuff, but I kinda wanted a
ready solution
But replication is a ready-solution just type a few commands and change a few configuration.
Using simple replication settings with one MASTER and one SLAVE, how can one ensure that the SLAVE and MASTER are fully synchronized?
Now yes, they both started from the exact same image and replication is working and reporting that everything is okay BUT:
* It has happened that there were errors stopping the replication and then the replication had to be stopped and later resumed.
* Perhaps a change accidentally occurred on the SLAVE and then it's not the same as the MASTER anymore.
* Other whichever scenarios that might break sync.
While it's possible to do a big mysqldump of both database and compare the files I would be interested in a method that can be implemented more easily and also can be checked automatically to ensure all is in sync.
Thanks
Have you tried Percona Toolkit (formerly known as Maatkit)? You can use one of their tools which is pt-table-checksum for your case. You can check other tools too at their website.
pt-table-checksum performs an online replication consistency check by
executing checksum queries on the master, which produces different
results on replicas that are inconsistent with the master. The
optional DSN specifies the master host. The tool’s exit status is
nonzero if any differences are found, or if any warnings or errors
occur.
The following command will connect to the replication master on
localhost, checksum every table, and report the results on every
detected replica:
If you have MySQL Server versions 5.6.14 or higher, you can use the MySQL Replication Synchronization Checker. It's included in the MySql server package. Is designed to work exclusively for servers that support global transaction identifiers (GTIDs) and have gtid_mode=ON.
This utility permits you to check replication servers for synchronization. It checks data consistency between a master and slaves or between two slaves. The utility reports missing objects as well as missing data.
The utility can operate on an active replication topology, applying a synchronization process to check the data. Those servers where replication is not active can still be checked but the synchronization process will be skipped. In that case, it is up to the user to manually synchronize the servers.
See MySQL Documentation for more information
You are right to be suspicious of a seemingly healthy master/slave replication setup! We were running fine when suddenly we got alerts from check_mk concerning a database that existed on our master that did not exist on our slave... but the master and slave status outputs were good! How unnerving is that? The way to prove integrity of the process is to use checksums to verify the data.
I have seen a lot of chatter on the Internet recommending pt-table-checksum . However, its limitations proved to be too onerous for us to be comfortable with. Most importantly, it requires and even sets statement-based replication (see the pt-table-checksum link). As it says in the mysql 5.6 online documentation, (for row-based replication...) "all changes can be replicated. This is the safest form of replication." There are other disadvantages to statement-based replication that make our developers nervous because some functions cannot be replicated properly; see the doc for a list.
We have already experienced issues with a master and slave using statement-based replication so we're specifically trying to avoid it.
We are going to try mysqlrplsync which specifically mentions that it "works independently of the binary log format (row, statement, or mixed)". It also mentions, however, that gtid-mode must be on and it requires MySQL 5.6.14 and higher... which means, I believe, that the MySQL delivered with RHEL7/CentOS 7 at least is out. You'll need to get the MySQL Community Edition, which is left as an exercise for the reader but you can go here for the packages or here for the repos, including RHEL derivatives and Debian.