We use RDS MySQL with a micro service architecture. Each service has its own database and we have been slowly moving them into their own instances for load management and whatnot.
We have a replica still attached to the primary instance (where most of the databases still reside) but it is missing the databases that are on their own instances. We would like to have a single read replica that sources from all of the instances so that we can run sanity checks on the data without needing to move data between instances.
Does RDS support a multi source read replica? Or will we need to manage our own MySQL install in order to accomplish this?
As far as I know, you will have to operate MySQL yourself on EC2 to get the multi-source replication feature. You may like to contact your AWS support representative to confirm this.
AWS RDS does not support multi-source replication. Configuring replication from an external master is limited because you have to configure it using a stored procedure provided by AWS. It does not have any support for configuring multiple replication channels. Each replica has exactly one master.
AWS Aurora 2.x (API-compatible with MySQL 5.7) also does not support multi-source replication. As of Aurora 2.07.x, they list multi-source replication among features of MySQL 5.7 they do not support. See https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/AuroraMySQL.Updates.2070.html
AWS Aurora 1.x (API-compatible with MySQL 5.6) has something they call multi-master cluster, where all the nodes in a cluster can be writers. I have not used this, and I don't know how reliable it is. See https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/aurora-multi-master.html for details.
It's weird that they seem to have lost functionality between Aurora 1.x and 2.x. My understanding is that they sort of "started over" with Aurora 2.x, and some features haven't been implemented in the new version yet.
This is my understanding as of December 2019. Of course future versions may add such features.
Related
I need to create an RDS MySQL instance in us-east-1 and would like to replicate this data in another region (say eu-west-1). I know about read-replicas, and I will be using them also, but I wanted to have a back-up non-read-replica in another region.
Has anyone done this?
Is there code out there that does this replication?
I highly recommend switching from RDS MySQL to RDS Aurora, which is MySQL compatible. You won't have to change anything in your code, the database will still appear to be MySQL to any apps that connect to it. Among the many advantages Aurora has over MySQL is that it supports cross-region replication.
There are several ways to do this. I will cover two easy methods. RDS features are based upon engine (MySQL, Aurora, PostreSQL, etc.). Review as required.
1) Create a read-replica of your existing RDS instance in another region. Then promote that read-replica to master. Now you have a separate instance running in another region. You can then stop this instance to minimize billing (for up to seven days).
2) Create an RDS snapshot. Copy that snapshot to another region. When needed launch a new RDS instance from the snapshot.
I've been looking around for best practices when setting up your database on the cloud but it still isn't clear to me which of the following solutions should we be going for?
Amazon RDS Aurora
Amazon RDS MySQL
MySQL on EC2 instances
I see Amazon Aurora being marketed as the better alternative however after some research it doesn't seem like people are using it. Is there a problem with it?
You should benchmark Aurora carefully before you consider it. Launch an instance and set up a test instance of your application and your database. Generate as high of load as you can. I did at my last company, and I found that despite Amazon's claims of high performance, Aurora failed spectacularly. Two orders of magnitude slower than RDS. Our app had a high rate of write traffic.
Our conclusion: if you have secondary indexes and have high write traffic, Aurora is not suitable. I bet it's good for read-only traffic though.
(Edit: the testing I'm describing was done in Q1 of 2017. As with most AWS services, I expect Aurora to improve over time. Amazon has an explicit strategy of "Release ideas at 70% and then iterate." From this, we should conclude that a new product from AWS is worth testing, but probably not production-ready for at least a few years after it's introduced).
At that company, I recommended RDS. They had no dedicated DBA staff, and the automation that RDS gives you for DB operations like upgrades and backups was very helpful. You sacrifice a little bit of flexibility on tuning options, but that shouldn't be a problem.
The worst inconvenience of RDS is that you can't have a MySQL user with SUPER privilege, but RDS provides stored procs for most common tasks you would need SUPER privilege for.
I compared a multi-AZ RDS instance versus a replica set of EC2 instances, managed by Orchestrator. Because Orchestrator requires three nodes so you can have quorum, RDS was the clear winner on cost here, as well as ease of setup and operations.
I don't use Aurora personally, but I can HIGHLY recommend RDS over running your own on EC2. Having the failover happen automatically and also the backups is just worth every penny. Especially since RDS isn't that much more expensive.
Aurara looks really good on paper, but the more flexible choice of instances has kept me at PostGreSQL until now. We're looking at migrating to Aurora though, mainly because of the autoscaling storage provisioning and the higher performance.
AWS RDS is the managed database solution which provides support for multiple database options Amazon Aurora, PostgreSQL, MySQL, MariaDB, Oracle, and Microsoft SQL Server. When you go with RDS it will provide inbuilt configuration options such as.
Database Replication for High Availability
Read Replicas for Scalability
Backups & Restore
Operating system and software patches & etc.
This simplifies the overhead of database administration. However the flexibility is limited to the RDS offerings.
Alternatively if you host your database in EC2 instance, you can install the required versions of the database engines, install needed extensions & etc. which provides more flexibility but also requires expertise & adds administration overhead.
When you consider Amazon Aurora in RDS, it differs from the rest of the engines because, its new and fully implemented by Amazon from ground up and offers higher performance, reliability out of the box (As marketed by Amazon) with reasonable pricing. However one limitation with Aurora is that its not included in AWS free-tier, where the smallest instance type it supports is "small".
Note: Some of the features offered by RDS and cost differs, based on the database option you select.
I have two rds which have the same structure, each db serves one of two apps. Some of tables are using by first app some by other. I want to set up two ways read/write replication between two rds. I can do it with stand alone mysql, set table replication (https://dev.mysql.com/doc/refman/5.7/en/replication-rules-table-options.html) but can not find any option to do so for rds
This is is probably what you'd call a Really Bad Idea™.
RDS does allow you to configure an RDS instance as a slave of another machine.
http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/mysql_rds_set_external_master.html
Of course, on the same page...
Warning
Do not use mysql.rds_set_external_master to manage replication between two Amazon RDS DB instances.
...however, that appears to be because that's not how you configure an RDS instance to be a replica of another instance. When you're configuring a read-only replica, you don't use this -- RDS manages all of the replication configuration for you.
mysql.rds_set_external_master() is a stored procedure that allows you to execute CHANGE MASTER TO... since, in RDS, you lack the SUPER privilege and would otherwise be unable to do this.
The feature is designed for hot migrations from a non-RDS MySQL server to RDS, by replicating the events from the external master into RDS during the transition.
...however, if there is a way to do what you are trying to do with RDS, this would be it. Each instance would be set to use the other as its master.
The two would need to have network connectivity, which means they'd need to be in the same region and same VPC, or you'd have to handle the peering or tunnel configuration yourself to establish that network path.
This is almost certainly an unsupported configuration, but again, if there is a way to accomplish it, this would be the way. "Unsupported" doesn't mean it won't work, but only that AWS support will not likely be able to provide assistance if it doesn't.
Did I mention this might not be a good idea?
I am struggling to setup replica with Amazon RDS. I want to make RDS as my master but for slave I want to setup my server outside of RDS. For now I am setting my slave at EC2. I have checked AWS doc for the information but could not find the exact answer to my question.
However I did found a blog post which have POC for doing it http://www.ruempler.eu/2013/07/07/replicating-aws-rds-mysql-databases-to-external-slaves/
But the problem that I facing is I am getting error
ERROR: Got error reading packet from server: Slave can not handle replication events with the checksum that master is configured to log
I tried to set binlog_checksum = none at RDS but RDS is not letting me change its config file.
Please suggest me folks!!
You can use AWS DMS (Data migration service) for this purpose. It will support replication between any 2 MySQL DBs.
Please refer our blog:
https://medium.com/tensult/cross-account-and-cross-region-rds-mysql-db-replication-part-1-55d307c7ae65
The question is quite old, but I bumped into it while searching for similar functionality. Amazon supports both replication from an external master and replicating to an external slave these days:
Replication from external master:
http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/MySQL.Procedural.Importing.External.Repl.html
Replication to external slave:
http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/MySQL.Procedural.Exporting.NonRDSRepl.html
Hope this helps future searchers.
It does not appear to be possible as of now (December, 2017).
Looking at #walter-heck's second link above, it is actually (another) explanation of "... replication between an external master instance and a DB instance on Amazon RDS".
It does not appear you can setup an external replica which connects to a RDS running as master.
AWS does offer this service, of course, however each replica must be within RDS.
With all the service offerings AWS has, one would think this would be possible. However, it may not work well with the AWS business model, because you don't pay based on external replicas - you pay based on RDS size.
I have an application database running with MySQL engine on Amazon RDS. For better availability of our data for users in all parts of the world I'm looking for the best solution.
In the previous version of the application, we mirrored our database in US and Singapore, so that users got a better performance in terms of speed and on our side, we had backup if any disaster occurred.
Now as we moved to Amazon, will having Multi A-Z Deployment serve us in the same way? I mean replicates the database in all regions but will RDS still work in a single region only?
I have done some studies but still not sure so please ask me any further questions if I'm being puzzling.
Thank you.
I think you need both the Multi-AZ and the Read Replica features of AWS RDS.
Multi-AZ just creates a non-accessible secondary DB in another availability zone and in case the primary fails, AWS would switch over to the secondary DB. So you have failover.
In the case you want to increase the performance, and your application can work in read-only mode in Singapore (for example), the Read Replica would be perfect. If writes are also required, you would need to route them to the primary read-write database.
AWS supports a combination of the two approaches.
RDS MySQL Multi-AZ deployments currently works only inside an Amazon EC2 region. RDS Read replica's also need to be present inside the same Amazon EC2 region. Inter region replication is the most requested feature in AWS RDS to replicate data to another RDS in Alternative Amazon EC2 region. It is in their road map currently.