At my company we are considering the possibility to migrate our MySQL dbs to AWS Aurora.
AWS documentation states:
What does "MySQL compatible" mean?
It means that most of the code, applications, drivers and tools you already use today with your MySQL databases can be used with Aurora with little or no change. The Amazon Aurora database engine is designed to be wire-compatible with MySQL 5.6 using the InnoDB storage engine. Certain MySQL features like the MyISAM storage engine are not available with Amazon Aurora.
I haven't been able to find a documentation exhaustively listing which MySQL features are not available with Amazon Aurora, and what kind of "little change" we might have to do if we decide to migrate.
What kind of non-BC change should we expect to encounter if we switch (appart from the non-support of the MyISAM storage engine)?
You should not expect any change. Most of the innovations of Amazon Aurora are in the storage and replication tiers, the query engine should behave exactly the same and code run unaltered.
However, MySQL is a completely separate development and organization, so it is quite challenging and risky to put compatibility in more precise terms than what you have found. Feel free to ask if there is any feature compatibility in specific you'd like to know more about.
Related
We use RDS MySQL with a micro service architecture. Each service has its own database and we have been slowly moving them into their own instances for load management and whatnot.
We have a replica still attached to the primary instance (where most of the databases still reside) but it is missing the databases that are on their own instances. We would like to have a single read replica that sources from all of the instances so that we can run sanity checks on the data without needing to move data between instances.
Does RDS support a multi source read replica? Or will we need to manage our own MySQL install in order to accomplish this?
As far as I know, you will have to operate MySQL yourself on EC2 to get the multi-source replication feature. You may like to contact your AWS support representative to confirm this.
AWS RDS does not support multi-source replication. Configuring replication from an external master is limited because you have to configure it using a stored procedure provided by AWS. It does not have any support for configuring multiple replication channels. Each replica has exactly one master.
AWS Aurora 2.x (API-compatible with MySQL 5.7) also does not support multi-source replication. As of Aurora 2.07.x, they list multi-source replication among features of MySQL 5.7 they do not support. See https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/AuroraMySQL.Updates.2070.html
AWS Aurora 1.x (API-compatible with MySQL 5.6) has something they call multi-master cluster, where all the nodes in a cluster can be writers. I have not used this, and I don't know how reliable it is. See https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/aurora-multi-master.html for details.
It's weird that they seem to have lost functionality between Aurora 1.x and 2.x. My understanding is that they sort of "started over" with Aurora 2.x, and some features haven't been implemented in the new version yet.
This is my understanding as of December 2019. Of course future versions may add such features.
I've been looking around for best practices when setting up your database on the cloud but it still isn't clear to me which of the following solutions should we be going for?
Amazon RDS Aurora
Amazon RDS MySQL
MySQL on EC2 instances
I see Amazon Aurora being marketed as the better alternative however after some research it doesn't seem like people are using it. Is there a problem with it?
You should benchmark Aurora carefully before you consider it. Launch an instance and set up a test instance of your application and your database. Generate as high of load as you can. I did at my last company, and I found that despite Amazon's claims of high performance, Aurora failed spectacularly. Two orders of magnitude slower than RDS. Our app had a high rate of write traffic.
Our conclusion: if you have secondary indexes and have high write traffic, Aurora is not suitable. I bet it's good for read-only traffic though.
(Edit: the testing I'm describing was done in Q1 of 2017. As with most AWS services, I expect Aurora to improve over time. Amazon has an explicit strategy of "Release ideas at 70% and then iterate." From this, we should conclude that a new product from AWS is worth testing, but probably not production-ready for at least a few years after it's introduced).
At that company, I recommended RDS. They had no dedicated DBA staff, and the automation that RDS gives you for DB operations like upgrades and backups was very helpful. You sacrifice a little bit of flexibility on tuning options, but that shouldn't be a problem.
The worst inconvenience of RDS is that you can't have a MySQL user with SUPER privilege, but RDS provides stored procs for most common tasks you would need SUPER privilege for.
I compared a multi-AZ RDS instance versus a replica set of EC2 instances, managed by Orchestrator. Because Orchestrator requires three nodes so you can have quorum, RDS was the clear winner on cost here, as well as ease of setup and operations.
I don't use Aurora personally, but I can HIGHLY recommend RDS over running your own on EC2. Having the failover happen automatically and also the backups is just worth every penny. Especially since RDS isn't that much more expensive.
Aurara looks really good on paper, but the more flexible choice of instances has kept me at PostGreSQL until now. We're looking at migrating to Aurora though, mainly because of the autoscaling storage provisioning and the higher performance.
AWS RDS is the managed database solution which provides support for multiple database options Amazon Aurora, PostgreSQL, MySQL, MariaDB, Oracle, and Microsoft SQL Server. When you go with RDS it will provide inbuilt configuration options such as.
Database Replication for High Availability
Read Replicas for Scalability
Backups & Restore
Operating system and software patches & etc.
This simplifies the overhead of database administration. However the flexibility is limited to the RDS offerings.
Alternatively if you host your database in EC2 instance, you can install the required versions of the database engines, install needed extensions & etc. which provides more flexibility but also requires expertise & adds administration overhead.
When you consider Amazon Aurora in RDS, it differs from the rest of the engines because, its new and fully implemented by Amazon from ground up and offers higher performance, reliability out of the box (As marketed by Amazon) with reasonable pricing. However one limitation with Aurora is that its not included in AWS free-tier, where the smallest instance type it supports is "small".
Note: Some of the features offered by RDS and cost differs, based on the database option you select.
I have a few DB snapshots that I made on Amazon RDS from a couple years ago. I'm trying to restore those DB snapshots but when I select the snapshot and click "Restore from DB Snapshot" I get the error:
"Cannot find version 5.1.42 for mysql"
I first thought this was because I didn't have an instance running, so I went to DB Instances, Launched DB Instance, and created one of the same size as this snapshot. However, the oldest MySQL that is available is 5.1.45 so that's what I used. I was able to connect to it from my local machine and I confirmed it had no data within it.
Then I went back to my DB Snapshots and tried again and I still receive the same error even though I now have a DB instance running.
What do I need to do to restore this?
(I'm doing this all from the aws web UI)
That's an 'interesting' (and worrisome) problem - section DB Engine Version Management within chapter MySQL Database Engine in the Amazon RDS Technical FAQ seems to imply that restoring an unsupported MySQL snapshot might not be possible anymore in fact (even though it is nowhere stated explicitly):
Does Amazon RDS provide guidelines for supporting new MySQL version releases and/or deprecating MySQL versions that are currently
supported?
[...]
We intend to support major MySQL version releases, including MySQL
5.1, for 3 years after they are initially supported by Amazon RDS.
We intend to support minor MySQL version releases (e.g. MySQL 5.1.45)
for at least 1 year after they are initially supported by Amazon RDS.
After a MySQL major or minor version has been “deprecated”, we expect
to provide a three month grace period for you to initiate an upgrade
to a supported version prior to an automatic upgrade being applied
during your scheduled maintenance window.
[emphasis mine]
According to Impossible to create a RDS instance in EU-west, MySQL 5.1.42 has been deprecated as of May 24, 2011 the latest already, so this three month grace period has long passed.
Obviously the apparent effect of this deprecation you encountered (i.e. the inability to restore respectively outdated snapshots) will come to a surprise for many, so there might be options to deal with it eventually still, but I'm afraid you'll need to contact AWS for a solution, either directly or via the Amazon Relational Database Service Forum - please post your findings as an answer here if possible, insofar I'd expect this problem to show up regularly as time goes by.
I have an app whose database is being migrated to amazon RDS.
I experienced a significant drop of performance, due to the latency of the queries between RDS and our server (like 30s of loading time only because of the queries). There is no explicit caching, and the requests could be optimized a bit more, but this is still more than 10x slower than with a local database.
I this kind of performance drop expected? If yes, is there a way to use a cloud database with similar performances as a local one?
There have been some reported issues with poor performance of RDS by people. Although amazon as far as I've seen hasn't acknowledged these issues.
RDS (which is just a custom version of mysql) uses the ESB for the storage backend, and as I'm sure you are well aware of the failure they just had with that service.
I've read a lot of companies just running their own mysql DBs in EC2 instances becuase it's showns to have more reliable performance.
Does MySql support for cluster environment? Whether Community edition support for clustering?
I want to know more details about it. Please guide me where can I get details?
You can read everything about MySQL Cluster (which is a different version than MySQL Community, but you can get both for free) on the MySQL website:
http://www.mysql.com/products/database/cluster/
MySQL Community edition can be set up to do master-slaves replication, but that's not as cool and effective as what MySQl Cluster can do.
Yes, MySQL Cluster does support clustering. If you have a Amazon EC2 account, I recommend this tool/website for setting up a MySQL Cluster quickly for testing: http://support.severalnines.com/entries/20916823-installing-on-amazon-ec2
MySQL Cluster is apparently the only MySQL solution right now from Oracle (outside of Percona and others) that allows for 99.999% availability (HA).
From my initial tests, MySQL Cluster is really fast at writes, but slightly slower at reads, though it aggressively caches reads after the first read. I'm still studying it myself.
Also, MySQL Cluster community edition should suffice for most use cases. I think the Carrier Grade edition is only for real heavy-duty uses.
You can get straight to the community (GPL) download for MySQL Cluster at http://dev.mysql.com/downloads/cluster/ (or the commercial version from Oracle e-Delivery - the functionality and underlying code base is the same).
An alternative to MySQL Cluster is SchoonerSQL, a cluster based on MySQL but uses the standard InnoDB storage engine rather than NDB. It supports 99.999% availability and has a very simple GUI management console. Go to SchoonerInfoTechcom