Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 1 year ago.
Improve this question
I have created one instance of Amazon Aurora in Sydney Region and restored my RDS snapshot on it. I am executing one simple query on one of my table which has roughly 6k records in it, returns a very slow result. I have not changed any parameter in default parameter group, which is linked to my instance. This query runs perfectly on my existing RDS instance with same parameters with 0.200sec and returns quick response. But the same query takes about 0.350sec on Aurora. My query plan (EXPLAIN) shows me no issue. It uses PRIMARY Index to get a result. So, I can't understand, Why it is so slow? Do I need to configure parameters? As they claim that Aurora is 5x faster than RDS. How Do I check?
Thanks.
Aurora is also a type of Amazon RDS based on MySQL.
How did you migrate data from RDS (which one?) to Aurora on RDS? Did you use Amazon DMS to migrate data between Mysql/MariaDB/Aurora RDS to Aurora RDS? You said you restored a snapshot - (it's impossible to restore Aurora from non-Aurora snapshot).
I had a performance issue with MariaDB and Aurora when I migrate data from other non RDS MariaDB through Amazon DMS. It was extremely slow! Migration process between MariaDB and RDS DM went without problems, no error/warning logs, but it just work extremely slow, almost all queries took 100 times more time than on standard (much smaller) EC2 instance with MariaDB. I tried to increase IOPS, resizing RDS, changing parameters, etc. Nothing helped!
My solution was to not use DMS migration (which changed a lot in tables creation schemas). I did a mysqldump on EC2 Instance with MariaDB and restored it into new MariaDB RDS. Everything started working as expected with a good performance.
We did not use Amazon DMS to migrate to Amazon Aurora. After considering that among options, instead we used an Aurora Read replica. This piece on database migration to Aurora describes are experience and maybe helpful
https://blog.codacy.com/database-migration-to-amazon-aurora/
Related
We have existing data in on-premise mysql version 5.7 and planning to move application and database on AWS. we provisioned one RDS Mysql database and one Aurora Mysql db and connect application to both server. we saved execution timing in database so we found that RDS MySql is running 2times fast rather then Aurora database.
AWS claim that performance of Aurora database is 5 times faster then RDS Mysql, but it seems this is not correct.
Please suggest is there any tuning required with Aurora db.
System configuration for both DB: db.r6g.large (2CPU and 16 GB RAM)
Note : refer db column prodQueryTime as 'Mysql performance time' and experimentQueryTime as 'Aurora DB performance time'
There are many similar reports like yours, e.g. here. And the answer is that it depends what you do. AWS rep writes:
The most important aspect to keep in mind is that Aurora is optimized for concurrent workloads and its benefits are best evaluated by running parallel benchmarks.
Totally agree.
Aurora MYSQL performance is hugely worse than common MYSQL for our Task (iOS & Android App, accesded by 500K users)
We migrated from RDS MYSQL (t3.medium) to Aurora MYSQL.
We tried Aurora MYSQL and Aurora MYSQL Severless V2.
Originally, our MYSQL t3.medium was just running fine, CPU (20-50%) all time.
The same size of Aurora could not handle the load. We had to duplicate the instance size and keep having issues during peak times.
Also, testing Aurora Serverless v2, we had to scale to 8ACU (16GB) to be able to handle the load.
As I said before, with MYSQL (No aurora), we could handle the load just fine with t3.medium (2CPU, 4GB).
With Aurora we had to duplicate (or more) the instance size, so the cost doubled (or more).
Going back to normal RDS MYSQL.
What better way to do a quick migration from RDS MySQL 5.7 from São Paulo/Brazil to RDS Aurora in Northern Virginia, from a large database (probably more than 25GB of dump).
But I can not leave the database stopped for more than 3 hours (
or probably less), because this database is production of a company.
Thank you very much in advance.
In the region of São Paulo no have MySQL Aurora (One of the reasons to do the migration, in addition to the costs being twice as much compared to Northern Virginia)
RDS may be publicly accessible only during migration if necessary.
I will not be able to use Multi-AZ. Would it be feasible to use "AWS Database Migration Service"?
I will also have to migrate the instances EC2 and S3 linked to this database, mainly EC2 to avoid latency problems.
After the migration will be stopped all services in the region of São Paulo.
The main reasons as I said before is the reduction of costs in the long and short term (will be considered the use of reserved instances) and also performance, and instances EC2 to avoid problems of latency and instability.
You are making a mistake trying to move the database and change the engine from MySQL to Aurora at the same time.
Migrate the MySQL 5.7 system now, and convert to Aurora later. You do not need to ask for trouble, and doing both at the same time is exactly that.
It is not possible to "quickly" migrate a primary database over distance, but it is possible to make the amount of setup time irrelevant, and activation time near zero.
Instead of trying to do a copy, create an RDS cross-region replica of your data, and at the last moment, promote that replica to master.
Creating a Read Replica in a Different AWS Region
With Amazon RDS, you can create a MariaDB, MySQL, or PostgreSQL Read Replica in a different AWS Region than the source DB instance. You create a Read Replica to do the following:
Improve your disaster recovery capabilities.
Scale read operations into an AWS Region closer to your users.
Make it easier to migrate from a data center in one AWS Region to a data center in another AWS Region.
https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_ReadRepl.html#USER_ReadRepl.XRgn
It doesn't matter how long it takes for RDS to copy the data and set up the replica, because as soon as it is copied, it starts replicating everything that changed on the master server since the process began.
Once you have verified that everything is correct and consistent, then you promote a replica. It is permanently and irrevocably detached from its original upstream instance, and becomes writable. This is the last thing you do and after the application starts writing to this new database, your original system in São Paulo is obsolete because changes to it will no longer replicate to the new system -- they're permanently isolated.
This arrangement does not require you to establish any networking or make the databases publicly accessible.
And, you can create and destroy multiple replicas to test this process, without disturbing production.
I need to create an RDS MySQL instance in us-east-1 and would like to replicate this data in another region (say eu-west-1). I know about read-replicas, and I will be using them also, but I wanted to have a back-up non-read-replica in another region.
Has anyone done this?
Is there code out there that does this replication?
I highly recommend switching from RDS MySQL to RDS Aurora, which is MySQL compatible. You won't have to change anything in your code, the database will still appear to be MySQL to any apps that connect to it. Among the many advantages Aurora has over MySQL is that it supports cross-region replication.
There are several ways to do this. I will cover two easy methods. RDS features are based upon engine (MySQL, Aurora, PostreSQL, etc.). Review as required.
1) Create a read-replica of your existing RDS instance in another region. Then promote that read-replica to master. Now you have a separate instance running in another region. You can then stop this instance to minimize billing (for up to seven days).
2) Create an RDS snapshot. Copy that snapshot to another region. When needed launch a new RDS instance from the snapshot.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about programming within the scope defined in the help center.
Closed 9 years ago.
Improve this question
How to I take backup of my mysql rds instance from amazon rds service.
and what is the available options to take a backup?
Means I want to take a backup daily, weekly and hourly, where I have to configure it, and amazon where to store that backups.
AWS basically provided two options for DB backup: automated backups and database snapshots.
When automated backups are turned on, Amazon RDS automatically performs a full daily snapshot of your data (during your preferred backup window) and captures transaction logs. By default and at no additional charge, Amazon RDS enables automated backups of your DB Instance with a 1 day retention period, but that's configurable. You can use ModifyDBInstance API to change the existing DB settings.
DB Snapshots are user-initiated and enable you to back up your DB Instance in a known state as frequently as you wish.
DB snapshots and automated backups are stored in S3.
Edit after datasage's comment:
I found this discussion What are the pros/cons of hosting a MySQL DB in Amazon RDS versus a DB instance in EC2. It talks about the problems of using MyISAM tables.
The RDS console allows you to set the backup options (frequency, how many to keep and so on) you don't need to worry about storage RDS does that for you.
If you want to backup an RDS instance outside RDS, then it depends on which type of database you are hosting on RDS, MySQL, Oracle or SQLServer - each have their own backup tools.
If you want to get the database backup to your local system,here is a bash script which does it for you. https://github.com/jeevandongre/aws-repo (I wrote this script and made it open source)
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 4 years ago.
Improve this question
I realize a couple of basic differences between the two, i.e.
EC2 is going to be cheaper
RDS I wouldn't have to do maintenance
Other than those two, are there any advantages to running my database from RDS as opposed to a separate EC2 server acting as a MySQL server. Assuming similar instance sizes, are both going to run into the same limitations in terms of being able to handle a load?
To give you a little bit more info about my use, I've got a database, nothing too big or anything (biggest table 1 million rows), just high SELECT volume.
This is a simple question with a very complicated answer!
In short: EC2 will provide maximum performance if you go with a RAID0 EBS. Doing RAID0 EBS requires a pretty significant amount of maintenance overhead, for example:
http://alestic.com/2009/06/ec2-ebs-raid
http://alestic.com/2009/09/ec2-consistent-snapshot
EC2 without RAID0 EBS will provide crappy I/O performance, thus it's not even really an option.
RDS will provide very good (though not maximum) performance out of the box. The management console is fantastic and it's easy to upgrade instances. High availability and read only slaves are a click away. It's REALLY awesome.
Short answer: Go with RDS. Still on the fence? Go with RDS!!! if you enjoy headaches and tuning every last little bit for maximum performance, then you can consider EC2 + EBS RAID 0. Vanilla EC2 is a terrible option for MySQL hosting.
In this post there is an excellent benchmark between:
Running MySql on a Small EC2 + EBS
Running MySql on a Small EC2 + EBS + adjusted MySql parameters
A Small RDS
The benchmark is very good since it is not focused only in ideal conditions (only one thread) but also in more realistic scenarios, with 50 threads hitting the database.
RDS is not really a high availability system. Read the fine print in the RDS faq. During a failover event it can take up to 3 minutes to failover. Additional amazon will decide it needs to "upgrade" your rds instance and do a failover at that point which will take your database down for "up to 3 minutes" (our experience is that it can take a longer than that).
RDS high availability is very different than master - master or master - slave replication and is much slower. They don't use mysql replication but uses some kind of ebs replication. So in a failover situation it will mount the ebs on the backup machine, start mysql, wait for mysql to do failure recover (hopefully nothing got corrupted too bad), then do a dns switch.
I hope this helps you with you evaluation.
We chose to use EC2 MySQL instances because we have a high read volume and need master-slave replication. Of course, you can spin up multiple RDS instances and setup MySQL replication between them yourself, but we use Scalr.net, which manages that for you using EC2 instances.
Basically, we just tell Scalr how many MySQL instances we want at it keeps them up, automates the setup of replication, handles automatic failover of slave promotion to master if the master gets terminated etc. It does both SQL dump backups and EBS volume snapshots of the master. So, when it needs to create a new slave, it automatically temporarily mounts an EBS volume of the last master snapshot to initialize the slave DB, then starts replication from the appropriate point. All point and click :)
(and no, I don't work for Scalr or anything. Scalr is available as Open Source if you don't want to use their service)
Regarding the maintenance window question. If you use Multi-AZ then RDS will create a standby replica in another availability zone so that there's no down time for maintenance and you protect yourself against a zone failure.
That's what I'm planning to do in the next week or so. Of course it's going to cost you more but I haven't worked that bit out yet.
MySQL on EC2 vs RDS MySQL
Advantages of MySQL on EC2
Amazon EC2 Inter Region Replication
Copy Snapshots across Amazon EC2 regions
RAID 0 with EBS Striping in MySQL EC2
More than 3TB of Disk space ( You will not need this for your size) can be attached on MySQL on EC2.
Disadvantages of MySQL on EC2
Configuration, Monitoring and Maintenance compared to RDS
Point in time backups available in RDS
IOPS lesser than RDS MySQL ( even after RAID 0) currently, 10800 with 6 disks for MySQL on EC2 whereas 12500 IOPS 16KB on RDS MySQL
I have been trying out RDS for a few months and here are some issues I have:
Using SQL profiler is tricky. Since you cannot connect profiler directly to the server, you have to run some stored procedures to create a log file that you can analyze. While they offer some suggestions about how that is done, it is far from user friendly. I would only recommend that you have a certified SQL professional do this kind of work.
while Amazon backs up your instance, you cannot restore an individual database. I have a web app with several separate customer-specific databases and my solution was to launch an EC2 instance with SQL running on it to attach to the production RDB database and import the data and then back it up on the EC2 instance. The other solution was to use a 3rd party tool that creates a massive SQL script (on the app server) that will recreate the schema and populate the data back to a restore point.
I had the same question this weekend. There is a 4 hour downtime window per week for RDS where they do maintenance. RDS seemed more expensive if you can get away with a micro instance of EC2. (This is true of test instances which has minimum traffic) I also wasn't able to change the timezone of the RDS instance because I dont have permission.
I am now actually looking at http://xeround.com/ which is mysql on EC2 by another company. They do not use InnoDB, instead they have their own engine called IDG. I am just starting to investigate that but they are in BETA and will give 500MB of space.