I have an app whose database is being migrated to amazon RDS.
I experienced a significant drop of performance, due to the latency of the queries between RDS and our server (like 30s of loading time only because of the queries). There is no explicit caching, and the requests could be optimized a bit more, but this is still more than 10x slower than with a local database.
I this kind of performance drop expected? If yes, is there a way to use a cloud database with similar performances as a local one?
There have been some reported issues with poor performance of RDS by people. Although amazon as far as I've seen hasn't acknowledged these issues.
RDS (which is just a custom version of mysql) uses the ESB for the storage backend, and as I'm sure you are well aware of the failure they just had with that service.
I've read a lot of companies just running their own mysql DBs in EC2 instances becuase it's showns to have more reliable performance.
Related
We have existing data in on-premise mysql version 5.7 and planning to move application and database on AWS. we provisioned one RDS Mysql database and one Aurora Mysql db and connect application to both server. we saved execution timing in database so we found that RDS MySql is running 2times fast rather then Aurora database.
AWS claim that performance of Aurora database is 5 times faster then RDS Mysql, but it seems this is not correct.
Please suggest is there any tuning required with Aurora db.
System configuration for both DB: db.r6g.large (2CPU and 16 GB RAM)
Note : refer db column prodQueryTime as 'Mysql performance time' and experimentQueryTime as 'Aurora DB performance time'
There are many similar reports like yours, e.g. here. And the answer is that it depends what you do. AWS rep writes:
The most important aspect to keep in mind is that Aurora is optimized for concurrent workloads and its benefits are best evaluated by running parallel benchmarks.
Totally agree.
Aurora MYSQL performance is hugely worse than common MYSQL for our Task (iOS & Android App, accesded by 500K users)
We migrated from RDS MYSQL (t3.medium) to Aurora MYSQL.
We tried Aurora MYSQL and Aurora MYSQL Severless V2.
Originally, our MYSQL t3.medium was just running fine, CPU (20-50%) all time.
The same size of Aurora could not handle the load. We had to duplicate the instance size and keep having issues during peak times.
Also, testing Aurora Serverless v2, we had to scale to 8ACU (16GB) to be able to handle the load.
As I said before, with MYSQL (No aurora), we could handle the load just fine with t3.medium (2CPU, 4GB).
With Aurora we had to duplicate (or more) the instance size, so the cost doubled (or more).
Going back to normal RDS MYSQL.
I am running Django Rest API on an AWS ec2-server. Right now the Api's are using MySql localhost database. Should I shift my database from MySql localhost to Amazon RDS instance?
As per what I Know for remote servers would take a little extra time to transmit the request and shared resources. Would this little extra time be worth migrating my database from MySql localhost to Amazon RDS instance?
I read this answer but it didn't helped me much.
MySql localhost vs Amazon RDS instance
An answer with all possible Pros and Cons will really be appreciated.
Pros for local MySQL
Slightly faster, because of proximity to the application
Cons for local MySQL
Not Easily Scalable
If you want to use autoscaling for your application load and traffic then you might have nightmares, because as you scale you will have even the MySQL servers running on each new node.
Pros for RDS
You don't have to worry about installing and maintaining MySQL server
You don't have to worry about scaling
You don't have to worry about load balancing
You don't have to worry about EC2 upgrades and patching
You don't have to worry about failure recovery because when you provision a Multi-AZ DB Instance, Amazon RDS synchronously replicates the data to a standby instance in a different Availability Zone (AZ)List item
Cons for RDS
Slightly slower due to network latency
It depends how database intensive your application is.
See this benchmark The local database blew RDS out of the water on query latency with low load.
The answer is probably use both? Use both a local Redis/MySQL for quick queries and an off server RDS for long queries over large data sets where paying the additional network latency makes sense.
Also think about using SQLite on S3. If you can easily shard your data, and most queries are read intensive it could be a lot cheaper, especially with something like Redis on the server to cache frequent queries.
If you want to really eek out performance per $$, you can use a lot of Pang's tricks by having a hierarchy of SQLite files.
I've been looking around for best practices when setting up your database on the cloud but it still isn't clear to me which of the following solutions should we be going for?
Amazon RDS Aurora
Amazon RDS MySQL
MySQL on EC2 instances
I see Amazon Aurora being marketed as the better alternative however after some research it doesn't seem like people are using it. Is there a problem with it?
You should benchmark Aurora carefully before you consider it. Launch an instance and set up a test instance of your application and your database. Generate as high of load as you can. I did at my last company, and I found that despite Amazon's claims of high performance, Aurora failed spectacularly. Two orders of magnitude slower than RDS. Our app had a high rate of write traffic.
Our conclusion: if you have secondary indexes and have high write traffic, Aurora is not suitable. I bet it's good for read-only traffic though.
(Edit: the testing I'm describing was done in Q1 of 2017. As with most AWS services, I expect Aurora to improve over time. Amazon has an explicit strategy of "Release ideas at 70% and then iterate." From this, we should conclude that a new product from AWS is worth testing, but probably not production-ready for at least a few years after it's introduced).
At that company, I recommended RDS. They had no dedicated DBA staff, and the automation that RDS gives you for DB operations like upgrades and backups was very helpful. You sacrifice a little bit of flexibility on tuning options, but that shouldn't be a problem.
The worst inconvenience of RDS is that you can't have a MySQL user with SUPER privilege, but RDS provides stored procs for most common tasks you would need SUPER privilege for.
I compared a multi-AZ RDS instance versus a replica set of EC2 instances, managed by Orchestrator. Because Orchestrator requires three nodes so you can have quorum, RDS was the clear winner on cost here, as well as ease of setup and operations.
I don't use Aurora personally, but I can HIGHLY recommend RDS over running your own on EC2. Having the failover happen automatically and also the backups is just worth every penny. Especially since RDS isn't that much more expensive.
Aurara looks really good on paper, but the more flexible choice of instances has kept me at PostGreSQL until now. We're looking at migrating to Aurora though, mainly because of the autoscaling storage provisioning and the higher performance.
AWS RDS is the managed database solution which provides support for multiple database options Amazon Aurora, PostgreSQL, MySQL, MariaDB, Oracle, and Microsoft SQL Server. When you go with RDS it will provide inbuilt configuration options such as.
Database Replication for High Availability
Read Replicas for Scalability
Backups & Restore
Operating system and software patches & etc.
This simplifies the overhead of database administration. However the flexibility is limited to the RDS offerings.
Alternatively if you host your database in EC2 instance, you can install the required versions of the database engines, install needed extensions & etc. which provides more flexibility but also requires expertise & adds administration overhead.
When you consider Amazon Aurora in RDS, it differs from the rest of the engines because, its new and fully implemented by Amazon from ground up and offers higher performance, reliability out of the box (As marketed by Amazon) with reasonable pricing. However one limitation with Aurora is that its not included in AWS free-tier, where the smallest instance type it supports is "small".
Note: Some of the features offered by RDS and cost differs, based on the database option you select.
I have a really simple bookshop webapplication written in Spring framework, just to test its scalability.
I deployed this bookshop on one EC2 instance (t1.micro), and database on Amazon RDS (t1.micro) with master/slave replication of one master instance and 3 slave instances (There's really a lot more reads than writes). One t1.micro RDS instance can have at maximum of 32 concurrent connections
Then I did stress testing with JMeter, figured out that the bottleneck is in the database, since you can have at maximum 32 concurrent connections to t1.micro RDS instance.
Should I auto scale RDS database instances, since creating new replica modifies master and it really takes long time to make it available?
Instead of using RDS should I create EC2 instances with MySQL master/replica and then auto scale these instances?
Should I shard my database instead of replication?
Application also uses com.mysql.jdbc.ReplicationDriver to load balance between master and slave instances. Should I use something different like HAProxy?
Have you ever consider Caching and Partitioning ? The web application we have worked have used Memcache. It really helps in performance issues. On the other hand If you have tables that have so much records, you should consider partitioning, accessing these tables on partitions can have remarkable affect.
I plan to move a site from a shared host to AWS. For that I plan to use a medium EC2 Ubuntu instance on which I will install wordpress.
The question I have is regarding my lack of understanding of the difference between installing MySQL into my EC2 instance and creating another RDS instance to manage the DB for Wordpress (apart from the cost of another instance just for the MySQL DB)
I do understand that RDS seems a more robust solution for big sites and lots of transactions, but for a WP site with 5000 visits a day what will be the adequate solution?
Thanks
With 5,000 visits a day, either one will be fine. The main benefit to RDS isn't power, but the fact that it's managed. Amazon handles version upgrades, backups, etc. for you, whereas with an EC2 instance you have to do that all yourself.
Don't waste your money on two instances, one should be more than enough for the wordpress + MySQL and should give you great results