We are in the process of migrating our databases from an MSSQL Server AWS RDS instance onto an AWS MySQL 8.0.17 RDS instance
Our web application uses ORM/hibernate for database interactions, with 1 application tied to 1 database
the database server currently contains 172 db's, with roughly 260 tables per db (44,479 tables in total), setup with 1 user with access to all db's (there are only 4 other system/aws users returned from "select * from mysql.user")
the process of starting up an application requires ORM to inspect the informaton_schema
unfortunately, this is currently taking over 10 minutes to start a single web application, as the MySQL database seems to be struggling with accessing the information_schema, usually stuck on the status "checking permissions" for upto 5 seconds and also seems to be performing hundreds of these lookups per database
on our staging server, the same web application started up in under a minute, due to only having 8 db's rather than 172
we have since recreated the same slowness issue by adding the additional 164 db's to the staging server, indicating that the issue lies with the number of db's/tables on the server
we've already applied the below settings but this hasn't improved the performance:
innodb_stats_on_metadata=0
innodb_stats_persistent=0
Does anyone have any ideas on how we can optimise MySQL further to get the desired performance.
Any help / advice to speed up our schema queries is greatly appreciated
Thanks
Related
We have existing data in on-premise mysql version 5.7 and planning to move application and database on AWS. we provisioned one RDS Mysql database and one Aurora Mysql db and connect application to both server. we saved execution timing in database so we found that RDS MySql is running 2times fast rather then Aurora database.
AWS claim that performance of Aurora database is 5 times faster then RDS Mysql, but it seems this is not correct.
Please suggest is there any tuning required with Aurora db.
System configuration for both DB: db.r6g.large (2CPU and 16 GB RAM)
Note : refer db column prodQueryTime as 'Mysql performance time' and experimentQueryTime as 'Aurora DB performance time'
There are many similar reports like yours, e.g. here. And the answer is that it depends what you do. AWS rep writes:
The most important aspect to keep in mind is that Aurora is optimized for concurrent workloads and its benefits are best evaluated by running parallel benchmarks.
Totally agree.
Aurora MYSQL performance is hugely worse than common MYSQL for our Task (iOS & Android App, accesded by 500K users)
We migrated from RDS MYSQL (t3.medium) to Aurora MYSQL.
We tried Aurora MYSQL and Aurora MYSQL Severless V2.
Originally, our MYSQL t3.medium was just running fine, CPU (20-50%) all time.
The same size of Aurora could not handle the load. We had to duplicate the instance size and keep having issues during peak times.
Also, testing Aurora Serverless v2, we had to scale to 8ACU (16GB) to be able to handle the load.
As I said before, with MYSQL (No aurora), we could handle the load just fine with t3.medium (2CPU, 4GB).
With Aurora we had to duplicate (or more) the instance size, so the cost doubled (or more).
Going back to normal RDS MYSQL.
I have MySQL database which is in size of 4 TB and when I'm dumping it using mysqldump then it is taking around 2 days to dump that database in the .sql format
Can anyone help to faster this process?
OS ubuntu 14
MySQL 5.6
The single database of size 4 TB
hundreds of table average tables size is around 100 to 200 GB
Please help if anyone have any solution to this
I would:
stop the database,
copy the files in a new database
restart the database
process the data from the new place (maybe in an other machine).
If you are replicating, just stop replication, process, start replication.
These methods should improve speed, because of lack of concurrent processes that access the database (and all lock logic).
On such large databases, I would try not to have to make dumps. Just use mysql table files if possible.
In any case 2 days seems a lot, also for a old machine. Check that you are not swapping, and try to check your mysql configuration for possible problems. In general, try to get a better machine. Computer are cheaper than time to optimize.
I'm planning to create a system which tracks visitors clicks into the database. I'm expecting around 1M inserts/day into the Database.
On the backend, I'll have an analytics system which will analyze all the data that's been collected over the days/weeks/months/years.
My question is: is it a practical approach to have 2 different MySQL Servers + 1 Web server? MySQL Server A would insert the clicks into it's DB and it would be connected to MySQL Server B by group replication, so whenever I create reports, etc on MySQL Server B, it doesn't load Server A heavily.
These 2 Database servers would then be connected to the Web Server which would handle all the click requests and displaying the backend reports also.
Is it a practical solution, or is it better to have one bigger server to handle all the MySQL data? Or have multiple MySQL servers that are load balancing each other? Anything else perhaps?
1M inserts/day is not a high load by modern standards. That's less than 12 per second on average.
On sufficiently powerful servers with fast storage and proper tuning of MySQL options, you can expect to support at least 100x that load with a single MySQL server.
A better reason to use multiple MySQL servers is redundancy. Inevitably, any MySQL server needs to be upgraded, or you might have hardware failures and need to replace a disk, or other components. To avoid downtime, you should have a standby database server, which stays in sync with the primary server, either using MySQL replication or by disk-level replication like DRBD.
I have a development database and a production one. Both the database servers, hosted on AWS RDS (MySQL environment), were working fine till the production one has encountered major speed issues with SELECT statements for strings (using either = and LIKE). The size of the table (on production) I am searching from is about 100k rows but it can take 1min + to do a simple string search on phpmyadmin. The development db table is about 30k rows and results are returned in less than 0.1s.
Is this an AWS issue? Or some MySQL config problem. Note that this has only started occurring recently.
Extra: After I do a string search on the table, my read/write latency and queue depth shoots up. The only way I can resolve it is by making a snapshot of the server (reboot does not help).
I've had a Rails application running in production for the past 6 months, with weekly deployments, without any issue.
Now, I've been having a recurring issue for about 3 weeks and it seems to get worst every week.
When my app boots and reaches the point where it's trying to connect to the DB, I get this error :
Can't connect to MySQL server on '***.amazonaws.com' (110) (Mysql2::Error)
AFAIK, this error tells me that I've reached MySQL's max connections limit.
From the configs, I should be able to open 296 connections. My app is set to run 7 instances with each a database connection pool of 5, so it can't really exceed 70 connections when deploying a new instance.
I've never seen the connection count go above 20 in either the AWS RDS Console or the SHOW PROCESSLIST command.
I don't think it has anything to do with either Rails or my application server (Puma), since I can't connect through the MySQL Command-Line Tool when the issue occurs.
Has anyone had a similar issue with MySQL on RDS or MySQL itself?
The database pool isn't per application, it's per process. If it's threading/multi process per instance it could be using more than that. Have you tried restarting mysql? It sounds like you have some hanging connections for whatever reason.
I've been getting these issues recently. Could it be related to the pending-restart change of parameter group on my RDS Instance? I sure hope not. As I understand a pending change should have no effect on the current performance.