When you move beyond using one instance for your database, what is the best practice when using EC2? If the first instance is a master and you're spinning up slaves, they would need to scan the transaction log and bring themselves up to date before the slaves are useable correct? If the master had been running awhile and was busy, this could take a very long time, right? Is it smarter to use something besides master-slave on EC2? I've seen MySQL Enterprise has support for EC2 but it wasn't clear (to me) on the MySQL site what features this adds. Does it have some added functionality that makes spawning new instances fast and turnkey-like?
Fundamentally, I'm trying to figure how you auto-scale the database.
You could also use the Amazon RDS (their version of MySQL in the cloud) and get out of the running a MySQL server business altogether (you'll pay slightly more per server instance but you can do database snapshots/etc.).
Amazon RDS currently supports five DB Instance Classes, starting at 11 cents an hour going all the way to $3.10 an hour:
* Small DB Instance: 1.7 GB memory, 1 ECU (1 virtual core with 1 ECU), 64-bit platform.
* Large DB Instance: 7.5 GB memory, 4 ECUs (2 virtual cores with 2 ECUs each), 64-bit platform
* Extra Large DB Instance: 15 GB of memory, 8 ECUs (4 virtual cores with 2 ECUs each), 64-bit platform
* Double Extra Large DB Instance: 34 GB of memory, 13 ECUs (4 virtual cores with 3,25 ECUs each), 64-bit platform
* Quadruple Extra Large DB Instance: 68 GB of memory, 26 ECUs (8 virtual cores with 3.25 ECUs each), 64-bit platform
Related
I'm in the process of both virtualizing and updating an old Linux server running a reporting system developed in house (Apache, MySQL, PHP).
The old physical server is running 64-bit Ubuntu 10.04.3 LTS, MySQL 5.1.41 and PHP 5.3. The server has an Intel Xeon CPU X3460 # 2.80GHz (4 cores), 4GB RAM.
We have ESXI 5.5 running on an HP DL380G6 with 2 x Intel Xeon X5650 6 Core 2.66ghz and 32GB RAM.
I created a new VM with 4 cores and 4GB RAM, and did a clean install of 64-bit Ubuntu 16.04.4 LTS, MySQL 5.7.21 and PHP 7.0, migrated our app, and everything is running much slower. I believe it's MySQL because when doing the same direct query on the old physical server vs the new virtualized one, queries can take 8 seconds (VM) instead of 1 (Physical). The tables all have appropriate indexes, running "EXPLAIN" on each server provides the same results, yet one is substantially slower. When a page is running numerous complex queries, it can take a minute+ to load instead of a few seconds.
Any idea why this can be? Same dataset, same query, same engine (MyISAM). The VM has much more recent versions of everything, same number of cores and same RAM. I even tried doubling the VM CPU to 2 sockets, 4 cores and 8GB RAM, and it doesn't seem to have a substantial impact.
I've compared the MySQL configuration and nothing is jumping out at me at being very different.
What might I be missing here? Is it the virtual host hardware?
Did you upgrade the vmware tools? if not do so.
Then, on this ESXi host, do you have other VMs on it? If yes, I would advise you to create a resource pool and then configure the resource limits.
In the resource pool the amount of resource you assign will be in a way reserved for your VM, so you won't have to share with other VMs.
I am using Couchbase Server on stage environment. Things were working fine until yesterday. But since today I am observing high CPU usage when the load is increased moderately. (PFA)
Couchbase cluster configuration:-
3 node cluster running (4.5.1-2844 Community Edition (build-2844))
each having m4.2xlarge(8 cores, 32 GB RAM) AWS machines.
Data RAM quota: 25000 MB
Index RAM quota: 2048MB
It has 9 buckets. And used bucket is having 9 GB RAM (i.e. 3 GB per cluster)
Note: - Since we are using community edition, each node is running Data, Full Text, Index, and query service.
Let me know if I've done some misconfiguration or if any optimization required.
I have an question about this 2 dedicated servers,...
WATH is BETTER to run an website with an large MySql Data Base (4 - 6 GB)
------My actual server-------
Intel Xeon E5-1650v2 - 6c/12t - 3,5 / 3,8 GHz
Ram: 128 GB
Memory: DDR3 ECC 2133 MHz
Disks: 3x600GB SAS HARD + Cache 80Go SSD
------I want to change to this server----------
Intel Xeon 2x E5-2630v3 - 16c/32t - 2,4 / 3,2 GHz
Ram: 128 GB
Memory: DDR4 ECC 1866 MHz
Disks: 2x480GB SSD SOFT
Of corse, the second server, have beeter procesor, better memory... but the disks?
About my website:
Is an very light web, dont use images, videos,... I dont have big files, and is developed with laravel and run in cPanel (centOS)
The problem is the DB, becouse all content (for example, images) are requested from external websites, and all this routes are stored in the DB. So, for now, my DB have 4GB, but in the next monts, can have 6GB.
I need an very fast server for the DB.
MANY THANKS.
Your database should rock on either of those specs. Your entire database can fit in RAM and that will definitely make your web app/DB very responsive. Your database is really not large. 6 GB these days for a DB is small-to-medium size.
I'd go the SSD right. If you can, go with RAID-1 for data protection.
I'm experiencing a very puzzling difference in MySQL performance with a new machine I've recently built compared to two older machines. The machines use MySQL as a backend for an IIS-based webserver.
All three use the same MySQL options (same ini file).
All are running the same tables...all MyISAM.
Caching is off on all.
The newest server (#3) generates web pages noticeably slower than the others and I've narrowed the problem to MySQL. The newest server (#3), equals or dramatically surpasses the other machines in pretty much every benchmark I've run.
It is ONLY mysql that seems to be slow.
At first I suspected some specific queries or configuration might be the problem but even the simplest queries are slower. I've kept making the queries simpler and simpler and the difference remained. No matter what query I run, simple or complex, the new server (#3) is about 2-3x slower, even without any table accesses. I've even run the following query and have still detected a 2x difference in speed:
SELECT 1;
One example of a fairly simple but more than just a trivial query is:
SELECT COUNT(*) FROM users WHERE email LIKE '%ab%'
where users has approximately 18000 rows with email indexed.
This query takes about 0.0090s on server #1.
This query takes about 0.0270s on server #3.
Server #3 is essentially idle with nothing else running on it. I haven't even installed antivirus software on it yet where #1 is under moderate load and #2 is nearly idle.
The OS is certainly different but I wouldn't have expected a 2-3x performance loss on a newer OS version on a much faster machine on one application and no others.
What am I missing????
Server Configurations....
Server #1 (circa 2008-09):
Intel Core i7 Extreme Edition, 3.2 GHz(Nehalem, 4 core)
Intel DX58SO Motherboard
6 GB RAM (3 x 2GB)
Windows 2008, 64 bit
2 x Western Digital Black 640GB HD in Raid 1
MySQL 5.6.16
Connector ODBC 5.2
Server #2 (circa 2010-11, with some upgrades):
Intel Core i7, X995 # 3.6 GHz (Westmere, 6-core)
Intel DX58SO2 Motherboard
12 GB RAM (3 x 4 GB)
Windows 7 Professional, 64 bit
1 x Intel SSD 480 GB (from 1-2 year ago)
MySQL 5.6.16
Connector ODBC 5.2
Server #3 (2013-14):
Intel Core i7-4770K # 3.5 GHz (Broadwell, 4 core)
ASUS Z87-WS Motherboard
16 GB RAM (2 x 8GB, 1866)
Windows Server 2012 R2, 64 bit.
2 x Intel SSD DC S3500 160 GB in Raid 1
MySQL 5.6.17
Connector ODBC 5.2
Here is a diff of the variables ("<" represents server 1 where ">" represents server 3...with log file and server name differences removed):
diff Variables.Server1.csv Variables.Server3.csv
18a19
> "block_encryption_mode","aes-128-ecb"
22c23
< "character_set_database","latin1"
---
> "character_set_database","utf8"
29c30
< "collation_database","latin1_swedish_ci"
---
> "collation_database","utf8_general_ci"
99c100
< "innodb_autoextend_increment","1000"
---
> "innodb_autoextend_increment","64"
206c207
< "innodb_version","5.6.16"
---
> "innodb_version","5.6.17"
212c213
< "key_buffer_size","8388608"
---
> "key_buffer_size","268435456"
335c336
< "pseudo_thread_id","12032"
---
> "pseudo_thread_id","4"
430c431
< "timestamp","1396976138.845271"
---
> "timestamp","1396976213.390847"
440c441
< "version","5.6.16"
---
> "version","5.6.17"
I ever experienced that. In Windows 2012 IPv6 is first priority.
Try to disable IPv6.
From PowerShell run this :
c:> New-ItemProperty -Path HKLM:\SYSTEM\CurrentControlSet\services\TCPIP6\Parameters -Name DisabledComponents -PropertyType DWord -Value 0xffffffff
Then restart.
In windows\system32\drivers\etc\host
uncomment 127.0.0.1 localhost.
While I have not yet tried the suggestion regarding IPv6, I have had some success....indirectly.
A few weeks ago, a patch/service pack was released for windows server. After applying the updates, mysql magically started performing similar to or better than the older server rather than being much slower. I don't exactly know what patch it was since there were several, but I noticed an immediate improvement.
Site is an ecommerce site: 5 Million records in table
Two servers: one for webserver and other contains mysql
Search is happenning through Sphinx server. So search queries do not come to MySQL
Mysql configuration: Dual Quad Core Zeoo 2.0, 146 GB, 16 GB RAM.
Webserver configuration: Dual Quad Core Zeoo 2.0, 146 GB, 16 GB RAM.
For past four days I find MySQL is using CPU continuously for at least 6-7 hours in a day. It becomes normal after that. Even if I restart, it doesnt stop. It again uses CPU in 2 to 3 mins. I even tried stopping Apache and made sure no one is connecting to Mysql.
I just noticed the same issue on my server and tried to do something about it. I upgraded mysql and all it did was reduce the amount of CPU used - from 100% to 55%
I think my issue is Leap Second related - I notice your question is 15 days after the lead second.
I found:
EC2 Amazon Linux AMI MySQL CPU # 62% When Idle?
Which lead me to:
Why Does the Leap Second Cause Problems?
Lastly I did the classic windows fix and rebooted the linux box - this worked (did not check if kernel had been upgraded in mean time)