I have two tables with millions of rows (20+ GB) in a MySQL database. Adding a foreign key through phpMyAdmin times out after 16 hours (current default). The server environment is as follows:
OS: Windows 2008 R2 Server
Stack: WAMP
CPU: Xeon X3464
RAM: 8Gb
(Virtualized)
Is this server setup/MySQL capable of handling a data set of this size?
Yes, MySQL can handle a table that size.
innodb_buffer_pool_size should be something like 5G on your 8GB machine. But it should not be so large that mysql swaps.
You should probably increase the timeout that is causing it to fail.
Read the details in http://dev.mysql.com/doc/refman/5.6/en/alter-table.html -- based on exactly which version you are using and whether you are using ALGORITHM=INPLACE and whether you have other alters in the same statement.
Related
Why MySQL server took time compare than MySQL inside wamp ?
Machine 1 installed MySQL 5.6.17(inside the wamp)
Machine 2 installed MySQL 5.7.24(separate server)
Both machines are same configuration and same OS.
I imported same DB dump file to Machine1 and Machine2.
Now I execute query (the query get data from 6 join tables) and return 400 rows.
Took time:
Machine 1 (5.6.17) inside wamp- Below 30 sec's
Machine 2 (5.7.24) - Morethan 230 sec's
Shall I use MySQL(wamp) instead of MySQL server?
I think MySQL server need to increase Innodb_bufferpool_size on my.ini which located from C;\Program Data (Default hidden folder)
Default Innodb_bufferpool_size is 8M
innodb_buffer_pool_size: This is a very important setting to look immediate after the installation using InnoDB. The InnoDB is the buffer pool where the data is indexed the cached, which has a very large possible size that will make sure and use the memory no the disk space for most of the read-write operations, generally the size of InnoDB values are 5-6GB for 8GB RAM.
Fix: Increase innodb_buffer_pool_size
innodb_buffer_pool_size=356M
I have performance issue in MySQL 5.6 upgrade.
OS version : Solaris 10
Language : Perl script
MySQL version : MySQL upgraded from 5.1 (logical upgrade- Installed 5.6 on same sever and restored it in 5.6)
Memory : 64 GB
I have upgraded MYSQL form 5.1 to 5.6 in solaris 10 and converted all mysql tables from MYISAM to INNODB because mysql 5.6 have default storage engine as INNODB. My database size is 4.5 GB and added the parameters for the innodb.
innodb_file_per_table
innodb_flush_method=O_DIRECT
innodb_log_file_size=512 M
innodb_buffer_pool_size=5 G
My application is creating some temporary tables while doing the transactions so I have created tmp_table_size and max_heap_table_size to 512 M .
In the application there are 3 modules and 2 are working fine and in third module it is very slow compared to MySQL 5.1. It contains nearly 20+ update statements and doing some joins with the temporary tables and master tables contains 2 million records in some tables.
I have the explain plan and done the profiling on the queries. In the profiling I observed sending data is taking huge time because of this performance is degraded.
Can anyone suggest on this to improve the performance.
I have also similar issues with solaris10 and mysql upgrade from 5.1 to 5.6.
I did the logical upgrade and while restoration it took lat time to restore 1Gb database. I have tried with inplace upgrde and found the similar issues with performance.
Just I have solaris 10 server which is running from the past 3.5 yeasrs. so i have restarted my server so that all the cache was cleared and after the restart I have started my all databases services.
NOTE : Before doing the restart stop all the services and better to create another 5.1 database version and take the backup and start the inplace upgrade activity. It is better to go with inplace rather than doing the logical upgrade. Both are same but while restore it will take more time.
The below command takes 2-3 seconds in a Linux MySQL 5.6 server running Php 5.4
exec("mysql --host=$db_host --user=$db_user --password=$db_password $db_name < $sql_file");
On windows with similar configuration it takes 10-15 seconds. The windows machine has a lot more ram (16gb) and similar hard drive. I installed MySQL 5.6 and made no configuration changes. This is on windows server 2012.
What are configurations I can change to fix this?
The database file creates about 40 innodb tables with very minimal inserts.
EDIT: Here is the file I am running:
https://www.dropbox.com/s/uguzgbbnyghok0o/database_14.4.sql?dl=0
UPDATE: On windows 8 and 7 it was 3 seconds. But on windows server 2012 it is 15+ seconds. I disabled System center 2012 and that made no difference.
UPDATE 2:
I also tried killing almost every service except for mysql and IIS and it still performed slowly. Is there something in windows server 2012 that causes this to be slow?
Update 3
I tried disable write cache buffer flush and performance is now great.
I didn't have to do this on other machines I tested with. Does this indicate a bottleneck With how disk is setup?
https://social.technet.microsoft.com/Forums/windows/en-US/282ea0fc-fba7-4474-83d5-f9bbce0e52ea/major-disk-speed-improvement-disable-write-cache-buffer-flushing?forum=w7itproperf
That is why we call it LAMP stack and no doubt why it is so popular mysql on windows vs Linux. But that has more to do more with stability and safety. Performance wise the difference should be minimal. While a Microsoft Professional can best tune the Windows Server explicitly for MySQL by enabling and disabling the services, but we would rather be interested to see the configuration of your my.ini. So what could be the contributing factors w.r.t Windows on MySQL that we should consider
The services and policies in Windows is sometimes a big impediment to performance because of all sorts of restrictions and protections.
We should also take into account the Apache(httpd.conf) and PHP(php.ini) configuration as MySQL is so tightly coupled with them.
Antivirus : Better disable this when benchmarking about performance
Must consider these parameters in my.ini as here you have 40 Innodb tables
innodb_buffer_pool_size, innodb_flush_log_at_trx_commit, query_cache_size, innodb_flush_method, innodb_log_file_size, innodb_file_per_table
For example: If file size of ib_logfile0 = 524288000, Then
524288000/1048576 = 500, Hence innodb_log_file_size should be 500M
innodb_flush_log_at_trx_commit = 2
innodb_flush_method = O_DIRECT
https://dev.mysql.com/doc/refman/5.1/en/innodb-tuning.html
When importing data into InnoDB, make sure that MySQL does not have autocommit mode enabled because that requires a log flush to disk for every insert
SET autocommit=0;
Most importantly innodb_flush_log_at_trx_commit as in this case it is about importing database. Setting this to '2' form '1' (default)hm can be a big performance booster specially during data import as log buffer will be flushed to OS file cache on every transaction commit
For reference :
https://dev.mysql.com/doc/refman/5.5/en/optimizing-innodb-bulk-data-loading.html
https://dba.stackexchange.com/a/72766/60318
http://kvz.io/blog/2009/03/31/improve-mysql-insert-performance/
Lastly, based on this
mysql --host=$db_host --user=$db_user --password=$db_password $db_name < $sql_file
If the mysqldump (.sql) file is not residing in the same host where you are importing, performance will be slow. Consider to copy the (.sql) file exactly in the server where you need to import the database, then try importing without --host option.
Windows is slower at creating files, period. 40 InnoDB tables involves 40 or 80 file creations. Since they are small InnoDB tables, you may as well set innodb_file_per_table=OFF before doing the CREATEs, thereby needing only 40 file creations.
Good practice in MySQL is to create tables once, and not be creating/dropping tables frequently. If your application is designed to do lots of CREATEs, we should focus on that. (Note that, even on Linux, table create time is non-trivial.)
If these are temporary tables... 5.7 will have significant changes that will improve the performance (on either OS) in this area. 5.7 is on the cusp of being GA.
(RAM size is irrelevant in this situation.)
I have a dedicated server (Intel® Core™ i7-2600 Quadcore incl. Hyper-Threading Technology 16GB DDR3, 2 x 3 TB SATA 6 Gb/s HDD 7200 rpm (Software-RAID 1)) and have installed nginx+apache+mysql from debian stable.
I have a db with a table of 2+ million rows (around 400MB of data). When I drop an index db is very slow. For example I am dropping now an index in a single column for around 8 minutes. From iotop I see mysql has around 8Mb/sec. Isn't this too slow?
When you are altering a table (including adding or dropping an index) in innodb, the whole table is rewritten on the disk. (data is copied, indexes are regenerated). This does not happen if you use InnoDB Plugin in MySQL 5.1, but by default MySQL 5.1 is not setup with InnoDB Plugin but with the old build-in InnoDB.
I have a server with 12GB RAM, and max_heap_table_size in my.cnf is set to 6GB. ("max_heap_table_size=6442450944"). I restarted the MySQL server after setting this.
The trouble is, whenever my table gets to just 2GB during inserts I get error "table full". Why is it not letting me add more than 2GB worth of data? (The 2GB figure is what is shown as the size in phpMyAdmin)
A 32 bit MySQL server (or any 32 bit application for that matter) only have 2-3Gb(Depending on the OS, etc.) of virtual memory available, and thus can't address more memory. You need a 64 bit OS, and a 64 bit MySQL server to take advantage of more memory.