I have MySQL 5.6 (InnoDB) installed on Win 2008 server with 1Gb. mysqld uses 415Mb memory. How to reduce memory usage? I read that it is possible to do by configuring my.ini: key_buffer_size, innodb_buffer_pool_size, etc. Please, tell me the correct settings to minimize memory usage.
You can check Configuring MySQL to use minimal memory:
# /etc/my.cnf:
innodb_buffer_pool_size=5M
innodb_log_buffer_size=256K
query_cache_size=0
max_connections=10
key_buffer_size=8
thread_cache_size=0
host_cache_size=0
innodb_ft_cache_size=1600000
innodb_ft_total_cache_size=32000000
# per thread or per operation settings
thread_stack=131072
sort_buffer_size=32K
read_buffer_size=8200
read_rnd_buffer_size=8200
max_heap_table_size=16K
tmp_table_size=1K
bulk_insert_buffer_size=0
join_buffer_size=128
net_buffer_length=1K
innodb_sort_buffer_size=64K
#settings that relate to the binary log (if enabled)
binlog_cache_size=4K
binlog_stmt_cache_size=4K
Related
What are the recommended configurations for allocating RAM to MySQL?
My environment: Ubuntu 14 machine with 96G RAM, MySQL Ver 14.14 Distrib 5.7.
And how to set those configurations?
Thank you in advance!
Assuming you are are using InnoDB as engine than please see here https://www.percona.com/blog/2013/09/20/innodb-performance-optimization-basics-updated/
MySQL Innodb Settings
From 5.5 InnoDB is the default engine, so these parameters are even more important for performance than before. The most important ones are:
innodb_buffer_pool_size: InnoDB relies heavily on the buffer pool and should be set correctly, so be sure to allocate enough memory to it. Typically a good value is 70%-80% of available memory. More precisely, if you have RAM bigger than your dataset setting it bit larger should be appropriate with that keep in account of your database growth and re-adjust innodb buffer pool size accordingly. Further, there is improvement in code for InnoDB buffer scalability if you are using Percona Server 5.1 or Percona Server 5.5 You can read more about it here.
innodb_buffer_pool_instances: Multiple innodb buffer pools introduced in InnoDB 1.1 and MySQL 5.5. In MySQL 5.5 the default value for it was 1 which is changed to 8 as new default value in MySQL 5.6. Minimum innodb_buffer_pool_instances should be lie between 1 (minimum) & 64 (maximum). Enabling innodb_buffer_pool_instances is useful in highly concurrent workload as it may reduce contention of the global mutexes.
Dump/Restore Buffer Pool: This feature speed up restarts by saving and restoring the contents of the buffer pool. This feature is first introduced in Percona Server 5.5 you can read about it here. Also Vadim benchmark this feature You can read more about it in this post. Oracle MySQL also introduced it in version 5.6, To automatically dump the database at startup and shutdown set innodb_buffer_pool_dump_at_shutdown & innodb_buffer_pool_load_at_startup parameters to ON.
innodb_log_file_size: Large enough InnoDB transaction logs are crucial for good, stable write performance. But also larger log files means that recovery process will slower in case of crash. However this is not such big issue since great improvements in 5.5. Default value has been changed in MySQL 5.6 to 50 MB from 5 MB (old default), but it’s still too small size for many workloads. Also, in MySQL 5.6, if innodb_log_file_size is changed between restarts then MySQL will automatically resize the logs to match the new desired size during the startup process. Combined log file size is increased to almost 512 GB in MySQL 5.6 from 4 GB. To get the optimal logfile size please check this blog post.
innodb_log_buffer_size: Innodb writes changed data record into lt’s log buffer, which kept in memory and it saves disk I/O for large transactions as it not need to write the log of changes to disk before transaction commit. 4 MB – 8 MB is good start unless you write a lot of huge blobs.
innodb_flush_log_at_trx_commit: When innodb_flush_log_at_trx_commit is set to 1 the log buffer is flushed on every transaction commit to the log file on disk and provides maximum data integrity but it also has performance impact. Setting it to 2 means log buffer is flushed to OS file cache on every transaction commit. The implication of 2 is optimal and improve performance if you are not concerning ACID and can lose transactions for last second or two in case of OS crashes.
innodb_thread_concurrency: With improvements to the InnoDB engine, it is recommended to allow the engine to control the concurrency by keeping it to default value (which is zero). If you see concurrency issues, you can tune this variable. A recommended value is 2 times the number of CPUs plus the number of disks. It’s dynamic variable means it can set without restarting MySQL server.
innodb_flush_method: DIRECT_IO relieves I/O pressure. Direct I/O is not cached, If it set to O_DIRECT avoids double buffering with buffer pool and filesystem cache. Given that you have hardware RAID controller and battery-backed write cache.
innodb_file_per_table: innodb_file_per_table is ON by default from MySQL 5.6. This is usually recommended as it avoids having a huge shared tablespace and as it allows you to reclaim space when you drop or truncate a table. Separate tablespace also benefits for Xtrabackup partial backup scheme.
I am trying many problems with MySQL, with high memory usage and especially with high CPU usage.
I have a dedicated server with the following configuration:
8 CPU Intel(R) Xeon(R) CPU E3-1240 v6 # 3.70GHz
16GB DDR3
SO Linux with cPanel/WHM + MySQL
Following is my.cnf file:
[mysqld]
max_connections=1000
wait_timeout=1000
interactive_timeout=1000
long_query_time=50
slow_query_log = 0
#slow_query_log_file = /var/log/mysql/slow-query.log
default-storage-engine=MyISAM
log-error=/var/lib/mysql/dc.leilaoweb.com.err
max_allowed_packet=268435456
local-infile=0
event_scheduler = on
tmp_table_size=300M
max_heap_table_size=128M
open_files_limit=65000
performance-schema=1
innodb_file_per_table=1
innodb_log_file_size=512M
innodb_buffer_pool_size=8G
key_buffer_size=512M
innodb_buffer_pool_instances=8
# config cache
query_cache_limit=8M
query_cache_size=256M
query_cache_type=1
table_open_cache=6000
table_cache=5000
thread_cache_size=96
#bind-address=127.0.0.1
#skip-networking
#performance_schema=ON
skip-name-resolve
How could I improve this setting to make queries faster and not raise server load?
It's a funny question that asks for help with query optimization, in which no specific query is mentioned.
Here are some tips on configuration options:
default-storage-engine=MyISAM
Change the default storage engine to InnoDB, and make sure all your existing tables are InnoDB. Don't use MyISAM.
query_cache_size=256M
query_cache_type=1
Set the query cache size and type to 0. The query cache is useful in such rare conditions that it has become deprecated, and removed in MySQL 8.0. It's better to cache query results in your application code, on a case-by-case basis.
innodb_buffer_pool_size=8G
If you have a lot more data than 8G, consider increasing the size of the buffer pool. The more of your data and indexes that resides in RAM, the better it will be for performance. But there's no further benefit to adding RAM once your data and indexes are 100% cached in the buffer pool.
And of course do not overallocate the buffer pool such that it causes the server to start swapping. That will kill performance (or else Linux's OOM killer will terminate mysqld if you have no swap).
key_buffer_size=512M
No need for extra memory allocated to the key buffer if you don't use MyISAM.
There may be other tuning parameters that can give benefit, but since you have said nothing about your queries or server activity, there's no way to guess what those would be.
You're better off focusing on indexes and query design.
In general, optimization naturally improves some queries at the expense of other queries. So you can make an optimization strategy only after you know which queries you need to optimize for.
I have a server (MS Windows Server 2012 R2 Datacenter 64GB RAM 2TB+ disk space) running mySQL 5.0. When I start the mySQL server, right off the bat it allocates 214,000 handles. Is that normal? I've been looking into this because I am trying to run an application that executes multiple unique queries over thousands of records and it is just crawling.
I have changed query_cache_size from 160M to 0M in the my.ini file as query caching will not benefit this application. Still no change in handles. I'm not sure what else I can do to fix this. Does anyone have any ideas?
The server is:
MySQL 5.0.60sp1-enterprise-gpl-nt
There are a ton of options. Here are what I think are the relevant ones (I could be wrong I am not an expert)
[mysqld]
default_storage_engine=InnoDB
innodb_file_per_table
innodb_flush_method=unbuffered
lower_case_table_names=2
max_allowed_packet=48M
max_heap_table_size=64777216
max_connections=3010
query_cache_size=0M
table_cache=6020
tmp_table_size=16M
thread_cache_size=64
myisam_max_sort_file_size=100G
myisam_max_extra_sort_file_size=100G
key_buffer_size=20M
read_buffer_size=64K
read_rnd_buffer_size=256K
innodb_additional_mem_pool_size=15M
innodb_flush_log_at_trx_commit=1
innodb_buffer_pool_size=709M
innodb_thread_concurrency=50
We are not able to create a stable configuration of a production server with MySQL and Tomcat Application Server. The MySQL throws very often an error:
MySQL: Out of memory (Needed 429496728 bytes)
It was a Windows Server 2012 with 64 GB of RAM. We see in the process tab, that the MySQLd commited 64 GB of RAM (that means all the available RAM in the server).
For your explaination, the Application uses completly inMemory based Tables in the MySQL Server.
See the my.ini Configuration
[client]
port=3306
[mysql]
default-character-set=utf8
[mysqld]
port=3306
basedir="F:/MySQL Server 5.6/"
datadir="F:/MySQL Server 5.6/data/"
character-set-server=utf8
default-storage-engine=MyISAM
sql-mode="STRICT_TRANS_TABLES,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION"
max_connections=200
query_cache_size=0
# table_cache=256
tmp_table_size=6G
max_heap_table_size=6G
max_tmp_tables=2048
open_files_limit=40000
thread_cache_size=8
myisam_max_sort_file_size=100G
myisam_sort_buffer_size=32M
myisam_use_mmap=0
concurrent_insert=2
key_buffer_size=2G
read_buffer_size=512M
read_rnd_buffer_size=512M
sort_buffer_size=128M
bulk_insert_buffer_size=32M
#skip-innodb
innodb_additional_mem_pool_size=15M
innodb_flush_log_at_trx_commit=1
innodb_log_buffer_size=7M
innodb_buffer_pool_size=686M
innodb_log_file_size=343M
innodb_thread_concurrency=34
slow_query_log=1
log-queries-not-using-indexes=1
long_query_time=1
log-output=FILE,TABLE
slow_query_log_file="F:/MySQL Server 5.6/localhost-slow-query.log"
event-scheduler=ON
Does anybody have a suitable solution to get this fixed?
We also use the MySQL Tuner to get informations, here the results:
What I cannot attach is the MySQL Tuner results as image (too few reputations). But look here.
Up for 39days
Data in Memory Tables 42G (Tables:281)
Data in MyISAM tables 31G (Tables 668)
total buffers: 2.3G global + 144.5M per Thread (200 max threads)
maximum possible memory usage: 30.5G (47% of installed RAM)
Key buffer size /total MyISAM indixes: 640M/7.0G
query cache disabled
joins performed without indixes: 337737
I'm not sure if stack is the right place to ask this, but I recently upgraded to Percona 5.6 from 5.5 and my memory usage has skyrocketed!
This is from PS:
mysql 4598 0.0 29.5 1583356 465312 ? Sl Oct17 9:07 /usr/sbin/mysqld --basedir=/usr --datadir=/var/lib/mysql --plugin-dir=/usr/lib6
I'm on a dedicated VSS
My server only has a gig of ram...how is this only memory usage only 30% according to PS?
I have my ram set in the config to be less than this, and when I run MySQLTuner i get:
[OK] Maximum possible memory usage: 338.9M (22% of installed RAM)
So how am I using almost 500MB of physical memory and over a gig and a half of virtual?
Is this a bug in mySQL or something with my server?
found out that in mysql 5.6 performance_schema is on by default. It was disabled by default prior in 5.5 and before. It has been enabled by default since 5.6.6
performance_schema=off to my config file fixes the issue.
I'd imagine anyone who doesn't have the memory to be running performance_schema would not be using it anyway.
This might affect other distros of mysql 5.6.6 as well.
I had this problem and fixing a few (increased) cache values in MySQL.ini sorted the problem out.
table_definition_cache - set to 400
From "http://bugs.mysql.com/bug.php?id=68287" where this is discussed
Yes, there are thresholds based on table_open_cache and table_definition_cache and max_connections and crossing the thresholds produces a big increase in RAM used. The thresholds work by first deciding if the server size is small, medium or large.
Small: all three are same as or less than defaults (2000, 400, 151).
Large: any of the three is more than twice the default.
Medium: others.
From memory. mine was set to 2000+ and dropping it sorted the problem.
What helped me was on CentOS was changing the memory allocator:
yum install jemalloc-devel
and add to my.cnf:
[mysqld_safe]
malloc-lib = /usr/lib64/libjemalloc.so.1