I have problem with my production mysql server. Usage of memory still grow up and I don`t know why. Trouble started when we changed the server.
My mysql version: 5.5.44-0+deb8u1-log - (Debian).
My my.cnf file:
[mysqld_safe]
pid-file = /var/run/mysqld/mysqld.pid
socket = /var/run/mysqld/mysqld.sock
nice = 0
[mysqld]
skip-external-locking
key_buffer_size = 5M
max_allowed_packet = 16M
thread_stack = 192K
thread_cache_size = 8
tmp_table_size = 384M
max_heap_table_size = 384M
table_open_cache = 7000
open_files_limit = 14000
interactive_timeout=3600
wait_timeout=3600
myisam-recover_options = BACKUP
max_connections = 150
query_cache_limit = 8M
query_cache_size = 127M
slow_query_log_file = /var/log/mysql/mysql-slow.log
slow_query_log = 1
long_query_time = 2
expire_logs_days = 10
max_binlog_size = 100M
innodb_file_per_table
innodb_buffer_pool_instances = 9
innodb_buffer_pool_size = 10000M
innodb_log_file_size = 2000M
It is something wrong in my configuration?
EDIT
For production we have dedicated server:
4 x Intel(R) Xeon(R) CPU E5-2680 v2 # 2.80GHz
20 GB RAM memory
40 GB na DHH - data base have now 15 GB
in the data base is on this moment 650 tables on the innoDB engine
Screenshot of htop:
htop of proces on server
Related
We are running multiple scripts that calculates data. The script was written in PHP(Laravel) and they were triggered by a cron(every minute). What I have noticed is when the cron triggers more than 300 of those process, mysql crashes "Connection Refused". This issue doesn't happen when there are less than 300 processes. I already have increased max_connections to 1000 and back_log t0 500 in my.cnf but the issue still persists(of course MySQL service was restarted). I have seen someone say that back_log is also at OS level but I can't find any article on how to adjust it. Any thoughts?
Here's some config values:
max_connections = 1000
back_log = 500
connect_timeout = 5
wait_timeout = 600
max_allowed_packet = 64M
thread_cache_size = 256
sort_buffer_size = 128M
bulk_insert_buffer_size = 128M
tmp_table_size = 128M
max_heap_table_size = 1G
myisam_recover_options = BACKUP
key_buffer_size = 128M
#open-files-limit = 2000
table_open_cache = 400
myisam_sort_buffer_size = 512M
concurrent_insert = 2
read_buffer_size = 8M
read_rnd_buffer_size = 4M
innodb_buffer_pool_size = 80G
innodb_log_buffer_size = 256M
innodb_file_per_table = 1
innodb_open_files = 400
innodb_io_capacity = 400
innodb_flush_method = O_DIRECT
I think it could be a performance issue, Did you monitor the mysql server cpu usage?
I use mysql 5.5 and centos 7. Some tables are InnoDB in my database and i optimized my.cnf file with i read articles in the internet. I use TRANSACTION and COMMIT. But insert and update very slow you can see.
I can't use MyISAM because these tables always get insert, update and read same time too much.
QUERY PHOTO
My.cnf file
# The following options will be passed to all MySQL clients
[client]
port = 3306
socket = /var/lib/mysql/mysql.sock
[mysqld]
#innodb_force_recovery=6
user = mysql
default-storage-engine = InnoDB
socket = /var/lib/mysql/mysql.sock
pid-file = /var/lib/mysql/mysql.pid
datadir=/var/lib/mysql/
log-error=/var/lib/mysql/server.mysql.err
symbolic-links=0
tmpdir=/var/tmp
skip-external-locking
table_cache = 2000
key_buffer_size=20G
join_buffer_size = 4M
sort_buffer_size = 2M
read_buffer_size = 2M
read_rnd_buffer_size = 1M
myisam_sort_buffer_size=2M
thread_cache_size = 512
query_cache_limit = 1G
query_cache_size = 40M
query_cache_type=1
thread_stack = 256K
tmp_table_size = 128M
max_heap_table_size = 128M
open_files_limit=65535
#thread_concurrency = 10
max_connect_errors=1
connect_timeout=60
interactive_timeout = 60
lock_wait_timeout=60
wait_timeout = 30
max_connections = 1000
slow_query_log=1
long_query_time=1
slow-query-log-file=/var/log/mysql-slow.log
#log-queries-not-using-indexes
innodb_buffer_pool_size=32G
innodb_additional_mem_pool_size=64M
innodb_data_file_path=ibdata1:100M:autoextend
innodb_log_buffer_size=128M
innodb-log-files-in-group = 2
innodb_change_buffering=all
innodb_flush_method=O_DIRECT
innodb_flush_log_at_trx_commit=2
#innodb_thread_concurrency=10
innodb_read_io_threads = 64
innodb_write_io_threads = 64
innodb_file_per_table=1
innodb_lock_wait_timeout = 60
innodb_table_locks=0
innodb_open_files=65535
innodb_io_capacity=2000
#innodb_doublewrite=0
#innodb_support_xa=0
[mysqldump]
max_allowed_packet=2G
quick
[mysql]
no-auto-rehash
[isamchk]
key_buffer = 128M
sort_buffer_size = 4M
read_buffer = 4M
write_buffer = 4M
[myisamchk]
tmpdir=/tmp
key_buffer_size=128M
sort_buffer_size=4M
read_buffer=4M
write_buffer=4M
[mysqlhotcopy]
interactive-timeout
And my server properties are
CPU
Intel(R) Xeon(R) CPU E5-1620 v2 # 3.70GHz
Cores : 8
Cache : 10240KB
RAM
64 GB
Disks
3 x 160 GB SSD
query_cache_limit = 1G -- should be less than the size.
query_cache_size = 40M -- this is reasonable.
key_buffer_size=20G -- too big; change to 6G. Note: only used for MyISAM indexes.
-- if you are not using MyISAM, then drop to 10M
slow_query_log=1 --
What does pt-query-digest point out as the 'worst' 3 queries? Show us them, together with SHOW CREATE TABLE and (if SELECT), EXPLAIN SELECT ....
Its not necessary that you have to tune optimized my.cnf file if your queries working slow. You might have to optimize your queries by proper indexes and joins.
as you may see ,for this server the CPU can be use for up to 20%.it would not raise even mysql start to stuck. this server runs mysql ONLY because such php nginx etc is in other server.i assume there is some error in the my.conf,please help.
screenshot after type "top" in centos is as flow
about hardware ,it is dual Xeon E5-2620 V2. 32G RAM.
cat /proc/cpuinfo | grep name | cut -f2 -d: | uniq -c
24 Intel(R) Xeon(R) CPU E5-2620 v2 # 2.10GHz
cat /proc/cpuinfo | grep "physical"| sort |uniq -c
24 address sizes : 46 bits physical, 48 bits virtual
12 physical id : 0
12 physical id : 1
here is my.conf(mysql 5.6.14)
[mysqld]
port = 3306
;socket = /tmp/mysql.sock
log-error=/var/log/mysql-error.log
slow_query_log=1
;log_queries_not_using_indexes=1
;log-slow-queries=/var/log/mysql-slow.log
;log_slow_queries=1
;slow_query_log_file=/var/log/mysql-slow.log
;long_query_time=10
log-output=FILE
slow_query_log_file=/var/log/mysql-slow.log
;long-query-time=10
skip-external-locking
table_open_cache = 8192
max_allowed_packet = 1024M
table_definition_cache = 1024
max_connections = 1000
max_connect_errors = 10
max_heap_table_size =64M
sort_buffer_size = 8M
join_buffer_size = 1M
thread_cache_size = 64
thread_concurrency = 24
query_cache_size = 256M
query_cache_limit = 4M
tmp_table_size = 1024M
key_buffer_size = 5G
read_buffer_size = 2M
read_rnd_buffer_size = 16M
bulk_insert_buffer_size= 64M
myisam_sort_buffer_size = 128M
myisam_max_sort_file_size = 256M
myisam_repair_threads = 1
myisam_recover
innodb_additional_mem_pool_size = 1024M
innodb_log_buffer_size = 256M
innodb_log_file_size = 1024M
innodb_buffer_pool_size = 10G
[mysqlhotcopy]
interactive-timeout
Thanks for your help!
I have a web server with Apache and MySQL running on AWS EC2 t2.small with Windows 2012 Server. AWS EC2 t2.small characteristics:
RAM 2 GB (used 65%)
1 CPU 2.50 GHz (used 1%)
Now MySQL process (mysqld.exe) uses 400 MB of RAM (too much for me).
MySQL current settings are (my.ini):
key_buffer = 16M
max_allowed_packet = 16M
sort_buffer_size = 512K
net_buffer_length = 8K
read_buffer_size = 256K
read_rnd_buffer_size = 512K
myisam_sort_buffer_size = 8M
tmp-table-size = 32M
max-heap-table-size = 32M
max-connections = 500
thread-cache-size = 50
open-files-limit = 65535
table-definition-cache = 1024
table-open-cache = 2048
query_cache_type = 1
query_cache_limit = 256K
query_cache_min_res_unit = 2k
query_cache_size = 80M
innodb-log-files-in-group = 2
innodb-log-file-size = 64M
innodb-flush-log-at-trx-commit = 1
innodb-file-per-table = 1
innodb_buffer_pool_size = 128M
innodb_additional_mem_pool_size = 2M
innodb_log_file_size = 5M
innodb_log_buffer_size = 8M
innodb_flush_log_at_trx_commit = 1
innodb_lock_wait_timeout = 50
Database is formed by 20 InnoDB tables and they are composed with 5/10 columns. The server has a low traffic.
How can I optimize my settings to be suitable with EC2 t2.small (2GB RAM)?
You have innodb_buffer_pool_size twice in your config. It should be with underscores, but check which one gets used with:
show variables like 'innodb_buffer_pool_size';
You could try halving innodb_buffer_pool_size and query_cache_size. Try if performance is ok with query_cache_size=0 too.
I am having load problems on my database server (dual quad core 50 gb ram)
load average: 4.26, 6.78, 11.25
here is the current my.cnf, can i optimize it?
[client]
port = 3306
socket = /var/run/mysqld/mysqld.sock
[mysqld_safe]
socket = /var/run/mysqld/mysqld.sock
nice = 0
[mysqld]
user = mysql
pid-file = /var/run/mysqld/mysqld.pid
socket = /var/run/mysqld/mysqld.sock
port = 3306
basedir = /usr
datadir = /var/lib/mysql
tmpdir = /tmp
language = /usr/share/mysql/english
skip-external-locking
key_buffer = 150M
key_buffer_size = 512M
max_allowed_packet = 16M
thread_stack = 128K
thread_cache_size = 128
open-files = 8192
myisam-recover = BACKUP
max_connections = 1000
table_cache = 8096
join_buffer_size = 2M
tmp_table_size = 10G
max_heap_table_size = 10G
innodb_buffer_pool_size = 10G
innodb_additional_mem_pool_size = 128M
innodb_log_file_size = 100M
log-slow-queries = /var/log/mysql/slow.log
sort_buffer_size = 5M
net_buffer_length = 5M
read_buffer_size = 2M
read_rnd_buffer_size = 12M
thread_concurrency = 16
ft_max_word_len = 30
ft_min_word_len = 2
#thread_concurrency = 10
query_cache_limit = 4M
query_cache_size = 512M
The website has about 2000 users online.
I always start out with a watch "mysqladmin pr -vv" to see which statements are the worst... then you should get an idea what variable to adjust ... most of the time it's a missing index.
You use mytop and mysqltuner to analyze using mysql resources (RAM and CPU).
Too enable the option to log slow queries:
log_slow_queries = /var/log/mysql/mysql-slow.log
long_query_time = 3
And check out this post about ntpd service:
MySQL high CPU usage
I have a dedicated server for a high rate of transactions:
max_allowed_packet=16M
key_buffer_size=8M
innodb_additional_mem_pool_size=10M
innodb_buffer_pool_size=512M
join_buffer_size=40M
table_open_cache=1024
query_cache_size=40M
table_definition_cache=256
innodb_additional_mem_pool_size=10M
key_buffer_size=16M
max_allowed_packet=32M
max_connections = 300
query_cache_limit = 10M
log_slow_queries = /var/log/mysql/mysql-slow.log
long_query_time = 3
Regards.