Recently my server CPU has been going very high.
Currently, My my.cnf file such as below:
[mysqld]
default-storage-engine=innodb
local-infile=0
skip-locking
max_connections=300
key_buffer = 128M
myisam_sort_buffer_size = 64M
join_buffer_size = 1M
read_buffer_size = 2M
sort_buffer_size = 2M
read_rnd_buffer_size = 2M
table_cache = 1024
record_buffer = 1M
thread_cache_size = 128
wait_timeout = 30
connect_timeout = 10
interactive_timeout = 10
tmp_table_size = 64M
max_heap_table_size = 64M
max_allowed_packet = 16M
max_connect_errors = 10
query_cache_limit = 1M
query_cache_size = 64M
query_cache_type = 1
[isamchk]
key_buffer=64M
sort_buffer=64M
read_buffer=16M
write_buffer=16M
[myisamchk]
key_buffer=64M
sort_buffer=64M
read_buffer=16M
write_buffer=16M
[mysqlhotcopy]
interactive-timeout
CPU Usage: u623.84 s78.87 cu0 cs0 - 8.12% CPU load
MYSQLD: 190%
My server information:
Intel® Pentium® E5400 2.70GHz 2M Cache
HDD: 250GB
RAM: 4GB
Please help me!
Set this variable (which is not in your my.cnf) to 70% of the RAM you have available:
innodb_buffer_pool_size=128M
Of course, if the CPU spikes are due to queries then you want to optimize those with indexes.
Related
I use vertx java to build an app and writing tests for mysql queries. I can successful get results before but after I modified my.cnf because of other stuffs. I cannot no long get returned results but "Fail to read any response from the server, the underlying connection might get lost unexpectedly". I searched google, but I cannot figure out exactly where is wrong and how to fix it.
I would post my.cnf file:
[client]
port = 3306
socket = /tmp/mysql.sock
# This was formally known as [safe_mysqld]. Both versions are currently parsed.
[mysqld_safe]
socket = /tmp/mysql.sock
nice = 0
syslog
[mysqld]
# Basic Settings
port = 3306
user = root
pid-file = /usr/local/mysql/data/mysqld.local.pid
socket = /tmp/mysql.sock
port = 3306
basedir = /usr/local/mysql/
datadir = /usr/local/mysql/data
tmpdir = /tmp
lc-messages-dir = /usr/local/mysql/share
skip-external-locking
# Engine
default-storage-engine = InnoDB
# Listening IP
bind-address = 0.0.0.0
# Safety
max-connect-errors = 1000000
max_allowed_packet = 64M
skip-name-resolve
sysdate-is-now = 1
innodb = FORCE
innodb-strict-mode = 1
tls_version = TLSv1,TLSv1.1,TLSv1.2
wait_timeout = 28800
interactive_timeout = 10000
# Buffers
sort_buffer_size = 4M
read_buffer_size = 2M
join_buffer_size = 8M
read_rnd_buffer_size = 16M
# MyISAM
key-buffer-size = 32M
# CACHES AND LIMITS #
tmp-table-size = 128M
max-heap-table-size = 128M
# Bin logs
binlog-format = ROW
log_bin = /tmp/binlog
max_binlog_size = 100M
server-id = 1 # randomize it incase of multiple servers
# InnoDB
innodb-buffer-pool-size = 2048M
innodb_buffer_pool_instances = 8
innodb_log_buffer_size = 8M
innodb-log-files-in-group = 2
innodb-log-file-size = 256M
innodb-file-per-table = 1
innodb-flush-log-at-trx-commit = 1
innodb-flush-method = O_DIRECT
# With virtual synchrony redundancy, make write queries faster
innodb_doublewrite = 1
# LOGGING
general_log_file = /usr/local/mysql/mysql.log
log-error = /usr/local/mysql/data/mysqld.local.err
log-queries-not-using-indexes = 1
[mysqldump]
quick
quote-names
max_allowed_packet = 64M
[isamchk]
key_buffer = 16M
#
# * IMPORTANT: Additional settings that can override those from this file!
# The files must end with '.cnf', otherwise they'll be ignored.
#
Can you tell me what's wrong with the configuration and what should I change? Thanks
I am using MySQL DB on Windows 2012 R2, x64 with 8 GB RAM. MySQL database many times shows the error - Out of Memory (needed 1048584 bytes). Below are the database Variable values i have set in my.ini file. Is there any change is required in this file?
[mysqld]
port= 3306
socket = "C:/xampp/mysql/mysql.sock"
basedir = "C:/xampp/mysql"
tmpdir = "C:/xampp/tmp"
datadir = "C:/xampp/mysql/data"
pid_file = "mysql.pid"
# enable-named-pipe
key_buffer_size = 16M
max_allowed_packet = 256M
table_cache = 256
net_buffer_length = 1M
read_buffer_size = 512K
read_rnd_buffer_size = 512K
myisam_sort_buffer_size = 8M
log_error = "mysql_error.log"
# Change here for bind listening
# bind-address="127.0.0.1"
# bind-address = ::1 # for ipv6
max_connect_errors = 500
max_connections = 5000
net_write_timeout = 1800
net_read_timeout = 1800
slow_launch_time = 10
slow_query_log = OFF
thread_cache_size = 2000
back_log = 100
performance_schema = OFF
query_cache_size = 512M
query_cache_limit = 5M
join_buffer_size = 4M
sort_buffer_size = 4M
max_heap_table_size = 64M
tmp_table_size = 128M
table_open_cache = 4500
table_definition_cache=4000
innodb_file_per_table=1
open_files_limit=50000
innodb_data_home_dir = "C:/xampp/mysql/data"
innodb_data_file_path = ibdata1:10M:autoextend
innodb_log_group_home_dir = "C:/xampp/mysql/data"
#innodb_log_arch_dir = "C:/xampp/mysql/data"
## You can set .._buffer_pool_size up to 50 - 80 %
## of RAM but beware of setting memory usage too high
innodb_buffer_pool_size = 2048M
innodb_buffer_pool_instances = 2
## Set .._log_file_size to 25 % of buffer pool size
innodb_log_file_size = 256M
innodb_log_buffer_size = 8M
innodb_flush_log_at_trx_commit = 2
innodb_lock_wait_timeout = 50
innodb_flush_method=O_DIRECT
innodb_io_capacity=1000
innodb_old_blocks_time=1000
innodb_open_files=5000
innodb_sort_buffer_size = 8M
Please reply if someone can manage to get this working again!
Thanks in advance.
Suggestions to consider for your my.ini [mysqld] section
Lead with # (or REMOVE) to allow defaults to serve you
. net_buffer_length
. read_buffer_size
. read_rnd_buffer_size
. MyISAM_sort_buffer_size
. join_buffer_size
. sort_buffer_size
. innodb_sort_buffer_size
max_connections=500 # from 5000 until you have a known need for more connections
thread_cache_size=100 # from 2000 per v8 refman CAP at 100 to avoid OOM
query_cache_type=0 # to avoid QC overhead
query_cache_size=0 # from 512M to conserve RAM for more useful purpose
query_cache_limit=0 # 0 is fine when not using QC
tmp_table_size=64M # from 128M until you can monitor created_tmp_disk_tables
tmp_table_size and max_heap_table_size should always be the same size.
Good luck.
For additional assistance please check my profile, clk Network Profile for contact info.
I am running version 10.0.16-MariaDB-1 (Debian GNU/Linux 8.1 (jessie)) on Dual Xeon CPU, total 32 cores with 128 GB of RAM. This is a dedicated DB server and has been recently migrated over from Percona DB server. My configuration looks like this,
################################################################################
# DATE: 19-01-2014
# DESCRIPTION: MySQL config
# RAM: 128GB RAM dedicated server
# Connections: 1000 connections
################################################################################
[mysql]
# CLIENT #
port = 1979
socket = /var/run/mysqld/mysqld.sock
[mysqld]
## Files
back_log = 300
open-files-limit = 8192
#open-files = 1024
# GENERAL #
user = mysql
default-storage-engine = InnoDB
port = 1979
socket = /var/run/mysqld/mysqld.sock
pid-file = /var/run/mysqld/mysql.pid
# DATA STORAGE #
datadir = /var/lib/mysql
# LOGGING #
log-error = /var/log/mysql/mysql-error.log
log-queries-not-using-indexes = 0
slow-query-log = 0
slow-query-log-file = /var/log/mysql/mysql-slow.log
## Per-Thread Buffers * (max_connections) = total per-thread mem usage
thread_stack = 512K #default: 32bit: 192K, 64bit: 256K
sort_buffer_size = 2M #default: 2M, larger may cause perf issues
read_buffer_size = 2M #default: 128K, change in increments of 4K
read_rnd_buffer_size = 2M #default: 256K
join_buffer_size = 2M #default: 128K
binlog_cache_size = 128K #default: 32K, size of buffer to hold TX queries
## total per-thread buffer memory usage: 17664000K = 17.250GB
## Query Cache
query_cache_size = 64M #global buffer
query_cache_limit = 2M #max query result size to put in cache
## Connections
max_connections = 900 #multiplier for memory usage via per-thread buffers
max_connect_errors = 1000 #default: 10
concurrent_insert = 2
connect_timeout = 10 #default -5.1.22: 5, +5.1.22: 10
max_allowed_packet = 32M #max size of incoming data to allow
## Default Table Settings
sql_mode = NO_AUTO_CREATE_USER
## Table and TMP settings
max_heap_table_size = 1G #recommend same size as tmp_table_size
bulk_insert_buffer_size = 1G #recommend same size as tmp_table_size
tmp_table_size = 1G #recommend 1G min
tmpdir = /dev/shm #Recommend using RAMDISK for tmpdir
## Table cache settings
#table_cache = 512 #5.0.x <default: 64>
#table_open_cache = 512 #5.1.x, 5.5.x <default: 64>
query-cache-type = 1
## Thread settings
thread_concurrency = 32 #recommend 2x CPU cores
thread_cache_size = 400 #recommend 5% of max_connections
## InnoDB Plugin Independent Settings
innodb_data_home_dir = /var/lib/mysql
innodb_data_file_path = ibdata1:128M;ibdata2:10M:autoextend
innodb_log_file_size = 768M #64G_RAM+ = 768, 24G_RAM+ = 512, 8G_RAM+ = 256, 2G_RAM+ = 128
innodb_log_files_in_group = 4 #combined size of all logs <4GB. <2G_RAM = 2, >2G_RAM = 4
innodb_buffer_pool_size = 96G #global buffer
innodb_additional_mem_pool_size = 8M #global buffer
innodb_status_file #extra reporting
innodb_file_per_table #enable always
innodb_flush_log_at_trx_commit = 2
innodb_table_locks = 0 #preserve table locks
innodb_log_buffer_size = 128M #global buffer
innodb_lock_wait_timeout = 600
#innodb_thread_concurrency = 0
innodb_commit_concurrency = 8 #recommend 4x num disks
innodb_flush_method = O_DIRECT_NO_FSYNC #O_DIRECT = local/DAS, O_DSYNC = SAN/iSCSI
innodb_support_xa = 0 #recommend 0, disable xa to negate extra disk flush
innodb_fast_shutdown = 1
skip-innodb-doublewrite
skip-host-cache
# skip-name-resolve
## Binlog sync settings
## XA transactions = 1, otherwise set to 0 for best performance
sync_binlog = 0
## TX Isolation
#transaction-isolation = REPEATABLE-READ #REPEATABLE-READ req for ACID, SERIALIZABLE req XA
## Per-Thread Buffer memory utilization equation:
#(read_buffer_size + read_rnd_buffer_size + sort_buffer_size + thread_stack + join_buffer_size + binlog_cache_size) * max_connections
## Replication
#server_id = 1
#log_bin = /var/log/mysql/mysql-bin.log
#log_bin_index = /var/log/mysql/mysql-bin.log.index
#relay_log = /var/log/mysql/mysql-relay-bin
#relay_log_index = /var/log/mysql/mysql-relay-bin.index
#expire_logs_days = 10
#max_binlog_size = 100M
#log_slave_updates = 1
#auto-increment-increment = 2
#auto-increment-offset = 1
## Global Buffer memory utilization equation:
# innodb_buffer_pool_size + innodb_additional_mem_pool_size + innodb_log_buffer_size + key_buffer_size + query_cache_size
long_query_time = 3
innodb_checksum_algorithm=NONE
interactive_timeout = 300
wait_timeout = 40
myisam_repair_threads = 8
myisam-block-size = 8096
innodb_adaptive_hash_index = off
innodb_use_sys_malloc
innodb_sync_array_size = 4
innodb_read_ahead_threshold = 56
innodb_purge_threads = 4
innodb_adaptive_flushing = off
innodb_use_native_aio = off
innodb_random_read_ahead
optimizer_search_depth = 24
table_definition_cache = 400
table_open_cache = 2048
delay-key-write = ALL
innodb_checksum_algorithm = innodb
innodb_write_io_threads = 64
innodb_read_io_threads = 64
thread_pool_size = 16
thread_handling = pool-of-threads
innodb_disable_sort_file_cache
innodb_buffer_pool_instances = 32
innodb_adaptive_hash_index = off
[mysqldump]
quick
quote-names
max_allowed_packet = 128M
I have multiple hosts connecting to the server and initiating queries. The problem I am facing is that randomly the DB server stops accepting connections. All connections start piling up and in around 3 or 5 seconds it comes back to normal. When I run show processlist I see a lot of connections piled up waiting on to be connected but since the DB server stops accepting connections, eventually the connection pool becomes full, no further connections are allowed.
During this blackout period, I am not able to see what's going on the DB server because i just won't give me any info even if the connection is established already. iostat -d -x 1 won't show anything unusual with the disks, CPU is also mostly idle. TCP connections piles up since the clients wants to open connections because they need to push data.
I am not able to figure out what's wrong here. Is this something specific to my DB, or the machine or the config. I need some help here on where and what to look for to identify and rectify the issue.
Thanks.
thread_stack = 512K #default: 32bit: 192K, 64bit: 256K
I have not heard of needing more than 256K. Did you discover a need for 512K? Together with
max_connections = 900
You potentially need 500MB for processes. (Granted, this is not a big deal since you have 128GB of ram.)
Back to the problem...
Turn on the slow_log with long_query_time = 1. After the next occurrence, run pt-query-digest against the slowlog. There may be some surprises there. Note that queries hung across the 3-5 second pause will register as taking several seconds, even if they should take only milliseconds. There's still a risk that it won't point the finger at the real cause of the Pause.
I am getting this error when I try starting the MySQL service through the XAMPP control panel:
2014-08-17 14:53:44 6028 [ERROR] TCP/IP, --shared-memory, or --named-pipe should be configured on NT OS
2014-08-17 14:53:44 6028 [ERROR] Aborting
Anyone ran into this issue before and have a configuration change that may be used to get it running?
For reference, these are the settings within my.ini:
[client]
port = 3306
socket = "G:/Dev/xampp/mysql/mysql.sock"
# The MySQL server
[mysqld]
port= 3306
socket = "G:/Dev/xampp/mysql/mysql.sock"
basedir = "G:/Dev/xampp/mysql"
tmpdir = "G:/Dev/xampp/tmp"
datadir = "G:/Dev/xampp/mysql/data"
pid_file = "mysql.pid"
# enable-named-pipe
key_buffer = 16M
max_allowed_packet = 1M
sort_buffer_size = 512K
net_buffer_length = 8K
read_buffer_size = 256K
read_rnd_buffer_size = 512K
myisam_sort_buffer_size = 8M
log_error = "mysql_error.log"
# SECURITY CHANGES
skip-networking
local-infile=0
plugin_dir = "G:/Dev/xampp/mysql/lib/plugin/"
skip-federated
server-id = 1
innodb_data_home_dir = "G:/Dev/xampp/mysql/data"
innodb_data_file_path = ibdata1:10M:autoextend
innodb_log_group_home_dir = "G:/Dev/xampp/mysql/data"
innodb_buffer_pool_size = 256M
innodb_additional_mem_pool_size = 2M
innodb_log_file_size = 32M
innodb_log_buffer_size = 8M
innodb_flush_log_at_trx_commit = 2
innodb_lock_wait_timeout = 50
innodb_thread_concurrency = 16
innodb_flush_method = normal
[mysqldump]
quick
max_allowed_packet = 16M
[mysql]
no-auto-rehash
[isamchk]
key_buffer = 20M
sort_buffer_size = 20M
read_buffer = 2M
write_buffer = 2M
[myisamchk]
key_buffer = 20M
sort_buffer_size = 20M
read_buffer = 2M
write_buffer = 2M
[mysqlhotcopy]
interactive-timeout
Either enable named pipes by removing the comment before the option enable-named-pipe
pid_file = "mysql.pid"
enable-named-pipe # this line
key_buffer = 16M
or let mysql listen on a TCP/IP port by commenting out the option skip-networking:
# SECURITY CHANGES
# skip-networking # this line
If networking is disabled and named pipes are not enabled then mysqld will useless.
With mysql 8 on windows, if you try to start it from command line with skip-grant-tables you will hit this error.
And the solution is adding shared-memory.
.\mysqld --skip-grant-tables --shared-memory
i was trying to enable slow_query_log. i have downloaded my.cnf file from server using WinSCP and added the code for enabling slow_query_log.
slow_query_log = ON
slow_query_log_file = /var/log/mysql_slow.log
And when i restart my server i am getting this error mysql has failed, please contact the sysadmin (result was "mysql is not running").
here i am adding my my.cnf file
i have tried with slow_query_log = 1 also. But getting the same error. when i remove the code which i have added the error is gone. Any one please help me...
my.cnf
[mysqld]
datadir=/home/mysql/
tmpdir=/home/mysqltmp
#max_connections = 175 #was 175
max_connections = 80
#max_connect_errors = 350 #was 250
max_connect_errors = 250
safe-show-database
skip-locking
key_buffer = 1024M # was 128M
max_allowed_packet = 6M
myisam_sort_buffer_size = 64M
#old settings, for 900 ish max maxconn
#sort_buffer_size = 32M
#read_buffer_size = 32M
#read_rnd_buffer_size = 32M
sort_buffer_size = 5M
read_buffer_size = 5M
read_rnd_buffer_size = 5M
query_cache_size= 1024M
query_cache_limit= 16M
max_heap_table_size = 128M
tmp_table_size = 128M
thread_concurrency = 16
wait_timeout = 10
innodb_file_per_table
innodb_log_file_size = 10485760
open_files_limit = 8192
low_priority_updates = 1
slow_query_log = ON
slow_query_log_file = /var/log/mysql_slow.log
# who set these? these are NOT memory settings, but rather integer settings.
#table_cache = 1024M
#thread_cache_size = 8M
table_cache = 512
thread_cache_size = 8
[mysqldump]
quick
max_allowed_packet = 16M
[mysql]
no-auto-rehash
[isamchk]
key_buffer = 128M
sort_buffer_size = 128M
read_buffer = 2M
write_buffer = 2M
[myisamchk]
key_buffer = 128M
sort_buffer_size = 128M
read_buffer = 2M
write_buffer = 2M
create the file mysql_slow.log in /var/log & change ownerhip of the file:
chown mysql.mysql -R /var/log/mysql_slow.log
either value (1 or ON) should work for the slow query log.
also, are you using regular statements?
check for ownership as mysql.mysql and permissions on file /var/log/mysql_slow.log
ls -al /var/log/mysql_slow.log