We have a set of PXC clusters, each with slaves that we are using for reads. The slaves are running percona server 5.7 with " max_connections" and "max_user_connections" set to 4000 and 4050 respectively. But every time we reboot our slaves, these values are automatically reset to the default values out of the box causing a lot of performance issues. Is this a bug, or are we missing something in our config?
Below is our config file (SSD):
#
# Default values.
[mysqld_safe]
flush_caches
numa_interleave
#
#
[mysqld]
back_log = 65535
binlog_format = ROW
character_set_server = utf8
collation_server = utf8_general_ci
#core_file
datadir = /var/lib/mysql
default_storage_engine = InnoDB
enforce-gtid-consistency = 1
expand_fast_index_creation = 1
expire_logs_days = 2
gtid_mode = ON
innodb_autoinc_lock_mode = 2
innodb_buffer_pool_instances = 64
innodb_buffer_pool_populate = 1
innodb_buffer_pool_size = 67G #77G
innodb_data_file_path = ibdata1:64M;ibdata2:64M:autoextend
innodb_file_format = Barracuda
innodb_file_per_table
#innodb_flush_log_at_trx_commit = 2
innodb_flush_log_at_trx_commit = 0
innodb_flush_method = O_DIRECT
innodb_io_capacity = 20000
innodb_large_prefix
innodb_locks_unsafe_for_binlog = 1
#innodb_log_file_size = 64M
innodb_log_file_size = 1G
innodb_print_all_deadlocks = 1
innodb_read_io_threads = 64
innodb_stats_on_metadata = FALSE
innodb_support_xa = FALSE
innodb_write_io_threads = 64
log-bin = mysqld-bin
#log-queries-not-using-indexes
log-slave-updates
long_query_time = 1
master_info_repository = TABLE
max_allowed_packet = 64M
max_connect_errors = 4294967295
max_connections = 4000
max_user_connections = 4050
min_examined_row_limit = 1000
port = 3306
read-only = 1
relay_log_info_repository = TABLE
relay-log-recovery = TRUE
skip-name-resolve
slave_parallel_workers = 8
slow_query_log = 1
slow_query_log_timestamp_always = 1
table_open_cache = 4096
thread_cache = 1024
tmpdir = /srv/tmp
transaction_isolation = REPEATABLE-READ
updatable_views_with_limit = 0
user = mysql
wait_timeout = 60
userstat
#innodb_buffer_pool_load_at_startup=1
#innodb_buffer_pool_dump_at_shutdown=1
#skip_slave_start
#
##for grafana dashboard monitoring
#query_response_time_stats = on
userstat = 1
server-id = 1019244
Goto /etc/my.cnf and set
max_connections=1000
Then restart mysql
Related
[mysqld] datadir = /mnt/mysql/lib/data socket =
/var/lib/mysql/mysql.sock user = mysql character-set-server = utf8
max_connections = 750 max_allowed_packet = 1G symbolic-links = 0
performance_schema server-id = 03082022 expire_log_days = 3
skip-name-resolve key_buffer_size = 12G innodb_buffer_pool_size = 5G
innodb_file_per_table = 1 innodb_data_file_path =
ibdata1:4G:autoextend innodb_log_file_size = 512M
innodb_thread_concurrency = 10 innodb_additional_mem_pool_size = 1024M
innodb_flush_log_at_trx_commit = 2 sort_buffer_size = 8M wait_timeout
= 60 sync_binlog = 0 innodb_io_capacity = 600 innodb_doublewrite = 0 event_scheduler = ON bingo-format = MIXED transaction-isolation =
READ_COMMITTED thread_cache_size = 750
when i set these and when i restart my server its failing
refer the website the issue will be solved no need of all these parameters
https://devopscube.com/setup-mysql-master-slave-replication/
My server config is this:
1- (Intel(R) Xeon(R) CPU E5-2673 v3 # 2.40GHz) x2
2- (Ram DDR4 8G) x16 128G total ram
And this is mysql configuration:
[mysqld]
# log-error=/var/lib/mysql/mysqld.err
# pid-file=/var/run/mysqld/mysqld.pid
innodb_undo_log_truncate = off
# general
table_open_cache = 200000
table_open_cache_instances = 64
back_log = 3500
max_connections = 100000
# files
innodb_file_per_table = ON # New
innodb_log_file_size = 16G # c
innodb_log_files_in_group = 2
innodb_open_files = 4000
# buffers
innodb_buffer_pool_size = 64G # c
innodb_buffer_pool_instances = 24
innodb_log_buffer_size = 64M
key_buffer_size = 64M
# tune
innodb_doublewrite = 1
innodb_thread_concurrency = 0
innodb_flush_log_at_trx_commit = 0
innodb_flush_method = O_DIRECT_NO_FSYNC
innodb_max_dirty_pages_pct = 90
innodb_max_dirty_pages_pct_lwm = 10
# 1innodb_lru_scan_depth = 2048
innodb_page_cleaners = 4
join_buffer_size = 512KB
sort_buffer_size = 512KB
innodb_use_native_aio = 1
#innodb_spin_wait_delay = 96
innodb_adaptive_flushing = 1
innodb_flush_neighbors = 0
innodb_read_io_threads = 16
innodb_write_io_threads = 16
innodb_io_capacity = 1500
innodb_io_capacity_max = 2500
innodb_purge_threads = 4
innodb_adaptive_hash_index = 0
max_prepared_stmt_count = 1000000
innodb_monitor_enable = '%'
performance_schema = ON
max_allowed_packet = 268435456
thread_handling = pool-of-threads
My website use huge data in join and sort and always at least 200 visitors online.
Response is too low speed and cpu usage more than 200%
mysql 0 CPU:220.53% Memory:22.08% /usr/sbin/mysqld
I think my Mysql configuration is wrong.
What should i do?
We have two mariadb servers Master and Slave each on PR and DR, We are observing high disk io above 90% and iowait some time above 20, when mariadb backup start at night iowait goes above 30 and system become unresponsive causing watch dog process timeout and restart mariadb. when running dd command
sudo dd if=/dev/zero of=/data/test2.img bs=512 count=1000 oflag=dsync
it write data between 300 to 500 Kb/s, we have checked storage level as well but no issue found at storage level as same storage has been in used for other DB's (Oracle) and no such IO issues there. i need help to identify this issue.
following is server.cnf of mariadb
[mysqld]
symbolic_links = 0
local_infile = 0
basedir = /usr
datadir = /data/mdb_data
pid_file = /var/lib/mysql/mysqld.pid
log_error = /var/log/mariadb/mysqld.log
bind-address = ::
port = 3306
userstat = 1
plugin-load-add = server_audit=server_audit.so
server_audit = FORCE_PLUS_PERMANENT
server_audit_logging = on
server_audit_events = CONNECT,QUERY_DCL,QUERY_DDL
server_audit_output_type = syslog
cracklib_password_check = off
log_bin_trust_function_creators = 1
lower_case_table_names = 1
character-set-server = utf8
init_connect = SET NAMES utf8
innodb_file_per_table = 1
innodb_flush_log_at_trx_commit = 1
skip-name-resolve
innodb_autoinc_lock_mode = 2
max_connections = 3000
innodb_buffer_pool_size = 58G
innodb_buffer_pool_instances = 8
innodb_log_file_size = 1024m
key_buffer_size = 16M
log-error = /var/log/mariadb/mysqld.log
skip-external-locking
slow_query_log = on
long_query_time = 1
slow_query_log_file = /var/log/mariadb/slowSQLs.log
performance_schema = on
innodb_flush_method = O_DIRECT
sync_binlog = 1
max_allowed_packet = 512m
slave_max_allowed_packet = 200m
innodb_read_io_threads = 32
innodb_write_io_threads = 32
innodb_io_capacity = 600
slave_parallel_threads = 50
slave_parallel_max_queued = 2097152
slave_parallel_mode = optimistic
rpl_semi_sync_master_enabled = 0
rpl_semi_sync_slave_enabled = 1
rpl_semi_sync_master_wait_point = AFTER_SYNC
binlog_commit_wait_usec = 10000
binlog_commit_wait_count = 1
server_audit_excl_users = repl#b.c,maxscale
slave-skip-errors = 1062,1032
[mysqld_safe]
syslog
[mariadb]
server_id = 228575792
log_slave_updates = on
log-bin
log-basename = db2
report_host = db2
from top
KiB Mem : 74046272 total, 32528492 free, 26723664 used, 14794120 buff/cache
KiB Swap: 33554428 total, 31482364 free, 2072064 used. 33507036 avail Mem
134359 [Note] Got an error reading communication packets
Appears in logs every 5 minutes, I have seen a few similar posts on here however none actually match the error I'm seeing here.
MySQL 5.7 - configuration:
[mysqld]
##skip-name-resolve=1
#bind-address=localhost
default-storage-engine=InnoDB
innodb_file_per_table=1
performance-schema=1
server-id=2
auto_increment_offset = 2
auto_increment_increment= 5
report-host=2
log-slave-updates
##
replicate-ignore-db=mysql
replicate-ignore-db=cphulkd
replicate-ignore-db=information_schema
replicate-ignore-db=performance_schema
replicate-ignore-db=sys
##
max_connections=1000
table_open_cache=2000
wait_timeout=86400
interactive_timeout=86400
net_read_timeout=28800
net_write_timeout=28800
table_definition_cache=2000
thread_cache_size=100
long_query_time=10000
log_queries_not_using_indexes=0
innodb_page_cleaners=128
sql_mode=""
max_allowed_packet=512M
local-infile=0
[mysql]
# CLIENT #
port = 3306
socket = /var/lib/mysql/mysql.sock
[mysqld]
# GENERAL #
user = mysql
default-storage-engine = InnoDB
socket = /var/lib/mysql/mysql.sock
pid-file = /var/lib/mysql/mysql.pid
# MyISAM #
key-buffer-size = 10M
##myisam-recover = FORCE,BACKUP
# SAFETY #
max-allowed-packet = 10000M
max-connect-errors = 1000000
# DATA STORAGE #
datadir = /var/lib/mysql/
# BINARY LOGGING #
log-bin = /var/lib/mysql/mysql-bin
expire-logs-days = 14
sync-binlog = 0
# CACHES AND LIMITS #
tmp-table-size = 100M
max-heap-table-size = 100M
query-cache-type = 0
query-cache-size = 0
#max-connections = 1000
thread-cache-size = 50
open-files-limit = 1024
table-definition-cache = 4096
table_open_cache = 2024
# INNODB #
innodb-flush-method = O_DIRECT
innodb-log-files-in-group = 2
innodb-log-file-size = 10G
innodb_log_buffer_size = 5G
innodb-flush-log-at-trx-commit = 1
innodb-file-per-table = 1
innodb-buffer-pool-size = 80G
innodb_buffer_pool_instances = 64
innodb_write_io_threads = 64
innodb_read_io_threads = 64
innodb_max_dirty_pages_pct = 90
innodb_io_capacity = 1000
innodb_stats_on_metadata = 0
innodb_adaptive_flushing = 1
innodb_lru_scan_depth = 1024
# LOGGING #
log-error = /var/lib/mysql/mysql-error.log
log-queries-not-using-indexes = 0
slow-query-log = 0
slow-query-log-file = /var/lib/mysql/mysql-slow.log
join_buffer_size = 55M
binlog_cache_size = 16M
key_buffer_size = 50M
The thread ID 134359 doesn't correspond to any actual threads if I enable General MySQL Logging, nor does anything appear in a tcpdump during the events occuring.
At a loss? I do feel it's something CPanel or WHM related
I am having load problems on my database server (dual quad core 50 gb ram)
load average: 4.26, 6.78, 11.25
here is the current my.cnf, can i optimize it?
[client]
port = 3306
socket = /var/run/mysqld/mysqld.sock
[mysqld_safe]
socket = /var/run/mysqld/mysqld.sock
nice = 0
[mysqld]
user = mysql
pid-file = /var/run/mysqld/mysqld.pid
socket = /var/run/mysqld/mysqld.sock
port = 3306
basedir = /usr
datadir = /var/lib/mysql
tmpdir = /tmp
language = /usr/share/mysql/english
skip-external-locking
key_buffer = 150M
key_buffer_size = 512M
max_allowed_packet = 16M
thread_stack = 128K
thread_cache_size = 128
open-files = 8192
myisam-recover = BACKUP
max_connections = 1000
table_cache = 8096
join_buffer_size = 2M
tmp_table_size = 10G
max_heap_table_size = 10G
innodb_buffer_pool_size = 10G
innodb_additional_mem_pool_size = 128M
innodb_log_file_size = 100M
log-slow-queries = /var/log/mysql/slow.log
sort_buffer_size = 5M
net_buffer_length = 5M
read_buffer_size = 2M
read_rnd_buffer_size = 12M
thread_concurrency = 16
ft_max_word_len = 30
ft_min_word_len = 2
#thread_concurrency = 10
query_cache_limit = 4M
query_cache_size = 512M
The website has about 2000 users online.
I always start out with a watch "mysqladmin pr -vv" to see which statements are the worst... then you should get an idea what variable to adjust ... most of the time it's a missing index.
You use mytop and mysqltuner to analyze using mysql resources (RAM and CPU).
Too enable the option to log slow queries:
log_slow_queries = /var/log/mysql/mysql-slow.log
long_query_time = 3
And check out this post about ntpd service:
MySQL high CPU usage
I have a dedicated server for a high rate of transactions:
max_allowed_packet=16M
key_buffer_size=8M
innodb_additional_mem_pool_size=10M
innodb_buffer_pool_size=512M
join_buffer_size=40M
table_open_cache=1024
query_cache_size=40M
table_definition_cache=256
innodb_additional_mem_pool_size=10M
key_buffer_size=16M
max_allowed_packet=32M
max_connections = 300
query_cache_limit = 10M
log_slow_queries = /var/log/mysql/mysql-slow.log
long_query_time = 3
Regards.