Optimizing MariaDB for wordpress - mysql

I have a server with 2 CPU cores and 1GB of RAM.The server only run one wordpress site.My Server Stack is LEMP.I ran mysql tuner two weeks after setting up the wordpress site.
Here are the results
[!!] Maximum reached memory usage: 884.8M (89.15% of installed RAM)
[!!] Maximum possible memory usage: 1.4G (139.86% of installed RAM)
[!!] Overall possible memory usage with other process exceeded memory
[!!] Slow queries: 15% (629K/4M)
[OK] Highest usage of available connections: 9% (19/200)
[OK] Aborted connections: 0.75% (4103/548857)
[!!] name resolution is active : a reverse name resolution is made for each new connection and can reduce performance
Here is my my.cnf configuration
[mysql]
# CLIENT #
port = 3306
socket = /var/lib/mysql/mysql.sock
[mysqld]
# GENERAL #
user = mysql
default-storage-engine = InnoDB
socket = /var/lib/mysql/mysql.sock
pid-file = /var/lib/mysql/mysql.pid
# MyISAM #
key-buffer-size = 32M
myisam-recover = FORCE,BACKUP
# SAFETY #
max-allowed-packet = 16M
max-connect-errors = 1000000
# DATA STORAGE #
datadir = /var/lib/mysql/
# BINARY LOGGING #
log-bin = /var/lib/mysql/mysql-bin
expire-logs-days = 14
sync-binlog = 1
# CACHES AND LIMITS #
tmp-table-size = 32M
max-heap-table-size = 32M
query-cache-type = 0
query-cache-size = 0
max-connections = 200
thread-cache-size = 20
open-files-limit = 65535
table-definition-cache = 1024
table-open-cache = 2048
# INNODB #
innodb-flush-method = O_DIRECT
innodb-log-files-in-group = 2
innodb-log-file-size = 64M
innodb-flush-log-at-trx-commit = 1
innodb-file-per-table = 1
innodb-buffer-pool-size = 624M
# LOGGING #
log-error = /var/lib/mysql/mysql-error.log
log-queries-not-using-indexes = 1
slow-query-log = 1
slow-query-log-file = /var/lib/mysql/mysql-slow.log
How can i optimize the configuation to fix those issues

There is one terribly bad setting:
innodb-buffer-pool-size = 624M
in a tiny 1GB server that probably includes both WP and MySQL? Change that to 200M. And watch for swapping. If there is any swapping, lower it more. Swapping leads to a huge amount of I/O; it is better to shrink the settings instead. Here's a head start:
tmp-table-size = 32M -> 8M
max-heap-table-size = 32M -> 8M
query-cache-type = 0 -- good
query-cache-size = 0 -- good
max-connections = 200 -> 50
thread-cache-size = 20
open-files-limit = 65535
table-definition-cache = 1024 -> 200
table-open-cache = 2048 -> 300
You have the slow log turned on? Let's see the worst query, as indicated by mysqldumpslow -s t or pt-query-digest.
Here's another tip. This vital table currently has lousy indexes; these will help:
CREATE TABLE wp_postmeta (
post_id …,
meta_key …,
meta_value …,
PRIMARY KEY(post_id, meta_key),
INDEX(meta_key)
) ENGINE=InnoDB;
IS WORDPRESS LISTENING?
Here's why:
AUTO_INCREMENT was a waste
This is a much better PK
Use 191 if necessary (5.6.3 thru 5.7.6)
InnoDB for clustered PK
More details: http://mysql.rjweb.org/doc.php/index_cookbook_mysql#speeding_up_wp_postmeta

Related

MYSQL 5.7 Note Got an error reading communication packets

134359 [Note] Got an error reading communication packets
Appears in logs every 5 minutes, I have seen a few similar posts on here however none actually match the error I'm seeing here.
MySQL 5.7 - configuration:
[mysqld]
##skip-name-resolve=1
#bind-address=localhost
default-storage-engine=InnoDB
innodb_file_per_table=1
performance-schema=1
server-id=2
auto_increment_offset = 2
auto_increment_increment= 5
report-host=2
log-slave-updates
##
replicate-ignore-db=mysql
replicate-ignore-db=cphulkd
replicate-ignore-db=information_schema
replicate-ignore-db=performance_schema
replicate-ignore-db=sys
##
max_connections=1000
table_open_cache=2000
wait_timeout=86400
interactive_timeout=86400
net_read_timeout=28800
net_write_timeout=28800
table_definition_cache=2000
thread_cache_size=100
long_query_time=10000
log_queries_not_using_indexes=0
innodb_page_cleaners=128
sql_mode=""
max_allowed_packet=512M
local-infile=0
[mysql]
# CLIENT #
port = 3306
socket = /var/lib/mysql/mysql.sock
[mysqld]
# GENERAL #
user = mysql
default-storage-engine = InnoDB
socket = /var/lib/mysql/mysql.sock
pid-file = /var/lib/mysql/mysql.pid
# MyISAM #
key-buffer-size = 10M
##myisam-recover = FORCE,BACKUP
# SAFETY #
max-allowed-packet = 10000M
max-connect-errors = 1000000
# DATA STORAGE #
datadir = /var/lib/mysql/
# BINARY LOGGING #
log-bin = /var/lib/mysql/mysql-bin
expire-logs-days = 14
sync-binlog = 0
# CACHES AND LIMITS #
tmp-table-size = 100M
max-heap-table-size = 100M
query-cache-type = 0
query-cache-size = 0
#max-connections = 1000
thread-cache-size = 50
open-files-limit = 1024
table-definition-cache = 4096
table_open_cache = 2024
# INNODB #
innodb-flush-method = O_DIRECT
innodb-log-files-in-group = 2
innodb-log-file-size = 10G
innodb_log_buffer_size = 5G
innodb-flush-log-at-trx-commit = 1
innodb-file-per-table = 1
innodb-buffer-pool-size = 80G
innodb_buffer_pool_instances = 64
innodb_write_io_threads = 64
innodb_read_io_threads = 64
innodb_max_dirty_pages_pct = 90
innodb_io_capacity = 1000
innodb_stats_on_metadata = 0
innodb_adaptive_flushing = 1
innodb_lru_scan_depth = 1024
# LOGGING #
log-error = /var/lib/mysql/mysql-error.log
log-queries-not-using-indexes = 0
slow-query-log = 0
slow-query-log-file = /var/lib/mysql/mysql-slow.log
join_buffer_size = 55M
binlog_cache_size = 16M
key_buffer_size = 50M
The thread ID 134359 doesn't correspond to any actual threads if I enable General MySQL Logging, nor does anything appear in a tcpdump during the events occuring.
At a loss? I do feel it's something CPanel or WHM related

MariaDB / Columnstore engine Memory getting chocked

we have installed mariadb along with columnstore engine and from the last few weeks we are facing memory chocking issue where memory getting chocked and all our DML/DDL operations are getting stuck, after restarting the services it gets fixed.
below are the stats :
total used free shared buff/cache available
Mem: 15 2 7 0 5 12
Swap: 4 0 4
[mysqld]
port = 3306
socket = /opt/evolv/mariadb/columnstore/mysql/lib/mysql/mysql.sock
datadir = /opt/evolv/mariadb/columnstore/mysql/db
skip-external-locking
key_buffer_size = 512M
max_allowed_packet = 1M
table_cache = 512
sort_buffer_size = 64M
read_buffer_size = 64M
read_rnd_buffer_size = 512M
myisam_sort_buffer_size = 64M
thread_cache_size = 8
query_cache_size = 0
# Try number of CPU's*2 for thread_concurrency
#thread_concurrency = 8
thread_stack = 512K
lower_case_table_names=1
group_concat_max_len=512
infinidb_use_import_for_batchinsert=1
# You can set .._buffer_pool_size up to 50 - 80 %
# of RAM but beware of setting memory usage too high
innodb_buffer_pool_size = 8192M
#innodb_additional_mem_pool_size = 20M
# Set .._log_file_size to 25 % of buffer pool size
#innodb_log_file_size = 100M
#innodb_log_buffer_size = 8M
#innodb_flush_log_at_trx_commit = 1
#innodb_lock_wait_timeout = 50
Here's an analysis of the VARIABLES and (suspicious) GLOBAL STATUS; nothing exciting:
Observations:
Version: 10.1.26-MariaDB
15 GB of RAM
Uptime = 03:04:25; Please rerun SHOW GLOBAL STATUS after several hours.
Are you sure this was a SHOW GLOBAL STATUS ?
You are not running on Windows.
Running 64-bit version
You appear to be running entirely (or mostly) InnoDB.
The More Important Issues:
Uptime = 03:04:25; Please rerun SHOW GLOBAL STATUS after several hours.
Are you sure this was a SHOW GLOBAL STATUS ?
key_buffer_size is excessively large (3G). If you don't need MyISAM for anything, set it to 50M.
Check infinidb_um_mem_limit to see if it makes sense for your application.
Suggest lowering innodb_buffer_pool_size to 2G until the "choking" is figured out.
Details and other observations:
( (key_buffer_size - 1.2 * Key_blocks_used * 1024) / _ram ) = (3072M - 1.2 * 0 * 1024) / 15360M = 20.0% -- Percent of RAM wasted in key_buffer.
-- Decrease key_buffer_size.
( Key_blocks_used * 1024 / key_buffer_size ) = 0 * 1024 / 3072M = 0 -- Percent of key_buffer used. High-water-mark.
-- Lower key_buffer_size to avoid unnecessary memory usage.
( innodb_buffer_pool_size / _ram ) = 6144M / 15360M = 40.0% -- % of RAM used for InnoDB buffer_pool
( Innodb_buffer_pool_pages_free * 16384 / innodb_buffer_pool_size ) = 392,768 * 16384 / 6144M = 99.9% -- buffer pool free
( innodb_print_all_deadlocks ) = innodb_print_all_deadlocks = OFF -- Whether to log all Deadlocks.
-- If you are plagued with Deadlocks, turn this on. Caution: If you have lots of deadlocks, this may write a lot to disk.
( local_infile ) = local_infile = ON
-- local_infile = ON is a potential security issue
( expire_logs_days ) = 0 -- How soon to automatically purge binlog (after this many days)
-- Too large (or zero) = consumes disk space; too small = need to respond quickly to network/machine crash.
(Not relevant if log_bin = OFF)
( long_query_time ) = 5 -- Cutoff (Seconds) for defining a "slow" query.
-- Suggest 2
Abnormally large:
read_buffer_size = 32MB
Acl_database_grants = 780
Acl_proxy_users = 4
Acl_users = 281
Columstore.xml
95% of all memory??
<MemoryCheckPercent>95</MemoryCheckPercent> <!-- Max real memory to limit growth of buffers to -->
<DataFileLog>OFF</DataFileLog>
I guess this is not relevant, since it is commented out??
<!-- enable if you want to limit how much memory may be used for hdfs read/write memory buffers.
<hdfsRdwrBufferMaxSize>8G</hdfsRdwrBufferMaxSize>
-->
Keep in mind that MySQL, other than Columnstore, is consuming a lot of memory:
<TotalUmMemory>25%</TotalUmMemory>
<TotalPmUmMemory>10%</TotalPmUmMemory>

Tuning MySql my.cnf for Many Users

I'm looking for some help tuning our my.cnf file to handle many current users (about 20 orders/minute). This particular site is in WordPress and running Woocommerce.
After a lot of reading online, I've come up with the settings below. The server is Debian 8 with 12 CPUs and 48GB RAM.
query_cache_type = 1
query_cache_limit = 256K
query_cache_min_res_unit = 2k
query_cache_size = 80M
key_buffer = 2G
max_allowed_packet = 512M
thread_cache_size = 256K
tmp_table_size = 4G
max_heap_table_size = 4G
table_cache = 1024
table_definition_cache = 1024
myisam_recover = FORCE,BACKUP
max_connections = 300
wait_timeout = 120
connect_timeout = 120
interactive_timeout = 120
innodb_flush_method = O_DIRECT
innodb_flush_log_at_trx_commit = 2
innodb_file_per_table = 1
innodb_buffer_pool_size = 2G
innodb_io_capacity = 1000
innodb_read_io_threads = 32
innodb_thread_concurrency = 0
innodb_write_io_threads = 32
It seems to be running pretty good for now. Any additional thoughts? Thanks for your input!
query_cache_type = 1
query_cache_limit = 256K
query_cache_min_res_unit = 2k
query_cache_size = 80M
See https://dom.as/tech/query-cache-tuner/
key_buffer = 2G
Key buffer is only for MyISAM. You shouldn't use MyISAM.
max_allowed_packet = 512M
thread_cache_size = 256K
tmp_table_size = 4G
max_heap_table_size = 4G
4G is probably way too high for the tmp table and heap table sizes. Keep in mind multiple threads can create temp tables concurrently.
table_cache = 1024
table_definition_cache = 1024
Probably overkill.
myisam_recover = FORCE,BACKUP
Also used only for MyISAM.
max_connections = 300
What does show global status like 'max_used_connections' say? Is it close to max_connections?
wait_timeout = 120
connect_timeout = 120
interactive_timeout = 120
innodb_flush_method = O_DIRECT
innodb_flush_log_at_trx_commit = 2
innodb_file_per_table = 1
innodb_buffer_pool_size = 2G
Fine but with 48G of RAM, you can probably increase the buffer pool size.
Run show engine innodb status and look for these lines:
Buffer pool size 131072
Free buffers 0
Database pages 128000
Is your buffer pool always pegged full? If so, increase its size.
What's the total size of your database? You don't need the buffer pool to be larger than the total size of your data+indexes, but large enough to hold the frequently-accessed pages would be good.
select round(sum(data_length+index_length)/1024/1024, 2) as mb
from information_schema.tables where engine='InnoDB'
innodb_io_capacity = 1000
innodb_read_io_threads = 32
innodb_thread_concurrency = 0
innodb_write_io_threads = 32
The io capacity might be greater than the ability of your disks to keep up. You didn't describe what your disk system is.
The io threads is way overkill for a Wordpress site with the traffic you describe. Run show engine innodb status and look for this line:
Pending normal aio reads: [0, 0, 0, 0] , aio writes: [0, 0, 0, 0] ,
If you always see 0's on that line, you don't need more than the default 4 & 4 io threads.

Mysql 5.5 InnoDB INSER/UPDATE very slow

I use mysql 5.5 and centos 7. Some tables are InnoDB in my database and i optimized my.cnf file with i read articles in the internet. I use TRANSACTION and COMMIT. But insert and update very slow you can see.
I can't use MyISAM because these tables always get insert, update and read same time too much.
QUERY PHOTO
My.cnf file
# The following options will be passed to all MySQL clients
[client]
port = 3306
socket = /var/lib/mysql/mysql.sock
[mysqld]
#innodb_force_recovery=6
user = mysql
default-storage-engine = InnoDB
socket = /var/lib/mysql/mysql.sock
pid-file = /var/lib/mysql/mysql.pid
datadir=/var/lib/mysql/
log-error=/var/lib/mysql/server.mysql.err
symbolic-links=0
tmpdir=/var/tmp
skip-external-locking
table_cache = 2000
key_buffer_size=20G
join_buffer_size = 4M
sort_buffer_size = 2M
read_buffer_size = 2M
read_rnd_buffer_size = 1M
myisam_sort_buffer_size=2M
thread_cache_size = 512
query_cache_limit = 1G
query_cache_size = 40M
query_cache_type=1
thread_stack = 256K
tmp_table_size = 128M
max_heap_table_size = 128M
open_files_limit=65535
#thread_concurrency = 10
max_connect_errors=1
connect_timeout=60
interactive_timeout = 60
lock_wait_timeout=60
wait_timeout = 30
max_connections = 1000
slow_query_log=1
long_query_time=1
slow-query-log-file=/var/log/mysql-slow.log
#log-queries-not-using-indexes
innodb_buffer_pool_size=32G
innodb_additional_mem_pool_size=64M
innodb_data_file_path=ibdata1:100M:autoextend
innodb_log_buffer_size=128M
innodb-log-files-in-group = 2
innodb_change_buffering=all
innodb_flush_method=O_DIRECT
innodb_flush_log_at_trx_commit=2
#innodb_thread_concurrency=10
innodb_read_io_threads = 64
innodb_write_io_threads = 64
innodb_file_per_table=1
innodb_lock_wait_timeout = 60
innodb_table_locks=0
innodb_open_files=65535
innodb_io_capacity=2000
#innodb_doublewrite=0
#innodb_support_xa=0
[mysqldump]
max_allowed_packet=2G
quick
[mysql]
no-auto-rehash
[isamchk]
key_buffer = 128M
sort_buffer_size = 4M
read_buffer = 4M
write_buffer = 4M
[myisamchk]
tmpdir=/tmp
key_buffer_size=128M
sort_buffer_size=4M
read_buffer=4M
write_buffer=4M
[mysqlhotcopy]
interactive-timeout
And my server properties are
CPU
Intel(R) Xeon(R) CPU E5-1620 v2 # 3.70GHz
Cores : 8
Cache : 10240KB
RAM
64 GB
Disks
3 x 160 GB SSD
query_cache_limit = 1G -- should be less than the size.
query_cache_size = 40M -- this is reasonable.
key_buffer_size=20G -- too big; change to 6G. Note: only used for MyISAM indexes.
-- if you are not using MyISAM, then drop to 10M
slow_query_log=1 --
What does pt-query-digest point out as the 'worst' 3 queries? Show us them, together with SHOW CREATE TABLE and (if SELECT), EXPLAIN SELECT ....
Its not necessary that you have to tune optimized my.cnf file if your queries working slow. You might have to optimize your queries by proper indexes and joins.

mysql - usage of memory still grow up

I have problem with my production mysql server. Usage of memory still grow up and I don`t know why. Trouble started when we changed the server.
My mysql version: 5.5.44-0+deb8u1-log - (Debian).
My my.cnf file:
[mysqld_safe]
pid-file = /var/run/mysqld/mysqld.pid
socket = /var/run/mysqld/mysqld.sock
nice = 0
[mysqld]
skip-external-locking
key_buffer_size = 5M
max_allowed_packet = 16M
thread_stack = 192K
thread_cache_size = 8
tmp_table_size = 384M
max_heap_table_size = 384M
table_open_cache = 7000
open_files_limit = 14000
interactive_timeout=3600
wait_timeout=3600
myisam-recover_options = BACKUP
max_connections = 150
query_cache_limit = 8M
query_cache_size = 127M
slow_query_log_file = /var/log/mysql/mysql-slow.log
slow_query_log = 1
long_query_time = 2
expire_logs_days = 10
max_binlog_size = 100M
innodb_file_per_table
innodb_buffer_pool_instances = 9
innodb_buffer_pool_size = 10000M
innodb_log_file_size = 2000M
It is something wrong in my configuration?
EDIT
For production we have dedicated server:
4 x Intel(R) Xeon(R) CPU E5-2680 v2 # 2.80GHz
20 GB RAM memory
40 GB na DHH - data base have now 15 GB
in the data base is on this moment 650 tables on the innoDB engine
Screenshot of htop:
htop of proces on server