I have 4 instance database-clustering (2 of them are mariadb 10.2 server and 2 of them is garbd server)
We use update-heavy queries in our databases, dealing with a huge tables which some of the tables have over 20 million records. Also we have lots of java application which use database and these threads mostly make "update" processes in tables. In the peak_time we use about 200 threads.
Our problem is that we have "Deadlock" in our applications. I believe that I can figure out by making some performance tuning in database configurations. My last configuration is below;
DB1: x.x.x.x
DB2: y.y.y.y
Garbd-1: w.w.w.w
Garbd-2: z.z.z.z
DB-1 : my.cnf
[client-server]
socket=/app/mysql/mysql.sock
port=3306
[mysqld]
user=mysql
port=3306
bind-address=0.0.0.0
socket=/app/mysql/mysql.sock
skip-name-resolve
lower_case_table_names=1
server-id=1
event-scheduler=ON
datadir=/app/mysql
pid-file=/app/mysql/db1.pid
log-error=/app/mysql/db1.err
log_bin_trust_function_creators=1
query_cache_type=0
query_cache_size=0
#query_cache_limit = 4M
default_table_type = InnoDB
table_open_cache = 4096
open_files_limit = 8192
max_connections = 800
wait_timeout=100
interactive_timeout=100
#net_read_timeout=3600
#net_write_timeout=3600
max_heap_table_size=64M
tmp_table_size=64M
thread_cache_size=256
sort_buffer_size = 2M
join_buffer_size = 256K
read_buffer_size=128K
read_rnd_buffer_size = 256M
transaction-isolation=READ-COMMITTED
log_warnings
slow_query_log
long_query_time=5
# Physical RAM is 32G (8-core)
innodb_buffer_pool_size =24G
innodb_buffer_pool_instances=6
key_buffer_size=512M
nnodb_write_io_threads=8
innodb_read_io_threads=8
innodb_thread_concurrency = 16
#innodb_data_file_path = ibdata1:1G:autoextend
#innodb_autoextend_increment=128M
#innodb_file_per_table
innodb_flush_log_at_trx_commit=2
#sync_binlog=1
innodb_log_buffer_size = 16M
innodb_log_file_size = 2G
innodb_max_dirty_pages_pct = 80
innodb_flush_method=O_DIRECT
innodb_lock_wait_timeout = 120
binlog_format=ROW
innodb_autoinc_lock_mode=2
[mysqldump]
quick
max_allowed_packet = 512M
[mysql]
no-auto-rehash
[mysqld_safe]
datadir=/app/mysql
pid-file=/app/mysql/db1.pid
log-error=/app/mysql/db1.err
[mariadb-10.2]
wsrep_on=ON
wsrep_provider=/usr/lib64/galera/libgalera_smm.so
wsrep_provider_options="gcache.size=10G;gmcast.listen_addr=tcp://0.0.0.0:4567;socket.checksum=2"
wsrep_cluster_address="gcomm://10.92.221.215,10.92.221.216,10.92.223.215,10.92.223.216"
wsrep_cluster_name='galera_cluster'
wsrep_node_address='10.92.221.215'
wsrep_node_name='ivrocsdbp00'
wsrep_sst_method=rsync
wsrep_sst_auth=sst_user:dbpass
These are also configured in second server, too with a server-id=2 and db2.pid, db2.err changes.
Is these changes might solve my problem?
I want to know that these parameters are well-configured ?
We have 32 GB RAM (8-core) in the servers.
Thanks for your kind help.
I'm guessing nnodb_write_io_threads is a typo as mysqld wouldn't start with it.
I would start with innodb_thread_concurrency = 0 to de-restrict the thread concurrency throttling.
Are your UPDATEs running fully indexed, ideally with PK in the WHERE?
Is there a good reason you have innodb_file_per_table commented out?
It might be worth verifying that your table_open_cache isn't too small (look at the process list and see if you regularly see queries in state of "opening|closing tables".
Are these servers VMs, and are they on a busy/overbooked host? I have seen very interesting spurious deadlocks in situations where clock ticks are unstable and unpredictable, e.g. on heavily overbooked virtual machines. There is some interesting thread racing happening under those conditions that seems to be completely un-reproducible on bare metal. Bumping up the instance size in the cloud environment usually cures the problem because bigger instances tend to be on less overbooked hosts.
Thanks for your responding.
I have corrected my typo and changed my configuration files as innodb_thread_concurrency = 0 as you mentioned.
I tried to configure very low-performing-consuming parameters and I have read about that innodb_file_per_table parameters could effect the RAM usage, so I decided to comment it out. Actually I couldn't figure out what this parameters use for clearly.
Let me tell you my some observation after I have restarted both DB servers.
First of all I stopped all application that making connection to db-1 (Actually we are making all db connections through DB1), then restarted both DB-servers. First of all, let me clear that there was no any traffic when I started applications.
After I start all appcalitons that we use, I realised that I was seeing about 120 processes that created by our user in PROCESSLIST in the INFORMATION_SCHEMA. Also I know that some of these connections don't make anything but using connnection_pool.
I was set up 100s wait_timeout, after 100s later, PROCESSLIST down to "27" and threads_cache became 92.
I also know that these 27 connection was used by some applications that running some check queries in spesific time interval. (10sec, 4sec, 2sec) So when they run queries , their connection_time return 0 and count again as they are never closed.
I believe that in the peak_time, we are gonna need thread_cache_size, so configuring this parameter as 256 would be good for our system.
I know it is so hard to understand for now but can you please check my innodb engine status and give me some advice as I am not very good at concept of database management systems.
=====================================
2020-05-08 01:10:28 0x7fd8cc55c700 INNODB MONITOR OUTPUT
=====================================
Per second averages calculated from the last 7 seconds
-----------------
BACKGROUND THREAD
-----------------
srv_master_thread loops: 4897 srv_active, 0 srv_shutdown, 371 srv_idle
srv_master_thread log flush and writes: 5266
----------
SEMAPHORES
----------
OS WAIT ARRAY INFO: reservation count 34869
OS WAIT ARRAY INFO: signal count 121009
RW-shared spins 0, rounds 214984, OS waits 15858
RW-excl spins 0, rounds 1540716, OS waits 11401
RW-sx spins 52309, rounds 782230, OS waits 3893
Spin rounds per wait: 214984.00 RW-shared, 1540716.00 RW-excl, 14.95 RW-sx
------------
TRANSACTIONS
------------
Trx id counter 211830460
Purge done for trx's n:o < 211830450 undo n:o < 0 state: running but idle
History list length 1
LIST OF TRANSACTIONS FOR EACH SESSION:
---TRANSACTION 422083415170480, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422083415166264, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422083415162048, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422083415157832, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422083415153616, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422083415149400, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422083415145184, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422083415140968, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422083415136752, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422083415132536, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422083415128320, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422083415124104, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422083415119888, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422083415115672, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422083415111456, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422083415107240, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422083415103024, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422083415098808, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422083415094592, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422083415090376, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422083415086160, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422083415081944, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422083415077728, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422083415073512, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422083415069296, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422083415065080, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422083415060864, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422083415056648, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422083415052432, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
--------
FILE I/O
--------
I/O thread 0 state: waiting for completed aio requests (insert buffer thread)
I/O thread 1 state: waiting for completed aio requests (log thread)
I/O thread 2 state: waiting for completed aio requests (read thread)
I/O thread 3 state: waiting for completed aio requests (read thread)
I/O thread 4 state: waiting for completed aio requests (read thread)
I/O thread 5 state: waiting for completed aio requests (read thread)
I/O thread 6 state: waiting for completed aio requests (read thread)
I/O thread 7 state: waiting for completed aio requests (read thread)
I/O thread 8 state: waiting for completed aio requests (read thread)
I/O thread 9 state: waiting for completed aio requests (read thread)
I/O thread 10 state: waiting for completed aio requests (write thread)
I/O thread 11 state: waiting for completed aio requests (write thread)
I/O thread 12 state: waiting for completed aio requests (write thread)
I/O thread 13 state: waiting for completed aio requests (write thread)
I/O thread 14 state: waiting for completed aio requests (write thread)
I/O thread 15 state: waiting for completed aio requests (write thread)
I/O thread 16 state: waiting for completed aio requests (write thread)
I/O thread 17 state: waiting for completed aio requests (write thread)
Pending normal aio reads: [0, 0, 0, 0, 0, 0, 0, 0] , aio writes: [0, 0, 0, 0, 0, 0, 0, 0] ,
ibuf aio reads:, log i/o's:, sync i/o's:
Pending flushes (fsync) log: 0; buffer pool: 0
363895 OS file reads, 105906 OS file writes, 18722 OS fsyncs
0.00 reads/s, 0 avg bytes/read, 6.14 writes/s, 2.43 fsyncs/s
-------------------------------------
INSERT BUFFER AND ADAPTIVE HASH INDEX
-------------------------------------
Ibuf: size 1, free list len 13430, seg size 13432, 2410 merges
merged operations:
insert 72630, delete mark 0, delete 0
discarded operations:
insert 0, delete mark 0, delete 0
Hash table size 6374293, node heap has 108 buffer(s)
Hash table size 6374293, node heap has 64 buffer(s)
Hash table size 6374293, node heap has 64 buffer(s)
Hash table size 6374293, node heap has 18 buffer(s)
Hash table size 6374293, node heap has 10 buffer(s)
Hash table size 6374293, node heap has 72 buffer(s)
Hash table size 6374293, node heap has 4 buffer(s)
Hash table size 6374293, node heap has 634 buffer(s)
3473.22 hash searches/s, 553.49 non-hash searches/s
---
LOG
---
Log sequence number 129572121784
Log flushed up to 129572121784
Pages flushed up to 129572121784
Last checkpoint at 129572120321
0 pending log flushes, 0 pending chkp writes
3127 log i/o's done, 0.71 log i/o's/second
----------------------
BUFFER POOL AND MEMORY
----------------------
Total large memory allocated 26398949376
Dictionary memory allocated 278784
Buffer pool size 1572672
Free buffers 1200805
Database pages 370893
Old database pages 137029
Modified db pages 0
Percent of dirty pages(LRU & free pages): 0.000
Max dirty pages percent: 80.000
Pending reads 0
Pending writes: LRU 0, flush list 0, single page 0
Pages made young 2349, not young 0
0.00 youngs/s, 0.00 non-youngs/s
Pages read 363786, created 7107, written 94745
0.00 reads/s, 0.00 creates/s, 4.43 writes/s
Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000
Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
LRU len: 370893, unzip_LRU len: 0
I/O sum[0]:cur[24], unzip sum[0]:cur[0]
----------------------
INDIVIDUAL BUFFER POOL INFO
----------------------
---BUFFER POOL 0
Buffer pool size 262112
Free buffers 198770
Database pages 63178
Old database pages 23341
Modified db pages 0
Percent of dirty pages(LRU & free pages): 0.000
Max dirty pages percent: 80.000
Pending reads 0
Pending writes: LRU 0, flush list 0, single page 0
Pages made young 372, not young 0
0.00 youngs/s, 0.00 non-youngs/s
Pages read 62294, created 884, written 22682
0.00 reads/s, 0.00 creates/s, 2.14 writes/s
Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000
Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
LRU len: 63178, unzip_LRU len: 0
I/O sum[0]:cur[4], unzip sum[0]:cur[0]
---BUFFER POOL 1
Buffer pool size 262112
Free buffers 198593
Database pages 63353
Old database pages 23406
Modified db pages 0
Percent of dirty pages(LRU & free pages): 0.000
Max dirty pages percent: 80.000
Pending reads 0
Pending writes: LRU 0, flush list 0, single page 0
Pages made young 365, not young 0
0.00 youngs/s, 0.00 non-youngs/s
Pages read 62005, created 1348, written 17745
0.00 reads/s, 0.00 creates/s, 0.86 writes/s
Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000
Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
LRU len: 63353, unzip_LRU len: 0
I/O sum[0]:cur[4], unzip sum[0]:cur[0]
---BUFFER POOL 2
Buffer pool size 262112
Free buffers 202987
Database pages 58961
Old database pages 21784
Modified db pages 0
Percent of dirty pages(LRU & free pages): 0.000
Max dirty pages percent: 80.000
Pending reads 0
Pending writes: LRU 0, flush list 0, single page 0
Pages made young 388, not young 0
0.00 youngs/s, 0.00 non-youngs/s
Pages read 58042, created 919, written 15919
0.00 reads/s, 0.00 creates/s, 1.00 writes/s
Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000
Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
LRU len: 58961, unzip_LRU len: 0
I/O sum[0]:cur[4], unzip sum[0]:cur[0]
---BUFFER POOL 3
Buffer pool size 262112
Free buffers 199895
Database pages 62058
Old database pages 22928
Modified db pages 0
Percent of dirty pages(LRU & free pages): 0.000
Max dirty pages percent: 80.000
Pending reads 0
Pending writes: LRU 0, flush list 0, single page 0
Pages made young 439, not young 0
0.00 youngs/s, 0.00 non-youngs/s
Pages read 60791, created 1267, written 12163
0.00 reads/s, 0.00 creates/s, 0.00 writes/s
Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000
Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
LRU len: 62058, unzip_LRU len: 0
I/O sum[0]:cur[4], unzip sum[0]:cur[0]
---BUFFER POOL 4
Buffer pool size 262112
Free buffers 201239
Database pages 60712
Old database pages 22431
Modified db pages 0
Percent of dirty pages(LRU & free pages): 0.000
Max dirty pages percent: 80.000
Pending reads 0
Pending writes: LRU 0, flush list 0, single page 0
Pages made young 382, not young 0
0.00 youngs/s, 0.00 non-youngs/s
Pages read 59190, created 1522, written 12539
0.00 reads/s, 0.00 creates/s, 0.43 writes/s
Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000
Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
LRU len: 60712, unzip_LRU len: 0
I/O sum[0]:cur[4], unzip sum[0]:cur[0]
---BUFFER POOL 5
Buffer pool size 262112
Free buffers 199321
Database pages 62631
Old database pages 23139
Modified db pages 0
Percent of dirty pages(LRU & free pages): 0.000
Max dirty pages percent: 80.000
Pending reads 0
Pending writes: LRU 0, flush list 0, single page 0
Pages made young 403, not young 0
0.00 youngs/s, 0.00 non-youngs/s
Pages read 61464, created 1167, written 13697
0.00 reads/s, 0.00 creates/s, 0.00 writes/s
Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000
Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
LRU len: 62631, unzip_LRU len: 0
I/O sum[0]:cur[4], unzip sum[0]:cur[0]
--------------
ROW OPERATIONS
--------------
0 queries inside InnoDB, 0 queries in queue
0 read views open inside InnoDB
Process ID=31986, Main thread ID=140569828304640, state: sleeping
Number of rows inserted 276741, updated 1143, deleted 276686, read 179703655
0.00 inserts/s, 0.29 updates/s, 0.00 deletes/s, 109971.15 reads/s
Number of system rows inserted 0, updated 0, deleted 0, read 0
0.00 inserts/s, 0.00 updates/s, 0.00 deletes/s, 0.00 reads/s
----------------------------
END OF INNODB MONITOR OUTPUT
============================
I will share threads status and INNODB ENGINE STATUS after 1-2 day later, too.
I hope we won't have face with any lock wait timeout issues.
Have a nice day.
Servers are running over 48 hours and let me share tuning-primer results in DB-1.
-- MYSQL PERFORMANCE TUNING PRIMER --
- By: Matthew Montgomery -
MySQL Version 10.2.27-MariaDB-log x86_64
Uptime = 2 days 3 hrs 54 min 46 sec
Avg. qps = 26
Total Questions = 5022294
Threads Connected = 17
Server has been running for over 48hrs.
It should be safe to follow these recommendations
To find out more information on how each of these
runtime variables effects performance visit:
http://dev.mysql.com/doc/refman/10.2/en/server-system-variables.html
Visit http://www.mysql.com/products/enterprise/advisors.html
for info about MySQL's Enterprise Monitoring and Advisory Service
SLOW QUERIES
The slow query log is NOT enabled.
Current long_query_time = 5.000000 sec.
You have 10 out of 5022308 that take longer than 5.000000 sec. to complete
Your long_query_time seems to be fine
BINARY UPDATE LOG
The binary update log is NOT enabled.
You will not be able to do point in time recovery
See http://dev.mysql.com/doc/refman/10.2/en/point-in-time-recovery.html
WORKER THREADS
Current thread_cache_size = 256
Current threads_cached = 103
Current threads_per_sec = 0
Historic threads_per_sec = 0
Your thread_cache_size is fine
MAX CONNECTIONS
Current max_connections = 800
Current threads_connected = 17
Historic max_used_connections = 120
The number of used connections is 15% of the configured maximum.
Your max_connections variable seems to be fine.
INNODB STATUS
Current InnoDB index space = 4.29 G
Current InnoDB data space = 4.72 G
Current InnoDB buffer pool free = 69 %
Current innodb_buffer_pool_size = 24.00 G
Depending on how much space your innodb indexes take up it may be safe
to increase this value to up to 2 / 3 of total system memory
MEMORY USAGE
Max Memory Ever Allocated : 54.82 G
Configured Max Per-thread Buffers : 202.07 G
Configured Max Global Buffers : 24.51 G
Configured Max Memory Limit : 226.59 G
Physical Memory : 31.26 G
Max memory limit exceeds 90% of physical memory
KEY BUFFER
Current MyISAM index space = 638 K
Current key_buffer_size = 512 M
Key cache miss rate is 1 : 144181
Key buffer free ratio = 81 %
Your key_buffer_size seems to be too high.
Perhaps you can use these resources elsewhere
QUERY CACHE
Query cache is supported but not enabled
Perhaps you should set the query_cache_size
SORT OPERATIONS
Current sort_buffer_size = 2 M
Current read_rnd_buffer_size = 256 M
Sort buffer seems to be fine
JOINS
Current join_buffer_size = 260.00 K
You have had 373030 queries where a join could not use an index properly
You should enable "log-queries-not-using-indexes"
Then look for non indexed joins in the slow query log.
If you are unable to optimize your queries you may want to increase your
join_buffer_size to accommodate larger joins in one pass.
Note! This script will still suggest raising the join_buffer_size when
ANY joins not using indexes are found.
OPEN FILES LIMIT
Current open_files_limit = 9031 files
The open_files_limit should typically be set to at least 2x-3x
that of table_cache if you have heavy MyISAM usage.
Your open_files_limit value seems to be fine
TABLE CACHE
Current table_open_cache = 4096 tables
Current table_definition_cache = 400 tables
You have a total of 186 tables
You have 428 open tables.
The table_cache value seems to be fine
TEMP TABLES
Current max_heap_table_size = 64 M
Current tmp_table_size = 64 M
Of 490764 temp tables, 0% were created on disk
Created disk tmp tables ratio seems fine
TABLE SCANS
Current read_buffer_size = 128 K
Current table scan ratio = 34 : 1
read_buffer_size seems to be fine
TABLE LOCKING
Current Lock Wait ratio = 1 : 383
You may benefit from selective use of InnoDB.
If you have long running SELECT's against MyISAM tables and perform
frequent updates consider setting 'low_priority_updates=1'
If you have a high concurrency of inserts on Dynamic row-length tables
consider setting 'concurrent_insert=ALWAYS'.
Should I decrease some parameters in my.cnf ? Any suggestion please ?
Thanks
-- MYSQL PERFORMANCE TUNING PRIMER --
- By: Matthew Montgomery -
MySQL Version 10.2.27-MariaDB-log x86_64
Uptime = 2 days 9 hrs 24 min 19 sec
Avg. qps = 69
Total Questions = 14381364
Threads Connected = 24
Server has been running for over 48hrs.
It should be safe to follow these recommendations
To find out more information on how each of these
runtime variables effects performance visit:
http://dev.mysql.com/doc/refman/10.2/en/server-system-variables.html
Visit http://www.mysql.com/products/enterprise/advisors.html
for info about MySQL's Enterprise Monitoring and Advisory Service
SLOW QUERIES
The slow query log is NOT enabled.
Current long_query_time = 5.000000 sec.
You have 608 out of 14381378 that take longer than 5.000000 sec. to complete
Your long_query_time seems to be fine
BINARY UPDATE LOG
The binary update log is NOT enabled.
You will not be able to do point in time recovery
See http://dev.mysql.com/doc/refman/10.2/en/point-in-time-recovery.html
WORKER THREADS
Current thread_cache_size = 256
Current threads_cached = 109
Current threads_per_sec = 0
Historic threads_per_sec = 0
Your thread_cache_size is fine
MAX CONNECTIONS
Current max_connections = 800
Current threads_connected = 24
Historic max_used_connections = 156
The number of used connections is 19% of the configured maximum.
Your max_connections variable seems to be fine.
INNODB STATUS
Current InnoDB index space = 5.79 G
Current InnoDB data space = 6.29 G
Current InnoDB buffer pool free = 63 %
Current innodb_buffer_pool_size = 24.00 G
Depending on how much space your innodb indexes take up it may be safe
to increase this value to up to 2 / 3 of total system memory
MEMORY USAGE
Max Memory Ever Allocated : 24.95 G
Configured Max Per-thread Buffers : 2.27 G
Configured Max Global Buffers : 24.51 G
Configured Max Memory Limit : 26.78 G
Physical Memory : 31.26 G
Max memory limit seem to be within acceptable norms
KEY BUFFER
Current MyISAM index space = 684 K
Current key_buffer_size = 512 M
Key cache miss rate is 1 : 111228
Key buffer free ratio = 81 %
Your key_buffer_size seems to be too high.
Perhaps you can use these resources elsewhere
QUERY CACHE
Query cache is supported but not enabled
Perhaps you should set the query_cache_size
SORT OPERATIONS
Current sort_buffer_size = 2 M
Current read_rnd_buffer_size = 256 K
Sort buffer seems to be fine
JOINS
Current join_buffer_size = 260.00 K
You have had 419644 queries where a join could not use an index properly
You should enable "log-queries-not-using-indexes"
Then look for non indexed joins in the slow query log.
If you are unable to optimize your queries you may want to increase your
join_buffer_size to accommodate larger joins in one pass.
Note! This script will still suggest raising the join_buffer_size when
ANY joins not using indexes are found.
OPEN FILES LIMIT
Current open_files_limit = 9031 files
The open_files_limit should typically be set to at least 2x-3x
that of table_cache if you have heavy MyISAM usage.
Your open_files_limit value seems to be fine
TABLE CACHE
Current table_open_cache = 4096 tables
Current table_definition_cache = 400 tables
You have a total of 186 tables
You have 1303 open tables.
The table_cache value seems to be fine
TEMP TABLES
Current max_heap_table_size = 64 M
Current tmp_table_size = 64 M
Of 842280 temp tables, 0% were created on disk
Created disk tmp tables ratio seems fine
TABLE SCANS
Current read_buffer_size = 128 K
Current table scan ratio = 50 : 1
read_buffer_size seems to be fine
TABLE LOCKING
Current Lock Wait ratio = 1 : 29
You may benefit from selective use of InnoDB.
If you have long running SELECT's against MyISAM tables and perform
frequent updates consider setting 'low_priority_updates=1'
If you have a high concurrency of inserts on Dynamic row-length tables
consider setting 'concurrent_insert=ALWAYS'.
Any suggest please ?
I also wanna status my table spesifications. We usually had deadlocks in 3 major tables which are cp_resource_allocation, cp_pending, cp_batch.
CREATE TABLE `cp_resource_allocation` (
`CAMPAIGN_ID` int(11) NOT NULL,
`RESOURCE_QUOTA` int(11) NOT NULL DEFAULT 0,
`RESOURCE_ALLOCATED` int(11) NOT NULL DEFAULT 0,
`DAILY_ATTEMPT` int(11) NOT NULL DEFAULT 0,
`DAILY_SUCCESSFUL` int(11) NOT NULL DEFAULT 0,
`DAILY_RESERVED` int(11) NOT NULL DEFAULT 0,
PRIMARY KEY (`CAMPAIGN_ID`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1 COMMENT='Campaign Resource Allocation Table'
CREATE TABLE `cp_pending` (
`RID` int(20) NOT NULL AUTO_INCREMENT,
`CAMPAIGN` int(10) NOT NULL,
`BATCH` int(11) NOT NULL,
`DEST_ADDR` char(20) CHARACTER SET latin1 NOT NULL COMMENT 'Aranan / Gönderilen adres.',
`TRIES_MADE` int(10) NOT NULL DEFAULT 0 COMMENT 'Specifies current retry count for the Called Number.',
`SELECT_LOCK` int(10) NOT NULL DEFAULT 0 COMMENT 'NULL’dan farklı ise kayıt seçilmiştir. ',
`SELECT_TIME` datetime DEFAULT NULL COMMENT 'Seçim zamanı.',
`OUTCOME` int(10) NOT NULL DEFAULT 0 COMMENT 'İşlem durumu. 0 ise işlem devam ediyor. 1 ise işlem başarılı. 2 ise işlem başarısız. 3 ise işlem iptal. 4 ise numara blacklist''de. 5 ise HLR''a tetik konmuş.',
`APP_INT1` int(10) DEFAULT NULL COMMENT 'Uygulamaya özel.',
`APP_INT2` int(10) unsigned zerofill DEFAULT NULL COMMENT 'Uygulamaya özel..',
`CID` int(11) DEFAULT NULL,
`CC_SERVER_ADDRESS` char(45) CHARACTER SET latin1 DEFAULT NULL,
`CALL_START_TIME` datetime DEFAULT NULL,
`CALL_ANSWER_TIME` datetime DEFAULT NULL,
`CALL_END_TIME` datetime DEFAULT NULL,
`CAUSE_VALUE` int(11) DEFAULT NULL COMMENT 'RELEASE CAUSE VALUE',
`CALL_REQUEST_TIME` datetime DEFAULT NULL COMMENT 'time when a call manager retrieves this record',
`CALL_CONNECT_DURATION` int(11) NOT NULL DEFAULT 0 COMMENT 'CALL_END_TIME - CALL_ANSWER_TIME',
`LIST_TYPE` int(11) NOT NULL DEFAULT 0 COMMENT 'List Group Type;\\n0 Start List. \\n1 Main List.\\n2 End List. \\n',
`CALL_ACTIVE_DURATION` int(11) NOT NULL DEFAULT 0 COMMENT 'CALL_END_TIME-CALL_START_TIME',
`DTMF` varchar(5) CHARACTER SET latin1 DEFAULT NULL,
`app_str1` varchar(2048) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`APP_STR2` varchar(2048) CHARACTER SET latin1 DEFAULT NULL,
`APP_STR3` varchar(2048) CHARACTER SET latin1 DEFAULT NULL,
`APP_STR4` varchar(80) CHARACTER SET latin1 DEFAULT NULL,
`APP_STR5` varchar(80) CHARACTER SET latin1 DEFAULT NULL,
`LAST_HLR_UPDATE` datetime DEFAULT NULL,
`TTS_STATUS` int(11) DEFAULT 0 COMMENT '0: initial, 1: Converted, 2: Convert failed, 3: Uploaded, 4: Upload Error, 5: Partial Upload Error',
`TTS_SELECT_LOCK` varchar(100) DEFAULT NULL,
PRIMARY KEY (`RID`) USING BTREE,
KEY `campaign` (`CAMPAIGN`),
KEY `ilock` (`SELECT_LOCK`) USING BTREE,
KEY `iDestAddr` (`DEST_ADDR`),
KEY `ix_cp_pending_2` (`DEST_ADDR`,`OUTCOME`),
KEY `ix_test` (`DEST_ADDR`,`CAMPAIGN`,`OUTCOME`),
KEY `ix_batch` (`OUTCOME`,`BATCH`),
KEY `ix_time` (`SELECT_TIME`,`OUTCOME`),
KEY `ix_mrcp_tts_client` (`TTS_STATUS`,`CAMPAIGN`,`TTS_SELECT_LOCK`)
) ENGINE=InnoDB AUTO_INCREMENT=148583478 DEFAULT CHARSET=utf8mb4
CREATE TABLE `cp_batch` (
`BATCH_ID` int(10) NOT NULL AUTO_INCREMENT,
`BATCH_NAME` varchar(200) NOT NULL,
`BATCH_STATUS` tinyint(3) unsigned NOT NULL COMMENT 'Batch status indicator. 1-Loading, 2-Active, 3-Aborted, 4-Completed, 5-Suspended, 6-Deleted',
`CAMPAIGN_ID` int(10) NOT NULL,
`LIST_ID` int(10) DEFAULT NULL,
`OPERATION_TIME` timestamp NOT NULL DEFAULT current_timestamp(),
`INPUT_FILE` varchar(200) DEFAULT NULL COMMENT 'If this batch was created automatically by text file placed in disk folder, the name of that file.',
`NUM_DESTINATIONS` int(10) unsigned NOT NULL DEFAULT 0 COMMENT 'Total number of destinations used for this batch.',
`NUM_SUCCESS` int(10) unsigned NOT NULL DEFAULT 0 COMMENT 'Total number of successfull calls',
`NUM_CANCEL` int(10) unsigned NOT NULL DEFAULT 0 COMMENT 'Total number of canceled calls\n',
`REMAINING_DESTINATIONS` int(10) unsigned DEFAULT NULL,
`NUM_UNTRIED` int(10) NOT NULL COMMENT 'Number of destinations to which no call attempt has been made yet.',
`NUM_BLACKLIST` int(11) NOT NULL DEFAULT 0 COMMENT 'total number of blacklisted calls',
`COMPLETION_REASON` int(10) unsigned DEFAULT NULL COMMENT 'Indicates how the campaign was completed (end date, max tries, etc.)',
`DEPLETION_TIME` datetime DEFAULT NULL COMMENT 'The time when all numbers in this batch have been processed. Set by CallMgr.',
`IS_DEPLETED` int(10) unsigned NOT NULL DEFAULT 0 COMMENT 'Shows if all numbers in this batch have been processed. 1: All numbers were processed 0: No',
`MIN_RID` bigint(20) NOT NULL DEFAULT 0 COMMENT 'beginning of rid for this campaign',
`MAX_RID` bigint(20) NOT NULL DEFAULT 0 COMMENT 'last rid for this campaign',
`CURRENT_RID` bigint(20) NOT NULL DEFAULT 0 COMMENT 'current pointer for this campaign',
`TRIES_MADE` int(11) NOT NULL DEFAULT 0,
`START_NOTIF` int(11) NOT NULL DEFAULT 0,
`IS_EXPANDABLE` tinyint(4) NOT NULL DEFAULT 0 COMMENT 'default:0, initial:1',
PRIMARY KEY (`BATCH_ID`),
KEY `ix1` (`BATCH_STATUS`,`IS_DEPLETED`),
KEY `ix2_campaign` (`CAMPAIGN_ID`),
KEY `batch_index` (`BATCH_NAME`)
) ENGINE=InnoDB AUTO_INCREMENT=1588730 DEFAULT CHARSET=latin1
We haven't face with any deadlock yet since yesterday. Do you see any suspect value in "create table" parameters ?
Thanks.
I'm currently trying to optimize the indices for a quite large table of a project and experiencing a very counter intuitive behavior between the explain result and the actual query runtime.
The server is running MariaDB version 10.1.26-MariaDB-0+deb9u1 with the following configuration options:
key_buffer_size = 5G
max_allowed_packet = 16M
thread_stack = 192K
thread_cache_size = 8
myisam_sort_buffer_size = 512M
read_buffer_size = 2M
read_rnd_buffer_size = 1M
query_cache_type = 0
query_cache_limit = 256K
query_cache_min_res_unit = 2k
query_cache_size = 0M
join_buffer_size = 8M
sort_buffer_size = 8M
tmp_table_size = 64M
max_heap_table_size = 64M
table_open_cache = 4K
performance_schema = ON
innodb_buffer_pool_size = 30G
innodb_log_buffer_size = 4MB
innodb_log_file_size = 1G
innodb_buffer_pool_instances = 10
The table looks contains about 6.8 million rows summing up to 12.1GB and looks like this:
CREATE TABLE `ad_master_test` (
`ID_AD_MASTER` BIGINT(20) UNSIGNED NOT NULL AUTO_INCREMENT,
/* Some more attribute fields (mainly integers) ... */
`FK_KAT` BIGINT(20) UNSIGNED NOT NULL,
/* Some more content fields (mainly varchars/integers) ... */
`STAMP_START` DATETIME NULL DEFAULT NULL,
`STAMP_END` DATETIME NULL DEFAULT NULL,
PRIMARY KEY (`ID_AD_MASTER`),
INDEX `TEST1` (`STAMP_START`, `FK_KAT`),
INDEX `TEST2` (`FK_KAT`, `STAMP_START`)
)
COLLATE='utf8_general_ci'
ENGINE=InnoDB
ROW_FORMAT=DYNAMIC
AUTO_INCREMENT=14149037;
I already simplyfied the query as far as possible to better illustrate the Problem. I'm using FORCE INDEX to illustrate my issue here.
This first index is optimized using the explain statement and looks pretty promising (regarding the explain output):
SELECT *
FROM `ad_master_test`
FORCE INDEX (TEST1)
WHERE FK_KAT IN
(94169,94163,94164,94165,94166,94167,94168,94170,94171,94172,
94173,94174,94175,94176,94177,94162,99606,94179,94180,94181,
94182,94183,94184,94185,94186,94187,94188,94189,94190,94191,
94192,94193,94194,94195,94196,94197,94198,94199,94200,94201,
94202,94203,94204,94205,94206,94207,94208,94209,94210,94211,
94212,94213,94214,94215,94216,94217,94218,94219,94220,94221,
94222,94223,94224,94225,94226,94227,94228,94229,94230,94231,
94232,94233,94234,94235,94236,94237,94238,94239,94240,94241,
94178,94161)
ORDER BY STAMP_START DESC
LIMIT 24
Results in this explain:
id select_type table type possible_keys key key_len ref rows Extra
1 SIMPLE ad_master_test index (NULL) TEST1 14 (NULL) 24 Using where
And this profile:
Status Duration
starting 0.000180
checking permissions 0.000015
Opening tables 0.000041
After opening tables 0.000013
System lock 0.000011
Table lock 0.000013
init 0.000115
optimizing 0.000044
statistics 0.000050
preparing 0.000039
executing 0.000009
Sorting result 0.000016
Sending data 4.827512
end 0.000023
query end 0.000008
closing tables 0.000004
Unlocking tables 0.000014
freeing items 0.000011
updating status 0.000132
cleaning up 0.000021
The second index is just the fields reversed (the way I understood it here: https://dev.mysql.com/doc/refman/8.0/en/order-by-optimization.html ) which looks pretty horrible (regarding the explain output):
SELECT *
FROM `ad_master_test`
FORCE INDEX (TEST2)
WHERE FK_KAT IN (94169,94163,94164,94165,94166,94167,94168,94170,94171,94172,94173,94174,94175,94176,94177,94162,99606,94179,94180,94181,94182,94183,94184,94185,94186,94187,94188,94189,94190,94191,94192,94193,94194,94195,94196,94197,94198,94199,94200,94201,94202,94203,94204,94205,94206,94207,94208,94209,94210,94211,94212,94213,94214,94215,94216,94217,94218,94219,94220,94221,94222,94223,94224,94225,94226,94227,94228,94229,94230,94231,94232,94233,94234,94235,94236,94237,94238,94239,94240,94241,94178,94161)
ORDER BY STAMP_START DESC
LIMIT 24
Results in this explain:
id select_type table type possible_keys key key_len ref rows Extra
1 SIMPLE ad_master_test range TEST2 TEST2 8 (NULL) 497.766 Using index condition; Using filesort
And this profile:
Status Duration
starting 0.000087
checking permissions 0.000007
Opening tables 0.000021
After opening tables 0.000007
System lock 0.000006
Table lock 0.000005
init 0.000058
optimizing 0.000023
statistics 0.000654
preparing 0.000480
executing 0.000008
Sorting result 0.433607
Sending data 0.001681
end 0.000010
query end 0.000007
closing tables 0.000003
Unlocking tables 0.000011
freeing items 0.000010
updating status 0.000158
cleaning up 0.000021
Edit: When not using force index the explain changes as following:
id select_type table type possible_keys key key_len ref rows Extra
1 SIMPLE ad_master_test index TEST2 TEST1 14 (NULL) 345 Using where
The profile and runtime stays (as expected) the same it was when using FORCE INDEX on the TEST1 index.
/Edit
I honestly can't wrap my head around this. Why does the explain and the actual query performance differ that extremely. What does the server do while the 5 seconds "Sending data"?
It looks like there are some TEXT or BLOB or even large VARCHAR columns?? 12.1GB/6.8M = 1.8KB. If you don't need them, don't fetch them; this may speed up any such query. How much RAM do you have?
The two indexes do seem to take a different time (4.8s vs 0.4s).
(STAMP_START, FK_KAT)
This avoids the "filesort" by scanning the index BTree in the desired order. It has to check every entry for a matching fk_kat. I think it will stop after 24 (see LIMIT) matching rows, but that could be the first 24 (fast), the last 24 (very slow), or something in between.
(FK_KAT, STAMP_START)
This show go directly to all 82 ids, scan each (assuming not unique), collecting perhaps hundreds of rows. Then do a "filesort". (Note: this will be a disk sort if any TEXT columns are being fetched.) Then deliver the first 24. (Oops; I don't think MariaDB 10.1 has that feature.)
Even though this takes more steps, by avoiding the full index scan it turns out to be faster.
Other Notes
key_buffer_size = 20G - Don't use MyISAM. But if you do, change this to 10% of RAM. If you don't, change it to 30M and give 70% of RAM to innodb_buffer_pool_size.
If you want to discuss further, please provide EXPLAIN FORMAT=JSON SELECT ... for each query. This will have the "cost" analysis, which should explain why it picked the worse index.
Another experiment
Instead of SELECT *, run the timings and EXPLAINs with just SELECT ID_AD_MASTER. If that proves to be "fast", then reformulate the query thus:
SELECT b.* -- (or selected columns from `b`)
FROM ( SELECT ID_AD_MASTER FROM ... ) AS a
JOIN ad_master_test AS b USING(ad_master_test)
ORDER BY STAMP_START DESC ; -- (yes, repeat the ORDER BY)
Suggestions to consider for your my.cnf [mysqld] section
(RPS is RatePerSecond)
thread_handling=pool-of-threads # from one-thread-per-connection see refman
max_connections=100 # from 151 because max_used_connections < 60
read_rnd_buffer_size=256K # from 1M to reduce RAM used, < handler_read_rnd_next RPS
aria_pagecache_division_limit=50 # from 100 for WARM cache for < aria_pagecache_reads RPS
key_cache_division_limit=50 # from 100 for WARM cache for < key_reads
key_buffer_size=2G # from 5G Mysqltuner reports 1G used (this could be WRONG-test it)
innodb_io_capacity=30000 # from 200 since you have SSD
innodb_buffer_pool_instances=8 # from 16 for your volume of data
innodb_lru_scan_depth=128 # from 1024 to conserve CPU every SECOND see refman
innodb_buffer_pool_size=36G # from 30G for effective size of 32G when
innodb_change_buffer_pool_size=10 # from 25% set aside for Del,Ins,Upd activities
for additional suggestions, view profile, Network profile, for contact info including my Skype ID. There are additional opportunities available to improve your configuration.
Remember the advice ONLY one change per day, monitor, if positive results proceed to next suggestion. Otherwise let me know any seriously adverse result and which change seemed to cause the problem, please.
Analysis of VARIABLES and GLOBAL STATUS:
Observations:
Version: 10.1.26-MariaDB-0+deb9u1
64 GB of RAM
Uptime = 7d 22:50:19
You are not running on Windows.
Running 64-bit version
You appear to be running entirely (or mostly) InnoDB.
The More Important Issues:
A "Load Average" of 1 (or more) usually indicates an inefficient query. This is further confirmed by the large value for Created_tmp_disk_tables and Handler_read_rnd_next for a "mere" 91 queries per second. Let's see the slowest query. See Recommendations for further investiataion.
thread_cache_size = 20
Having gotten rid of MyISAM, there is no need to such a large key_buffer_size; decrease from 5G to 50M.
I'm not a fan of ROW_FORMAT=COMPRESSED; this has two relevant impacts for your Question: Increased CPU for compress/uncompress, and need for extra buffer_pool space. On the other hand, the GLOBAL STATUS does not indicate that 30GB is "too small". Is there a need for shrinking the disk space usage?
You have turned off some optimizations? Was this in response to some other problem?
Details and other observations:
( (key_buffer_size - 1.2 * Key_blocks_used * 1024) / _ram ) = (5120M - 1.2 * 25 * 1024) / 65536M = 7.8% -- Percent of RAM wasted in key_buffer.
-- Decrease key_buffer_size.
( Key_blocks_used * 1024 / key_buffer_size ) = 25 * 1024 / 5120M = 0.00% -- Percent of key_buffer used. High-water-mark.
-- Lower key_buffer_size to avoid unnecessary memory usage.
( innodb_buffer_pool_size / _ram ) = 30720M / 65536M = 46.9% -- % of RAM used for InnoDB buffer_pool
( table_open_cache ) = 4,096 -- Number of table descriptors to cache
-- Several hundred is usually good.
( Innodb_os_log_written / (Uptime / 3600) / innodb_log_files_in_group / innodb_log_file_size ) = 6,714,002,432 / (687019 / 3600) / 2 / 1024M = 0.0164 -- Ratio
-- (see minutes)
( Uptime / 60 * innodb_log_file_size / Innodb_os_log_written ) = 687,019 / 60 * 1024M / 6714002432 = 1,831 -- Minutes between InnoDB log rotations Beginning with 5.6.8, this can be changed dynamically; be sure to also change my.cnf.
-- (The recommendation of 60 minutes between rotations is somewhat arbitrary.) Adjust innodb_log_file_size. (Cannot change in AWS.)
( default_tmp_storage_engine ) = default_tmp_storage_engine =
( Innodb_rows_deleted / Innodb_rows_inserted ) = 1,319,619 / 2015717 = 0.655 -- Churn
-- "Don't queue it, just do it." (If MySQL is being used as a queue.)
( innodb_thread_concurrency ) = 0 -- 0 = Let InnoDB decide the best for concurrency_tickets.
-- Set to 0 or 64. This may cut back on CPU.
( innodb_print_all_deadlocks ) = innodb_print_all_deadlocks = OFF -- Whether to log all Deadlocks.
-- If you are plagued with Deadlocks, turn this on. Caution: If you have lots of deadlocks, this may write a lot to disk.
( innodb_buffer_pool_populate ) = OFF = 0 -- NUMA control
( query_prealloc_size / _ram ) = 24,576 / 65536M = 0.00% -- For parsing. Pct of RAM
( query_alloc_block_size / _ram ) = 16,384 / 65536M = 0.00% -- For parsing. Pct of RAM
( net_buffer_length / max_allowed_packet ) = 16,384 / 16M = 0.10%
( bulk_insert_buffer_size / _ram ) = 8M / 65536M = 0.01% -- Buffer for multi-row INSERTs and LOAD DATA
-- Too big could threaten RAM size. Too small could hinder such operations.
( Created_tmp_tables ) = 19,436,364 / 687019 = 28 /sec -- Frequency of creating "temp" tables as part of complex SELECTs.
( Created_tmp_disk_tables ) = 17,887,832 / 687019 = 26 /sec -- Frequency of creating disk "temp" tables as part of complex SELECTs
-- increase tmp_table_size and max_heap_table_size.
Check the rules for temp tables on when MEMORY is used instead of MyISAM. Perhaps minor schema or query changes can avoid MyISAM.
Better indexes and reformulation of queries are more likely to help.
( Created_tmp_disk_tables / Questions ) = 17,887,832 / 62591791 = 28.6% -- Pct of queries that needed on-disk tmp table.
-- Better indexes / No blobs / etc.
( Created_tmp_disk_tables / Created_tmp_tables ) = 17,887,832 / 19436364 = 92.0% -- Percent of temp tables that spilled to disk
-- Maybe increase tmp_table_size and max_heap_table_size; improve indexes; avoid blobs, etc.
( tmp_table_size ) = 64M -- Limit on size of MEMORY temp tables used to support a SELECT
-- Decrease tmp_table_size to avoid running out of RAM. Perhaps no more than 64M.
( Handler_read_rnd_next ) = 703,386,895,308 / 687019 = 1023824 /sec -- High if lots of table scans
-- possibly inadequate keys
( Handler_read_rnd_next / Com_select ) = 703,386,895,308 / 58493862 = 12,024 -- Avg rows scanned per SELECT. (approx)
-- Consider raising read_buffer_size
( Select_full_join ) = 15,981,913 / 687019 = 23 /sec -- joins without index
-- Add suitable index(es) to tables used in JOINs.
( Select_full_join / Com_select ) = 15,981,913 / 58493862 = 27.3% -- % of selects that are indexless join
-- Add suitable index(es) to tables used in JOINs.
( Select_scan ) = 1,510,902 / 687019 = 2.2 /sec -- full table scans
-- Add indexes / optimize queries (unless they are tiny tables)
( sort_buffer_size ) = 8M -- One per thread, malloced at full size until 5.6.4, so keep low; after that bigger is ok.
-- This may be eating into available RAM; recommend no more than 2M.
( binlog_format ) = binlog_format = STATEMENT -- STATEMENT/ROW/MIXED. ROW is preferred; it may become the default.
( slow_query_log ) = slow_query_log = OFF -- Whether to log slow queries. (5.1.12)
( long_query_time ) = 10 -- Cutoff (Seconds) for defining a "slow" query.
-- Suggest 2
( Threads_created / Connections ) = 3,081 / 303642 = 1.0% -- Rapidity of process creation
-- Increase thread_cache_size (non-Windows)
Abnormally large:
Connection_errors_peer_address = 2
Handler_icp_attempts = 71206 /sec
Handler_icp_match = 71206 /sec
Handler_read_next / Handler_read_key = 283
Handler_read_prev = 12522 /sec
Handler_read_rnd_deleted = 16 /sec
Innodb_rows_read = 1255832 /sec
Key_blocks_unused = 4.24e+6
Performance_schema_table_instances_lost = 32
Select_range / Com_select = 33.1%
Sort_scan = 27 /sec
Tc_log_page_size = 4,096
innodb_lru_scan_depth / innodb_io_capacity = 5.12
innodb_max_dirty_pages_pct_lwm = 0.10%
max_relay_log_size = 100MB
myisam_sort_buffer_size = 512MB
Abnormal strings:
Compression = ON
innodb_cleaner_lsn_age_factor = HIGH_CHECKPOINT
innodb_empty_free_list_algorithm = BACKOFF
innodb_fast_shutdown = 1
innodb_foreground_preflush = EXPONENTIAL_BACKOFF
innodb_log_checksum_algorithm = INNODB
myisam_stats_method = NULLS_UNEQUAL
opt_s__engine_condition_pushdown = off
opt_s__mrr = off
opt_s__mrr_cost_based = off