Related
MySQL v8.0.23, InnoDB engine
I have configured innodb buffer pool size =80GB, and max threads used are 423. However, there is significant difference in memory consumed by mysql vs expected max value of RAM usage. What could be the contributing factors for the extra 17GB?
SELECT ( ##key_buffer_size + ##innodb_buffer_pool_size + 423 * (##read_buffer_size + ##read_rnd_buffer_size + ##sort_buffer_size + ##join_buffer_size + ##tmp_table_size )) / (1024*1024*1024)
AS MAX_MEMORY_GB;
+---------------+
| MAX_MEMORY_GB |
+---------------+
| 86.5277 |
+---------------+
As per above, max usage of memory ~87GB. However, top and free show following:
free -mh
total used free shared buff/cache available
Mem: 119G 104G 876M 908K 14G 13G
Swap: 0B 0B 0B
top -o %MEM
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
26502 mysql 20 0 0.103t 0.102t 12120 S 113.3 86.7 265894:41 mysqld
129344 root 19 -1 352404 138448 133868 S 0.0 0.1 9:12.45 systemd-journal
Output of pt-mysql-summary
# InnoDB #####################################################
Version | 8.0.23
Buffer Pool Size | 80.0G
Buffer Pool Fill | 90%
Buffer Pool Dirty | 0%
File Per Table | ON
Page Size | 16k
Log File Size | 2 * 512.0M = 1.0G
Log Buffer Size | 32M
Flush Method | fsync
Flush Log At Commit | 0
XA Support |
Checksums |
Doublewrite | ON
R/W I/O Threads | 32 32
I/O Capacity | 200
Thread Concurrency | 0
Concurrency Tickets | 5000
Commit Concurrency | 0
Txn Isolation Level |
Adaptive Flushing | ON
Adaptive Checkpoint |
Checkpoint Age | 3M
InnoDB Queue | 0 queries inside InnoDB, 0 queries in queue
Oldest Transaction | 0 Seconds
History List Len | 36
Read Views | 0
Undo Log Entries | 1 transactions, 2 total undo, 2 max undo
Pending I/O Reads | 0 buf pool reads, 0 normal AIO, 0 ibuf AIO, 0 preads
Pending I/O Writes | 0 buf pool (0 LRU, 0 flush list, 0 page); 0 AIO, 0 sync, 0 log IO (0 log, 0 chkp); 0 pwrites
Pending I/O Flushes | 478553 buf pool, 0 log
Transaction States | 1xACTIVE (PREPARED), 4xnot started
You are missing a critical ( and ) in your query, the following may be more accurate for you.
SELECT ( ##key_buffer_size + ##innodb_buffer_pool_size + ( 423 * (##read_buffer_size + ##read_rnd_buffer_size + ##sort_buffer_size + ##join_buffer_size + ##tmp_table_size ))) / (1024*1024*1024)
AS MAX_MEMORY_GB;
I have searched the internet in many places to find some answers to problems in the database of a Magento installation.
The website is not available at such a moment (lasting most of the time about 15 minutes and resolves itself), not frontend and not backend.
I have tried various tuningscripts for mysql, setting indexing to manual, moving the DB to a dedicated server, etc. The logs of the Ubuntu OS nor the Magento logs show any unusual error message. Fail2Ban is installed against DDOS.
The mysql process is OR using 100+ percent OR using 0.0 percent (!!!) but the whole server seems te freeze. Sometimes even my typed characted with Putty/SSH are not shown by the server until it restores itself.
The dedicated DB-server has 32Gb RAM, mysql uses about 25G.
Top of ToP:
840 mysql 20 0 25,640g 0,011t 6632 S 26,2 34,2 270:08.31 mysqld
Settings my.cnf:
[mysqld]
key_buffer = 512M
max_allowed_packet = 64M
thread_stack = 512K
thread_cache_size = 512
sort_buffer_size = 24M
read_buffer_size = 8M
read_rnd_buffer_size = 24M
join_buffer_size = 128M
max_connections = 1024
concurrent_insert = 2
connect_timeout = 5
table_open_cache = 12288
tmp_table_size = 512M
max_heap_table_size = 512M
bulk_insert_buffer_size = 512M
open-files-limit = 8192
open-files = 1024
query_cache_type=1
query_cache_limit = 4M
query_cache_size = 512M
slow_query_log_file = /var/log/mysql/slow.log
slow_query_log = 0
long_query_time = 0.2
expire_logs_days = 10
max_binlog_size = 1024M
binlog_cache_size = 32K
innodb_thread_concurrency = 16
innodb_commit_concurrency = 2
innodb_buffer_pool_size = 16G
innodb_log_file_size = 512M
innodb_additional_mem_pool_size = 8M
#innodb_data_file_path = ibdata1:1024M:autoextend
innodb_file_per_table
innodb_flush_log_at_trx_commit = 2
innodb_log_buffer_size = 2G
Output of Show full processlist:
| Id | User | Host | db | Command | Time | State | Info |
++
| 76636 | juwelierwebshop | ...:40316 | juwelierwebshop | Query | 328 | query end | INSERT INTO abctools_merkgevuld (sku) VALUES ('JUPA22655') |
| 76638 | juwelierwebshop | ...:40344 | juwelierwebshop | Sleep | 328 | | NULL |
| 76642 | juwelierwebshop | ...:40484 | juwelierwebshop | Query | 54 | query end | SELECT `main_table`.* FROM `bolconnect_items_be` AS `main_table` WHERE (product_id = '62622') |
| 76695 | juwelierwebshop | ...:54112 | juwelierwebshop | Sleep | 603 | | NULL |
| 76696 | juwelierwebshop | ...:54126 | juwelierwebshop | Query | 54 | update | INSERT INTO `bolconnect_offerlist` (`ean`, `prijs`, `leverancier`, `sellerid`, `sellerrating`, `avdesc`, `koopbox`, `updated_at`) VALUES ('8718834545421', '619', 'abc', '1121928', '8.8', '5 - 7 dagen', '1', '2018-09-07 12:10:10') |
| 77039 | juwelierwebshop | ...:40290 | juwelierwebshop | Query | 54 | Sending data | INSERT INTO `sales_flat_order_grid` (`entity_id`, `status`, `store_id`, `customer_id`, `base_grand_total`, `base_total_paid`, `grand_total`, `total_paid`, `increment_id`, `base_currency_code`, `order_currency_code`, `store_name`, `created_at`, `updated_at`, `billing_name`, `shipping_name`) SELECT `main_table`.`entity_id`, `main_table`.`status`, `main_table`.`store_id`, `main_table`.`customer_id`, `main_table`.`base_grand_total`, `main_table`.`base_total_paid`, `main_table`.`grand_total`, `main_table`.`total_paid`, `main_table`.`increment_id`, `main_table`.`base_currency_code`, `main_table`.`order_currency_code`, `main_table`.`store_name`, `main_table`.`created_at`, `main_table`.`updated_at`, CONCAT(IFNULL(table_billing_name.firstname, ''), ' ', IFNULL(table_billing_name.lastname, '')) AS `billing_name`, CONCAT(IFNULL(table_shipping_name.firstname, ''), ' ', IFNULL(table_shipping_name.lastname, '')) AS `shipping_name` FROM `sales_flat_order` AS `main_table`
LEFT JOIN `sales_flat_order_address` AS `table_billing_name` ON `main_table`.`billing_address_id`=`table_billing_name`.`entity_id`
LEFT JOIN `sales_flat_order_address` AS `table_shipping_name` ON `main_table`.`shipping_address_id`=`table_shipping_name`.`entity_id` WHERE (main_table.entity_id IN('27894')) ON DUPLICATE KEY UPDATE `entity_id` = VALUES(`entity_id`), `status` = VALUES(`status`), `store_id` = VALUES(`store_id`), `customer_id` = VALUES(`customer_id`), `base_grand_total` = VALUES(`base_grand_total`), `base_total_paid` = VALUES(`base_total_paid`), `grand_total` = VALUES(`grand_total`), `total_paid` = VALUES(`total_paid`), `increment_id` = VALUES(`increment_id`), `base_currency_code` = VALUES(`base_currency_code`), `order_currency_code` = VALUES(`order_currency_code`), `store_name` = VALUES(`store_name`), `created_at` = VALUES(`created_at`), `updated_at` = VALUES(`updated_at`), `billing_name` = VALUES(`billing_name`), `shipping_name` = VALUES(`shipping_name`) |
| 77041 | juwelierwebshop | ...:40578 | juwelierwebshop | Query | 54 | statistics | SELECT `catalog_product_entity`.* FROM `catalog_product_entity` WHERE (entity_id ='45702') |
| 77042 | juwelierwebshop | ...:40766 | juwelierwebshop | Query | 54 | Sending data | SELECT COUNT(DISTINCT e.entity_id) FROM `catalog_product_flat_2` AS `e`
INNER JOIN `report_viewed_product_index` AS `idx_table` ON (idx_table.product_id=e.entity_id) AND (idx_table.visitor_id = '')
INNER JOIN `catalog_category_product_index` AS `cat_index` ON cat_index.product_id=e.entity_id AND cat_index.store_id=2 AND cat_index.category_id = '2'
LEFT JOIN `core_store` AS `store_index` ON store_index.store_id = idx_table.store_id
LEFT JOIN `core_store_group` AS `store_group_index` ON store_index.group_id = store_group_index.group_id
LEFT JOIN `catalog_category_product_index` AS `store_cat_index` ON store_cat_index.product_id = e.entity_id AND store_cat_index.store_id = idx_table.store_id AND store_cat_index.category_id=store_group_index.root_category_id WHERE (cat_index.visibility IN(3, 2, 4) OR store_cat_index.visibility IN(3, 2, 4)) |
| 77043 | juwelierwebshop | ...:40930 | juwelierwebshop | Query | 54 | statistics | SELECT `catalog_category_entity`.* FROM `catalog_category_entity` WHERE (entity_id ='37') |
| 77047 | juwelierwebshop | ...:42478 | juwelierwebshop | Query | 53 | Sending data | SELECT COUNT(DISTINCT e.entity_id) FROM `catalog_product_entity` AS `e`
INNER JOIN `catalog_product_entity_int` AS `at_status` ON (`at_status`.`entity_id` = `e`.`entity_id`) AND (`at_status`.`attribute_id` = '96') AND (`at_status`.`store_id` = 0)
INNER JOIN `catalog_product_entity_int` AS `at_visibility` ON (`at_visibility`.`entity_id` = `e`.`entity_id`) AND (`at_visibility`.`attribute_id` = '102') AND (`at_visibility`.`store_id` = 0)
INNER JOIN `catalog_product_entity_varchar` AS `at_name` ON (`at_name`.`entity_id` = `e`.`entity_id`) AND (`at_name`.`attribute_id` = '71') AND (`at_name`.`store_id` = 0) WHERE (at_name.value LIKE '%BUDDHA TO BUDDHA 811 SIGNIFY CORD BRACELET BROWN%') |
| 77050 | juwelierwebshop | ...:43538 | juwelierwebshop | Query | 54 | updating | UPDATE `sales_flat_quote_address` SET `quote_id` = '62660', `created_at` = '2018-09-02 19:38:03', `updated_at` = '2018-09-07 12:10:10', `customer_id` = '22445', `save_in_address_book` = '0', `customer_address_id` = '45277', `address_type` = 'billing', `email` = 'abc#email.nl', `prefix` = NULL, `firstname` = 'Pieter', `middlename` = NULL, `lastname` = 'Slits', `suffix` = NULL, `company` = NULL, `street` = 'Steurgat\n92', `city` = 'Gorinchem', `region` = 'Zuid-Holland', `region_id` = NULL, `postcode` = '4208BL', `country_id` = 'NL', `telephone` = '0621263156', `fax` = NULL, `same_as_billing` = '0', `free_shipping` = '0', `collect_shipping_rates` = '0', `shipping_method` = NULL, `shipping_description` = NULL, `weight` = '0', `subtotal` = '0', `base_subtotal` = '0', `subtotal_with_discount` = '0', `base_subtotal_with_discount` = '0', `tax_amount` = '0', `base_tax_amount` = '0', `shipping_amount` = '0', `base_shipping_amount` = '0', `shipping_tax_amount` = '0', `base_shipping_tax_amount` = '0', `discount_amount` = '0', `base_discount_amount` = '0', `grand_total` = '0', `base_grand_total` = '0', `customer_notes` = NULL, `applied_taxes` = 'a:0:{}', `discount_description` = NULL, `shipping_discount_amount` = NULL, `base_shipping_discount_amount` = NULL, `subtotal_incl_tax` = '0', `base_subtotal_total_incl_tax` = NULL, `hidden_tax_amount` = NULL, `base_hidden_tax_amount` = NULL, `shipping_hidden_tax_amount` = NULL, `base_shipping_hidden_tax_amnt` = NULL, `shipping_incl_tax` = '0', `base_shipping_incl_tax` = '0', `vat_id` = NULL, `vat_is_valid` = NULL, `vat_request_id` = NULL, `vat_request_date` = NULL, `vat_request_success` = NULL, `gift_message_id` = NULL, `kp_id` = NULL, `payment_fee` = '0', `base_payment_fee` = '0', `payment_fee_tax` = '0', `base_payment_fee_tax` = '0' WHERE (address_id='121757') |
| 77052 | juwelierwebshop | ...:43840 | juwelierwebshop | Query | 54 | init | commit |
| 77053 | juwelierwebshop | ...:44590 | juwelierwebshop | Query | 204 | removing tmp table | SELECT `customer_address_entity_varchar`.`entity_id`, `customer_address_entity_varchar`.`attribute_id`, `customer_address_entity_varchar`.`value` FROM `customer_address_entity_varchar` WHERE (entity_type_id =2) AND (entity_id IN (45277)) AND (attribute_id IN ('26', '24', '27', '32', '20', '22', '21', '30', '19', '28', '23', '31', '36', '39', '38')) UNION ALL SELECT `customer_address_entity_int`.`entity_id`, `customer_address_entity_int`.`attribute_id`, `customer_address_entity_int`.`value` FROM `customer_address_entity_int` WHERE (entity_type_id =2) AND (entity_id IN (45277)) AND (attribute_id IN ('29', '37', '40')) UNION ALL SELECT `customer_address_entity_text`.`entity_id`, `customer_address_entity_text`.`attribute_id`, `customer_address_entity_text`.`value` FROM `customer_address_entity_text` WHERE (entity_type_id =2) AND (entity_id IN (45277)) AND (attribute_id IN ('25')) |
| 77055 | juwelierwebshop | ...:44704 | juwelierwebshop | Query | 203 | Opening tables | SELECT `customer_address_entity_varchar`.`entity_id`, `customer_address_entity_varchar`.`attribute_id`, `customer_address_entity_varchar`.`value` FROM `customer_address_entity_varchar` WHERE (entity_type_id =2) AND (entity_id IN (45277)) AND (attribute_id IN ('26', '24', '27', '32', '20', '22', '21', '30', '19', '28', '23', '31', '36', '39', '38')) UNION ALL SELECT `customer_address_entity_int`.`entity_id`, `customer_address_entity_int`.`attribute_id`, `customer_address_entity_int`.`value` FROM `customer_address_entity_int` WHERE (entity_type_id =2) AND (entity_id IN (45277)) AND (attribute_id IN ('29', '37', '40')) UNION ALL SELECT `customer_address_entity_text`.`entity_id`, `customer_address_entity_text`.`attribute_id`, `customer_address_entity_text`.`value` FROM `customer_address_entity_text` WHERE (entity_type_id =2) AND (entity_id IN (45277)) AND (attribute_id IN ('25')) |
| 77056 | juwelierwebshop | ...:44950 | juwelierwebshop | Query | 199 | Opening tables | SELECT `customer_address_entity_varchar`.`entity_id`, `customer_address_entity_varchar`.`attribute_id`, `customer_address_entity_varchar`.`value` FROM `customer_address_entity_varchar` WHERE (entity_type_id =2) AND (entity_id IN (45277)) AND (attribute_id IN ('26', '24', '27', '32', '20', '22', '21', '30', '19', '28', '23', '31', '36', '39', '38')) UNION ALL SELECT `customer_address_entity_int`.`entity_id`, `customer_address_entity_int`.`attribute_id`, `customer_address_entity_int`.`value` FROM `customer_address_entity_int` WHERE (entity_type_id =2) AND (entity_id IN (45277)) AND (attribute_id IN ('29', '37', '40')) UNION ALL SELECT `customer_address_entity_text`.`entity_id`, `customer_address_entity_text`.`attribute_id`, `customer_address_entity_text`.`value` FROM `customer_address_entity_text` WHERE (entity_type_id =2) AND (entity_id IN (45277)) AND (attribute_id IN ('25')) |
| 77058 | juwelierwebshop | ...:45508 | juwelierwebshop | Query | 54 | Sending data | SELECT `catalog_product_website`.`website_id` FROM `catalog_product_website` WHERE (product_id = 16722) |
| 77063 | juwelierwebshop | ...:45836 | juwelierwebshop | Query | 54 | update | INSERT INTO `sales_flat_quote_address` (`quote_id`, `created_at`, `updated_at`, `customer_id`, `address_type`, `city`, `region`, `region_id`, `postcode`, `country_id`, `same_as_billing`, `collect_shipping_rates`, `shipping_method`, `shipping_description`, `shipping_amount`, `base_shipping_amount`) VALUES ('65679', '2018-09-07 12:10:10', '2018-09-07 12:10:10', NULL, 'shipping', NULL, NULL, '0', NULL, 'NL', '1', '1', 'flatrate_flatrate', NULL, '0', '0') |
| 77067 | juwelierwebshop | ...:46232 | juwelierwebshop | Query | 229 | update | INSERT INTO `sales_flat_quote` (`store_id`, `created_at`, `updated_at`, `store_to_base_rate`, `store_to_quote_rate`, `base_currency_code`, `store_currency_code`, `quote_currency_code`, `remote_ip`, `global_currency_code`, `base_to_global_rate`, `base_to_quote_rate`, `is_changed`, `allow_alerts`) VALUES ('2', '2018-09-07 12:07:15', '2018-09-07 12:07:15', '1', '1', 'EUR', 'EUR', 'EUR', '81.169.242.132', 'EUR', '1', '1', '1', '1') |
| 77069 | juwelierwebshop | ...:46678 | juwelierwebshop | Query | 222 | query end | SELECT `t_d`.`entity_id`, `t_d`.`attribute_id`, `t_d`.`value` AS `default_value`, `t_s`.`value` AS `store_value`, IF(t_s.value_id IS NULL, t_d.value, t_s.value) AS `value` FROM `catalog_category_entity_varchar` AS `t_d`
LEFT JOIN `catalog_category_entity_varchar` AS `t_s` ON t_s.attribute_id = t_d.attribute_id AND t_s.entity_id = t_d.entity_id AND t_s.store_id = 2 WHERE (t_d.entity_type_id = 3) AND (t_d.entity_id IN (260)) AND (t_d.attribute_id IN ('41')) AND (t_d.store_id = 0) |
| 77074 | juwelierwebshop | ...:46984 | juwelierwebshop | Query | 216 | query end | SELECT `e`.*, IF(at_is_active.value_id > 0, at_is_active.value, at_is_active_default.value) AS `is_active` FROM `catalog_category_entity` AS `e`
INNER JOIN `catalog_category_entity_int` AS `at_is_active_default` ON (`at_is_active_default`.`entity_id` = `e`.`entity_id`) AND (`at_is_active_default`.`attribute_id` = '42') AND `at_is_active_default`.`store_id` = 0
LEFT JOIN `catalog_category_entity_int` AS `at_is_active` ON (`at_is_active`.`entity_id` = `e`.`entity_id`) AND (`at_is_active`.`attribute_id` = '42') AND (`at_is_active`.`store_id` = 2) WHERE (`e`.`entity_type_id` = '3') AND (`e`.`entity_id` IN('365')) AND (IF(at_is_active.value_id > 0, at_is_active.value, at_is_active_default.value) = '1') |
| 77076 | juwelierwebshop | ...:47082 | juwelierwebshop | Query | 214 | query end | SELECT `t_d`.`entity_id`, `t_d`.`attribute_id`, `t_d`.`value` AS `default_value`, `t_s`.`value` AS `store_value`, IF(t_s.value_id IS NULL, t_d.value, t_s.value) AS `value` FROM `catalog_category_entity_varchar` AS `t_d`
LEFT JOIN `catalog_category_entity_varchar` AS `t_s` ON t_s.attribute_id = t_d.attribute_id AND t_s.entity_id = t_d.entity_id AND t_s.store_id = 2 WHERE (t_d.entity_type_id = 3) AND (t_d.entity_id IN (187)) AND (t_d.attribute_id IN ('41')) AND (t_d.store_id = 0)
.. and more
Output ulimit -a:
core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 1030957
max locked memory (kbytes, -l) 64
max memory size (kbytes, -m) unlimited
open files (-n) 1024
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 10240
cpu time (seconds, -t) unlimited
max user processes (-u) 1030957
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited
Rate Per Second=RPS
Suggestions to consider for your my.cnf [mysqld] section
thread_cache_size=100 # from 512 see refman 5.7 for CAP of 100 suggestions to avoid OOM
innodb_log_buffer_size=512M # from 2G - BUFFER should NOT be > file size
innodb_log_file_size=2G # from 512M for a reasonable relationship to BUFFER
the following 4 are per CONNECTION values and you are OVER the TOP
sort_buffer_size=4M # from 24M default is 2M
read_buffer_size=256K # from 8M default is 128K to reduce handler_read_next RPS
read_rnd_buffer_size=512K # from 24M default is 256K to reduce handler_read_rnd_next RPS
join_buffer_size=256K # from 128M default is 128K for row pointers
Please view my profile, Network profile for contact info including my Skype ID and get in touch.
As long as my post is getting too long, i will post the additional info here...
SHOW GLOBAL STATUS:
Aborted_clients
146
Aborted_connects
286
Binlog_cache_disk_use
0
Binlog_cache_use
0
Binlog_stmt_cache_disk_use
0
Binlog_stmt_cache_use
0
Bytes_received
262381827589
Bytes_sent
934591758226
Com_admin_commands
13
Com_assign_to_keycache
0
Com_alter_db
0
Com_alter_db_upgrade
0
Com_alter_event
0
Com_alter_function
0
Com_alter_procedure
0
Com_alter_server
0
Com_alter_table
0
Com_alter_tablespace
0
Com_alter_user
0
Com_analyze
0
Com_begin
21257645
Com_binlog
0
Com_call_procedure
0
Com_change_db
303
Com_change_master
0
Com_check
0
Com_checksum
0
Com_commit
21257476
Com_create_db
0
Com_create_event
0
Com_create_function
0
Com_create_index
0
Com_create_procedure
0
Com_create_server
0
Com_create_table
1981
Com_create_trigger
0
Com_create_udf
0
Com_create_user
0
Com_create_view
0
Com_dealloc_sql
0
Com_delete
8394541
Com_delete_multi
63886
Com_do
0
Com_drop_db
0
Com_drop_event
0
Com_drop_function
0
Com_drop_index
0
Com_drop_procedure
0
Com_drop_server
0
Com_drop_table
1980
Com_drop_trigger
0
Com_drop_user
0
Com_drop_view
0
Com_empty_query
0
Com_execute_sql
0
Com_flush
11
Com_get_diagnostics
0
Com_grant
0
Com_ha_close
0
Com_ha_open
0
Com_ha_read
0
Com_help
0
Com_insert
11568107
Com_insert_select
1143891
Com_install_plugin
0
Com_kill
0
Com_load
0
Com_lock_tables
0
Com_optimize
0
Com_preload_keys
0
Com_prepare_sql
0
Com_purge
0
Com_purge_before_date
0
Com_release_savepoint
0
Com_rename_table
0
Com_rename_user
0
Com_repair
0
Com_replace
12
Com_replace_select
0
Com_reset
0
Com_resignal
0
Com_revoke
0
Com_revoke_all
0
Com_rollback
151
Com_rollback_to_savepoint
0
Com_savepoint
0
Com_select
450132043
Com_set_option
2690585
Com_signal
0
Com_show_binlog_events
0
Com_show_binlogs
10
Com_show_charsets
0
Com_show_collations
0
Com_show_create_db
0
Com_show_create_event
0
Com_show_create_func
0
Com_show_create_proc
0
Com_show_create_table
38
Com_show_create_trigger
0
Com_show_databases
0
Variable_name
Value
Com_show_engine_logs
0
Com_show_engine_mutex
0
Com_show_engine_status
0
Com_show_events
0
Com_show_errors
0
Com_show_fields
18447
Com_show_function_code
0
Com_show_function_status
0
Com_show_grants
17
Com_show_keys
9599
Com_show_master_status
26
Com_show_open_tables
0
Com_show_plugins
3
Com_show_privileges
0
Com_show_procedure_code
0
Com_show_procedure_status
0
Com_show_processlist
0
Com_show_profile
0
Com_show_profiles
0
Com_show_relaylog_events
0
Com_show_slave_hosts
0
Com_show_slave_status
26
Com_show_status
3
Com_show_storage_engines
1
Com_show_table_status
20700
Com_show_tables
236
Com_show_triggers
0
Com_show_variables
193
Com_show_warnings
56103
Com_slave_start
0
Com_slave_stop
0
Com_stmt_close
0
Com_stmt_execute
0
Com_stmt_fetch
0
Com_stmt_prepare
0
Com_stmt_reprepare
0
Com_stmt_reset
0
Com_stmt_send_long_data
0
Com_truncate
379
Com_uninstall_plugin
0
Com_unlock_tables
0
Com_update
13681401
Com_update_multi
38290135
Com_xa_commit
0
Com_xa_end
0
Com_xa_prepare
0
Com_xa_recover
0
Com_xa_rollback
0
Com_xa_start
0
Compression
OFF
Connection_errors_accept
0
Connection_errors_internal
0
Connection_errors_max_connections
0
Connection_errors_peer_address
0
Connection_errors_select
0
Connection_errors_tcpwrap
0
Connections
1754922
Created_tmp_disk_tables
32299788
Created_tmp_files
2827
Created_tmp_tables
67858347
Delayed_errors
0
Delayed_insert_threads
0
Delayed_writes
0
Flush_commands
1
Handler_commit
537556955
Handler_delete
21203603
Handler_discover
0
Handler_external_lock
2080617356
Handler_mrr_init
0
Handler_prepare
0
Handler_read_first
5701366
Handler_read_key
26262529583
Handler_read_last
31617
Handler_read_next
109928062206
Handler_read_prev
212093135
Handler_read_rnd
6690959918
Handler_read_rnd_next
26130974483
Handler_rollback
99880
Handler_savepoint
0
Handler_savepoint_rollback
0
Handler_update
80365845
Handler_write
3262327514
Innodb_buffer_pool_dump_status
not started
Innodb_buffer_pool_load_status
not started
Innodb_buffer_pool_pages_data
384476
Innodb_buffer_pool_bytes_data
6299254784
Innodb_buffer_pool_pages_dirty
95
Innodb_buffer_pool_bytes_dirty
1556480
Innodb_buffer_pool_pages_flushed
16849536
Innodb_buffer_pool_pages_free
617389
Innodb_buffer_pool_pages_misc
46707
Innodb_buffer_pool_pages_total
1048572
Innodb_buffer_pool_read_ahead_rnd
0
Innodb_buffer_pool_read_ahead
26706
Innodb_buffer_pool_read_ahead_evicted
0
Innodb_buffer_pool_read_requests
235232077708
Innodb_buffer_pool_reads
157875
Innodb_buffer_pool_wait_free
0
Innodb_buffer_pool_write_requests
523402314
Innodb_data_fsyncs
8277097
Variable_name
Value
Innodb_data_pending_fsyncs
0
Innodb_data_pending_reads
0
Innodb_data_pending_writes
0
Innodb_data_read
3187642368
Innodb_data_reads
195006
Innodb_data_writes
46743559
Innodb_data_written
669383931392
Innodb_dblwr_pages_written
16849536
Innodb_dblwr_writes
2819209
Innodb_have_atomic_builtins
ON
Innodb_log_waits
0
Innodb_log_write_requests
231729452
Innodb_log_writes
26991919
Innodb_os_log_fsyncs
1177300
Innodb_os_log_pending_fsyncs
0
Innodb_os_log_pending_writes
0
Innodb_os_log_written
117223886848
Innodb_page_size
16384
Innodb_pages_created
244392
Innodb_pages_read
194427
Innodb_pages_written
16849536
Innodb_row_lock_current_waits
0
Innodb_row_lock_time
19483692
Innodb_row_lock_time_avg
2345
Innodb_row_lock_time_max
51715
Innodb_row_lock_waits
8306
Innodb_rows_deleted
20957957
Innodb_rows_inserted
27091885
Innodb_rows_read
156431546126
Innodb_rows_updated
25463060
Innodb_num_open_files
573
Innodb_truncated_status_writes
0
Innodb_available_undo_logs
128
Key_blocks_not_flushed
0
Key_blocks_unused
247176
Key_blocks_used
181543
Key_read_requests
478108001
Key_reads
171224
Key_write_requests
42044666
Key_writes
32000663
Last_query_cost
0.000000
Last_query_partial_plans
0
Max_used_connections
108
Not_flushed_delayed_rows
0
Open_files
132
Open_streams
0
Open_table_definitions
695
Open_tables
2157
Opened_files
129209548
Opened_table_definitions
5035
Opened_tables
98173
Performance_schema_accounts_lost
0
Performance_schema_cond_classes_lost
0
Performance_schema_cond_instances_lost
0
Performance_schema_digest_lost
0
Performance_schema_file_classes_lost
0
Performance_schema_file_handles_lost
0
Performance_schema_file_instances_lost
0
Performance_schema_hosts_lost
0
Performance_schema_locker_lost
0
Performance_schema_mutex_classes_lost
0
Performance_schema_mutex_instances_lost
0
Performance_schema_rwlock_classes_lost
0
Performance_schema_rwlock_instances_lost
0
Performance_schema_session_connect_attrs_lost
0
Performance_schema_socket_classes_lost
0
Performance_schema_socket_instances_lost
0
Performance_schema_stage_classes_lost
0
Performance_schema_statement_classes_lost
0
Performance_schema_table_handles_lost
0
Performance_schema_table_instances_lost
0
Performance_schema_thread_classes_lost
0
Performance_schema_thread_instances_lost
0
Performance_schema_users_lost
0
Prepared_stmt_count
0
Qcache_free_blocks
1
Qcache_free_memory
536852816
Qcache_hits
0
Qcache_inserts
0
Qcache_lowmem_prunes
0
Qcache_not_cached
450105606
Qcache_queries_in_cache
0
Qcache_total_blocks
1
Queries
570344429
Questions
570344415
Select_full_join
336026
Select_full_range_join
1914512
Select_range
21082218
Select_range_check
0
Select_scan
37881116
Slave_heartbeat_period
Slave_last_heartbeat
Slave_open_temp_tables
0
Slave_received_heartbeats
Slave_retried_transactions
Slave_running
OFF
Slow_launch_threads
0
Slow_queries
359667
Sort_merge_passes
1690
Sort_range
24277510
Variable_name
Value
Sort_rows
2017426115
Sort_scan
66464668
Ssl_accept_renegotiates
0
Ssl_accepts
0
Ssl_callback_cache_hits
0
Ssl_cipher
Ssl_cipher_list
Ssl_client_connects
0
Ssl_connect_renegotiates
0
Ssl_ctx_verify_depth
0
Ssl_ctx_verify_mode
0
Ssl_default_timeout
0
Ssl_finished_accepts
0
Ssl_finished_connects
0
Ssl_server_not_after
Ssl_server_not_before
Ssl_session_cache_hits
0
Ssl_session_cache_misses
0
Ssl_session_cache_mode
NONE
Ssl_session_cache_overflows
0
Ssl_session_cache_size
0
Ssl_session_cache_timeouts
0
Ssl_sessions_reused
0
Ssl_used_session_cache_entries
0
Ssl_verify_depth
0
Ssl_verify_mode
0
Ssl_version
Table_locks_immediate
1022116993
Table_locks_waited
155
Table_open_cache_hits
1022210351
Table_open_cache_misses
2690
Table_open_cache_overflows
0
Tc_log_max_pages_used
0
Tc_log_page_size
0
Tc_log_page_waits
0
Threads_cached
103
Threads_connected
5
Threads_created
108
Threads_running
1
Uptime
929242
Uptime_since_flush_status
929242
SHOW GLOBAL VARIABLES:
auto_increment_increment
1
auto_increment_offset
1
autocommit
ON
automatic_sp_privileges
ON
back_log
254
basedir
/usr
big_tables
OFF
bind_address
*
binlog_cache_size
32768
binlog_checksum
CRC32
binlog_direct_non_transactional_updates
OFF
binlog_format
STATEMENT
binlog_max_flush_queue_time
0
binlog_order_commits
ON
binlog_row_image
FULL
binlog_rows_query_log_events
OFF
binlog_stmt_cache_size
32768
bulk_insert_buffer_size
536870912
character_set_client
latin1
character_set_connection
latin1
character_set_database
latin1
character_set_filesystem
binary
character_set_results
latin1
character_set_server
latin1
character_set_system
utf8
character_sets_dir
/usr/share/mysql/charsets/
collation_connection
latin1_swedish_ci
collation_database
latin1_swedish_ci
collation_server
latin1_swedish_ci
completion_type
NO_CHAIN
concurrent_insert
ALWAYS
connect_timeout
5
core_file
OFF
datadir
/var/lib/mysql/
date_format
%Y-%m-%d
datetime_format
%Y-%m-%d %H:%i:%s
default_storage_engine
InnoDB
default_tmp_storage_engine
InnoDB
default_week_format
0
delay_key_write
ON
delayed_insert_limit
100
delayed_insert_timeout
300
delayed_queue_size
1000
disconnect_on_expired_password
ON
div_precision_increment
4
end_markers_in_json
OFF
enforce_gtid_consistency
OFF
eq_range_index_dive_limit
10
event_scheduler
OFF
expire_logs_days
10
explicit_defaults_for_timestamp
OFF
flush
OFF
flush_time
0
foreign_key_checks
ON
ft_boolean_syntax
+ -><()~*:""&|
ft_max_word_len
84
ft_min_word_len
4
ft_query_expansion_limit
20
ft_stopword_file
(built-in)
general_log
OFF
general_log_file
/var/lib/mysql/xxx.log
group_concat_max_len
1024
gtid_executed
gtid_mode
OFF
gtid_owned
gtid_purged
have_compress
YES
have_crypt
YES
have_dynamic_loading
YES
have_geometry
YES
have_openssl
DISABLED
have_profiling
YES
have_query_cache
YES
have_rtree_keys
YES
have_ssl
DISABLED
have_symlink
YES
host_cache_size
654
hostname
h2776883.stratoserver.net
ignore_builtin_innodb
OFF
ignore_db_dirs
init_connect
init_file
init_slave
innodb_adaptive_flushing
ON
innodb_adaptive_flushing_lwm
10
innodb_adaptive_hash_index
ON
innodb_adaptive_max_sleep_delay
150000
innodb_additional_mem_pool_size
8388608
innodb_api_bk_commit_interval
5
innodb_api_disable_rowlock
OFF
innodb_api_enable_binlog
OFF
innodb_api_enable_mdl
OFF
innodb_api_trx_level
0
innodb_autoextend_increment
64
innodb_autoinc_lock_mode
1
innodb_buffer_pool_dump_at_shutdown
OFF
innodb_buffer_pool_dump_now
OFF
innodb_buffer_pool_filename
ib_buffer_pool
innodb_buffer_pool_instances
8
innodb_buffer_pool_load_abort
OFF
Variable_name
Value
innodb_buffer_pool_load_at_startup
OFF
innodb_buffer_pool_load_now
OFF
innodb_buffer_pool_size
17179869184
innodb_change_buffer_max_size
25
innodb_change_buffering
all
innodb_checksum_algorithm
innodb
innodb_checksums
ON
innodb_cmp_per_index_enabled
OFF
innodb_commit_concurrency
0
innodb_compression_failure_threshold_pct
5
innodb_compression_level
6
innodb_compression_pad_pct_max
50
innodb_concurrency_tickets
5000
innodb_data_file_path
ibdata1:12M:autoextend
innodb_data_home_dir
innodb_disable_sort_file_cache
OFF
innodb_doublewrite
ON
innodb_fast_shutdown
1
innodb_file_format
Antelope
innodb_file_format_check
ON
innodb_file_format_max
Antelope
innodb_file_per_table
ON
innodb_flush_log_at_timeout
1
innodb_flush_log_at_trx_commit
2
innodb_flush_method
innodb_flush_neighbors
1
innodb_flushing_avg_loops
30
innodb_force_load_corrupted
OFF
innodb_force_recovery
0
innodb_ft_aux_table
innodb_ft_cache_size
8000000
innodb_ft_enable_diag_print
OFF
innodb_ft_enable_stopword
ON
innodb_ft_max_token_size
84
innodb_ft_min_token_size
3
innodb_ft_num_word_optimize
2000
innodb_ft_result_cache_limit
2000000000
innodb_ft_server_stopword_table
innodb_ft_sort_pll_degree
2
innodb_ft_total_cache_size
640000000
innodb_ft_user_stopword_table
innodb_io_capacity
200
innodb_io_capacity_max
2000
innodb_large_prefix
OFF
innodb_lock_wait_timeout
50
innodb_locks_unsafe_for_binlog
OFF
innodb_log_buffer_size
2147483648
innodb_log_compressed_pages
ON
innodb_log_file_size
536870912
innodb_log_files_in_group
2
innodb_log_group_home_dir
./
innodb_lru_scan_depth
1024
innodb_max_dirty_pages_pct
75
innodb_max_dirty_pages_pct_lwm
0
innodb_max_purge_lag
0
innodb_max_purge_lag_delay
0
innodb_mirrored_log_groups
1
innodb_monitor_disable
innodb_monitor_enable
innodb_monitor_reset
innodb_monitor_reset_all
innodb_old_blocks_pct
37
innodb_old_blocks_time
1000
innodb_online_alter_log_max_size
134217728
innodb_open_files
12288
innodb_optimize_fulltext_only
OFF
innodb_page_size
16384
innodb_print_all_deadlocks
OFF
innodb_purge_batch_size
300
innodb_purge_threads
1
innodb_random_read_ahead
OFF
innodb_read_ahead_threshold
56
innodb_read_io_threads
4
innodb_read_only
OFF
innodb_replication_delay
0
innodb_rollback_on_timeout
OFF
innodb_rollback_segments
128
innodb_sort_buffer_size
1048576
innodb_spin_wait_delay
6
innodb_stats_auto_recalc
ON
innodb_stats_method
nulls_equal
innodb_stats_on_metadata
OFF
innodb_stats_persistent
ON
innodb_stats_persistent_sample_pages
20
innodb_stats_sample_pages
8
innodb_stats_transient_sample_pages
8
innodb_status_output
OFF
innodb_status_output_locks
OFF
innodb_strict_mode
OFF
innodb_support_xa
ON
innodb_sync_array_size
1
innodb_sync_spin_loops
30
innodb_table_locks
ON
innodb_thread_concurrency
0
innodb_thread_sleep_delay
10000
innodb_undo_directory
.
innodb_undo_logs
128
innodb_undo_tablespaces
0
innodb_use_native_aio
ON
innodb_use_sys_malloc
ON
Variable_name
Value
innodb_version
5.6.16
innodb_write_io_threads
4
interactive_timeout
28800
join_buffer_size
131072
keep_files_on_create
OFF
key_buffer_size
536870912
key_cache_age_threshold
300
key_cache_block_size
1024
key_cache_division_limit
100
large_files_support
ON
large_page_size
0
large_pages
OFF
lc_messages
en_US
lc_messages_dir
/usr/share/mysql/
lc_time_names
en_US
license
GPL
local_infile
ON
lock_wait_timeout
31536000
locked_in_memory
OFF
log_bin
OFF
log_bin_basename
log_bin_index
log_bin_trust_function_creators
OFF
log_bin_use_v1_row_events
OFF
log_error
log_output
FILE
log_queries_not_using_indexes
OFF
log_slave_updates
OFF
log_slow_admin_statements
OFF
log_slow_slave_statements
OFF
log_throttle_queries_not_using_indexes
0
log_warnings
1
long_query_time
0.200000
low_priority_updates
OFF
lower_case_file_system
OFF
lower_case_table_names
0
master_info_repository
FILE
master_verify_checksum
OFF
max_allowed_packet
67108864
max_binlog_cache_size
18446744073709547520
max_binlog_size
1073741824
max_binlog_stmt_cache_size
18446744073709547520
max_connect_errors
100
max_connections
1024
max_delayed_threads
20
max_error_count
64
max_heap_table_size
536870912
max_insert_delayed_threads
20
max_join_size
18446744073709551615
max_length_for_sort_data
1024
max_prepared_stmt_count
16382
max_relay_log_size
0
max_seeks_for_key
18446744073709551615
max_sort_length
1024
max_sp_recursion_depth
0
max_tmp_tables
32
max_user_connections
128
max_write_lock_count
18446744073709551615
metadata_locks_cache_size
1024
metadata_locks_hash_instances
8
min_examined_row_limit
0
multi_range_count
256
myisam_data_pointer_size
6
myisam_max_sort_file_size
9223372036853727232
myisam_mmap_size
18446744073709551615
myisam_recover_options
OFF
myisam_repair_threads
1
myisam_sort_buffer_size
8388608
myisam_stats_method
nulls_unequal
myisam_use_mmap
OFF
net_buffer_length
16384
net_read_timeout
30
net_retry_count
10
net_write_timeout
60
new
OFF
old
OFF
old_alter_table
OFF
old_passwords
0
open_files_limit
25610
optimizer_prune_level
1
optimizer_search_depth
62
optimizer_switch
index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,engine_condition_pushdown=on,index_condition_pushdown=on,mrr=on,mrr_cost_based=on,block_nested_loop=on,batched_key_access=off,materialization=on,semijoin=on,loosescan=on,firstmatch=on,subquery_materialization_cost_based=on,use_index_extensions=on
optimizer_trace
enabled=off,one_line=off
optimizer_trace_features
greedy_search=on,range_optimizer=on,dynamic_range=on,repeated_subselect=on
optimizer_trace_limit
1
optimizer_trace_max_mem_size
16384
optimizer_trace_offset
-1
performance_schema
ON
performance_schema_accounts_size
100
performance_schema_digests_size
10000
performance_schema_events_stages_history_long_size
10000
performance_schema_events_stages_history_size
10
performance_schema_events_statements_history_long_size
10000
performance_schema_events_statements_history_size
10
performance_schema_events_waits_history_long_size
10000
performance_schema_events_waits_history_size
10
performance_schema_hosts_size
100
performance_schema_max_cond_classes
80
performance_schema_max_cond_instances
8196
performance_schema_max_file_classes
50
Variable_name
Value
performance_schema_max_file_handles
32768
performance_schema_max_file_instances
39400
performance_schema_max_mutex_classes
200
performance_schema_max_mutex_instances
27144
performance_schema_max_rwlock_classes
40
performance_schema_max_rwlock_instances
14448
performance_schema_max_socket_classes
10
performance_schema_max_socket_instances
2068
performance_schema_max_stage_classes
150
performance_schema_max_statement_classes
168
performance_schema_max_table_handles
24576
performance_schema_max_table_instances
12500
performance_schema_max_thread_classes
50
performance_schema_max_thread_instances
2148
performance_schema_session_connect_attrs_size
512
performance_schema_setup_actors_size
100
performance_schema_setup_objects_size
100
performance_schema_users_size
100
pid_file
/var/lib/mysql/xxx.stratoserver.net.pid
plugin_dir
/usr/lib/mysql/plugin/
port
3306
preload_buffer_size
32768
profiling
OFF
profiling_history_size
15
protocol_version
10
query_alloc_block_size
8192
query_cache_limit
4194304
query_cache_min_res_unit
4096
query_cache_size
536870912
query_cache_type
OFF
query_cache_wlock_invalidate
OFF
query_prealloc_size
8192
range_alloc_block_size
4096
read_buffer_size
131072
read_only
OFF
read_rnd_buffer_size
262144
relay_log
relay_log_basename
relay_log_index
relay_log_info_file
relay-log.info
relay_log_info_repository
FILE
relay_log_purge
ON
relay_log_recovery
OFF
relay_log_space_limit
0
report_host
report_password
report_port
3306
report_user
rpl_stop_slave_timeout
31536000
secure_auth
ON
secure_file_priv
server_id
0
server_id_bits
32
server_uuid
f625f81f-3416-11e8-b131-02ab55a84b80
skip_external_locking
ON
skip_name_resolve
OFF
skip_networking
OFF
skip_show_database
OFF
slave_allow_batching
OFF
slave_checkpoint_group
512
slave_checkpoint_period
300
slave_compressed_protocol
OFF
slave_exec_mode
STRICT
slave_load_tmpdir
/tmp
slave_max_allowed_packet
1073741824
slave_net_timeout
3600
slave_parallel_workers
0
slave_pending_jobs_size_max
16777216
slave_rows_search_algorithms
TABLE_SCAN,INDEX_SCAN
slave_skip_errors
OFF
slave_sql_verify_checksum
ON
slave_transaction_retries
10
slave_type_conversions
slow_launch_time
2
slow_query_log
OFF
slow_query_log_file
/var/log/mysql/slow.log
socket
/var/run/mysqld/mysqld.sock
sort_buffer_size
2097152
sql_auto_is_null
OFF
sql_big_selects
ON
sql_buffer_result
OFF
sql_log_bin
ON
sql_log_off
OFF
sql_mode
NO_ENGINE_SUBSTITUTION
sql_notes
ON
sql_quote_show_create
ON
sql_safe_updates
OFF
sql_select_limit
18446744073709551615
sql_slave_skip_counter
0
sql_warnings
OFF
ssl_ca
ssl_capath
ssl_cert
ssl_cipher
ssl_crl
ssl_crlpath
ssl_key
storage_engine
InnoDB
stored_program_cache
256
sync_binlog
0
Variable_name
Value
sync_frm
ON
sync_master_info
10000
sync_relay_log
10000
sync_relay_log_info
10000
system_time_zone
CEST
table_definition_cache
2000
table_open_cache
12288
table_open_cache_instances
1
thread_cache_size
128
thread_concurrency
10
thread_handling
one-thread-per-connection
thread_stack
262144
time_format
%H:%i:%s
time_zone
SYSTEM
timed_mutexes
OFF
tmp_table_size
67108864
tmpdir
/tmp
transaction_alloc_block_size
8192
transaction_prealloc_size
4096
tx_isolation
REPEATABLE-READ
tx_read_only
OFF
unique_checks
ON
updatable_views_with_limit
YES
version
5.6.16-1~exp1
version_comment
(Ubuntu)
version_compile_machine
x86_64
version_compile_os
debian-linux-gnu
wait_timeout
28800
Output mysqltuner before any change:
>> MySQLTuner 1.7.12 - Major Hayden <major#mhtx.net>
>> Bug reports, feature requests, and downloads at
>> Run with '--help' for additional options and output filtering
[--] Skipped version check for MySQLTuner script
[OK] Logged in using credentials from Debian maintenance account.
[OK] Currently running supported MySQL version 5.6.16-1~exp1
[OK] Operating on 64-bit architecture
-------- Log file Recommendations ------------------------------------------------------------------
[--] Log file: /var/lib/mysql/xxx.stratoserver.net.err(0B)
[!!] Log file /var/lib/mysql/xxx.stratoserver.net.err doesn't exist
[!!] Log file /var/lib/mysql/xxx.stratoserver.net.err isn't readable.
-------- Storage Engine Statistics -----------------------------------------------------------------
[--] Status: +ARCHIVE +BLACKHOLE +CSV -FEDERATED +InnoDB +MEMORY +MRG_MYISAM +MyISAM +PERFORMANCE_SCHEMA
[--] Data in MyISAM tables: 984.2M (Tables: 30)
[--] Data in InnoDB tables: 4.1G (Tables: 568)
[--] Data in MEMORY tables: 1.3M (Tables: 17)
[!!] Total fragmented tables: 1
-------- Analysis Performance Metrics --------------------------------------------------------------
[--] innodb_stats_on_metadata: OFF
[OK] No stat updates during querying INFORMATION_SCHEMA.
-------- Security Recommendations ------------------------------------------------------------------
[OK] There are no anonymous accounts for any database users
[OK] All database users have passwords assigned
[!!] There is no basic password file list!
-------- CVE Security Recommendations --------------------------------------------------------------
[--] Skipped due to --cvefile option undefined
-------- Performance Metrics -----------------------------------------------------------------------
[--] Up for: 10d 23h 42m 17s (581M q [612.461 qps], 1M conn, TX: 889G, RX: 249G)
[--] Reads / Writes: 93% / 7%
[--] Binary logging is disabled
[--] Physical Memory : 32.0G
[--] Max MySQL memory : 22.6G
[--] Other process memory: 33.5M
[--] Total buffers: 19.1G global + 2.8M per thread (1024 max threads)
[--] P_S Max memory usage: 754M
[--] Galera GCache Max memory usage: 0B
[OK] Maximum reached memory usage: 20.1G (62.80% of installed RAM)
[OK] Maximum possible memory usage: 22.6G (70.49% of installed RAM)
[OK] Overall possible memory usage with other process is compatible with memory available
[OK] Slow queries: 0% (365K/581M)
[OK] Highest usage of available connections: 10% (108/1024)
[OK] Aborted connections: 0.02% (338/1784303)
[!!] name resolution is active : a reverse name resolution is made for each new connection and can reduce performance
[!!] Query cache may be disabled by default due to mutex contention.
[!!] Query cache efficiency: 0.0% (0 cached / 459M selects)
[OK] Query cache prunes per day: 0
[OK] Sorts requiring temporary tables: 0% (1K temp sorts / 92M sorts)
[!!] Joins performed without indexes: 341633
[!!] Temporary tables created on disk: 47% (32M on disk / 69M total)
[OK] Thread cache hit rate: 99% (108 created / 1M connections)
[!!] Table cache hit rate: 2% (2K open / 98K opened)
[OK] Open file limit used: 0% (132/25K)
[OK] Table locks acquired immediately: 99% (1B immediate / 1B locks)
-------- Performance schema ------------------------------------------------------------------------
[--] Memory used by P_S: 754.8M
[--] Sys schema isn't installed.
-------- ThreadPool Metrics ------------------------------------------------------------------------
[--] ThreadPool stat is disabled.
-------- MyISAM Metrics ----------------------------------------------------------------------------
[!!] Key buffer used: 52.9% (284M used / 536M cache)
[OK] Key buffer size / total MyISAM indexes: 512.0M/378.4M
[OK] Read Key buffer hit rate: 100.0% (484M cached / 171K reads)
[!!] Write Key buffer hit rate: 76.0% (42M cached / 32M writes)
-------- InnoDB Metrics ----------------------------------------------------------------------------
[--] InnoDB is enabled.
[--] InnoDB Thread Concurrency: 0
[OK] InnoDB File per table is activated
[OK] InnoDB buffer pool / data size: 16.0G/4.1G
[!!] Ratio InnoDB log file size / InnoDB Buffer pool size (6.25 %): 512.0M * 2/16.0G should be equal 25%
[!!] InnoDB buffer pool instances: 8
[--] InnoDB Buffer Pool Chunk Size not used or defined in your version
[OK] InnoDB Read buffer efficiency: 100.00% (238662528177 hits/ 238662686106 total)
[!!] InnoDB Write Log efficiency: 88.34% (208711294 hits/ 236256359 total)
[OK] InnoDB log waits: 0.00% (0 waits / 27545065 writes)
-------- AriaDB Metrics ----------------------------------------------------------------------------
[--] AriaDB is disabled.
-------- TokuDB Metrics ----------------------------------------------------------------------------
[--] TokuDB is disabled.
-------- XtraDB Metrics ----------------------------------------------------------------------------
[--] XtraDB is disabled.
-------- Galera Metrics ----------------------------------------------------------------------------
[--] Galera is disabled.
-------- Replication Metrics -----------------------------------------------------------------------
[--] Galera Synchronous replication: NO
[--] No replication slave(s) for this server.
[--] Binlog format: STATEMENT
[--] XA support enabled: ON
[--] Semi synchronous replication Master: Not Activated
[--] Semi synchronous replication Slave: Not Activated
[--] This is a standalone server
-------- Recommendations ---------------------------------------------------------------------------
General recommendations:
Run OPTIMIZE TABLE to defragment tables for better performance
OPTIMIZE TABLE `juwelierwebshop`.`cron_schedule`; -- can free 1064 MB
Total freed space after theses OPTIMIZE TABLE : 1064 Mb
Configure your accounts with ip or subnets only, then update your configuration with skip-name-resolve=1
Adjust your join queries to always utilize indexes
When making adjustments, make tmp_table_size/max_heap_table_size equal
Reduce your SELECT DISTINCT queries which have no LIMIT clause
Increase table_open_cache gradually to avoid file descriptor limits
Read this before increasing table_open_cache over 64:
This is MyISAM only table_cache scalability problem, InnoDB not affected.
See more details here:
This bug already fixed in MySQL 5.7.9 and newer MySQL versions.
Beware that open_files_limit (25610) variable
should be greater than table_open_cache (12288)
Consider installing Sys schema from
Before changing innodb_log_file_size and/or innodb_log_files_in_group read this:
Variables to adjust:
query_cache_size (=0)
query_cache_type (=0)
query_cache_limit (> 4M, or use smaller result sets)
join_buffer_size (> 128.0K, or always use indexes with JOINs)
tmp_table_size (> 64M)
max_heap_table_size (> 512M)
table_open_cache (> 12288)
innodb_log_file_size should be (=2G) if possible, so InnoDB total log files size equals to 25% of buffer pool size.
innodb_buffer_pool_instances(=16)
Do not blindly raise values in hopes they help. The following are dangerously high and are likely to lead to swapping, which is much worse than having lower settings:
key_buffer = 512M
thread_cache_size = 512
join_buffer_size = 128M
max_connections = 1024
table_open_cache = 12288
tmp_table_size = 512M
max_heap_table_size = 512M
query_cache_size = 512M
innodb_log_buffer_size = 2G
Consider moving the rest of your tables from MyISAM to InnoDB.
You seem to have a lot of poorly formulated queries and/or missing indexes. The Slowlog is configured, but turned off, so you cannot readily get further info. Turn it back on.
Are these indexed? Are they numbers or strings?
main_table.product_id
main_table.entity_id
catalog_category_entity.entity_id
sales_flat_quote_address.address_id
This is vicious to optimize; can it be rewritten some other way?..
WHERE (cat_index.visibility IN(3, 2, 4)
OR store_cat_index.visibility IN(3, 2, 4))
For example, is the "visibility" always the same in both those tables?
Can you somehow do the COUNT before JOINing? cf SELECT COUNT(DISTINCT e.entity_id) ...
Ouch! EAV made worse. customer_address_entity_varchar/_int/_text
WHERE (t_d.entity_type_id = 3)
AND (t_d.entity_id IN (260))
AND (t_d.attribute_id IN ('41'))
AND (t_d.store_id = 0)
begs for
`INDEX(entity_type_id, store_id, -- first, in either order
entity_id, attribute_id)
Fix those; run for a while; refetch VARIABLES and GLOBAL STATUS but unwrapped, then I'll look at the rest of them.
Just in general, when catalog is larger then 50k
Enable Flat catalog
Set indexer modes to manual and setup a nightly
cron for them.
I am facing an issue with high mysql load in particularly whole of the CPU gets consumed. We have 16core cpu's. A job runs which imports certain data in csv format to database. When this starts the CPU goes high and mysql gets hung. Below
INNODB ENGINE STATUS
mysql> show engine innodb status\G;
*************************** 1. row ***************************
Type: InnoDB
Name:
Status:
=====================================
180614 21:27:28 INNODB MONITOR OUTPUT
=====================================
Per second averages calculated from the last 27 seconds
-----------------
BACKGROUND THREAD
-----------------
srv_master_thread loops: 1113759 1_second, 1113758 sleeps, 111273 10_second, 1177 background, 1177 flush
srv_master_thread log flush and writes: 1120485
----------
SpgaPHORES
----------
OS WAIT ARRAY INFO: reservation count 6324138, signal count 163497140
Mutex spin waits 3563712357, rounds 2684759736, OS waits 3547098
RW-shared spins 28930883, rounds 72633117, OS waits 977277
RW-excl spins 8651762, rounds 274697595, OS waits 1300580
Spin rounds per wait: 0.75 mutex, 2.51 RW-shared, 31.75 RW-excl
------------------------
LATEST DETECTED DEADLOCK
------------------------
180609 5:00:01
*** (1) TRANSACTION:
TRANSACTION 192A64A2D3, ACTIVE 1 sec starting index read
UPDATE products SET hjmpTS = 80060 ,modifiedTS='2018-06-09 05:00:01' WHERE GK = 8816144056321
*** (2) HOLDS THE LOCK(S):
RECORD LOCKS space id 971922 page no 31354 n bits 136 index `PRIMARY` of table `pgagmcb_prd`.`products` trx id 192A64ACB2 lock_mode X locks rec but not gap
*** WE ROLL BACK TRANSACTION (1)
------------
TRANSACTIONS
------------
Trx id counter 19C4CE52B2
Purge done for trx's n:o < 19C4CE1594 undo n:o < 0
History list length 821
LIST OF TRANSACTIONS FOR EACH SESSION:
---TRANSACTION 0, not started
MySQL thread id 392730, OS thread handle 0x7f09af186700, query id 5877276140 localhost root
show engine innodb status
---TRANSACTION 19C4CE4A6D, not started
MySQL thread id 392695, OS thread handle 0x7f09c5bef700, query id 5877273919 prgkptomcat02 118.21.1.11 pgag_oauth_prd
---TRANSACTION 19C4CA0181, not started
MySQL thread id 392677, OS thread handle 0x7f09c8a5d700, query id 5876980085 prgkptomcat04 118.21.1.13 pgag_oauth_prd
---TRANSACTION 19C4CE52B1, not started
MySQL thread id 392667, OS thread handle 0x7f09aebf0700, query id 5877276137 prgkptomcat04 118.21.1.13 pgagmcb_prd
---TRANSACTION 19C4CDC68D, not started
MySQL thread id 392642, OS thread handle 0x7f11a4161700, query id 5877238586 118.21.1.14 pgagmcb_prd
---TRANSACTION 19C4CDBF35, not started
MySQL thread id 392641, OS thread handle 0x7f09af0c3700, query id 5877236562 118.21.1.14 pgagmcb_prd
---TRANSACTION 19C4C412C3, not started
MySQL thread id 392586, OS thread handle 0x7f09c5a28700, query id 5876568407 prgkptomcat02 118.21.1.11 pgagmcb_prd
---TRANSACTION 19C4C70266, not started
MySQL thread id 392584, OS thread handle 0x7f09ae966700, query id 5876773008 prgkptomcat02 118.21.1.11 pgagmcb_prd
---TRANSACTION 19C4CAFCC7, not started
MySQL thread id 392525, OS thread handle 0x7f09c43d0700, query id 5877046769 prgkptomcat02 118.21.1.11 pgag_oauth_prd
---TRANSACTION 19C4CAFD1F, not started
MySQL thread id 392523, OS thread handle 0x7f09c8daa700, query id 5877046862 prgkptomcat02 118.21.1.11 pgag_oauth_prd
---TRANSACTION 19C4CAFD07, not started
MySQL thread id 392524, OS thread handle 0x7f09c5820700, query id 5877046837 prgkptomcat02 118.21.1.11 pgag_oauth_prd
---TRANSACTION 19C4CDE3C3, not started
MySQL thread id 392490, OS thread handle 0x7f09c5d75700, query id 5877246447 118.21.1.14 pgag_oauth_prd
---TRANSACTION 19C4CDCB36, not started
MySQL thread id 392485, OS thread handle 0x7f09aefbf700, query id 5877239893 prgkptomcat03 118.21.1.12 pgagmcb_prd
---TRANSACTION 19C4CE52A3, not started
MySQL thread id 392451, OS thread handle 0x7f09af71c700, query id 5877276127 prgkptomcat04 118.21.1.13 pgagmcb_prd
---TRANSACTION 19C4C92CF9, not started
MySQL thread id 392449, OS thread handle 0x7f09c5f7d700, query id 5876922937 prgkptomcat04 118.21.1.13 pgagmcb_prd
---TRANSACTION 19C4CE52A6, not started
MySQL thread id 392442, OS thread handle 0x7f09af38e700, query id 5877276129 prgkptomcat04 118.21.1.13 pgagmcb_prd
---TRANSACTION 19C4CE4F03, not started
MySQL thread id 392421, OS thread handle 0x7f09af30c700, query id 5877275162 prgkptomcat02 118.21.1.11 pgagmcb_prd
---TRANSACTION 19C4C9B259, not started
MySQL thread id 392415, OS thread handle 0x7f09c4597700, query id 5876958837 prgkptomcat02 118.21.1.11 pgagmcb_prd
---TRANSACTION 19C4CAE196, not started
MySQL thread id 392402, OS thread handle 0x7f09c8c65700, query id 5877039508 prgkptomcat01 118.21.1.10 pgagmcb_prd
---TRANSACTION 19C4CE52A1, not started
MySQL thread id 392383, OS thread handle 0x7f09aeaab700, query id 5877276126 prgkptomcat04 118.21.1.13 pgagmcb_prd
---TRANSACTION 19C4CE52AA, not started
MySQL thread id 392382, OS thread handle 0x7f09c4c72700, query id 5877276132 prgkptomcat04 118.21.1.13 pgagmcb_prd
---TRANSACTION 19C4CDBDE9, not started
MySQL thread id 392352, OS thread handle 0x7f09c5082700, query id 5877236203 118.21.1.14 pgagmcb_prd
---TRANSACTION 19C4CE5279, not started
MySQL thread id 392350, OS thread handle 0x7f09c4db7700, query id 5877276084 118.21.1.14 pgagmcb_prd
---TRANSACTION 19C4CDEAEF, not started
MySQL thread id 392323, OS thread handle 0x7f09c48e4700, query id 5877248380 118.21.1.14 pgagmcb_prd
---TRANSACTION 19C4CE52A5, not started
MySQL thread id 392322, OS thread handle 0x7f09c56db700, query id 5877276112 118.21.1.14 pgagmcb_prd
---TRANSACTION 19C4CE4BD9, not started
MySQL thread id 392319, OS thread handle 0x7f09c6523700, query id 5877274310 prgkptomcat02 118.21.1.11 pgagmcb_prd
---TRANSACTION 19C4CC77FB, not started
MySQL thread id 392297, OS thread handle 0x7f09c538e700, query id 5877148868 prgkptomcat02 118.21.1.11 pgagmcb_prd
---TRANSACTION 19C4C70074, not started
MySQL thread id 392296, OS thread handle 0x7f09c8a9e700, query id 5876772482 prgkptomcat02 118.21.1.11 pgagmcb_prd
---TRANSACTION 19C4CD042D, not started
MySQL thread id 392295, OS thread handle 0x7f09ae5d8700, query id 5877186230 prgkptomcat02 118.21.1.11 pgagmcb_prd
---TRANSACTION 19C4CE2C0A, not started
MySQL thread id 392293, OS thread handle 0x7f09c5aeb700, query id 5877265779 118.21.1.14 pgagmcb_prd
---TRANSACTION 19C4CE3571, not started
MySQL thread id 392283, OS thread handle 0x7f09c6627700, query id 5877268313 prgkptomcat04 118.21.1.13 pgagmcb_prd
---TRANSACTION 19C4CC96A6, not started
MySQL thread id 392282, OS thread handle 0x7f09c40c4700, query id 5877156935 prgkptomcat04 118.21.1.13 pgagmcb_prd
---TRANSACTION 19C4C930C7, not started
MySQL thread id 392281, OS thread handle 0x7f09c8e2c700, query id 5876923943 prgkptomcat04 118.21.1.13 pgagmcb_prd
---TRANSACTION 19C4C6F0A5, not started
MySQL thread id 392278, OS thread handle 0x7f09c569a700, query id 5876768228 prgkptomcat02 118.21.1.11 pgagmcb_prd
---TRANSACTION 19C4CC399F, not started
MySQL thread id 392276, OS thread handle 0x7f09c471d700, query id 5877132036 prgkptomcat02 118.21.1.11 pgagmcb_prd
---TRANSACTION 19C4CE52A0, not started
MySQL thread id 392234, OS thread handle 0x7f09c5249700, query id 5877276138 118.21.1.14 pgagmcb_prd Opening tables
SELECT item_t0.GK FROM cartentries item_t0 WHERE ( item_t0.p_order =8995720658987) AND (item_t0.TypeGKString=8796094496850 ) order by item_t0.EntryNumber ASC, item_t0.createdTS ASC
---TRANSACTION 19C4CE369D, not started
MySQL thread id 392232, OS thread handle 0x7f09c8be3700, query id 5877268619 prgkptomcat04 118.21.1.13 pgagmcb_prd
---TRANSACTION 19C4CAE1B0, not started
MySQL thread id 392136, OS thread handle 0x7f09c4d76700, query id 5877039534 prgkptomcat01 118.21.1.10 pgagmcb_prd
---TRANSACTION 19C4CAE1AA, not started
MySQL thread id 392135, OS thread handle 0x7f09c4fbf700, query id 5877039527 prgkptomcat01 118.21.1.10 pgagmcb_prd
---TRANSACTION 19C4CE5272, not started
MySQL thread id 392114, OS thread handle 0x7f09c50c3700, query id 5877276077 prgkptomcat01 118.21.1.10 pgagmcb_prd
---TRANSACTION 19C4CE32D8, not started
MySQL thread id 392027, OS thread handle 0x7f11a409e700, query id 5877267604 prgkptomcat03 118.21.1.12 pgagmcb_prd
---TRANSACTION 19C4CDBDEE, not started
MySQL thread id 391944, OS thread handle 0x7f11a4224700, query id 5877236208 118.21.1.14 pgagmcb_prd
---TRANSACTION 19C4CA0332, not started
MySQL thread id 391874, OS thread handle 0x7f09c5aaa700, query id 5876980541 prgkptomcat04 118.21.1.13 pgag_oauth_prd
---TRANSACTION 19C4CE50B6, not started
MySQL thread id 391814, OS thread handle 0x7f09c5eba700, query id 5877275617 prgkptomcat02 118.21.1.11 pgagmcb_prd
---TRANSACTION 19C4CE1588, not started
MySQL thread id 391776, OS thread handle 0x7f09c5d34700, query id 5877259853 118.21.1.14 pgagmcb_prd
---TRANSACTION 19C4CE52AF, not started
mysql tables in use 1, locked 1
MySQL thread id 391692, OS thread handle 0x7f11a40df700, query id 5877276141 118.21.1.14 pgagmcb_prd init
UPDATE cronjobs SET hjmpTS = 8629763 ,modifiedTS='2018-06-14 21:27:28',p_endtime='2018-06-14 21:27:28' WHERE GK = 8811756159477
Trx read view will not see trx with id >= 19C4CE527F, sees < 19C4CE4C4D
---TRANSACTION 19C4CE524E, ACTIVE 0 sec fetching rows
mysql tables in use 2, locked 0
MySQL thread id 392064, OS thread handle 0x7f09afb2c700, query id 5877276038 prgkptomcat01 118.21.1.10 pgagmcb_prd Sending data
MySQL thread id 392062, OS thread handle 0x7f09c55d7700, query id 5877275109 prgkptomcat01 118.21.1.10 pgagmcb_prd Sending data
SELECT item_t0.GK FROM addresses item_t0 WHERE ( item_t0.p_pkid ='29001499' AND item_t0.p_billingaddress =0 AND item_t0.p_shippingaddress =1) AND (item_t0.TypeGKString=8796094103634 ) UNION ALL SELECT item_t0.GK FROM pointofserviceaddress item_t0 WHERE ( item_t0.p_pkid ='29001499' AND item_t0.p_billingaddress =0 AND item_t0.p_shippingaddress =1) AND (item_t0.TypeGKString=8797338533970 )
Trx read view will not see trx with id >= 19C4CE4ECF, sees < 19C4CE4A1C
---TRANSACTION 19C4CE4EC4, ACTIVE 0 sec
mysql tables in use 2, locked 0
MySQL thread id 392067, OS thread handle 0x7f09affff700, query id 5877275099 prgkptomcat01 118.21.1.10 pgagmcb_prd Sending data
h id >= 19C4CE4C62, sees < 19C4CE48E5
---TRANSACTION 19C4CE4C4D, ACTIVE 0 sec fetching rows
mysql tables in use 2, locked 0
MySQL thread id 392059, OS thread handle 0x7f09c430d700, query id 5877274440 prgkptomcat01 118.21.1.10 emagmcb_prd Sending data
SELECT item_t0.GK FROM addresses item_t0 WHERE ( item_t0.p_pkid ='20087352' AND item_t0.p_billingaddress =0 AND item_t0.p_shippingaddress =1) AND (item_t0.TypeGKString=8796094103634 ) UNION ALL SELECT item_t0.GK FROM pointofserviceaddress item_t0 WHERE ( item_t0.p_pkid ='20087352' AND item_t0.p_billingaddress =0 AND item_t0.p_shippingaddress =1) AND (item_t0.TypeGKString=8797338533970 )
Trx read view will not see trx with id >= 19C4CE4C4E, sees < 19C4CE48E5
--------
FILE I/O
--------
I/O thread 0 state: waiting for i/o request (insert buffer thread)
I/O thread 1 state: waiting for i/o request (log thread)
I/O thread 2 state: waiting for i/o request (read thread)
I/O thread 3 state: waiting for i/o request (read thread)
I/O thread 4 state: waiting for i/o request (read thread)
I/O thread 5 state: waiting for i/o request (read thread)
I/O thread 6 state: waiting for i/o request (write thread)
I/O thread 7 state: waiting for i/o request (write thread)
I/O thread 8 state: waiting for i/o request (write thread)
I/O thread 9 state: waiting for i/o request (write thread)
Pending normal aio reads: 0 [0, 0, 0, 0] , aio writes: 0 [0, 0, 0, 0] ,
ibuf aio reads: 0, log i/o's: 0, sync i/o's: 0
Pending flushes (fsync) log: 0; buffer pool: 0
689458 OS file reads, 21884164 OS file writes, 5163807 OS fsyncs
0.00 reads/s, 0 avg bytes/read, 22.07 writes/s, 5.22 fsyncs/s
-------------------------------------
INSERT BUFFER AND ADAPTIVE HASH INDEX
-------------------------------------
Ibuf: size 1, free list len 156723, seg size 156725, 39554 merges
merged operations:
insert 106864, delete mark 147895, delete 47175
discarded operations:
insert 0, delete mark 0, delete 0
Hash table size 63749393, node heap has 66107 buffer(s)
11918.00 hash searches/s, 20228.44 non-hash searches/s
---
LOG
---
Log sequence number 3872974342573
Log flushed up to 3872974307599
Last checkpoint at 3872972346157
0 pending log writes, 0 pending chkp writes
1520749 log i/o's done, 1.33 log i/o's/second
----------------------
BUFFER POOL AND MEMORY
----------------------
Total memory allocated 32967229440; in additional pool allocated 0
Dictionary memory allocated 5343676
Buffer pool size 1966080
Free buffers 901470
Database pages 998503
Old database pages 368568
Modified db pages 1090
Pending reads 0
Pending writes: LRU 0, flush list 0, single page 0
Pages made young 4341, not young 0
0.00 youngs/s, 0.00 non-youngs/s
Pages read 889753, created 108750, written 32815581
0.00 reads/s, 0.15 creates/s, 30.44 writes/s
Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000
Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
LRU len: 998503, unzip_LRU len: 0
I/O sum[0]:cur[0], unzip sum[0]:cur[0]
--------------
ROW OPERATIONS
--------------
0 queries inside InnoDB, 0 queries in queue
22 read views open inside InnoDB
Main thread process no. 11805, id 139679969232640, state: sleeping
Number of rows inserted 4605702, updated 110022489, deleted 3897802, read 309526373829
0.81 inserts/s, 49.85 updates/s, 0.56 deletes/s, 1280085.96 reads/s
----------------------------
END OF INNODB MONITOR OUTPUT
============================
[client]
port = 3306
socket = /AB/mysql/logs/mysql.sock
[mysqld]
port = 3306
bind-address = 0.0.0.0
socket = /AB/mysql/logs/mysql.sock
basedir = /AB/mysql
user = mysql
tmpdir = /AB/mysql_temp_files
character-set-server = utf8
datadir = /AB/mysql/data
default-storage-engine = InnoDB
expire_logs_days = 10
general_log = 0
general_log_file = /AB/mysql/logs/general-log.log
innodb_buffer_pool_size = 30G
innodb_file_per_table = 1
innodb_flush_log_at_trx_commit = 0
key_buffer_size = 16M
log_error = /AB/mysql/logs/mysqld-error.log
log_queries_not_using_indexes = 1
long_query_time = 5
log-bin = /AB/mysql/binlog/mysql-bin
max_allowed_packet = 35M
max_binlog_size = 100M
max_connect_errors = 100000
max_connections = 910
myisam_recover = BACKUP
query_cache_limit = 2M
query_cache_size = 0
query_cache_type = 0
skip-external-locking
slow_query_log = 0
slow_query_log_file = /AB/mysql/logs/mysqld-slow-queries.log
table_open_cache = 256
thread_cache_size = 8
thread_stack = 256K
transaction-isolation = READ-COMMITTED
############################
# MySql replication - Master
############################
server_id = 1
binlog_format = row
binlog-ignore-db=mysql # input the database that should be ignored for replication
binlog-ignore-db=performance_schema
binlog-ignore-db=pkgmcb_eldas
replicate-wild-ignore-table=mysql.% #disable slave replication for mysql db
relay-log = /AB/mysql/mysql-relay-bin
relay-log-index = /AB/mysql/mysql-relay-bin.index
master-info-file = /AB/mysql/mysql-master.info
relay-log-info-file = /AB/mysql/mysql-relay-log-info
[mysqld_safe]
log-error = /AB/mysql/logs/mysqld.log
socket = /AB/mysql/logs/mysql.sock
pid-file = /AB/mysql/logs/mysqld.pid
[mysqldump]
max_allowed_packet = 16M
quick
[mysql]
no_auto_rehash
mysql> show global status;
+------------------------------------------+---------------+
| Variable_name | Value |
+------------------------------------------+---------------+
| Aborted_clients | 13 |
| Aborted_connects | 3753 |
| Binlog_cache_disk_use | 32815 |
| Binlog_cache_use | 92017182 |
| Bytes_received | 1617086123131 |
| Bytes_sent | 2825568326969 |
| Com_admin_commands | 37462 |
| Com_change_db | 2 |
| Com_commit | 128435552 |
| Com_create_table | 18730 |
| Com_delete | 15234374 |
| Com_delete_multi | 37450 |
| Com_drop_table | 18725 |
| Com_insert | 4603989 |
| Com_insert_select | 18725 |
| Com_purge_before_date | 13 |
| Com_rollback | 15159 |
| Com_select | 5360296742 |
| Com_set_option | 257428949 |
| Com_show_charsets | 4 |
| Com_show_collations | 6371 |
| Com_show_engine_status | 4870 |
| Com_show_processlist | 1187 |
| Com_show_status | 95630 |
| Com_show_tables | 21 |
| Com_show_triggers | 0 |
| Com_show_variables | 21400 |
| Com_update | 110599528 |
| Compression | OFF |
| Connections | 392731 |
| Created_tmp_disk_tables | 77030 |
| Created_tmp_files | 445171 |
| Created_tmp_tables | 665507495 |
| Delayed_errors | 0 |
| Delayed_insert_threads | 0 |
| Delayed_writes | 0 |
| Flush_commands | 1 |
| Handler_commit | 5774785433 |
| Handler_delete | 3897802 |
| Handler_discover | 0 |
| Handler_prepare | 402020246 |
| Handler_read_first | 161347921 |
| Handler_read_key | 32213287627 |
| Handler_read_last | 2 |
| Handler_read_next | 47210732684 |
| Handler_read_prev | 32726 |
| Handler_read_rnd | 215354676 |
| Handler_read_rnd_next | 482467384071 |
| Handler_rollback | 170 |
| Handler_savepoint | 0 |
| Handler_savepoint_rollback | 0 |
| Handler_update | 221011669 |
| Handler_write | 21797614285 |
| Innodb_buffer_pool_pages_data | 998503 |
| Innodb_buffer_pool_bytes_data | 16359473152 |
| Innodb_buffer_pool_pages_dirty | 1029 |
| Innodb_buffer_pool_bytes_dirty | 16859136 |
| Innodb_buffer_pool_pages_flushed | 32815553 |
| Innodb_buffer_pool_pages_free | 901470 |
| Innodb_buffer_pool_pages_misc | 66107 |
| Innodb_buffer_pool_pages_total | 1966080 |
| Innodb_buffer_pool_read_ahead_rnd | 0 |
| Innodb_buffer_pool_read_ahead | 207758 |
| Innodb_buffer_pool_read_ahead_evicted | 0 |
| Innodb_buffer_pool_read_requests | 390826699904 |
| Innodb_buffer_pool_reads | 645677 |
| Innodb_buffer_pool_wait_free | 0 |
| Innodb_buffer_pool_write_requests | 887071687 |
| Innodb_data_fsyncs | 5163787 |
| Innodb_data_pending_fsyncs | 0 |
| Innodb_data_pending_reads | 0 |
| Innodb_data_pending_writes | 0 |
| Innodb_data_read | 14588825600 |
| Innodb_data_reads | 689458 |
| Innodb_data_writes | 21884132 |
| Innodb_data_written | 1155002425856 |
| Innodb_dblwr_pages_written | 32815553 |
| Innodb_dblwr_writes | 918315 |
| Innodb_have_atomic_builtins | ON |
| Innodb_log_waits | 0 |
| Innodb_log_write_requests | 234974909 |
| Innodb_log_writes | 1390975 |
| Innodb_os_log_fsyncs | 1505727 |
| Innodb_os_log_pending_fsyncs | 0 |
| Innodb_os_log_pending_writes | 0 |
| Innodb_os_log_written | 79636049408 |
| Innodb_page_size | 16384 |
| Innodb_pages_created | 108750 |
| Innodb_pages_read | 889753 |
| Innodb_pages_written | 32815553 |
| Innodb_row_lock_current_waits | 0 |
| Innodb_row_lock_time | 195660 |
| Innodb_row_lock_time_avg | 0 |
| Innodb_row_lock_time_max | 2243 |
| Innodb_row_lock_waits | 1286068 |
| Innodb_rows_deleted | 3897802 |
| Innodb_rows_inserted | 4605700 |
| Innodb_rows_read | 309520074518 |
| Innodb_rows_updated | 110022225 |
| Innodb_truncated_status_writes | 0 |
| Key_blocks_not_flushed | 0 |
| Key_blocks_unused | 13396 |
| Key_blocks_used | 5921 |
| Key_read_requests | 1782635934 |
| Key_reads | 0 |
| Key_write_requests | 292962271 |
| Key_writes | 0 |
| Last_query_cost | 0.000000 |
| Max_used_connections | 266 |
| Not_flushed_delayed_rows | 0 |
| Open_files | 5 |
| Open_streams | 0 |
| Open_table_definitions | 400 |
| Open_tables | 256 |
| Opened_files | 959559 |
| Opened_table_definitions | 43623 |
| Opened_tables | 21935444 |
| Performance_schema_cond_classes_lost | 0 |
| Performance_schema_cond_instances_lost | 0 |
| Performance_schema_file_classes_lost | 0 |
| Performance_schema_file_handles_lost | 0 |
| Performance_schema_file_instances_lost | 0 |
| Performance_schema_locker_lost | 0 |
| Performance_schema_mutex_classes_lost | 0 |
| Performance_schema_mutex_instances_lost | 0 |
| Performance_schema_rwlock_classes_lost | 0 |
| Performance_schema_rwlock_instances_lost | 0 |
| Performance_schema_table_handles_lost | 0 |
| Performance_schema_table_instances_lost | 0 |
| Performance_schema_thread_classes_lost | 0 |
| Performance_schema_thread_instances_lost | 0 |
| Prepared_stmt_count | 0 |
| Qcache_free_blocks | 0 |
| Qcache_free_memory | 0 |
| Qcache_hits | 0 |
| Qcache_inserts | 0 |
| Qcache_lowmem_prunes | 0 |
| Qcache_not_cached | 0 |
| Qcache_queries_in_cache | 0 |
| Qcache_total_blocks | 0 |
| Queries | 5877263773 |
| Questions | 5877218009 |
| Rpl_status | AUTH_MASTER |
| Select_full_join | 1085194 |
| Select_full_range_join | 0 |
| Select_range | 106321329 |
| Select_range_check | 0 |
| Select_scan | 491596929 |
| Slave_heartbeat_period | 0.000 |
| Slave_open_temp_tables | 0 |
| Slave_received_heartbeats | 0 |
| Slave_retried_transactions | 0 |
| Slave_running | OFF |
| Slow_launch_threads | 0 |
| Slow_queries | 407546261 |
| Sort_merge_passes | 631000 |
| Sort_range | 4293576025 |
| Sort_rows | 17877617809 |
| Sort_scan | 13049200 |
| Table_locks_immediate | 7035027122 |
| Table_locks_waited | 0 |
| Tc_log_max_pages_used | 0 |
| Tc_log_page_size | 0 |
| Tc_log_page_waits | 7 |
| Threads_cached | 5 |
| Threads_connected | 102 |
| Threads_created | 7052 |
| Threads_running | 21 |
| Uptime | 1123494 |
| Uptime_since_flush_status | 1123494 |
+------------------------------------------+---------------+
289 rows in set (0.00 sec)
Any help is really appreciated. Thanks
Analysis of Global Status and Variables:
Observations:
Version: 5.5.47-log
30 GB of RAM
Uptime = 13d 00:04:54
You are not running on Windows.
Running 64-bit version
You appear to be running entirely (or mostly) InnoDB.
The More Important Issues:
What did you mean by "30G assigned for mysql"? I see that the buffer_pool is set to that. That is reasonable if you have more than 30G of RAM available for all of MySQL. If the 40GB machine is mostly dedicated to MySQL, then 30G for buffer_pool is good.
Time to upgrade to 5.6. The table_cache is being hit rather hard, but table_open_cache_hits and _misses are not available until 5.6. Meanwhile, see if this helps:
table_open_cache = 500
InnoDB's log is churning like mad. Recommend the following. Caution -- it is complex to change:
innodb_log_file_size = 128M
What are you doing to cause so many rows to be deleted? Perhaps you are "reloading" a table? There are faster ways than using DELETE. Tell us what is up so we can tailor a solution.
There are lots of clues of frequent and/or slow queries. Please turn on the slowlog and have the following
log_queries_not_using_indexes = OFF -- ON is just clutter
Don't decrease long_query_time until you have fixed some of the 362 slow queries/sec.
After a day, let's summarize and analyze what is in the slowlog.
Creating and dropping table once a minute? What's up?
tx_isolation = READ-COMMITTED -- What's up?
These will help some:
innodb_buffer_pool_instances = 16
thread_cache_size = 20
join_buffer_size = 256K
Details and other observations:
( Key_blocks_used * 1024 / key_buffer_size ) = 5,921 * 1024 / 16M = 36.1% -- Percent of key_buffer used. High-water-mark.
-- Lower key_buffer_size to avoid unnecessary memory usage.
( innodb_buffer_pool_size / _ram ) = 30720M / 30720M = 100.0% -- % of RAM used for InnoDB buffer_pool
( (key_buffer_size / 0.20 + innodb_buffer_pool_size / 0.70) / _ram ) = (16M / 0.20 + 30720M / 0.70) / 30720M = 143.1% -- Most of available ram should be made available for caching.
-- http://mysql.rjweb.org/doc.php/memory
( Opened_tables ) = 21,935,444 / 1123494 = 20 /sec -- Frequency of opening Tables
-- increase table_open_cache
( innodb_buffer_pool_size / innodb_buffer_pool_instances ) = 30720M / 1 = 30720MB -- Size of each buffer_pool instance.
-- An instance should be at least 1GB. In very large RAM, have 16 instances.
( Innodb_buffer_pool_pages_free / Innodb_buffer_pool_pages_total ) = 901,470 / 1966080 = 45.9% -- Pct of buffer_pool currently not in use
-- innodb_buffer_pool_size is bigger than necessary?
( Innodb_os_log_written / (Uptime / 3600) / innodb_log_files_in_group / innodb_log_file_size ) = 79,636,049,408 / (1123494 / 3600) / 2 / 5M = 24.3 -- Ratio
-- (see minutes)
( Uptime / 60 * innodb_log_file_size / Innodb_os_log_written ) = 1,123,494 / 60 * 5M / 79636049408 = 1.23 -- Minutes between InnoDB log rotations Beginning with 5.6.8, this can be changed dynamically; be sure to also change my.cnf.
-- (The recommendation of 60 minutes between rotations is somewhat arbitrary.) Adjust innodb_log_file_size. (Cannot change in AWS.)
( Innodb_rows_deleted / Innodb_rows_inserted ) = 3,897,802 / 4605700 = 0.846 -- Churn
-- "Don't queue it, just do it." (If MySQL is being used as a queue.)
( Innodb_row_lock_waits ) = 1,286,068 / 1123494 = 1.1 /sec -- How often there is a delay in getting a row lock.
-- May be caused by complex queries that could be optimized.
( innodb_stats_on_metadata ) = ON -- Re-analyze table when touching stats.
-- ON is likely to slow down certain SHOWs and information_schema accesses.
( innodb_thread_concurrency ) = 0 -- 0 = Let InnoDB decide the best for concurrency_tickets.
-- Set to 0 or 64. This may cut back on CPU.
( innodb_print_all_deadlocks ) = OFF -- Whether to log all Deadlocks.
-- If you are plagued with Deadlocks, turn this on. Caution: If you have lots of deadlocks, this may write a lot to disk.
( join_buffer_size / _ram ) = 131,072 / 30720M = 0.00% -- 0-N per thread. May speed up JOINs (better to fix queries/indexes) (all engines) Used for index scan, range index scan, full table scan, each full JOIN, etc.
-- If large, decrease join_buffer_size to avoid memory pressure. Suggest less than 1% of RAM. If small, increase to 0.01% of RAM to improve some queries.
( query_prealloc_size / _ram ) = 8,192 / 30720M = 0.00% -- For parsing. Pct of RAM
( query_alloc_block_size / _ram ) = 8,192 / 30720M = 0.00% -- For parsing. Pct of RAM
( net_buffer_length / max_allowed_packet ) = 16,384 / 35M = 0.04%
( local_infile ) = ON
-- local_infile = ON is a potential security issue
( Questions ) = 5,877,218,009 / 1123494 = 5231 /sec -- Queries (outside SP) -- "qps"
-- >2000 may be stressing server
( Queries ) = 5,877,263,773 / 1123494 = 5231 /sec -- Queries (including inside SP)
-- >3000 may be stressing server
( Created_tmp_tables ) = 665,507,495 / 1123494 = 592 /sec -- Frequency of creating "temp" tables as part of complex SELECTs.
( Com_delete / Com_insert ) = 15,234,374 / 4603989 = 330.9% -- Deletes / Inserts (as a pct). (Ignores LOAD, REPLACE, etc.)
( Select_scan ) = 491,596,929 / 1123494 = 437 /sec -- full table scans
-- Add indexes / optimize queries (unless they are tiny tables)
( Select_scan / Com_select ) = 491,596,929 / 5360296742 = 9.2% -- % of selects doing full table scan. (May be fooled by Stored Routines.)
-- Add indexes / optimize queries
( Sort_merge_passes ) = 631,000 / 1123494 = 0.56 /sec -- Heafty sorts
-- Increase sort_buffer_size and/or optimize complex queries.
( innodb_autoinc_lock_mode ) = 1 -- Galera: desires 2 -- 2 = "interleaved"; 1 = "consecutive" is typical; 0 = "traditional".
( log_slow_queries ) = OFF -- Whether to log slow queries. (Before 5.1.29, 5.6.1)
( slow_query_log ) = OFF -- Whether to log slow queries. (5.1.12)
( long_query_time ) = 5.000000 = 5 -- Cutoff (Seconds) for defining a "slow" query.
-- Suggest 2
( Slow_queries ) = 407,546,261 / 1123494 = 362 /sec -- Frequency (Slow queries per sec)
-- Rework slow guys; improve indexes; watch disk space for slow log file
( Slow_queries / Questions ) = 407,546,261 / 5877218009 = 6.9% -- Frequency (% of all queries)
-- Find slow queries; check indexes.
( log_queries_not_using_indexes ) = ON -- Whether to include such in slowlog.
-- This clutters the slowlog; turn it off so you can see the real slow queries. And decrease long_query_time to catch most interesting queries.
( max_connect_errors ) = 100,000 -- A small protection against hackers.
-- Perhaps no more than 200.
( Threads_running - 1 ) = 21 - 1 = 20 -- Active threads (concurrency when data collected)
-- Optimize queries and/or schema
( Threads_created / Connections ) = 7,052 / 392731 = 1.8% -- Rapidity of process creation
-- Increase thread_cache_size (non-Windows)
( thread_cache_size / max_connections ) = 8 / 910 = 0.88%
-- (0 for Windows)
( Threads_running / thread_cache_size ) = 21 / 8 = 2.62 -- Threads: current / cached (Not relevant when using thread pooling)
-- Optimize queries
Abnormally large:
Binlog_cache_use = 82 /sec
Bytes_received = 1439336 /sec
Com_commit = 114 /sec
Com_create_table = 60 /HR
Com_delete = 14 /sec
Com_delete_multi = 0.033 /sec
Com_drop_table = 60 /HR
Com_purge_before_date = 0.042 /HR
Com_select = 4771 /sec
Com_set_option = 229 /sec
Com_update = 98 /sec
Created_tmp_files = 0.4 /sec
Handler_commit = 5140 /sec
Handler_prepare = 357 /sec
Handler_read_first = 143 /sec
Handler_write = 19401 /sec
Innodb_buffer_pool_pages_free = 901,470
Select_range = 95 /sec
Sort_range = 3821 /sec
Sort_rows = 15912 /sec
Tc_log_page_waits = 7
Threads_running = 21
Additional suggestions (Jun 21, 2018) to consider for your Linux and my.cnf-ini [mysqld] section
. for Linux since your limit is 1024, in Linux command,
ulimit -n 30000 # to increase Open Files handle count
. for your my.cnf
table_open_cache=10000 # from 256 to lower 20 opened_tables per SECOND
table_definition_cache=800 # from 400 to lower 43,623 opened_table_definitions
open_files_limit=13000 # from 4550 to lower opened_files of 959,559 in 13 days, 1 RPS.
Suggestions to consider for your my.cnf-ini [mysqld] section
max_connections=500 # from 910 only 266 max_used in 13 days
thread_cache_size=100 # from 8 to minimize thread churn and reduce threads_created
read_rnd_buffer_size=128K # from 256K to lower handler_read_rnd_next RPS
key_cache_age_threshold=64800 # from 300 seconds to read same data again
key_cache_division_limit=50 # from 100 for Hot/Warm cache
innodb_buffer_pool_size=24G # from 32G 17G was in use when GLOBAL STATUS recorded
. with your version 5.5.nn special handling required, see REFMAN for next 2.
innodb_log_buffer_size=128M # from 8M to support ~30 minutes
innodb_log_file_size=512M # from 5M for rotation every couple hours
max_write_lock_count=16 # to allow RD after nn locks vs possible wait on 4Billion locks
sort_buffer_size=3M # from 2M to reduce sort_merge_passes
thread_concurrency=30 # from 10 for expedited completion
for additional assistance please find contact information in my profile, Network profile.
(Second answer -- addressing the queries.)
No PRIMARY KEY? Naughty. Does that table have a column (or combination of columns) that is 'naturally' Unique? If so consider making the PK from it/them.
INDEX(p_pkid, p_billingaddress, p_shippingaddress, TypeGKString), in any order, will help the SELECT performance significantly, thereby speeding things up and decreasing CPU consumption.
INDEX(GK) is needed for the UPDATE, (Where's the SHOW CREATE TABLE cronjobs?)
Don't use BIGINT (8 bytes) for columns that need it. (How many different "genders" are there??) (Consider CHAR(2) CHARACTER SET ascii for country.) See TINYINT (1 byte) and its friends.
(Yes, this in my 3rd answer. This time to spell out findings for newer SHOW GLOBAL STATUS.)
Hmmm... I used the old SHOW VARIABLEs, so some of the advice is based on values from that. If you have followed my previous advice, then I need a new copy of SHOW GLOBAL VARIABLES.
Observations:
Version: 5.5.47-log
40 GB of RAM
Uptime = 11d 23:29:02
You are not running on Windows.
Running 64-bit version
You appear to be running entirely (or mostly) InnoDB.
The More Important Issues:
See this about changing innodb_log_file_size. (Or see the 5.5 manual, etc)
innodb_log_file_size = 80M -- too much waiting for the log. See above
innodb_buffer_pool_instances = 16 -- minor contention relief
table_open_cache = 512 -- bumping entries out of this cache has some impact
thread_cache_size = 200 -- process creation is somewhat costly
log_queries_not_using_indexes = OFF -- otherwise clutter
See this for finding the 'worst' queries for us to help you optimize.
Yikes, 268 queries/sec taking more than 5 seconds.
Deleting a lot more often than inserting? What is going on?
9K SETs per second? What is going on?
Why did you choose READ-COMMITTED?
Hundreds of Threads_running usually implies that MySQL is stumbling over itself, and it might be better to throttle the number of connections in the client. Some benchmarks show that 5.6 and 5.7 actually handle this situation better; 5.5 tends to decline in throughtput (queries/sec) after a few dozen connections. Consider upgrading, also since you are 3 major revisions behind.
Details and other observations:
( Opened_tables ) = 13,001,730 / 1034942 = 13 /sec -- Frequency of opening Tables
-- increase table_open_cache
( innodb_buffer_pool_size / innodb_buffer_pool_instances ) = 30720M / 1 = 30720MB -- Size of each buffer_pool instance.
-- An instance should be at least 1GB. In very large RAM, have 16 instances.
( Innodb_os_log_written / (Uptime / 3600) / innodb_log_files_in_group / innodb_log_file_size ) = 45,704,499,200 / (1034942 / 3600) / 2 / 5M = 15.2 -- Ratio
-- (see minutes)
( Uptime / 60 * innodb_log_file_size / Innodb_os_log_written ) = 1,034,942 / 60 * 5M / 45704499200 = 1.98 -- Minutes between InnoDB log rotations Beginning with 5.6.8, this can be changed dynamically; be sure to also change my.cnf.
-- (The recommendation of 60 minutes between rotations is somewhat arbitrary.) Adjust innodb_log_file_size. (Cannot change in AWS.)
( Innodb_rows_deleted / Innodb_rows_inserted ) = 3,412,394 / 4031801 = 0.846 -- Churn
-- "Don't queue it, just do it." (If MySQL is being used as a queue.)
( Innodb_row_lock_waits ) = 256,120 / 1034942 = 0.25 /sec -- How often there is a delay in getting a row lock.
-- May be caused by complex queries that could be optimized.
( innodb_stats_on_metadata ) = innodb_stats_on_metadata = ON -- Re-analyze table when touching stats.
-- ON is likely to slow down certain SHOWs and information_schema accesses.
( innodb_thread_concurrency ) = 0 -- 0 = Let InnoDB decide the best for concurrency_tickets.
-- Set to 0 or 64. This may cut back on CPU.
( innodb_print_all_deadlocks ) = innodb_print_all_deadlocks = OFF -- Whether to log all Deadlocks.
-- If you are plagued with Deadlocks, turn this on. Caution: If you have lots of deadlocks, this may write a lot to disk.
( net_buffer_length / max_allowed_packet ) = 16,384 / 35M = 0.04%
( local_infile ) = local_infile = ON
-- local_infile = ON is a potential security issue
( bulk_insert_buffer_size / _ram ) = 8M / 40960M = 0.02% -- Buffer for multi-row INSERTs and LOAD DATA
-- Too big could threaten RAM size. Too small could hinder such operations.
( Questions ) = 15,894,065,519 / 1034942 = 15357 /sec -- Queries (outside SP) -- "qps"
-- >2000 may be stressing server
( Queries ) = 15,894,107,993 / 1034942 = 15357 /sec -- Queries (including inside SP)
-- >3000 may be stressing server
( Created_tmp_tables ) = 501,042,067 / 1034942 = 484 /sec -- Frequency of creating "temp" tables as part of complex SELECTs.
( (Com_insert + Com_update + Com_delete + Com_replace) / Com_commit ) = (4031334 + 54661875 + 13451091 + 0) / 4930449084 = 0.0146 -- Statements per Commit (assuming all InnoDB)
-- Low: Might help to group queries together in transactions; High: long transactions strain various things.
( Com_delete / Com_insert ) = 13,451,091 / 4031334 = 333.7% -- Deletes / Inserts (as a pct). (Ignores LOAD, REPLACE, etc.)
( Select_scan ) = 348,481,657 / 1034942 = 336 /sec -- full table scans
-- Add indexes / optimize queries (unless they are tiny tables)
( Select_scan / Com_select ) = 348,481,657 / 1029506656 = 33.8% -- % of selects doing full table scan. (May be fooled by Stored Routines.)
-- Add indexes / optimize queries
( Sort_merge_passes ) = 297,216 / 1034942 = 0.29 /sec -- Heafty sorts
-- Increase sort_buffer_size and/or optimize complex queries.
( Com_insert + Com_delete + Com_delete_multi + Com_replace + Com_update + Com_update_multi ) = (4031334 + 13451091 + 34494 + 0 + 54661875 + 0) / 1034942 = 70 /sec -- writes/sec
-- 50 writes/sec + log flushes will probably max out I/O write capacity of normal drives
( log_slow_queries ) = log_slow_queries = OFF -- Whether to log slow queries. (Before 5.1.29, 5.6.1)
( slow_query_log ) = slow_query_log = OFF -- Whether to log slow queries. (5.1.12)
( long_query_time ) = 5 -- Cutoff (Seconds) for defining a "slow" query.
-- Suggest 2
( Slow_queries ) = 277,531,287 / 1034942 = 268 /sec -- Frequency (Slow queries per sec)
-- Rework slow guys; improve indexes; watch disk space for slow log file
( Slow_queries / Questions ) = 277,531,287 / 15894065519 = 1.7% -- Frequency (% of all queries)
-- Find slow queries; check indexes.
( log_queries_not_using_indexes ) = log_queries_not_using_indexes = ON -- Whether to include such in slowlog.
-- This clutters the slowlog; turn it off so you can see the real slow queries. And decrease long_query_time to catch most interesting queries.
( max_connect_errors ) = 100,000 -- A small protection against hackers.
-- Perhaps no more than 200.
( Threads_running - 1 ) = 215 - 1 = 214 -- Active threads (concurrency when data collected)
-- Optimize queries and/or schema
( Threads_created / Connections ) = 6,390 / 361954 = 1.8% -- Rapidity of process creation
-- Increase thread_cache_size (non-Windows)
( thread_cache_size / max_connections ) = 8 / 910 = 0.88%
-- (0 for Windows)
( Threads_running / thread_cache_size ) = 215 / 8 = 26.9 -- Threads: current / cached (Not relevant when using thread pooling)
-- Optimize queries
36 issues flagged, out of 162 computed Variables/Status/Expressions
Abnormally large:
Bytes_received = 820009 /sec
Com_commit = 4763 /sec
Com_delete = 13 /sec
Com_delete_multi = 0.033 /sec
Com_drop_table = 60 /HR
Com_purge_before_date = 0.042 /HR
Com_set_option = 9528 /sec
Com_show_engine_status = 21 /HR
Created_tmp_files = 0.21 /sec
Handler_commit = 1207 /sec
Handler_read_first = 105 /sec
Handler_read_key = 72824 /sec
Handler_write = 25131 /sec
Innodb_buffer_pool_pages_misc = 100,245
Key_write_requests = 324 /sec
Select_range = 73 /sec
Sort_range = 221 /sec
Tc_log_page_waits = 1
Threads_running = 215
Abnormal strings:
innodb_fast_shutdown = 1
innodb_use_native_aio = OFF
relay_log_info_file = /PK/mysql/mysql-relay-log-info
time_zone = SYSTPK
tx_isolation = READ-COMMITTED
I have read a number of answers here on SO, as well as gone through this excellent post from Percona. My max innodb_buffer_pool_size is 50MB, with one instance. Query cache is off, and MYISAM is pretty much not in use.
Relevant output of pt-mysql-summary:
# InnoDB #####################################################
Version | 5.6.35
Buffer Pool Size | 50.0M
Buffer Pool Fill | 5%
Buffer Pool Dirty | 0%
File Per Table | ON
Page Size | 16k
Log File Size | 2 * 48.0M = 96.0M
Log Buffer Size | 8M
Flush Method |
Flush Log At Commit | 1
XA Support | ON
Checksums | ON
Doublewrite | ON
R/W I/O Threads | 4 4
I/O Capacity | 200
Thread Concurrency | 0
Concurrency Tickets | 5000
Commit Concurrency | 0
Txn Isolation Level | REPEATABLE-READ
Adaptive Flushing | ON
Adaptive Checkpoint |
Checkpoint Age | 0
InnoDB Queue | 0 queries inside InnoDB, 0 queries in queue
Oldest Transaction | 0 Seconds
History List Len | 0
Read Views | 0
Undo Log Entries | 0 transactions, 0 total undo, 0 max undo
Pending I/O Reads | 0 buf pool reads, 0 normal AIO, 0 ibuf AIO, 0 preads
Pending I/O Writes | 0 buf pool (0 LRU, 0 flush list, 0 page); 0 AIO, 0 sync, 0 log IO (0 log, 0 chkp); 0 pwrites
Pending I/O Flushes | 0 buf pool, 0 log
Transaction States | 1xnot started
# MyISAM #####################################################
Key Cache | 8.0M
Pct Used | 20%
Unflushed | 0%
# Query cache ################################################
query_cache_type | OFF
Size | 0.0
Usage | 0%
HitToInsertRatio | 0%
There are no connections/processes. Yet top shows:
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
1 mysql 20 0 973460 465988 12780 S 0.0 11.5 0:00.85 mysqld
Almost 500MB of RAM! According to all my memory settings, it shouldn't be more than maybe 60MB or 70MB.
What am I missing here?
you should disable the performance schema among other tweaks. this is what I use, which produces a mysqld that consumes less than 100MB:
https://github.com/gbolo/vSummary/blob/master/docker/my.cnf
[mysqld]
performance_schema = 0
skip-host-cache
skip-name-resolve
user = mysql
pid-file = /var/run/mysqld/mysqld.pid
socket = /var/run/mysqld/mysqld.sock
port = 3306
basedir = /usr
datadir = /data/mysql/data
tmpdir = /tmp
skip-external-locking
max_connections = 100
connect_timeout = 5
wait_timeout = 600
max_allowed_packet = 16M
thread_cache_size = 128
sort_buffer_size = 4M
bulk_insert_buffer_size = 16M
tmp_table_size = 32M
max_heap_table_size = 32M
myisam_recover_options = BACKUP
key_buffer_size = 128M
table_open_cache = 400
myisam_sort_buffer_size = 512M
concurrent_insert = 2
read_buffer_size = 2M
read_rnd_buffer_size = 1M
query_cache_limit = 128K
query_cache_size = 64M
slow_query_log_file = /data/mysql/logs/mariadb-slow.log
long_query_time = 10
expire_logs_days = 10
max_binlog_size = 100M
default_storage_engine = InnoDB
innodb_buffer_pool_size = 256M
innodb_log_buffer_size = 8M
innodb_file_per_table = 1
innodb_open_files = 400
innodb_io_capacity = 400
innodb_flush_method = O_DIRECT
[mysqldump]
quick
quote-names
max_allowed_packet = 16M
[isamchk]
key_buffer = 16M
I recently restarted my MariaDB (Mysql) server after updating to the last version (Mariadb 10.1.9). But since then, my nagios report a constant memory used increase and I can't figure out where it comes from.
The memory used was already recorded by Nagios as 150.000 and currently it increase constantly. The request of this morning show 620.000 :
QUERY WARNING: 'SELECT ROUND(VARIABLE_VALUE/1024/1024) FROM information_schema.GLOBAL_STATUS WHERE VARIABLE_NAME = "Memory_used"' returned 620.000000
Is it possible that the problem come from the update + restart, or a different configuration file since package update or simply something else?
Thanks for your help
UPDATE
show variables like "%cache%"
Variable_name
Value
aria_pagecache_age_threshold
300
aria_pagecache_buffer_size
134217728
aria_pagecache_division_limit
100
aria_pagecache_file_hash_size
512
binlog_cache_size
32768
binlog_stmt_cache_size
32768
have_query_cache
YES
host_cache_size
279
innodb_disable_sort_file_cache
OFF
innodb_ft_cache_size
8000000
innodb_ft_result_cache_limit
2000000000
innodb_ft_total_cache_size
640000000
join_cache_level
2
key_cache_age_threshold
300
key_cache_block_size
1024
key_cache_division_limit
100
key_cache_file_hash_size
512
key_cache_segments
0
max_binlog_cache_size
18446744073709547520
max_binlog_stmt_cache_size
18446744073709547520
metadata_locks_cache_size
1024
query_cache_limit
1048576
query_cache_min_res_unit
4096
query_cache_size
1048576
query_cache_strip_comments
OFF
query_cache_type
OFF
query_cache_wlock_invalidate
OFF
stored_program_cache
256
table_definition_cache
400
table_open_cache
64
thread_cache_size
16
show variables like "%buffer%"
Variable_name
Value
aria_pagecache_buffer_size
134217728
aria_sort_buffer_size
268434432
bulk_insert_buffer_size
8388608
innodb_buffer_pool_dump_at_shutdown
OFF
innodb_buffer_pool_dump_now
OFF
innodb_buffer_pool_filename
ib_buffer_pool
innodb_buffer_pool_instances
8
innodb_buffer_pool_load_abort
OFF
innodb_buffer_pool_load_at_startup
OFF
innodb_buffer_pool_load_now
OFF
innodb_buffer_pool_populate
OFF
innodb_buffer_pool_size
134217728
innodb_change_buffer_max_size
25
innodb_change_buffering
all
innodb_log_buffer_size
16777216
innodb_sort_buffer_size
1048576
join_buffer_size
262144
join_buffer_space_limit
2097152
key_buffer_size
16777216
mrr_buffer_size
262144
myisam_sort_buffer_size
8388608
net_buffer_length
8192
preload_buffer_size
32768
read_buffer_size
262144
read_rnd_buffer_size
524288
sort_buffer_size
524288
sql_buffer_result
OFF
show variables like "%max_connections%"
Variable_name
Value
extra_max_connections
1
max_connections
151
3 more parameters are missing for the exact calculation:
tmp_table_size,
thread_stack,
innodb_additional_mem_pool_size
But 620MB of memory consumption is normal for the configuration you provided.
I assume that the upgrade also modified the configuration files of MariaDB.