We have InnoDB on dedicated server and from time to time a simple join query (normally very fast) causes an infinite wait and dies after 30 seconds.
The query looks like "SELECT * FROM orders LEFT JOIN orders_products USING(code)"
This is EXPLAIN:
I am convinced InnoDB is f this up and I don't know why, but it's when more people access the script at the same time.
I am NOT doing any updates or inserts in that script, so it makes no sense.
It could be that somewhere is a script that does a concurrent write and maybe then this main script fails? I have no clue. I never had issue on MyIsam and I don't use transactions.
Can we change settings somehow to fix this?
We have a big server, but neither the admin or I know how to set this properly.
Suggestions are welcome.
This is current settings:
Variable_name
Value
innodb_adaptive_flushing
ON
innodb_adaptive_flushing_lwm
10.000000
innodb_adaptive_hash_index
ON
innodb_adaptive_hash_index_partitions
1
innodb_adaptive_max_sleep_delay
150000
innodb_additional_mem_pool_size
8388608
innodb_api_bk_commit_interval
5
innodb_api_disable_rowlock
OFF
innodb_api_enable_binlog
OFF
innodb_api_enable_mdl
OFF
innodb_api_trx_level
0
innodb_autoextend_increment
64
innodb_autoinc_lock_mode
1
innodb_background_scrub_data_check_interval
3600
innodb_background_scrub_data_compressed
OFF
innodb_background_scrub_data_interval
604800
innodb_background_scrub_data_uncompressed
OFF
innodb_buf_dump_status_frequency
0
innodb_buffer_pool_dump_at_shutdown
OFF
innodb_buffer_pool_dump_now
OFF
innodb_buffer_pool_dump_pct
100
innodb_buffer_pool_filename
ib_buffer_pool
innodb_buffer_pool_instances
2
innodb_buffer_pool_load_abort
OFF
innodb_buffer_pool_load_at_startup
OFF
innodb_buffer_pool_load_now
OFF
innodb_buffer_pool_populate
OFF
innodb_buffer_pool_size
2147483648
innodb_change_buffer_max_size
25
innodb_change_buffering
all
innodb_checksum_algorithm
CRC32
innodb_checksums
ON
innodb_cleaner_lsn_age_factor
HIGH_CHECKPOINT
innodb_cmp_per_index_enabled
OFF
innodb_commit_concurrency
0
innodb_compression_algorithm
zlib
innodb_compression_failure_threshold_pct
5
innodb_compression_level
6
innodb_compression_pad_pct_max
50
innodb_concurrency_tickets
5000
innodb_corrupt_table_action
assert
innodb_data_file_path
ibdata1:12M:autoextend
innodb_data_home_dir
innodb_default_encryption_key_id
1
innodb_default_row_format
compact
innodb_defragment
OFF
innodb_defragment_fill_factor
0.900000
innodb_defragment_fill_factor_n_recs
20
innodb_defragment_frequency
40
innodb_defragment_n_pages
7
innodb_defragment_stats_accuracy
0
innodb_disable_sort_file_cache
OFF
innodb_disallow_writes
OFF
innodb_doublewrite
ON
innodb_empty_free_list_algorithm
BACKOFF
innodb_encrypt_log
OFF
innodb_encrypt_tables
OFF
innodb_encryption_rotate_key_age
1
innodb_encryption_rotation_iops
100
innodb_encryption_threads
0
innodb_fake_changes
OFF
innodb_fast_shutdown
1
innodb_fatal_semaphore_wait_threshold
600
innodb_file_format
Barracuda
innodb_file_format_check
ON
innodb_file_format_max
Antelope
innodb_file_per_table
ON
innodb_flush_log_at_timeout
1
innodb_flush_log_at_trx_commit
2
innodb_flush_method
O_DIRECT
innodb_flush_neighbors
1
innodb_flushing_avg_loops
30
innodb_force_load_corrupted
OFF
innodb_force_primary_key
OFF
innodb_force_recovery
0
innodb_foreground_preflush
EXPONENTIAL_BACKOFF
innodb_ft_aux_table
innodb_ft_cache_size
8000000
innodb_ft_enable_diag_print
OFF
innodb_ft_enable_stopword
ON
innodb_ft_max_token_size
84
innodb_ft_min_token_size
3
innodb_ft_num_word_optimize
2000
innodb_ft_result_cache_limit
2000000000
innodb_ft_server_stopword_table
innodb_ft_sort_pll_degree
2
innodb_ft_total_cache_size
640000000
innodb_ft_user_stopword_table
innodb_idle_flush_pct
100
innodb_immediate_scrub_data_uncompressed
OFF
innodb_instrument_semaphores
OFF
innodb_io_capacity
200
innodb_io_capacity_max
2000
innodb_kill_idle_transaction
0
innodb_large_prefix
ON
innodb_lock_schedule_algorithm
fcfs
innodb_lock_wait_timeout
50
innodb_locking_fake_changes
ON
innodb_locks_unsafe_for_binlog
OFF
innodb_log_arch_dir
./
Variable_name
Value
innodb_log_arch_expire_sec
0
innodb_log_archive
OFF
innodb_log_block_size
512
innodb_log_buffer_size
16777216
innodb_log_checksum_algorithm
INNODB
innodb_log_compressed_pages
ON
innodb_log_file_size
268435456
innodb_log_files_in_group
2
innodb_log_group_home_dir
./
innodb_lru_scan_depth
1024
innodb_max_bitmap_file_size
104857600
innodb_max_changed_pages
1000000
innodb_max_dirty_pages_pct
75.000000
innodb_max_dirty_pages_pct_lwm
0.001000
innodb_max_purge_lag
0
innodb_max_purge_lag_delay
0
innodb_mirrored_log_groups
1
innodb_monitor_disable
innodb_monitor_enable
innodb_monitor_reset
innodb_monitor_reset_all
innodb_mtflush_threads
8
innodb_old_blocks_pct
37
innodb_old_blocks_time
1000
innodb_online_alter_log_max_size
134217728
innodb_open_files
5000
innodb_optimize_fulltext_only
OFF
innodb_page_size
16384
innodb_prefix_index_cluster_optimization
OFF
innodb_print_all_deadlocks
OFF
innodb_print_lock_wait_timeout_info
OFF
innodb_purge_batch_size
300
innodb_purge_threads
1
innodb_random_read_ahead
OFF
innodb_read_ahead_threshold
56
innodb_read_io_threads
4
innodb_read_only
OFF
innodb_replication_delay
0
innodb_rollback_on_timeout
OFF
innodb_rollback_segments
128
innodb_sched_priority_cleaner
19
innodb_scrub_log
OFF
innodb_scrub_log_speed
256
innodb_show_locks_held
10
innodb_show_verbose_locks
0
innodb_simulate_comp_failures
0
innodb_sort_buffer_size
1048576
innodb_spin_wait_delay
6
innodb_stats_auto_recalc
ON
innodb_stats_include_delete_marked
OFF
innodb_stats_method
nulls_equal
innodb_stats_modified_counter
0
innodb_stats_on_metadata
OFF
innodb_stats_persistent
ON
innodb_stats_persistent_sample_pages
20
innodb_stats_sample_pages
8
innodb_stats_traditional
ON
innodb_stats_transient_sample_pages
8
innodb_status_output
OFF
innodb_status_output_locks
OFF
innodb_strict_mode
OFF
innodb_support_xa
ON
innodb_sync_array_size
1
innodb_sync_spin_loops
30
innodb_table_locks
ON
innodb_thread_concurrency
0
innodb_thread_sleep_delay
10000
innodb_tmpdir
innodb_track_changed_pages
OFF
innodb_undo_directory
.
innodb_undo_logs
128
innodb_undo_tablespaces
0
innodb_use_atomic_writes
OFF
innodb_use_fallocate
OFF
innodb_use_global_flush_log_at_trx_commit
ON
innodb_use_mtflush
OFF
innodb_use_native_aio
ON
innodb_use_stacktrace
OFF
innodb_use_sys_malloc
ON
innodb_use_trim
OFF
innodb_version
5.6.46-86.2
innodb_write_io_threads
4
So, I just split the query to 2 queries (many rows/conditions) and it works now.
Probably was difficult query (despite the fact it was quick) and when having more hits, it stalled. Idk.
Related
We migrated several development databases to mysql8 a couple months ago and it was onlyin running large imports/sql scripts that we realized how slow our inserts are.
As an example I created a table as follows and inserted 1000 records,
CREATE TABLE IF NOT EXISTS `ciiu_test2` (
`id` int(11) DEFAULT NULL,
`codbut` varchar(11) DEFAULT NULL,
`ciiu` varchar(8) DEFAULT NULL,
`dt` float(14,2) DEFAULT NULL
)
delete from ciiu_test2;
select UNIX_TIMESTAMP(CURTIME(3)) into #ctime;
INSERT INTO `ciiu_test2` (`id`, `codbut`, `ciiu`,dt) VALUES (1, '1', '1',UNIX_TIMESTAMP(CURTIME(3))-#ctime);
INSERT INTO `ciiu_test2` (`id`, `codbut`, `ciiu`,dt) VALUES (2, '2', '2',UNIX_TIMESTAMP(CURTIME(3))-#ctime);
etc...
With autocommit on, it took 45 seconds to insert 1000 records, with autocommit off, putting everything under one transaction, it took around .6 seconds
Now, I have read about logging and I've turned off logging that might cause a problem. WE don't have replication running. In fact, I have a mysql8 running off docker with the same configuration and no issue.
Here is the configuration for the mysql8 server and the one for docker
Variable_name
167value
51_8value
auto_generate_certs
ON
ON
auto_increment_increment
1
1
auto_increment_offset
1
1
autocommit
ON
ON
automatic_sp_privileges
ON
ON
avoid_temporal_upgrade
OFF
OFF
back_log
151
151
basedir
/usr/
/usr/
big_tables
OFF
OFF
bind_address
*
*
binlog_cache_size
32768
32768
binlog_checksum
CRC32
CRC32
binlog_direct_non_transactional_updates
OFF
OFF
binlog_error_action
ABORT_SERVER
ABORT_SERVER
binlog_format
ROW
ROW
binlog_group_commit_sync_delay
0
0
binlog_group_commit_sync_no_delay_count
0
0
binlog_gtid_simple_recovery
ON
ON
binlog_max_flush_queue_time
0
0
binlog_order_commits
ON
ON
binlog_row_image
FULL
FULL
binlog_rows_query_log_events
OFF
OFF
binlog_stmt_cache_size
32768
32768
binlog_transaction_dependency_history_size
25000
25000
binlog_transaction_dependency_tracking
COMMIT_ORDER
COMMIT_ORDER
block_encryption_mode
aes-128-ecb
aes-128-ecb
bulk_insert_buffer_size
8388608
8388608
character_sets_dir
/usr/share/mysql/charsets/
/usr/share/mysql-8.0/charsets/
check_proxy_users
OFF
OFF
concurrent_insert
AUTO
AUTO
connect_timeout
10
10
core_file
OFF
OFF
datadir
/var/lib/mysql/
/var/lib/mysql/
date_format
not_defined
not_defined
datetime_format
not_defined
not_defined
default_authentication_plugin
mysql_native_password
caching_sha2_password
default_password_lifetime
0
0
default_storage_engine
InnoDB
InnoDB
default_tmp_storage_engine
InnoDB
InnoDB
default_week_format
0
0
delay_key_write
ON
ON
delayed_insert_limit
100
100
delayed_insert_timeout
300
300
delayed_queue_size
1000
1000
disabled_storage_engines
disconnect_on_expired_password
ON
ON
div_precision_increment
4
4
end_markers_in_json
OFF
OFF
enforce_gtid_consistency
OFF
OFF
eq_range_index_dive_limit
200
200
error_count
0
0
event_scheduler
ON
ON
expire_logs_days
0
0
explicit_defaults_for_timestamp
ON
ON
external_user
flush
OFF
OFF
flush_time
0
0
foreign_key_checks
ON
ON
ft_boolean_syntax
+ -><()~*:""&|
+ -><()~*:""&|
ft_max_word_len
84
84
ft_min_word_len
4
4
ft_query_expansion_limit
20
20
ft_stopword_file
(built-in)
(built-in)
general_log
OFF
OFF
general_log_file
/var/lib/mysql/ns515408.log
/var/lib/mysql/093b8c5ba5be.log
group_concat_max_len
1024
1024
gtid_executed_compression_period
1000
1000
gtid_mode
OFF
OFF
gtid_next
AUTOMATIC
AUTOMATIC
gtid_owned
gtid_purged
have_compress
YES
YES
have_crypt
not_defined
not_defined
have_dynamic_loading
YES
YES
have_geometry
YES
YES
have_openssl
YES
YES
have_profiling
YES
YES
have_query_cache
NO
NO
have_rtree_keys
YES
YES
have_ssl
YES
YES
have_statement_timeout
YES
YES
have_symlink
DISABLED
DISABLED
host_cache_size
279
279
hostname
ns515408.ip-167-114-174.net
093b8c5ba5be
identity
0
0
ignore_builtin_innodb
not_defined
not_defined
ignore_db_dirs
not_defined
not_defined
init_connect
init_file
init_slave
innodb_adaptive_flushing
ON
ON
innodb_adaptive_flushing_lwm
10
10
innodb_adaptive_hash_index
ON
ON
innodb_adaptive_hash_index_parts
8
8
innodb_adaptive_max_sleep_delay
150000
150000
innodb_api_bk_commit_interval
5
5
innodb_api_disable_rowlock
OFF
OFF
innodb_api_enable_binlog
OFF
OFF
innodb_api_enable_mdl
OFF
OFF
innodb_api_trx_level
0
0
innodb_autoextend_increment
64
64
innodb_autoinc_lock_mode
2
2
innodb_buffer_pool_chunk_size
134217728
134217728
innodb_buffer_pool_dump_at_shutdown
ON
ON
innodb_buffer_pool_dump_now
OFF
OFF
innodb_buffer_pool_dump_pct
25
25
innodb_buffer_pool_filename
ib_buffer_pool
ib_buffer_pool
innodb_buffer_pool_instances
1
1
innodb_buffer_pool_load_abort
OFF
OFF
innodb_buffer_pool_load_at_startup
ON
ON
innodb_buffer_pool_load_now
OFF
OFF
innodb_buffer_pool_size
134217728
134217728
innodb_change_buffer_max_size
25
25
innodb_change_buffering
all
all
innodb_checksum_algorithm
crc32
crc32
innodb_checksums
not_defined
not_defined
innodb_cmp_per_index_enabled
OFF
OFF
innodb_commit_concurrency
0
0
innodb_compression_failure_threshold_pct
5
5
innodb_compression_level
6
6
innodb_compression_pad_pct_max
50
50
innodb_concurrency_tickets
5000
5000
innodb_data_file_path
ibdata1:12M:autoextend
ibdata1:12M:autoextend
innodb_data_home_dir
innodb_deadlock_detect
ON
ON
innodb_default_row_format
dynamic
dynamic
innodb_disable_sort_file_cache
OFF
OFF
innodb_doublewrite
ON
ON
innodb_fast_shutdown
1
1
innodb_file_format
not_defined
not_defined
innodb_file_format_check
not_defined
not_defined
innodb_file_format_max
not_defined
not_defined
innodb_file_per_table
ON
ON
innodb_fill_factor
100
100
innodb_flush_log_at_timeout
1
1
innodb_flush_log_at_trx_commit
1
1
innodb_flush_method
fsync
fsync
innodb_flush_neighbors
0
0
innodb_flush_sync
ON
ON
innodb_flushing_avg_loops
30
30
innodb_force_load_corrupted
OFF
OFF
innodb_force_recovery
0
0
innodb_ft_aux_table
innodb_ft_cache_size
8000000
8000000
innodb_ft_enable_diag_print
OFF
OFF
innodb_ft_enable_stopword
ON
ON
innodb_ft_max_token_size
84
84
innodb_ft_min_token_size
3
3
innodb_ft_num_word_optimize
2000
2000
innodb_ft_result_cache_limit
2000000000
2000000000
innodb_ft_server_stopword_table
innodb_ft_sort_pll_degree
2
2
innodb_ft_total_cache_size
640000000
640000000
innodb_ft_user_stopword_table
innodb_io_capacity
200
200
innodb_io_capacity_max
2000
2000
innodb_large_prefix
not_defined
not_defined
innodb_lock_wait_timeout
50
50
innodb_locks_unsafe_for_binlog
not_defined
not_defined
innodb_log_buffer_size
16777216
16777216
innodb_log_checksums
ON
ON
innodb_log_compressed_pages
ON
ON
innodb_log_file_size
50331648
50331648
innodb_log_files_in_group
2
2
innodb_log_group_home_dir
./
./
innodb_log_write_ahead_size
8192
8192
innodb_lru_scan_depth
1024
1024
innodb_max_dirty_pages_pct
90
90
innodb_max_dirty_pages_pct_lwm
10
10
innodb_max_purge_lag
0
0
innodb_max_purge_lag_delay
0
0
innodb_max_undo_log_size
1073741824
1073741824
innodb_monitor_disable
innodb_monitor_enable
innodb_monitor_reset
innodb_monitor_reset_all
innodb_numa_interleave
OFF
OFF
innodb_old_blocks_pct
37
37
innodb_old_blocks_time
1000
1000
innodb_online_alter_log_max_size
134217728
134217728
innodb_open_files
4000
4000
innodb_optimize_fulltext_only
OFF
OFF
innodb_page_cleaners
1
1
innodb_page_size
16384
16384
innodb_print_all_deadlocks
OFF
OFF
innodb_purge_batch_size
300
300
innodb_purge_rseg_truncate_frequency
128
128
innodb_purge_threads
4
4
innodb_random_read_ahead
OFF
OFF
innodb_read_ahead_threshold
56
56
innodb_read_io_threads
4
4
innodb_read_only
OFF
OFF
innodb_replication_delay
0
0
innodb_rollback_on_timeout
OFF
OFF
innodb_rollback_segments
128
128
innodb_sort_buffer_size
1048576
1048576
innodb_spin_wait_delay
6
6
innodb_stats_auto_recalc
ON
ON
innodb_stats_include_delete_marked
OFF
OFF
innodb_stats_method
nulls_equal
nulls_equal
innodb_stats_on_metadata
OFF
OFF
innodb_stats_persistent
ON
ON
innodb_stats_persistent_sample_pages
20
20
innodb_stats_sample_pages
not_defined
not_defined
innodb_stats_transient_sample_pages
8
8
innodb_status_output
OFF
OFF
innodb_status_output_locks
OFF
OFF
innodb_strict_mode
ON
ON
innodb_support_xa
not_defined
not_defined
innodb_sync_array_size
1
1
innodb_sync_spin_loops
30
30
innodb_table_locks
ON
ON
innodb_temp_data_file_path
ibtmp1:12M:autoextend
ibtmp1:12M:autoextend
innodb_thread_concurrency
0
0
innodb_thread_sleep_delay
10000
10000
innodb_tmpdir
innodb_undo_directory
./
./
innodb_undo_log_truncate
ON
ON
innodb_undo_logs
not_defined
not_defined
innodb_undo_tablespaces
2
2
innodb_use_native_aio
ON
ON
innodb_version
8.0.21
8.0.22
innodb_write_io_threads
4
4
insert_id
0
0
interactive_timeout
28800
28800
internal_tmp_disk_storage_engine
not_defined
not_defined
join_buffer_size
262144
262144
keep_files_on_create
OFF
OFF
key_buffer_size
8388608
8388608
key_cache_age_threshold
300
300
key_cache_block_size
1024
1024
key_cache_division_limit
100
100
keyring_operations
ON
ON
large_files_support
ON
ON
large_page_size
0
0
large_pages
OFF
OFF
last_insert_id
0
0
lc_messages
en_US
en_US
lc_messages_dir
/usr/share/mysql/
/usr/share/mysql-8.0/
lc_time_names
en_US
en_US
license
GPL
GPL
local_infile
OFF
OFF
lock_wait_timeout
31536000
31536000
locked_in_memory
OFF
OFF
log_bin
ON
ON
log_bin_basename
/var/lib/mysql/binlog
/var/lib/mysql/binlog
log_bin_index
/var/lib/mysql/binlog.index
/var/lib/mysql/binlog.index
log_bin_trust_function_creators
OFF
OFF
log_bin_use_v1_row_events
OFF
OFF
log_builtin_as_identified_by_password
not_defined
not_defined
log_error
/var/log/mysql/mysqld.log
stderr
log_error_verbosity
2
2
log_output
FILE
FILE
log_queries_not_using_indexes
OFF
OFF
log_slave_updates
ON
ON
log_slow_admin_statements
OFF
OFF
log_slow_slave_statements
OFF
OFF
log_statements_unsafe_for_binlog
ON
ON
log_syslog
not_defined
not_defined
log_syslog_facility
not_defined
not_defined
log_syslog_include_pid
not_defined
not_defined
log_syslog_tag
not_defined
not_defined
log_throttle_queries_not_using_indexes
0
0
log_timestamps
UTC
UTC
log_warnings
not_defined
not_defined
long_query_time
10
10
max_allowed_packet
67108864
67108864
max_binlog_cache_size
1.84467E+19
1.84467E+19
max_binlog_size
1073741824
1073741824
max_binlog_stmt_cache_size
1.84467E+19
1.84467E+19
max_connect_errors
100
100
max_connections
151
151
max_delayed_threads
20
20
max_digest_length
1024
1024
max_error_count
1024
1024
max_execution_time
0
0
max_heap_table_size
16777216
16777216
max_insert_delayed_threads
20
20
max_join_size
1.84467E+19
1.84467E+19
max_length_for_sort_data
4096
4096
max_points_in_geometry
65536
65536
max_prepared_stmt_count
16382
16382
max_relay_log_size
0
0
max_seeks_for_key
1.84467E+19
1.84467E+19
max_sort_length
1024
1024
max_sp_recursion_depth
0
0
max_tmp_tables
not_defined
not_defined
max_user_connections
0
0
max_write_lock_count
1.84467E+19
1.84467E+19
metadata_locks_cache_size
not_defined
not_defined
metadata_locks_hash_instances
not_defined
not_defined
min_examined_row_limit
0
0
multi_range_count
not_defined
not_defined
myisam_data_pointer_size
6
6
myisam_max_sort_file_size
9.22337E+18
9.22337E+18
myisam_mmap_size
1.84467E+19
1.84467E+19
myisam_recover_options
OFF
OFF
myisam_repair_threads
1
1
myisam_sort_buffer_size
8388608
8388608
myisam_stats_method
nulls_unequal
nulls_unequal
myisam_use_mmap
OFF
OFF
mysql_native_password_proxy_users
OFF
OFF
net_buffer_length
16384
16384
net_read_timeout
30
30
net_retry_count
10
10
net_write_timeout
60
60
new
OFF
OFF
open_files_limit
10000
1048576
optimizer_prune_level
1
1
optimizer_search_depth
62
62
optimizer_switch
index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,engine_condition_pushdown=on,index_condition_pushdown=on,mrr=on,mrr_cost_based=on,block_nested_loop=on,batched_key_access=off,materialization=on,semijoin=on,loosescan=on,firstmatch=on,duplicateweedout=on,subquery_materialization_cost_based=on,use_index_extensions=on,condition_fanout_filter=on,derived_merge=on,use_invisible_indexes=off,skip_scan=on,hash_join=on,subquery_to_derived=off,prefer_ordering_index=on
index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,engine_condition_pushdown=on,index_condition_pushdown=on,mrr=on,mrr_cost_based=on,block_nested_loop=on,batched_key_access=off,materialization=on,semijoin=on,loosescan=on,firstmatch=on,duplicateweedout=on,subquery_materialization_cost_based=on,use_index_extensions=on,condition_fanout_filter=on,derived_merge=on,use_invisible_indexes=off,skip_scan=on,hash_join=on,subquery_to_derived=off,prefer_ordering_index=on,hypergraph_optimizer=off,derived_condition_pushdown=on
optimizer_trace
enabled=off,one_line=off
enabled=off,one_line=off
optimizer_trace_features
greedy_search=on,range_optimizer=on,dynamic_range=on,repeated_subselect=on
greedy_search=on,range_optimizer=on,dynamic_range=on,repeated_subselect=on
optimizer_trace_limit
1
1
optimizer_trace_max_mem_size
1048576
1048576
optimizer_trace_offset
-1
-1
parser_max_mem_size
1.84467E+19
1.84467E+19
performance_schema
ON
ON
performance_schema_accounts_size
-1
-1
performance_schema_digests_size
10000
10000
performance_schema_events_stages_history_long_size
10000
10000
performance_schema_events_stages_history_size
10
10
performance_schema_events_statements_history_long_size
10000
10000
performance_schema_events_statements_history_size
10
10
performance_schema_events_transactions_history_long_size
10000
10000
performance_schema_events_transactions_history_size
10
10
performance_schema_events_waits_history_long_size
10000
10000
performance_schema_events_waits_history_size
10
10
performance_schema_hosts_size
-1
-1
performance_schema_max_cond_classes
100
100
performance_schema_max_cond_instances
-1
-1
performance_schema_max_digest_length
1024
1024
performance_schema_max_file_classes
80
80
performance_schema_max_file_handles
32768
32768
performance_schema_max_file_instances
-1
-1
performance_schema_max_index_stat
-1
-1
performance_schema_max_memory_classes
450
450
performance_schema_max_metadata_locks
-1
-1
performance_schema_max_mutex_classes
300
300
performance_schema_max_mutex_instances
-1
-1
performance_schema_max_prepared_statements_instances
-1
-1
performance_schema_max_program_instances
-1
-1
performance_schema_max_rwlock_classes
60
60
performance_schema_max_rwlock_instances
-1
-1
performance_schema_max_socket_classes
10
10
performance_schema_max_socket_instances
-1
-1
performance_schema_max_sql_text_length
1024
1024
performance_schema_max_stage_classes
175
175
performance_schema_max_statement_classes
218
218
performance_schema_max_statement_stack
10
10
performance_schema_max_table_handles
-1
-1
performance_schema_max_table_instances
-1
-1
performance_schema_max_table_lock_stat
-1
-1
performance_schema_max_thread_classes
100
100
performance_schema_max_thread_instances
-1
-1
performance_schema_session_connect_attrs_size
512
512
performance_schema_setup_actors_size
-1
-1
performance_schema_setup_objects_size
-1
-1
performance_schema_users_size
-1
-1
port
3306
3306
preload_buffer_size
32768
32768
profiling
ON
ON
profiling_history_size
15
15
protocol_version
10
10
proxy_user
pseudo_slave_mode
OFF
OFF
pseudo_thread_id
13855
13
query_alloc_block_size
8192
8192
query_cache_limit
not_defined
not_defined
query_cache_min_res_unit
not_defined
not_defined
query_cache_size
not_defined
not_defined
query_cache_type
not_defined
not_defined
query_cache_wlock_invalidate
not_defined
not_defined
query_prealloc_size
8192
8192
rand_seed1
0
0
rand_seed2
0
0
range_alloc_block_size
4096
4096
range_optimizer_max_mem_size
8388608
8388608
rbr_exec_mode
STRICT
STRICT
read_buffer_size
131072
131072
read_only
OFF
OFF
read_rnd_buffer_size
262144
262144
relay_log
ns515408-relay-bin
093b8c5ba5be-relay-bin
relay_log_basename
/var/lib/mysql/ns515408-relay-bin
/var/lib/mysql/093b8c5ba5be-relay-bin
relay_log_index
/var/lib/mysql/ns515408-relay-bin.index
/var/lib/mysql/093b8c5ba5be-relay-bin.index
relay_log_info_file
relay-log.info
relay-log.info
relay_log_info_repository
TABLE
TABLE
relay_log_purge
ON
ON
relay_log_recovery
OFF
OFF
relay_log_space_limit
0
0
replication_optimize_for_static_plugin_config
not_defined
not_defined
replication_sender_observe_commit_only
not_defined
not_defined
rpl_stop_slave_timeout
31536000
31536000
secure_auth
not_defined
not_defined
secure_file_priv
/var/lib/mysql-files/
NULL
server_id
1
1
server_id_bits
32
32
server_uuid
96c5e0e3-73bc-11eb-873b-a4bf011f053a
c16243c8-1f86-11eb-ab83-0242ac110002
session_track_gtids
OFF
OFF
session_track_schema
ON
ON
session_track_state_change
OFF
OFF
session_track_system_variables
time_zone,autocommit,character_set_client,character_set_results,character_set_connection
time_zone,autocommit,character_set_client,character_set_results,character_set_connection
session_track_transaction_info
OFF
OFF
sha256_password_auto_generate_rsa_keys
ON
ON
sha256_password_private_key_path
private_key.pem
private_key.pem
sha256_password_proxy_users
OFF
OFF
sha256_password_public_key_path
public_key.pem
public_key.pem
show_compatibility_56
not_defined
not_defined
show_create_table_verbosity
OFF
OFF
show_old_temporals
OFF
OFF
skip_external_locking
ON
ON
skip_name_resolve
OFF
ON
skip_networking
OFF
OFF
skip_show_database
OFF
OFF
slave_allow_batching
OFF
OFF
slave_checkpoint_group
512
512
slave_checkpoint_period
300
300
slave_compressed_protocol
OFF
OFF
slave_exec_mode
STRICT
STRICT
slave_load_tmpdir
/var/tmp
/tmp
slave_max_allowed_packet
1073741824
1073741824
slave_net_timeout
60
60
slave_parallel_type
DATABASE
DATABASE
slave_parallel_workers
0
0
slave_pending_jobs_size_max
134217728
134217728
slave_preserve_commit_order
OFF
OFF
slave_rows_search_algorithms
INDEX_SCAN,HASH_SCAN
INDEX_SCAN,HASH_SCAN
slave_skip_errors
OFF
OFF
slave_sql_verify_checksum
ON
ON
slave_transaction_retries
10
10
slave_type_conversions
slow_launch_time
2
2
slow_query_log
OFF
OFF
slow_query_log_file
/var/lib/mysql/ns515408-slow.log
/var/lib/mysql/093b8c5ba5be-slow.log
socket
/var/lib/mysql/mysql.sock
/var/run/mysqld/mysqld.sock
sort_buffer_size
262144
262144
sql_auto_is_null
OFF
OFF
sql_big_selects
ON
ON
sql_buffer_result
OFF
OFF
sql_log_bin
OFF
OFF
sql_log_off
OFF
OFF
sql_mode
ONLY_FULL_GROUP_BY,STRICT_TRANS_TABLES,ERROR_FOR_DIVISION_BY_ZERO,NO_ENGINE_SUBSTITUTION
ONLY_FULL_GROUP_BY,STRICT_TRANS_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,NO_ENGINE_SUBSTITUTION
sql_notes
ON
ON
sql_quote_show_create
ON
ON
sql_safe_updates
OFF
OFF
sql_select_limit
1.84467E+19
1.84467E+19
sql_slave_skip_counter
0
0
stored_program_cache
256
256
super_read_only
OFF
OFF
sync_binlog
1
1
sync_frm
not_defined
not_defined
sync_master_info
10000
10000
sync_relay_log
10000
10000
sync_relay_log_info
10000
10000
system_time_zone
UTC
UTC
table_definition_cache
2000
2000
table_open_cache
4000
4000
table_open_cache_instances
16
16
thread_cache_size
9
9
thread_handling
one-thread-per-connection
one-thread-per-connection
thread_stack
286720
286720
time_format
not_defined
not_defined
time_zone
SYSTEM
SYSTEM
timestamp
1620141370
1620235604
tls_version
TLSv1,TLSv1.1,TLSv1.2,TLSv1.3
TLSv1,TLSv1.1,TLSv1.2,TLSv1.3
tmp_table_size
16777216
16777216
tmpdir
/var/tmp
/tmp
transaction_alloc_block_size
8192
8192
transaction_allow_batching
OFF
OFF
transaction_isolation
REPEATABLE-READ
REPEATABLE-READ
transaction_prealloc_size
4096
4096
transaction_read_only
OFF
OFF
transaction_write_set_extraction
XXHASH64
XXHASH64
tx_isolation
not_defined
not_defined
tx_read_only
not_defined
not_defined
unique_checks
ON
ON
updatable_views_with_limit
YES
YES
version
8.0.21
8.0.22
version_comment
Source distribution
MySQL Community Server - GPL
version_compile_machine
x86_64
x86_64
version_compile_os
Linux
Linux
wait_timeout
28800
28800
warning_count
0
0
activate_all_roles_on_login
OFF
OFF
admin_address
binlog_encryption
OFF
OFF
binlog_expire_logs_seconds
2592000
2592000
binlog_rotate_encryption_master_key_at_startup
OFF
OFF
binlog_row_event_max_size
8192
8192
binlog_row_metadata
MINIMAL
MINIMAL
binlog_row_value_options
binlog_transaction_compression
OFF
OFF
binlog_transaction_compression_level_zstd
3
3
caching_sha2_password_auto_generate_rsa_keys
ON
ON
caching_sha2_password_private_key_path
private_key.pem
private_key.pem
caching_sha2_password_public_key_path
public_key.pem
public_key.pem
create_admin_listener_thread
OFF
OFF
cte_max_recursion_depth
1000
1000
default_collation_for_utf8mb4
utf8mb4_0900_ai_ci
utf8mb4_0900_ai_ci
default_table_encryption
OFF
OFF
generated_random_password_length
20
20
group_replication_consistency
EVENTUAL
EVENTUAL
gtid_executed
histogram_generation_max_mem_size
20000000
20000000
immediate_server_version
999999
999999
information_schema_stats_expiry
86400
86400
innodb_buffer_pool_in_core_file
ON
ON
innodb_dedicated_server
OFF
OFF
innodb_directories
innodb_doublewrite_batch_size
0
0
innodb_doublewrite_dir
innodb_doublewrite_files
2
2
innodb_doublewrite_pages
4
4
innodb_fsync_threshold
0
0
innodb_idle_flush_pct
100
100
innodb_log_spin_cpu_abs_lwm
80
80
innodb_log_spin_cpu_pct_hwm
50
50
innodb_log_wait_for_flush_spin_hwm
400
400
innodb_parallel_read_threads
4
4
innodb_print_ddl_logs
OFF
OFF
innodb_redo_log_archive_dirs
innodb_redo_log_encrypt
OFF
OFF
innodb_spin_wait_pause_multiplier
50
50
innodb_temp_tablespaces_dir
./#innodb_temp/
./#innodb_temp/
innodb_undo_log_encrypt
OFF
OFF
innodb_validate_tablespace_paths
ON
ON
internal_tmp_mem_storage_engine
TempTable
TempTable
log_error_services
log_filter_internal; log_sink_internal
log_filter_internal; log_sink_internal
log_error_suppression_list
log_raw
OFF
OFF
log_slow_extra
OFF
OFF
mandatory_roles
mysqlx_bind_address
*
*
mysqlx_compression_algorithms
DEFLATE_STREAM,LZ4_MESSAGE,ZSTD_STREAM
DEFLATE_STREAM,LZ4_MESSAGE,ZSTD_STREAM
mysqlx_connect_timeout
30
30
mysqlx_deflate_default_compression_level
3
3
mysqlx_deflate_max_client_compression_level
5
5
mysqlx_document_id_unique_prefix
0
0
mysqlx_enable_hello_notice
ON
ON
mysqlx_idle_worker_thread_timeout
60
60
mysqlx_interactive_timeout
28800
28800
mysqlx_lz4_default_compression_level
2
2
mysqlx_lz4_max_client_compression_level
8
8
mysqlx_max_allowed_packet
67108864
67108864
mysqlx_max_connections
100
100
mysqlx_min_worker_threads
2
2
mysqlx_port
33060
33060
mysqlx_port_open_timeout
0
0
mysqlx_read_timeout
30
30
mysqlx_socket
/var/lib/mysql/mysqlx.sock
/var/run/mysqld/mysqlx.sock
mysqlx_ssl_ca
mysqlx_ssl_capath
mysqlx_ssl_cert
mysqlx_ssl_cipher
mysqlx_ssl_crl
mysqlx_ssl_crlpath
mysqlx_ssl_key
mysqlx_wait_timeout
28800
28800
mysqlx_write_timeout
60
60
mysqlx_zstd_default_compression_level
3
3
mysqlx_zstd_max_client_compression_level
11
11
original_commit_timestamp
3.60288E+16
3.60288E+16
original_server_version
999999
999999
partial_revokes
OFF
OFF
password_history
0
0
password_require_current
OFF
OFF
password_reuse_interval
0
0
performance_schema_error_size
4772
4808
performance_schema_max_digest_sample_age
60
60
persist_only_admin_x509_subject
persisted_globals_load
ON
ON
print_identified_with_as_hex
OFF
OFF
protocol_compression_algorithms
zlib,zstd,uncompressed
zlib,zstd,uncompressed
regexp_stack_limit
8000000
8000000
regexp_time_limit
32
32
require_row_format
OFF
OFF
resultset_metadata
FULL
FULL
rpl_read_size
8192
8192
schema_definition_cache
256
256
secondary_engine_cost_threshold
100000
100000
show_create_table_skip_secondary_engine
OFF
OFF
sql_require_primary_key
OFF
OFF
ssl_fips_mode
OFF
OFF
stored_program_definition_cache
256
256
table_encryption_privilege_check
OFF
OFF
tablespace_definition_cache
256
256
temptable_max_ram
1073741824
1073741824
temptable_use_mmap
ON
ON
tls_ciphersuites
use_secondary_engine
ON
ON
version_compile_zlib
1.2.11
1.2.11
windowing_use_high_precision
ON
ON
innodb_extend_and_initialize
only_in_51_8
ON
innodb_log_writer_threads
only_in_51_8
ON
performance_schema_show_processlist
only_in_51_8
OFF
select_into_buffer_size
only_in_51_8
131072
select_into_disk_sync
only_in_51_8
OFF
select_into_disk_sync_delay
only_in_51_8
0
We have a problem with peak loading while wring data to database
This is example of error
SQLSTATE[HY000]: General error: 1205 Lock wait timeout exceeded; try restarting transaction
SQL: insert intoresponses(participant_id,survey_id,question_id,question_option_id) values (2204017, 188131, 5, 23)
The table is quite big, 4.5 millions rows, per day at peak we write about 150-200 thosands rows to it.
MySQL version is 5.6.39
Table uses InnoDb engine
MySQL instance configuration at RDS (Amazon service)
Instance class db.t2.xlarge
vCPU 4
RAM 16 GB
Table structure
id int(10) unsigned Auto Increment
participant_id int(10) unsigned
survey_id int(10) unsigned
question_id int(10) unsigned
question_option_id int(10) unsigned NULL
response_answer text NULL
created_at timestamp NULL
updated_at timestamp NULL
Indexes
PRIMARY id
INDEX participant_id
INDEX survey_id
INDEX question_id
INDEX question_option_id
Foreign keys
participant_id -> participants(id)
question_id -> questions(id)
question_option_id -> question_options(id)
survey_id -> surveys(id)
InnoDB Settings
innodb_adaptive_flushing ON
innodb_adaptive_flushing_lwm 10
innodb_adaptive_hash_index ON
innodb_adaptive_max_sleep_delay 150000
innodb_additional_mem_pool_size 8388608
innodb_api_bk_commit_interval 5
innodb_api_disable_rowlock OFF
innodb_api_enable_binlog OFF
innodb_api_enable_mdl OFF
innodb_api_trx_level 0
innodb_autoextend_increment 64
innodb_autoinc_lock_mode 1
innodb_buffer_pool_dump_at_shutdown OFF
innodb_buffer_pool_dump_now OFF
innodb_buffer_pool_filename ib_buffer_pool
innodb_buffer_pool_instances 8
innodb_buffer_pool_load_abort OFF
innodb_buffer_pool_load_at_startup OFF
innodb_buffer_pool_load_now OFF
innodb_buffer_pool_size 12465471488
innodb_change_buffer_max_size 25
innodb_change_buffering all
innodb_checksum_algorithm innodb
innodb_checksums ON
innodb_cmp_per_index_enabled OFF
innodb_commit_concurrency 0
innodb_compression_failure_threshold_pct 5
innodb_compression_level 6
innodb_compression_pad_pct_max 50
innodb_concurrency_tickets 5000
innodb_data_file_path ibdata1:12M:autoextend
innodb_data_home_dir /rdsdbdata/db/innodb
innodb_disable_sort_file_cache OFF
innodb_doublewrite ON
innodb_fast_shutdown 1
innodb_file_format Antelope
innodb_file_format_check ON
innodb_file_format_max Antelope
innodb_file_per_table ON
innodb_flush_log_at_timeout 1
innodb_flush_log_at_trx_commit 1
innodb_flush_method O_DIRECT
innodb_flush_neighbors 1
innodb_flushing_avg_loops 30
innodb_force_load_corrupted OFF
innodb_force_recovery 0
innodb_ft_aux_table
innodb_ft_cache_size 8000000
innodb_ft_enable_diag_print OFF
innodb_ft_enable_stopword ON
innodb_ft_max_token_size 84
innodb_ft_min_token_size 3
innodb_ft_num_word_optimize 2000
innodb_ft_result_cache_limit 2000000000
innodb_ft_server_stopword_table
innodb_ft_sort_pll_degree 2
innodb_ft_total_cache_size 640000000
innodb_ft_user_stopword_table
innodb_io_capacity 200
innodb_io_capacity_max 2000
innodb_large_prefix OFF
innodb_lock_wait_timeout 50
innodb_locks_unsafe_for_binlog OFF
innodb_log_buffer_size 8388608
innodb_log_compressed_pages ON
innodb_log_file_size 134217728
innodb_log_files_in_group 2
innodb_log_group_home_dir /rdsdbdata/log/innodb
innodb_lru_scan_depth 1024
innodb_max_dirty_pages_pct 75
innodb_max_dirty_pages_pct_lwm 0
innodb_max_purge_lag 0
innodb_max_purge_lag_delay 0
innodb_mirrored_log_groups 1
innodb_monitor_disable
innodb_monitor_enable
innodb_monitor_reset
innodb_monitor_reset_all
innodb_numa_interleave OFF
innodb_old_blocks_pct 37
innodb_old_blocks_time 1000
innodb_online_alter_log_max_size 134217728
innodb_open_files 2000
innodb_optimize_fulltext_only OFF
innodb_page_size 16384
innodb_print_all_deadlocks OFF
innodb_purge_batch_size 300
innodb_purge_threads 1
innodb_random_read_ahead OFF
innodb_read_ahead_threshold 56
innodb_read_io_threads 4
innodb_read_only OFF
innodb_replication_delay 0
innodb_rollback_on_timeout OFF
innodb_rollback_segments 128
innodb_sort_buffer_size 1048576
innodb_spin_wait_delay 6
innodb_stats_auto_recalc ON
innodb_stats_include_delete_marked OFF
innodb_stats_method nulls_equal
innodb_stats_on_metadata OFF
innodb_stats_persistent ON
innodb_stats_persistent_sample_pages 20
innodb_stats_sample_pages 8
innodb_stats_transient_sample_pages 8
innodb_status_output OFF
innodb_status_output_locks OFF
innodb_strict_mode OFF
innodb_support_xa ON
innodb_sync_array_size 1
innodb_sync_spin_loops 30
innodb_table_locks ON
innodb_thread_concurrency 0
innodb_thread_sleep_delay 10000
innodb_tmpdir
innodb_undo_directory .
innodb_undo_logs 128
innodb_undo_tablespaces 0
innodb_use_native_aio ON
innodb_use_sys_malloc ON
innodb_version 5.6.39
innodb_write_io_threads 4
I think increasing target & max IOPS above the defaults to avoid excessive SSD wear also disable proactively flushing pages could help
innodb_adaptive_flushing_lwm = 0
Innodb_flush_neighbors = off
innodb_io_capacity=1000
innodb_io_capacity_max=2500
Reference:
https://www.percona.com/blog/2019/12/18/give-love-to-your-ssds-reduce-innodb_io_capacity_max/
I am currently assisting with integration testing for migrating an application that currently runs against a mysql 5.5 DB to mysql 5.7. We have run into issues related to functionality that was optimised in mysql 5.5 to run about 30% faster by having certain parts be encapsulated in a read only transaction now throwing the below error:
Error Code: 1792 Cannot execute statement in a READ ONLY transaction.
I have scoured the internet and havent found any solutions. I know there is a related limitation added in mysql 5.6 documentation that mentions:
"In read-only mode, it remains possible to change tables created with
the TEMPORARY keyword using DML statements. Changes made with DDL
statements are not permitted, just as with permanent tables."
This presents a major limitation to us as some of these read only transactions call stored procedures that create temporary tables and it is not feasible for us to rewrite the application as this read only optimisation is widespread.
Essentially we are hoping to find a work around or a configuration we can tweak to allow the below example to not give a read only error.
START TRANSACTION READ only;
CREATE TEMPORARY TABLE IF NOT EXISTS table1 (account_id INT(10) UNSIGNED);
ROLLBACK;
Creating the table beforehand is not an option, mysql server session variables below if required. Any help would be much appreciated!
Session Variables:
Variable_name Value
auto_increment_increment 1
auto_increment_offset 1
autocommit ON
automatic_sp_privileges ON
avoid_temporal_upgrade OFF
back_log 80
basedir C:\\Program Files\\MySQL\\MySQL Server 5.7\\
big_tables OFF
bind_address *
binlog_cache_size 32768
binlog_checksum CRC32
binlog_direct_non_transactional_updates OFF
binlog_error_action ABORT_SERVER
binlog_format ROW
binlog_group_commit_sync_delay 0
binlog_group_commit_sync_no_delay_count 0
binlog_gtid_simple_recovery ON
binlog_max_flush_queue_time 0
binlog_order_commits ON
binlog_row_image FULL
binlog_rows_query_log_events OFF
binlog_stmt_cache_size 32768
binlog_transaction_dependency_history_size 25000
binlog_transaction_dependency_tracking COMMIT_ORDER
block_encryption_mode aes-128-ecb
bulk_insert_buffer_size 8388608
character_set_client utf8
character_set_connection utf8
character_set_database utf8
character_set_filesystem binary
character_set_results utf8
character_set_server utf8
character_set_system utf8
character_sets_dir C:\\Program Files\\MySQL\\MySQL Server 5.7\\share\\charsets\\
check_proxy_users OFF
collation_connection utf8_general_ci
collation_database utf8_general_ci
collation_server utf8_general_ci
completion_type NO_CHAIN
concurrent_insert AUTO
connect_timeout 10
core_file OFF
datadir C:\\ProgramData\\MySQL\\MySQL Server 5.7\\Data\\
date_format %Y-%m-%d
datetime_format %Y-%m-%d %H:%i:%s
default_authentication_plugin mysql_native_password
default_password_lifetime 0
default_storage_engine InnoDB
default_tmp_storage_engine InnoDB
default_week_format 0
delay_key_write ON
delayed_insert_limit 100
delayed_insert_timeout 300
delayed_queue_size 1000
disabled_storage_engines
disconnect_on_expired_password ON
div_precision_increment 4
end_markers_in_json OFF
enforce_gtid_consistency OFF
eq_range_index_dive_limit 200
error_count 0
event_scheduler OFF
expire_logs_days 0
explicit_defaults_for_timestamp OFF
external_user
flush OFF
flush_time 0
foreign_key_checks ON
ft_boolean_syntax "+ -><()~*:""""&|"
ft_max_word_len 84
ft_min_word_len 4
ft_query_expansion_limit 20
ft_stopword_file (built-in)
general_log OFF
general_log_file DESKTOP-11VS9FN.log
group_concat_max_len 1024
gtid_executed_compression_period 1000
gtid_mode OFF
gtid_next AUTOMATIC
gtid_owned
gtid_purged
have_compress YES
have_crypt NO
have_dynamic_loading YES
have_geometry YES
have_openssl DISABLED
have_profiling YES
have_query_cache YES
have_rtree_keys YES
have_ssl DISABLED
have_statement_timeout YES
have_symlink YES
host_cache_size 279
hostname BLUEOA-LP005
identity 0
ignore_builtin_innodb OFF
ignore_db_dirs
init_connect
init_file
init_slave
innodb_adaptive_flushing ON
innodb_adaptive_flushing_lwm 10
innodb_adaptive_hash_index ON
innodb_adaptive_hash_index_parts 8
innodb_adaptive_max_sleep_delay 150000
innodb_api_bk_commit_interval 5
innodb_api_disable_rowlock OFF
innodb_api_enable_binlog OFF
innodb_api_enable_mdl OFF
innodb_api_trx_level 0
innodb_autoextend_increment 64
innodb_autoinc_lock_mode 1
innodb_buffer_pool_chunk_size 8388608
innodb_buffer_pool_dump_at_shutdown ON
innodb_buffer_pool_dump_now OFF
innodb_buffer_pool_dump_pct 25
innodb_buffer_pool_filename ib_buffer_pool
innodb_buffer_pool_instances 1
innodb_buffer_pool_load_abort OFF
innodb_buffer_pool_load_at_startup ON
innodb_buffer_pool_load_now OFF
innodb_buffer_pool_size 8388608
innodb_change_buffer_max_size 25
innodb_change_buffering all
innodb_checksum_algorithm crc32
innodb_checksums ON
innodb_cmp_per_index_enabled OFF
innodb_commit_concurrency 0
innodb_compression_failure_threshold_pct 5
innodb_compression_level 6
innodb_compression_pad_pct_max 50
innodb_concurrency_tickets 5000
innodb_data_file_path ibdata1:12M:autoextend
innodb_data_home_dir
innodb_deadlock_detect ON
innodb_default_row_format dynamic
innodb_disable_sort_file_cache OFF
innodb_doublewrite ON
innodb_fast_shutdown 1
innodb_file_format Barracuda
innodb_file_format_check ON
innodb_file_format_max Barracuda
innodb_file_per_table ON
innodb_fill_factor 100
innodb_flush_log_at_timeout 1
innodb_flush_log_at_trx_commit 1
innodb_flush_method
innodb_flush_neighbors 1
innodb_flush_sync ON
innodb_flushing_avg_loops 30
innodb_force_load_corrupted OFF
innodb_force_recovery 0
innodb_ft_aux_table
innodb_ft_cache_size 8000000
innodb_ft_enable_diag_print OFF
innodb_ft_enable_stopword ON
innodb_ft_max_token_size 84
innodb_ft_min_token_size 3
innodb_ft_num_word_optimize 2000
innodb_ft_result_cache_limit 2000000000
innodb_ft_server_stopword_table
innodb_ft_sort_pll_degree 2
innodb_ft_total_cache_size 640000000
innodb_ft_user_stopword_table
innodb_io_capacity 200
innodb_io_capacity_max 2000
innodb_large_prefix ON
innodb_lock_wait_timeout 50
innodb_locks_unsafe_for_binlog OFF
innodb_log_buffer_size 1048576
innodb_log_checksums ON
innodb_log_compressed_pages ON
innodb_log_file_size 50331648
innodb_log_files_in_group 2
innodb_log_group_home_dir .\\
innodb_log_write_ahead_size 8192
innodb_lru_scan_depth 1024
innodb_max_dirty_pages_pct 75
innodb_max_dirty_pages_pct_lwm 0
innodb_max_purge_lag 0
innodb_max_purge_lag_delay 0
innodb_max_undo_log_size 1073741824
innodb_monitor_disable
innodb_monitor_enable
innodb_monitor_reset
innodb_monitor_reset_all
innodb_old_blocks_pct 37
innodb_old_blocks_time 1000
innodb_online_alter_log_max_size 134217728
innodb_open_files 300
innodb_optimize_fulltext_only OFF
innodb_page_cleaners 1
innodb_page_size 16384
innodb_print_all_deadlocks OFF
innodb_purge_batch_size 300
innodb_purge_rseg_truncate_frequency 128
innodb_purge_threads 4
innodb_random_read_ahead OFF
innodb_read_ahead_threshold 56
innodb_read_io_threads 4
innodb_read_only OFF
innodb_replication_delay 0
innodb_rollback_on_timeout OFF
innodb_rollback_segments 128
innodb_sort_buffer_size 1048576
innodb_spin_wait_delay 6
innodb_stats_auto_recalc ON
innodb_stats_include_delete_marked OFF
innodb_stats_method nulls_equal
innodb_stats_on_metadata OFF
innodb_stats_persistent ON
innodb_stats_persistent_sample_pages 20
innodb_stats_sample_pages 8
innodb_stats_transient_sample_pages 8
innodb_status_output OFF
innodb_status_output_locks OFF
innodb_strict_mode OFF
innodb_support_xa ON
innodb_sync_array_size 1
innodb_sync_spin_loops 30
innodb_table_locks ON
innodb_temp_data_file_path ibtmp1:12M:autoextend
innodb_thread_concurrency 9
innodb_thread_sleep_delay 0
innodb_tmpdir
innodb_undo_directory .\\
innodb_undo_log_truncate OFF
innodb_undo_logs 128
innodb_undo_tablespaces 0
innodb_use_native_aio ON
innodb_version 5.7.22
innodb_write_io_threads 4
insert_id 0
interactive_timeout 28800
internal_tmp_disk_storage_engine InnoDB
join_buffer_size 262144
keep_files_on_create OFF
key_buffer_size 8388608
key_cache_age_threshold 300
key_cache_block_size 1024
key_cache_division_limit 100
keyring_operations ON
large_files_support ON
large_page_size 0
large_pages OFF
last_insert_id 0
lc_messages en_US
lc_messages_dir C:\\Program Files\\MySQL\\MySQL Server 5.7\\share\\
lc_time_names en_US
license GPL
local_infile ON
lock_wait_timeout 31536000
log_bin OFF
log_bin_basename
log_bin_index
log_bin_trust_function_creators OFF
log_bin_use_v1_row_events OFF
log_builtin_as_identified_by_password OFF
log_error .\\DESKTOP-11VS9FN.err
log_error_verbosity 3
log_output FILE
log_queries_not_using_indexes OFF
log_slave_updates OFF
log_slow_admin_statements OFF
log_slow_slave_statements OFF
log_statements_unsafe_for_binlog ON
log_syslog ON
log_syslog_tag
log_throttle_queries_not_using_indexes 0
log_timestamps UTC
log_warnings 2
long_query_time 10
low_priority_updates OFF
lower_case_file_system ON
lower_case_table_names 1
master_info_repository FILE
master_verify_checksum OFF
max_allowed_packet 173741056
max_binlog_cache_size 1.84467E+19
max_binlog_size 1073741824
max_binlog_stmt_cache_size 1.84467E+19
max_connect_errors 100
max_connections 151
max_delayed_threads 20
max_digest_length 1024
max_error_count 64
max_execution_time 0
max_heap_table_size 70254592
max_insert_delayed_threads 20
max_join_size 1.84467E+19
max_length_for_sort_data 1024
max_points_in_geometry 65536
max_prepared_stmt_count 16382
max_relay_log_size 0
max_seeks_for_key 4294967295
max_sort_length 1024
max_sp_recursion_depth 0
max_tmp_tables 32
max_user_connections 0
max_write_lock_count 4294967295
metadata_locks_cache_size 1024
metadata_locks_hash_instances 8
min_examined_row_limit 0
multi_range_count 256
myisam_data_pointer_size 6
myisam_max_sort_file_size 1.07374E+11
myisam_mmap_size 1.84467E+19
myisam_recover_options OFF
myisam_repair_threads 1
myisam_sort_buffer_size 8388608
myisam_stats_method nulls_unequal
myisam_use_mmap OFF
mysql_native_password_proxy_users OFF
named_pipe OFF
net_buffer_length 16384
net_read_timeout 30
net_retry_count 10
net_write_timeout 60
new OFF
ngram_token_size 2
offline_mode OFF
old OFF
old_alter_table OFF
old_passwords 0
open_files_limit 6209
optimizer_prune_level 1
optimizer_search_depth 62
optimizer_switch "index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,engine_condition_pushdown=on,index_condition_pushdown=on,mrr=on,mrr_cost_based=on,block_nested_loop=on,batched_key_access=off,materialization=on,semijoin=on,loosescan=on,firstmatch=on,duplicateweedout=on,subquery_materialization_cost_based=on,use_index_extensions=on,condition_fanout_filter=on,derived_merge=on"
optimizer_trace "enabled=off,one_line=off"
optimizer_trace_features "greedy_search=on,range_optimizer=on,dynamic_range=on,repeated_subselect=on"
optimizer_trace_limit 1
optimizer_trace_max_mem_size 16384
optimizer_trace_offset -1
parser_max_mem_size 1.84467E+19
performance_schema ON
performance_schema_accounts_size -1
performance_schema_digests_size 10000
performance_schema_events_stages_history_long_size 10000
performance_schema_events_stages_history_size 10
performance_schema_events_statements_history_long_size 10000
performance_schema_events_statements_history_size 10
performance_schema_events_transactions_history_long_size 10000
performance_schema_events_transactions_history_size 10
performance_schema_events_waits_history_long_size 10000
performance_schema_events_waits_history_size 10
performance_schema_hosts_size -1
performance_schema_max_cond_classes 80
performance_schema_max_cond_instances -1
performance_schema_max_digest_length 1024
performance_schema_max_file_classes 80
performance_schema_max_file_handles 32768
performance_schema_max_file_instances -1
performance_schema_max_index_stat -1
performance_schema_max_memory_classes 320
performance_schema_max_metadata_locks -1
performance_schema_max_mutex_classes 210
performance_schema_max_mutex_instances -1
performance_schema_max_prepared_statements_instances -1
performance_schema_max_program_instances -1
performance_schema_max_rwlock_classes 40
performance_schema_max_rwlock_instances -1
performance_schema_max_socket_classes 10
performance_schema_max_socket_instances -1
performance_schema_max_sql_text_length 1024
performance_schema_max_stage_classes 150
performance_schema_max_statement_classes 193
performance_schema_max_statement_stack 10
performance_schema_max_table_handles -1
performance_schema_max_table_instances -1
performance_schema_max_table_lock_stat -1
performance_schema_max_thread_classes 50
performance_schema_max_thread_instances -1
performance_schema_session_connect_attrs_size 512
performance_schema_setup_actors_size -1
performance_schema_setup_objects_size -1
performance_schema_users_size -1
pid_file C:\\ProgramData\\MySQL\\MySQL Server 5.7\\Data\\BLUEOA-LP005.pid
plugin_dir C:\\Program Files\\MySQL\\MySQL Server 5.7\\lib\\plugin\\
port 3306
preload_buffer_size 32768
profiling OFF
profiling_history_size 15
protocol_version 10
proxy_user
pseudo_slave_mode OFF
pseudo_thread_id 503
query_alloc_block_size 8192
query_cache_limit 1048576
query_cache_min_res_unit 4096
query_cache_size 1048576
query_cache_type OFF
query_cache_wlock_invalidate OFF
query_prealloc_size 8192
rand_seed1 0
rand_seed2 0
range_alloc_block_size 4096
range_optimizer_max_mem_size 8388608
rbr_exec_mode STRICT
read_buffer_size 8192
read_only OFF
read_rnd_buffer_size 1
relay_log
relay_log_basename C:\\ProgramData\\MySQL\\MySQL Server 5.7\\Data\\BLUEOA-LP005-relay-bin
relay_log_index C:\\ProgramData\\MySQL\\MySQL Server 5.7\\Data\\BLUEOA-LP005-relay-bin.index
relay_log_info_file relay-log.info
relay_log_info_repository FILE
relay_log_purge ON
relay_log_recovery OFF
relay_log_space_limit 0
report_host
report_password
report_port 3306
report_user
require_secure_transport OFF
rpl_stop_slave_timeout 31536000
secure_auth ON
secure_file_priv C:\\ProgramData\\MySQL\\MySQL Server 5.7\\Uploads\\
server_id 1
server_id_bits 32
server_uuid 215d32a3-5340-11e8-93c8-106530971942
session_track_gtids OFF
session_track_schema ON
session_track_state_change OFF
session_track_system_variables "time_zone,autocommit,character_set_client,character_set_results,character_set_connection"
session_track_transaction_info OFF
sha256_password_proxy_users OFF
shared_memory OFF
shared_memory_base_name MYSQL
show_compatibility_56 OFF
show_create_table_verbosity OFF
show_old_temporals OFF
skip_external_locking ON
skip_name_resolve OFF
skip_networking OFF
skip_show_database OFF
slave_allow_batching OFF
slave_checkpoint_group 512
slave_checkpoint_period 300
slave_compressed_protocol OFF
slave_exec_mode IDEMPOTENT
slave_load_tmpdir C:\\WINDOWS\\SERVIC~3\\NETWOR~1\\AppData\\Local\\Temp
slave_max_allowed_packet 1073741824
slave_net_timeout 60
slave_parallel_type DATABASE
slave_parallel_workers 0
slave_pending_jobs_size_max 16777216
slave_preserve_commit_order OFF
slave_rows_search_algorithms "TABLE_SCAN,INDEX_SCAN"
slave_skip_errors OFF
slave_sql_verify_checksum ON
slave_transaction_retries 10
slave_type_conversions
slow_launch_time 2
slow_query_log ON
slow_query_log_file DESKTOP-11VS9FN-slow.log
socket MySQL
sort_buffer_size 262144
sql_auto_is_null OFF
sql_big_selects ON
sql_buffer_result OFF
sql_log_bin ON
sql_log_off OFF
sql_mode
sql_notes ON
sql_quote_show_create ON
sql_safe_updates OFF
sql_select_limit 1.84467E+19
sql_slave_skip_counter 0
sql_warnings OFF
ssl_ca
ssl_capath
ssl_cert
ssl_cipher
ssl_crl
ssl_crlpath
ssl_key
stored_program_cache 256
super_read_only OFF
sync_binlog 1
sync_frm ON
sync_master_info 10000
sync_relay_log 10000
sync_relay_log_info 10000
system_time_zone AUS Eastern Standard Time
table_definition_cache 1400
table_open_cache 2000
table_open_cache_instances 16
thread_cache_size 10
thread_handling one-thread-per-connection
thread_stack 262144
time_format %H:%i:%s
time_zone SYSTEM
timestamp 1527639658
tls_version "TLSv1,TLSv1.1"
tmp_table_size 15728640
tmpdir C:\\WINDOWS\\SERVIC~3\\NETWOR~1\\AppData\\Local\\Temp
transaction_alloc_block_size 8192
transaction_allow_batching OFF
transaction_isolation REPEATABLE-READ
transaction_prealloc_size 4096
transaction_read_only OFF
transaction_write_set_extraction OFF
tx_isolation REPEATABLE-READ
tx_read_only OFF
unique_checks ON
updatable_views_with_limit YES
version 5.7.22-log
version_comment MySQL Community Server (GPL)
version_compile_machine x86_64
version_compile_os Win64
wait_timeout 28800
warning_count 0
*/ EDIT 1 */ We have decided to abandon the read only optimisations that were put in place for MySql 5.5 as we were unable to resolve this roadblock.
I know this is an old question but we have ended up finding a very easy solution to this problem.
The solution was simple as to not execute the stored procedure calls within a transaction. The only extra care we had to make was to ensure none of the affected stored procedures would make data change but apart from that the solutions was as simple as that.
I have a large database application that uses mysqls query cache ability. I'd like to inspect the contents of the query cache to identify entries that should not get cached (e.g. by using SELECT SQL_NO_CACHE...) because I want to optimize my application.
Is there a way to get a list of cached entries from the query cache?
-- Edit: additional info as suggested by Rick ---
1) RAM size is 16GB
2) show variables;
auto_increment_increment 1
auto_increment_offset 1
autocommit ON
automatic_sp_privileges ON
back_log 50
basedir /usr/
big_tables OFF
binlog_cache_size 32768
binlog_direct_non_transactional_updates OFF
binlog_format STATEMENT
bulk_insert_buffer_size 4294967296
character_set_client utf8
character_set_connection utf8
character_set_database utf8
character_set_filesystem binary
character_set_results utf8
character_set_server utf8
character_set_system utf8
character_sets_dir /usr/share/mysql/charsets/
collation_connection utf8_general_ci
collation_database utf8_general_ci
collation_server utf8_general_ci
completion_type 0
concurrent_insert 1
connect_timeout 10
datadir /var/lib/mysql/
date_format %Y-%m-%d
datetime_format %Y-%m-%d %H:%i:%s
default_week_format 0
delay_key_write ON
delayed_insert_limit 100
delayed_insert_timeout 300
delayed_queue_size 1000
div_precision_increment 4
engine_condition_pushdown ON
error_count 0
event_scheduler ON
expire_logs_days 0
flush OFF
flush_time 0
foreign_key_checks ON
ft_boolean_syntax + -><()~*:""&|
ft_max_word_len 84
ft_min_word_len 4
ft_query_expansion_limit 20
ft_stopword_file (built-in)
general_log OFF
general_log_file /var/run/mysqld/mysqld.log
group_concat_max_len 1024
have_community_features YES
have_compress YES
have_crypt YES
have_csv YES
have_dynamic_loading YES
have_geometry YES
have_innodb YES
have_ndbcluster NO
have_openssl DISABLED
have_partitioning NO
have_query_cache YES
have_rtree_keys YES
have_ssl DISABLED
have_symlink YES
hostname www2
identity 0
ignore_builtin_innodb ON
init_connect
init_file
init_slave
innodb_adaptive_flushing ON
innodb_adaptive_hash_index ON
innodb_additional_mem_pool_size 67108864
innodb_autoextend_increment 8
innodb_autoinc_lock_mode 1
innodb_buffer_pool_size 1073741824
innodb_change_buffering inserts
innodb_checksums ON
innodb_commit_concurrency 0
innodb_concurrency_tickets 500
innodb_data_file_path ibdata1:10M:autoextend
innodb_data_home_dir
innodb_doublewrite ON
innodb_fast_shutdown 1
innodb_file_format Antelope
innodb_file_format_check Barracuda
innodb_file_per_table ON
innodb_flush_log_at_trx_commit 1
innodb_flush_method
innodb_force_recovery 0
innodb_io_capacity 200
innodb_lock_wait_timeout 50
innodb_locks_unsafe_for_binlog OFF
innodb_log_buffer_size 67108864
innodb_log_file_size 268435456
innodb_log_files_in_group 2
innodb_log_group_home_dir ./
innodb_max_dirty_pages_pct 75
innodb_max_purge_lag 0
innodb_mirrored_log_groups 1
innodb_old_blocks_pct 37
innodb_old_blocks_time 0
innodb_open_files 300
innodb_read_ahead_threshold 56
innodb_read_io_threads 4
innodb_replication_delay 0
innodb_rollback_on_timeout OFF
innodb_spin_wait_delay 6
innodb_stats_method nulls_equal
innodb_stats_on_metadata ON
innodb_stats_sample_pages 8
innodb_strict_mode OFF
innodb_support_xa ON
innodb_sync_spin_loops 30
innodb_table_locks ON
innodb_thread_concurrency 0
innodb_thread_sleep_delay 10000
innodb_use_sys_malloc ON
innodb_version 1.0.15
innodb_write_io_threads 4
insert_id 0
interactive_timeout 30
join_buffer_size 2147483648
keep_files_on_create OFF
key_buffer_size 4294967296
key_cache_age_threshold 300
key_cache_block_size 1024
key_cache_division_limit 100
language /usr/share/mysql/english/
large_files_support ON
large_page_size 0
large_pages OFF
last_insert_id 0
lc_time_names en_US
license GPL
local_infile OFF
locked_in_memory OFF
log OFF
log_bin OFF
log_bin_trust_function_creators OFF
log_bin_trust_routine_creators OFF
log_error /var/log/mysql/mysqld.err
log_output FILE
log_queries_not_using_indexes OFF
log_slave_updates OFF
log_slow_queries ON
log_warnings 2
long_query_time 2.000000
low_priority_updates OFF
lower_case_file_system OFF
lower_case_table_names 0
max_allowed_packet 268435456
max_binlog_cache_size 18446744073709547520
max_binlog_size 1073741824
max_connect_errors 100000
max_connections 120
max_delayed_threads 20
max_error_count 64
max_heap_table_size 268435456
max_insert_delayed_threads 20
max_join_size 18446744073709551615
max_length_for_sort_data 1024
max_prepared_stmt_count 16382
max_relay_log_size 0
max_seeks_for_key 18446744073709551615
max_sort_length 1024
max_sp_recursion_depth 0
max_tmp_tables 32
max_user_connections 0
max_write_lock_count 18446744073709551615
min_examined_row_limit 0
multi_range_count 256
myisam_data_pointer_size 6
myisam_max_sort_file_size 9223372036853727232
myisam_mmap_size 18446744073709551615
myisam_recover_options OFF
myisam_repair_threads 1
myisam_sort_buffer_size 4294967296
myisam_stats_method nulls_unequal
myisam_use_mmap OFF
net_buffer_length 1048576
net_read_timeout 30
net_retry_count 10
net_write_timeout 60
new OFF
old OFF
old_alter_table OFF
old_passwords OFF
open_files_limit 8322
optimizer_prune_level 1
optimizer_search_depth 62
optimizer_switch index_merge=on,index_merge_union=on,index_merge_so...
pid_file /var/run/mysqld/mysqld.pid
plugin_dir /usr/lib64/mysql/plugin
port 3306
preload_buffer_size 32768
protocol_version 10
pseudo_thread_id 3468076
query_alloc_block_size 8192
query_cache_limit 4294967296
query_cache_min_res_unit 4096
query_cache_size 1073741824
query_cache_type ON
query_cache_wlock_invalidate OFF
query_prealloc_size 8192
rand_seed1
rand_seed2
range_alloc_block_size 4096
read_buffer_size 131072
read_only OFF
read_rnd_buffer_size 8388608
relay_log
relay_log_index
relay_log_info_file relay-log.info
relay_log_purge ON
relay_log_space_limit 0
report_host
report_password
report_port 3306
report_user
rpl_recovery_rank 0
secure_auth OFF
secure_file_priv
server_id 0
skip_external_locking ON
skip_name_resolve OFF
skip_networking OFF
skip_show_database OFF
slave_compressed_protocol OFF
slave_exec_mode STRICT
slave_load_tmpdir /var/tmp/
slave_net_timeout 3600
slave_skip_errors OFF
slave_transaction_retries 10
slow_launch_time 2
slow_query_log ON
slow_query_log_file /var/log/mysql/slowqueries.log
socket /var/run/mysqld/mysqld.sock
sort_buffer_size 262144
sql_auto_is_null ON
sql_big_selects ON
sql_big_tables OFF
sql_buffer_result OFF
sql_log_bin ON
sql_log_off OFF
sql_log_update ON
sql_low_priority_updates OFF
sql_max_join_size 18446744073709551615
sql_mode
sql_notes ON
sql_quote_show_create ON
sql_safe_updates OFF
sql_select_limit 18446744073709551615
sql_slave_skip_counter
sql_warnings OFF
ssl_ca
ssl_capath
ssl_cert
ssl_cipher
ssl_key
storage_engine MyISAM
sync_binlog 0
sync_frm ON
system_time_zone CEST
table_definition_cache 256
table_lock_wait_timeout 50
table_open_cache 4096
table_type MyISAM
thread_cache_size 16
thread_handling one-thread-per-connection
thread_stack 262144
time_format %H:%i:%s
time_zone SYSTEM
timed_mutexes OFF
timestamp 1449649902
tmp_table_size 2147483648
tmpdir /var/tmp/
transaction_alloc_block_size 8192
transaction_prealloc_size 4096
tx_isolation READ-COMMITTED
unique_checks ON
updatable_views_with_limit YES
version 5.1.56-log
version_comment Gentoo Linux mysql-5.1.56
version_compile_machine x86_64
version_compile_os pc-linux-gnu
wait_timeout 30
warning_count 0
3) show global status;
Aborted_clients 2094
Aborted_connects 4
Binlog_cache_disk_use 0
Binlog_cache_use 0
Bytes_received 171903979482
Bytes_sent 2515206884666
Com_admin_commands 339464
Com_assign_to_keycache 0
Com_alter_db 0
Com_alter_db_upgrade 0
Com_alter_event 0
Com_alter_function 0
Com_alter_procedure 0
Com_alter_server 0
Com_alter_table 340
Com_alter_tablespace 0
Com_analyze 0
Com_backup_table 0
Com_begin 5508
Com_binlog 0
Com_call_procedure 0
Com_change_db 3205067
Com_change_master 0
Com_check 2735
Com_checksum 0
Com_commit 5508
Com_create_db 1
Com_create_event 0
Com_create_function 0
Com_create_index 0
Com_create_procedure 0
Com_create_server 0
Com_create_table 148
Com_create_trigger 0
Com_create_udf 0
Com_create_user 0
Com_create_view 0
Com_dealloc_sql 2
Com_delete 10465739
Com_delete_multi 15995
Com_do 0
Com_drop_db 0
Com_drop_event 0
Com_drop_function 0
Com_drop_index 0
Com_drop_procedure 0
Com_drop_server 0
Com_drop_table 121
Com_drop_trigger 0
Com_drop_user 0
Com_drop_view 0
Com_empty_query 0
Com_execute_sql 2
Com_flush 342
Com_grant 2
Com_ha_close 0
Com_ha_open 0
Com_ha_read 0
Com_help 0
Com_insert 13583029
Com_insert_select 409
Com_install_plugin 0
Com_kill 0
Com_load 112042
Com_load_master_data 0
Com_load_master_table 0
Com_lock_tables 2068
Com_optimize 44
Com_preload_keys 0
Com_prepare_sql 2
Com_purge 0
Com_purge_before_date 0
Com_release_savepoint 0
Com_rename_table 0
Com_rename_user 0
Com_repair 2
Com_replace 489
Com_replace_select 0
Com_reset 0
Com_restore_table 0
Com_revoke 0
Com_revoke_all 0
Com_rollback 0
Com_rollback_to_savepoint 0
Com_savepoint 0
Com_select 115580660
Com_set_option 6028458
Com_show_authors 0
Com_show_binlog_events 0
Com_show_binlogs 11
Com_show_charsets 128
Com_show_collations 105
Com_show_column_types 0
Com_show_contributors 0
Com_show_create_db 10
Com_show_create_event 0
Com_show_create_func 0
Com_show_create_proc 0
Com_show_create_table 172358
Com_show_create_trigger 0
Com_show_databases 3956
Com_show_engine_logs 0
Com_show_engine_mutex 0
Com_show_engine_status 0
Com_show_events 0
Com_show_errors 0
Com_show_fields 839418
Com_show_function_status 0
Com_show_grants 11
Com_show_keys 3212
Com_show_master_status 0
Com_show_new_master 0
Com_show_open_tables 0
Com_show_plugins 0
Com_show_privileges 0
Com_show_procedure_status 0
Com_show_processlist 2
Com_show_profile 0
Com_show_profiles 0
Com_show_slave_hosts 0
Com_show_slave_status 0
Com_show_status 31038
Com_show_storage_engines 0
Com_show_table_status 207707
Com_show_tables 55951
Com_show_triggers 172340
Com_show_variables 803242
Com_show_warnings 0
Com_slave_start 0
Com_slave_stop 0
Com_stmt_close 627128
Com_stmt_execute 627128
Com_stmt_fetch 0
Com_stmt_prepare 627128
Com_stmt_reprepare 0
Com_stmt_reset 0
Com_stmt_send_long_data 0
Com_truncate 4722
Com_uninstall_plugin 0
Com_unlock_tables 2068
Com_update 1754805
Com_update_multi 1
Com_xa_commit 0
Com_xa_end 0
Com_xa_prepare 0
Com_xa_recover 0
Com_xa_rollback 0
Com_xa_start 0
Compression OFF
Connections 3467815
Created_tmp_disk_tables 2151917
Created_tmp_files 16517
Created_tmp_tables 3928177
Delayed_errors 0
Delayed_insert_threads 0
Delayed_writes 0
Flush_commands 1
Handler_commit 90276170
Handler_delete 9422032
Handler_discover 0
Handler_prepare 0
Handler_read_first 16985165
Handler_read_key 727922287
Handler_read_next 1575370461
Handler_read_prev 1178376
Handler_read_rnd 114364819
Handler_read_rnd_next 40377965120
Handler_rollback 326
Handler_savepoint 0
Handler_savepoint_rollback 0
Handler_update 33675553
Handler_write 137057026
Innodb_buffer_pool_pages_data 65003
Innodb_buffer_pool_pages_dirty 44
Innodb_buffer_pool_pages_flushed 29673396
Innodb_buffer_pool_pages_free 1
Innodb_buffer_pool_pages_misc 531
Innodb_buffer_pool_pages_total 65535
Innodb_buffer_pool_read_ahead 30144135
Innodb_buffer_pool_read_ahead_evicted 4691916
Innodb_buffer_pool_read_requests 5269018735
Innodb_buffer_pool_reads 15291059
Innodb_buffer_pool_wait_free 0
Innodb_buffer_pool_write_requests 196667000
Innodb_data_fsyncs 23264996
Innodb_data_pending_fsyncs 1
Innodb_data_pending_reads 0
Innodb_data_pending_writes 0
Innodb_data_read 745229520896
Innodb_data_reads 15847178
Innodb_data_writes 40737707
Innodb_data_written 1007283227136
Innodb_dblwr_pages_written 29673396
Innodb_dblwr_writes 727171
Innodb_have_atomic_builtins ON
Innodb_log_waits 0
Innodb_log_write_requests 61077243
Innodb_log_writes 17124326
Innodb_os_log_fsyncs 17565254
Innodb_os_log_pending_fsyncs 1
Innodb_os_log_pending_writes 0
Innodb_os_log_written 34746359296
Innodb_page_size 16384
Innodb_pages_created 893340
Innodb_pages_read 45485066
Innodb_pages_written 29673396
Innodb_row_lock_current_waits 0
Innodb_row_lock_time 75432
Innodb_row_lock_time_avg 23
Innodb_row_lock_time_max 10927
Innodb_row_lock_waits 3264
Innodb_rows_deleted 8649356
Innodb_rows_inserted 12313551
Innodb_rows_read 3890223870
Innodb_rows_updated 1493452
Key_blocks_not_flushed 0
Key_blocks_unused 3089739
Key_blocks_used 864035
Key_read_requests 1336336475
Key_reads 2772018
Key_write_requests 105012022
Key_writes 7764232
Last_query_cost 0.000000
Max_used_connections 69
Not_flushed_delayed_rows 0
Open_files 3006
Open_streams 0
Open_table_definitions 2265
Open_tables 3982
Opened_files 16841374
Opened_table_definitions 140845
Opened_tables 327918
Prepared_stmt_count 0
Qcache_free_blocks 56315
Qcache_free_memory 164432360
Qcache_hits 539352560
Qcache_inserts 111849501
Qcache_lowmem_prunes 54731591
Qcache_not_cached 3376334
Qcache_queries_in_cache 182383
Qcache_total_blocks 472186
Queries 697132110
Questions 695877858
Rpl_status NULL
Select_full_join 30156
Select_full_range_join 4732
Select_range 36065208
Select_range_check 76799
Select_scan 4058097
Slave_open_temp_tables 0
Slave_retried_transactions 0
Slave_running OFF
Slow_launch_threads 0
Slow_queries 13817
Sort_merge_passes 24563
Sort_range 2573998
Sort_rows 93946492
Sort_scan 1603263
Ssl_accept_renegotiates 0
Ssl_accepts 0
Ssl_callback_cache_hits 0
Ssl_cipher
Ssl_cipher_list
Ssl_client_connects 0
Ssl_connect_renegotiates 0
Ssl_ctx_verify_depth 0
Ssl_ctx_verify_mode 0
Ssl_default_timeout 0
Ssl_finished_accepts 0
Ssl_finished_connects 0
Ssl_session_cache_hits 0
Ssl_session_cache_misses 0
Ssl_session_cache_mode NONE
Ssl_session_cache_overflows 0
Ssl_session_cache_size 0
Ssl_session_cache_timeouts 0
Ssl_sessions_reused 0
Ssl_used_session_cache_entries 0
Ssl_verify_depth 0
Ssl_verify_mode 0
Ssl_version
Table_locks_immediate 157822475
Table_locks_waited 3513
Tc_log_max_pages_used 0
Tc_log_page_size 0
Tc_log_page_waits 0
Threads_cached 14
Threads_connected 3
Threads_created 2076
Threads_running 3
Uptime 4656056
Uptime_since_flush_status 4656056
You can know the amount of queries cached with
Show status like 'Qcache_queries_in_cache'
It will give you the amount of cached query in value column.
Only the hash of queries are stored in Qcache.
It is not possible to find which queries are actually cached.
One possible way to know it is to execute one of your query and to check changes of value column.
Observations:
Version: 5.1.56-log
16 GB of RAM
Uptime = 53d 21:20:56
You are not running on Windows.
Running 64-bit version
It appears that you are running both MyISAM and InnoDB.
The More Important Issues
key_buffer_size = 1500M
innodb_buffer_pool_size = 2000M
join_buffer_size = 1M -- This may be the most important
tmp_table_size = 256M -- Also important
query_cache_size = 50M -- as already mentioned
Use slowlog to find tmp tables spilling to disk
Three settings were especially high and could lead to paging/swapping. MySQL performance suffers terribly when paged/swapped. It is better to lower settings than to have that happen.
Details and Minor Issues
( innodb_buffer_pool_size / _ram ) = 1024M / 16384M = 6.2% -- % of RAM used for InnoDB buffer_pool
( Innodb_pages_written / Innodb_buffer_pool_write_requests ) = 29,673,396 / 196667000 = 15.1% -- Write requests that had to hit disk
-- Check innodb_buffer_pool_size
( Innodb_buffer_pool_read_ahead_evicted / Innodb_buffer_pool_read_ahead ) = 4,691,916 / 30144135 = 15.6% -- Utility of read_ahead.
-- Turn off read-ahead
( Innodb_os_log_written / (Uptime / 3600) / innodb_log_files_in_group / innodb_log_file_size ) = 34,746,359,296 / (4656056 / 3600) / 2 / 256M = 0.05 -- Ratio
( Uptime / 60 * innodb_log_file_size / Innodb_os_log_written ) = 4,656,056 / 60 * 256M / 34746359296 = 599 -- Minutes between InnoDB log rotations Beginning with 5.6.8, this can be changed dynamically; be sure to also change my.cnf.
-- (The recommendation of 60 minutes between rotations is somewhat arbitrary.)
( table_open_cache ) = 4,096 -- Number of table descriptors to cache
-- Several hundred is usually good.
( innodb_stats_on_metadata ) = ON -- Re-analyze table when touching stats.
-- ON is likely to slow down certain SHOWs and information_schema accesses.
( join_buffer_size ) = 2,147,483,648 = 2048MB -- 0-N per thread. May speed up JOINs (better to fix queries/indexes) (all engines) Used for index scan, range index scan, full table scan, each full JOIN, etc.
-- Use the default.
( join_buffer_size / _ram ) = 2,147,483,648 / 16384M = 12.5% -- 0-N per thread. May speed up JOINs (better to fix queries/indexes) (all engines) Used for index scan, range index scan, full table scan, each full JOIN, etc.
-- Decrease join_buffer_size to avoid memory pressure. Suggest less than 1% of RAM.
( min( tmp_table_size, max_heap_table_size ) / _ram ) = min( 2147483648, 256M ) / 16384M = 1.6% -- Percent of RAM to allocate when needing MEMORY table (per table), or temp table inside a SELECT (per temp table per some SELECTs). Too high may lead to swapping.
-- Decrease tmp_table_size and max_heap_table_size to, say, 1% of ram.
( (key_buffer_size - 1.2 * Key_blocks_used * 1024) / _ram ) = (4294967296 - 1.2 * 864035 * 1024) / 16384M = 18.8% -- Percent of RAM wasted in key_buffer.
-- Decrease key_buffer_size
( Key_blocks_used * 1024 / key_buffer_size ) = 864,035 * 1024 / 4294967296 = 20.6% -- Percent of key_buffer used . High-water-mark.
-- Lower key_buffer_size to avoid unnecessary memory usage.
( Key_writes / Key_write_requests ) = 7,764,232 / 105012022 = 7.4% -- key_buffer effectiveness for writes
( query_cache_size ) = 1024M -- Size of QC
-- Too small = not of much use. Too large = too much overhead. Recommend either 0 or no more than 50M.
( Created_tmp_disk_tables / (Created_tmp_disk_tables + Created_tmp_tables) ) = 2,151,917 / (2151917 + 3928177) = 35.4% -- Percent of temp tables that spilled to disk
( tmp_table_size ) = 2,147,483,648 = 2048MB -- Limit on size of MEMORY temp tables used to support a SELECT
-- Decrease tmp_table_size to avoid running out of RAM. Perhaps no more than 64M.
( back_log / max_connections ) = 50 / 120 = 41.7%
( max_connect_errors ) = 100,000 -- A small protection against hackers.
-- Perhaps no more than 200.
If you have a "large" db app, this implies that there are a lot of writes? Keep in mind that all QC entries for a table are purged when any write to that table occurs. That implies that you may not benefit from the QC.
Furthermore, that purging takes effort, thereby slowing down the system. For query_cache_size under, say, 50M, the cost is not big. But if you have a 1G QC, you will spend more time purging the QC than using it!
The actual queries in the QC cannot be discovered. Here is what might help: Put SQL_NO_CACHE on any queries involving table(s) that change frequently. Put SQL_CACHE on the rest. Then use DEMAND. Monitor STATUS LIKE Qc%.
If you would like an analysis of your system (QC and otherwise), provide (1) RAM size, (2) SHOW VARIABLES, (3) SHOW GLOBAL STATUS after the server has been up for at least a day.
I have a problem with a MySql server causing it to freeze all the time. In phpMyAdmin when this happens I can see in "Show Processlist" are lots of processes for User "unauthenticated user" with Command "connect" and State "reading from net".
The database is using InnoDB but I also use one table running MyISAM to be able to perform full text search. The server has about 4GB memory, less than 500MB in use.
I have been using the slow Query Log from MySql to find the queries not using indexes. I also think I need to do some changes / tweaking on the server variables.
I really could use some help here and therefore I'm posting both SHOW GLOBAL STATUS and SHOW GLOBAL VARIABLES here and maybe you can give me some ideas how I should set the variables?
Current SHOW GLOBAL STATUS:
Variable_name Value
Aborted_clients 730
Aborted_connects 35
Binlog_cache_disk_use 0
Binlog_cache_use 0
Binlog_stmt_cache_disk_use 0
Binlog_stmt_cache_use 0
Bytes_received 31558148370
Bytes_sent 556410688609
Com_admin_commands 4565
Com_assign_to_keycache 0
Com_alter_db 0
Com_alter_db_upgrade 0
Com_alter_event 0
Com_alter_function 0
Com_alter_procedure 0
Com_alter_server 0
Com_alter_table 3
Com_alter_tablespace 0
Com_analyze 0
Com_begin 168
Com_binlog 0
Com_call_procedure 0
Com_change_db 26791502
Com_change_master 0
Com_check 0
Com_checksum 0
Com_commit 0
Com_create_db 0
Com_create_event 0
Com_create_function 0
Com_create_index 0
Com_create_procedure 0
Com_create_server 0
Com_create_table 0
Com_create_trigger 0
Com_create_udf 0
Com_create_user 0
Com_create_view 0
Com_dealloc_sql 0
Com_delete 584152
Com_delete_multi 0
Com_do 0
Com_drop_db 0
Com_drop_event 0
Com_drop_function 0
Com_drop_index 0
Com_drop_procedure 0
Com_drop_server 0
Com_drop_table 0
Com_drop_trigger 0
Com_drop_user 0
Com_drop_view 0
Com_empty_query 0
Com_execute_sql 0
Com_flush 0
Com_grant 0
Com_ha_close 0
Com_ha_open 0
Com_ha_read 0
Com_help 0
Com_insert 5871056
Com_insert_select 2833
Com_install_plugin 0
Com_kill 160
Com_load 0
Com_lock_tables 0
Com_optimize 0
Com_preload_keys 0
Com_prepare_sql 0
Com_purge 0
Com_purge_before_date 0
Com_release_savepoint 0
Com_rename_table 0
Com_rename_user 0
Com_repair 0
Com_replace 0
Com_replace_select 0
Com_reset 0
Com_resignal 0
Com_revoke 0
Com_revoke_all 0
Com_rollback 0
Com_rollback_to_savepoint 0
Com_savepoint 0
Com_select 21739455
Com_set_option 53600251
Com_signal 0
Com_show_authors 0
Com_show_binlog_events 0
Com_show_binlogs 217
Com_show_charsets 3
Com_show_collations 26786645
Com_show_contributors 0
Com_show_create_db 0
Com_show_create_event 0
Com_show_create_func 0
Com_show_create_proc 0
Com_show_create_table 6412
Com_show_create_trigger 84
Com_show_databases 213
Variable_name Value
Com_show_engine_logs 0
Com_show_engine_mutex 0
Com_show_engine_status 0
Com_show_events 0
Com_show_errors 0
Com_show_fields 4389
Com_show_function_status 14
Com_show_grants 2
Com_show_keys 25
Com_show_master_status 5
Com_show_open_tables 0
Com_show_plugins 560
Com_show_privileges 0
Com_show_procedure_status 14
Com_show_processlist 189
Com_show_profile 0
Com_show_profiles 0
Com_show_relaylog_events 0
Com_show_slave_hosts 0
Com_show_slave_status 5
Com_show_status 156
Com_show_storage_engines 16
Com_show_table_status 4154
Com_show_tables 280
Com_show_triggers 4144
Com_show_variables 26786703
Com_show_warnings 2363
Com_slave_start 0
Com_slave_stop 0
Com_stmt_close 0
Com_stmt_execute 0
Com_stmt_fetch 0
Com_stmt_prepare 0
Com_stmt_reprepare 0
Com_stmt_reset 0
Com_stmt_send_long_data 0
Com_truncate 0
Com_uninstall_plugin 0
Com_unlock_tables 168
Com_update 2572452
Com_update_multi 35171
Com_xa_commit 0
Com_xa_end 0
Com_xa_prepare 0
Com_xa_recover 0
Com_xa_rollback 0
Com_xa_start 0
Compression OFF
Connections 26787752
Created_tmp_disk_tables 119645
Created_tmp_files 7069
Created_tmp_tables 58427880
Delayed_errors 0
Delayed_insert_threads 0
Delayed_writes 0
Flush_commands 1
Handler_commit 30698922
Handler_delete 177424
Handler_discover 0
Handler_prepare 0
Handler_read_first 1239039
Handler_read_key 2226723080
Handler_read_last 9200
Handler_read_next 4186985860
Handler_read_prev 6811735740
Handler_read_rnd 8942473
Handler_read_rnd_next 16431972655
Handler_rollback 495
Handler_savepoint 0
Handler_savepoint_rollback 0
Handler_update 9720709
Handler_write 15014651212
Innodb_buffer_pool_pages_data 8173
Innodb_buffer_pool_pages_dirty 138
Innodb_buffer_pool_pages_flushed 34780758
Innodb_buffer_pool_pages_free 0
Innodb_buffer_pool_pages_misc 18
Innodb_buffer_pool_pages_total 8191
Innodb_buffer_pool_read_ahead_rnd 0
Innodb_buffer_pool_read_ahead 26163447
Innodb_buffer_pool_read_ahead_evicted 669245354
Innodb_buffer_pool_read_requests 19793139859
Innodb_buffer_pool_reads 1156881183
Innodb_buffer_pool_wait_free 0
Innodb_buffer_pool_write_requests 81288931
Innodb_data_fsyncs 8482576
Innodb_data_pending_fsyncs 0
Innodb_data_pending_reads 0
Innodb_data_pending_writes 0
Innodb_data_read 19409628139520
Innodb_data_reads 1184668281
Innodb_data_writes 25747879
Innodb_data_written 576917547008
Innodb_dblwr_pages_written 17390379
Innodb_dblwr_writes 255445
Innodb_have_atomic_builtins ON
Innodb_log_waits 0
Innodb_log_write_requests 6930648
Innodb_log_writes 7850155
Innodb_os_log_fsyncs 7976564
Variable_name Value
Innodb_os_log_pending_fsyncs 0
Innodb_os_log_pending_writes 0
Innodb_os_log_written 7007020032
Innodb_page_size 16384
Innodb_pages_created 78647
Innodb_pages_read 1184669628
Innodb_pages_written 17390379
Innodb_row_lock_current_waits 0
Innodb_row_lock_time 1116
Innodb_row_lock_time_avg 39
Innodb_row_lock_time_max 963
Innodb_row_lock_waits 28
Innodb_rows_deleted 177424
Innodb_rows_inserted 5863854
Innodb_rows_read 14138431339
Innodb_rows_updated 4886889
Innodb_truncated_status_writes 0
Key_blocks_not_flushed 0
Key_blocks_unused 13
Key_blocks_used 13
Key_read_requests 3458723878
Key_reads 49092342
Key_write_requests 462162
Key_writes 436874
Last_query_cost 0.000000
Max_used_connections 137
Not_flushed_delayed_rows 0
Open_files 0
Open_streams 0
Open_table_definitions 113
Open_tables 4
Opened_files 6485943
Opened_table_definitions 116
Opened_tables 31771706
Performance_schema_cond_classes_lost 0
Performance_schema_cond_instances_lost 0
Performance_schema_file_classes_lost 0
Performance_schema_file_handles_lost 0
Performance_schema_file_instances_lost 0
Performance_schema_locker_lost 0
Performance_schema_mutex_classes_lost 0
Performance_schema_mutex_instances_lost 0
Performance_schema_rwlock_classes_lost 0
Performance_schema_rwlock_instances_lost 0
Performance_schema_table_handles_lost 0
Performance_schema_table_instances_lost 0
Performance_schema_thread_classes_lost 0
Performance_schema_thread_instances_lost 0
Prepared_stmt_count 0
Qcache_free_blocks 0
Qcache_free_memory 0
Qcache_hits 0
Qcache_inserts 0
Qcache_lowmem_prunes 0
Qcache_not_cached 0
Qcache_queries_in_cache 0
Qcache_total_blocks 0
Queries 191630861
Questions 191562751
Rpl_status AUTH_MASTER
Select_full_join 7193
Select_full_range_join 0
Select_range 957755
Select_range_check 0
Select_scan 57577013
Slave_heartbeat_period 0.000
Slave_open_temp_tables 0
Slave_received_heartbeats 0
Slave_retried_transactions 0
Slave_running OFF
Slow_launch_threads 0
Slow_queries 445
Sort_merge_passes 3531
Sort_range 5655320
Sort_rows 43857009
Sort_scan 757681
Ssl_accept_renegotiates 0
Ssl_accepts 0
Ssl_callback_cache_hits 0
Ssl_cipher
Ssl_cipher_list
Ssl_client_connects 0
Ssl_connect_renegotiates 0
Ssl_ctx_verify_depth 0
Ssl_ctx_verify_mode 0
Ssl_default_timeout 0
Ssl_finished_accepts 0
Ssl_finished_connects 0
Ssl_session_cache_hits 0
Ssl_session_cache_misses 0
Ssl_session_cache_mode NONE
Ssl_session_cache_overflows 0
Ssl_session_cache_size 0
Ssl_session_cache_timeouts 0
Ssl_sessions_reused 0
Ssl_used_session_cache_entries 0
Ssl_verify_depth 0
Ssl_verify_mode 0
Ssl_version
Table_locks_immediate 51310065
Variable_name Value
Table_locks_waited 14101
Tc_log_max_pages_used 0
Tc_log_page_size 0
Tc_log_page_waits 0
Threads_cached 0
Threads_connected 4
Threads_created 26787751
Threads_running 1
Uptime 1208447
Uptime_since_flush_status 1208447
Current SHOW GLOBAL VARIABLES:
Variable_name Value
auto_increment_increment 1
auto_increment_offset 1
autocommit ON
automatic_sp_privileges ON
back_log 50
basedir /usr
big_tables OFF
binlog_cache_size 32768
binlog_direct_non_transactional_updates OFF
binlog_format STATEMENT
binlog_stmt_cache_size 32768
bulk_insert_buffer_size 8388608
character_set_client latin1
character_set_connection latin1
character_set_database latin1
character_set_filesystem binary
character_set_results latin1
character_set_server latin1
character_set_system utf8
character_sets_dir /usr/share/mysql/charsets/
collation_connection latin1_swedish_ci
collation_database latin1_swedish_ci
collation_server latin1_swedish_ci
completion_type NO_CHAIN
concurrent_insert AUTO
connect_timeout 10
datadir /var/lib/mysql/
date_format %Y-%m-%d
datetime_format %Y-%m-%d %H:%i:%s
default_storage_engine InnoDB
default_week_format 0
delay_key_write ON
delayed_insert_limit 100
delayed_insert_timeout 300
delayed_queue_size 1000
div_precision_increment 4
engine_condition_pushdown ON
event_scheduler OFF
expire_logs_days 0
flush OFF
flush_time 0
foreign_key_checks ON
ft_boolean_syntax + -><()~*:""&|
ft_max_word_len 84
ft_min_word_len 3
ft_query_expansion_limit 20
ft_stopword_file (built-in)
general_log OFF
general_log_file /var/lib/mysql/db.log
group_concat_max_len 1024
have_compress YES
have_crypt YES
have_csv YES
have_dynamic_loading YES
have_geometry YES
have_innodb YES
have_ndbcluster NO
have_openssl DISABLED
have_partitioning YES
have_profiling YES
have_query_cache YES
have_rtree_keys YES
have_ssl DISABLED
have_symlink YES
hostname silldb5
ignore_builtin_innodb OFF
init_connect
init_file
init_slave
innodb_adaptive_flushing ON
innodb_adaptive_hash_index ON
innodb_additional_mem_pool_size 8388608
innodb_autoextend_increment 8
innodb_autoinc_lock_mode 1
innodb_buffer_pool_instances 1
innodb_buffer_pool_size 134217728
innodb_change_buffering all
innodb_checksums ON
innodb_commit_concurrency 0
innodb_concurrency_tickets 500
innodb_data_file_path ibdata1:10M:autoextend
innodb_data_home_dir
innodb_doublewrite ON
innodb_fast_shutdown 1
innodb_file_format Antelope
innodb_file_format_check ON
innodb_file_format_max Antelope
innodb_file_per_table OFF
innodb_flush_log_at_trx_commit 1
innodb_flush_method
innodb_force_load_corrupted OFF
innodb_force_recovery 0
innodb_io_capacity 200
innodb_large_prefix OFF
innodb_lock_wait_timeout 50
innodb_locks_unsafe_for_binlog OFF
innodb_log_buffer_size 8388608
innodb_log_file_size 5242880
innodb_log_files_in_group 2
innodb_log_group_home_dir ./
Variable_name Value
innodb_max_dirty_pages_pct 75
innodb_max_purge_lag 0
innodb_mirrored_log_groups 1
innodb_old_blocks_pct 37
innodb_old_blocks_time 0
innodb_open_files 300
innodb_purge_batch_size 20
innodb_purge_threads 0
innodb_random_read_ahead OFF
innodb_read_ahead_threshold 56
innodb_read_io_threads 4
innodb_replication_delay 0
innodb_rollback_on_timeout OFF
innodb_rollback_segments 128
innodb_spin_wait_delay 6
innodb_stats_method nulls_equal
innodb_stats_on_metadata ON
innodb_stats_sample_pages 8
innodb_strict_mode OFF
innodb_support_xa ON
innodb_sync_spin_loops 30
innodb_table_locks ON
innodb_thread_concurrency 0
innodb_thread_sleep_delay 10000
innodb_use_native_aio ON
innodb_use_sys_malloc ON
innodb_version 1.1.8
innodb_write_io_threads 4
interactive_timeout 28800
join_buffer_size 131072
keep_files_on_create OFF
key_buffer_size 16384
key_cache_age_threshold 300
key_cache_block_size 1024
key_cache_division_limit 100
large_files_support ON
large_page_size 0
large_pages OFF
lc_messages en_US
lc_messages_dir /usr/share/mysql/
lc_time_names en_US
license GPL
local_infile ON
lock_wait_timeout 31536000
locked_in_memory OFF
log OFF
log_bin OFF
log_bin_trust_function_creators OFF
log_error /var/lib/mysql/db.err
log_output FILE
log_queries_not_using_indexes OFF
log_slave_updates OFF
log_slow_queries OFF
log_warnings 1
long_query_time 10.000000
low_priority_updates OFF
lower_case_file_system OFF
lower_case_table_names 0
max_allowed_packet 1048576
max_binlog_cache_size 18446744073709547520
max_binlog_size 1073741824
max_binlog_stmt_cache_size 18446744073709547520
max_connect_errors 10
max_connections 2000
max_delayed_threads 20
max_error_count 64
max_heap_table_size 16777216
max_insert_delayed_threads 20
max_join_size 18446744073709551615
max_length_for_sort_data 1024
max_long_data_size 1048576
max_prepared_stmt_count 16382
max_relay_log_size 0
max_seeks_for_key 18446744073709551615
max_sort_length 1024
max_sp_recursion_depth 0
max_tmp_tables 32
max_user_connections 0
max_write_lock_count 18446744073709551615
metadata_locks_cache_size 1024
min_examined_row_limit 0
multi_range_count 256
myisam_data_pointer_size 6
myisam_max_sort_file_size 9223372036853727232
myisam_mmap_size 18446744073709551615
myisam_recover_options OFF
myisam_repair_threads 1
myisam_sort_buffer_size 8388608
myisam_stats_method nulls_unequal
myisam_use_mmap OFF
net_buffer_length 2048
net_read_timeout 30
net_retry_count 10
net_write_timeout 60
new OFF
old OFF
old_alter_table OFF
old_passwords OFF
open_files_limit 10000
optimizer_prune_level 1
Variable_name Value
optimizer_search_depth 62
optimizer_switch index_merge=on,index_merge_union=on,index_merge_so...
performance_schema OFF
performance_schema_events_waits_history_long_size 10000
performance_schema_events_waits_history_size 10
performance_schema_max_cond_classes 80
performance_schema_max_cond_instances 1000
performance_schema_max_file_classes 50
performance_schema_max_file_handles 32768
performance_schema_max_file_instances 10000
performance_schema_max_mutex_classes 200
performance_schema_max_mutex_instances 1000000
performance_schema_max_rwlock_classes 30
performance_schema_max_rwlock_instances 1000000
performance_schema_max_table_handles 100000
performance_schema_max_table_instances 50000
performance_schema_max_thread_classes 50
performance_schema_max_thread_instances 1000
pid_file /var/run/mysqld/mysqld.pid
plugin_dir /usr/lib64/mysql/plugin
port 3306
preload_buffer_size 32768
profiling OFF
profiling_history_size 15
protocol_version 10
query_alloc_block_size 8192
query_cache_limit 1048576
query_cache_min_res_unit 4096
query_cache_size 8 388 608
query_cache_type ON
query_cache_wlock_invalidate OFF
query_prealloc_size 8192
range_alloc_block_size 4096
read_buffer_size 262144
read_only OFF
read_rnd_buffer_size 262144
relay_log
relay_log_index
relay_log_info_file relay-log.info
relay_log_purge ON
relay_log_recovery OFF
relay_log_space_limit 0
report_host
report_password
report_port 3306
report_user
rpl_recovery_rank 0
secure_auth OFF
secure_file_priv
server_id 1
skip_external_locking ON
skip_name_resolve ON
skip_networking OFF
skip_show_database OFF
slave_compressed_protocol OFF
slave_exec_mode STRICT
slave_load_tmpdir /tmp
slave_net_timeout 3600
slave_skip_errors OFF
slave_transaction_retries 10
slave_type_conversions
slow_launch_time 2
slow_query_log OFF
slow_query_log_file /var/lib/mysql/slow.log
socket /var/lib/mysql/mysql.sock
sort_buffer_size 65536
sql_auto_is_null OFF
sql_big_selects ON
sql_big_tables OFF
sql_buffer_result OFF
sql_log_bin ON
sql_log_off OFF
sql_low_priority_updates OFF
sql_max_join_size 18446744073709551615
sql_mode
sql_notes ON
sql_quote_show_create ON
sql_safe_updates OFF
sql_select_limit 18446744073709551615
sql_slave_skip_counter 0
sql_warnings OFF
ssl_ca
ssl_capath
ssl_cert
ssl_cipher
ssl_key
storage_engine InnoDB
sync_binlog 0
sync_frm ON
sync_master_info 0
sync_relay_log 0
sync_relay_log_info 0
system_time_zone CET
table_definition_cache 400
table_open_cache 4
thread_cache_size 100
thread_concurrency 8
thread_handling one-thread-per-connection
thread_stack 524288
time_format %H:%i:%s
Variable_name Value
time_zone SYSTEM
timed_mutexes OFF
tmp_table_size 16777216
tmpdir /tmp
transaction_alloc_block_size 8192
transaction_prealloc_size 4096
tx_isolation REPEATABLE-READ
unique_checks ON
updatable_views_with_limit YES
version 5.5.20
version_comment MySQL Community Server (GPL) by Remi
version_compile_machine x86_64
version_compile_os Linux
wait_timeout 28800
Optimising mysql is very well commented on the net, and you’ll find huge information on how to do this. There is never “best parameters”, the best parameters is those fits your needs, box hardware, mysql usage…
So, I’ll not give the best parameters but rather how to define these ones. Make some tests, and you’ll quickly find your own parameters.
There a lot of available parameters but only few one are very important to tweak your mysql box.
The most important variables are (for me, and it is not exhaustive):
- max_connections
- wait_timeout
- thread_cache_size
- table_cache
- key_buffer_size
- query_cache_size
- tmp_table_size
To obtain the stat of your mysql server since it has been loaded, run mysqladmin processlist extended-status as mentionned above.
1 - The two most important variables : Table_cache and Key_buffer_size
If Opened_tables is big, then your table_cache variable is probably
too small.
table_cache 64
Open_tables 64
Opened_tables 544468
This is the first serious problem. "The table_cache is the number of open
tables for all threads. MySQL, being multi-threaded, may be running many
queries on the table at one time, and each of these will open a table."
Therefore, even though we only have a few tables, we will need many more
open_tables.
The Opened_tables value is high and shows the number of
cache misses. Getting the table_cache size correct is one of the two best
things you can do to improve performance.
If Key_reads is big, then your key_buffer_size variable is probably
too small. The cache hit rate can be calculated with
Key_reads/Key_read_requests.
key_buffer_size 16M
Key_read_requests 2973620399
Key_reads 8490571
(cache hit rate = 0.0028)
“The key_buffer_size affects the size of the index buffers and the speed
of index handling, particularly reading." The MySQL manual (and other sources) say that
"Key_reads/Key_read_request ratio should normally be < 0.01." This is the
other most important thing to get correct. Here the value seems to be correct (< 0.01)
Also check key_write_requests and key_writes.
The key_writes/key_writes_request should normally be < 1 (near 0.5 seems to be fine)
Here is a very interesting web pointer : table_cache and key_buffer_size
2 - Others important settings are : Wait_timeout, max_connexion, thread_cache
A little explanation :
Generaly you have a lot of mysql process that are sleeping because wait_timeout are not set low. So I make sure that the wait_timeout is set to a very low value: 15 seconds (for me) . That means MySQL would close any connection that was idle for more than 15 seconds.
The problem is you also have to increment your max_connexion (mine is set to 300) to be sure there is not a lot of idle clients holding connections and blocking out new clients from connecting and getting real work done.
The pbm is that the box has to create new threads (MySQL is a multi-threaded server) at a very high rate. That may sucks up a measurable amount of CPU time.
So the solution is to use the Thread_cache (from mysql doc) :
“How many threads we should keep in a cache for reuse. When a client disconnects, the client's threads are put in the cache if there aren't more than thread_cache_size threads from before. All new threads are first taken from the cache, and only when the cache is empty is a new thread created. This variable can be increased to improve performance if you have a lot of new connections. (Normally this doesn't give a notable performance improvement if you have a good thread implementation.) By examing the difference between the Connections and Threads_created you can see how efficient the current thread cache is for you.”
If Threads_created is big, you may want to increase the
thread_cache_size variable. The cache hit rate can be calculated with
Threads_created/Connections.
thread_cache_size 0
Threads_created 150022
Connections 150023
This is the second problem that should be fixed. A cache size of zero is the default for my-medium.cnf but the recommended size in my-large.cnf is 8.
you may try this formula : table_cache = opened table / max_used_connection
3 - Finally, you may also have a look at : tmp_table_size and Handler_read_rnd / Handler_read_rnd_next
If Created_tmp_disk_tables is big, you may want to increase the
tmp_table_size variable to get the temporary tables memory-based instead
of disk based.
tmp_table_size 32M
Created_tmp_disk_tables 3227
Created_tmp_tables 159832
Created_tmp_files 4444
Created_tmp_disk_tables are the "number of implicit temporary tables on
disk created while executing statements" and Created_tmp_tables are
memory-based. Obviously it is bad if you have to go to disk instead of
memory. About 2% of temp tables go to disk, which doesn't seem too bad
but increasing the tmp_table_size probably couldn't hurt either.
If Handler_read_rnd is big, then you probably have a lot of queries
that require MySQL to scan whole tables or you have joins that don't use
keys properly.
Handler_read_rnd 27712353
Handler_read_rnd_next 283536234
These values are high, that we could probably stand to improve
the indexes and queries.
I hope this will help some of you to more understand how it is possible to optimise MYSQL to fit your needs, hardaware box, or mysql current usage.
For those who'll read this response later :
ONLY FOR MYSQL SERVER LESS THAN 6.0 If you change tmp_table_size, don't forget to carry about max_heap_table_size value too.
Explanation here
Another topic about this VARS : here
For people just discovering this very helpful post, other possible changes include:
max_connections of 2000 could be safely reduced since max_used_connections is 137.
key_buffer_size at 16384 bytes seems very small for the workload.
key_cache_division_limit = 50 would keep repetetively used keys in hot/warm memory and avoid disk reads.
key_age_threshold = 3600 seconds would keep keys available in memory for 1 hr vs 5 minutes.
open_files_limit = 1024 would support the workload.