TokuDB alone beat InnoDB in our benchmarks by about 25% but when I turn master-slave on InnoDB now beats TokuDB by about 20%
Any idea what's going on
here's the conf that's running on a r4.8xlarge aws machine
[mysql_safe]
malloc-lib=/usr/include/jemalloc
[mysqld]
user = mysql
pid-file = /var/run/mysqld/mysqld.pid
socket = /var/run/mysqld/mysqld.sock
port = 3306
server-id = 1
log_bin = /var/log/mysql/mysql-bin.log
binlog_do_db = mt
sync_binlog=1
binlog_format=ROW
binlog_row_image=FULL
basedir = /usr
datadir = /var/lib/mysql
tmpdir = /tmp
lc-messages-dir = /usr/share/mysql
explicit_defaults_for_timestamp
#malloc-lib=/usr/include/jemalloc
key_buffer_size= 250G
read_buffer_size=2G
read_rnd_buffer_size=50M
join_buffer_size=25M
tmp_table_size = 5G
sort_buffer_size = 2G
query_cache_limit = 10M
query_cache_size = 100M
innodb_buffer_pool_instances=64
tokudb_fanout = 128
tokudb_commit_sync = 0
tokudb_fsync_log_period = 1000
tokudb_directio = 1
# Instead of skip-networking the default is now to listen only on
# localhost which is more compatible and is not less secure.
innodb-flush-method = O_DIRECT
innodb-log-files-in-group = 2
innodb-log-file-size = 512M
innodb-flush-log-at-trx-commit = 2
innodb-file-per-table = 1
innodb-buffer-pool-size = 200G
log-error = /var/log/mysql/error.log
# Recommended in standard MySQL setup
sql_mode=NO_ENGINE_SUBSTITUTION,STRICT_ALL_TABLES
# Disabling symbolic-links is recommended to prevent assorted security risks
symbolic-links=0
Related
I use vertx java to build an app and writing tests for mysql queries. I can successful get results before but after I modified my.cnf because of other stuffs. I cannot no long get returned results but "Fail to read any response from the server, the underlying connection might get lost unexpectedly". I searched google, but I cannot figure out exactly where is wrong and how to fix it.
I would post my.cnf file:
[client]
port = 3306
socket = /tmp/mysql.sock
# This was formally known as [safe_mysqld]. Both versions are currently parsed.
[mysqld_safe]
socket = /tmp/mysql.sock
nice = 0
syslog
[mysqld]
# Basic Settings
port = 3306
user = root
pid-file = /usr/local/mysql/data/mysqld.local.pid
socket = /tmp/mysql.sock
port = 3306
basedir = /usr/local/mysql/
datadir = /usr/local/mysql/data
tmpdir = /tmp
lc-messages-dir = /usr/local/mysql/share
skip-external-locking
# Engine
default-storage-engine = InnoDB
# Listening IP
bind-address = 0.0.0.0
# Safety
max-connect-errors = 1000000
max_allowed_packet = 64M
skip-name-resolve
sysdate-is-now = 1
innodb = FORCE
innodb-strict-mode = 1
tls_version = TLSv1,TLSv1.1,TLSv1.2
wait_timeout = 28800
interactive_timeout = 10000
# Buffers
sort_buffer_size = 4M
read_buffer_size = 2M
join_buffer_size = 8M
read_rnd_buffer_size = 16M
# MyISAM
key-buffer-size = 32M
# CACHES AND LIMITS #
tmp-table-size = 128M
max-heap-table-size = 128M
# Bin logs
binlog-format = ROW
log_bin = /tmp/binlog
max_binlog_size = 100M
server-id = 1 # randomize it incase of multiple servers
# InnoDB
innodb-buffer-pool-size = 2048M
innodb_buffer_pool_instances = 8
innodb_log_buffer_size = 8M
innodb-log-files-in-group = 2
innodb-log-file-size = 256M
innodb-file-per-table = 1
innodb-flush-log-at-trx-commit = 1
innodb-flush-method = O_DIRECT
# With virtual synchrony redundancy, make write queries faster
innodb_doublewrite = 1
# LOGGING
general_log_file = /usr/local/mysql/mysql.log
log-error = /usr/local/mysql/data/mysqld.local.err
log-queries-not-using-indexes = 1
[mysqldump]
quick
quote-names
max_allowed_packet = 64M
[isamchk]
key_buffer = 16M
#
# * IMPORTANT: Additional settings that can override those from this file!
# The files must end with '.cnf', otherwise they'll be ignored.
#
Can you tell me what's wrong with the configuration and what should I change? Thanks
I am using MySQL DB on Windows 2012 R2, x64 with 8 GB RAM. MySQL database many times shows the error - Out of Memory (needed 1048584 bytes). Below are the database Variable values i have set in my.ini file. Is there any change is required in this file?
[mysqld]
port= 3306
socket = "C:/xampp/mysql/mysql.sock"
basedir = "C:/xampp/mysql"
tmpdir = "C:/xampp/tmp"
datadir = "C:/xampp/mysql/data"
pid_file = "mysql.pid"
# enable-named-pipe
key_buffer_size = 16M
max_allowed_packet = 256M
table_cache = 256
net_buffer_length = 1M
read_buffer_size = 512K
read_rnd_buffer_size = 512K
myisam_sort_buffer_size = 8M
log_error = "mysql_error.log"
# Change here for bind listening
# bind-address="127.0.0.1"
# bind-address = ::1 # for ipv6
max_connect_errors = 500
max_connections = 5000
net_write_timeout = 1800
net_read_timeout = 1800
slow_launch_time = 10
slow_query_log = OFF
thread_cache_size = 2000
back_log = 100
performance_schema = OFF
query_cache_size = 512M
query_cache_limit = 5M
join_buffer_size = 4M
sort_buffer_size = 4M
max_heap_table_size = 64M
tmp_table_size = 128M
table_open_cache = 4500
table_definition_cache=4000
innodb_file_per_table=1
open_files_limit=50000
innodb_data_home_dir = "C:/xampp/mysql/data"
innodb_data_file_path = ibdata1:10M:autoextend
innodb_log_group_home_dir = "C:/xampp/mysql/data"
#innodb_log_arch_dir = "C:/xampp/mysql/data"
## You can set .._buffer_pool_size up to 50 - 80 %
## of RAM but beware of setting memory usage too high
innodb_buffer_pool_size = 2048M
innodb_buffer_pool_instances = 2
## Set .._log_file_size to 25 % of buffer pool size
innodb_log_file_size = 256M
innodb_log_buffer_size = 8M
innodb_flush_log_at_trx_commit = 2
innodb_lock_wait_timeout = 50
innodb_flush_method=O_DIRECT
innodb_io_capacity=1000
innodb_old_blocks_time=1000
innodb_open_files=5000
innodb_sort_buffer_size = 8M
Please reply if someone can manage to get this working again!
Thanks in advance.
Suggestions to consider for your my.ini [mysqld] section
Lead with # (or REMOVE) to allow defaults to serve you
. net_buffer_length
. read_buffer_size
. read_rnd_buffer_size
. MyISAM_sort_buffer_size
. join_buffer_size
. sort_buffer_size
. innodb_sort_buffer_size
max_connections=500 # from 5000 until you have a known need for more connections
thread_cache_size=100 # from 2000 per v8 refman CAP at 100 to avoid OOM
query_cache_type=0 # to avoid QC overhead
query_cache_size=0 # from 512M to conserve RAM for more useful purpose
query_cache_limit=0 # 0 is fine when not using QC
tmp_table_size=64M # from 128M until you can monitor created_tmp_disk_tables
tmp_table_size and max_heap_table_size should always be the same size.
Good luck.
For additional assistance please check my profile, clk Network Profile for contact info.
I am getting this error when I try starting the MySQL service through the XAMPP control panel:
2014-08-17 14:53:44 6028 [ERROR] TCP/IP, --shared-memory, or --named-pipe should be configured on NT OS
2014-08-17 14:53:44 6028 [ERROR] Aborting
Anyone ran into this issue before and have a configuration change that may be used to get it running?
For reference, these are the settings within my.ini:
[client]
port = 3306
socket = "G:/Dev/xampp/mysql/mysql.sock"
# The MySQL server
[mysqld]
port= 3306
socket = "G:/Dev/xampp/mysql/mysql.sock"
basedir = "G:/Dev/xampp/mysql"
tmpdir = "G:/Dev/xampp/tmp"
datadir = "G:/Dev/xampp/mysql/data"
pid_file = "mysql.pid"
# enable-named-pipe
key_buffer = 16M
max_allowed_packet = 1M
sort_buffer_size = 512K
net_buffer_length = 8K
read_buffer_size = 256K
read_rnd_buffer_size = 512K
myisam_sort_buffer_size = 8M
log_error = "mysql_error.log"
# SECURITY CHANGES
skip-networking
local-infile=0
plugin_dir = "G:/Dev/xampp/mysql/lib/plugin/"
skip-federated
server-id = 1
innodb_data_home_dir = "G:/Dev/xampp/mysql/data"
innodb_data_file_path = ibdata1:10M:autoextend
innodb_log_group_home_dir = "G:/Dev/xampp/mysql/data"
innodb_buffer_pool_size = 256M
innodb_additional_mem_pool_size = 2M
innodb_log_file_size = 32M
innodb_log_buffer_size = 8M
innodb_flush_log_at_trx_commit = 2
innodb_lock_wait_timeout = 50
innodb_thread_concurrency = 16
innodb_flush_method = normal
[mysqldump]
quick
max_allowed_packet = 16M
[mysql]
no-auto-rehash
[isamchk]
key_buffer = 20M
sort_buffer_size = 20M
read_buffer = 2M
write_buffer = 2M
[myisamchk]
key_buffer = 20M
sort_buffer_size = 20M
read_buffer = 2M
write_buffer = 2M
[mysqlhotcopy]
interactive-timeout
Either enable named pipes by removing the comment before the option enable-named-pipe
pid_file = "mysql.pid"
enable-named-pipe # this line
key_buffer = 16M
or let mysql listen on a TCP/IP port by commenting out the option skip-networking:
# SECURITY CHANGES
# skip-networking # this line
If networking is disabled and named pipes are not enabled then mysqld will useless.
With mysql 8 on windows, if you try to start it from command line with skip-grant-tables you will hit this error.
And the solution is adding shared-memory.
.\mysqld --skip-grant-tables --shared-memory
Hi my server have 24 cores and 32GB of memory.
Am doing multiple "INSERT INTO SELECT" of 50 millions row at a time.
This takes about 15h a query but it is ticking along at 100% of only one CPU, I'am trying to get mySQL(5.5)(InnoDB) to use more of the resources.
I have read multiple threads about it, but I do not get it to work.
Most info is about adding innodb_thread_concurrency = 0
But I still get no results.
port = 3306
socket = /var/run/mysqld/mysqld.sock
[mysqld_safe]
socket = /var/run/mysqld/mysqld.sock
nice = 0
[mysqld]
user = mysql
pid-file = /var/run/mysqld/mysqld.pid
socket = /var/run/mysqld/mysqld.sock
port = 3306
basedir = /usr
datadir = /media/ssd/db
tmpdir = /tmp
lc-messages-dir = /usr/share/mysql
skip-external-locking
innodb_buffer_pool_size=26G
innodb_thread_concurrency = 0
bind-address = 127.0.0.1
#
# * Fine Tuning
#
key_buffer = 1000M
max_allowed_packet = 160M
thread_stack = 192K
thread_cache_size = 8
# This replaces the startup script and checks MyISAM tables if needed
# the first time they are touched
myisam-recover = BACKUP
table_cache = 800
query_cache_limit = 5000M
query_cache_size = 1600M
join_buffer_size = 1000M
log_error = /var/log/mysql/error.log
# Here you can see queries with especially long duration
log_slow_queries = /var/log/mysql/mysql-slow.log
long_query_time = 2
Try this parameters:
innodb_io_capacity=5000 (or even 20000 depending on your IO subsystem)
innodb_buffer_pool_size=4G (for example)
innodb_log_file_size=1G
innodb_write_io_threads = 64
innodb_read_io_threads = 64
innodb_thread_concurrency = 0
i was trying to enable slow_query_log. i have downloaded my.cnf file from server using WinSCP and added the code for enabling slow_query_log.
slow_query_log = ON
slow_query_log_file = /var/log/mysql_slow.log
And when i restart my server i am getting this error mysql has failed, please contact the sysadmin (result was "mysql is not running").
here i am adding my my.cnf file
i have tried with slow_query_log = 1 also. But getting the same error. when i remove the code which i have added the error is gone. Any one please help me...
my.cnf
[mysqld]
datadir=/home/mysql/
tmpdir=/home/mysqltmp
#max_connections = 175 #was 175
max_connections = 80
#max_connect_errors = 350 #was 250
max_connect_errors = 250
safe-show-database
skip-locking
key_buffer = 1024M # was 128M
max_allowed_packet = 6M
myisam_sort_buffer_size = 64M
#old settings, for 900 ish max maxconn
#sort_buffer_size = 32M
#read_buffer_size = 32M
#read_rnd_buffer_size = 32M
sort_buffer_size = 5M
read_buffer_size = 5M
read_rnd_buffer_size = 5M
query_cache_size= 1024M
query_cache_limit= 16M
max_heap_table_size = 128M
tmp_table_size = 128M
thread_concurrency = 16
wait_timeout = 10
innodb_file_per_table
innodb_log_file_size = 10485760
open_files_limit = 8192
low_priority_updates = 1
slow_query_log = ON
slow_query_log_file = /var/log/mysql_slow.log
# who set these? these are NOT memory settings, but rather integer settings.
#table_cache = 1024M
#thread_cache_size = 8M
table_cache = 512
thread_cache_size = 8
[mysqldump]
quick
max_allowed_packet = 16M
[mysql]
no-auto-rehash
[isamchk]
key_buffer = 128M
sort_buffer_size = 128M
read_buffer = 2M
write_buffer = 2M
[myisamchk]
key_buffer = 128M
sort_buffer_size = 128M
read_buffer = 2M
write_buffer = 2M
create the file mysql_slow.log in /var/log & change ownerhip of the file:
chown mysql.mysql -R /var/log/mysql_slow.log
either value (1 or ON) should work for the slow query log.
also, are you using regular statements?
check for ownership as mysql.mysql and permissions on file /var/log/mysql_slow.log
ls -al /var/log/mysql_slow.log