I have MySQL Community Server 5.6.16 running, I have a mysql dump file of size 290MB,
and I have the following my.ini settings
innodb_lock_wait_timeout = 60000000
max_allowed_packet = 8000000000M
When i try to restore the database from the dump file, I get error below
#2006 - MySQL server has gone away
How to resolve this error.
RAM Size: 3.00 GB
my.ini
----------------
key_buffer = 16M
max_allowed_packet = 1M
sort_buffer_size = 512K
net_buffer_length = 8K
read_buffer_size = 256K
read_rnd_buffer_size = 512K
myisam_sort_buffer_size = 8M
innodb_buffer_pool_size = 16M
innodb_additional_mem_pool_size = 2M
## Set .._log_file_size to 25 % of buffer pool size
innodb_log_file_size = 5M
innodb_log_buffer_size = 8M
innodb_flush_log_at_trx_commit = 1
innodb_lock_wait_timeout = 60000000
my.ini file download link
Change as per below:
max_allowed_packet = 64M
innodb_buffer_pool_size = 1536M
Also add below lines same as given at the end of your ini file.
[mysqldump]
quick ##it is already there in your file.
max_allowed_packet = 80000M # change it to 64M as this is more than your total memory.
After these changes restart mysql and check now.
If still getting error then share todays error logs from your error log file.
Related
I have a software application built with Electron, there is an Ajax call every seconds.
Everything seems to work however the server gets smashed and we are experiencing massive overload, slowness. The sites would be fast, then slow, and suddently fast again or sometimes keeps spinning. We have 8 users using the application which result to 80 queries per seconds.
Here my cnf file
[client]
port = 3306
socket = /var/lib/mysql/mysql.sock
[mysqld]
port = 3306
socket = /var/lib/mysql/mysql.sock
sql_mode = ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION
character-set-server = utf8mb4
collation-server = utf8mb4_unicode_ci
max_allowed_packet = 64M
max_user_connections = 100
max_delayed_threads = 0
tmpdir = /tmp
wait_timeout = 300
read_buffer_size = 2M
read_rnd_buffer_size = 4M
myisam-recover-options = DEFAULT
myisam_sort_buffer_size = 128M
innodb_buffer_pool_size = 128M
innodb_log_buffer_size = 32M
innodb_data_home_dir = /var/lib/mysql/
innodb_file_per_table = 1
innodb_log_group_home_dir = /var/lib/mysql/
innodb_log_file_size = 16M
query_cache_type = 0
query_cache_size = 0
tmp_table_size = 64M
max_heap_table_size = 64M
table_definition_cache = 8192
table_open_cache = 16384
log-error = /var/log/mysql.error.log
log-warnings = 3
[mysql]
no-auto-rehash
default-character-set = utf8mb4
[mysqldump]
quick
max_allowed_packet = 32M
[isamchk]
key_buffer = 128M
sort_buffer_size = 128M
read_buffer = 2M
write_buffer = 2M
[myisamchk]
key_buffer = 128M
sort_buffer_size = 128M
read_buffer = 2M
write_buffer = 2M
[mysqlhotcopy]
interactive-timeout
!include /etc/my.cnf.d/hwconf.conf
I'm notb sure and have no clue of what i can do to avoid this, i spoke to the hosting and they say that there is some sleeping queries sometimes, not much help there.
Things to do in order:
Check the process list using SHOW PROCESSLIST, see what's slow, maybe it's something obvious. If it's slow you'll see the query.
Check the slow query log.
Use a tool like Percona Toolkit or JetProfiler to zoom in on any problem not obvious from 1 or 2.
I have a simple master-slave config. For an unknown reason, relay log files are created on the master as well as on the slave server. Everything works fine on the slave and the slave delete relay log files as it go however, since the master is not a slave, I do not understand why relay log files are created there too and these relay log files are never deleted neither.
So, am I missing something?
How to instruct the master to stop creating relay log files?
Mariadb version on these master/slave servers is: 10.0.38-MariaDB
Here is my master server cnf config file:
[mysqld]
expire_logs_days=21
innodb_support_xa=1
sync_binlog=1
sync_master_info=1
sync_relay_log=1
sync_relay_log_info=1
binlog_format=mixed
datadir=/data/mysql
max_heap_table_size = 256M
log_bin_trust_function_creators = 1
net_read_timeout = 3600
net_write_timeout = 3600
port = 3306
socket = /var/lib/mysql/mysql.sock
skip-external-locking
key_buffer_size = 384M
max_allowed_packet = 256M
max_connections = 500
table_open_cache = 201000
table_definition_cache = 201000
sort_buffer_size = 2M
read_buffer_size = 2M
read_rnd_buffer_size = 8M
myisam_sort_buffer_size = 64M
thread_cache_size = 128
query_cache_size = 128M
innodb_buffer_pool_size = 16G
slave_compressed_protocol = 1
log-bin=mysql-bin
server-id = 1
Thank you for your time.
I can't find the source of the issue on our Master/Slave replication .
Today i was updating the Master and suddenly got the following error from the slave
Error 'The table 'caching_api' is full' on query.
Query: '
ALTER TABLE `caching_api`
ADD UNIQUE INDEX `id` (`id`) USING BTREE ,
ADD INDEX `search` (`component`, `method`) USING BTREE
It's not a disk issue , the Slave is an exact replicate of the Master
And my.cnf config :
[mysqld]
#
# * Basic Settings
#
user = mysql
pid-file = /var/run/mysqld/mysqld.pid
socket = /var/run/mysqld/mysqld.sock
port = 3306
basedir = /usr
datadir = /data/mysql
tmpdir = /tmp
lc-messages-dir = /usr/share/mysql
skip-external-locking
key_buffer = 16M
max_allowed_packet = 128M
thread_stack = 192K
thread_cache_size = 64
table_open_cache = 3000
join_buffer_size = 128k
# This replaces the startup script and checks MyISAM tables if needed
# the first time they are touched
myisam-recover = BACKUP
max_connections = 4000
wait_timeout = 150
interactive_timeout = 30
innodb_buffer_pool_size = 25G
innodb_log_file_size = 1G
innodb_buffer_pool_instances = 10
tmp_table_size = 256M
max_heap_table_size = 256M
innodb_flush_log_at_trx_commit = 2
query_cache_limit = 64M
query_cache_size = 256M
relay_log_space_limit = 10G
server-id = 2
relay-log = /var/log/mysql/mysqld-relay-bin
expire_logs_days = 1
max_binlog_size = 100M
slave-skip-errors = 1062,1054
[mysqldump]
quick
quote-names
[mysql]
#no-auto-rehash # faster start of mysql but no tab completition
[isamchk]
key_buffer = 16M
Update to questions from comments :
When i try to run the query directly on the SLAVE :
caching_api table status
Slave Disk Info
ibdata1 about 36GB
ulimit -a
The manual doesn't rule out the possibility of a full disk.
If a table-full error occurs, it may be that the disk is full or that
the table has reached its maximum size. The effective maximum table
size for MySQL databases is usually determined by operating system
constraints on file sizes, not by MySQL internal limits.
But since have checked that already before posting this question. So the only other possiblility is that maximum table size has been reached.
I'm running this server for data mining purposes. It runs several compute intensive data mining applications parallely and does simultaneous access to the MySQL server.
Here are the configuarations.
Server config: 8 core Intel Xeon, 16gb RAM, 500 GB SAS drive
MySQL my.cnf
[client]
#password = [your_password]
port = 3306
socket = /var/lib/mysql/mysql.sock
[mysqld]
# generic configuration options
port = 3306
socket = /var/lib/mysql/mysql.sock
datadir = /database/mysql
log_bin = OFF
expire-logs-days = 3
pid-file = /database/mysql/localhost.localdomain.pid
back_log = 50
max_connections = 3000
max_connect_errors = 100
table_open_cache = 2048
max_allowed_packet = 16M
binlog_cache_size = 1M
max_heap_table_size = 64M
read_buffer_size = 128M
read_rnd_buffer_size = 32M
sort_buffer_size = 32M
join_buffer_size = 8M
thread_cache_size = 8
thread_concurrency = 4
query_cache_size = 64M
query_cache_limit = 2M
ft_min_word_len = 4
default-storage-engine = innodb
thread_stack = 192K
transaction_isolation = REPEATABLE-READ
tmp_table_size = 64M
log-bin = mysql-bin
binlog_format = mixed
server-id = 1
key_buffer_size = 32M
bulk_insert_buffer_size = 64M
myisam_sort_buffer_size = 128M
myisam_max_sort_file_size = 10G
myisam_repair_threads = 1
myisam_recover
innodb_additional_mem_pool_size = 32M
innodb_buffer_pool_size = 4G
innodb_data_file_path = ibdata1:10M:autoextend
#innodb_data_home_dir = <directory>
innodb_write_io_threads = 8
innodb_read_io_threads = 8
#innodb_force_recovery = 6
innodb_thread_concurrency = 0
innodb_flush_log_at_trx_commit= 2
#innodb_fast_shutdown
innodb_log_buffer_size = 8M
innodb_log_file_size = 1G
innodb_log_files_in_group = 3
#innodb_log_group_home_dir
innodb_max_dirty_pages_pct = 90
#innodb_flush_method = O_DSYNC
innodb_lock_wait_timeout = 120
[mysqldump
quick
max_allowed_packet = 16M
[mysql]
auto-rehash
[myisamchk]
key_buffer_size = 512M
sort_buffer_size = 512M
read_buffer = 8M
write_buffer = 8M
[mysqlhotcopy]
interactive-timeout
[mysqld_safe]
open-files-limit = 8192
There are only 2 users who access this server which includes me. On peak hour I get this
mysql > show processlist
...
120 rows in set
Which shows that around 120 connections are established to the mysql server during peak computation hours. MySQL consumes around 9.5gb of memory and uses 98-99% CPU which I can still live with. But during this time a front end site build with php/javascript takes around 1 - 2 min to load which is because mysql responds very slowly during these hours. While normally it takes somewhere around 890ms to 4 seconds.
I want to know how to further optimize the mysql server configuration. Currently as can be seen from the posted my.cnf , buffer pool is at 4GB and max number of connections are set at 3000 . All the tables are Innodb with proper indexes, but in my case transaction safety is not a issue the main and the only issue is performance. The data mining applications uses MySQL C API Connector and each has around 24 parallel threads running which equals to 24 simultaneous connections to MySQL
How can I further optimize the mysql server configuration so that I may get a reasonable response time of around 10 - 15 seconds for front end access . Please let me know if there is any way to optimize this further.
You really should dedicate another server just for data mining and set up replication between your MySQL servers. Data mining application should use transactions to union multiple small queries into blocks. This way your site will not wait for other queries to be executed and synchronization would be made in background without visible lag.
Another options is to cache as much as possible and hope that user will not request data that is not in cache when heavy hours are.
But I prefer to do both of this things so you'll have 100% reliable service.
I don't know if this problem is specific to my set up but when I add the line
log = /var/log/mysql.log
to the mysqld section of a copied my-large.cnf file and try to restart the mysql server I get the error
Starting MySQL. ERROR! The server quit without updating PID file (/var/lib/mysql/centos-server.pid).
I've created the file /var/log/mysql.log, set its owner and group to mysql and set the permissions on /var/log to 777 (for the moment)
I'm on centos, with mysql 5.6.5 m8 (the development release).
This is a snippet of the my.cnf file
[mysqld]
port = 3306
socket = /var/lib/mysql/mysql.sock
skip-external-locking
key_buffer_size = 256M
max_allowed_packet = 1M
table_open_cache = 256
sort_buffer_size = 1M
read_buffer_size = 1M
read_rnd_buffer_size = 4M
myisam_sort_buffer_size = 64M
thread_cache_size = 8
query_cache_size= 16M
# Try number of CPU's*2 for thread_concurrency
thread_concurrency = 8
log = /var/log/mysql.log
Seems like the log directive is outdated and my.cnf requires the directive
general-log = 1
If specified like this the log file will be created in a default location (which on centos is /var/lib/mysql/centos-server.log)