I have a tomcat app connecting to a MySQL Db / java application
I keep getting
Packet for query is too large 1080>1024
I tried changing my.cnf:
in my.cnf the Max packet size is defined as 50 MB and
socket = /var/run/mysqld/mysqld.sock
port = 3306
basedir = /usr
datadir = /var/lib/mysql
tmpdir = /tmp
lc-messages-dir = /usr/share/mysql
lower_case_table_names = 1
skip-external-locking
bind-address = 0.0.0.0
key_buffer = 16M
max_allowed_packet = 50M
thread_stack = 192K
thread_cache_size = 8
group_concat_max_len=100000
innodb_lock_wait_timeout=300
innodb_buffer_pool_size=22G
innodb_locks_unsafe_for_binlog = ON
innodb_additional_mem_pool_size=40M
I have even tried passing the Param as part of the connection string
jdbc:mysql://serverIP:3306/dbname?maxAllowedPacket=2048000
to the jdbc driver - still I keep getting
Packet for query is too large 1080>1024
This keeps coming every few hours.
What should I check?
MySQL version is 5.5
Thanks for the help.
For those who just want to temparery increase the size of max_allowed_packet and don't want to make this change permanent, try execute sql:
use your_db;
set global max_allowed_packet = 1024*1024*10; # set size to 10M
to verify whether it takes effect or not, you need to open a new query session and execute:
show VARIABLES like '%max_allowed_packet%';
Please note this change is temparery and will restore to default when mysql restart.
It looks like something isn't configured right... Did you restart MySQL after making the config changes? You have to configure the size on both ends:
Both the client and the server have their own max_allowed_packet variable, so if you want to handle big packets, you must increase this variable both in the client and in the server.
However, the server's default size is 1mb (1024 kb). Given your error says 1080>1024 I'm guessing your configuration changes didn't take place (at least not on both client and server).
This happens because either a query, row or a binary log event are greater than 1mb:
A communication packet is a single SQL statement sent to the MySQL server, a single row that is sent to the client, or a binary log event sent from a master replication server to a slave.
I'd try bumping to 5mb on the client and the server (including your replication instance if you have one). You'll have to bounce MySQL for changes to take effect.
https://dev.mysql.com/doc/refman/5.5/en/packet-too-large.html
Related
I'm facing an issue that the only way I found myself to solve it is making a factory reset of my machine.
The problem I have is that the inserts, updates, deletes and schema modifications are taking a lot of time. The disk goes up to 100%, but performs with very slow speeds. This happens when making modifications by:
Restoring backups from Workbench (speeds up to 5MB/s)
Applying migrations and seeding a new db from a NodeJS process (speeds up to 1MB/s): this process used to be finished in 2 minutes, but now it takes about 30/40 minutes.
I think this started happening when I started trying to restore big backups (like 14GB of data). I believe this is not a coincidence.
The computer works great. Copying and pasting big files speeds the disk up to 400MB/s, what is a great performance. I use it for gaming and never see any related issues, so seems like the problem only happens with MYSQL.
My environment is:
Razer Blade 15, 16GB RAM, 512 GB SSD NvME (CA5-8D512), I7 10TH processor
Windows 10
WSL 2 (Ubuntu 20.04)
MySql 8.0 (running on WSL)
MySQL Workbench (running on Windows)
My mysqld.cnf file looks like this:
[mysqld_safe]
socket = /var/run/mysqld/mysqld.sock
nice = 0
[mysqld]
user = mysql
pid-file = /var/run/mysqld/mysqld.pid
socket = /var/run/mysqld/mysqld.sock
port = 3308
basedir = /usr
datadir = /var/lib/mysql
tmpdir = /tmp
lc-messages-dir = /usr/share/mysql
skip-external-locking
skip-log-bin
bind-address = 127.0.0.1
mysqlx-bind-address = 127.0.0.1
key_buffer_size = 16M
max_allowed_packet = 16M
thread_stack = 192K
thread_cache_size = 8
myisam-recover-options = BACKUP
log_error = /var/log/mysql/error.log
max_binlog_size = 100M
expire_logs_days = 10
sql_mode=''
default-time-zone=+00:00
Additional information: I tried also using Ubuntu OS, but the same issue happened.
Schema changes are supposed to be rare. You seem to be doing them a lot.
The reason 8.0 is significantly slower than previous versions is that schema changes are now ACID, and can be rolled back. This necessitates hanging on to dropped tables, column changes, etc. That costs a lot of I/O.
Meanwhile, I suggest that the inserts, updates, and deletes are not any slower than in previous versions.
If you rarely load copies of the dataset, then consider LVM techniques. They are orders of magnitude faster than any "restore" -- even before 8.0.
I have the same effect that MySQL 8.0 ist much slower than 5.7.
To prepare an upgrade I put an 8.0 as a replication secondary to the 5.7 production primary.
The 8.0 ist unable to keep up with primary changes, seconds-behind-master is growing and growing.
When I change the secondary to a 5.7 the replication works fine.
I have looked at many similar questions to this but I can't seem to find the answer. I would like to set up the slow query log for my MySQL database. I have seen many answers saying I should access the MySQL command line tool. I am not sure exactly how to find this tool but I tried accessing it by going to:
c:/xampp/mysql/bin/mysql -u root -p -h localhost
But here I get MariaDB, which seems to be different from any other answers/tutorials I have seen before. Typing in:
set log_slow_queries = ON;
gives me the error
ERROR 1193 (HY000): Unknown system variable 'log_slow_queries'
SET GLOBAL slow_query_log=1;
The Slow Query Log consists of log events for queries taking up to long_query_time seconds to finish. For instance, up to 10 seconds to complete. To see the time threshold currently set, issue the following:
SELECT ##long_query_time;
+-------------------+
| ##long_query_time |
+-------------------+
| 10.000000 |
+-------------------+
It can be set as a GLOBAL variable, in my.cnf or my.ini file. Or it can be set by the connection, though this is unusual. The value can be set between 0 to 10 (seconds). What value to use?
10 is so high as to be almost useless;
2 is a compromise;
0.5 and other fractions are possible;
0 captures everything; this could fill up disk dangerously fast, but can be very useful.
The capturing of slow queries is either turned on or off. And the file logged to is also specified. The below captures these concepts:
SELECT ##slow_query_log; -- Is capture currently active? (1=On, 0=Off)
SELECT ##slow_query_log_file; -- filename for capture. Resides in datadir
SELECT ##datadir; -- to see current value of the location for capture file
SET GLOBAL slow_query_log=0; -- Turn Off
-- make a backup of the Slow Query Log capture file. Then delete it.
SET GLOBAL slow_query_log=1; -- Turn it back On (new empty file is created)
For more information, please see the MySQL Manual Page The Slow Query Log
Note: The above information on turning on/off the slowlog was changed in 5.6(?); older version had another mechanism.
The "best" way to see what is slowing down your system:
long_query_time=...
turn on the slowlog
run for a few hours
turn off the slowlog (or raise the cutoff)
run pt-query-digest to find the 'worst' couple of queries. Or mysqldumpslow -s t
Go to xampp control panel click on config button for mysql and select my.ini then add these lines in my.ini file
slow_query_log = 1
slow-query-log-file=/path/of/the/log/file.log
I put above two lines under the log_error = "mysql_error.log". the modified part of the my.ini file should look like this
# The MySQL server
[mysqld]
port= 3306
socket = "C:/xampp/mysql/mysql.sock"
basedir = "C:/xampp/mysql"
tmpdir = "C:/xampp/tmp"
datadir = "C:/xampp/mysql/data"
pid_file = "mysql.pid"
# enable-named-pipe
key_buffer = 16M
max_allowed_packet = 1M
sort_buffer_size = 512K
net_buffer_length = 8K
read_buffer_size = 256K
read_rnd_buffer_size = 512K
myisam_sort_buffer_size = 8M
log_error = "mysql_error.log"
slow_query_log = 1
slow-query-log-file=/var/log/mysql-slow.log
Then Restart the MySQL server in xampp control panel. and now slow_query_log should be enabled, you can confirm it by running following command in the MySQL shell
show variables like '%slow%';
It might be obvious but it took me time before I realized my mistake: in the my.ini file you should put the slow_query_log settings in the [mysqld] group, not simply at the end of the my.ini file....
I want to increase max_allowed_packet variable on my server and I don't know why, when I add the sentence in /etc/my.cnf, it doesn't work!! This is the line I added:
max_allowed_packet=62M
But when I check the current value through mysqladmin variables, it shows 10M!! I've even restarted my server! And nothing happens!
What happens here?
The settings file will normally have several sections (what MySQL calls groups). You're possibly disregarding that fact:
# The following options will be passed to all MySQL clients
[client]
port = 3306
socket = /tmp/mysql.sock
# Here follows entries for some specific programs
# The MySQL server
[mysqld]
port = 3306
socket = /tmp/mysql.sock
skip-external-locking
key_buffer_size = 384M
max_allowed_packet = 1M
I am hoping that someone on here can help with an issue I am having on a webserver. Currently the server hosts a web application which we have purchased for helpdesking. Every 8 hours when someone navigates to the page they will be presented with this error:
[Warning]: PDO::__construct(): MySQL server has gone away (Database/class.SWIFT_Database.php:334)
I have tried setting the wait_timeout to the max value in the /etc/my.cnf file ie;
wait_timeout=31536000
Also from the mysql I have set the global wait_timeout to this value and set the session wait_time out to the same.
I have noticed some strange behaviour, when you restart the mysql service, the wait timeout resets to the 28800 default, despite it being set in the my.cnf. I can't find any other config files for mysql though happy to look if someone can point me in the direction.
Also this morning when I logged into the server and ran the following command, the session wait_timeout value had reverted!
mysql> select ##global.wait_timeout, ##session.wait_timeout;
+-----------------------+------------------------+
| ##global.wait_timeout | ##session.wait_timeout |
+-----------------------+------------------------+
| 31536000 | 28800 |
+-----------------------+------------------------+
1 row in set (0.00 sec)
I see that this is an issue for a lot of people online with various webapps running on mysql, but no one seems to have a fix. A lot of advice online points to the wait_timeout but it doesn't seem to be changing the bug that I am seeing. I have tried out the fixes on the mysql manual about this issue, but still no luck (link: http://dev.mysql.com/doc/refman/5.0/en/gone-away.html)
Any advice would be greatly appreciated. Server details and product versions below:
Server: OpenSuse 11.4
MySql Version: 5.1.53
Many Thanks In Advance!
While not very elegant, the following snippet has helped me get rid of the timeouts and keep persistent connections. It will throw the exception if the connection fails $limit times in a row, though if the problem is timeouts you will only need 1 retry at most.
$db = null;
$limit = 10;
$counter = 0;
while (true) {
try {
$db = new PDO('mysql:host=' . db_host . ';dbname=' . db_name, db_user, db_pass);
$db->exec( "SET CHARACTER SET utf8" );
$db->setAttribute( PDO::ATTR_DEFAULT_FETCH_MODE, PDO::FETCH_ASSOC );
$db->setAttribute( PDO::ATTR_PERSISTENT, true );
break;
}
catch (Exception $e) {
$db = null;
$counter++;
if ($counter == $limit)
throw $e;
}
}
This article on Drupal.org may be helpful in your situation:
MySQL comes with a default configuration of the resources it is going to use, specified in "my.cnf" (Linux) or "my.ini" (Windows) during the installation of MySQL.
In Linux this file is located at /etc/my.cnf to set global options, or /usr/local/var/mysql-data-dir/my.cnf to set server-specific options.
In Windows this file is located by default at C:\Program Files\MySQL\MySQL Server X.Y\my.ini.
Resources allowed by the default configuration are normally insufficient to run a resource-intensive application. You must modify the following resource specifications if they are available in your original configuration file, or add them to the configuration file if they are not already specified (because some are not present by default) :
Important: Remember to keep backup files before you do anything! You will also have to reload the MySQL service after making changes to these configuration files.
MyISAM specifications:
[mysqld]
port = 3306
socket = /tmp/mysql.sock
skip-external-locking
key_buffer = 384M
max_allowed_packet = 64M
table_cache = 4096
sort_buffer_size = 2M
read_buffer_size = 2M
read_rnd_buffer_size = 64M
myisam_sort_buffer_size = 64M
thread_cache_size = 8
query_cache_size = 32M
InnoDB specifications:
innodb_buffer_pool_size = 384M
innodb_additional_mem_pool_size = 20M
innodb_log_file_size = 10M
innodb_log_buffer_size = 64M
innodb_flush_log_at_trx_commit = 1
innodb_lock_wait_timeout = 180
Note: It is assumed here that you are using the InnoDB database tables, as Drupal is a resource intensive application. If you are not using the InnoDB database tables try to change this, in view of the fact that you are getting the Warning: MySQL server has gone away - apparently meaning that your setup is resource intensive.
There are various reasons that a connection gets closed.
Reference:
https://dev.mysql.com/doc/refman/5.0/en/gone-away.html
I too faced the similar problem on using PDO where the hosting administrator kills the connection if it sleeps more than a minute. Hence I came up with my own class which will wrap the PDO class. This will detect whether a connection is closed and will try to reconnect on query execution.
Answer Below
PDO: MySQL server has gone away
I am running a drupal site. I got an error in my site user warning: Got a packet bigger than 'max_allowed_packet' bytes query. I have set the value as high as 128M . Even after that same error is reported.
What is the issue here?? Why is it not working ??
Is there a maximum limit for the value max_allowed_packet ?
This is the bleeding edge:
set global max_allowed_packet=1073741824;
Although, it is probably not a good idea to set it that high in your case.
As a side note, I experienced this error with mysqldump, and setting this maximum didn't help. This did the trick: $ mysqldump --max_allowed_packet=999M -u root -p table_name > table_name.sql
Often this can be caused by the variables not actually taking effect- you make the configuration change, but in the wrong my.cnf, or you forget to bounce the app, etc.
An easy way to check a running mysql instance is to do something like this in a shell:
mysqladmin variables -u root -p
and enter in your root password. This will dump all of the current variables (including max_allowed_packet), and will let you verify what it's set to. If it's set to 128M and you're still choking on it, then you'll need to increase it- but it's pretty unlikely.
first is you need to set your max_allowed_packet to 128M in your my.cnf file.
to find it, use "locate my.cnf" command in your command line.
the file should look like this:
#
!includedir /etc/my.cnf.d
#max_allowed_packet = 1024M
[mysqld]
port = 3306
key_buffer_size = 256M
# max_allowed_packet = 100M
table_open_cache = 256
sort_buffer_size = 1M
read_buffer_size = 1M
read_rnd_buffer_size = 4M
myisam_sort_buffer_size = 64M
thread_cache_size = 8
query_cache_size= 16M
thread_concurrency = 8
bind-address = 202.90.158.47
# skip-networking
log = /var/log/mysql.access.log
log-error = /var/log/mysql.error.log
wait_timeout = 1
[mysqldump]
#max_allowed_packet = 101M
be sure to uncomment(remove the # sign before the max_allowed_packet = 128M line)
and finally, restart your sql using command "/etc/init.d/mysqld restart"
that should do the trick. :D
Edit your /etc/my.cnf, adding the max_allowed_packet variable.
It should look like this:
[mysqld]
max_allowed_packet=1000000000
Then restart your server.
Try setting max_allowed_packet = 128M as the very last option under the [mysqld] category of my.cnf.
When I had it as the first option, it would not work, but when I had it as the last option, it worked! I think this is because some other variables were over-riding max_allowed_packet.
After changing my.cnf, restart MySQL using sudo service mysql restart, and check the setting using SHOW VARIABLES LIKE 'max_allowed_packet';
You need to set the setting in all sections that apply to the action you are doing, and always in the [MySQLd] section. The setting applies to the buffer of the elements you are using. So under [MySQLd] is for your MySQL server proces deamon on linux / service on windows. And if you want to make a dump with MySQLDump add it as parameter on the command line or make a section [MySQLDump] in your my.ini as well for this tool with the same parameter to make it permanent. If you want to import the dump again with MySQL again use the parameter on the command line or make a section [MySQL] with again the same parameter in your my.ini to make the choice permanent for this tools also.
I kept on talking about my.ini because i am on windows but on linux that is my.cnf of course.
I decided to explain it here because it took me ages to figure this out because it is not explained anywhere. In examples however i noticed some ppl having the setting under multiple sections so i started to google more and found correlation between the sections and the actions they where doing. Now i never have this problem anymore and settings as high as the mentioned 128M here are not needed in most cases. However because it's the maximum the server will use for this buffer if you have the memory just set it high enough to never get into trouble with your actions. The size you actually need is a little bit larger than the largest record in your database.
I experienced this error with mysqldump with LONGBLOB fields, and setting this maximum didn't help. This did the trick:
$ mysqldump --max_allowed_packet=999M -u root -p table_name > table_name.sql