I used mysql 5.5 in my project and encontered a problem that I can't solve. I googled but without result. My client can connect the server normally however after a while about 10 minutes the connection lost. The client would reconnect the server but it hangs up and never return a connection. I tried with two kinds of clients: The native mysql console client and the mysql workbench and same result. Here are some details:
MariaDB [(none)]> show variables like "%timeout%";
+----------------------------+----------+
| Variable_name | Value |
+----------------------------+----------+
| connect_timeout | 10 |
| deadlock_timeout_long | 50000000 |
| deadlock_timeout_short | 10000 |
| delayed_insert_timeout | 300 |
| innodb_lock_wait_timeout | 50 |
| innodb_rollback_on_timeout | OFF |
| interactive_timeout | 28800 |
| lock_wait_timeout | 31536000 |
| net_read_timeout | 30 |
| net_write_timeout | 60 |
| slave_net_timeout | 3600 |
| thread_pool_idle_timeout | 60 |
| wait_timeout | 28800 |
+----------------------------+----------+
so the wait_timeoutand interactive_timeout seems normal and I can't get some other logs about the problem.
EDIT: I find this problem only occures when I connect the server via remote clients. The local client doesn't have the problem.
Related
I have a remote MySQL database, and my deployed application (a few java microservice applications) consistently encounters the error ERROR 2013 (HY000): Lost connection to MySQL during query with some SQL queries. Then I tried to connect to mysql with MySQL command line client, executed the same query, and I managed to replicate the same error. Below is Mysql setup after following this page https://dev.mysql.com/doc/refman/5.7/en/error-lost-connection.html
+------------------------------+----------+
| Variable_name | Value |
+------------------------------+----------+
| connect_timeout | 3600 |
| delayed_insert_timeout | 300 |
| have_statement_timeout | YES |
| innodb_flush_log_at_timeout | 1 |
| innodb_lock_wait_timeout | 5 |
| innodb_rollback_on_timeout | OFF |
| interactive_timeout | 7200 |
| lock_wait_timeout | 50 |
| net_read_timeout | 3000 |
| net_write_timeout | 6000 |
| rpl_semi_sync_master_timeout | 10000 |
| rpl_stop_slave_timeout | 31536000 |
| slave_net_timeout | 4 |
| wait_timeout | 7200 |
+------------------------------+----------+
+--------------------------+------------+
| Variable_name | Value |
+--------------------------+------------+
| max_allowed_packet | 1073741824 |
| slave_max_allowed_packet | 1073741824 |
+--------------------------+------------+
However, I systematically have the same error after roughly 10 minutes after executing a big SQL query from the mysql command line client.
Note: On the DB server side, I notice the log [Note] Aborted connection xxxxx to db: 'xxxxx' user 'xxxxx' host 'xxxxx' (Got an error writing communication packets)
DBA executed the query successfully on the server on which mysql is installed. But we have to use mysql remotely as a service.
I will be grateful if anybody can help.
Finally, the root cause was a network issue. Our remote MySQL instance is exposed as service. There's an additional proxy between the client and MySQL DB. By default, the connection will be closed if no communication between client and server over 10 mins.
Like setup MySQL connection with MYSQL_OPT_RECONNECT is true in this way
my_bool reconnect = true;
mysql_options(ptrMYSQL, MYSQL_OPT_RECONNECT, &reconnect);
then raised one SELECT immediately, log file shows less than 1ms, performed OK.
295ms later, after did some other work, the process raised another query with the same ptrMYSQL, this one failed for the reason
errno 2013 error Lost connection to MySQL server during query
Since I had the AUTO RECONNECT option on, how can MySQL query failed in such way?
This is what timeout variables look like:
mysql> show variables like '%timeout%';
+------------------------------+-----------+
| Variable_name | Value |
+------------------------------+-----------+
| connect_timeout | 10 |
| delayed_insert_timeout | 300 |
| have_statement_timeout | YES |
| innodb_flush_log_at_timeout | 1 |
| innodb_lock_wait_timeout | 8 |
| innodb_rollback_on_timeout | OFF |
| interactive_timeout | 1800 |
| lock_wait_timeout | 31536000 |
| net_read_timeout | 30 |
| net_write_timeout | 60 |
| rpl_semi_sync_master_timeout | 999999999 |
| rpl_stop_slave_timeout | 31536000 |
| slave_net_timeout | 30 |
| wait_timeout | 1800 |
+------------------------------+-----------+
14 rows in set (0.00 sec)
And the version:
mysql> select version();
+------------+
| version() |
+------------+
| 5.7.17-log |
+------------+
I have a large amount of connections but when I issue a show full processlist I am not showing anything close to the connections I see. Are these connections orphans of some sort? I tried the flush hosts command and the connections persist, even with a reboot of the server and also restarting the mysql server.
I believe these connections are causing issues with making new connections to the database. User's are getting a "server went away" error. How do I clear these?
See commands below:
mysql> show status like '%onn%';
+--------------------------+-------+
| Variable_name | Value |
+--------------------------+-------+
| Aborted_connects | 5 |
| Connections | 11743 |
| Max_used_connections | 24 |
| Ssl_client_connects | 0 |
| Ssl_connect_renegotiates | 0 |
| Ssl_finished_connects | 0 |
| Threads_connected | 6 |
+--------------------------+-------+
7 rows in set (0.00 sec)
mysql> show full processlist;
+-------+---------+----------------------+--------------------+---------+-------+-------+-----------------------+
| Id | User | Host | db | Command | Time | State | Info |
+-------+---------+----------------------+--------------------+---------+-------+-------+-----------------------+
| 4494 | rode | localhost:43411 | NULL | Sleep | 11159 | | NULL |
| 4506 | rode | localhost:43423 | information_schema | Sleep | 11159 | | NULL |
| 4554 | rode | localhost:43511 | performance_schema | Sleep | 11112 | | NULL |
| 11500 | ass | serv:1243 | Home-Tech | Sleep | 0 | | NULL |
| 11743 | root | localhost | NULL | Query | 0 | NULL | show full processlist |
| 11744 | ass | out:6070 | Home-Tech | Sleep | 4 | | NULL |
| 11745 | ass | out:6074 | HTGlobal | Sleep | 8 | | NULL
The MySQL server has gone away (error 2006) has two main causes
Server timed out and closed the connection. To fix, check that “wait_timeout” mysql variable in your my.cnf configuration file is large enough.
Server dropped an incorrect or too large packet. If mysqld gets a packet that is too large or incorrect, it assumes that something has gone wrong with the client and closes the connection. To fix, you can increase the maximal packet size limit “max_allowed_packet” in my.cnf file, eg. set max_allowed_packet = 128M, then sudo /etc/init.d/mysql restart.
there are two main ways to fix this. if the above change doesn't there may be an issue with your linux or windows mysql database server; you either need to increase ram on your server or watch it's process.
is this on a windows or linux box?
I run this command:
mysql> show status like '%onn%';
+--------------------------+-------+
| Variable_name | Value |
+--------------------------+-------+
| Aborted_connects | 0 |
| Connections | 37226 |
| Max_used_connections | 45 |
| Ssl_client_connects | 0 |
| Ssl_connect_renegotiates | 0 |
| Ssl_finished_connects | 0 |
| Threads_connected | 4 |
+--------------------------+-------+
I know I don't have a high traffic site, so the Connections number is confusing me.
I'm positive I close all my connections when I use PHP.
Are there other reasons why Connections keep growing?
As documented here:
Connections
The number of connection attempts (successful or not) to the MySQL
server.
It is not the number of simultaneous connections, but a cumulative number.
You can Run Below command to know Traffic on your server.
Show global status like '%Thread%';
I have a problem, my app have to open a lot of connections with MySQL and write data into it, but for after some time I'm getting more and more connections that are in idle/sleep state, and my app's speed reduces until it completely stops. I blame 'wait_timeout' variable to be too high,
mysql> show variables like '%timeout%';
+----------------------------+----------+
| Variable_name | Value |
+----------------------------+----------+
| connect_timeout | 10 |
| delayed_insert_timeout | 300 |
| innodb_lock_wait_timeout | 50 |
| innodb_rollback_on_timeout | OFF |
| interactive_timeout | 28800 |
| lock_wait_timeout | 31536000 |
| net_read_timeout | 30 |
| net_write_timeout | 60 |
| slave_net_timeout | 3600 |
| wait_timeout | 28800 |
+----------------------------+----------+
10 rows in set (0.05 sec)
and
mysql> SHOW FULL PROCESSLIST;
+------+----------+----------------------------------------------------+-------------+---------+------+-------+-----------------------+
| Id | User | Host | db | Command | Time | State | Info |
+------+----------+----------------------------------------------------+-------------+---------+------+-------+-----------------------+
| 5425 | root | ip-10-xxxxxxxx.ec2.internal:60544 | my_db | Sleep | 2344 | | NULL |
| 5426 | root | ip-10-xxxxxxxx.ec2.internal:60561 | my_db | Sleep | 2136 | | NULL |
And there are a lot of connections like this.
So my primary question is how to reset connection? I know I can just kill PID but I would really like to reset it.
And do you think wait_timeout would solve an issue. By the way everything is on the Amazon RDS.
I would suggest that wait_timeout is probably the wrong solution, because, as #datasage implied, your application may not handle it gracefully. The solution is for your application to properly close the connections when it is finished with them.
KILL thread_id is the correct way to reset it from the server side, but that's just a patch for the actual solution, assuming this is actually what's slowing down your application.
I commonly have hundreds of sleeping threads on my MySQL servers, because the application holds them in a pool for reuse. Sleeping threads are sleeping. They aren't slowing anything down unless you have so many of them that you're hitting the max_connections limit on RDS.
You could reduce wait_timeout but you will need to be confident that your application can handle a connection being closed by the server instead of the client.