I am using Peewee ORM to update and modify database table from my python project.
I have set max connection limit to 15 using:
set global max_connections = 15
to get total connection I run the command,
> select count(*) from information_schema.processlist;
> 12
now that connection limit is 15 and even if I run a my code to do some work on db by opening a connection the number of connection goes up by 2
> select count(*) from information_schema.processlist;
> 14
now even if am done doing the task, I close python terminal I still see total number of connection from process list count to be 14, it seems like old connections get reused or what, if I run the same command to update db table, from different terminal I add 2 more connection but it gives error saying too many connections. but the first terminal I had opened still work.
I can post peewee code if required.
If you are using the regular MySQLDatabase class, then calling .close() on the connection will close it.
On the other hand if you are using PooledMySQLDatabase, .close() will recycle the connection to the pool of available connections. You can manage the connections in the pool using the following APIs: http://docs.peewee-orm.com/en/latest/peewee/playhouse.html#PooledDatabase
Related
I'm looking into aborted connection -
2022-11-21T20:10:43.215738Z 640870 [Note] Aborted connection 640870 to db: '' user: '' host: '10.0.0.**' (Got timeout reading communication packets)
My understanding is that I need to figure out whether it is an interactive or not connection, and increase wait_timeout (or interactive_timeout) accordingly. If it has no effect, then I'll need to adjust net_read_timeout or net_write_timeout and see.
I'd like to ask:
Is there a meta table that I can query for the connection type
(interactive or not)?
There are how-to's on the internet on adjusting wait_timeout (or
interactive_timeout) and all of them have rebooting the database as
the last step. Is that really required? Given that immediate effect
is not required, the sessions are supposed to come and go, and new
sessions will pick up the new value (after the system value is set),
I suppose if there is a way to track how many connections are left
with the old values, then it will be ok?
Finally, can someone suggest any blog (strategy) on handling aborted
connection or adjusting the timeout values?
Thank you!
RDS MySQL version 5.7
There is only one client that sets the interactive flag by default: the mysql command-line client. All other client tools and connectors do not set this flag by default. You can choose to set the interactive flag, because it's a flag in the MySQL client API mysql_real_connect(). So you would know if you did it. In some connectors, you aren't calling the MySQL client API directly, and it isn't even an option to set this flag.
So for practical purposes, you can ignore the difference between wait_timeout and interactive_timeout, unless you're trying to tune the timeout of the mysql client in a shell window.
You should never need to restart the MySQL Server. The timeout means the client closed the session after there has been no activity for wait_timeout seconds. The default value is 28800, which is 8 hours.
The proper way of handling this in application code is to catch exceptions, reconnect if necessary, and then retry whatever query was interrupted.
Some connectors have an auto-reconnect option. Auto-reconnect does not automatically retry the query.
In many applications, you are borrowing a connection from a connection pool, and the connection pool manager is supposed to test the connection before returning it to the caller. For example running SELECT 1; is a common test. The action of testing the connection causes a reconnect if the connection was not used for 8 hours.
If you don't use a connection pool (for example if your client program is PHP, which doesn't support connection pools as far as I know), then your client opens a new connection on request, so naturally it can't be idle for 8 hours if it's a new connection. Then the connection is closed as the request finishes, and presumably this request lasts less than 8 hours.
So this comes up only if your client opens a long-lived MySQL connection that is inactive for periods of 8 hours or more. In such cases, it's your responsibility to test the connection and reopen it if necessary before running a query.
I am using .net core 3.1 with mysql connector 8.0.21.
In mysql i have max_connection around 3000 and max_user_connection as 0 (unlimited)
In my connection pools setting in various application my total connection reserved are around 2800.
Still sometime i am getting this above error occasionally.
Any idea what could be the problem.
In my application there are two implementation for db access. one is dapper and other is old vanilla ado.net implementation.
Strange thing is when we queried the number of connection with all users via query
SELECT IFNULL(usr,'All Users') USER,IFNULL(hst,'All Hosts') HOST,COUNT(1) Connections
FROM
(
SELECT USER usr,LEFT(HOST,LOCATE(':',HOST) - 1) hst
FROM information_schema.processlist
WHERE USER NOT IN ('system user','root')
) A GROUP BY usr,hst WITH ROLLUP;
Result is well under the limits of connection. nothing much in mysql error logs as well
https://bugs.mysql.com/bug.php?id=24761
Our server also have setup nagios
Problem was due to third aprty monitoring tool nagios doing telnet on ip/port of mysql server
Problem was removed after removing the monitoring and doing alternative way for monitoring as mentioned in
https://serverfault.com/questions/578682/monit-mysql-server-blocked-because-of-too-many-bad-connections
How to find out MYSQL connection time not connection_timeout?
I am able to find out the connection timeout value using
SHOW VARIABLES LIKE 'connect_timeout'
But I am not able to get the variable for connection time.
for some reason when I open a connection the the Percona MySQL database on my HostGator website, after fetching the query, it will disconnect/ close the connection about 10 seconds later.
I typically wouldn't care, but HeidiSQL freezes up, preventing exporting or sorting the returned rows with it's UI unless I connect again.
Any thoughts on making the connection last longer? is it something I can do myself, or will it require a dedicated server or some upgrade? (I'm currently on a shared one). Thanks!
Sounds like it may be the "wait" timeout on the MySQL connection.
SHOW VARIABLES LIKE 'wait_timeout'
That's the amount of time (in seconds) that MySQL will leave the session (the database connection) open while it's idle, waiting for another statement to be issued. After this amount of time expires, MySQL can close the connection.
You should be able to change this for a session, to change the timeout to 5 minutes
SET wait_timeout = 300
Verify the setting with the SHOW VARIABLES statement again.
NOTE: This is per connection. It only affects the current session. Every new connection will inherit their own wait_timeout value from the global setting.
(This is only a guess. There's insufficient information in the question to make a precise diagnosis. It could be something other than MySQL server that's closing the database connection, e.g. it could be your connection pool settings (if you are using a connection pool).
We have 8 phusion passengers with 20 connections each. should be 160 max connection. Today our Mysql connection crossed 300 and our server stopped responding.
What would happen if the thread dies unnaturally ? how do db connections associated with it get cleaned-up?
How to debug this type of scenario ?
What would happen if the thread dies unnaturally ?
If you are using your statements under transaction then all statements under un-successful transactions will be rolled back but if you are executing your statements individually then how many have been executed will be saved in db but other un-successful will not and data can be in-consistant.
how do db connections associated with it get cleaned-up?
As per my assumption your application is opening connections more than required and also not closing properly. In this case if you check connections in mysql admininstrator then you will get so many connections in sleep mode. So you can kill all sleep connections to clean them.
How to debug this type of scenario ?
Step1: Enable general logs on your server:
Step2: execute below command in any gui tool like sqlyog etc.
show full processlist;
Step3: Take first sleep process from above command:
Step3: find/grep above process id (suppose it is 235798) in your general log. You can use below command.
$ cat /var/log/mysqldquery.log | grep 235798
Note: there can be different file name and path for your general log file.
Above command will show you few lines, check if you are closing connection at the end, should show "quite" statement at the end of line. In this way you need to check few processes those are in sleep mode and you can jugde which type statements/from which module opening extra connectiions and not closing and accordingly you can take action.