I am using .net core 3.1 with mysql connector 8.0.21.
In mysql i have max_connection around 3000 and max_user_connection as 0 (unlimited)
In my connection pools setting in various application my total connection reserved are around 2800.
Still sometime i am getting this above error occasionally.
Any idea what could be the problem.
In my application there are two implementation for db access. one is dapper and other is old vanilla ado.net implementation.
Strange thing is when we queried the number of connection with all users via query
SELECT IFNULL(usr,'All Users') USER,IFNULL(hst,'All Hosts') HOST,COUNT(1) Connections
FROM
(
SELECT USER usr,LEFT(HOST,LOCATE(':',HOST) - 1) hst
FROM information_schema.processlist
WHERE USER NOT IN ('system user','root')
) A GROUP BY usr,hst WITH ROLLUP;
Result is well under the limits of connection. nothing much in mysql error logs as well
https://bugs.mysql.com/bug.php?id=24761
Our server also have setup nagios
Problem was due to third aprty monitoring tool nagios doing telnet on ip/port of mysql server
Problem was removed after removing the monitoring and doing alternative way for monitoring as mentioned in
https://serverfault.com/questions/578682/monit-mysql-server-blocked-because-of-too-many-bad-connections
Anyone knows what this error can be?
It happens when I try to connect to an instance.
MySQL is on.
Thank you very much.
Make sure that your firewall is not blocking anything regarding MySQL. Try to do what you are doing by shutting down your antivirus for 5 minutes. It can simply be your firewall.
To interpret the error you are getting, it simply means that your MySQL client not getting to connect to the MySQL server engine.
Another possibility:
Open your mySQL Client and verify that it si set to use the default port number which is 3306. Another thing to lookup would be the bind-address in my.cnf. Set it with the following bind-address = 0.0.0.0 in mysql/my.cnf
The problem was some heavy SQLs being executed. After solving the SQL problems everything got back to normal.
For some reason I am having a problem with too many connections. I close my connections when I am done with them.
After the connections pool hits 100. I get the "too many connections" error and it closes the current connection.
When I run the command show proccesslist, it show 100 connections, but when I run "show variables like 'max_connections'" it shows the max connections is 500.
My questions is why, why am I getting this error when my max connections are set to 500? Any ideas?
You need to do changes in "my.cnf",
max_connections = 500
Note : if this line is not in my.cnf then add this line.
For MySQL 5.7 you find the configuration file under: C:\ProgramData\MySQL\MySQL Server 5.7. Which is my.ini. Edit it and set the write parameter for connections number, as:
# The maximum amount of concurrent sessions the MySQL server will
# allow. One of these connections will be reserved for a user with
# SUPER privileges to allow the administrator to login even if the
# connection limit has been reached.
max_connections=500
For some versions of MySQL, there is apparently an open_file_limit that can overrided the max_connections limit. Also note the this can also be affected by ulimit -n.
See http://blog.endpoint.com/2013/12/increasing-mysql-55-maxconnections-on.html for more details.
I am continuously getting below error while connecting to mysql-
Can\'t connect to MySQL server on /ip/address
but mysql is not generating any log for this error. I have below entries in my.cnf file
log=/var/log/mysql_err.log
log-error=/var/log/mysql/mysql_error.log
I am using correct credentials. No issue of conections.
Now how will I determine the exact issue If I'll not get any log ?
Any Idea, how to solve this ?
This is not surprising:
the server can only log an incident it is aware of. However your client does not even reach the server at all! So how should the server know some client has attempted to contact it?
The error message you get clearly indicates that you do have a connection issue.
You can easily make a test to check the most common problems: just open a telnet connection from the system trying to connect to the system the sql server runs on, connect to the mysql port: telnet <ip-of-mysql-server> mysql
On typical unixoid systems "mysql" will be substituted by the "well known port number of mysql, which is 3306. otherwise you have to specify it manually. Do you get a connection at all? I would guess not. This means either the mysql server is not listening where expected (not running or configured otherwise) or the connection is blocked on network level (firewall).
After our Ruby on Rails application has run for a while, it starts throwing 500s with "MySQL server has gone away". Often this happens overnight. It's started doing this recently, with no obvious change in our server configuration.
Mysql::Error: MySQL server has gone away: SELECT * FROM `widgets`
Restarting the mongrels (not the MySQL server) fixes it.
How can we fix this?
Ruby on Rails 2.3 has a reconnect option for your database connection:
production:
# Your settings
reconnect: true
See:
Ruby on Rails 2.3 Release Notes, sub section 4.8 Reconnecting MySQL Connections.
MySQL auto-reconnect revisited
Good luck!
This is probably caused by the persistent connections to MySQL going away (time out is likely if it's happening over night) and Ruby on Rails is failing to restore the connection, which it should be doing by default:
In the file vendor/rails/actionpack/lib/action_controller/dispatcher.rb is the code:
if defined?(ActiveRecord)
before_dispatch { ActiveRecord::Base.verify_active_connections! }
to_prepare(:activerecord_instantiate_observers) {ActiveRecord::Base.instantiate_observers }
end
The method verify_active_connections! performs several actions, one of which is to recreate any expired connections.
The most likely cause of this error is that this is because a monkey patch has redefined the dispatcher to not call verify_active_connections!, or verify_active_connections! has been changed, etc.
Try ActiveRecord::Base.connection.verify! in Ruby on Rails 4. Verify pings the server and reconnects if it is not connected.
I had this problem when sending really large statements to MySQL. MySQL limits the size of statements and will close the connection if you go over the limit.
set global max_allowed_packet = 1048576; # 2^20 bytes (1 MB) was enough in my case
As the other contributors to this thread have said, it is most likely that MySQL server has closed the connection to your Ruby on Rails application because of inactivity. The default timeout is 28800 seconds, or 8 hours.
set-variable = wait_timeout=86400
Adding this line to your /etc/my.cnf will raise the timeout to 24 hours
http://dev.mysql.com/doc/refman/5.0/en/server-system-variables.html#option_mysqld_wait_timeout.
Although the documentation doesn't indicate it, a value of 0 may disable the timeout completely, but you would need to experiment as this is just speculation.
There are however three other situations that I know of that can generate that error. The first is the MySQL server being restarted. This will obviously drop all the connections, but as the MySQL client is passive, and this won't be noticed till you do the next query.
The second condition is if someone kills your query from the MySQL command line, and this also drops the connection, because it could leave the client in an undefined state.
The last is if your MySQL server restarts itself due to a fatal internal error. That is, if you are doing a simple query against a table and instantly see 'MySQL has gone away', I'd take a close look at your server's logs to check for hardware error, or database corruption.
First, determine the max_connections in MySQL:
show variables like "max_connections";
You need to make sure that the number of connections you're making in your Ruby on Rails application is less than the maximum allowed number of connections. Note that extra connections can be coming from your cron jobs, delayed_job processes (each would have the same pool size in your database.yml), etc.
Monitor the SQL connections as you go through your application, run processes, etc. by doing the following in MySQL:
show status where variable_name = 'Threads_connected';
You might want to consider closing connections after a Thread finishes execution as database connections do not get closed automatically (I think this is less of an issue with Ruby on Rails 4 applications Reaper):
Thread.new do
begin
# Thread work here
ensure
begin
if (ActiveRecord::Base.connection && ActiveRecord::Base.connection.active?)
ActiveRecord::Base.connection.close
end
rescue
end
end
end
The connection to the MySQL server is probably timing out.
You should be able to increase the timeout in MySQL, but for a proper fix, have your code check that the database connection is still alive, and re-connect if it's not.
Using reconnect: true in the database.yml will cause the database connection to be re-established AFTER the ActiveRecord::StatementInvalid error is raised (As Dave Cheney mentioned).
Unfortunately adding a retry on the database operation seemed necessary to guard against the connection timeout:
begin
do_some_active_record_operation
rescue ActiveRecord::StatementInvalid => e
Rails.logger.debug("Got statement invalid #{e.message} ... trying again")
# Second attempt, now that db connection is re-established
do_some_active_record_operation
end
Do you monitor the number of open MySQL connections or threads? What is your mysql.ini settings for max_connections?
mysql> show status;
Look at Connections, Max_used_connections, Threads_connected, and Threads_created.
You may need to increase the limits in your MySQL configuration, or perhaps rails is not closing the connection properly*.
Note: I've only used Ruby on Rails briefly...
The MySQL documentation for server status is in http://dev.mysql.com/doc/refman/5.0/en/server-status-variables.html.
Something else to check is Unicorn config is correct. See before_fork and after_fork handling of ActiveRecord connection here: https://gist.github.com/nebiros/2776085#file-unicorn-rb
I had this problem in a Ruby on Rails 3 application, using the mysql2 gem. I copied out the offending query and tried running it in MySQL directly, and I got the same error, "MySQL server has gone away.".
The query in question was very, very large. A very large insert (+1 MB). The field I was trying to insert into was a TEXT column and their max size is 64 KB. Rather than throwing an errorm, the connection went away.
I increased the size of the field and got the same thing, so I'm still not sure what the exact issue was. The point is that it was in the database due to some strange query. Anyway!
While forking in Rails.
For anyone running into this while forking in Rails, try clearing the existing connections before forking and then establish a new connection for each fork, like this:
# Clear existing connections before forking to ensure they do not get inherited.
::ActiveRecord::Base.clear_all_connections!
fork do
# Establish a new connection for each fork.
::ActiveRecord::Base.establish_connection
# The rest of the code for each fork...
end
See this StackOverflow answer here: https://stackoverflow.com/a/8915353/293280