As a standard procedure, MySql connection is lost after stated number of hours (default:8) of inactivity. To reconnect to the mysql server after identifying such connection lost I simply do connection = DriverManager.getConnection(url, user, password);
I am not using connection pool and as this trick has not been mentioned in previous connection lost related posts, that makes me wonder if my code will generate any side-effects later? (I say so because testing this code after above instance, I found the sessionlistner is not called after session.invalidate() call.)
You'll loose temporary tables and session settings if your connection drops. It sounds like a connection pool would be useful in your situation.
Depending on how you handle connection object(s), it can create small client-side memory leak for connection object that was lost. But probably this effect will be so small that you will never see any problems from it.
To minimize this risk, you can do something as simple as SELECT 1 every few minutes from your connection during idle time, such that your connection is still considered active (unless your client dies off completely).
Related
The scenario:
I'm testing out a project written by another engineer in Go to interface with a mysql database. It came to my attention that in Go, whenever a new mysql connection is created, the defaults for this connection are set such that connections never expire.
We recently had a bug in our code that resulted in the proposed statement queue in mysql being completely filled up, resulting in all database requests being denied.
The proposed solution for this bug is to clear out all mysql connections every 24 hours to prevent the proposed statement queue from being filled up.
It is my understanding that our code spawns new connections automatically as needed, and hence clearing out all connections every 24 hours should have no unintended consequences.
The underlying reason why this queue is being filled up seems to be a disconnect between GO language cancelling a connection or request; whereas in mysql, the connection still exists but is only idle. So the proposed solution is to clear out these idle requests and allow them to timeout, using the following code:
conn.SetConnMaxLifetime(24 * time.Hour)
conn.SetConnMaxIdleTime(1 * time.Hour)
conn.SetMaxOpenConns(30)
conn.SetMaxIdleConns(1)
What I seek to understand:
What advantage is there for keeping idle connections, and could there be a situation where more than 1 idle connection is needed?
Are there any unintended consequences that haven't been considered; that could result in a future issue where all database requests get denied?
I want to write a function for mysql connection to use it any where.
there is tow way in my mind
1)in top of function open connection and execute query and close connection. in this way each query open connection and close it but have easy usage because no need to handle open and close connection in separate function and call them in top and bottom of code
2)write a function for opening connection and another function for closing it and call them in top an bottom of code. in this way one connection open and use for multiple query then close
my question is:
is there any difference to open and close the db connection multiple times or it doesn't matter?
which way have a better performance(in second way connection may open some minutes)?
I use some programming language like java, php and ... .but my question is general
Each time you open and close a connection you are using resources. Imagine you have 100 requests per second. Each time you perform 2 operations. Open and close a connection. The response time increase.
Is better to use pool Connections. So, you have 5 open connections waiting for querys. When the connection is not in use, the connection returns to the pool and wait for another request.
Or you can try a persist connection. Both have pros and cons.
But never, never ever open and close connections for each request. Look this response
MySQL - Persistent connection vs connection pooling
i think there will be not so much difference but it will be better if you close your connection frequently so mysql connection limit will not be increase and if you keep connection open, So there will be chances to cross your connection limit.
I have written a web server using Delphi and the Indy TIdHttpServer component. I am managing a pool of TAdoConnection connections to a MySql database. When a request comes in I query my pool for available database connections. If one is not available then a new TAdoConnection is created and added to the pool.
Problems occur when a connection becomes "stale" (i.e. it has not been used in quite some time). I think in this instance the query results in the "MySql has gone away" error.
Does anyone have a method for getting around this? Or would I have manage it myself by one of the following:
Writing a thread that will periodically "refresh" all connections.
Keeping track of the last active query, and if too old pass up using the connection and instead free it.
Two suggestions:
store a 'last used' time stamp with every pooled connection, and if a connection is requested check if the connection is too old - in this case, create a new one
add a validateObject() method which issues a no-op SQL query to detect if the connection is still healthy
a background thread which cleans up the pool in regular intervals: removing idle connections allows to reduce the pool size back to a minimum after peak usage
For some suggestions, see this article about the Apache Commons Pool Framework: http://www.javaworld.com/article/2071834/build-ci-sdlc/pool-resources-using-apache-s-commons-pool-framework.html
I have program that constantly query a mysql server. Each time I access the server I made a connection then query it. I wonder if I can actually save time by reuse the same connection and only reconnect when the connection is closed. Given that I can fit many of my queries within the duration of connection timeout and my program has only one thread.
yes - this is a good idea.
remember to use the timeout so you don't leave connections open permanently.
also, remember to close it when the program exits. (even after exceptions)
Yes by all means, re-use the connection!
If you are also doing updates/delete/inserts through that connection make sure you commit (or rollback) you transactions properly so that once you are "done" with the connection, it is left in a clean state.
Another option would be to use a connection pooler.
Yes, you should reuse the connection, within reason. Don't leave a connection open indefinitely, but you can batch your queries together so that you get everything done, and then close it immediately afterwards.
Leaving a connection open too long means that under high traffic you might hit the maximum number of possible connections to your server.
Reconnecting often is just slow, causes a lot of unnecessary chatter, and is simply a waste.
Instead, you should look into using mysql_pconnect function which will create persistent connection to the database. You can read about it here:
http://php.net/manual/en/function.mysql-pconnect.php
I'm doing my first foray with mysql and I have a doubt about how to handle the connection(s) my applications has.
What I am doing now is opening a connection and keeping it alive until I terminate my program. I do a mysql_ping() every now and then and the connection is started with MYSQL_OPT_RECONNECT.
The other option (I can think of), would be to start a new connection before doing anything that requires my connection to the database and closing it after I'm done with it.
What are the pros and cons of these two approaches?
what are the "side effects" of a long connection?
What is the most used method of handling this?
Cheers ;)
Some extra details
At this point I am keeping the connection alive and I ping it every now and again to now it's status and reconnect if needed.
In spite of this, when there is some consistent concurrency with queries happening in quick succession, I get a "Server has gone away" message and after a while the connection is re-established.
I'm left wondering if this is a side effect of a prolonged connection or if this is just a case of bad mysql server configuration.
Any ideas?
In general there is quite some amount of overhead incurred when opening a connection. Depending on how often you expect this to happen it might be ok, but if you are writing any kind of application that executes more than just a very few commands per program run, I would recommend a connection pool (for server type apps) or at least a single or very few connections from your standalone app to be kept open for some time and reused for multiple transactions.
That way you have better control over how many connections get opened at the application level, even before the database server gets involved. This is a service an application server offers you, but it can also be rolled up rather easily if you want to keep it smaller.
Apart from performance reasons a pool is also a good idea to be prepared for peaks in demand. When a lot of requests come in and each of them tries to open a separate connection to the database - or as you suggested even more (per transaction) - you are quickly going to run out of resources. Keep in mind that every connection consumes memory inside MySQL!
Also you want to make sure to use a non-root user to connect, because if you don't (I think it is tied to the MySQL SUPER privilege), you might find yourself locked out. MySQL reserves at least one connection for an administrator for problem fixing, but if your app connects with that privilege, all connections would already be used up when you try to put out the fire manually.
Unless you are worried about having too many connections open (i.e. over 1,000), you she leave the connection open. There is overhead in connecting/reconnecting that will only slow things down. If you know you are going to need the connection to stay open for a while, run this query instead of pinging periodically:
SET SESSION wait_timeout=#
Where # is the number of seconds to leave an idle connection open.
What kind of application are you writing? If it's a webscript: keep it open. If it's an executable, pool your connections (if necessary, most of the times a singleton will do).