I have developed a application which send large number of emails to different users in batches. The most common problem I faced in this application is the problem of mysql connection timeout. In between the batches when there is no queries executed in the previously opened connection and connection remained idle for long time, mysql itself close the connection. After sending the current batch when I again try to execute any sql query it gives me mysql connection error.
Right now I am using mysql_ping($conn) function to check whether the connection id timedout or not. If the connect is timed out I connect again with mysql_connect() function. Now I am moving to doctrine rather than native PHP function. Is there a recconnect() function in Doctorine as well ?
Never faced this problem with my batch actions but I think you could probably do something like this in those places in your code where you think there's a risk the connection will time out:
// Fetch current connection
$conn = Doctrine_Manager::connection();
if(!$conn) {
// Open a new connection
$conn = Doctrine_Manager::connection('mysql://username:password#localhost/whatever', 'connection 1');
}
http://docs.doctrine-project.org/projects/doctrine1/en/latest/en/manual/connections.html
There's a custom bundle that allow connection to reopen:
https://github.com/facile-it/doctrine-mysql-come-back
You can set a maximum amount of retry, or you can manually call method retry on Connection!
Related
I'm using Symfony 1.4 for my web application which uses doctrine 1.2 to connect to my MySQL database. I have a Symfony task which first fetches some data using doctrine queries and then it does a very time-consuming non- database involved operation (ex: a never-ending for loop) which doesn't need a database connection. Since the latter operation doesn't need database connection to be open, I need to close the database connection after the database operations are done. So I tried the following code to close the database connection.
sfContext::createInstance($configuration);
sfContext::getInstance()->getDatabaseConnection('doctrine')->setAttribute(Doctrine_Core::ATTR_AUTO_FREE_QUERY_OBJECTS, true );
// Do the doctrine queries and fetch data
sfContext::getInstance()->getDatabaseManager()->shutdown();
gc_collect_cycles();
// Do the non-database operation
While the non-database operation is executing, I checked the active mysql processes using SHOW PROCESSLIST command running on MySQL. It still shows that the MySQL connection created by the Symfony application is not closed but in the Sleep state.
Is there any way to close the database connection permanently in Symfony.
Note: I found that the doctrine connection closes if the connection is closed before execuring the doctrine queries.
This is most probably because Doctrine uses PDO and PDO needs to unset all the references before making the MySQL connection close otherwise it keeps the connection active. But I don't know how to clear the PDO references in Doctrine.
That is some really old stuff you are working with. I'm not sure why you are getting things from the sfContext singleton. You should be able to do this instead:
$conn = Doctrine_Manager::connection();
$conn->close();
This should close the connection, but leave it registered with the connection manager. This is basically right out of the Doctrine1 manual.
This will let you iterate through the connections and close them while also removing them from the manager:
$manager = Doctrine_Manager::getInstance();
foreach($manager as $conn) {
$manager->closeConnection($conn);
}
I am using root as username.
My program will refresh every 5 seconds.
What it does is, it query from mysql table and display the data.
Problem is, every after 5 seconds, the connection on mysql will append, reason that it will give an error of "TOO MUCH CONNECTIONS" when it reach the limit.
Is it possible to kill the previous connection since it is unused already?
Here is my code on opening a connection.
connectionPool = connectionPool.getConnectionPool("root", "*****", "");
This is normal behavior if you are using a connection pool. When your job is over be sure that you free the connection instance, or close all pool connections when your code execution is done.
When you are done with a connection, you need to close it. this will return a connection back to the pool.
import MySQLdb
MySQLdb.connect fails sometimes unable to connect, is there a way to set a timeout and try this repeatedly without writing application specific code
Abstract your connection code into a method you can call and have it return the connection. Inside that method try to connect to the database up to 3 or 5 times. Do a time.sleep(.01) between each attempt. That is a 100th of a second. If you can go less, that would be better.
If we don't close or end mysql connection in node js, is it effects in feature or not.
I am doing like this
var mysql = require('mysql');
var connection = mysql.createConnection(...);
connection.query('SELECT 1', function(err, rows) {
// connected! (unless `err` is set)
});
Am not ending mysql connection any where in my code. My question is
Is it necessary to close the mysql connection.
If we don't close the mysql connection will i face any other problems in future.
Please help me am new to nodejs
EDIT1
I will not face any problems like unable to connect, too many connections to open etc, means any resource related issues? right.
EDIT2
At which instant mysql connection will be close if we don't end it manually or by using end function?
You should close the connection.
The point at which you do it depends on what your program does with that connection.
If the program is long lived and only needs a single connection but uses that connection continuously then you can just leave that one connection open, but you should be be prepared to re-open the connection if required - e.g. if the server gets restarted.
If the program just opens one connection, does some stuff, and then doesn't use the connection for some time you should close the connection while you're not using it.
If the program is short lived, i.e. it makes a connection, does some stuff, and then exits you can get away without closing the connection because it'll get closed automagically when your program exits.
Actually, you will run into problems such as this one from MySQL:
"[host] is blocked because of many connection errors; unblock with ' mysqladmin flush-hosts"
As MySQL will start counting those failed disconnects as "connection errors". And then, your database remains unusable until you execute FLUSH HOSTS on the MySQL server. I know this for a fact, it happened with our NodeJS project.
Write some code to explicitly close the db connection before any output or render statements using connection.end() or connection.destroy()
No you don't need to close it.
If you question is "is there any specific cleanup on mysql side when I tell server that I'm closing connection" then the answer is "no, if you just exit your client process or close socket its the same as calling connection.end()"
I found a very good module (node-mysql) to connect to Mysql database.
The module is very good, I Only have a question about "WHEN" to open the connection to Mysql.
I always have used php-mysql before starting with node, for each request i opened a connection...then query....then close.
Is the same with node? for each request do I have to open a connection and then close it? or can i use persistent connection?
Thank you
The open-query-close pattern generally relies on connection pooling to perform well. Node-mysql doesn't have any built in connection pooling, so if you use this pattern you'll be paying the cost of establishing a new connection each time you run a query (which may or may not be fine in your case).
Because node is single threaded, you can get away with a single persistent connection (especially since node-mysql will attempt to reconnect if the connection dies), but there are possible problems with that approach if you intend to use transactions (since all users of the node client are sharing the same connection and so same transaction state). Also, a single connection can be a limit in throughput since only one sql command can be executed at a time.
So, for transactional safety and for performance, the best case is really to use some sort of pooling. You could build a simple pool yourself in your app or investigate what other packages are out there to provide that capability. But either open-query-close, or persistent connection approaches may work in your case also.
felixge/node-mysql now has connection pooling (at the time of this writing.)
https://github.com/felixge/node-mysql#pooling-connections
Here's a sample code from the above link:
var mysql = require('mysql');
var pool = mysql.createPool(...);
pool.getConnection(function(err, connection) {
// Use the connection
connection.query( 'SELECT something FROM sometable', function(err, rows) {
// And done with the connection.
connection.end();
// Don't use the connection here, it has been returned to the pool.
});
});
So to answer your question (and same as #Geoff Chappell's answer): best case would be to utilize pooling to manage connections.