Lame question here. When running
async with engine.acquire() as conn:
await conn.execute(...)
will the context manager close connection automatically? And does it imply that postgres connection will be closed too?
From the code, it looks as if exiting the context manager will close the SQLAlchemy connection, but return it to the connection pool, so the Postgresql connection will remain open until the pool is released.
Related
I am using sql with Go, using the mysql driver and database/sql.
While reading about it I found that this driver handles the things related to connection pooling itself and db, err := db.Open("conn string") returns the connection pool instead of a single connection and when we use db.Query("some query"), it actually picks a free connection from connection pool (if no connection in pool is free it opens a new one) and uses it.
Please correct me, if I am wrong in my above understanding.
Now my question is, what happens when I call db.Close():
Will it close all connections in the connection pool?
Will it close then even if they they are passed to different goroutines and are being used there?
Following the documentation at https://hc.apache.org/httpcomponents-client-ga/tutorial/html/connmgmt.html
2.3.4. Connection manager shutdown
When an HttpClient instance is no longer needed and is about to go out of scope it is important to shut down its connection manager to ensure that all connections kept alive by the manager get closed and system resources allocated by those connections are released.
CloseableHttpClient httpClient = <...>
httpClient.close();
My confusion is about conflating the the instance going out of scope and needing to shutdown the connection manager.
In my use case, I am using the PoolingConnection so I want to keep the connections open, but of course return them back to the pool.
In my client code I have
ResponseHandler<Integer> rh = new ResponseHandler<Integer>()
.... elided ....
CloseableHttpClient httpclient = this.httpClientBuilder.build();
Integer statusCode = httpclient.execute(httpPost, rh);
My understanding from the docs is that that the useage of ResponseHandler takes care of return the lease
When using a ResponseHandler, HttpClient will automatically take care of ensuring release of the connection back to the connection manager
You understanding is correct. One needs to shut down the connection manager and the underlying connection pool only once it is no longer needed in order to ensure immediate shutdown and dealocation of persistent connections kept alive in the pool.
ResponseHandler ensures the connection leased from the pool gets released back to the manager no matter the outcome of request execution but it is up to the manager either to close the connection or keep it alive for re-use by subsequent requests.
We have developed a project in .NET Core and Entity Framework Core using the MySql nuget package.
The context is added to dependancy injection using the following line:
services.AddDbContext<ReadWriteContext>(options => options.UseMySQL(Configuration["Machine:ReadWriteConnectionString"]));
Then in a controller, this is injected as such:
public class SystemController : Controller
{
private readonly ReadWriteContext _dataContext;
public SystemController(ReadWriteContext dataContext)
{
_dataContext = dataContext;
}
...
}
And used as such:
var hasServices = await _dataContext.Services.AnyAsync();
In the logs we see the opening and closing log lines:
Opening connection to database 'config_service' on server '10.211.55.5'.
Closing connection to database 'config_service' on server '10.211.55.5'.
However, when we look at the MySql server and run "show full processlist", the connections are still showing as being in the sleep state and never close. When you stop the .NET process, the connections then close and disappear from MySql process list.
How do I get the connections to close when the request is finished. The AddDbContext should be scoped to the current request, but it does not appear to properly close the connections.
Any help would be great?
I'm not aware with MySql. That said if you were in a SQL server context you would be facing the "connection pool": https://learn.microsoft.com/en-us/dotnet/framework/data/adonet/sql-server-connection-pooling.
That is your application keep a set of active connection to speed up the process of reaching the data server. Thus even if you sais to the pool "I don't need this connection anymore", it does not believe you and keep the connection open... just in case.
For how long, well it depends on logic beyond the scope of the application. One use to say: it is the connection pool realm, just let it do his job.
So the answer for your question is: you can't explicitly close the connection to the server. The connection pool will decide when to close or not.
If we don't close or end mysql connection in node js, is it effects in feature or not.
I am doing like this
var mysql = require('mysql');
var connection = mysql.createConnection(...);
connection.query('SELECT 1', function(err, rows) {
// connected! (unless `err` is set)
});
Am not ending mysql connection any where in my code. My question is
Is it necessary to close the mysql connection.
If we don't close the mysql connection will i face any other problems in future.
Please help me am new to nodejs
EDIT1
I will not face any problems like unable to connect, too many connections to open etc, means any resource related issues? right.
EDIT2
At which instant mysql connection will be close if we don't end it manually or by using end function?
You should close the connection.
The point at which you do it depends on what your program does with that connection.
If the program is long lived and only needs a single connection but uses that connection continuously then you can just leave that one connection open, but you should be be prepared to re-open the connection if required - e.g. if the server gets restarted.
If the program just opens one connection, does some stuff, and then doesn't use the connection for some time you should close the connection while you're not using it.
If the program is short lived, i.e. it makes a connection, does some stuff, and then exits you can get away without closing the connection because it'll get closed automagically when your program exits.
Actually, you will run into problems such as this one from MySQL:
"[host] is blocked because of many connection errors; unblock with ' mysqladmin flush-hosts"
As MySQL will start counting those failed disconnects as "connection errors". And then, your database remains unusable until you execute FLUSH HOSTS on the MySQL server. I know this for a fact, it happened with our NodeJS project.
Write some code to explicitly close the db connection before any output or render statements using connection.end() or connection.destroy()
No you don't need to close it.
If you question is "is there any specific cleanup on mysql side when I tell server that I'm closing connection" then the answer is "no, if you just exit your client process or close socket its the same as calling connection.end()"
I found a very good module (node-mysql) to connect to Mysql database.
The module is very good, I Only have a question about "WHEN" to open the connection to Mysql.
I always have used php-mysql before starting with node, for each request i opened a connection...then query....then close.
Is the same with node? for each request do I have to open a connection and then close it? or can i use persistent connection?
Thank you
The open-query-close pattern generally relies on connection pooling to perform well. Node-mysql doesn't have any built in connection pooling, so if you use this pattern you'll be paying the cost of establishing a new connection each time you run a query (which may or may not be fine in your case).
Because node is single threaded, you can get away with a single persistent connection (especially since node-mysql will attempt to reconnect if the connection dies), but there are possible problems with that approach if you intend to use transactions (since all users of the node client are sharing the same connection and so same transaction state). Also, a single connection can be a limit in throughput since only one sql command can be executed at a time.
So, for transactional safety and for performance, the best case is really to use some sort of pooling. You could build a simple pool yourself in your app or investigate what other packages are out there to provide that capability. But either open-query-close, or persistent connection approaches may work in your case also.
felixge/node-mysql now has connection pooling (at the time of this writing.)
https://github.com/felixge/node-mysql#pooling-connections
Here's a sample code from the above link:
var mysql = require('mysql');
var pool = mysql.createPool(...);
pool.getConnection(function(err, connection) {
// Use the connection
connection.query( 'SELECT something FROM sometable', function(err, rows) {
// And done with the connection.
connection.end();
// Don't use the connection here, it has been returned to the pool.
});
});
So to answer your question (and same as #Geoff Chappell's answer): best case would be to utilize pooling to manage connections.