I set unreturnedConnectionTimeout to release stale connections. I assumed that this was only going to close connections without any activity but it looks like this just closes every connection after the specified time.
Is this a bug or is this 'as designed'?
The manual states:
unreturnedConnectionTimeout defines a limit (in seconds) to how long a
Connection may remain checked out. If set to a nozero value,
unreturned, checked-out Connections that exceed this limit will be
summarily destroyed, and then replaced in the pool. Obviously, you
must take care to set this parameter to a value large enough that all
intended operations on checked out Connections have time to complete.
You can use this parameter to merely workaround unreliable client apps
that fail to close() Connections
From this I conclude that activity is not influencing the throwing away of connections. To me that sounds strange. Why throw away active connections?
Thanks,
Milo
i'm the author of c3p0, and of the paragraph you quote.
unreturnedConnectionTimeout is exactly what its name and documentation state: a timeout for unreturned Connections. it was implemented reluctantly, in response to user feedback, because it would never be necessary or useful if clients reliably check-in the Connections they check out. when it was implemented, I added a second unsolicited config param, debugUnreturnedConnectionStackTraces, to encourage developers to fix client applications rather than rely lazily on unreturnedConnectionTimeout.
there is nothing strange about the definition of unreturnedConnectionTimeout. generally, applications that use a Connection pool do not keep Connections checked out for long periods of time. doing so defies the purpose of a Connection pool, which is to allow applications to acquire Connections on an as-needed basis without a large performance penalty. the alternative to a Connection pool is for an application to check out Connections and retain them for long-periods of time, so they are always available for use. but maintaining long-lived Connections turns out to be complicated, so most applications delegate this to a pooling library like c3p0.
i understand that you have a preexisting application that maintains long-lived Connections, that you cannot easily modify. you would like a hybrid architecture between applications that maintain long-lived Connections directly and applications that delegate to a pool. in particular, what you want is for a library that helps you maintain the long-lived Connections that your application is already designed to retain.
c3p0 is not that library, unfortunately. c3p0 (like most Connection pooling libraries) considers checked-out Connections to be the client's property, and does no maintenance work on them until they are checked back in. there are two exceptions to this: unreturnedConnectionTimeout will close() Connections out from underneath clients if they have been checked out for too long, and c3p0 will invisibly test checked-out Connection when Exceptions occur, in order to determine whether Connections that have experienced Exceptions are suitable for return to the pool or else must be destroyed on check-in.
unreturnedConnectionTimeout is not the parameter you want. you would like something that automatically closes Connections when they are inactive for a period of time, but that permits Connections to be checked out indefinitely. such a parameter might be called inactiveConnectionTimout, and is a feature that could conceivably be added to c3p0, but has not been. it probably will not be, because few application hold checked-out Connections for long periods, and c3p0 is full of features that help you observe failures once Connections are checked-in, or when Connections transition between checked-out and checked in.
in your (pretty unusual) case, this means there is a feature you would like that simply is not provided by the library. i am sorry about that!
The unreturnedConnections can be active, it depends on how long it takes to execute eg. the query on the database. You should set the timeout for it to the value bigger then the longest operation you can expect with your application. Sometimes if you know that the value should be enough and c3p0 is still closing active connections than it means that the connection leaked somewhere (maybe not closed properly).
Related
I would like to create a Cloud function to call a Postgres Cloud SQL DB. Currently I followed the documentation and create a Hikari based connection...
val config = new HikariConfig
config.setJdbcUrl(jdbcURL)
config.setDataSourceProperties(connProps)
config.setMaximumPoolSize(10)
config.setMinimumIdle(4)
config.addDataSourceProperty("ipTypes", "PUBLIC,PRIVATE") // TODO: Make configureable
println("Config created")
val pool : DataSource = new HikariDataSource(config) // Do we really need Hikari here if it doesn't need pooling?
println("Returning the datasource")
Some(pool)
This works but it causes a 25 sec delay due to "cold start"s. I would like to try using PG driver directly and see if that is faster but I think that isn't possible thanks the the UNIX socket/SQL Cloud proxy stuff based on the documentation.
Is there a way to connect to Cloud SQL from a Cloud function using a basic PG Driver connection and not the Hikari stuff?
As mentioned in the thread:
With all "serverless" compute providers, there is always going to be
some form of cold start cost that you can't eliminate. Even if you are
able to keep a single instance alive by pinging it, the system may
spin up any number of other instances to handle current load. Those
new instances will have a cold start cost. Then, when load decreases,
the unnecessary instances will be shut down.
you can now specify a minimum number of instances to keep active. This
can help reduce (but not eliminate) cold starts. Read the Google
Cloud blog and the documentation.
If you absolutely demand hot servers to handle requests 24/7, then you
need to manage your own servers that run 24/7 (and pay the cost of
those servers running 24/7). As you can see, the benefit of serverless
is that you don't manage or scale your own servers, and you only pay
for what you use, but you have unpredictable cold start costs
associated with your project. That's the tradeoff.
For more information related to dependencies you can refer to the link provided by guillaume blaquiere.
To answer your exact question:
Can I connect without using HikariCP?
The answer is sure; you can use any number of connection pooling libraries avaible in Java. The examples often show HikariCP because it is far and away the most popular and highest performing.
So it's unlikely that switching connection pools will improve your performance. A slightly different question implied by your first question might be:
Can I connect without using a connection pool?
And again the answer is sure, you could use the driver directly -- but you probably shouldn't. Connection creation and management is expensive (and hard), and using a connection pool is a best practice. I wouldn't consider code "production quality" without one. While it might save you boot time, it's likely to introduce more overhead and latency into the request itself, costing you more overall. Additionally, it'll remove helpful error handling and retries around connections that you'll now have to deal with yourself.
So it seems you question really might be:
How can I reduce my cold start time?
Well with a start time of 25 seconds, the problem likely isn't limited to just Hikari. I would check out this GCP doc page on performance, and look into other articles on how to improve start up time for JVMs or your specific frameworks.
However, one way that HikariCP might be impacting your start up time is that HikariCP blocks on the connection creation until the initialization is complete. There are a few things you can do to improve this (but likely will only help, not eliminate the 25s cold start)
You can lower your number of connections to 1. Cloud function instances only handle requests one at a time, so specifying a min-idle of 4 and a max connection to 10 is likely leading to wasted connections.
You can move the initialization of Hikari to happen outside of your start up. The GCP docs page I mentioned above shows how to use lazy initialization, so expensive object's aren't created until you need them. This will move the cost of initializing Hikari out of your functions start up. This could make the first request that calls it more expensive -- if that is a concern, I would suggest combining lazy initialization along with triggering that initialization in async way on start up. This way the pool is created in the background, without blocking startup.
As an alternative to #2, you could also lower min-idle connections to 0 - e.i., initialize the Hikari Pool with 0 connections in it. While this might be easier to implement, it will mean that requests without a warmed up connection will have to wait for a new connection to be established. (which makes #2 more optimal in terms of performance).
Assuming I'll execute every 2 seconds a query, should I open the connection on every request or should I keep the connection alive until the application(server) stops?
In my experience, establishing connections is unlikely to be a bottleneck for a mysql server (connection overhead is fairly low in mysql). That having been said, reusing existing connections is often an appropriate approach, but it requires some careful considerations: if the database server is temporarily unavailable, the code must reconnect; if the server is replaced, it must reconnect (mysql implementations tend towards failover solutions rather than true high availability); if the application uses multiple connections to mysql, you must be sure not to cross your connections between users or sessions (active database, timezone, charset, and so on are sessions variables, essentially tied to a connection). If you're not up to the task of making your reusable connection reliable in these and other edge cases, creating a new connection every 2 seconds may provide this durability for free.
In short, there can be less-than-obvious benefits to short lived connections. I would not bother to add intelligence around maintaining a persistent connection unless you have reason to believe it will actually make a meaningful difference in your case (eg benchmarks).
I'm doing my first foray with mysql and I have a doubt about how to handle the connection(s) my applications has.
What I am doing now is opening a connection and keeping it alive until I terminate my program. I do a mysql_ping() every now and then and the connection is started with MYSQL_OPT_RECONNECT.
The other option (I can think of), would be to start a new connection before doing anything that requires my connection to the database and closing it after I'm done with it.
What are the pros and cons of these two approaches?
what are the "side effects" of a long connection?
What is the most used method of handling this?
Cheers ;)
Some extra details
At this point I am keeping the connection alive and I ping it every now and again to now it's status and reconnect if needed.
In spite of this, when there is some consistent concurrency with queries happening in quick succession, I get a "Server has gone away" message and after a while the connection is re-established.
I'm left wondering if this is a side effect of a prolonged connection or if this is just a case of bad mysql server configuration.
Any ideas?
In general there is quite some amount of overhead incurred when opening a connection. Depending on how often you expect this to happen it might be ok, but if you are writing any kind of application that executes more than just a very few commands per program run, I would recommend a connection pool (for server type apps) or at least a single or very few connections from your standalone app to be kept open for some time and reused for multiple transactions.
That way you have better control over how many connections get opened at the application level, even before the database server gets involved. This is a service an application server offers you, but it can also be rolled up rather easily if you want to keep it smaller.
Apart from performance reasons a pool is also a good idea to be prepared for peaks in demand. When a lot of requests come in and each of them tries to open a separate connection to the database - or as you suggested even more (per transaction) - you are quickly going to run out of resources. Keep in mind that every connection consumes memory inside MySQL!
Also you want to make sure to use a non-root user to connect, because if you don't (I think it is tied to the MySQL SUPER privilege), you might find yourself locked out. MySQL reserves at least one connection for an administrator for problem fixing, but if your app connects with that privilege, all connections would already be used up when you try to put out the fire manually.
Unless you are worried about having too many connections open (i.e. over 1,000), you she leave the connection open. There is overhead in connecting/reconnecting that will only slow things down. If you know you are going to need the connection to stay open for a while, run this query instead of pinging periodically:
SET SESSION wait_timeout=#
Where # is the number of seconds to leave an idle connection open.
What kind of application are you writing? If it's a webscript: keep it open. If it's an executable, pool your connections (if necessary, most of the times a singleton will do).
My site has always used persistent connections, based on my understanding of them there's no reason not to. Why close the connection when it can be reused? I have a site that in total accesses about 7 databases. It's not a huge traffic site, but it's big enough. What's your take on persistent, should I use them?
With persistent connections:
You cannot build transaction processing effectively
impossible user sessions on the same connection
app are not scalable. With time you may need to extend it and it will require management/tracking of persistent connections
if the script, for whatever reason, could not release the lock on the table, then any following scripts will block indefinitely and one should restart the db server. Using transactions, transaction block will also pass to the next script (using the same connection) if script execution ends before the transaction block completes, etc.
Persistent connections do not bring anything you can do with non-persistent connections.
Then, why to use them, at all?
The only possible reason is performance, to use them when overhead of creating a link to your SQL Server is high. And this depends on many factors like:
database type
whether MySQl server is on the same machine and, if not, how far? might be out of your local network /domain?
how much overloaded by other processes the machine on which MySQL sits
One always can replace persistent connections with non-persistent connections. It might change the performance of the script, but not its behavior!
Commercial RDMS might be licensed by the number of concurrent opened connections and here the persistent connections can misserve
My knowledge on the area is kinda limited so I can't give you many details on the subject but, as far as I know, the process of creating connections and handing them to a thread really costs resources, so I would avoid it if I were you. Anyhow I think that most of this decisions can't be generalized and depend on the business.
If, for instance, your application communicates continuously with the Database and will only stop when the application is closed, then perhaps persistent connections are the way to go, for you avoid the process mentioned before.
However, if your application only communicates with the Database sporadically to get minor information then closing the connection might be more sane, for you won't waste resources on opened connections that are not being used.
Also there is a technique called "Connection Pooling", in which you create a series of connections a priori and keep them there for other applications to consume. In this case connections are persistent to the database but non-persistent to the applications.
Note: Connections in MSSQL are always persistent to the database because connection pooling is the default behavior.
I've been thinking, why does Apache start a new connection to the MySQL server for each page request? Why doesn't it just keep ONE connection open at all times and send all sql queries through that one connection (obviously with client id attached to each req)?
It cuts down on the handshake time overhead, and a couple of other advantages that I see.
It's like plugging in a computer every time you want to use it. Why go to the outlet each time when you can just leave it plugged in?
MySQL does not support multiple sessions over a single connection.
Oracle, for instance, allows this, and you can setup Apache to mutliplex several logical sessions over a single TCP connection.
This is limitation of MySQL, not Apache or script languages.
There are modules that can do session pooling:
Precreate a number of connections
Pick a free connection on demand
Create additional connections if not free connection is available.
the reason is: it's simpler.
to re-use connections, you have to invent and implement connection pooling. this adds another almost-layer of code that has to be developed, maintained, etc.
plus pooled connections invite a whole other class of bugs that you have to watch out for while developing your application. for example, if you define a user variable but the next user of that connection goes down a code path that branches based on the existence of that variable or not then that user runs the wrong code. other problems include: temporary tables, transaction deadlocks, session variables, etc. all of these become very hard to duplicate because it depends on the subsequent actions of two different users that appear to have no ties to each other.
besides, the connection overhead on a mysql connection is tiny. in my experience, connection pooling does increase the number of users a server can support by very much.
Because that's the purpose of the mod_dbd module.