I have a Grails application running under Tomcat 7 and am having problems managing MySQL connections.
The problem is that each new request to the application (i.e., a page load) is creating a new MySQL connection, and these connections are not closing. Instead, they stay in a SLEEP state until the MySQL server finally refuses to accept more connections. Therefore, simply by reloading individual pages on the site at a fast rate, one can create numerous database connections.
So it seems that my connection pool isn't closing the connections with MySQL fast enough. There are a number of configuration settings for the connection pool, but I'm not sure what needs to be tuned to avoid this problem.
Here is the configuration from my context.xml file:
<Resource name="jdbc/Production" auth="Container" type="javax.sql.DataSource"
maxActive="100"
maxIdle="30"
maxWait="10000"
minEvictableIdleTimeMillis="1800000"
timeBetweenEvictionRunsMillis="1800000"
numTestsPerEvictionRun="3"
removeAbandoned="true"
removeAbandonedTimeout="60"
logAbandoned="true"
testOnBorrow="true"
testWhileIdle="true"
testOnReturn="true"
validationQuery="SELECT 1"
username=""
password=""
driverClassName="com.mysql.jdbc.Driver"
url="jdbc:mysql://localhost/Production"
/>
Thanks very much for any suggestions.
You did not define a connection pool.
Add following code to the context.xml(it seems to be a JNDI data source):
pooled = "true"
factory="org.apache.tomcat.jdbc.pool.DataSourceFactory"
If you haven't put JDBC Pool in to your dependency configuration, add following to the plugins closure in BuildConfig.groovy:
compile ":jdbc-pool:1.0.9.3"
You may use other connection pools, but JDBC Pool will be my recommendation.
Related
How can you configure Tomcat to Not use Database Connection Pooling ?
I want to do this because I have an application that makes very few calls to the mysql database, typically less than 10 in a 24 hour period and some calls fail on retrieving the connection. It appears that Mysql is sometimes giving up on the connection because it has been idle for so long without telling the database pooler.
Worth noting that I have another application that makes thousands of requests to the exact same database and this never fails, so I am convinced the issue is related to the rarity of calls to the database. Mysql is probably part of the problem but updating to Postgres would be a larger task than I want to commit to right now.
So I tried configuring context.xml to testWhileIdle but it is stil failing
<Context path="/store" privileged="true">
<Resource name="jdbc/myapp" auth="Container" type="javax.sql.DataSource"
maxActive="100" maxIdle="100" maxWait="10000" testWhileIdle="true"
username="usrnm" password="pwd" driverClassName="com.mysql.jdbc.Driver"
url="jdbc:mysql://localhost:3306/myapp"/>
</Context>
So now I want to disable pooling so its get a new connection every time but I cant work out how to do it, Ive looked at
https://tomcat.apache.org/tomcat-7.0-doc/jdbc-pool.html#Plain_Ol%27_Java
but everything seems to be centered out database pooling.
Attempted Solution
So I tried out suggestion in answer and will see what happens over next few days
<Context path="/store" privileged="true">
<Resource name="jdbc/myapp" auth="Container" type="javax.sql.DataSource"
maxActive="10" maxIdle="10"
testWhileIdle="true" validationQuery="select 1" validationQueryTimeout="5"
username="usrnm" password="pwd"
driverClassName="com.mysql.jdbc.Driver" testOnBorrow="true"
url="jdbc:mysql://localhost:3306/myapp"/>
</Context>
Nothing should be preventing you from making your own connections and using them. But, I think you are missing configuration to make the connection validation work properly. When a connection is tested and found to be bad it should be replaced by a new one prior to the pool giving you the connection. I think you are missing validationQuery which would be utilized when testing the connection. Check out the docs at https://tomcat.apache.org/tomcat-8.0-doc/jdbc-pool.html and look at supplying the necessary values for the validation to work. Try testOnBorrow and validationQuery so that each time you go for a connection it will validation it prior to returning. Also, you appear to be setting up way too many connections based on your frequency of usage. Tuning that should help avoid keeping unnecessary connections open.
I'm using the tomcat connection pool via JNDI resources.
To avoid that the connection is lost after a long inactivity (more of 8 hours, that is the default value of MySQL variable wait_timeout), I have put validationQuery and testOnBorrow in the context.xml.
My context.xml is:
<Resource name="jdbc/mydb" auth="Container" type="javax.sql.DataSource"
removeAbandoned="true" removeAbandonedTimeout="60"
maxActive="30" maxIdle="30" maxWait="10000"
username="myuser" password="mypwd" driverClassName="com.mysql.jdbc.Driver"
url="jdbc:mysql://localhost:3306/mydb?useEncoding=true&characterEncoding=UTF-8"
factory="org.apache.tomcat.jdbc.pool.DataSourceFactory" closeMethod="close"
validationQuery="select 1" testOnBorrow="true" />
It works, but if a I use a SSL connection, it doesn't work anymore.
I obtain:
com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException: No
operations allowed after connection closed.
...
Caused by: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: The last packet
successfully received from the server was 328,606,914 milliseconds ago. The last
packet sent successfully to the server was 328,606,914 milliseconds ago. is longer
than the server configured value of 'wait_timeout'. You should consider either
expiring and/or testing connection validity before use in your application,
increasing the server configured values for client timeouts, or using the
Connector/J connection property 'autoReconnect=true' to avoid this problem.
I wouldn't to use autoreconnect=true, 'cause it is not recommended by MySQL team itself
What could be the issue? Why this difference between SSL and no-SSL?
EDIT
It seems to work by putting ssl=true in the query string of the connection to db:
url="jdbc:mysql://localhost:3306/mydb?useEncoding=true&characterEncoding=UTF-8&ssl=true"
I've come across similar questions many times here on Stack Overflow, however, I haven't been able to get it right.
I'm trying to use Database connection pooling in Tomcat 6 + MySQL ( on AWS RDS )
These are the parameters I have configured.
( I'm closing the connection from my Java code also )
<Resource name="jdbc/awsDB" auth="Container" type="javax.sql.DataSource"
factory="org.apache.tomcat.jdbc.pool.DataSourceFactory"
initialSize="10"
testWhileIdle="true"
maxActive="30"
maxAge="3600"
maxIdle="5"
maxWait="3000"
removeAbandoned="true"
logAbandoned="false"
validationQuery="SELECT 1"
removeAbandonedTimeout="60"
timeBetweenEvictionRunsMillis="15000"
username="sbose78" password="XXXXX"
driverClassName="com.mysql.jdbc.Driver"
url="jdbc:mysql://xxxxrds.amazonaws.com:3306/health?autoReconnect=true" />
The application works great for the first few database queries and after that the database connection hangs indefinitely.
From the connection logs I've noticed that the connection pooling works good , however, Even when there are 10+ connections in SLEEP state, a new database query takes indefinite time to execute.
I end up re-starting the server in most cases.
What configuration mistake am I doing?
Thanks a lot in advance!
Check whether your query is using all the indexed columns in the where clause.If you are sure about the connection pool, then most probaably the problem will be with query itself.
I'm using tomcat 7 and the tomcat jdbc connection pool to dish out mysql connections.
During night times we don't have any activity so all connections become idle for longer than 8 hours and are dropped by mysql. (mysql's wait_timeout default).
We use the following pool configuration:
<Resource name="jdbc/dbName"
auth="Container"
factory="org.apache.tomcat.jdbc.pool.DataSourceFactory"
type="javax.sql.DataSource"
maxActive="50"
maxIdle="30"
maxWait="5000"
driverClassName="com.mysql.jdbc.Driver"
validationQuery="SELECT 1"
testOnBorrow="true"
testWhileIdle="true"
timeBetweenEvictionRunsMillis="10000"
removeAbandoned="true"
removeAbandonedTimeout="60"
logAbandoned="true"
username="xxx"
password="xxx"
url="jdbc:mysql://host:3306/xxx"/>
I was expecting the EvictionPolicy to remove idle connections way before they ever get closed by MySql. Somehow after one day we get the following exception:
com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException: No operations allowed after connection closed.Connection was implicitly closed by the driver.
I guess this problem should be something the jdbc connection pool can fix, but there are many configuration properties and I haven't used this pool before. Anybody got a good set of properties to configure the pool to not dish out closed connections?
Kind regards,
Albert
Solved it. Turned out it wasn't a pooling problem after all. We were using squeryl and lift together which isn't a happy combi (just yet). Connections got closed before returned to the pool.
Ditching lift's DB connection management in favor of squeryl's solved it.
I should manage a connection to a database from my web application (Tomcatserver/frameworkJSF) ,theconnection from my login page until the deconnection.
I have found that I should let the container manage the connection/deconnection to the database but the example that I found show how to configure tomcat server (http://christophej.developpez.com/tutoriel/j2ee/pooltomcat/)and use the connection from a servlet but I want to use the connection from my beans.
how can I instanciate a connection in my classess each time without creation of a new connection?
You create a datasource in Tomcat or somewhere else which is a pool of connections. You take a connection from that pool and then release it when you're done with it. Tomcat example (in context.xml):
<Resource auth="Container" driverClassName="com.mysql.jdbc.Driver"
maxActive="100" maxIdle="30" maxWait="10000" name="jdbc/yourname"
password="..." type="javax.sql.DataSource"
url="jdbc:mysql://localhost:3306/mydb?characterEncoding=utf8"
username="..."/>
This creates a pool with a JNDI name 'jdbc/yourname'. You can get this from Spring like this:
<jee:jndi-lookup id="dataSource" jndi-name="java:comp/env/jdbc/yourname" />
This will create a DataSource object that you can inject into your Java code as needed.
Or there are plenty of JNDI lookup examples on the web to do this programmatically instead of declaratively.
There are also loads of pool implementations, like c3po.