I am using Slick and Mysql. Getting the Error:
Failed to validate connection com.mysql.jdbc.JDBC4Connection#58e7cfcd (No operations allowed after connection closed.). Possibly consider using a shorter maxLifetime value."
I understand the issue but don't know how to set maxLifetime in slick on config. Here is my config:
mysqldb {
driver = "com.mysql.cj.jdbc.Driver"
url = "jdbc:mysql://localhost:3306"
user = root
password = root
connectionTimeout = 10000
queueSize = 50000
minConnections = 20
maxConnections = 40
numThreads = 40
}
Any help would be appreciated.
you can set under db config:
mysqldb {
driver = "com.mysql.cj.jdbc.Driver"
url = "jdbc:mysql://localhost:3306"
user = root
password = root
connectionTimeout = 10000
queueSize = 50000
minConnections = 20
maxConnections = 40
numThreads = 40
idleTimeout = 85000 // should be less than maxLifetime
maxLifetime = 90000 //time is in millsecods and should be less than database wait_timeout
}
Related
I'm getting this error "No operations allowed after connection closed" from grils2x/mysql/dbcp occasionally and I couldn't find a solution.
because I get the error hours later, like next day after a restart, it's difficult to fix it.
I feel like I need to replicate it in a predictable manner and so I can find a definitive fix.
What parameter set can I use for MySQL and Grails side that ends up with that error immediately after it is run?
I ended up setting two timeouts manually , and make a connection afterwards , and see the following in mysql logs , to confirm connection was aborted after 10 seconds.
2022-02-12T22:03:53.690960Z 493 [Note] Aborted connection 493 to db: 'quantanywhere_2' user: 'root' host: '172.17.0.1' (Got timeout reading communication packets)
SET ##GLOBAL.wait_timeout=10;
SET ##SESSION.wait_timeout=10;
And following should be ok with the default wait_timeout of 28800 seconds=8 hours.
validationInterval = 28000
testWhileIdle=true
maxActive = 50
maxIdle = 25
maxWait = 10000
maxAge: 600000
minIdle = 5
validationQuery="select 1"
validationQueryTimeout=3
initialSize = 10
minEvictableIdleTimeMillis = 60000
timeBetweenEvictionRunsMillis = 5000
numTestsPerEvictionRun = 3
testOnBorrow = true
testWhileIdle = true
testOnReturn = false
removeAbandoned = true
removeAbandonedTimeout = 120
I have attempted official document for postgresql and change org.postgresql.ds.PGSimpleDataSource to slick.driver.MySQLDriver$
dataSourceClass = "slick.driver.MySQLDriver$"
properties = {
databaseName = "mydb"
user = "user"
password = "password"
}
numThreads = 10
scala code:
val config = DatabaseConfig.forConfig[JdbcProfile]("slick.mysql.local")
Output error:
Exception in thread "main" com.typesafe.config.ConfigException$Missing:
No configuration setting found for key 'slick.mysql.local.profile'
Seems it need a profile property but what value should be added?
============UPDATE========
config properties are under brackets:
slick.mysql.local {
dataSourceClass = "slick.driver.MySQLDriver$"
properties = {
databaseName = "mydb"
user = "user"
password = "password"
}
numThreads = 10
}
If you carefully read the docs, what they say is:
define a mydb key in some application.conf with your configuration
refer to that configuration using val db = Database.forConfig("mydb")
So I can only assume you're missing your slick.mysql.local key.
I also found it difficult to configure the database connector. Here is my config:
mysql {
dataSourceClass = "com.mysql.jdbc.jdbc2.optional.MysqlDataSource"
properties = {
url = "jdbc:mysql://127.0.0.1:3306/dbName"
databaseName = "dbName" <= maybe not necessaray as also in the url
user = "root"
password = ""
}
numThreads = 10
}
Thanks to https://groups.google.com/forum/#!topic/scalaquery/WbR6yHzBj_8
I struggle some days and find a config can works.
driver = "slick.driver.MySQLDriver$"
db {
url = "jdbc:mysql://127.0.0.1:3306/develop_tdsystem?user=user&password=password"
driver = com.mysql.jdbc.Driver
maxThreads = 10
}
please notice there are some warning:
Loading class `com.mysql.jdbc.Driver'. This is deprecated. The new driver class is `com.mysql.cj.jdbc.Driver'. The driver is automatically registered via the SPI and manual loading of the driver class is generally unnecessary.
[warn] c.z.h.u.DriverDataSource - Registered driver with driverClassName=com.mysql.jdbc.Driver was not found, trying direct instantiation.
Most examples of JNDI, mysql 5.6 and tomcat 7 have this defined in DataSource.groovy:
dataSource {
pooled = true
driverClassName = "com.mysql.jdbc.Driver"
dialect = 'org.hibernate.dialect.MySQL5InnoDBDialect'
}
environments:
production {
dataSource {
dbCreate = "update"
jndiName = "java:comp/env/myDatasourceName
}
}
and in the conf/context.xml in tomcat:
<Context>
<WatchedResource>WEB-INF/web.xml</WatchedResource>
<Resource name="myDatasourceName" auth="Container" type="javax.sql.DataSource"
maxActive="100" maxIdle="30" maxWait="10000"
username="root" password="password" driverClassName="com.mysql.jdbc.Driver"
url="jdbc:mysql://localhost:3306/my_db_name"/>
</Context>
The question is, is this the correct setup for a basic low volume production system, which should reconnect if the db goes down then up etc?
In the DataSource.xml is a hugely complex example:
dataSource {
dbCreate = "update"
//url = "jdbc:h2:prodDb;MVCC=TRUE;LOCK_TIMEOUT=10099;DB_CLOSE_ON_EXIT=FALSE"
properties {
// Documentation for Tomcat JDBC Pool
// http://tomcat.apache.org/tomcat-7.0-doc/jdbc-pool.html#Common_Attributes
// https://tomcat.apache.org/tomcat-7.0-doc/api/org/apache/tomcat/jdbc/pool/PoolConfiguration.html
jmxEnabled = true
initialSize = 5
maxActive = 50
minIdle = 5
maxIdle = 25
maxWait = 10000
maxAge = 10 * 60000
timeBetweenEvictionRunsMillis = 5000
minEvictableIdleTimeMillis = 60000
validationQuery = "SELECT 1"
validationQueryTimeout = 3
validationInterval = 15000
testOnBorrow = true
testWhileIdle = true
testOnReturn = false
ignoreExceptionOnPreLoad = true
// http://tomcat.apache.org/tomcat-7.0-doc/jdbc-pool.html#JDBC_interceptors
jdbcInterceptors = "ConnectionState;StatementCache(max=200)"
defaultTransactionIsolation = java.sql.Connection.TRANSACTION_READ_COMMITTED // safe default
// controls for leaked connections
abandonWhenPercentageFull = 100 // settings are active only when pool is full
removeAbandonedTimeout = 120000
removeAbandoned = true
// use JMX console to change this setting at runtime
logAbandoned = false // causes stacktrace recording overhead, use only for debugging
/*
// JDBC driver properties
// Mysql as example
dbProperties {
// Mysql specific driver properties
// http://dev.mysql.com/doc/connector-j/en/connector-j-reference-configuration-properties.html
// let Tomcat JDBC Pool handle reconnecting
autoReconnect=false
// truncation behaviour
jdbcCompliantTruncation=false
// mysql 0-date conversion
zeroDateTimeBehavior='convertToNull'
// Tomcat JDBC Pool's StatementCache is used instead, so disable mysql driver's cache
cachePrepStmts=false
cacheCallableStmts=false
// Tomcat JDBC Pool's StatementFinalizer keeps track
dontTrackOpenResources=true
// performance optimization: reduce number of SQLExceptions thrown in mysql driver code
holdResultsOpenOverStatementClose=true
// enable MySQL query cache - using server prep stmts will disable query caching
useServerPrepStmts=false
// metadata caching
cacheServerConfiguration=true
cacheResultSetMetadata=true
metadataCacheSize=100
// timeouts for TCP/IP
connectTimeout=15000
socketTimeout=120000
// timer tuning (disable)
maintainTimeStats=false
enableQueryTimeouts=false
// misc tuning
noDatetimeStringSync=true
}
*/
}
}
Now I dont know what 95% of those parameters do, but I guess I need them all?
What I dont understand is this
surely tomcat will pool the connections, so should pooled=true in the grails dataSource?
Surely tomcat will handle reconnecting and validation of connection, so why define validationQuery = "SELECT 1"? Expecially as its saying "autoREconnect=false in the mysql dbPrperties section.
Could someone come up with a general purpose default dataSource definition suitable for mysql for a small number of users? I can tune the min/max active/idle etc, but is this not all done in tomcat, not grails?
"userServerPrepStmts=false" why would anyone want to disable prepared statements? We have been drilled for the last 15 years to only ever use prepared statements otherwise the DB cant cache them.
If I ignore all this complex example, and went with the super smiple one everyone posts, will it work properly? e.g. will it reconnect when the db goes down and up? will it not leak etc.
Thanks!
AFAIK, Tomcat has nothing to do with managing the database connection. Since Grails 2.3.6, these are the recommended default datasource properties:
properties {
//see http://grails.org/doc/latest/guide/conf.html#dataSource for documentation
jmxEnabled = true
initialSize = 5
maxActive = 50
minIdle = 5
maxIdle = 25
maxWait = 10000
maxAge = 10 * 60000
timeBetweenEvictionRunsMillis = 5000
minEvictableIdleTimeMillis = 60000
validationQuery = "SELECT 1"
validationQueryTimeout = 3
validationInterval = 15000
testOnBorrow = true
testWhileIdle = true
testOnReturn = false
jdbcInterceptors = "ConnectionState;StatementCache(max=200)"
defaultTransactionIsolation = java.sql.Connection.TRANSACTION_READ_COMMITTED
}
I'm using lighttpd and pfp-fpm.
All work correctly, but if I call a mysql_connect() on my php scripts I get an empty page.
Like a "denied operation".
This is my php-fpm.conf pool
[example.com]
listen = 127.0.0.1:9001
listen.backlog = -1
user = example.com
group = example.com
pm = dynamic
pm.max_requests = 0
pm.max_children = 2
pm.start_servers = 1
pm.min_spare_servers = 1
pm.max_spare_servers = 1
chroot = /home/vhosts/example.com/
request_terminate_timeout = 2
request_slowlog_timeout = 1
slowlog = /home/vhosts/example.com/log/php-slow.log
catch_workers_output = yes
env[HOSTNAME] = $HOSTNAME
env[PATH] = /usr/local/bin:/usr/bin:/bin
env[TMP] = /tmp
env[TMPDIR] = /tmp
env[TEMP] = /tmp
This is my lighttpd vhost:
$HTTP["host"] =~ "(^|.)example\.com$" {
server.document-root = "/home/vhosts/example.com/web"
server.errorlog = "/home/vhosts/example.com/log/error.log"
accesslog.filename = "/home/vhosts/example.com/log/access.log"
fastcgi.server = (
".php" => (
"localhost" => (
"docroot" => "/web",
"host" => "127.0.0.1",
"port" => "9001"
)
)
)
}
What is wrong? If I don't use mysql I can see the result of the php script correctly.
Turning on error display and posting those errors would be helpful, but since you can use mysql without chrooting php, I guess you are getting some connection error, since php tries to connect to mysql on localhost using socket ( php compiled with --with-mysql-sock=/var/mysql.sock), and most likely this socket is outside your chrooted environment.
You can try:
Recompiling php to not use socket
creating hard link to socket in your chrooted path
using some internal ip (like 10.0.0.1,192.168.0.1) instead of loopback
Seems to me all the problems are because of the user and group being example.com
i guess that user doesnt have rights to perform the requested tasks
I have a small grails application running on Tomcat in Ubuntu on a VPS. I use MySql as my datastore and everything works fine unless I leave the application for more than half a day (8 hours?). I did some searching and apparently this is the default wait_timeout in mysql.cnf so after 8 hours the connection will die but Tomcat won't know so when the next user tries to view the site they will see the connection failure error. Refreshing the page will fix this but I want to get rid of the error altogether. For my version of MySql (5.0.75) I have only my.cnf and it doesn't contain such a parameter, In any case changing this parameter doesn't solve the problem.
This Blog Post seems to be reporting a similar error but I still don't fully understand what I need to configure to get this fixed and also I am hoping that there is a simpler solution than another third party library. The machine I'm running on has 256MB ram and I'm trying to keep the number of programs/services running to a minimum.
Is there something I can configure in Grails / Tomcat / MySql to get this to go away?
Thanks in advance,
Gav
From my Catalina.out;
2010-04-29 21:26:25,946 [http-8080-2] ERROR util.JDBCExceptionReporter - The last packet successfully received from the server was 102,906,722 milliseconds$
2010-04-29 21:26:25,994 [http-8080-2] ERROR errors.GrailsExceptionResolver - Broken pipe
java.net.SocketException: Broken pipe
at java.net.SocketOutputStream.socketWrite0(Native Method)
...
2010-04-29 21:26:26,016 [http-8080-2] ERROR util.JDBCExceptionReporter - Already closed.
2010-04-29 21:26:26,016 [http-8080-2] ERROR util.JDBCExceptionReporter - Already closed.
2010-04-29 21:26:26,017 [http-8080-2] ERROR servlet.GrailsDispatcherServlet - HandlerInterceptor.afterCompletion threw exception
org.hibernate.exception.GenericJDBCException: Cannot release connection
at java.lang.Thread.run(Thread.java:619)
Caused by: java.sql.SQLException: Already closed.
at org.apache.commons.dbcp.PoolableConnection.close(PoolableConnection.java:84)
at org.apache.commons.dbcp.PoolingDataSource$PoolGuardConnectionWrapper.close(PoolingDataSource.java:181)
... 1 more
Referring to this article, you have stale connections in your DBCP connections pool that are silently dropped by OS or firewall.
The solution is to define a validation query and do a sanity check of the connection before you actually use it in your application.
In grails this is actually done by modifying the grails-app/conf/spring/Resource.groovy file and add the following:
beans = {
dataSource(BasicDataSource) {
//run the evictor every 30 minutes and evict any connections older than 30 minutes.
minEvictableIdleTimeMillis=1800000
timeBetweenEvictionRunsMillis=1800000
numTestsPerEvictionRun=3
//test the connection while its idle, before borrow and return it
testOnBorrow=true
testWhileIdle=true
testOnReturn=true
validationQuery="SELECT 1"
}
}
In grails 1.3.X, you can modify the evictor values in the DataSource.groovy file to make sure pooled connections are used during idle. This will make sure the mysql server will not time out the connection.
production {
dataSource {
pooled = true
// Other database parameters..
properties {
maxActive = 50
maxIdle = 25
minIdle = 5
initialSize = 5
minEvictableIdleTimeMillis = 1800000
timeBetweenEvictionRunsMillis = 1800000
maxWait = 10000
}
}
A quick way to verify this works is to modify the MySQL my.cnf configuration file [mysql] element and add wait_time parameter with a low value.
Try increasing the number of open MySQL connections by putting the following in your DataSources.groovy:
dataSource {
driverClassName = "com.mysql.jdbc.Driver"
pooled=true
maxActive=10
initialSize=5
// Remaining connection params
}
If you want to go the whole hog, try implementing a connection pool; here is a useful link on this.
For grails 1.3.X, I had to add the following code to Bootstrap.groovy :
def init = {servletContext ->
def ctx=servletContext.getAttribute(ApplicationAttributes.APPLICATION_CONTEXT)
//implement test on borrow
def dataSource = ctx.dataSource
dataSource.targetDataSource.setMinEvictableIdleTimeMillis(1000 * 60 * 30)
dataSource.targetDataSource.setTimeBetweenEvictionRunsMillis(1000 * 60 * 30)
dataSource.targetDataSource.setNumTestsPerEvictionRun(3)
dataSource.targetDataSource.setTestOnBorrow(true)
dataSource.targetDataSource.setTestWhileIdle(true)
dataSource.targetDataSource.setTestOnReturn(false)
dataSource.targetDataSource.setValidationQuery("SELECT 1")
}
I also had to import org.codehaus.groovy.grails.commons.ApplicationAttributes
Add these parameters to dataSource
testOnBorrow = true
testWhileIdle = true
testOnReturn = true
See this article for more information
http://sacharya.com/grails-dbcp-stale-connections/
Starting from grails 2.3.6 default configuration already has options for preventing closing connection by timeout
These are the new defaults.
properties {
// See http://grails.org/doc/latest/guide/conf.html#dataSource for documentation
....
minIdle = 5
maxIdle = 25
maxWait = 10000
maxAge = 10 * 60000
timeBetweenEvictionRunsMillis = 5000
minEvictableIdleTimeMillis = 60000
validationQuery = "SELECT 1"
validationQueryTimeout = 3
validationInterval = 15000
testOnBorrow = true
testWhileIdle = true
testOnReturn = false
jdbcInterceptors = "ConnectionState;StatementCache(max=200)"
defaultTransactionIsolation = java.sql.Connection.TRANSACTION_READ_COMMITTED
}
What does your JDBC connection string look like? You can set an autoReconneect param in your data source config, e.g.
jdbc:mysql://hostname/mydb?autoReconnect=true