broken pipe exception with MySql in Grails app - mysql

Question

I have deployed a Grails 2.2.1 application to a remote server. After a few hours, when I access to home page of the application, I see the following exception:
Error 500: Internal Server Error
URI
/DocGemStudioZoccali/
Class
java.net.SocketException
Message
Broken pipe
Trace
Line | Method
->> 1145 | runWorker in java.util.concurrent.ThreadPoolExecutor
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
| 615 | run in java.util.concurrent.ThreadPoolExecutor$Worker
^ 722 | run . . . in java.lang.Thread
Caused by GroovyPagesException: Error processing GroovyPageView: could not execute query; nested exception is org.hibernate.exception.JDBCConnectionException: could not execute query
->> 631 | runWorker in /index.gsp
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Caused by DataAccessResourceFailureException: could not execute query; nested exception is org.hibernate.exception.JDBCConnectionException: could not execute query
->> 108 | doCall in org.grails.datastore.gorm.GormStaticApi$_methodMissing_closure2
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
| 25 | doCall in gsp_docGemStudioZoccaliindex_gsp$_run_closure1
| 38 | run . . . in gsp_docGemStudioZoccaliindex_gsp
| 1145 | runWorker in java.util.concurrent.ThreadPoolExecutor
| 615 | run . . . in java.util.concurrent.ThreadPoolExecutor$Worker
^ 722 | run in java.lang.Thread
Caused by JDBCConnectionException: could not execute query
->> 108 | doCall in org.grails.datastore.gorm.GormStaticApi$_methodMissing_closure2
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
| 25 | doCall in gsp_docGemStudioZoccaliindex_gsp$_run_closure1
| 38 | run . . . in gsp_docGemStudioZoccaliindex_gsp
| 1145 | runWorker in java.util.concurrent.ThreadPoolExecutor
| 615 | run . . . in java.util.concurrent.ThreadPoolExecutor$Worker
^ 722 | run in java.lang.Thread
Caused by CommunicationsException: The last packet successfully received from the server was 138,684,859 milliseconds ago. The last packet sent successfully to the server was 138,684,859 milliseconds ago. is longer than the server configured value of 'wait_timeout'. You should consider either expiring and/or testing connection validity before use in your application, increasing the server configured values for client timeouts, or using the Connector/J connection property 'autoReconnect=true' to avoid this problem.
->> 411 | handleNewInstance in com.mysql.jdbc.Util
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
| 1116 | createCommunicationsException in com.mysql.jdbc.SQLError
| 3851 | send . . in com.mysql.jdbc.MysqlIO
| 2471 | sendCommand in ''
| 2651 | sqlQueryDirect in ''
| 2683 | execSQL in com.mysql.jdbc.ConnectionImpl
| 2144 | executeInternal in com.mysql.jdbc.PreparedStatement
| 2310 | executeQuery in ''
| 96 | executeQuery in org.apache.commons.dbcp.DelegatingPreparedStatement
| 108 | doCall in org.grails.datastore.gorm.GormStaticApi$_methodMissing_closure2
| 25 | doCall . in gsp_docGemStudioZoccaliindex_gsp$_run_closure1
| 38 | run in gsp_docGemStudioZoccaliindex_gsp
| 1145 | runWorker in java.util.concurrent.ThreadPoolExecutor
| 615 | run in java.util.concurrent.ThreadPoolExecutor$Worker
^ 722 | run . . . in java.lang.Thread
Caused by SocketException: Broken pipe
->> 109 | socketWrite in java.net.SocketOutputStream
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
| 153 | write in ''
| 82 | flushBuffer in java.io.BufferedOutputStream
| 140 | flush in ''
| 3832 | send . . in com.mysql.jdbc.MysqlIO
| 2471 | sendCommand in ''
| 2651 | sqlQueryDirect in ''
| 2683 | execSQL in com.mysql.jdbc.ConnectionImpl
| 2144 | executeInternal in com.mysql.jdbc.PreparedStatement
| 2310 | executeQuery in ''
| 96 | executeQuery in org.apache.commons.dbcp.DelegatingPreparedStatement
| 108 | doCall in org.grails.datastore.gorm.GormStaticApi$_methodMissing_closure2
| 25 | doCall . in gsp_docGemStudioZoccaliindex_gsp$_run_closure1
| 38 | run in gsp_docGemStudioZoccaliindex_gsp
| 1145 | runWorker in java.util.concurrent.ThreadPoolExecutor
| 615 | run in java.util.concurrent.ThreadPoolExecutor$Worker
^ 722 | run . . . in java.lang.Thread
I've tried to solve the problem as seen in another question in SO, but it does not work for me.
Here is what I've done inside config file:
dataSource {
dbCreate = "update"
url = "jdbc:mysql://urlofapp/myappname?useUnicode=yes&characterEncoding=UTF-8"
username = "root"
password = "root"
//run the evictor every 30 minutes and evict any connections older than 30 minutes.
minEvictableIdleTimeMillis=1800000
timeBetweenEvictionRunsMillis=1800000
numTestsPerEvictionRun=3
//test the connection while its idle, before borrow and return it
testOnBorrow=true
testWhileIdle=true
testOnReturn=true
validationQuery="SELECT 1"
}
Notice that, if I refresh the page, everything works correctly. Anybody knows something to solve this issue?
EDIT 1:
I've edited the my.cnf file in /etc folder, adding the following:
wait_timeout = 28800
interactive_timeout = 28800
but nothing changes
EDIT 2:
This is the actual data source that I'm using. Broken pipe still appears
dataSource {
dbCreate = "update"
url = "jdbc:mysql://urlofapp/myappname?useUnicode=yes&characterEncoding=UTF-8&autoReconnect=true"
username = "root"
password = "root"
minIdle = 5
maxIdle = 25
maxWait = 10000
maxAge = 10 * 60000
timeBetweenEvictionRunsMillis = 5000
minEvictableIdleTimeMillis = 60000
validationQuery = "SELECT 1"
validationQueryTimeout = 3
validationInterval = 15000
testOnBorrow = true
testWhileIdle = true
testOnReturn = false
jdbcInterceptors = "ConnectionState;StatementCache(max=200)"
defaultTransactionIsolation = java.sql.Connection.TRANSACTION_READ_COMMITTED
}
EDIT 3:
Here is the stacktrace after the suggestion given by James Kleeh
2015-12-04 17:41:36,924 [http-bio-8080-exec-63] ERROR context.GrailsContextLoader - Error initializing the application: Error creating bean with name 'transactionManagerPostProcessor': Initialization of bean failed; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'transactionManager': Cannot resolve reference to bean 'sessionFactory' while setting bean property 'sessionFactory'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'sessionFactory': Cannot resolve reference to bean 'dataSource' while setting bean property 'dataSource'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'dataSource': Cannot resolve reference to bean 'dataSourceUnproxied' while setting constructor argument; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'dataSourceUnproxied': Error setting property values; nested exception is org.springframework.beans.NotWritablePropertyException: Invalid property 'maxAge' of bean class [org.apache.commons.dbcp.BasicDataSource]: Bean property 'maxAge' is not writable or has an invalid setter method. Does the parameter type of the setter match the return type of the getter?
org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'transactionManagerPostProcessor': Initialization of bean failed; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'transactionManager': Cannot resolve reference to bean 'sessionFactory' while setting bean property 'sessionFactory'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'sessionFactory': Cannot resolve reference to bean 'dataSource' while setting bean property 'dataSource'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'dataSource': Cannot resolve reference to bean 'dataSourceUnproxied' while setting constructor argument; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'dataSourceUnproxied': Error setting property values; nested exception is org.springframework.beans.NotWritablePropertyException: Invalid property 'maxAge' of bean class [org.apache.commons.dbcp.BasicDataSource]: Bean property 'maxAge' is not writable or has an invalid setter method. Does the parameter type of the setter match the return type of the getter?
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:722)
Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'transactionManager': Cannot resolve reference to bean 'sessionFactory' while setting bean property 'sessionFactory'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'sessionFactory': Cannot resolve reference to bean 'dataSource' while setting bean property 'dataSource'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'dataSource': Cannot resolve reference to bean 'dataSourceUnproxied' while setting constructor argument; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'dataSourceUnproxied': Error setting property values; nested exception is org.springframework.beans.NotWritablePropertyException: Invalid property 'maxAge' of bean class [org.apache.commons.dbcp.BasicDataSource]: Bean property 'maxAge' is not writable or has an invalid setter method. Does the parameter type of the setter match the return type of the getter?
... 3 more
Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'sessionFactory': Cannot resolve reference to bean 'dataSource' while setting bean property 'dataSource'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'dataSource': Cannot resolve reference to bean 'dataSourceUnproxied' while setting constructor argument; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'dataSourceUnproxied': Error setting property values; nested exception is org.springframework.beans.NotWritablePropertyException: Invalid property 'maxAge' of bean class [org.apache.commons.dbcp.BasicDataSource]: Bean property 'maxAge' is not writable or has an invalid setter method. Does the parameter type of the setter match the return type of the getter?
... 3 more
Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'dataSource': Cannot resolve reference to bean 'dataSourceUnproxied' while setting constructor argument; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'dataSourceUnproxied': Error setting property values; nested exception is org.springframework.beans.NotWritablePropertyException: Invalid property 'maxAge' of bean class [org.apache.commons.dbcp.BasicDataSource]: Bean property 'maxAge' is not writable or has an invalid setter method. Does the parameter type of the setter match the return type of the getter?
... 3 more
Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'dataSourceUnproxied': Error setting property values; nested exception is org.springframework.beans.NotWritablePropertyException: Invalid property 'maxAge' of bean class [org.apache.commons.dbcp.BasicDataSource]: Bean property 'maxAge' is not writable or has an invalid setter method. Does the parameter type of the setter match the return type of the getter?
... 3 more
Caused by: org.springframework.beans.NotWritablePropertyException: Invalid property 'maxAge' of bean class [org.apache.commons.dbcp.BasicDataSource]: Bean property 'maxAge' is not writable or has an invalid setter method. Does the parameter type of the setter match the return type of the getter?
... 3 more
2015-12-04 17:41:36,931 [http-bio-8080-exec-63] ERROR context.GrailsContextLoader - Error initializing Grails: Error creating bean with name 'transactionManagerPostProcessor': Initialization of bean failed; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'transactionManager': Cannot resolve reference to bean 'sessionFactory' while setting bean property 'sessionFactory'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'sessionFactory': Cannot resolve reference to bean 'dataSource' while setting bean property 'dataSource'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'dataSource': Cannot resolve reference to bean 'dataSourceUnproxied' while setting constructor argument; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'dataSourceUnproxied': Error setting property values; nested exception is org.springframework.beans.NotWritablePropertyException: Invalid property 'maxAge' of bean class [org.apache.commons.dbcp.BasicDataSource]: Bean property 'maxAge' is not writable or has an invalid setter method. Does the parameter type of the setter match the return type of the getter?
org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'transactionManagerPostProcessor': Initialization of bean failed; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'transactionManager': Cannot resolve reference to bean 'sessionFactory' while setting bean property 'sessionFactory'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'sessionFactory': Cannot resolve reference to bean 'dataSource' while setting bean property 'dataSource'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'dataSource': Cannot resolve reference to bean 'dataSourceUnproxied' while setting constructor argument; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'dataSourceUnproxied': Error setting property values; nested exception is org.springframework.beans.NotWritablePropertyException: Invalid property 'maxAge' of bean class [org.apache.commons.dbcp.BasicDataSource]: Bean property 'maxAge' is not writable or has an invalid setter method. Does the parameter type of the setter match the return type of the getter?
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:722)
Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'transactionManager': Cannot resolve reference to bean 'sessionFactory' while setting bean property 'sessionFactory'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'sessionFactory': Cannot resolve reference to bean 'dataSource' while setting bean property 'dataSource'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'dataSource': Cannot resolve reference to bean 'dataSourceUnproxied' while setting constructor argument; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'dataSourceUnproxied': Error setting property values; nested exception is org.springframework.beans.NotWritablePropertyException: Invalid property 'maxAge' of bean class [org.apache.commons.dbcp.BasicDataSource]: Bean property 'maxAge' is not writable or has an invalid setter method. Does the parameter type of the setter match the return type of the getter?
... 3 more
Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'sessionFactory': Cannot resolve reference to bean 'dataSource' while setting bean property 'dataSource'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'dataSource': Cannot resolve reference to bean 'dataSourceUnproxied' while setting constructor argument; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'dataSourceUnproxied': Error setting property values; nested exception is org.springframework.beans.NotWritablePropertyException: Invalid property 'maxAge' of bean class [org.apache.commons.dbcp.BasicDataSource]: Bean property 'maxAge' is not writable or has an invalid setter method. Does the parameter type of the setter match the return type of the getter?
... 3 more
Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'dataSource': Cannot resolve reference to bean 'dataSourceUnproxied' while setting constructor argument; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'dataSourceUnproxied': Error setting property values; nested exception is org.springframework.beans.NotWritablePropertyException: Invalid property 'maxAge' of bean class [org.apache.commons.dbcp.BasicDataSource]: Bean property 'maxAge' is not writable or has an invalid setter method. Does the parameter type of the setter match the return type of the getter?
... 3 more
Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'dataSourceUnproxied': Error setting property values; nested exception is org.springframework.beans.NotWritablePropertyException: Invalid property 'maxAge' of bean class [org.apache.commons.dbcp.BasicDataSource]: Bean property 'maxAge' is not writable or has an invalid setter method. Does the parameter type of the setter match the return type of the getter?
... 3 more
Caused by: org.springframework.beans.NotWritablePropertyException: Invalid property 'maxAge' of bean class [org.apache.commons.dbcp.BasicDataSource]: Bean property 'maxAge' is not writable or has an invalid setter method. Does the parameter type of the setter match the return type of the getter?
... 3 more
Dec 04, 2015 5:41:36 PM org.apache.catalina.core.StandardContext startInternal
SEVERE: Error listenerStart
Dec 04, 2015 5:41:36 PM org.apache.catalina.core.StandardContext startInternal
SEVERE: Context [/DocGemStudioTest] startup failed due to previous errors
2015-12-04 17:41:36,955 [http-bio-8080-exec-63] WARN lifecycle.ShutdownOperations - Error occurred running shutdown operation: BeanFactory not initialized or already closed - call 'refresh' before accessing beans via the ApplicationContext
java.lang.IllegalStateException: BeanFactory not initialized or already closed - call 'refresh' before accessing beans via the ApplicationContext
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:722)
Dec 04, 2015 5:41:36 PM org.apache.catalina.loader.WebappClassLoader clearReferencesJdbc
SEVERE: The web application [/DocGemStudioTest] registered the JDBC driver [com.mysql.jdbc.Driver] but failed to unregister it when the web application was stopped. To prevent a memory leak, the JDBC Driver has been forcibly unregistered.
EDIT 4:
I've edited the properties block and now application starts. I need to check if error occurs after few hours.
properties{
minIdle = 5
maxIdle = 25
maxWait = 10000
timeBetweenEvictionRunsMillis = 5000
minEvictableIdleTimeMillis = 60000
validationQuery = "SELECT 1"
validationQueryTimeout = 3
testOnBorrow = true
testWhileIdle = true
testOnReturn = false
defaultTransactionIsolation = java.sql.Connection.TRANSACTION_READ_COMMITTED
}

Answer 1

So I believe everything other than dbCreate, url, username, and password needs to be in a properties block:
dataSource {
dbCreate = "update"
url = "jdbc:mysql://urlofapp/myappname?useUnicode=yes&characterEncoding=UTF-8&autoReconnect=true"
username = "root"
password = "root"
properties {
minIdle = 5
maxIdle = 25
maxWait = 10000
maxAge = 10 * 60000
timeBetweenEvictionRunsMillis = 5000
minEvictableIdleTimeMillis = 60000
validationQuery = "SELECT 1"
validationQueryTimeout = 3
validationInterval = 15000
testOnBorrow = true
testWhileIdle = true
testOnReturn = false
jdbcInterceptors = "ConnectionState;StatementCache(max=200)"
defaultTransactionIsolation = java.sql.Connection.TRANSACTION_READ_COMMITTED
}
}

Answer 2

Although, this should be strange but if the Grails app and the MySQL server both are on the same machine then use localhost or 127.0.0. instead of real IP address xxx.xxx.xxx.xxx
dataSource {
dbCreate = "update"
url = "jdbc:mysql://localhost:3306/docgem_test_db?useUnicode=yes&characterEncoding=UTF-8&autoReconnect=true"
// Your other properties
}

Answer 3

Please try the below not the addition of driverClassName and dialect and I also added pooled in case you need that
dataSource {
pooled = true
driverClassName = "com.mysql.jdbc.Driver"
dialect = "org.hibernate.dialect.MySQL5InnoDBDialect"
dbCreate = "update"
url = "jdbc:mysql://urlofapp/myappname? useUnicode=yes&characterEncoding=UTF-8&autoReconnect=true"
username = "root"
password = "root"
properties {
minIdle = 5
maxIdle = 25
maxWait = 10000
maxAge = 10 * 60000
timeBetweenEvictionRunsMillis = 5000
minEvictableIdleTimeMillis = 60000
validationQuery = "SELECT 1"
validationQueryTimeout = 3
validationInterval = 15000
testOnBorrow = true
testWhileIdle = true
testOnReturn = false
jdbcInterceptors = "ConnectionState;StatementCache(max=200)"
defaultTransactionIsolation = java.sql.Connection.TRANSACTION_READ_COMMITTED
}
}

Answer 4

Looks to me like MySQL is timing out it's connection to you. Probably not much you can do about that.
Can you consider closing the connection if there's no traffic on it for a while and then re-open it when it's needed again?