After a day grails program hangs. Need solution - mysql

Here is the log. Problem occurse after a day with no interaction whatsoever. Is it the asynchronous mail plugin? is it a setting? Who can help me with this, any help is appreciated.
Grails 2.4.0, ubuntu 16.04
The last packet successfully received from the server was 278.019 milliseconds ago. The last packet sent successfully to the server was 218.021 milliseconds ago.
ERROR 2017-01-19 10:37:47,642 interceptor.TransactionInterceptor Application exception overridden by rollback exception
org.springframework.dao.RecoverableDataAccessException: Hibernate operation: could not prepare statement; SQL [update async_mail_mess set status=? where end_date<? and (status=? or status=?)]; Communications link failure
The last packet successfully received from the server was 278.019 milliseconds ago. The last packet sent successfully to the server was 218.021 milliseconds ago.; nested exception is com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure
The last packet successfully received from the server was 278.019 milliseconds ago. The last packet sent successfully to the server was 218.021 milliseconds ago.
at grails.plugin.asyncmail.AsynchronousMailPersistenceService$_updateExpiredMessages_closure4.doCall(AsynchronousMailPersistenceService.groovy:66)
at org.grails.datastore.gorm.GormStaticApi.withTransaction(GormStaticApi.groovy:814)
at org.grails.datastore.gorm.GormStaticApi.withTransaction(GormStaticApi.groovy:714)
at grails.plugin.asyncmail.AsynchronousMailPersistenceService.updateExpiredMessages(AsynchronousMailPersistenceService.groovy:63)
at grails.plugin.asyncmail.ExpiredMessagesCollectorJob.execute(ExpiredMessagesCollectorJob.groovy:13)
at grails.plugins.quartz.GrailsJobFactory$GrailsJob.execute(GrailsJobFactory.java:104)
at org.quartz.core.JobRunShell.run(JobRunShell.java:202)
at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:573)
Caused by: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure
The last packet successfully received from the server was 278.019 milliseconds ago. The last packet sent successfully to the server was 218.021 milliseconds ago.
at com.mysql.jdbc.Util.handleNewInstance(Util.java:411)
at com.mysql.jdbc.SQLError.createCommunicationsException(SQLError.java:1116)
at com.mysql.jdbc.MysqlIO.send(MysqlIO.java:3853)
at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2471)
at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2651)
at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2671)
at com.mysql.jdbc.ConnectionImpl.setAutoCommit(ConnectionImpl.java:5024)
... 8 more
Caused by: java.net.SocketException: Broken pipe (Write failed)
at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:109)
at java.net.SocketOutputStream.write(SocketOutputStream.java:153)
at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
at com.mysql.jdbc.MysqlIO.send(MysqlIO.java:3834)
... 12 more
ERROR 2017-01-19 10:37:47,642 listeners.ExceptionPrinterJobListener Exception occurred in job: Grails Job
org.quartz.JobExecutionException: org.springframework.transaction.TransactionSystemException: Could not roll back Hibernate transaction; nested exception is org.hibernate.TransactionException: rollback failed [See nested exception: org.springframework.transaction.TransactionSystemException: Could not roll back Hibernate transaction; nested exception is org.hibernate.TransactionException: rollback failed]
at grails.plugins.quartz.GrailsJobFactory$GrailsJob.execute(GrailsJobFactory.java:111)
at org.quartz.core.JobRunShell.run(JobRunShell.java:202)
at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:573)
Caused by: org.springframework.transaction.TransactionSystemException: Could not roll back Hibernate transaction; nested exception is org.hibernate.TransactionException: rollback failed
at grails.plugin.asyncmail.ExpiredMessagesCollectorJob.execute(ExpiredMessagesCollectorJob.groovy:13)
at grails.plugins.quartz.GrailsJobFactory$GrailsJob.execute(GrailsJobFactory.java:104)
... 2 more
Caused by: org.hibernate.TransactionException: rollback failed
... 4 more
Caused by: org.hibernate.TransactionException: unable to rollback against JDBC connection
... 4 more
Caused by: com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException: Communications link failure during rollback(). Transaction resolution unknown.
at com.mysql.jdbc.Util.handleNewInstance(Util.java:411)
at com.mysql.jdbc.Util.getInstance(Util.java:386)
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:1013)
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:987)
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:982)
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:927)
at com.mysql.jdbc.ConnectionImpl.rollback(ConnectionImpl.java:4757)
... 4 more
ERROR 2017-01-19 10:38:09,620 spi.SqlExceptionHelper Communications link failure

It could be because of two reasons
1.) your grails box got disconnected from the network.
2.) Your database server stops.

You can add configuration parameters to your dataSource config to ensure the DB connection is kept alive
dataSource {
properties {
validationQuery=SELECT 1 FROM DUAL
validationQueryTimeout=3
validationInterval=15000
}
}
The validationQuery will depend on your database vendor e.g. the above works with Oracle

Related

MySQL SELECT fails without LIMIT

I'm connecting to a MySQL server with the node jdbc package. The purpose is to duplicate data from an old AS400 into a MySQL server for more advanced querying.
My connection string looks like this: jdbc:mysql://1.1.1.1:3306/db_name?allowPublicKeyRetrieval=true&useSSL=false.
I'm able to use the connection normally to truncate and insert records for the cache. I can select from the table if I include a LIMIT, but when I try to retrieve the data with the below sql I get the following error.
select * from table_name;
Complete error output:
node:internal/process/promises:218
triggerUncaughtException(err, true /* fromPromise */);
^
[Error: Error running instance method
com.mysql.cj.jdbc.exceptions.CommunicationsException: Communications link failure
The last packet successfully received from the server was 1,756,928 milliseconds ago. The last packet sent successfully to the server was 1,756,972 milliseconds ago.
at com.mysql.cj.jdbc.exceptions.SQLError.createCommunicationsException(SQLError.java:174)
at com.mysql.cj.jdbc.exceptions.SQLExceptionsMapping.translateException(SQLExceptionsMapping.java:64)
at com.mysql.cj.jdbc.StatementImpl.executeQuery(StatementImpl.java:1202)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
Caused by: com.mysql.cj.exceptions.CJCommunicationsException: Communications link failure
The last packet successfully received from the server was 1,756,928 milliseconds ago. The last packet sent successfully to the server was 1,756,972 milliseconds ago.
at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490)
at com.mysql.cj.exceptions.ExceptionFactory.createException(ExceptionFactory.java:61)
at com.mysql.cj.exceptions.ExceptionFactory.createException(ExceptionFactory.java:105)
at com.mysql.cj.exceptions.ExceptionFactory.createException(ExceptionFactory.java:151)
at com.mysql.cj.exceptions.ExceptionFactory.createCommunicationsException(ExceptionFactory.java:167)
at com.mysql.cj.protocol.a.NativeProtocol.readMessage(NativeProtocol.java:519)
at com.mysql.cj.protocol.a.NativeProtocol.checkErrorMessage(NativeProtocol.java:683)
at com.mysql.cj.protocol.a.NativeProtocol.sendCommand(NativeProtocol.java:622)
at com.mysql.cj.protocol.a.NativeProtocol.sendQueryPacket(NativeProtocol.java:970)
at com.mysql.cj.protocol.a.NativeProtocol.sendQueryString(NativeProtocol.java:916)
at com.mysql.cj.NativeSession.execSQL(NativeSession.java:660)
at com.mysql.cj.jdbc.StatementImpl.executeQuery(StatementImpl.java:1170)
... 4 more
Caused by: java.net.SocketException: Connection reset
at java.base/java.net.SocketInputStream.read(SocketInputStream.java:186)
at java.base/java.net.SocketInputStream.read(SocketInputStream.java:140)
at com.mysql.cj.protocol.ReadAheadInputStream.fill(ReadAheadInputStream.java:107)
at com.mysql.cj.protocol.ReadAheadInputStream.readFromUnderlyingStreamIfNecessary(ReadAheadInputStream.java:150)
at com.mysql.cj.protocol.ReadAheadInputStream.read(ReadAheadInputStream.java:180)
at java.base/java.io.FilterInputStream.read(FilterInputStream.java:133)
at com.mysql.cj.protocol.FullReadInputStream.readFully(FullReadInputStream.java:64)
at com.mysql.cj.protocol.a.SimplePacketReader.readHeader(SimplePacketReader.java:63)
at com.mysql.cj.protocol.a.SimplePacketReader.readHeader(SimplePacketReader.java:45)
at com.mysql.cj.protocol.a.TimeTrackingPacketReader.readHeader(TimeTrackingPacketReader.java:52)
at com.mysql.cj.protocol.a.TimeTrackingPacketReader.readHeader(TimeTrackingPacketReader.java:41)
at com.mysql.cj.protocol.a.MultiPacketReader.readHeader(MultiPacketReader.java:54)
at com.mysql.cj.protocol.a.MultiPacketReader.readHeader(MultiPacketReader.java:44)
at com.mysql.cj.protocol.a.NativeProtocol.readMessage(NativeProtocol.java:513)
... 10 more
] {
cause: nodeJava_com_mysql_cj_jdbc_exceptions_CommunicationsException {}
}
Any help in clarifying why this is happening would be helpful and most appreciated!

Communications link failure Error sometimes happens in AppEngine when connecting to google Cloud SQL

I published a backend project in google App Engine since 1 year, and all seems good, 1 week ago this Exception started to throw in logs sometimes (not always) during executing a JPA select query in Cloud SQL and sometimes the query is just returning the result successfully:
Internal Exception: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure
The last packet successfully received from the server was 309,979 milliseconds ago. The last packet sent successfully to the server was 309,981 milliseconds ago.
Error Code: 0
Call: SELECT ID, created_time, CURRENCY, IP, NAME FROM country_table
Query: ReadAllQuery(name="Country.findAll" referenceClass=Country sql="SELECT ID, created_time, CURRENCY, IP, NAME FROM country_table")
and sometimes this exception is stopping some queries to be executed:
The last packet successfully received from the server was 219,443 milliseconds ago. The last packet sent successfully to the server was 219,446 milliseconds ago.
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:44)
at com.mysql.jdbc.Util.handleNewInstance(Util.java:411)
at com.mysql.jdbc.SQLError.createCommunicationsException(SQLError.java:1117)
at com.mysql.jdbc.MysqlIO.send(MysqlIO.java:3851)
at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2471)
at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2651)
at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2739)
at com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:2149)
at com.mysql.jdbc.PreparedStatement.executeQuery(PreparedStatement.java:2313)
at org.eclipse.persistence.internal.databaseaccess.DatabaseAccessor.executeSelect(DatabaseAccessor.java:1007)
at org.eclipse.persistence.internal.databaseaccess.DatabaseAccessor.basicExecuteCall(DatabaseAccessor.java:642)
at org.eclipse.persistence.internal.databaseaccess.DatabaseAccessor.executeCall(DatabaseAccessor.java:558)
at org.eclipse.persistence.internal.sessions.AbstractSession.basicExecuteCall(AbstractSession.java:2002)
at org.eclipse.persistence.sessions.server.ServerSession.executeCall(ServerSession.java:570)
at org.eclipse.persistence.sessions.server.ClientSession.executeCall(ClientSession.java:250)
This is because MySQL closed the connection.
Probably JPA will automatically reconnect to the database. If the transaction result is correct, you can ignore this message.
Or you can extend the timer until connection close with interactive_timeout and wait_timeout on developer console. This can reduce the message.

Percona connection fails after timeout

We have multimodule project with database, and sometimes we recieve this exception:
org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [SELECT data FROM table]; No operations allowed after connection closed.; nested exception is com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException: No operations allowed after connection closed.
at org.springframework.jdbc.support.SQLExceptionSubclassTranslator.doTranslate(SQLExceptionSubclassTranslator.java:79)
...
at java.lang.Thread.run(Thread.java:745)
Caused by: com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException: No operations allowed after connection closed.
at sun.reflect.GeneratedConstructorAccessor108.newInstance(Unknown Source)
... 87 common frames omitted
Caused by: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: The last packet successfully received from the server was 51,072,384 milliseconds ago. The last packet sent successfully to the server was 51,072,384 milliseconds ago. is longer than the server configured value of 'wait_timeout'. You should consider either expiring and/or testing connection validity before use in your application, increasing the server configured values for client timeouts, or using the Connector/J connection property 'autoReconnect=true' to avoid this problem.
at sun.reflect.GeneratedConstructorAccessor89.newInstance(Unknown Source)
... 78 common frames omitted
Caused by: java.net.SocketException: Broken pipe
at java.net.SocketOutputStream.socketWrite0(Native Method)
... 83 common frames omitted
Google suggested that it could be because of database closes connections when no queries in time specified in wait_timeout variable.
So I reprodused this situation in that way:
Use Percona server 5.7.16, SpringBoot 1.3.3
Set timeouts in mysql to demonstrate problem (in production sets default value 8 hours):
SET GLOBAL interactive_timeout = 15;
SET GLOBAL wait_timeout = 15;
Set heartbeat for mysql in springboot properties:
spring.datasource.validationQuery=SELECT 1
spring.datasource.testWhileIdle = true
Run test:
#Test
#DatabaseSetup("/....xml")
public void test() throws Exception {
mockMvc
.perform(get("/request_that_fetches_data_from_db"))
.andExpect(status().isOk());
sleep(1000 * 60);
mockMvc
.perform(get("/request_that_fetches_data_from_db"))
.andExpect(status().isOk());
}
Result is:
java.sql.SQLException: Could not retrieve transation read-only status server
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:957)
...
Caused by: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure
The last packet successfully received from the server was 40 414 milliseconds ago. The last packet sent successfully to the server was 1 milliseconds ago.
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
...
Caused by: java.io.EOFException: Can not read response from server. Expected to read 4 bytes, read 0 bytes before connection was unexpectedly lost.
at com.mysql.jdbc.MysqlIO.readFully(MysqlIO.java:2957)
... 46 more
I looked at mysql general log. There are query SELECT 1 passes every 10 seconds, but when after that recieve business query, then straight recieve command quit. Error log was empty for testing period.
What is wrong and how to fix this problem without increasing wait_timeout?
P.S.jdbc:mysql://url?autoReconnect=true don't works too.

What's causing "error executing query: connection timed out" in my ColdFUsion/MySql?

I get this somewhat frequent error and I don't know what's causing the trouble. My site and MySQL database are hosted at hosting.com.
Can someone help me decode this? I want to understand what this error is desperately trying to tell me so I can fix the problem.
com.mysql.jdbc.CommunicationsException: Communications link failure due to underlying exception: ** BEGIN NESTED EXCEPTION ** java.net.ConnectException MESSAGE: Connection timed out: connect STACKTRACE: java.net.ConnectException: Connection timed out: connect at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:333) at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:195) at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:182) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366) at java.net.Socket.connect(Socket.java:519) at java.net.Socket.connect(Socket.java:469) at java.net.Socket.<init>(Socket.java:366) at java.net.Socket.<init>(Socket.java:209) at com.mysql.jdbc.StandardSocketFactory.connect(StandardSocketFactory.java:173) at com.mysql.jdbc.MysqlIO.<init>(MysqlIO.java:267) at com.mysql.jdbc.Connection.createNewIO(Connection.java:2739) at com.mysql.jdbc.Connection.<init>(Connection.java:1553) at com.mysql.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:266) at coldfusion.server.j2ee.sql.pool.JDBCPool.createPhysicalConnection(JDBCPool.java:589) at coldfusion.server.j2ee.sql.pool.ConnectionRunner$RunnableConnection.run(ConnectionRunner.java:67) at java.lang.Thread.run(Thread.java:619) ** END NESTED EXCEPTION ** Last packet sent to the server was 0 ms ago. at com.mysql.jdbc.Connection.createNewIO(Connection.java:2814) at com.mysql.jdbc.Connection.<init>(Connection.java:1553) at com.mysql.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:266) at coldfusion.server.j2ee.sql.pool.JDBCPool.createPhysicalConnection(JDBCPool.java:589) at coldfusion.server.j2ee.sql.pool.ConnectionRunner$RunnableConnection.run(ConnectionRunner.java:67) at java.lang.Thread.run(Thread.java:619)
It's pretty simple. It seems their database is on a remote machine and as the exception says: there is a network failure preventing the CF application talking to the database.
You need to get hosting.com support on the case. Provide the exception details and the times they occurred.
Hope that helps.

ibatis + glassfish: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure [duplicate]

This question already has answers here:
com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure
(51 answers)
Closed 6 years ago.
I keep getting the error randomly:
com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure
Anyone got ideas?
ERROR: java.sql.Connection - Error calling Connection.setAutoCommit:
com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure
The last packet successfully received from the server was 129,064 milliseconds ago. The last packet sent successfully to the server was 0 milliseconds ago.
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
at com.mysql.jdbc.Util.handleNewInstance(Util.java:409)
at com.mysql.jdbc.SQLError.createCommunicationsException(SQLError.java:1118)
at com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:3055)
at com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:2941)
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3489)
Caused by: java.io.EOFException: Can not read response from server. Expected to read 4 bytes, read 0 bytes before connection was unexpectedly lost.
at com.mysql.jdbc.MysqlIO.readFully(MysqlIO.java:2502)
at com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:2952)
... 83 more
The MySQL server does not listen on the TCP port 3306 by default. To allow (remote) TCP connections, comment the following line in /etc/mysql/my.cnf:
skip-networking