Session is closed Exception - mysql

I am building an application with Struts 2 hibernate spring 3 with my sql as back end and c3p0 connection pooling(c3p0-0.9.1.1 jar).
Sometimes, when executing a query, I get the below error. When ever I am executing query I check if the connection is closed or not, and if it's closed I will open a new connection before executing the query.
public List<T> find(final Query query, final Object[] values) throws HibernateException, NullPointerException {
if (values != null) {
for (int i = 0; i < values.length; i++) {
query.setParameter(i, values[i]);
}
}
List<T> resultList = null;
if (hibernateSession == null || !hibernateSession.isOpen()) {
try {
openHibernateSession();
resultList = query.list();
} catch (Exception e) {
e.printStackTrace();
} finally {
closeHibernateSession();
}
} else {
resultList = query.list();
}
return resultList;
}
public List<T> find(final String queryString, final Object[] values) throws HibernateException {
if (hibernateSession == null || !hibernateSession.isOpen()) {
openHibernateSession();
}
Query queryObject = hibernateSession.createQuery(queryString);
return find(queryObject, values);
}
private void openHibernateSession() throws HibernateException {
try {
hibernateSession = HibernateUtil.getSessionFactory().openSession();
hibernateSession.setFlushMode(FlushMode.MANUAL);
hibernateTransaction = hibernateSession.getTransaction();
hibernateTransaction.begin();
} catch (Exception e) {
e.printStackTrace();
}
}
private void closeHibernateSession() {
try {
if (hibernateTransaction != null) {
hibernateSession.flush();
if (!hibernateTransaction.wasCommitted()) {
hibernateTransaction.commit();
}
}
if (hibernateSession != null) {
if (hibernateSession.isOpen()) {
hibernateSession.close();
}
}
} catch (Exception e) {
e.printStackTrace();
}
}
<property name="hibernate.dialect">org.hibernate.dialect.MySQLDialect</property>
<property name="hibernate.connection.driver_class">com.mysql.jdbc.Driver</property>
<property name="hibernate.connection.url">jdbc:mysql://localhost:3306/projectdb</property>
<property name="hibernate.connection.username">root</property>
<property name="hibernate.connection.password">root</property>
<property name="show_sql">true</property>
<property name="format_sql">true</property>
<property name="hbm2ddl.auto">update</property>
<property name="hibernate.c3p0.min_size">5</property>
<property name="hibernate.c3p0.max_size">20</property>
<property name="hibernate.c3p0.timeout">1800</property>
<property name="hibernate.c3p0.max_statements">50</property>
<property name="hibernate.c3p0.idle_test_period">50</property>
<property name="hibernate.connection.provider_class">org.hibernate.connection.C3P0ConnectionProvider</property>
<property name="hibernate.c3p0.acquire_increment">2</property>
<property name="hibernate.c3p0.usesTraditionalReflectiveProxies">true</property>
<property name="connection.pool_size">20</property>
<property name="current_session_context_class">thread</property>
<property name="cache.provider_class">org.hibernate.cache.NoCacheProvider</property>
<property name="hibernate.connection.isolation">4</property>

you can probably fix your most immediate problem pretty easily. have your closeHibernateSession() reliably set hibernateSession to null. that will force your query functions to create new Sessions rather than try to reuse an old, closed session.
but don't just do this. you have much bigger problems.
you need to be much cleaner in how you organize your hibernate sessions. from the code above, it's clear that you have a member variable called hibernateSession that may be shared by across multiple callers or by multiple queries, that is sometimes null and sometimes not, which various functions lazily initialize but may not close until some time down the line. that's no good at all.
in the simplest arrangement, don't hold the session as a member variable. hold it as a local variable. let openHibernateSession() return a session that will be closed with perfect certainty in a finally block. if for some reason you need sessions to outlive what can be encapsulated by a single function call (which might of course call many other functions, each of which might take the session as a parameter), you will have to be very, very careful. sessions that are open must reliably be close()ed, and you must understand their lifecycles. your current approach, which uses a session if one is already open, but creates a temporary session otherwise, is no good. if you insist on maintaining a Session member variable, you must know with perfect certainty, whenever the member variable will be initialized, exactly how and when it will be close()ed and then reset to null or dereferenced for garbage collection.
you also should understand your transactions, none of this "if i have an uncommitted transaction before close(), commit it."
also, please consider migrating to c3p0-0.9.2.1

When you are trying to save or update a transient instance make sure the entities associated with instance are persistent. This is a common error of users trying to persist a new instance of the entity with the detached references. Get the entity from the persistence context before you set its reference to the transient or detached instance that you are going to persist.

Related

c3p0 DataSource monitor deadlock - all threads hang - how to fix

We have a Spring based application and recently we went into production. We are using Spring #Controller that ultimately hit DAOs that use JDBCTemplate. It is using c3p0's ComboPooledDataSource
On an increased load (something like 150 concurrent users), the application hangs for all users - the DataSource gets locked by something - on a thread dump, there are like 200 threads that say - obviously the DataSource is deadlocked.
"http-bio-8080-exec-440" - Thread t#878
java.lang.Thread.State: WAITING
at java.lang.Object.wait(Native Method)
- waiting on <146d984e> (a com.mchange.v2.resourcepool.BasicResourcePool)
at com.mchange.v2.resourcepool.BasicResourcePool.awaitAvailable(BasicResourcePool.java:1418)
at com.mchange.v2.resourcepool.BasicResourcePool.prelimCheckoutResource(BasicResourcePool.java:606)
at com.mchange.v2.resourcepool.BasicResourcePool.checkoutResource(BasicResourcePool.java:526)
at com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool.checkoutAndMarkConnectionInUse(C3P0PooledConnectionPool.java:756)
at com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool.checkoutPooledConnection(C3P0PooledConnectionPool.java:683)
at com.mchange.v2.c3p0.impl.AbstractPoolBackedDataSource.getConnection(AbstractPoolBackedDataSource.java:140)
at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:111)
at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:77)
at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:573)
at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637)
at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666)
at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:674)
at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:718)
After that point the application becomes unusable unless restarted. When this happened DBA team didn't observe any load on the database.
At that time c3p0 was configured like that:
app_en.driverClass=com.mysql.jdbc.Driver
app_en.user=tapp_en
app_en.password=tapp_en
app_en.jdbcUrl=jdbc:mysql://10.10.0.102:3306/tapp_en?useUnicode=true&characterEncoding=utf-8&autoReconnect=true
app_en.acquireIncrement=5
app_en.maxIdleTime=3600
app_en.maxIdleTimeExcessConnections=300
app_en.unreturnedConnectionTimeout=3600
app_en.numHelperThreads=6
app_en.minPoolSize=20
app_en.maxPoolSize=100
app_en.idleConnectionTestPeriod=120
app_en.testConnectionOnCheckin=true
After that, I changed c3p0's configuration as follows - and enabled DEBUG logging for com.mchange.v2.c3p0 package:
app_en.driverClass=com.mysql.jdbc.Driver
app_en.user=tapp_en
app_en.password=tapp_en
app_en.jdbcUrl=jdbc:mysql://10.10.0.102:3306/tapp_en? useUnicode=true&characterEncoding=utf-8&autoReconnect=true
app_en.acquireIncrement=5
app_en.maxIdleTime=180
app_en.maxIdleTimeExcessConnections=60
app_en.unreturnedConnectionTimeout=30
app_en.checkoutTimeout=10000
app_en.numHelperThreads=12
app_en.debugUnreturnedConnectionStackTraces=true
app_en.initialPoolSize=10
app_en.maxPoolSize=100
app_en.idleConnectionTestPeriod=120
app_en.preferredTestQuery="select 1 from tbl_users"
With this configuration in place, I again ran load tests and the application still hanged... although the threads recover after they are unable to obtain connection to the database. Even though, the game hanged for too many users even though the threads recovered unlike the previous configuration - so they had to restart their clients.
Although all logging was enabled, the c3p0 logs don't log any deadlock messages. The error messages I see are just that:
[06/24/2015 12:20:54] [C3P0PooledConnectionPoolManager[identityToken->1oed6dl9a9ak8qsgqfvdu|4d6145af]-HelperThread-#10] DEBUG NewPooledConnection - com.mchange.v2.c3p0.impl.NewPooledConnection#7f0bc55a closed by a client.
java.lang.Exception: DEBUG -- CLOSE BY CLIENT STACK TRACE
at com.mchange.v2.c3p0.impl.NewPooledConnection.close(NewPooledConnection.java:659)
at com.mchange.v2.c3p0.impl.NewPooledConnection.closeMaybeCheckedOut(NewPooledConnection.java:255)
at com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool$1PooledConnectionResourcePoolManager.destroyResource(C3P0PooledConnectionPool.java:621)
at com.mchange.v2.resourcepool.BasicResourcePool$1DestroyResourceTask.run(BasicResourcePool.java:1024)
at com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread.run(ThreadPoolAsynchronousRunner.java:696)
There aren't any transactions in the application made, nor are we using any TransactionManager or TransactionTemplate. I wonder if this may be some kind of bug in the frameworks used, or misconfiguration. These are the relevant frameworks used:
c3p0-0.9.5-pre8
mysql-connector-java-5.1.24
spring-core-3.2.1.RELEASE
spring-web-3.2.1.RELEASE
mchange-commons-java-0.2.7
We really appreciate any help because this is blocking our efforts to release our product.
P.S. EDIT: Here is the configuration of the DataSource:
<bean id="app_en_DataSource" class="com.mchange.v2.c3p0.ComboPooledDataSource"
destroy-method="close">
<property name="driverClass" value="${app_en.driverClass}" />
<property name="jdbcUrl" value="${app_en.jdbcUrl}" />
<property name="user" value="${app_en.user}" />
<property name="password" value="${app_en.password}" />
<property name="acquireIncrement" value="${app_en.acquireIncrement}"></property>
<property name="maxIdleTime" value="${app_en.maxIdleTime}"></property>
<property name="maxIdleTimeExcessConnections" value="${app_en.maxIdleTimeExcessConnections}"></property>
<property name="unreturnedConnectionTimeout" value="${app_en.unreturnedConnectionTimeout}"></property>
<property name="checkoutTimeout" value="${app_en.checkoutTimeout}"></property>
<property name="numHelperThreads" value="${app_en.numHelperThreads}"></property>
<property name="debugUnreturnedConnectionStackTraces" value="${app_en.debugUnreturnedConnectionStackTraces}"></property>
<property name="initialPoolSize" value="${app_en.initialPoolSize}"></property>
<property name="maxPoolSize" value="${app_en.maxPoolSize}"></property>
<property name="idleConnectionTestPeriod" value="${app_en.idleConnectionTestPeriod}"></property>
<property name="preferredTestQuery" value="${app_en.preferredTestQuery}"></property>
</bean>
And here is some code inside the application that isn't using the jdbcTemplate directly. There is nothing else that does that, everything else is jdbcTemplate.update, jdbcTemplate.query:
Connection conn = null;
ResultSet getItemsRS = null;
try {
JdbcTemplate jdbcTemplate = getJdbcTemplate(database);
conn = jdbcTemplate.getDataSource().getConnection();
UserItems items;
if (!action.areItemsNew()) {
conn.setAutoCommit(false);
conn.setTransactionIsolation(Connection.TRANSACTION_SERIALIZABLE);
PreparedStatement getItemsPS = conn.prepareStatement("select * from tbl_items where ownerId = ? for update",
ResultSet.TYPE_FORWARD_ONLY,
ResultSet.CONCUR_UPDATABLE);
getItemsPS.setLong(1, userId);
getItemsRS = getItemsPS.executeQuery();
getItemsRS.next();
items = new UserItemsRowMapper().mapRow(getItemsRS, getItemsRS.getRow());
} else {
items = new UserItems();
}
action.doUserItemsAction(items);
ByteArrayOutputStream baos = new ByteArrayOutputStream();
ObjectOutputStream oos = new ObjectOutputStream(baos);
oos.writeObject(items.getItemContainers());
oos.close();
byte[] data = baos.toByteArray();
Blob blob = conn.createBlob();
blob.setBytes(1, data);
if (!action.areItemsNew()) {
getItemsRS.updateBlob("data", blob);
getItemsRS.updateRow();
} else {
jdbcTemplate.update("insert into tbl_items(ownerId,data) values(?,?)", userId, data);
}
} catch (Exception e) {
logger.error(e);
throw new RuntimeException(e);
} finally {
if (!action.areItemsNew()) {
try {
conn.commit();
conn.close();
} catch (SQLException e) {
logger.error(e);
throw new RuntimeException(e);
}
}
}
The reason for this code is that I would like to block reading/writing to the user's items before they are updated by this operation action.doUserItemsAction(items) as written above.
The code you have is potentially dangerous and has a connection leak, when checking out a connection yourself you should always close it, there might be a case it fails to close the connection.
Instead I strongly suggest using Spring to manage your transactions and connections.
First annotate your method with #Transactional(isolation=SERIALIZABLE). Next add a DataSourceTransactionManager and <tx:annotation-driven /> to your configuration. After these changes rewrite the data access code you have.
JdbcTemplate jdbcTemplate = getJdbcTemplate(database);
final UserItems items;
if (!action.areItemsNew()) {
items = jdbcTemplate.queryForObject("select * from tbl_items where ownerId = ? for update", userId, new UserItemsRowMapper());
} else {
items = new UserItems();
}
action.doUserItemsAction(items);
String query = !action.areItemsNew() ? "update tbl_items set data=? where ownerId=?" : "insert into tbl_items(data,ownerId) values(?,?)";
byte[] data = SerializationUtils.serialize(items.getItemContainers());
jdbcTemplate.update(query, new SqlLobValue(data), userId);
Something like that (together with the aformentioned modification should work). (This was more or less from the top of my head, so it might need some tweaking). The use of proper transaction management ensures that everything is reusing the same single connection instead of multiple connections, it also ensures that a connection is returned to the pool when finished or when something goes wrong.
I would still suggest a different datasource as C3P0 is pretty old.
So, a few things.
1) The "error" messages that you see are not errors, when c3p0 logs an Exception whose message begins with DEBUG, that means you are logging at DEBUG levels and c3p0 has generated the Exception just to capture the stack trace. (c3p0 is an old library; Thread.getStackTrace() didn't exist back in the day, creating an Exception was a convenient way to capture and dump the stack.) You are just logging the expected destruction of pooled Connections due to expiration or test failures. In general, c3p0 expects to log at INFO, it will be very verbose at DEBUG levels.
2) You are not deadlocking c3p0's Thread pool. If you were, you'd see APPARENT DEADLOCK messages and then recoveries. You are experiencing a condition of pool exhaustion: clients are awaiting Connections, but the pool is at maxPoolSize and unable to acquire them.
3) The usual cause of pool exhaustion is a Connection leak: Somewhere in your application's code path, under some (probably Exceptional) circumstances, Connections are acquired and then never close()ed. You need to be very careful to ensure that Connections are reliably close()ed in finally blocks in ways that cannot be skipped due to prior failures within the finally block. In Java 7+, use try-with-resources. In older versions, use the reliable resource cleanup idiom.
4) To test whether a Connection leak is the issue, set the c3p0 config params unreturnedConnectionTimeout and debugUnreturnedConnectionStackTraces. unreturnedConnectionTimeout will work around the problem, but yuck. More importantly, debugUnreturnedConnectionStackTraces will show you where the problem is so that you can fix it, logging the stack trace that opened the unclosed Exception at INFO. (You must set unreturnedConnectionTimeout for debugUnreturnedConnectionStackTraces to have any effect; the stack trace is logged when a Connection times out as abandoned.)
5) Although 0.9.5-pre8 is probably ok, the current production version of c3p0 is c3p0-0.9.5.1 (which depends upon mchange-commons-java v.0.2.10). You might think about using that. I don't think it has anything at all to do with your issue, but still.
I hope this helps!
Update: Since you've now posted code that shows the likely Connection leak, here's a suggestion for how to fix it. Replace your finally block with:
} finally {
if ( conn != null ) {
try { if (!action.areItemsNew()) conn.commit(); }
catch (SQLException e) {
logger.error(e);
throw new RuntimeException(e);
} finally {
conn.close()
}
}
}
Update 2: The redone finally block above will solve the Connection leak, but if I were you I'd also change the logic of this code regarding commit(). Here's a suggested revision:
Connection conn = null;
ResultSet getItemsRS = null;
try {
JdbcTemplate jdbcTemplate = getJdbcTemplate(database);
conn = jdbcTemplate.getDataSource().getConnection();
UserItems items;
if (!action.areItemsNew()) {
conn.setAutoCommit(false);
conn.setTransactionIsolation(Connection.TRANSACTION_SERIALIZABLE);
PreparedStatement getItemsPS = conn.prepareStatement("select * from tbl_items where ownerId = ? for update",
ResultSet.TYPE_FORWARD_ONLY,
ResultSet.CONCUR_UPDATABLE);
getItemsPS.setLong(1, userId);
getItemsRS = getItemsPS.executeQuery();
getItemsRS.next();
items = new UserItemsRowMapper().mapRow(getItemsRS, getItemsRS.getRow());
} else {
items = new UserItems();
}
action.doUserItemsAction(items);
ByteArrayOutputStream baos = new ByteArrayOutputStream();
ObjectOutputStream oos = new ObjectOutputStream(baos);
oos.writeObject(items.getItemContainers());
oos.close();
byte[] data = baos.toByteArray();
Blob blob = conn.createBlob();
blob.setBytes(1, data);
if (!action.areItemsNew()) {
getItemsRS.updateBlob("data", blob);
getItemsRS.updateRow();
conn.commit();
} else {
jdbcTemplate.update("insert into tbl_items(ownerId,data) values(?,?)", userId, data);
}
} catch (Exception e) {
logger.error(e);
throw new RuntimeException(e);
} finally {
try { if ( conn != null ) conn.close(); }
catch ( Exception e )
{ logger.error(e); }
}
Now commit() will get called only if (!action.areItemsNew()) AND all expected operations have succeeded. Before commit() would get called even if something went wrong. The resource cleanup code is much simpler and cleaner too. Note that in the suggested version, if there's an Exception on close() it's logged, but it is not wrapped and rethrown as a RuntimeException. Usually if there's an Exception on close(), there will have been a more informative Exception prior and that's the one you want to see. If the only place an Exception occurs is on close(), it means that all database operations have succeeded so your application can proceed correctly despite the fault. (If there are lots of Exceptions on close(), eventually you'd exhaust the Connection pool, but in practice that would happen only if there's something badly wrong with your database or network.)

Hibernate using C3P0 Works fine on Oracle but does not close connection with MySQL 5.6

I am using Servlet 2.4, Hibernate 4.2.4 Final, c3p0 0.9.2.1, Tomcat 7.0.42, MySQL 5.6 & JSP.
I had completed development using Oracle 11gR2 DB but at a later point was asked to switch to MySQL as the Database.
I have a rather unusual problem at hand.
The problem is Multiple MySQL Process/Connections being created for every single DB request, which are neither closed nor returned to the pool despite issuing the SessionFactoryUtil.close(); which was not the case with Oracle DB.
I tested the exact same code on these two different Databases, i.e after executing a Function/Request (ex: Login)
The application when tested with Oracle (11gR2) the DB created a single connection and used it for all requests henceforth.
SELECT * FROM V$RESOURCE_LIMIT
Gives me the following Output
RESOURCE_NAME: processes
CURRENT_UTILIZATION: 32
MAX_UTILIZATION: 36
INITIAL_ALLOCATION: 300
LIMIT_VALUE: 300
No matter how many users log in the Connection pool maintains it gracefully.
Now on the other hand when the same application was run on MySQL:
I did a SHOW PROCESSLIST; on MySQL which shows two processes being created for every request; c3p0 successfully terminates one connection but the other connection remains till the DB crashes because it exceeded the max connections available.
My SessionFactoryUtil is quite simple and straightforward and is as follows:
public class SessionFactoryUtil {
private static SessionFactory sessionFactory;
public static SessionFactory getSessionFactory() {
return sessionFactory = new Configuration().configure()
.buildSessionFactory();//deprecated method not changed due to official reason
}
public Session getCurrentSession() {
return sessionFactory.getCurrentSession();
}
public static void close() {
if (sessionFactory != null) {
sessionFactory.close();
}
sessionFactory = null;
}
My DAO Method is as follows
public User getUserByName(String userName) throws FetchException {
User user = null;
Session session = SessionFactoryUtil.getSessionFactory().getCurrentSession();
try {
session.beginTransaction();
user = (User) session.createQuery("from User where userName = '" + userName + "'").uniqueResult();
} catch (Exception e) {
logger.info("UserDaoImpl -> getUserByName() : Error : " +e);
e.printStackTrace();
} finally {
SessionFactoryUtil.close();
}
return user;
The stack trace where c3p0 destroys a connection is as follows:
20:45:43,692 INFO com.mchange.v2.resourcepool.BasicResourcePool:1493 - A checked-out resource is overdue, and will be destroyed: com.mchange.v2.c3p0.impl.NewPooledConnection#61f31fff
20:45:43,692 INFO com.mchange.v2.resourcepool.BasicResourcePool:1496 - Logging the stack trace by which the overdue resource was checked-out.
java.lang.Exception: DEBUG STACK TRACE: Overdue resource check-out stack trace.
at com.mchange.v2.resourcepool.BasicResourcePool.checkoutResource(BasicResourcePool.java:555)
at com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool.checkoutAndMarkConnectionInUse(C3P0PooledConnectionPool.java:755)
at com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool.checkoutPooledConnection(C3P0PooledConnectionPool.java:682)
at com.mchange.v2.c3p0.impl.AbstractPoolBackedDataSource.getConnection(AbstractPoolBackedDataSource.java:140)
at org.hibernate.service.jdbc.connections.internal.C3P0ConnectionProvider.getConnection(C3P0ConnectionProvider.java:84)
at org.hibernate.internal.AbstractSessionImpl$NonContextualJdbcConnectionAccess.obtainConnection(AbstractSessionImpl.java:292)
at org.hibernate.engine.jdbc.internal.LogicalConnectionImpl.obtainConnection(LogicalConnectionImpl.java:214)
at org.hibernate.engine.jdbc.internal.LogicalConnectionImpl.getConnection(LogicalConnectionImpl.java:157)
at org.hibernate.engine.transaction.internal.jdbc.JdbcTransaction.doBegin(JdbcTransaction.java:67)
at org.hibernate.engine.transaction.spi.AbstractTransactionImpl.begin(AbstractTransactionImpl.java:160)
at org.hibernate.internal.SessionImpl.beginTransaction(SessionImpl.java:1426)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.hibernate.context.internal.ThreadLocalSessionContext$TransactionProtectionWrapper.invoke(ThreadLocalSessionContext.java:352)
at com.sun.proxy.$Proxy7.beginTransaction(Unknown Source)
at com.demo.access.impl.ConfDaoImp.showAllEvents(ConfDaoImp.java:939)
at com.demo.business.impl.ConfServiceImpl.showAllEvents(ConfServiceImpl.java:404)
at com.demo.controller.UserController.getControls(UserController.java:112)
at com.demo.controller.UserController.validateUser(UserController.java:93)
at com.demo.controller.UserController.process(UserController.java:42)
at com.demo.controller.ApplicationServlet.process(ApplicationServlet.java:75)
at com.demo.controller.ApplicationServlet.doPost(ApplicationServlet.java:53)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:641)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:722)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:304)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at com.demo.controller.LoginFilter.doFilter(LoginFilter.java:37)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:224)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:185)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:472)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:151)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:100)
at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:929)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:405)
at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:269)
at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:515)
at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:302)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
I have read almost all the question related to this particular scenario but none seems to work, or the thread was abandoned half way, or I am missing out something; could some one please help me get through with this.
This piece of your code did the trick for me:
public static void close() {
if(sessionFactory instanceof SessionFactoryImpl) {
SessionFactoryImpl sf = (SessionFactoryImpl)sessionFactory;
ConnectionProvider conn = sf.getConnectionProvider();
if(conn instanceof C3P0ConnectionProvider) {
((C3P0ConnectionProvider)conn).close();
}
}
sessionFactory.close(); }
Until then Tomcat had (correctly) complained about memory leaks at each hot deployment. Thanks!
a few ideas:
1) you are never closing the Session you create (implicitly by asking for a "current session"). that's a straightforward reason why you might have an unreturned Connection that eventually times out.
2) you are treating your SessionFactory like a Session, building up then tearing down the whole thing (which includes a Connection pool) just to get and use one Connection. not so good. your SessionFamily should have a long lifecycle, your sessions should be for one-time, short-term use.
Its been a while since I found out the answer to my strange problem and I thought that sharing it would help most.
To start with, A couple of things i did wrong was...
Firstly, I migrated from hibernate 3.6 to 4.2 and on doing so I was still using the deprecated buildSessionFactory() method.
Secondly, I was using SessionFactoryUtil.close() after the end of each query statement in my DAO, which defeats the purpose of using Connection Pooling.
Finally, The strange issue where Oracle seems to close the connection successfully after execution of a statement whereas MySql was unable to close the same still remains a mystery.
This, I suspect happened because I was asking SessionFactoryUtil to close a connection which was originally opened by C3P0ConnectionProvider (which I believe was in-turn causing the connection leaks).
After a lot of research and looking around I re-wrote the SessionFactoryUtil as follows...
public class SessionFactoryUtil {
private static SessionFactory sessionFactory;
private static ServiceRegistry serviceRegistry;
public static SessionFactory getSessionFactory() {
Configuration configuration = new Configuration();
configuration.configure();
serviceRegistry = new ServiceRegistryBuilder().applySettings(configuration.getProperties()).buildServiceRegistry();
sessionFactory = configuration.buildSessionFactory(serviceRegistry);
return sessionFactory;
}
public static Session getCurrentSession() {
if(sessionFactory == null){
getSessionFactory();
}
return sessionFactory.getCurrentSession();
}
public static void close() {
if(sessionFactory instanceof SessionFactoryImpl) {
SessionFactoryImpl sf = (SessionFactoryImpl)sessionFactory;
ConnectionProvider conn = sf.getConnectionProvider();
if(conn instanceof C3P0ConnectionProvider) {
((C3P0ConnectionProvider)conn).close();
}
}
sessionFactory.close();
}
Note that all my connections are opened by C3P0ConnectionProvider so it is only logical to close it using C3P0ConnectionProvider itself.
The following is my hibernate.cfg.xml along with c3p0 settings.
<!-- Database connection settings -->
<property name="hibernate.connection.driver_class">com.mysql.jdbc.Driver</property>
<property name="hibernate.connection.url">jdbc:mysql://localhost:3306/application</property>
<property name="hibernate.connection.username">root</property>
<property name="hibernate.connection.password">root</property>
<property name="show_sql">true</property>
<property name="format_sql">false</property>
<!-- SQL dialect -->
<property name="dialect">org.hibernate.dialect.MySQLDialect</property>
<!-- Enable Hibernate's automatic session context management -->
<property name="hibernate.current_session_context_class">thread</property>
<property name="hibernate.connection.release_mode">auto</property>
<!-- Create or update the database schema on startup -->
<property name="hibernate.hbm2ddl.auto">none</property>
<!-- DEPRECATED -->
<!-- <property name="connection.provider_class">org.hibernate.connection.C3P0ConnectionProvider</property> -->
<!-- C3p0 connection pooling configuration -->
<property name="hibernate.connection.provider_class">org.hibernate.service.jdbc.connections.internal.C3P0ConnectionProvider</property>
<property name="c3p0.unreturnedConnectionTimeout">600</property>
<property name="c3p0.debugUnreturnedConnectionStackTraces">false</property>
<!-- configuration pool via c3p0 -->
<property name="c3p0.acquire_increment">1</property>
<property name="c3p0.idle_test_period">600</property>
<property name="c3p0.max_size">75</property>
<property name="c3p0.max_statements">5</property>
<property name="c3p0.min_size">5</property>
<property name="c3p0.timeout">600</property>
<property name="c3p0.checkoutTimeout">6000000</property>
<property name="c3p0.testConnectionOnCheckout">false</property>
<property name="c3p0.testConnectionOnCheckin">true</property>
<!-- Mapping -->
</session-factory>
This is again one of the methods from my DAO Class...
public User getUserByName(String userName) throws FetchException {
User user = null;
Session session = SessionFactoryUtil.getCurrentSession();
try {
session.beginTransaction();
user = (User) session.createQuery("from User where userName = '" + userName + "'").uniqueResult();
session.getTransaction().commit();
} catch (Exception e) {
logger.info("UserDaoImpl -> getUserByName() : Error : " +e);
e.printStackTrace();
} finally {
}
return user;
Note that in my DAO at the finally block i don't have to close my connection any more I let c3p0 handle the connection pool.
And voila...!! The Application runs!!! Had more than 2000 transaction hits on a single day in a span of 2 hours.
I hope this helps noob hibernate users like me.

Binary File To SQL Database Apache Camel

I need some guidance around which approach to use to load binary files from a folder into a MySQL Database using Camel. Basically I want to store voice logs from our PBX system into a database. The directory with the voice logs will be a remote directory
I have designed a prototype but I am not sure if this is really efficient, it works but I am not happy with the design. Let me explain what I am doing. Camel route as follows:
<camelContext xmlns="http://camel.apache.org/schema/spring">
<package>com.hia.camelone</package>
<route>
<from uri="file://c:/CTest/Inbox?noop=true&recursive=true&delay=3000"/>
<to uri="bean://fileToSQL"/>
<to uri="jdbc://timlogdb"/>
</route>
</camelContext>
<bean id="timlogdb" class="org.springframework.jdbc.datasource.DriverManagerDataSource">
<property name="driverClassName" value=" com.mysql.jdbc.Driver"/>
<property name="url" value="jdbc:mysql://127.0.0.1:3306/TimLog" />
<property name="username" value="root" />
<property name="password" value="blahblah" />
</bean>
<bean id="fileToSQL" class="com.hia.camelone.fileToSQL"/>
And the code to fileToSQL bean is:
public class fileToSQL {
public String toString(#Headers Map<String,Object> header, #Body Object body){
StringBuilder sb = new StringBuilder();
String filename =(String)header.get("CamelFileNameOnly");
String escapedFileName = StringEscapeUtils.escapeJava(filename).replace("\'", "");
String filePath = StringEscapeUtils.escapeJava((String)header.get("CamelFilePath"));
sb.append("insert into FileLog ");
sb.append("(FileName,FileData) values (");
sb.append("'").append(escapedFileName).append("',").append("LOAD_FILE(\"").append(filePath).append("\")");
sb.append(")");
System.out.println(sb.toString());
System.out.println(body);
System.out.println(header.toString());
return sb.toString();
}
}
Ok short explanation I get the file component to consume the files then I build a SQL string using the MySQL LOAD_FILE() function to load the file.
My thoughts around this:
The LOAD_FILE function only works on the local machine and thus this route will only with the files being on the local machine. I could use a file producer to copy the files from some remote directory to a local directory and then use the route. My route would be something like this then:
<route>
<from uri="file://c:/CTest/Inbox?noop=true&recursive=true&delay=3000"/>
<to uri="file://c:/outbox"/>
<to uri="bean://fileToSQL"/>
<to uri="jdbc://timlogdb"/>
</route>
However since I have access to the files content in the message from the files consumer I should be able to theoretically be able to access the body/content of the string and build a SQL command that does NOT use the LOAD_FILE() function.
The only way I know how to build such a string is by using the prepared statement of JDBC. This would be first prize if I could somehow build a insert statement with the content from the file consumer.
Can I create a prepared statement in my fileToSQL bean and pass it to my jdbc component?
Or how do I build a INSERT statement without the LOAD_FILE() function?
Since I have to use the LOAD_FILE() function I would now have to cater for both unix and windows filepaths. While this should not be difficult I just dont like the idea of putting OS specific code into my applications(feels like a work around).
Anybody here ever uploaded binary files to a MySQL database using Camel who can give me some guidance on the points above. While I could work around the problems I just want to make sure I dont miss a obvious way of doing things.
I had a look around here and only found people working with mostly text files. Guys please don't even go down the route of me storing the file on the files system and linking it to the database. We have some very specific disaster recovery requirements and legal requirements that enforce the need for me to store it in a database.
Right so I managed to find a way and it was not that difficult. What I essentially did was get rid of the JDBC Camel Component in the route. I then injected the data source bean into my fileToSQL bean. I then used a simple prepared statement to insert the file and its name into MySQL.
As always code is much more explicit than my english.
<camelContext xmlns="http://camel.apache.org/schema/spring">
<package>com.hia.camelone</package>
<route>
<from uri="file://c:/CTest/Inbox?noop=true&recursive=true&delay=3000"/>
<to uri="bean://fileToSQL"/>
<!--<to uri="jdbc://timlogdb"/>-->
</route>
</camelContext>
<bean id="timlogdb" class="org.springframework.jdbc.datasource.DriverManagerDataSource">
<property name="driverClassName" value=" com.mysql.jdbc.Driver"/>
<property name="url" value="jdbc:mysql://127.0.0.1:3306/TimLog" />
<property name="username" value="root" />
<property name="password" value="lalala" />
</bean>
<bean id="fileToSQL" class="com.hia.camelone.fileToSQL">
<property name="dataSource" ref="timlogdb"/>
</bean>
As you can see I inject my timlogdb bean into my fileToSQL bean. Spring ROCKS!
So here is my fileToSQL bean.
public class fileToSQL {
private DriverManagerDataSource dataSource;
private static final String SQL_INSERT="insert into FileLog(FileName,FileData)values(?,?)";
#Handler
public void toString(#Headers Map<String,Object> header,Exchange exchange){
Connection conn = null;
PreparedStatement stmt=null;
String filename =StringEscapeUtils.escapeJava(((String)header.get("CamelFileNameOnly")).replace("\'", ""));
try {
conn= dataSource.getConnection();
stmt =conn.prepareStatement(SQL_INSERT);
stmt.setString(1, filename);
byte[] filedata = exchange.getIn().getBody(byte[].class);
stmt.setBytes(2,filedata );
int s = stmt.executeUpdate();
}
catch (Exception e)
{
System.out.println(e.getMessage());
}
finally{
try
{
if (stmt!=null)
{
stmt.close();
}
if (conn!=null)
{
conn.close();
}
}
catch(SQLException e)
{
System.out.println(e.getMessage());
}
}
}
/**
* #param dataSource the dataSource to set
*/
public void setDataSource(DriverManagerDataSource dataSource) {
this.dataSource = dataSource;
}
}
The guys from Camel did a great job. Camel is truly flexible especially when you combine it with Spring.
What a ride!

Hibernate list using cache? -Not updating entity attribute

I have an application that uses hibernate.
I did the following:
Used List to list some entities on database
Logged in my Mysql Database manualy and updated a field in some
entities
Used List again in hibernate doing an identical query as 1
The entity that hibernate listed was not updated.
If I close and open the application. it then shows the entity updated correctly.
Is hibernate using some kind of cache by default?
<property name="hibernate.dialect">org.hibernate.dialect.MySQLDialect</property>
<property name="hibernate.connection.driver_class">com.mysql.jdbc.Driver</property>
<property name="hibernate.connection.url">jdbc:mysql://127.0.0.1:3306/XXX</property>
<property name="hibernate.connection.username">XXXXXXXXXX</property>
<property name="hibernate.connection.password">XXXXXXXXXX</property>
<property name="show_sql">true</property>
Code that lists the entity:
Session s = HibernateUtil.getSession();
Criteria c = s.createCriteria(Solicitacao.class, "s");
//Add some Restrictions here
List<Solicitacao> ls = c.list();
s.close();
My Session factory:
public class HibernateUtil {
private static SessionFactory sessionFactory = null;
static {
// Configurações iniciais de caminho dos diretórios e arquivos
URL url = HibernateUtil.class.getProtectionDomain().getCodeSource().getLocation();
File myFile = null;
try {
myFile = new File(url.toURI());
} catch (Exception ex) {
}
File dir = myFile.getParentFile();
File xml = new File(dir, "hibernate.cfg.xml");
/*
* sessionFactory = new AnnotationConfiguration() .configure("br/com/netradiobrasil/pcp/" +
* "hibernate/hibernate.cfg.xml") .buildSessionFactory();
*/
sessionFactory = new AnnotationConfiguration().configure(xml).buildSessionFactory();
}
public static Session getSession() {
return sessionFactory.openSession();
}
}
I tryed to add those lines in my hibernate.cfg.xml
<property name="hibernate.cache.use_query_cache">false</property>
<property name="hibernate.cache.use_second_level_cache">false</property>
<property name="hibernate.cache.provider_class">org.hibernate.cache.NoCacheProvider</property>
Also tryed to use: session.setCacheMode(CacheMode.IGNORE)
but still didnt solve my problem
Let me guess
After executing this
Session s = HibernateUtil.getSession();
Criteria c = s.createCriteria(Solicitacao.class, "s");
//Add some Restrictions here
List<Solicitacao> ls = c.list();
You changed entries in database manually and then reran the query ? If yes then can you close the session and then rerun your code ?
Adding those lines in my hibernate.cfg.xml - that enables c3p0 fixed my problem
<property name="hibernate.c3p0.min_size">5</property>
<property name="hibernate.c3p0.max_size">40</property>
<property name="hibernate.c3p0.timeout">300</property>
<property name="hibernate.c3p0.max_statements">50</property>
<property name="hibernate.c3p0.idle_test_period">100</property>

Handle with exception on auto proxy / proxy factory object

I start learning Spring.NET framework and I am very confusing with behavior of proxy, auto-proxy and exception handling.
for example I defined simple business object and from this object I will throw custom exception.
namespace Aspect.Managers
{
public interface IDbCustomerManager
{
Customer GetCustomerById(long id);
}
public class DbCustomerManager:IDbCustomerManager
{
public Customer GetCustomerById(long id)
{
throw new DbException(string.Format("Problem load customer with Id: {0}",id));
}
}
}
Second I defined Advice for handling with exception.
public class LogExThrowsAdvice:IThrowsAdvice
{
public void AfterThrowing(MethodInfo method, Object[] args,
Object target, DbException exception)
{
Console.WriteLine(exception.Message);
}
}
And last I join togheter business object and advice with proxy.
In app.confing
Advice:
<object id="theLogExThrowsAdvice"
type="Aspect.LogExThrowsAdvice, Log4NetInSpringNet"/>
Auto-Proxy
<object id="theProxyCreator"
type="Spring.Aop.Framework.AutoProxy.TypeNameAutoProxyCreator, Spring.Aop">
<property name="TypeNames" value="Aspect.Managers.DbCustomerManager*"/>
<property name="InterceptorNames">
<list>
<value>theLogExThrowsAdvice</value>
</list>
</property>
</object>
And test it:
var springContext = ContextRegistry.GetContext();
var dbMgr = (IDbCustomerManager)springContext["theDbCustomerManager"];
dbMgr.GetCustomerById(1);
Exception is throwed, method AfterThrowing from LogExThrowsAdvice is not calling.
I try changed type of advice for type BeforeAdvice.
public class DbAccessAdvice:IMethodBeforeAdvice
{
#region Implementation of IMethodBeforeAdvice
public void Before(MethodInfo method, object[] args, object target)
{
Console.WriteLine("You try access to DB");
}
#endregion
}
and in app.config:
<object id="theDbAccessAdvice"
type="Aspect.DbAccessAdvice, Log4NetInSpringNet"/>
<object id="theProxyCreator"
type="Spring.Aop.Framework.AutoProxy.TypeNameAutoProxyCreator, Spring.Aop">
<property name="TypeNames" value="Aspect.Managers.DbCustomerManager*"/>
<property name="InterceptorNames">
<list>
<value>theDbAccessAdvice</value>
<value>theLogExThrowsAdvice</value>
</list>
</property>
</object>
BeforeAdvice is fire but ThrowsAdvice no. Why?
I tried change auto proxy for proxy object factory and tried proxying interfaces IDbCustomerManager.
<object id="theProxy"
type="Spring.Aop.Framework.ProxyFactoryObject, Spring.Aop">
<property name="ProxyInterfaces" value="Aspect.Managers.IDbCustomerManager"/>
<property name="Target">
<object type="Aspect.Managers.DbCustomerManager">
</object>
</property>
<property name="InterceptorNames">
<list>
<value>theDbAccessAdvice</value>
<value>theLogAdvice</value>
</list>
</property>
</object>
var springContext = ContextRegistry.GetContext();
var dbMgr = (IDbCustomerManager)springContext["theProxy"];
dbMgr.GetCustomerById(1);
Before advice is fired but throws advice are not? why? Only exception is throwed.
For me is magic how it really work.
I tried used Advisors instead advices:
<!--Advisor-->
<object id="theDbAccessAdvisor"
type="Spring.Aop.Support.RegularExpressionMethodPointcutAdvisor, Spring.Aop">
<property name="Pattern" value="Aspect*"/>
<property name="Advice" ref="theDbAccessAdvice"/>
</object>
<object id="theLogAdvisor"
type="Spring.Aop.Support.RegularExpressionMethodPointcutAdvisor, Spring.Aop">
<property name="Pattern" value="Aspect*"/>
<property name="Advice" ref="theLogAdvice"/>
</object>
But same result before advice is fired but throws advice not.
I tried use also ExceptionHandleAdvice aspect from Spring.NET same exception is throwed but advice not.
<object id="exAdvice"
type="Spring.Aspects.Exceptions.ExceptionHandlerAdvice, Spring.Aop">
<property name="ExceptionHandlers">
<list>
<value>on exception name DbException swallow</value>
</list>
</property>
</object>
This project is for me magic I upload all VS project here:
http://hotfile.com/dl/135485464/93558e0/Log4Net.7z.html
Here is stackTrace of exception:
at Aspect.Managers.DbCustomerManager.GetCustomerById(Int64 id) in
E:\C#
PROJECTS\STUDY\SPRING.NET\Study.Spring.Net\Aspects\Logging\Log4Net\Managers\DbCustomerManager.cs:line
20 at
_dynamic_Aspect.Managers.DbCustomerManager.GetCustomerById(Object , Object[] ) at Spring.Reflection.Dynamic.SafeMethod.Invoke(Object
target, Object[] arguments) at
Spring.Aop.Framework.DynamicMethodInvocation.InvokeJoinpoint() at
Spring.Aop.Framework.AbstractMethodInvocation.Proceed() at
Spring.Aspects.Exceptions.ExceptionHandlerAdvice.Invoke(IMethodInvocation
invocation)
Also if I try catch exception something like this:
try
{
var springContext = ContextRegistry.GetContext();
var dbMgr = (IDbCustomerManager)springContext["theDbCustomerManager"];
dbMgr.GetCustomerById(1);
}
catch (Exception ex)
{
Console.WriteLine("{0}\n{1}", ex.GetType(), ex.Message);
}
It is not possible..system show message is unhandled exception....
I tested your package. Everything works fine (I'm using v1.3.2).
The exception is caught by the AfterThrows advice (use a breakpoint) but not ignored.
Here is my config:
public void AfterThrowing(MethodInfo method, Object[] args,
Object target, DbException exception)
{
Console.WriteLine(exception.Message);
}
<!--DAO-->
<object id="theCustomerDao"
type="Aspect.Dao.CustomerDao"/>
<!--Business object-->
<object id="theDbCustomerManager"
type="Aspect.Managers.DbCustomerManager">
<!--<property name="CustomerDao" ref="theCustomerDao"/>-->
</object>
<!--Advices-->
<object id="theLogAdvice"
type="Aspect.LogExThrowsAdvice, Log4NetInSpringNet"/>
<!--Proxy creator-->
<object type="Spring.Aop.Framework.AutoProxy.TypeNameAutoProxyCreator, Spring.Aop">
<property name="TypeNames" value="Aspect.Managers.DbCustomerManager*"/>
<property name="InterceptorNames">
<list>
<value>theLogAdvice</value>
</list>
</property>
</object>
If you take your code and combine it with bbaia's config, then you're halfway there I think.
From your question and comments, I understand that you want to swallow the DbExceptions.
Note that generally speaking, you'd never want your logger swallow exceptions, but for the question's sake assume you do - but promise me you'll never do this in a production environment :-)
I suggest you slightly adjust your LogExThrowsAdvice to:
public class LogExThrowsAdvice : Spring.Aspects.Exceptions.ExceptionHandlerAdvice, IThrowsAdvice
{
public void AfterThrowing(MethodInfo method, Object[] args,
Object target, Exception exception)
{
Console.WriteLine(exception.Message);
}
}
Inheriting from ExceptionHandlerAdvice allows you to specify how to handle thrown exceptions, for instance swallow it.
Then take bbaia's config and add an event handler to it:
<object id="theLogAdvice"
type="Aspect.LogExThrowsAdvice, Log4NetInSpringNet">
<property name="exceptionHandlers">
<list>
<value>on exception name DbException swallow</value>
</list>
</property>
</object>
Now all DbExceptions are swallowed.
With respect to your question as to how this still throws in Visual Studio: could it be that your VS is set to "break on a thrown exception"? Go to Debug -> Exceptions and _un_check the Thrown checkbox for Common Language Runtime Exceptions.
Note that if you keep this checked, you can still continue when the exception is raised: the program will not crash, because the exception is handled.