Improve performance javax.persistence transactions to remote database - mysql

Question

When developing the application against my local database there were no problems with the speed of transactions, although the CPU usage was constantly at about 30 percents when performing several transactions per second, and when profiling most of the time was spent in javax methods handling the transactions with an average of 2.6 seconds per transaction. Therefore I'm using an ArrayList as a buffer and only sending the transaction when the size of the buffer exceeds 300 instances, which significantly lowered the CPU usage.
When I'm changing my persistence.xml to use a remote database instead (checked both RDS and a personal, off-site database) the minimum time for persisting/committing a batch of instances is about 20 seconds, which is too high since a transaction of 300 instances is required once every 5 seconds (on average).
I've tried to change the flushmode of the EntityManager to FlushModeType.COMMIT but it didn't change the performance noticeably. Increasing the size of the buffer before sending causes a stack overflow with the javax.persistence library for some (to me) unknown reason.
persistence.xml
<persistence-unit name="PU-data" transaction-type="RESOURCE_LOCAL">
<mapping-file>META-INF/orm.xml</mapping-file>
... // class, shared-cache-mode=none, validation-mode=none ...
<properties>
... // Authentication ...
<!-- Optimization attempts -->
<property name="eclipselink.jdbc.bind-parameters" value="true" />
<property name="eclipselink.jdbc.batch-writing" value="JDBC"/>
<property name="eclipselink.jdbc.batch-writing.size" value="300" />
<property name="eclipselink.jdbc.cache-statements" value="true" />
<property name="eclipselink.cache.shared.default" value="false" />
<property name="eclipselink.persistence-context.close-on-commit" value="true" />
<property name="eclipselink.persistence-context.flush-mode" value="commit" />
<property name="eclipselink.persistence-context.persist-on-commit" value="false" />
</properties>
</persistence-unit>
Facade handling the transactions
MouseFacade.bufferSemaphore.acquireUninterruptibly(1);
if (MouseFacade.buffer.size() >= 300) {
EntityManager entityManager = EMF.getEntityManager();
try {
entityManager.getTransaction().begin();
for (Mouse mouse : MouseFacade.buffer) {
entityManager.persist(mouse);
}
entityManager.getTransaction().commit();
} finally {
if (entityManager.getTransaction().isActive()) {
entityManager.getTransaction().rollback();
}
entityManager.close();
MouseFacade.buffer.clear();
}
}
MouseFacade.bufferSemaphore.release(1);
ORM mapping
<entity-mappings version="2.1" xmlns="http://www.eclipse.org/eclipselink/xsds/persistence/orm" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<entity class="se.my.package.Mouse">
<table-generator name="ORD_SEQ" allocation-size="300"/>
</entity>
</entity-mappings>
Update
I've gone through the suggestions found at this page, called How to improve JPA performance by 1,825% (http://java-persistence-performance.blogspot.se/2011/06/how-to-improve-jpa-performance-by-1825.html), but there is no difference what so ever which makes me wonder whether I'm missing a key point about batch writing and MySQL. I've rewritten the entities not to rely on relationships and minimized my read-operations to 1 for the entire application in order to just focus on the write problems.
When looking through the EclipseLink log it doesn't look like batch-writing is being used at all, but instead 2 log entires are written for every instance which seems about right (300 instances * 2 connections * 24 latency = 14.4 seconds).
[EL Fine]: sql: 2013-03-31 01:35:29.249--ClientSession(1213059092)--Connection(662811604)--Thread(Thread[pool-1-thread-1,5,main])--SELECT LAST_INSERT_ID()
[EL Fine]: sql: 2013-03-31 01:35:29.274--ClientSession(1213059092)--Connection(662811604)--Thread(Thread[pool-1-thread-1,5,main])--INSERT INTO mouse (event, posX, posY, created, uid) VALUES (?, ?, ?, ?, ?)
bind => [12, 241, 250, 1364690113727, 1]
[EL Fine]: sql: 2013-03-31 01:35:29.298--ClientSession(1213059092)--Connection(662811604)--Thread(Thread[pool-1-thread-1,5,main])--SELECT LAST_INSERT_ID()
[EL Fine]: sql: 2013-03-31 01:35:29.323--ClientSession(1213059092)--Connection(662811604)--Thread(Thread[pool-1-thread-1,5,main])--INSERT INTO mouse (event, posX, posY, created, uid) VALUES (?, ?, ?, ?, ?)
bind => [12, 233, 296, 1364690113443, 1]
...
Progress
By changing to #GeneratedValue(strategy = GenerationType.TABLE) and allocationSize=300 I've managed to reduce the number of requests by 50%, although it looks as if bind are still sent on their own when checking the EclipseLink log, even though batch writing is supposedly enabled.
[EL Fine]: sql: 2013-03-31 01:35:29.323--ClientSession(1213059092)--Connection(662811604)--Thread(Thread[pool-1-thread-1,5,main])--INSERT INTO mouse (event, posX, posY, created, uid) VALUES (?, ?, ?, ?, ?)
bind => [..., ..., ..., ..., ...]
bind => [..., ..., ..., ..., ...]
bind => [..., ..., ..., ..., ...]
bind => [..., ..., ..., ..., ...]
bind => [..., ..., ..., ..., ...]
bind => [..., ..., ..., ..., ...]
bind => [..., ..., ..., ..., ...]
bind => [..., ..., ..., ..., ...]
bind => [..., ..., ..., ..., ...]
bind => [..., ..., ..., ..., ...]

Answer 1

Change your sequencing to use table sequencing allowing sequence numbers to be preallocated. What you have now forces each insert into its own statement so that the id can be found right after - which prevents batching. Table and other strategies allowing preallocation will give better performance if matched up with the size of your batches. Optimization #6 in http://java-persistence-performance.blogspot.se/2011/06/how-to-improve-jpa-performance-by-1825.html

Answer 2

Try enabling JDBC batch writing. I'm not sure what difference it would make, but it may be worth trying.

Answer 3

For batch writing in MySQL the MySQL JDBC driver does not batch statement unless you have set the following property in your conneciton URL,
?rewriteBatchedStatements=true
i.e.
jdbc:mysql://localhost:3306/db1?rewriteBatchedStatements=true