How to recover from database errors in Grails within a transaction - exception

In short, what I am trying solve is how to recover from certain database errors in a Grails application using Hibernate and continue on with the transaction skipping over the failed row updates that are part of a batch of changes.
The application uses Grails 2.3.11 but I have also tried with version 1.3.8 with similar failed results.
Basically there is a Grails service class that iterates over a list of imported records and attempts to update associated master records appropriately. In certain situations exceptions might occur during the domain.save(flush:true) call e.g. org.hibernate.exception.DataException thrown due to issues like (Data truncation: Data too long for column ...).
At this point I have tried:
Disabling transactions
Using domainObj.withTransaction() for each individual record
Trying various #Transactional annotations
Calling domain.clearErrors() and domain.discard() after catching the exception
Tried using a nested service with Transactional annotation with noRollbackFor as shown below
A number of other approaches but nothing I've tried has worked
Example code:
#Transactional
class UpdateService {
public updateBatch(Integer batchId) {
...
list.each { record ->
record.value = 123
try {
nestedService.saveDomain()
} catch (e) {
record.clearErrors()
record.discard()
}
}
batch.status = "POSTED"
batch.save()
}
}
#Transactional
class NestedService {
#Transactional(propagation = Propagation.REQUIRED, noRollbackFor = RuntimeException.class)
public void saveDomain(domainObj) throws RuntimeException {
if (domainObj.validate() && domainObj.save(flush:true) {
log.info "domain $domain was saved"
}
}
}
Once an error occurs I can't seem to clear out the hibernate session. On each subsequent record being updated I receive the error:
org.hibernate.StaleObjectStateException: Row was updated or deleted by another transaction
where it indicates the original failed domain id.
Revision:
Vahid, Thanks for the suggestions. I have tried that. I realized one issue is that I am passing objects across transactional boundaries. So I experimented with the NestedService class do something along the lines of:
#Transactional(propagation = Propagation.REQUIRE_NEW)
public void saveDomain(domainObj) {
def newObj = new Domain.get(domainObj.id)
newObj.properties = domainObj.properties
if (newObj.validate() && newObj.save(force:true) ) { ... }
I expected that to work but the original domainObj still fails even though I'm not calling the save on it. Very strange...

A simple approach would be to loop and then use validate(). If it does fail, then just store the id of the failed entity and proceed.
if(!domainObject.validate()){
// store Id for trying it again later ?
}else{
// Save
}

Related

Spring Boot+ MySQL: LAZY Loading issue - No operations allowed after statement closed

I implement a Dashboard functionality that checks every time at program start a list of Requirement-Objects for a bunch of different characteristics like progress, missing data and alike and sets for each characteristic a dedicated beacon on the UI.
protected void initializePerformanceIndicator() {
try {
updateA();
updateB();
...
updateF();
updateG();
} catch (Exception e) {
ErrorHandler.showError("Cannot show KPI Performance", e);
}
}
The checks have different compute demands some are faster some slower, therefore each of this checks runs in a dedicated Task to provide some feedback to the user. The skeleton of such a Task is always the same
protected void updateA() throws Exception {
Task<Void> task = new Task<Void>() {
#Override
protected Void call() throws Exception {
embeddedBudgetKPIController.setHyperlink("Budget", null);
embeddedBudgetKPIController.setToolTip("...");
ObservableList<UserRequirement> issues = FXCollections.observableArrayList();
List<UserRequirement> requirements = reqService.getAllUserRequirements(false); // all requirements of the selected product
for(UserRequirement req: requirements) {
if(*some criteria*) {
issues.add(req);
}
}
if(issues.isEmpty()) {
embeddedBudgetKPIController.setBeaconColor(Color.GREEN);
} else {
embeddedBudgetKPIController.setBeaconColor(Color.RED);
}
return null;
};
};
task.setOnSucceeded(e -> {
// Nothing to do
});
Thread tt = new Thread(task);
tt.start();
}
Before initializePerformanceIndicator is called, I retrieved already elsewhere the data from the database querying a number Spring Repositories:
protected final ObservableList<UserRequirement> allUserRequirements = FXCollections.observableArrayList();
public synchronized ObservableList<UserRequirement> getAllUserRequirements(boolean forceUpdate) throws Exception {
logger.debug(""); // show that this method is called
Product selectedProduct = SelectedScope.getSelectedProduct();
if(selectedProduct == null) {
throw new Exception("No selProduct selected");
}
if(forceUpdate || allUserRequirements.isEmpty()) {
allUserRequirements.clear();
allUserRequirements.addAll(epicRepository.findByProductAndRevisionSuccessorIsNull(selectedProduct));
allUserRequirements.addAll(themeRepository.findByProductAndRevisionSuccessorIsNull(selectedProduct));
allUserRequirements.addAll(userStoryRepository.findByProductAndRevisionSuccessorIsNull(selectedProduct));
allUserRequirements.addAll(tangibleRepository.findByProductAndRevisionSuccessorIsNull(selectedProduct));
}
return allUserRequirements;
}
and as you see updateBudgetKPIController calls getallUserRequirements with the parameter false. Therefore it returns the buffered result set and is not re-fetching data from database. So far everything is fine.
I can run each of these Tasks individually without problem. I tried a number combinations with 2 Tasks. Works fine, but the program will never show more than three or four beacons. Which ones are shown differs as well - what is expected as a consequence of the different Tasks. If I exceed three or four Tasks I often get no error at all, but the UI is just not showing more than three to four beacons.
Sometimes I do get an error message, which is
WARN 08:14 o.h.e.j.s.SqlExceptionHelper.logExceptions:137: SQL Error: 0, SQLState: S1009
ERROR 08:14 o.h.e.j.s.SqlExceptionHelper.logExceptions:142: No operations allowed after statement closed.
I debugged it, and realized that I was generating way too many select statements. The UserRequirement entity has almost a dozen OneToMany relations, some where defined with FetchType.LAZY, so I thought it would be better anyway to configure all these relations as
#OneToMany(fetch = FetchType.LAZY, mappedBy="parent", cascade = CascadeType.ALL)
Because of the LAZY loading, every Task tries to load additional data in the if(*some criteria*) part.
The problem did not disappear but I get more information, as the error is now
WARN 11:02 o.h.c.i.AbstractPersistentCollection.withTemporarySessionIfNeeded:278: Unable to close temporary session used to load lazy collection associated to no session
WARN 11:02 o.h.e.j.s.SqlExceptionHelper.logExceptions:137: SQL Error: 0, SQLState: S1009
ERROR 11:02 o.h.e.j.s.SqlExceptionHelper.logExceptions:142: No operations allowed after statement closed.
So I do have a LAZY loading issue.
I am using Spring Boot 2.1.6, MySQL 8.0.15 Community Server, Hibernate Core {5.3.10.Final}, Java 1.8.0_211 and the com.mysql.cj.jdbc.Driver
From a former issue, I have in my properties file the following configuration
# Prevent LazyInitializationException
spring.jpa.properties.hibernate.enable_lazy_load_no_trans=true
Don't know whether this has a side effect?!
Probably changing the LAZY loading to EAGER will fix it - haven't tried yet - but it would delay program start significantly. Therefore I would prefer a solution with LAZY loading.
Any ideas? I also appreciate any ideas regarding how to further isolate the root cause as the error message is not really explicit and I can't see which part of my code triggers it. Plus when I debug it, the behavior changes as I compute all Tasks sequentially rather then in parallel. Thank you in advance.
The issue was caused by different Tasks accessing the same getter of some of the entities. If the first getter call opened a connection, the second call got on it, and then the first call closed the ResultSet, the second call one was in trouble. Synchronizing the getter method solved the problem.

How to know exception occurred within grails transaction?

I have a service method which does some operation inside a transaction.
public User method1() {
// some code...
Vehicle.withTransaction { status ->
// some collection loop
// some other delete
vehicle.delete(failOnError:true)
}
if (checkSomething outside transaction) {
return throw some user defined exception
}
return user
}
If there is a runtime exception we dont have to catch that exception and the transaction will be rolled back automatically. But how to determine that transaction rolled back due to some exception and I want to throw some user friendly error message. delete() call also wont return anything.
If I add try/catch block inside the transaction by catching the Exception (super class) it is not getting into that exception block. But i was expecting it to go into that block and throw user friendly exception.
EDIT 1: Is it a good idea to add try/catch arround withTransaction
Any idea how to solver this?? Thanks in advance.
If I understand you question correctly, you want to know how to catch an exception, determine what the exception is, and return a message to the user. There are a few ways to do this. I will show you how I do it.
Before I get to the code there are a few things I might suggest. First, you don't need to explicitly declare the transaction in a service (I'm using v2.2.5). Services are transactional by default (not a big deal).
Second, the transaction will automatically roll back if any exception occurs while executing the service method.
Third, I would recommend removing failOnError:true from save() (I don't think it works on delete()... I may be wrong?). I find it is easier to run validate() or save() in the service then return the model instance to the controller where the objects errors can be used in a flash message.
The following is a sample of how I like to handle exceptions and saves using a service method and try/catch in the controller:
class FooService {
def saveFoo(Foo fooInstance) {
return fooInstance.save()
}
def anotherSaveFoo(Foo fooInstance) {
if(fooInstance.validate()){
fooInstance.save()
}else{
do something else or
throw new CustomException()
}
return fooInstance
}
}
class FooController {
def save = {
def newFoo = new Foo(params)
try{
returnedFoo = fooService.saveFoo(newFoo)
}catch(CustomException | Exception e){
flash.warning = [message(code: 'foo.validation.error.message',
args: [org.apache.commons.lang.exception.ExceptionUtils.getRootCauseMessage(e)],
default: "The foo changes did not pass validation.<br/>{0}")]
redirect('to where ever you need to go')
return
}
if(returnedFoo.hasErrors()){
def fooErrors = returnedFoo.errors.getAllErrors()
flash.warning = [message(code: 'foo.validation.error.message',
args: [fooErrors],
default: "The foo changes did not pass validation.<br/>${fooErrors}")]
redirect('to where ever you need to go')
return
}else {
flash.success = [message(code: 'foo.saved.successfully.message',
default: "The foo was saved successfully")]
redirect('to where ever you need to go')
}
}
}
Hope this helps, or gets some other input from more experienced Grails developers.
Here are a few other ways I've found to get exception info to pass along to your user:
request.exception.cause
request.exception.cause.message
response.status
A few links to other relevant questions that may help:
Exception handling in Grails controllers
Exception handling in Grails controllers with ExceptionMapper in Grails 2.2.4 best practice
https://commons.apache.org/proper/commons-lang/javadocs/api-2.6/org/apache/commons/lang/exception/ExceptionUtils.html

Exception while calling savechange method while adding, removing or modifying entity. IMP

I am working on Entity Framework 4.1 . Here Adding control into database using AddObject() and save that suing SaveChange() methods.
But Once I delete that added control and try to add again same I am getting this error again and again (Store update, insert, or delete statement affected an unexpected number of rows (0). Entities may have been modified or deleted since entities were loaded. Refresh ObjectStateManager entries.) but not able to add it. Once i close my application then try to add then I am able to add that control.
I tried to search a lot here and there how it going wrong but could not find solution. As I am new born in field in Entity Framework.
As in this scenario i was calling SaveChange() method of Entity framework object context after every operation like add, delete and modification. But i was getting exception back to back.It got solved by
By calling method like this
public void Save(object entity)
{
using (var transaction = Connection.BeginTransaction())
{
try
{
SaveChanges();
transaction.Commit();
}
catch (OptimisticConcurrencyException)
{
if (ObjectStateManager.GetObjectStateEntry(entity).State == EntityState.Deleted || ObjectStateManager.GetObjectStateEntry(entity).State == EntityState.Modified)
this.Refresh(RefreshMode.StoreWins, entity);
else if (ObjectStateManager.GetObjectStateEntry(entity).State == EntityState.Added)
Detach(entity);
AcceptAllChanges();
transaction.Commit();
}
}
}

Grails UnexpectedRollbackException occurred: Not Sure Why

I have the following code:
class ServiceA {
def save(Object object) {
if (somethingBadComesBack) {
throw new CustomRuntimeException(data)
}
}
}
class ServiceB {
def serviceA
def save(Object object) {
try {
serviceA.save(object)
// do more stuff if good to go
} catch(CustomRuntimeException e) {
// populate some objects with errors based on exception
}
}
}
class ServiceC {
def serviceB
def process(Object object) {
serviceB.save(object)
if (object.hasErrors() {
// do some stuff
}else{
// do some stuff
}
def info = someMethod(object)
return info
}
}
class SomeController {
def serviceC
def process() {
def object = .....
serviceC.save(object) // UnexpectedRollbackException is thrown here
}
}
When ServiceA.save() is called and an exception occurs, ServiceC.save() is throwing an UnexpectedRollbackException when it tries to return.
I did the following:
try {
serviceC.process(object)
}catch(UnexpectedRollbackException e) {
println e.getMostSpecificCause()
}
and I am getting:
org.springframework.transaction.UnexpectedRollbackException: Transaction rolled back because it has been marked as rollback-only
I'm not sure where to start looking for how to fix this.
You're using a runtime exception to roll back the transaction, but that's cheating - it's taking advantage of a side effect. Runtime exceptions roll back transactions automatically since you don't need to catch them so it's assumed that if one is thrown, it wasn't expected and the default behavior is to roll back. You can configure methods to not roll back for specific expected runtime exceptions, but this is somewhat rare. Checked exceptions don't roll back exceptions because in Java they must be caught or declared in the throws, so you have to have either explicitly thrown it or ducked it; either way you had a chance to try again.
The correct way to intentionally roll back a transaction is to call setRollbackOnly() on the current TransactionStatus but this isn't directly accessible in a service method (it is in a withTransaction block since it's the argument to the closure). But it's easy to get to: import org.springframework.transaction.interceptor.TransactionAspectSupport and call TransactionAspectSupport.currentTransactionStatus().setRollbackOnly(). This will required that you rework your code since there won't be an exception to catch, so you'll need to check that it was rolled back with TransactionAspectSupport.currentTransactionStatus().isRollbackOnly().
I'm not sure if it's a Grails issue or standard behavior, but when I was debugging this there were 3 commit calls with 3 different TransactionStatus instances. Only the first had the rollback flag set, but the second was aware of the first and was ok. The third one was considered a new transaction and was the one that triggered the same exception that you were seeing. So to work around this I added this to the 2nd and 3rd service methods:
def status = TransactionAspectSupport.currentTransactionStatus()
if (!status.isRollbackOnly()) status.setRollbackOnly()
to chain the rollback flag. That worked and I didn't get the UnexpectedRollbackException.
It might be easier to combine this with a checked exception. It's still overly expensive since it will fill in the stacktrace unnecessarily, but if you call setRollbackOnly() and throw a checked exception, you will be able to use the same general workflow you have now.
It seems like the default transactionality of services is biting you, and the unchecked exception thrown in Service A is dooming the transaction to rollback only, even once caught.
The above docs say the txn propagation level is PROPAGATION_REQUIRED, which should mean the same transaction is shared from your service C on up to A, if my memory serves me. Can you have Service A's save method throw a checked exception instead of RuntimeException, to avoid the auto-rollback from the latter? Or disable transactions on your services, if that's an option for you?

grails rollback db operation on some error

I have a service where to save a lot of data to db.
Using MYSQL
I have used like this
Domain1.withTransaction {text->
def domain1=//Create my domain object to save
if(!domain1.save()){
domain1.errors.each {
println it
}
throw new RuntimeException('unable to save domain1')
}
Domain2.withTransaction {text->
def domain2=//Create my domain object to save
if(!domain2.save()){
domain2.errors.each {
println it
}
throw new RuntimeException('unable to save domain2')
}
My problem if there occurred any problem in saving domain2 i need to roll back domain1 save also.
I need to remove domain1 from db.
Instead of using programatic transaction handling, the Service artifact allows for automatic transaction handling. This typically leads to cleaner and more maintainable code.
You can also use the failOnError:true when you save() to force a RuntimeException to be thrown.
Example below:
class SomeService {
static transactional = false
def serviceMethod() {
def domain1=//Create my domain object to save
def domain2=//Create my domain object to save
domain1.save(failOnError:true)
domain2.save(failOnError:true)
}
}
UPDATE
I'm revisiting this topic, after I read a response to another topic. Grails Integration Test Does NOT Rollback
Please verify that your dialect is setup as InnoDB as MyISAM tables aren't transactional. This is configured in your Config.groovy
dataSource {
dialect= org.hibernate.dialect.MySQLInnoDBDialect
...
}
Try removing the Domain2.withTransaction {text-> part.
You are already inside a transaction with your first call.
If you do further work inside the brackets, you should stay within the same transaction and domain1 should be rolled back if you throw and exception after checking domain2.
Put that Domain2.withTransaction closure inside Domain1.withTransaction closure, so error in Domain2 transaction will rollback both Domain1 and Domain2 transaction
Like this
Domain1.withTransaction {
//....
Domain2.withTransaction {
//....
}
}
If you just want a single transaction which is rolled back when an unhandled unchecked exception is thrown, then don't start a nested transaction. The necessaary code changes are shown below:
Domain1.withTransaction {text->
def domain1=//Create my domain object to save
if (!domain1.save()) {
domain1.errors.each {
println it
}
throw new RuntimeException('unable to save domain1')
}
def domain2=//Create my domain object to save
if (!domain2.save()) {
domain2.errors.each {
println it
}
throw new RuntimeException('unable to save domain2')
}
}