I implement a Dashboard functionality that checks every time at program start a list of Requirement-Objects for a bunch of different characteristics like progress, missing data and alike and sets for each characteristic a dedicated beacon on the UI.
protected void initializePerformanceIndicator() {
try {
updateA();
updateB();
...
updateF();
updateG();
} catch (Exception e) {
ErrorHandler.showError("Cannot show KPI Performance", e);
}
}
The checks have different compute demands some are faster some slower, therefore each of this checks runs in a dedicated Task to provide some feedback to the user. The skeleton of such a Task is always the same
protected void updateA() throws Exception {
Task<Void> task = new Task<Void>() {
#Override
protected Void call() throws Exception {
embeddedBudgetKPIController.setHyperlink("Budget", null);
embeddedBudgetKPIController.setToolTip("...");
ObservableList<UserRequirement> issues = FXCollections.observableArrayList();
List<UserRequirement> requirements = reqService.getAllUserRequirements(false); // all requirements of the selected product
for(UserRequirement req: requirements) {
if(*some criteria*) {
issues.add(req);
}
}
if(issues.isEmpty()) {
embeddedBudgetKPIController.setBeaconColor(Color.GREEN);
} else {
embeddedBudgetKPIController.setBeaconColor(Color.RED);
}
return null;
};
};
task.setOnSucceeded(e -> {
// Nothing to do
});
Thread tt = new Thread(task);
tt.start();
}
Before initializePerformanceIndicator is called, I retrieved already elsewhere the data from the database querying a number Spring Repositories:
protected final ObservableList<UserRequirement> allUserRequirements = FXCollections.observableArrayList();
public synchronized ObservableList<UserRequirement> getAllUserRequirements(boolean forceUpdate) throws Exception {
logger.debug(""); // show that this method is called
Product selectedProduct = SelectedScope.getSelectedProduct();
if(selectedProduct == null) {
throw new Exception("No selProduct selected");
}
if(forceUpdate || allUserRequirements.isEmpty()) {
allUserRequirements.clear();
allUserRequirements.addAll(epicRepository.findByProductAndRevisionSuccessorIsNull(selectedProduct));
allUserRequirements.addAll(themeRepository.findByProductAndRevisionSuccessorIsNull(selectedProduct));
allUserRequirements.addAll(userStoryRepository.findByProductAndRevisionSuccessorIsNull(selectedProduct));
allUserRequirements.addAll(tangibleRepository.findByProductAndRevisionSuccessorIsNull(selectedProduct));
}
return allUserRequirements;
}
and as you see updateBudgetKPIController calls getallUserRequirements with the parameter false. Therefore it returns the buffered result set and is not re-fetching data from database. So far everything is fine.
I can run each of these Tasks individually without problem. I tried a number combinations with 2 Tasks. Works fine, but the program will never show more than three or four beacons. Which ones are shown differs as well - what is expected as a consequence of the different Tasks. If I exceed three or four Tasks I often get no error at all, but the UI is just not showing more than three to four beacons.
Sometimes I do get an error message, which is
WARN 08:14 o.h.e.j.s.SqlExceptionHelper.logExceptions:137: SQL Error: 0, SQLState: S1009
ERROR 08:14 o.h.e.j.s.SqlExceptionHelper.logExceptions:142: No operations allowed after statement closed.
I debugged it, and realized that I was generating way too many select statements. The UserRequirement entity has almost a dozen OneToMany relations, some where defined with FetchType.LAZY, so I thought it would be better anyway to configure all these relations as
#OneToMany(fetch = FetchType.LAZY, mappedBy="parent", cascade = CascadeType.ALL)
Because of the LAZY loading, every Task tries to load additional data in the if(*some criteria*) part.
The problem did not disappear but I get more information, as the error is now
WARN 11:02 o.h.c.i.AbstractPersistentCollection.withTemporarySessionIfNeeded:278: Unable to close temporary session used to load lazy collection associated to no session
WARN 11:02 o.h.e.j.s.SqlExceptionHelper.logExceptions:137: SQL Error: 0, SQLState: S1009
ERROR 11:02 o.h.e.j.s.SqlExceptionHelper.logExceptions:142: No operations allowed after statement closed.
So I do have a LAZY loading issue.
I am using Spring Boot 2.1.6, MySQL 8.0.15 Community Server, Hibernate Core {5.3.10.Final}, Java 1.8.0_211 and the com.mysql.cj.jdbc.Driver
From a former issue, I have in my properties file the following configuration
# Prevent LazyInitializationException
spring.jpa.properties.hibernate.enable_lazy_load_no_trans=true
Don't know whether this has a side effect?!
Probably changing the LAZY loading to EAGER will fix it - haven't tried yet - but it would delay program start significantly. Therefore I would prefer a solution with LAZY loading.
Any ideas? I also appreciate any ideas regarding how to further isolate the root cause as the error message is not really explicit and I can't see which part of my code triggers it. Plus when I debug it, the behavior changes as I compute all Tasks sequentially rather then in parallel. Thank you in advance.
The issue was caused by different Tasks accessing the same getter of some of the entities. If the first getter call opened a connection, the second call got on it, and then the first call closed the ResultSet, the second call one was in trouble. Synchronizing the getter method solved the problem.
Related
In short, what I am trying solve is how to recover from certain database errors in a Grails application using Hibernate and continue on with the transaction skipping over the failed row updates that are part of a batch of changes.
The application uses Grails 2.3.11 but I have also tried with version 1.3.8 with similar failed results.
Basically there is a Grails service class that iterates over a list of imported records and attempts to update associated master records appropriately. In certain situations exceptions might occur during the domain.save(flush:true) call e.g. org.hibernate.exception.DataException thrown due to issues like (Data truncation: Data too long for column ...).
At this point I have tried:
Disabling transactions
Using domainObj.withTransaction() for each individual record
Trying various #Transactional annotations
Calling domain.clearErrors() and domain.discard() after catching the exception
Tried using a nested service with Transactional annotation with noRollbackFor as shown below
A number of other approaches but nothing I've tried has worked
Example code:
#Transactional
class UpdateService {
public updateBatch(Integer batchId) {
...
list.each { record ->
record.value = 123
try {
nestedService.saveDomain()
} catch (e) {
record.clearErrors()
record.discard()
}
}
batch.status = "POSTED"
batch.save()
}
}
#Transactional
class NestedService {
#Transactional(propagation = Propagation.REQUIRED, noRollbackFor = RuntimeException.class)
public void saveDomain(domainObj) throws RuntimeException {
if (domainObj.validate() && domainObj.save(flush:true) {
log.info "domain $domain was saved"
}
}
}
Once an error occurs I can't seem to clear out the hibernate session. On each subsequent record being updated I receive the error:
org.hibernate.StaleObjectStateException: Row was updated or deleted by another transaction
where it indicates the original failed domain id.
Revision:
Vahid, Thanks for the suggestions. I have tried that. I realized one issue is that I am passing objects across transactional boundaries. So I experimented with the NestedService class do something along the lines of:
#Transactional(propagation = Propagation.REQUIRE_NEW)
public void saveDomain(domainObj) {
def newObj = new Domain.get(domainObj.id)
newObj.properties = domainObj.properties
if (newObj.validate() && newObj.save(force:true) ) { ... }
I expected that to work but the original domainObj still fails even though I'm not calling the save on it. Very strange...
A simple approach would be to loop and then use validate(). If it does fail, then just store the id of the failed entity and proceed.
if(!domainObject.validate()){
// store Id for trying it again later ?
}else{
// Save
}
We've noticed a couple of times in our mobile applications that users have reported the application hanging or seeming to become unresponsive between views / rare crashes when switching between views. We've tracked down these cases to when our view model constructors throw uncaught exceptions.
We want to put a solution in place so that if a view model fails to construct for some reason then we can notify the user and provide some message that will be useful to us when it's logged through support.
I've been taking a look at doing this but haven't found a reliable way to achieve this.
The first thing we tried was at the IMvxViewModelLocator level. We already have a custom implementation of IMvxViewModelLocator so we've modified this. We allow all exceptions to be thrown and then we have an IErrorHandler interface which each platform implements. We then call this to attempt to show a dialog. This has proved to be unreliable and the dialog does not always display. Something along the lines of: (note - here ResolveViewModel will always return true or throw)
public override bool TryLoad(Type viewModelType, IMvxBundle parameterValues, IMvxBundle savedState, out IMvxViewModel viewModel)
{
try
{
return ResolveViewModel(viewModelType, parameterValues, savedState, out viewModel);
}
catch (Exception exception)
{
_errorHandler.HandleViewModelConstructionException(viewModelType, exception);
viewModel = null;
return false;
}
}
What we would ideally like to do is intercept any failure to construct a view model and then re-request an ErrorViewModel. We've tried to do this 2 ways:
1)
We've tried defining a custom IMvxViewDispatcher for each platform and we're trying to intercept failures as below but if an exception in the constructor is thrown we never get back this far:
public class TouchDispatcher : MvxTouchUIThreadDispatcher, IMvxViewDispatcher
{
private readonly IMvxTouchViewPresenter _presenter;
public TouchDispatcher(IMvxTouchViewPresenter presenter)
{
_presenter = presenter;
}
public bool ShowViewModel(MvxViewModelRequest request)
{
Action action = () =>
{
_presenter.Show(request);
};
try
{
bool success = RequestMainThreadAction(action);
return !success ? HandleError() : success;
}
catch (Exception)
{
return HandleError();
}
}
// Other bits
}
2)
We thought we might have more success at the presenter level. We modified our ViewPresenter for each platform and we have overridden void Show(MvxViewModelRequest request). This has not proved to be successful either as exceptions don't propagate back this far.
This leaves me thinking that maybe we are better attempting this at the IMvxViewModelLocator level again.
Has anyone found a way to reliably intercept failures to construct view models and then ideally re-request a different view model / present some dialog to the user?
It seems you've identified that the core of the problem is when: "view model constructors throw uncaught exceptions."
This is going to be slightly problematic as the ViewModel's are generally constructed during View lifecycle overrides like ViewDidLoad, OnCreate or NavigatedTo - which is generally after the Presenter has finished requesting presentation.
As you've already found an easy place to identify when ViewModel construction has failed is in a custom IMvxViewModelLocator - others likeIMvxViewModelLoader are also possible. This is probably the easiest place to catch the error and to trigger the error handling - you can then get hold of the IMvxViewDispatcher (or presenter) there in order to change the display. However, you will still need to make sure your Views can handle null created ViewModels - as the ViewDidLoad, etc calls will still need to complete.
I am working on Entity Framework 4.1 . Here Adding control into database using AddObject() and save that suing SaveChange() methods.
But Once I delete that added control and try to add again same I am getting this error again and again (Store update, insert, or delete statement affected an unexpected number of rows (0). Entities may have been modified or deleted since entities were loaded. Refresh ObjectStateManager entries.) but not able to add it. Once i close my application then try to add then I am able to add that control.
I tried to search a lot here and there how it going wrong but could not find solution. As I am new born in field in Entity Framework.
As in this scenario i was calling SaveChange() method of Entity framework object context after every operation like add, delete and modification. But i was getting exception back to back.It got solved by
By calling method like this
public void Save(object entity)
{
using (var transaction = Connection.BeginTransaction())
{
try
{
SaveChanges();
transaction.Commit();
}
catch (OptimisticConcurrencyException)
{
if (ObjectStateManager.GetObjectStateEntry(entity).State == EntityState.Deleted || ObjectStateManager.GetObjectStateEntry(entity).State == EntityState.Modified)
this.Refresh(RefreshMode.StoreWins, entity);
else if (ObjectStateManager.GetObjectStateEntry(entity).State == EntityState.Added)
Detach(entity);
AcceptAllChanges();
transaction.Commit();
}
}
}
I am trying learn how to best use the Reactive Extensions library and have set up simple test WPF application to view a logging database table. In a ViewModel class I am populating an ObservableCollection with the first 100 log entries from a Linq to Sql DataContext and I'm trying to use Rx to keep the UI responsive.
The following snippet works unless the database is unavailable at which point the app throws an exception and crashes. Where would be the best place to handle database connection exceptions and why are they not handled by the OnError method of the Observer?
ObservableCollection<LogEntry> _logEntries = new ObservableCollection<LogEntry>();
DataContext dataContext = new DataContext( "connection string" );
(from e in dataContext.LogEntries
select e).Take( 100 ).ToObservable()
.SubscribeOn( Scheduler.ThreadPool )
.ObserveOnDispatcher()
.Subscribe( _logEntries.Add, ex => System.Diagnostics.Debug.WriteLine( ex.ToString() ) );
Try this instead of ToObservable:
public static IObservable<T> SafeToObservable(this IEnumerable<T> This)
{
return Observable.Create(subj => {
try {
foreach(var v in This) {
subj.OnNext(v);
}
subj.OnCompleted();
} catch (Exception ex) {
subj.OnError(ex);
}
return Disposable.Empty;
});
}
In general though, this isn't a great use of Rx since the data source isn't very easy to Rx'ify - in fact, the code will execute most of the work on the UI thread, send it out to random worker threads, then send it back (i.e. completely wasted work). Task + Dispatcher.BeginInvoke might suit you better here.
I have implemented a DAL using Rob Conery's spin on the repository pattern (from the MVC Storefront project) where I map database objects to domain objects using Linq and use Linq to SQL to actually get the data.
This is all working wonderfully giving me the full control over the shape of my domain objects that I want, but I have hit a problem with concurrency that I thought I'd ask about here. I have concurrency working but the solution feels like it might be wrong (just one of those gitchy feelings).
The basic pattern is:
private MyDataContext _datacontext
private Table _tasks;
public Repository(MyDataContext datacontext)
{
_dataContext = datacontext;
}
public void GetTasks()
{
_tasks = from t in _dataContext.Tasks;
return from t in _tasks
select new Domain.Task
{
Name = t.Name,
Id = t.TaskId,
Description = t.Description
};
}
public void SaveTask(Domain.Task task)
{
Task dbTask = null;
// Logic for new tasks omitted...
dbTask = (from t in _tasks
where t.TaskId == task.Id
select t).SingleOrDefault();
dbTask.Description = task.Description,
dbTask.Name = task.Name,
_dataContext.SubmitChanges();
}
So with that implementation I've lost concurrency tracking because of the mapping to the domain task. I get it back by storing the private Table which is my datacontext list of tasks at the time of getting the original task.
I then update the tasks from this stored Table and save what I've updated
This is working - I get change conflict exceptions raised when there are concurrency violations, just as I want.
However, it just screams to me that I've missed a trick.
Is there a better way of doing this?
I've looked at the .Attach method on the datacontext but that appears to require storing the original version in a similar way to what I'm already doing.
I also know that I could avoid all this by doing away with the domain objects and letting the Linq to SQL generated objects all the way up my stack - but I dislike that just as much as I dislike the way I'm handling concurrency.
I worked through this and found the following solution. It works in all the test cases I (and more importantly, my testers!) can think of.
I am using the .Attach() method on the datacontext, and a TimeStamp column. This works fine for the first time that you save a particular primary key back to the database but I found that the datacontext throws a System.Data.Linq.DuplicateKeyException "Cannot add an entity with a key that is already in use."
The work around for this I created was to add a dictionary that stored the item I attach the first time around and then every subsequent time I save I reuse that item.
Example code is below, I do wonder if I've missed any tricks - concurrency is pretty fundamental so the hoops I'm jumping through seem a little excessive.
Hopefully the below proves useful, or someone can point me towards a better implementation!
private Dictionary<int, Payment> _attachedPayments;
public void SavePayments(IList<Domain.Payment> payments)
{
Dictionary<Payment, Domain.Payment> savedPayments =
new Dictionary<Payment, Domain.Payment>();
// Items with a zero id are new
foreach (Domain.Payment p in payments.Where(p => p.PaymentId != 0))
{
// The list of attached payments that works around the linq datacontext
// duplicatekey exception
if (_attachedPayments.ContainsKey(p.PaymentId)) // Already attached
{
Payment dbPayment = _attachedPayments[p.PaymentId];
// Just a method that maps domain to datacontext types
MapDomainPaymentToDBPayment(p, dbPayment, false);
savedPayments.Add(dbPayment, p);
}
else // Attach this payment to the datacontext
{
Payment dbPayment = new Payment();
MapDomainPaymentToDBPayment(p, dbPayment, true);
_dataContext.Payments.Attach(dbPayment, true);
savedPayments.Add(dbPayment, p);
}
}
// There is some code snipped but this is just brand new payments
foreach (var payment in newPayments)
{
Domain.Payment payment1 = payment;
Payment newPayment = new Payment();
MapDomainPaymentToDBPayment(payment1, newPayment, false);
_dataContext.Payments.InsertOnSubmit(newPayment);
savedPayments.Add(newPayment, payment);
}
try
{
_dataContext.SubmitChanges();
// Grab the Timestamp into the domain object
foreach (Payment p in savedPayments.Keys)
{
savedPayments[p].PaymentId = p.PaymentId;
savedPayments[p].Timestamp = p.Timestamp;
_attachedPayments[savedPayments[p].PaymentId] = p;
}
}
catch (ChangeConflictException ex)
{
foreach (ObjectChangeConflict occ in _dataContext.ChangeConflicts)
{
Payment entityInConflict = (Payment) occ.Object;
// Use the datacontext refresh so that I can display the new values
_dataContext.Refresh(RefreshMode.OverwriteCurrentValues, entityInConflict);
_attachedPayments[entityInConflict.PaymentId] = entityInConflict;
}
throw;
}
}
I would look at trying to utilise the .Attach method by passing the 'original' and 'updated' objects thus achieving true optimistic concurrency checking from LINQ2SQL. This IMO would be preferred to using version or datetime stamps either in the DBML objects or your Domain objects. I'm not sure how MVC allows for this idea of persisting the 'original' data however.. i've been trying to investigate the validation scaffolding in the hope that it's storing the 'original' data.. but i suspect that it is as only as good as the most recent post (and/or failed validation). So that idea may not work.
Another crazy idea i had was this: override the GetHashCode() for all of your domain objects where the hash represents the unique set of data for that object (minus the ID of course). Then, either manually or with a helper bury that hash in a hidden field in the HTML POST form and send it back to your service layer with your updated domain object - do the concurrency checking in your service layer or data layer (by comparing the original hash with a newly extracted domain object's hash) but be aware that you need to be checking for and raising concurrency exceptions yourself. It's nice to use the DMBL functions but the idea of abstracting away the data layer is so to not depend on the particular implementation's features etc. So having full control of the optimistic concurrency checking on your domain objects in your service layer (for example) seems like a good approach to me.