Handling database connection exceptions with Linq to SQL and Rx - linq-to-sql

I am trying learn how to best use the Reactive Extensions library and have set up simple test WPF application to view a logging database table. In a ViewModel class I am populating an ObservableCollection with the first 100 log entries from a Linq to Sql DataContext and I'm trying to use Rx to keep the UI responsive.
The following snippet works unless the database is unavailable at which point the app throws an exception and crashes. Where would be the best place to handle database connection exceptions and why are they not handled by the OnError method of the Observer?
ObservableCollection<LogEntry> _logEntries = new ObservableCollection<LogEntry>();
DataContext dataContext = new DataContext( "connection string" );
(from e in dataContext.LogEntries
select e).Take( 100 ).ToObservable()
.SubscribeOn( Scheduler.ThreadPool )
.ObserveOnDispatcher()
.Subscribe( _logEntries.Add, ex => System.Diagnostics.Debug.WriteLine( ex.ToString() ) );

Try this instead of ToObservable:
public static IObservable<T> SafeToObservable(this IEnumerable<T> This)
{
return Observable.Create(subj => {
try {
foreach(var v in This) {
subj.OnNext(v);
}
subj.OnCompleted();
} catch (Exception ex) {
subj.OnError(ex);
}
return Disposable.Empty;
});
}
In general though, this isn't a great use of Rx since the data source isn't very easy to Rx'ify - in fact, the code will execute most of the work on the UI thread, send it out to random worker threads, then send it back (i.e. completely wasted work). Task + Dispatcher.BeginInvoke might suit you better here.

Related

Spring Boot+ MySQL: LAZY Loading issue - No operations allowed after statement closed

I implement a Dashboard functionality that checks every time at program start a list of Requirement-Objects for a bunch of different characteristics like progress, missing data and alike and sets for each characteristic a dedicated beacon on the UI.
protected void initializePerformanceIndicator() {
try {
updateA();
updateB();
...
updateF();
updateG();
} catch (Exception e) {
ErrorHandler.showError("Cannot show KPI Performance", e);
}
}
The checks have different compute demands some are faster some slower, therefore each of this checks runs in a dedicated Task to provide some feedback to the user. The skeleton of such a Task is always the same
protected void updateA() throws Exception {
Task<Void> task = new Task<Void>() {
#Override
protected Void call() throws Exception {
embeddedBudgetKPIController.setHyperlink("Budget", null);
embeddedBudgetKPIController.setToolTip("...");
ObservableList<UserRequirement> issues = FXCollections.observableArrayList();
List<UserRequirement> requirements = reqService.getAllUserRequirements(false); // all requirements of the selected product
for(UserRequirement req: requirements) {
if(*some criteria*) {
issues.add(req);
}
}
if(issues.isEmpty()) {
embeddedBudgetKPIController.setBeaconColor(Color.GREEN);
} else {
embeddedBudgetKPIController.setBeaconColor(Color.RED);
}
return null;
};
};
task.setOnSucceeded(e -> {
// Nothing to do
});
Thread tt = new Thread(task);
tt.start();
}
Before initializePerformanceIndicator is called, I retrieved already elsewhere the data from the database querying a number Spring Repositories:
protected final ObservableList<UserRequirement> allUserRequirements = FXCollections.observableArrayList();
public synchronized ObservableList<UserRequirement> getAllUserRequirements(boolean forceUpdate) throws Exception {
logger.debug(""); // show that this method is called
Product selectedProduct = SelectedScope.getSelectedProduct();
if(selectedProduct == null) {
throw new Exception("No selProduct selected");
}
if(forceUpdate || allUserRequirements.isEmpty()) {
allUserRequirements.clear();
allUserRequirements.addAll(epicRepository.findByProductAndRevisionSuccessorIsNull(selectedProduct));
allUserRequirements.addAll(themeRepository.findByProductAndRevisionSuccessorIsNull(selectedProduct));
allUserRequirements.addAll(userStoryRepository.findByProductAndRevisionSuccessorIsNull(selectedProduct));
allUserRequirements.addAll(tangibleRepository.findByProductAndRevisionSuccessorIsNull(selectedProduct));
}
return allUserRequirements;
}
and as you see updateBudgetKPIController calls getallUserRequirements with the parameter false. Therefore it returns the buffered result set and is not re-fetching data from database. So far everything is fine.
I can run each of these Tasks individually without problem. I tried a number combinations with 2 Tasks. Works fine, but the program will never show more than three or four beacons. Which ones are shown differs as well - what is expected as a consequence of the different Tasks. If I exceed three or four Tasks I often get no error at all, but the UI is just not showing more than three to four beacons.
Sometimes I do get an error message, which is
WARN 08:14 o.h.e.j.s.SqlExceptionHelper.logExceptions:137: SQL Error: 0, SQLState: S1009
ERROR 08:14 o.h.e.j.s.SqlExceptionHelper.logExceptions:142: No operations allowed after statement closed.
I debugged it, and realized that I was generating way too many select statements. The UserRequirement entity has almost a dozen OneToMany relations, some where defined with FetchType.LAZY, so I thought it would be better anyway to configure all these relations as
#OneToMany(fetch = FetchType.LAZY, mappedBy="parent", cascade = CascadeType.ALL)
Because of the LAZY loading, every Task tries to load additional data in the if(*some criteria*) part.
The problem did not disappear but I get more information, as the error is now
WARN 11:02 o.h.c.i.AbstractPersistentCollection.withTemporarySessionIfNeeded:278: Unable to close temporary session used to load lazy collection associated to no session
WARN 11:02 o.h.e.j.s.SqlExceptionHelper.logExceptions:137: SQL Error: 0, SQLState: S1009
ERROR 11:02 o.h.e.j.s.SqlExceptionHelper.logExceptions:142: No operations allowed after statement closed.
So I do have a LAZY loading issue.
I am using Spring Boot 2.1.6, MySQL 8.0.15 Community Server, Hibernate Core {5.3.10.Final}, Java 1.8.0_211 and the com.mysql.cj.jdbc.Driver
From a former issue, I have in my properties file the following configuration
# Prevent LazyInitializationException
spring.jpa.properties.hibernate.enable_lazy_load_no_trans=true
Don't know whether this has a side effect?!
Probably changing the LAZY loading to EAGER will fix it - haven't tried yet - but it would delay program start significantly. Therefore I would prefer a solution with LAZY loading.
Any ideas? I also appreciate any ideas regarding how to further isolate the root cause as the error message is not really explicit and I can't see which part of my code triggers it. Plus when I debug it, the behavior changes as I compute all Tasks sequentially rather then in parallel. Thank you in advance.
The issue was caused by different Tasks accessing the same getter of some of the entities. If the first getter call opened a connection, the second call got on it, and then the first call closed the ResultSet, the second call one was in trouble. Synchronizing the getter method solved the problem.

Hystrix/Feign to solely react on HTTP status 429

I'm using Feign from the spring-cloud-starter-feign to send requests to a defined backend. I would like to use Hystrix as a circuit-breaker but for only one type of use-case: If the backend responds with a HTTP 429: Too many requests code, my Feign client should wait exactly one hour until it contacts the real backend again. Until then, a fallback method should be executed.
How would I have to configure my Spring Boot (1.5.10) application in order to accomplish that? I see many configuration possibilities but only few examples which are - in my opinion - unfortunately not resolved around use-cases.
This can be achieved by defining an ErrorDecoder and taking manual control of the Hystrix Circuit Breaker. You can inspect the response codes from the exceptions and provide your own fallback. In addition, if you wish to retry the request, wrap and throw your exception in a RetryException.
To meet your Retry requirement, also register a Retryer bean with the appropriate configuration. Keep in mind that using a Retryer will tie up a thread for the duration. The default implementation of Retryer does use an exponential backoff policy as well.
Here is an example ErrorDecoder taken from the OpenFeign documentation:
public class StashErrorDecoder implements ErrorDecoder {
#Override
public Exception decode(String methodKey, Response response) {
if (response.status() >= 400 && response.status() <= 499) {
return new StashClientException(
response.status(),
response.reason()
);
}
if (response.status() >= 500 && response.status() <= 599) {
return new StashServerException(
response.status(),
response.reason()
);
}
return errorStatus(methodKey, response);
}
}
In your case, you would react to 419 as desired.
You can forcibly open the Circuit Breaker setting this property at runtime
hystrix.command.HystrixCommandKey.circuitBreaker.forceOpen
ConfigurationManager.getConfigInstance()
.setProperty(
"hystrix.command.HystrixCommandKey.circuitBreaker.forceOpen", true);
Replace HystrixCommandKey with your own command. You will need to restore this circuit breaker back to closed after the desired time.
I could solve it with the following adjustments:
Properties in application.yml:
hystrix.command.commandKey:
execution.isolation.thread.timeoutInMilliseconds: 10_000
metrics.rollingStats.timeInMilliseconds: 10_000
circuitBreaker:
errorThresholdPercentage: 1
requestVolumeThreshold: 1
sleepWindowInMilliseconds: 3_600_000
Code in the respective Java class:
#HystrixCommand(fallbackMethod = "fallbackMethod", commandKey = COMMAND_KEY)
public void doCall(String parameter) {
try {
feignClient.doCall(parameter);
} catch (FeignException e) {
if (e.status() == 429) {
throw new TooManyRequestsException(e.getMessage());
}
}
}

MVVMCross ViewModel construction failure notifications

We've noticed a couple of times in our mobile applications that users have reported the application hanging or seeming to become unresponsive between views / rare crashes when switching between views. We've tracked down these cases to when our view model constructors throw uncaught exceptions.
We want to put a solution in place so that if a view model fails to construct for some reason then we can notify the user and provide some message that will be useful to us when it's logged through support.
I've been taking a look at doing this but haven't found a reliable way to achieve this.
The first thing we tried was at the IMvxViewModelLocator level. We already have a custom implementation of IMvxViewModelLocator so we've modified this. We allow all exceptions to be thrown and then we have an IErrorHandler interface which each platform implements. We then call this to attempt to show a dialog. This has proved to be unreliable and the dialog does not always display. Something along the lines of: (note - here ResolveViewModel will always return true or throw)
public override bool TryLoad(Type viewModelType, IMvxBundle parameterValues, IMvxBundle savedState, out IMvxViewModel viewModel)
{
try
{
return ResolveViewModel(viewModelType, parameterValues, savedState, out viewModel);
}
catch (Exception exception)
{
_errorHandler.HandleViewModelConstructionException(viewModelType, exception);
viewModel = null;
return false;
}
}
What we would ideally like to do is intercept any failure to construct a view model and then re-request an ErrorViewModel. We've tried to do this 2 ways:
1)
We've tried defining a custom IMvxViewDispatcher for each platform and we're trying to intercept failures as below but if an exception in the constructor is thrown we never get back this far:
public class TouchDispatcher : MvxTouchUIThreadDispatcher, IMvxViewDispatcher
{
private readonly IMvxTouchViewPresenter _presenter;
public TouchDispatcher(IMvxTouchViewPresenter presenter)
{
_presenter = presenter;
}
public bool ShowViewModel(MvxViewModelRequest request)
{
Action action = () =>
{
_presenter.Show(request);
};
try
{
bool success = RequestMainThreadAction(action);
return !success ? HandleError() : success;
}
catch (Exception)
{
return HandleError();
}
}
// Other bits
}
2)
We thought we might have more success at the presenter level. We modified our ViewPresenter for each platform and we have overridden void Show(MvxViewModelRequest request). This has not proved to be successful either as exceptions don't propagate back this far.
This leaves me thinking that maybe we are better attempting this at the IMvxViewModelLocator level again.
Has anyone found a way to reliably intercept failures to construct view models and then ideally re-request a different view model / present some dialog to the user?
It seems you've identified that the core of the problem is when: "view model constructors throw uncaught exceptions."
This is going to be slightly problematic as the ViewModel's are generally constructed during View lifecycle overrides like ViewDidLoad, OnCreate or NavigatedTo - which is generally after the Presenter has finished requesting presentation.
As you've already found an easy place to identify when ViewModel construction has failed is in a custom IMvxViewModelLocator - others likeIMvxViewModelLoader are also possible. This is probably the easiest place to catch the error and to trigger the error handling - you can then get hold of the IMvxViewDispatcher (or presenter) there in order to change the display. However, you will still need to make sure your Views can handle null created ViewModels - as the ViewDidLoad, etc calls will still need to complete.

Lock statement vs AutoResetEvent for LocalDb Thread Synchronization

I am writing an application that utilizes Windows Phone's LocalDB feature. I realized that I need to ensure that only one thread is performing operations on a given database, so I have created an AutoResetEvent object to coordinate the various threads vying for access to the database. My code goes pretty much like this:
class SomeClass
{
AutoResetEvent DatabaseLock = new AutoResetEvent(true);
public async void AddData(Person person)
{
await Task.Run(() =>
{
MyDataContext db = null;
try
{
this.DatabaseLock.WaitOne();
db = MyDataContext.GetInstance();
db.People.InsertOnSubmit(person);
db.SubmitChanges();
}
finally
{
if (db == null)
db.Dispose();
this.DatabaseLock.Set();
}
}
}
}
Obviously that's not the real code, but it's the same general pattern. Anyway, I decided to use the AutoResetEvent object here, as I have seen suggested online in multiple locations. However, I would be inclined to use a lock {...} statement instead.
Is there any reason to use AutoResetEvent? I feel like it's slow compared to locking an object.
You should use an AutoResetEvent when you need to signal to another thread.
In this case, you're just locking a resource, so the lock statement would be a better choice.

How can I wait on tasks without throwing TaskCanceledExceptions?

I have a method that creates some Tasks, and then waits on them with WaitAll before returning. The problem is, if those tasks got canceled, then WaitAll throws an AggregateException containing lots of TaskCanceledExceptions.
That means that WaitAll will throw exceptions in two different circumstances:
Exceptions that indicate a genuine error. These mean that there was a condition we didn't know how to handle; they need to propagate as unhandled exceptions, until they eventually terminate the process.
Exceptions that indicate that the user clicked a Cancel button. These mean that the task was canceled and cleaned up, and the program should continue running normally.
The latter fits squarely into the definition of a vexing exception: it's an exception thrown in a completely non-exceptional circumstance, so I have to catch it in order to resume normal control flow. Fortunately it's easy to catch, right? Just add catch (AggregateException) and -- oh wait, that's the same type that gets thrown when there's a fatal error.
I do need to wait for the tasks to finish running before I return (I need to know that they're no longer using their database connections, file handles, or anything else), so I do need the WaitAll or something similar. And if any of the tasks faulted, I do want those exceptions to propagate as unhandled exceptions. I just don't want exceptions for cancel.
How can I prevent WaitAll from throwing exceptions for canceled tasks?
The AggregateException provides a Handle method that can be used for these situations. If for example you want to ignore TaskCanceledException you can do:
var all = new AggregateException(
new NullReferenceException(),
new TaskCanceledException(),
new TaskCanceledException(),
new InvalidOperationException(),
new TaskCanceledException());
try
{
throw all;
}
catch (AggregateException errors)
{
errors.Handle(e => e is TaskCanceledException);
}
If all the exceptions are of type TaskCanceledException, the Handle method will not throw any exception; otherwise a new AggregateException containing only the unhandled exceptions will be thrown.
Based on João Angelo's suggestion, here goes a Task class extension
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading;
using System.Threading.Tasks;
namespace MySharedLibrary.Extensions
{
public static class TaskExtensions
{
// This code is based João Angelo's stackoverflow suggestion https://stackoverflow.com/a/8681687/378115
// Use this when a CancellationTokenSource is used
public static void SafeWait(this Task TargetTask, CancellationTokenSource TargetTaskCancellationTokenSource)
{
if (TargetTaskCancellationTokenSource.IsCancellationRequested == false)
{
TargetTaskCancellationTokenSource.Cancel();
}
SafeWait(TargetTask);
}
// Use this when no CancellationTokenSource is used
public static void SafeWait(this Task TargetTask)
{
try
{
if (TargetTask.IsCanceled == false)
{
TargetTask.Wait();
}
}
catch (AggregateException errors)
{
errors.Handle(e => e is TaskCanceledException);
}
}
}
}