Crash reporting and C++ Coroutines? - c++20

I use a crash reporting feature that allows the user to submit a crash report if the application crashed with an uncaught exception.
After adopting C++20 coroutines entered the application.
If there is an unexpected exception thrown in a coroutine the exception is caught before it is rethrown.
This causes crashreports to not show the stacktrace needed to figure out what happened, but only the stacktrace to the coroutine that rethrew the exception. This basically makes any crash reporting useless.
As far as I could find there is no way to prevent the catching of any exceptions by the coroutine because it is a required part of the design.
Is there a way to improve this I cant see?
I am curious because I found nobody else complaining yet. :->
Edit: To clarify the app is running on Windows, I mean the stacktrace of a minidump that is created at the point of the unhandled exception using: SetUnhandledExceptionFilter + MiniDumpWriteDump

C++ does not have standard stack tracing yet, so there is no nice builtin way to do this.
However, there are ways, which rely on keeping information in the promise objects.
Clang has documentation for some common debugging methods for coroutines.

The best solution we have found is as follows (Windows specific!):
Until now we used SetUnhandledExceptionFilter at the start of the app to set an exceptionfilter function that writes a minidump.
Instead we now use _set_se_translator.
If we want the program to just crash (f.e. if windows is set to write dumps) we set a function which calls std::abort.
If we want to handle it interactively we set a function which asks the user whether to send a minidump, the dump is written as before at this point.
Both cases provide the full callstack in the dump.
The only downside remaining is we cant let the program crash for "normal" exceptions to dump, this was possible before. But the "most important" exceptions (f.e. access violations) work.

Related

when will SetUnhandledExceptionFilter not work? e.g. stack corruptions?

I would like to have my code create a dump for unhandled exceptions.
I'd thought of using the SetUnhandledExceptionFilter. But what are the cases when SetUnhandledExceptionFilter may not work as expected. For example what about stack corruption issues when, for instance, a buffer overrun occurs on stack?
what will happen in this case? are there any additional solutions which will always work?
I've been using SetUnhandledExceptionFilter for quite a while and have not noticed any crashes/problems that are not trapped correctly. And, if an exception is not handled somewhere in the code, it should get handled by the filter. From MSDN regarding the filter...
After calling this function, if an exception occurs in a process that
is not being debugged, and the exception makes it to the unhandled
exception filter, that filter will call the exception filter function
specified by the lpTopLevelExceptionFilter parameter.
There is no mention that the above applies to only certain types of exceptions.
I don't use the filter to create a dump file because the application uses the Microsoft WER system to report crashes. Rather, the filter is used to provide an opportunity to collect two additional files to attach to the crash report (and dump file) that Microsoft will collect.
Here's an example of Microsoft's crash report dashboard for the application with module names redacted.
You'll see that there's a wide range of crash types collected, including, stack buffer overrun.
Also make sure no other code calls the SetUnhandledExceptionFilter() after you set it to your handler.
I had a similar issue and in my case it was caused by another linked library (ImageMagick) which called SetUnhandledExceptionFilter() from its Magick::InitializeMagick() which was called just in some situations in our application. Then it replaced our handler with ImageMagick's handler.
I found it by setting a breakpoint on SetUnhandledExceptionFilter() in gdb and checked the backtrace.

Disabling .NET handling of native exceptions

I have a C# application that calls a mixed mode C++ dll. I enabled dump generation via HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\Windows Error Reporting\LocalDumps.
When the dll accesses invalid memory, the runtime converts the win32 exception to a managed System.AccessViolationException, and unwinds the stack before generating the dump, destroying the native stack information. I could catch the exception myself before .net gets at it and generate the dump manually, but running code on an already corrupt program could hang it, according to the msdn. So, how can I disable SEH translation?
You cannot disable that. The CLR will not unwind the stack unless you catch the exception. Make sure you don't. This needs to go through an AppDomain.UnhandledException event handler. The essential function you need is Marshal.GetExceptionPointers(), that's the one that will pinpoint the exception when you open the minidump.
You'll find resources in my answer in this MSDN forum thread and this pinvoke.net snippet, should be enough to cobble your own together.

Why not catch general Exceptions

My VS just told me;
Warning 2 CA1031 : Microsoft.Design : Modify 'Program.Main(string[])' to catch a more specific exception than 'Exception' or rethrow the exception.
Why should I do that? If I do so, and don't catch all exceptions to handle them, my program crashes with the all-popular report-screen. I don't want my users to get such error-crap!
Why should I not catch all exceptions at once to display a nice warning to the user saying: "Something went wrong, don't care about it, I will handle it, just be patient"?
Edit: Just saw I have a dupe here, sorry for that Dupe
Edit2: To clarify things; I do exit the program after any exception has been catched! I just don't want my user to see that "report to microsoft" dialog that show up when an unhandled exception is raised in a console-application!
Swallowing exceptions is a dangerous practice because:
It can cause the user to think something succeeded when it actually failed.
It can put your application into states that you didn't plan for.
It complicates debugging, since it's much harder to find out where the failure happened when you're dealing with bizarre/broken behavior instead of a stack trace.
As you can probably imagine, some of these outcomes can be extremely catastrophic, so doing this right is an important habbit.
Best Practice
First off, code defensively so that exceptions don't occur any more than necessary. They're computationally expensive.
Handle the expected exceptions at a granular level (for example: FileNotFoundException) when possible.
For unexpected exceptions, you can do one of two things:
Let them bubble up normally and cause a crash
Catch them and fail gracefully
Fail Gracefully?
Let's say you're working in ASP.Net and you don't want to show the yellow screen of death to your users, but you also don't want problems to be hidden from the dev team.
In our applications, we usually catch unhandled exceptions in global.asax and then do logging and send out notification emails. We also show a more friendly error page, which can be configured in web.config using the customErrors tag.
That's our last line of defense, and if we end up getting an email we jump on it right away.
That type of pattern is not the same as just swallowing exceptions, where you have an empty Catch block that only exists to "pretend" that the exception did not occur.
Other Notes
In VS2010, there's something called intellitrace coming that will allow you to actually email the application state back home and step through code, examine variable values at the time of the exception, and so on. That's going to be extremely useful.
Because programs that swallow (catch) exceptions indiscriminately, (and then continue), cannot be relied upon to do what it is they are expected to do. This is because you have no idea what kind of exception was "ignored". What if there was an overflow or memory access error that causes the wrong amount to be debited from a financial account? What if it steers the ship into the iceberg instead of away from it ? Unexpected failures should always cause the application to terminate. That forces the development process to identify and correct the exceptions it finds, (crashes during demos are a wonderful motivator), and, in production, allows appropriately designed backup systems to react when the software experiences an "unexpected" inability to do what it was designed to do.
EDIT: To clarify distinctions between UI components, and service or middleware componentrs.
In Service or Middleware components, where there is no user interacting with the code component from within the same process space that the code is running in, the component needs to "pass On" the exception to whatever client component imnitiated the call it is currently processing. No matter the exception, it should make every possible attempt to do this. It is still the case, however, tjhat in cases where an unexpected, or unanticipated exception occurs, the component should finally terminate the process it is running in. For anticipated or expected exceptions, a velopment analysis should be done to determine whether or not, for that specific exception, the component and it's host process can continue to operate (handling future requests), or whether it should be terminated.
You should handle the exact exceptions you are capable of handling and let all others bubble up. If it displays a message to the user that means you don't quite know what you can handle.
Having worked on equipment used by emergency responders, I would rather the user see an ugly error message than to accidently swallow an exception that misleads the user into believing everything is "ok". Depending on your application, the consequence could be anything from nothing to a lost sale to a catastrophic loss of life.
If a person were going to catch all exception, show a better error dialog, and then quit the application, that's ok.. but if they are going to continue running after swallowing an unknown exception, I would fire a person for that. It's not ok. Ever.
Good coding is about practices that assume humans make mistakes. Assuming all "critical" exceptions have been caught and handled is a bad idea.
Simple answer: you are supposed to fix your bug. Find the place that throws the exception and unless it is beyond your control - fix it.
Also catching (without rethrowing) all kinds of exception violates exception neutrality. In general you do not want to do this (although catching exceptions in main does look like special case)
Since your warning message shows that this is in Main(), I'll assume that in lower levels, you do catch only more specific Exceptions.
For Main(), I'd consider two cases:
Your own (debugging) build, where you want all the exception information you can get: Do not catch any Exceptions here, so the debugger breaks and you have your call stack,
Your public releases, where you want the application to behave normally: Catch Exception and display a nice message. This is always better (for the average user) than the 'send report' window.
To do this nicely, just check if DEBUG is defined (and define it, if VS doesn't do this automatically):
#if DEBUG
yadda(); // Check only specific Exception types here
#else
try
{
yadda();
}
catch (Exception e)
{
ShowMessage(e); // Show friendly message to user
}
#endif
I'd disable the warning about catching general Exceptions, but only for your Main() function, catching Exception in any other method is unwise, as other posters have said already.
There is a way to suppress certain messages from code analysis. I've used this for this exact reason (catching the general exception for logging purposes) and it's been pretty handy. When you add this attribute, it shows you've at least acknowledged that you are breaking the rule for a specific reason. You also still get your warning for catch blocks that are incorrect (catching the general exception for purposes other than logging).
MSDN SuppressMessageAttribute
I am all for catching specific known exceptions and handling state...but I use general catch exceptions to quickly localize problems and pass errors up to calling methods which handle state just fine. During development as those are caught, they have a place right next to the general exception and are handled once in release.
I believe one should attempt to remove these once the code goes into production, but to constantly be nagged during the initial code creation is a bit much.
Hence turn off (uncheck) the warning by the project settings as found in Microsoft.CodeQuality.Analyzers. That is found in the project settings under Code Analysis:
All answers are good here. But I would mention one more option.
The intention of author to show some fancy message is understandable.
Also, default Windows error message is really ugly. Besides, if application is not submitted to "Windows Excellence Program" the developer will not receive information about this problem. So what is the point to use default runtime handler if it does not help?
The thing here is that default exception handler of CLR host ( https://learn.microsoft.com/en-us/previous-versions/visualstudio/visual-studio-2008/9x0wh2z3(v=vs.90)?redirectedfrom=MSDN ) works in a very safe way. The purpose of it is clear: log the error, send it to developer, set the return code of your process and kill it. The general way of how to change that is to write your own host. In this case you can provide your own way of handling exceptions.
Still, there is an easy solution which satisfies CA1031 and still most of your needs.
When catching the exception, you can handle it your own way (log, show the message etc) and at the end you can set the process result code and do the exit (using the mix of Thread.Abort and "Exit" methods, for example). Still, at the end of your catch block you can just put "throw;" (which will never be called because of ThreadAbortedException, but will satisfy the rule). Still there are some cases, like StackOverflowException, which can't be handled like that and you will see that default message box, for fixing which you need to fallback to custom CLR host option.
Additionally, just for your information, you application can run several threads (besides that one which execute Main method). To receive exceptions from all of them you can use AppDomain.UnhandledException. This event does not allow you to "mark" the exception as handled, still you can freeze the thread using Thread.Join() and then do the job (log, msgbox, exit) using another (one more) thread.
I understand all this looks a little tricky and may be not right, but we have to deal with the implementation of AppDomain.UnhandledException, ThreadAbortException, CorruptedState exceptions and default CLR host. All of this eventually does not leave us much of choice.
When you catch general exceptions, you get the side effect of potentially hiding run-time problems from the user which, in turn, can complicate debugging. Also, by catching general exception, you're ignoring a problem (which you're probably throwing elsewhere).
You can set up your try catch to catch multiple different behavior types and handle the exception based on the type. For most methods and properties in the framework, you can also see what exceptions they are capable of throwing. So unless you are catching an exception from an extremely small block of code, you should probably catch specific exceptions.
In VS you can setup a custom error page to show your users when something goes wrong instead of catching it in a try-catch. I'm assuming since you're using VS that you're using ASP .NET. If so add this tag to your Web.Config under the System.Web tag:
<customErrors mode="RemoteOnly" defaultRedirect="~/CustomErrorPage.aspx" redirectMode="ResponseRewrite" />
You can also catch all uncaught exceptions in the Global.asax file (if you don't have it already: Right-click on web project, select Add Item, and search for it). There are a bunch of application wide event handlers in that file like "Application_Error" that catches every exception that isn't caught within your application so you don't have to use Try-Catch all the time. This is good to use to send yourself an email if an exception occurs and possibly redirect them to your homepage or something if you don't want to use the customErrors tag above.
But ultimately you don't want to wrap your entire application in a try-catch nor do you want to catch a general Exception. Try-catches generally slow down your application and a lot of times if you catch every general exception than it could be possible that you wouldn't know a bug exists until months or years later because the try-catch caused you to overlook it.

Exceptions over remote methods

What are the best practices for exceptions over remote methods?
I'm sure that you need to handle all exceptions at the level of a remote method implementation, because you need to log it on the server side. But what should you do afterwards?
Should you wrap the exception in a RemoteException (java) and throw it to the client? This would mean that the client would have to import all exceptions that could be thrown. Would it be better to throw a new custom exception with fewer details? Because the client won't need to know all the details of what went wrong. What should you log on the client? I've even heard of using return codes(for efficiency maybe?) to tell the caller about what happened.
The important thing to keep in mind, is that the client must be informed of what went wrong. A generic answer of "Something failed" or no notification at all is unacceptable. And what about runtime (unchecked) exceptions?
It seems like you want to be able to differentiate if the failure was due to a system failure (e.g. a service or machine is down) or a business logic failure (e.g. the user does not exist).
I'd recommend wrapping all system exceptions from the RMI call with your own custom exception. You can still maintain the information in the exception by passing it to your custom exception as the cause (this is possible in Java, not sure about other languages). That way client only need to know how to handle the one exception in the cause of system failure. Whether this custom exception is checked or runtime is up for debate (probably depends on your project standards). I would definitely log this type of failure.
Business type failures can be represented as either a separate exception or some type of default (or null) response object. I would attempt to recover (i.e. take some alternative action) from this type of failure and log only if the recovery fails.
In past projects we'd catch all service layer (tier) exceptions at the very top of the layer, passing the application specific error codes/information to the UI via DTO's/VO's. It's a simple approach in that there's an established pattern of all error handling happening in the same place for each service instead of scattered about the service and UI layers.
Then all the UI has to do is inspect the DTO/VO for a flag (hasError?) and display the error message(s), it doesn't have to know nor care what the actual exception was.
I would always log the exception within my application (at the server side as defined in your question).
I would then throw an exception, to be caught by the client. If the caller could take corrective action to prevent the exception then I would ensure that the exception contained this information (e.g. DateTime argName must not be in the past). If the error was caused by some outage of a third party system then I might pass this information up the call stack to the caller.
If, however, the exception was essentially caused by a bug in my system then I would structure my exception handling such that a non-informative exception message (e.g. General failure) was used.
Here's what I did. Every Remote Method implementation catches all Exceptions on the server side and logs them. Then they are wrapped in a Custom Exception, which will contain a description of the problem. This description must be useful to the client, so it won't contain all the details of the caught Exception, because the client doesn't need them. They have already been logged on the server side. Now, on the client, these Exceptions can be handled how the user wishes.
Why I chose using Exceptions and not return codes is because of one very important drawback of return codes: you can't throw them to higher levels without some effort. This means you have to check for an error right after the call and handle it there. But this may not be what I want.

How to implement top level exception handling?

I recently had to develop an additional module for an existing service developed by a colleague. He had placed a try/catch block in the main working function for catching all unhadled exceptions that bubbled up to this level, logging them together with stack trace info etc:
try
{
// do main work
}
catch(Exception ex)
{
// log exception info
}
While this makes the program very stable (as in 'unlikely to crash'), I hate it because when I am testing my code, I do not see the exceptions caused by it. Of course I can look at the exception log and see if there are new entries, but I very much prefer the direct feedback of getting the exception the moment it is thrown (with the cursor on the right line in the code, please).
I removed this top level try/catch at least while i was still coding and testing. But now my task is finished and I have to decide wether to put it back in for the release, or not.
I think that I should do it, because it makes the service more stable, and the whole point of it is that it runs in the background without needing any supervision. On the other hand I have read that one should only call specific exceptions (as in IoException), not generically Exception.
What is your advice on this issue?
By the way, the project is written in C#, but I am also interested in answers for non .NET languages.
Put it back.
The exception should be only relevant while testing. Otherwise it doesn't make sense pop it to the user.
Logging is fine.
You may additionally use the DEBUG symbol defined by Visual Studio for debug builds as a flag.
...
} catch( Exception e ) {
#if DEBUG
throw;
#else
log as usual
#endif
}
So next time a modification is needed the debug flag should be set to true and the exception will pop up.
In any Java application, you just about always want to define an exception handler for uncaught exceptions with something like this:
Thread.setDefaultUncaughtExceptionHandler( ... );
where the object that will catch these uncaught exceptions will at least log the failure so you have a chance to know about it. Otherwise, there is no guarantee that you'll even be notified that a Thread took an Exception -- not unless it's your main Thread.
In addition to doing this, most of my threads have a try/catch where I'll catch RunnableException (but not Error) and log it ... some Threads will die when this happens, others will log and ignore, others will display a complaint to the user and let the user decide, depending on the needs of the Application. This try/catch is at the very root of the Thread, in the Runnable.run() method or equivalent. Since the try/catch is at the root of the Thread, there's no need to sometimes disable this catch.
When I write in C#, I code similarly. But this all depends on the need of the application. Is the Exception one that will corrupt data? Well then, don't catch and ignore it. ALWAYS log it, but then Let the application die. Most Exceptions are not of this sort, however.
Ideally you want to handle an exception as close to where it occured as possible but that doesn't mean a global exception handler is a bad idea. Especially for a service which must remain running at all costs. I would continue what you have been doing. Disable it while debugging but leave it in place for production.
Keep in mind it should be used as a safety net. Still try to catch all exceptions before they elevate that far.
The urge to catch all exceptions and make the program 'stable' is very strong and the idea seems very enticing indeed to everyone. The problem as you point out is that this is just a ruse and the program may very well be buggy and worse, w/o indications of failures. No one monitors the logs regularly.
My advice would be to try and convince the other developer to do extensive testing and then deploy it in production w/o the outer catch.
If you want to see the exception when it occurs, in Visual Studio you can go to the DEBUG menu, select EXCEPTIONS and you can tell the debugger to Break as soon as an Exception is thrown. You even get to pick what type of Exceptions. :)