My question is really simple. Please take a look at screenshot:
How could that happen? I explicitly put call to Async.RunSyncronously into try ... with.
Try this:
let withTimeout (timeOut: option<int>) (operation: Async<'x>) : Async<option<'x>> =
match timeOut with
| None -> async {
let! result = operation
return Some result
}
| Some timeOut -> async {
let! child = Async.StartChild (operation, timeOut)
try
let! result = child
return Some result
with :? System.TimeoutException ->
return None
}
You should not use Async.RunSynchronously within async blocks, because doing so makes for suboptimal use of native threads and can lead to stack overflows. Async.RunSynchronously is for running Async computations from outside of such computations. Within an async block you can use plain let! and do! or, for example, Async.StartChild to run Async computations. This makes for more effective use of native threads and does not suffer from similar issues with potential stack overflows.
try/with in F# async workflows do not map directly to CLR protected blocks - instead if exception is raised in user code, library code will catch it and re-route to the nearest error continuation (which can be i.e with block, finally block, custom error continuation supplied in Async.StartWithContinuations etc...). This has a consequence that debugger might report about unhandled exceptions in user code that can be handled and processed later.
Snippet below reports the similar error in debugger but nevertheless execution is finished successfully
let error (): int = raise (System.TimeoutException())
let run() = async {
try
let result = error()
return result
with
:? System.TimeoutException -> return -1
}
let r = Async.RunSynchronously (run())
printfn "%d" r
Related
Given the F# task computation expression I can write:-
task {
try
let! accessToken = getAccessTokenAsync a b
try
let! resource = getResourceAsync accessToken uri
// do stuff
with
| ex -> printfn "Failed to get API resource. %s" ex.Message
with
| ex -> printfn "Failed to get access token. %s" ex.Message
return ()
}
but what I want to do is have non-nested exception handling around the two getBlahAsync function calls. This can be done in C# quite easily in an async method with multiple awaits.
How to do so in an F# computation expression? If I try it in the simple and obvious way, accessToken from the first try..with doesn't flow into the second try..with.
(The trouble with nesting is that the // do stuff section could grow a bit, pushing the outer with further and further away from its try.)
How to do it in C#:-
static async Task MainAsync()
{
String accessToken = null;
try
{
accessToken = await GetAccessTokenAsync("e", "p");
}
catch (Exception ex)
{
Console.Error.WriteLine("Failed to get access token. " + ex.Message);
return;
}
String resource = null;
try
{
resource = await GetResourceAsync(accessToken);
}
catch (Exception ex)
{
Console.Error.WriteLine("Failed to get API resource. " + ex.Message);
return;
}
// do stuff
}
The main problem with translating the C# code is that F# does not let you use return to jump out of the function body early. You can avoid nesting exceptions in various ways, but you will not be able to return early. This can be implemented as another computatione expression, but that's more of a curiosity than something you'd actually want to use here.
My recommendation would be to just split the function into one that gets all the resources and handles exceptions and another one that does the stuff. That does not eliminate nesting, but it will make the code fairly readable.
let doStuff accessToken resource = task {
// do stuff
}
let getResourcesAndDoStuff a b uri = task {
try
let! accessToken = getAccessTokenAsync a b
try
let! resource = getResourceAsync accessToken uri
return! doStuff accessToken resource
with ex ->
printfn "Failed to get API resource. %s" ex.Message
with ex ->
printfn "Failed to get access token. %s" ex.Message
}
As an aside, do you have some particular reason for using task rather than the normal built-in F# async workflow? It is not necessarily a problem, but async composes better and supports cancellation, so it is often a sensible default choice.
After your edit, I see that what you actually want is "early return" - an ability to "interrupt" the flow of execution before reaching the end point. This is generally not possible in F# (though some computation builders might offer specialized facilities for that), because F# is fundamentally expression-based, not statement-based.
A lack of early return is a good thing, because it forces you to think through carefully what your program is supposed to do, as opposed to just bailing. But that is a philosophical discussion for another time.
However, there are other ways of achieving a similar effect. In this specific case, I would put the two operations, together with their exception handling, into separate functions, then chain those functions together:
task {
let token = task {
try
let! t = getAccessTokenAsync a b
return Some t
with
| ex -> printfn "Failed to get access token. %s" ex.Message
return None
}
let resouce t = task {
try
let! r = getResourceAsync accessToken uri
// do stuff
with
| ex -> printfn "Failed to get API resource. %s" ex.Message
}
let! t = token
match t with
| None -> return ()
| Some token -> do! resource token
}
If you find yourself facing similar issues regularly, you may want to invest in a few helper functions that wrap exception handling and Option chaining:
// Applies given Task-returning function to the given Option value,
// if the Option value is None, returns None again.
// This is essentially Option.map wrapped in a task.
let (<*>) f x = task {
match x with
| None -> return None
| Some r -> let! r' = f r
return Some r'
}
// Executes given Option-returning task, returns None if an exception was thrown.
let try'' errMsg f = task {
try return! f
with ex ->
printfn "%s %s" errMsg ex.Message
return None
}
// Executes given task, returns its result wrapped in Some,
// or returns None if an exception was thrown.
let try' errMsg f = try'' errMsg <| task { let! r = f
return Some r }
task {
let! token = getAccessTokenAsync a b |> try' "Failed to get access token."
let! resource = getResourceAsync uri <*> token |> try'' "Failed to get API resource."
do! doStuff <*> resource
}
This illustrates the preferred F# way of dealing with exceptions: avoid them, never throw them, instead return error types (the example above uses Option<_>, but also see e.g. Result<_,_>), and if you must interact with library code that does throw exceptions, put them inside wrappers that convert exceptions to error types.
After I installed Xcode 7 beta and convert my swift code to Swift 2, I got some issue with the code that I can't figure out. I know Swift 2 is new so I search and figure out since there is nothing about it, I should write a question.
Here is the error:
Call can throw, but it is not marked with 'try' and the error is not
handled
Code:
func deleteAccountDetail(){
let entityDescription = NSEntityDescription.entityForName("AccountDetail", inManagedObjectContext: Context!)
let request = NSFetchRequest()
request.entity = entityDescription
//The Line Below is where i expect the error
let fetchedEntities = self.Context!.executeFetchRequest(request) as! [AccountDetail]
for entity in fetchedEntities {
self.Context!.deleteObject(entity)
}
do {
try self.Context!.save()
} catch _ {
}
}
Snapshot:
You have to catch the error just as you're already doing for your save() call and since you're handling multiple errors here, you can try multiple calls sequentially in a single do-catch block, like so:
func deleteAccountDetail() {
let entityDescription = NSEntityDescription.entityForName("AccountDetail", inManagedObjectContext: Context!)
let request = NSFetchRequest()
request.entity = entityDescription
do {
let fetchedEntities = try self.Context!.executeFetchRequest(request) as! [AccountDetail]
for entity in fetchedEntities {
self.Context!.deleteObject(entity)
}
try self.Context!.save()
} catch {
print(error)
}
}
Or as #bames53 pointed out in the comments below, it is often better practice not to catch the error where it was thrown. You can mark the method as throws then try to call the method. For example:
func deleteAccountDetail() throws {
let entityDescription = NSEntityDescription.entityForName("AccountDetail", inManagedObjectContext: Context!)
let request = NSFetchRequest()
request.entity = entityDescription
let fetchedEntities = try Context.executeFetchRequest(request) as! [AccountDetail]
for entity in fetchedEntities {
self.Context!.deleteObject(entity)
}
try self.Context!.save()
}
When calling a function that is declared with throws in Swift, you must annotate the function call site with try or try!. For example, given a throwing function:
func willOnlyThrowIfTrue(value: Bool) throws {
if value { throw someError }
}
this function can be called like:
func foo(value: Bool) throws {
try willOnlyThrowIfTrue(value)
}
Here we annotate the call with try, which calls out to the reader that this function may throw an exception, and any following lines of code might not be executed. We also have to annotate this function with throws, because this function could throw an exception (i.e., when willOnlyThrowIfTrue() throws, then foo will automatically rethrow the exception upwards.
If you want to call a function that is declared as possibly throwing, but which you know will not throw in your case because you're giving it correct input, you can use try!.
func bar() {
try! willOnlyThrowIfTrue(false)
}
This way, when you guarantee that code won't throw, you don't have to put in extra boilerplate code to disable exception propagation.
try! is enforced at runtime: if you use try! and the function does end up throwing, then your program's execution will be terminated with a runtime error.
Most exception handling code should look like the above: either you simply propagate exceptions upward when they occur, or you set up conditions such that otherwise possible exceptions are ruled out. Any clean up of other resources in your code should occur via object destruction (i.e. deinit()), or sometimes via defered code.
func baz(value: Bool) throws {
var filePath = NSBundle.mainBundle().pathForResource("theFile", ofType:"txt")
var data = NSData(contentsOfFile:filePath)
try willOnlyThrowIfTrue(value)
// data and filePath automatically cleaned up, even when an exception occurs.
}
If for whatever reason you have clean up code that needs to run but isn't in a deinit() function, you can use defer.
func qux(value: Bool) throws {
defer {
print("this code runs when the function exits, even when it exits by an exception")
}
try willOnlyThrowIfTrue(value)
}
Most code that deals with exceptions simply has them propagate upward to callers, doing cleanup on the way via deinit() or defer. This is because most code doesn't know what to do with errors; it knows what went wrong, but it doesn't have enough information about what some higher level code is trying to do in order to know what to do about the error. It doesn't know if presenting a dialog to the user is appropriate, or if it should retry, or if something else is appropriate.
Higher level code, however, should know exactly what to do in the event of any error. So exceptions allow specific errors to bubble up from where they initially occur to the where they can be handled.
Handling exceptions is done via catch statements.
func quux(value: Bool) {
do {
try willOnlyThrowIfTrue(value)
} catch {
// handle error
}
}
You can have multiple catch statements, each catching a different kind of exception.
do {
try someFunctionThatThowsDifferentExceptions()
} catch MyErrorType.errorA {
// handle errorA
} catch MyErrorType.errorB {
// handle errorB
} catch {
// handle other errors
}
For more details on best practices with exceptions, see http://exceptionsafecode.com/. It's specifically aimed at C++, but after examining the Swift exception model, I believe the basics apply to Swift as well.
For details on the Swift syntax and error handling model, see the book The Swift Programming Language (Swift 2 Prerelease).
After perusing through forums and Swift documentation (not completely, I admit), it appears that instead of try-catch mechanisms, in Swift we are encouraged to write code that is more safe from exceptions. In light of that, I have a question about a sample API, and would like to learn how to more safely handle this situation:
For example, I can create the following class using the NSDecimalNumberHandler:
class MathWhiz {
init() {
let defaultBehavior: NSDecimalNumberHandler =
NSDecimalNumberHandler.defaultDecimalNumberHandler()
}
func add(op1: String, op2: String) ->NSDecimalNumber {
return NSDecimalNumber.decimalNumberWithString(op1).decimalNumberByAdding(NSDecimalNumber.decimalNumberWithString(op2))
}
}
If I use the following, I get a number:
let brain = MathWhiz()
brain.add("1", op2: "1e127")
However, if I cause an overflow exception,:
brain.add("1", op2: "1e128")
I will crash the program as expected.
So, my question is, the API raises exceptions, but I don't handle them here. There are other posts out there of people pointing out that Swift does not have exception handling, but this question is seeking a nice way to handle this problem in the way that the language creators were thinking it should be done.
Is there a recommended way to handle this without having to write my own code to check for overflow, underflow, loss of precision, ect...? I am wanting the NSDecimalNumberHandler to do that for me.
If you are designing a function (or method) in Swift, you have at least 3 choices for dealing with errors:
Choice 1: Return an Optional Type
If your function might fail, and this happens on a regular basis, then consider returning an optional type variable. For example, in your case, your method add could return an NSDecimalNumber? instead of a plain NSDecimalNumber. In that case, your method would check for everything that could go wrong, and return nil in those situations. Overflow and underflow would return nil, and all other cases would return an NSDecimalNumber. The callers would have to check for and unwrap the optional NSDecimalNumber like this:
let brain = MathWhiz()
if let sum = brain.add("1", op2: "1e127") {
println("the result was \(sum)")
} else
println("something went wrong with MathWhiz add")
}
Choice 2: Return an Enumerated Type
If you want to return more information about the thing that went wrong, you could create an enumerated type with a value for each error and one for success that embeds the answer. For example, you could do:
enum MathWhizResult {
case Overflow
case Underflow
case Success(NSDecimalNumber)
}
Then add would be defined to return MathWhizResult:
func add(op1: String, op2: String) -> MathWhizResult
In the case of an error, add would return .Overflow or .Underflow. In the case of success, add would return Success(result). The caller would have to check the enumeration and unpack the result. A switch could be used for this:
switch (brain.add("1", op2: "1e128")) {
case .Overflow
println("darn, it overflowed")
case .Underflow
println("underflow condition happened")
case .Success(let answer)
println("the result was \(answer)"
}
Choice 3: Choose not to handle errors explicitly
Unpacking the result in the first two choices might be too much overhead for an error that is very rare. You could chose to just return a result, and let the caller deal with the possibility of an underflow or overflow condition. In that case, they would have to check for these conditions themselves before calling add. The benefit is, it they know that their program will never cause an underflow or overflow (because they are dealing with single digit numbers for instance), they are not burdened with unpacking the result.
I created a small app to demonstrate how you could do this with NSDecimalNumbers. I created a Single View Application in Xcode. In the ViewController in the StoryBoard I added 3 TextFields (one each for operand 1, operand 2, and the result) and a Button that I labelled +.
ViewController.swift
import UIKit
class ViewController: UIViewController {
#IBOutlet var operand1 : UITextField!
#IBOutlet var operand2 : UITextField!
#IBOutlet var result : UITextField!
var brain = MathWhiz()
override func viewDidLoad() {
super.viewDidLoad()
// Do any additional setup after loading the view, typically from a nib.
}
override func didReceiveMemoryWarning() {
super.didReceiveMemoryWarning()
// Dispose of any resources that can be recreated.
}
#IBAction func addButton(sender : UIButton) {
var op1 = operand1.text
var op2 = operand2.text
// Perform the add with the contents of the operand fields.
// Print the answer, or "No Result" if add returns nil.
if let answer = brain.add(op1, op2: op2)?.description {
result.text = answer
} else {
result.text = "No Result"
}
}
}
MathWhiz.swift
import UIKit
// Declare that we implement NSDecimalNumberBehaviors so that we can handle
// exceptions without them being raised.
class MathWhiz: NSDecimalNumberBehaviors {
var badException = false
// Required function of NSDecimalNumberBehaviors protocol
func roundingMode() -> NSRoundingMode {
return .RoundPlain
}
// Required function of NSDecimalNumberBehaviors protocol
func scale() -> CShort {
return CShort(NSDecimalNoScale)
}
// Required function of NSDecimalNumberBehaviors protocol
// Here we process the exceptions
func exceptionDuringOperation(operation: Selector, error: NSCalculationError, leftOperand: NSDecimalNumber, rightOperand: NSDecimalNumber) -> NSDecimalNumber? {
var errorstr = ""
switch(error) {
case .NoError:
errorstr = "NoError"
case .LossOfPrecision:
errorstr = "LossOfPrecision"
case .Underflow:
errorstr = "Underflow"
badException = true
case .Overflow:
errorstr = "Overflow"
badException = true
case .DivideByZero:
errorstr = "DivideByZero"
badException = true
}
println("Exception called for operation \(operation) -> \(errorstr)")
return nil
}
// Add two numbers represented by the strings op1 and op2. Return nil
// if a bad exception occurs.
func add(op1: String, op2: String) -> NSDecimalNumber? {
let dn1 = NSDecimalNumber(string: op1)
let dn2 = NSDecimalNumber(string: op2)
// Init badException to false. It will be set to true if an
// overflow, underflow, or divide by zero exception occur.
badException = false
// Add the NSDecimalNumbers, passing ourselves as the implementor
// of the NSDecimalNumbersBehaviors protocol.
let dn3 = dn1.decimalNumberByAdding(dn2, withBehavior: self)
// Return nil if a bad exception happened, otherwise return the result
// of the add.
return badException ? nil : dn3
}
}
Well, you're using an ObjC API. So just handle the exceptions inside ObjC. Write an ObjC class that handles the exceptions and returns values for Swift code to consume.
One possibility would be to write MathWhiz as an ObjC class and return an inout NSError parameter (ie do it the way Core Data does, take an **NSError) and fill it with the proper value when you hit a recoverable error. Then you can eat exceptions from NSDecimalNumber and convert them into NSError values.
You could also write an entire NSDecimalNumber wrapper for Swift consumption that you then use in place of NSDecimalNumber in your Swift code. Perhaps you could overload operator + and its siblings for this class, and then work out how to represent the various possible errors without exceptions.
update answer for swift 2.0
As you mentioned, now Swift 2.0 support with try, throw, and catch keywords.
here is official announcement.
Error handling model: The new error handling model in Swift 2.0 will
instantly feel natural, with familiar try, throw, and catch keywords.
Best of all, it was designed to work perfectly with the Apple SDKs and
NSError. In fact, NSError conforms to a Swift’s ErrorType. You’ll
definitely want to watch the WWDC session on What’s New in Swift to
hear more about it.
e.g
func loadData() throws { }
func test() {
do {
try loadData()
} catch {
print(error)
}
}
I would like to release a resource when any exception is raised during the usage of the resource.
In C++ this task is easy: I put the release into the destructor, which gets called automatically, whatever happens. In Java one uses the 'finally' clause. What is the practice for this same task in Standard ML?
I tried to catch all exception with a variable pattern 'e' and re-raise it:
datatype FileReadResult = FileReadOkay of string | FileReadError
fun read_file (file_path_string : string) : FileReadResult =
let
val istream = TextIO.openIn file_path_string
(* this file is my resource *)
in
TextIO.closeIn istream;
FileReadOkay "" (* the content of the file will go here *)
handle e => (TextIO.closeIn istream; raise e)
end
handle Io => FileReadError
My compiler (MLton) accepts it, but because I am new in ML, I ask here for some assurance that this is really the right thing | best practice to do.
As this is a common design pattern, I created the below utility function to express it:
(* Uses the given resource in the given way while releasing it if any exception occurs. *)
fun use_resource (resource : 'Resource) (releaser : 'Resource -> unit) (usage : unit -> 'Result) : 'Result =
let
val r = usage ()
in
releaser resource;
r
end
handle e => (releaser resource; raise e)
This function plays the same role as the 'using' feature in C#.
Yes, that's the usual pattern, with two caveats:
The inner handle is around the FileReadOkay "" only in your code, which won't ever throw. You want to put parentheses around a larger part of the code, so that the handler applies to all of it.
Your outer handler catches Io. I think you mean IO.Io _ here, otherwise you will catch every exception (because Io is just a random fresh variable).
You can also try to abstract it into a function if it occurs frequently. Something along the lines of
(* withTextFile : string -> (TextIO.instream -> 'a) -> 'a
fun withTextFile name f =
let
val is = TextIO.openIn name
in
(f is before TextIO.closeIn is)
handle e => (TextIO.closeIn is; raise e)
end
(The infix operator before evaluates its left-hand and right-hand expression and returns the result of the former). Use it like:
fun echo file = withTextFile file (fn is => print(TextIO.inputAll is))
Im playing around with writing something like a really simple asynchronous testing framework.
But I think I'm hitting some kind of limitation or bug. Sorry but I was not able to reproduce this on a smaller codebase.
This is the basic Framework I came up with:
module TestRunner
open System
type TestOptions = {
Writer : ConsoleColor -> string -> unit}
type TestResults = {
Time : TimeSpan
Failure : exn option
}
type Test = {
Name : string
Finished : IEvent<TestResults>
SetFinished : TestResults -> unit
TestFunc : TestOptions -> Async<TestResults> }
let createTest name f =
let ev = new Event<TestResults>()
{
Name = name
Finished = ev.Publish
SetFinished = (fun res -> ev.Trigger res)
TestFunc =
(fun options -> async {
let watch = System.Diagnostics.Stopwatch.StartNew()
try
do! f options
watch.Stop()
return { Failure = None; Time = watch.Elapsed }
with exn ->
watch.Stop()
return { Failure = Some exn; Time = watch.Elapsed }
})}
let simpleTest name f =
createTest name (fun options -> f options.Writer)
/// Create a new Test and change the result
let mapResult mapping test =
{ test with
TestFunc =
(fun options -> async {
let! result = test.TestFunc options
return mapping result})}
let writeConsole color f =
let old = System.Console.ForegroundColor
try
System.Console.ForegroundColor <- color
f()
finally
System.Console.ForegroundColor <- old
let printColor color (text:String) =
writeConsole color (fun _ -> Console.WriteLine(text))
type WriterMessage =
| NormalWrite of ConsoleColor * String
| StartTask of AsyncReplyChannel<int> * String
| WriteMessage of int * ConsoleColor * String
| EndTask of int
/// will handle printing jobs for two reasons
/// 1. Nice output grouped by tests (StartTask,WriteMessage,EndTask)
/// 2. Print Summary after all tests finished (NormalWrite)
let writer = MailboxProcessor.Start (fun inbox ->
let currentTask = ref 0
let newHandle (returnHandle:AsyncReplyChannel<int>) =
let handle = System.Threading.Interlocked.Increment currentTask
returnHandle.Reply handle
handle
// the tasks describe which tasks are currently waiting to be processed
let rec loop tasks = async {
let! newTasks =
match tasks with
/// We process the Task with the number t and the name name
| (t, name) :: next ->
inbox.Scan
(fun msg ->
match msg with
| EndTask (endTask) ->
// if the message is from the current task finish it
if t = endTask then
Some (async { return next })
else None
| WriteMessage(writeTask, color, message) ->
if writeTask = t then
Some (async {
printColor color (sprintf "Task %s: %s" name message)
return tasks
})
else None
| StartTask (returnHandle, name) ->
// Start any tasks instantly and add them to the list (because otherwise they would just wait for the resonse)
Some (async {
let handle = newHandle returnHandle
return (List.append tasks [handle, name]) })
| _ -> None)
// No Current Tasks so just start ones or process the NormalWrite messages
| [] ->
inbox.Scan
(fun msg ->
match msg with
| StartTask (returnHandle, name) ->
Some (async {
let handle = newHandle returnHandle
return [handle, name] })
| NormalWrite(color, message) ->
Some (async {
printColor color message
return []
})
| _ -> None)
return! loop newTasks
}
loop [])
/// Write a normal message via writer
let writerWrite color (text:String) =
writer.Post(NormalWrite(color, text))
/// A wrapper around the communication (to not miss EndTask for a StartTask)
let createTestWriter name f = async {
let! handle = writer.PostAndAsyncReply(fun reply -> StartTask(reply, name))
try
let writer color s =
writer.Post(WriteMessage(handle,color,s))
return! f(writer)
finally
writer.Post (EndTask(handle))
}
/// Run the given test and print the results
let testRun t = async {
let! results = createTestWriter t.Name (fun writer -> async {
writer ConsoleColor.Green (sprintf "started")
let! results = t.TestFunc { Writer = writer }
match results.Failure with
| Some exn ->
writer ConsoleColor.Red (sprintf "failed with %O" exn)
| None ->
writer ConsoleColor.Green (sprintf "succeeded!")
return results})
t.SetFinished results
}
/// Start the given task with the given amount of workers
let startParallelMailbox workerNum f =
MailboxProcessor.Start(fun inbox ->
let workers = Array.init workerNum (fun _ -> MailboxProcessor.Start f)
let rec loop currentNum = async {
let! msg = inbox.Receive()
workers.[currentNum].Post msg
return! loop ((currentNum + 1) % workerNum)
}
loop 0 )
/// Runs all posted Tasks
let testRunner =
startParallelMailbox 10 (fun inbox ->
let rec loop () = async {
let! test = inbox.Receive()
do! testRun test
return! loop()
}
loop ())
/// Start the given tests and print a sumary at the end
let startTests tests = async {
let! results =
tests
|> Seq.map (fun t ->
let waiter = t.Finished |> Async.AwaitEvent
testRunner.Post t
waiter
)
|> Async.Parallel
let testTime =
results
|> Seq.map (fun res -> res.Time)
|> Seq.fold (fun state item -> state + item) TimeSpan.Zero
let failed =
results
|> Seq.map (fun res -> res.Failure)
|> Seq.filter (fun o -> o.IsSome)
|> Seq.length
let testCount = results.Length
if failed > 0 then
writerWrite ConsoleColor.DarkRed (sprintf "--- %d of %d TESTS FAILED (%A) ---" failed testCount testTime)
else
writerWrite ConsoleColor.DarkGray (sprintf "--- %d TESTS FINISHED SUCCESFULLY (%A) ---" testCount testTime)
}
Now the Exception is only triggered when i use a specific set of tests
which do some crawling on the web (some fail and some don't which is fine):
#r #"Yaaf.GameMediaManager.Primitives.dll";; // See below
open TestRunner
let testLink link =
Yaaf.GameMediaManager.EslGrabber.getMatchMembers link
|> Async.Ignore
let tests = [
// Some working links (links that should work)
yield!
[ //"TestMatch", "http://www.esl.eu/eu/wire/anti-cheat/css/anticheat_test/match/26077222/"
"MatchwithCheater", "http://www.esl.eu/de/csgo/ui/versus/match/3035028"
"DeletedAccount", "http://www.esl.eu/de/css/ui/versus/match/2852106"
"CS1.6", "http://www.esl.eu/de/cs/ui/versus/match/2997440"
"2on2Versus", "http://www.esl.eu/de/css/ui/versus/match/3012767"
"SC2cup1on1", "http://www.esl.eu/eu/sc2/go4sc2/cup230/match/26964055/"
"CSGO2on2Cup", "http://www.esl.eu/de/csgo/cups/2on2/season_08/match/26854846/"
"CSSAwpCup", "http://www.esl.eu/eu/css/cups/2on2/awp_cup_11/match/26811005/"
] |> Seq.map (fun (name, workingLink) -> simpleTest (sprintf "TestEslMatches_%s" name) (fun o -> testLink workingLink))
]
startTests tests |> Async.Start;; // this will produce the Exception now and then
https://github.com/matthid/Yaaf.GameMediaManager/blob/core/src/Yaaf.GameMediaManager.Primitives/EslGrabber.fs is the code and you can download https://github.com/downloads/matthid/Yaaf.GameMediaManager/GameMediaManager.%200.9.3.1.wireplugin (this is basically a renamed zip archive) and extract it to get the Yaaf.GameMediaManager.Primitives.dll binary
(you can paste it into FSI instead of downloading when you want but then you have to reference the HtmlAgilityPack)
I can reproduce this with Microsoft (R) F# 2.0 Interactive, Build 4.0.40219.1. The Problem is that the Exception will not be triggered always (but very often) and the stacktrace is telling me nothing
System.Exception: multiple waiting reader continuations for mailbox
bei <StartupCode$FSharp-Core>.$Control.-ctor#1860-3.Invoke(AsyncParams`1 _arg11)
bei <StartupCode$FSharp-Core>.$Control.loop#413-40(Trampoline this, FSharpFunc`2 action)
bei Microsoft.FSharp.Control.Trampoline.ExecuteAction(FSharpFunc`2 firstAction)
bei Microsoft.FSharp.Control.TrampolineHolder.Protect(FSharpFunc`2 firstAction)
bei <StartupCode$FSharp-Core>.$Control.finishTask#1280[T](AsyncParams`1 _arg3, AsyncParamsAux aux, FSharpRef`1 firstExn, T[] results, TrampolineHolder trampolineHolder, Int32 remaining)
bei <StartupCode$FSharp-Core>.$Control.recordFailure#1302[T](AsyncParams`1 _arg3, AsyncParamsAux aux, FSharpRef`1 count, FSharpRef`1 firstExn, T[] results, LinkedSubSource innerCTS, TrampolineHolder trampolineHolder, FSharpChoice`2 exn)
bei <StartupCode$FSharp-Core>.$Control.Parallel#1322-3.Invoke(Exception exn)
bei Microsoft.FSharp.Control.AsyncBuilderImpl.protectedPrimitive#690.Invoke(AsyncParams`1 args)
bei <StartupCode$FSharp-Core>.$Control.loop#413-40(Trampoline this, FSharpFunc`2 action)
bei Microsoft.FSharp.Control.Trampoline.ExecuteAction(FSharpFunc`2 firstAction)
bei Microsoft.FSharp.Control.TrampolineHolder.Protect(FSharpFunc`2 firstAction)
bei <StartupCode$FSharp-Core>.$Control.-ctor#473-1.Invoke(Object state)
bei System.Threading.QueueUserWorkItemCallback.WaitCallback_Context(Object state)
bei System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean ignoreSyncCtx)
bei System.Threading.QueueUserWorkItemCallback.System.Threading.IThreadPoolWorkItem.ExecuteWorkItem()
bei System.Threading.ThreadPoolWorkQueue.Dispatch()
bei System.Threading._ThreadPoolWaitCallback.PerformWaitCallback()
Because this is will be triggered on a worker thread, which I have no control of, this will crash the application (not FSI but the exception will be displayed here too).
I found http://cs.hubfs.net/topic/Some/2/59152 and http://cs.hubfs.net/topic/None/59146 but I do not use StartChild and I don't think I'm invoking Receive from multiple Threads at the same time somehow?
Is there anything wrong with my Code or is this indeed a bug? How can I workaround this if possible?
I noticed that in FSI that all tests will run as expected when the Exception is silently ignored. How can I do the same?
EDIT: I noticed after I fixed the failing unit tests it will work properly. However I can stil not reproduce this with a smaller codebase. For example with my own failing tests.
Thanks, matthid
My feeling is that the limitation would be within the MailboxProcessor itself rather than async.
To be honest I would err on the side of caution with the Scan functions. I wrote a blog post on the dangers of using them.
Is it possible to process the tasks with the standard receiving mechanism rather than using Scan functions?
As a note, inside async there is trampoline that is used so that the same thread is reused a set number of time to avoid unnecessary thread pool usage, (I think this is set to 300) so when debugging you may see this behaviour.
I would approach this problem slightly differently decomposing the separate components into pipeline stages rather than the nested async blocks. I would create a supervisor component and routing component.
The Supervisor would look after the initial tests and post messages to a routing component that would round-robin the requests to other agents. When the tasks are completed they could post back to the supervisor.
I realise this does not really help with the problem in the current code but I think you will have to decompose the problem anyway in order to debug the async parts of the system.
I do believe there was a bug in the 2.0 implementation of Scan/TryScan/Receive that might spuriously cause the
multiple waiting reader continuations for mailbox
exception; I think that bug is now fixed in the 3.0 implementation. I haven't looked carefully at your code to try to ensure you're only trying to receive one message at a time in your implementation, so it's also possible this might be a bug in your code. If you can try it out against F# 3.0, it would be great to know if this goes away.
Sadly I never actually could reproduce this on a smaller code base, and now I would use NUnit with async test support instead of my own implementation. I used agents (MailboxProcessor) and asyncs in various projects since them and never encountered this again...
Some notes, in case someone finds my experiences useful (it took a long time debugging multiple processes in order to locate the problem):
Execution and throughput started to get clogged up with just 50 Agents/Mailboxes. Sometimes with a light load it would work for the first round of messages but anything as significant as making a call to a logging library triggered the longer delay.
Debugging using the Threads/Parallel Stacks window in the VS IDE, the runtime is waiting on the results of
FSharpAsync.RunSynchronously -> CancellationTokenOps.RunSynchronously call by Trampoline.ExecuteAction
I suspect that the underlying ThreadPool is throttling startup (after the first time it seems to run ok). It's a very long delay. I'm using agents to serialise within certain queues minor computations, while allowing the main dispatching agent to remain responsive, so the delay is somewhere in the CLR.
I found that running MailboxProcessor Receive with a Timeout within a try-with, stopped the delay, but that this needed to be wrapped in an async block to stop the rest of the program slowing down, however short the delay. Despite a little bit of twiddling around, very happy with the F# MailboxProcessor for implementing the actor model.