NServiceBus: Messages handled multiple times - castle-windsor

I am at a complete loss as to why I am experiencing this problem. I am new to NServiceBus and have so far set up a dead simple 'server' which listens for messages sent by a web application. The server asks for custom initialisation (IWantCustomInitialization) and uses a custom builder for Castle Windsor 2.5.1. This custom builder is basically a copy of the one that comes with the NServiceBus source code, with two minor changes to move away from methods deprecated in Windsor 2.5.
Note that my code shares the container instance with NServiceBus.
The problem I experience is that every message sent by the web application is processed five (5) times by the server. The log files have five entries for each attempt, with the fifth attempt looking like this:
2011-03-28 16:04:10,326 [Worker.8] DEBUG NServiceBus.Unicast.UnicastBus [] - Calling 'HandleEndMessage' on NServiceBus.SagaPersisters.NHibernate.NHibernateMessageModule
2011-03-28 16:04:10,327 [Worker.8] DEBUG NServiceBus.Unicast.UnicastBus [] - Calling 'HandleEndMessage' on Server.NHibernateSessionMessageModule
2011-03-28 16:04:10,341 [Worker.8] DEBUG NServiceBus.Unicast.UnicastBus [] - Calling 'HandleError' on NServiceBus.SagaPersisters.NHibernate.NHibernateMessageModule
2011-03-28 16:04:10,342 [Worker.8] DEBUG NServiceBus.Unicast.UnicastBus [] - Calling 'HandleError' on Server.NHibernateSessionMessageModule
2011-03-28 16:04:10,344 [Worker.8] ERROR NServiceBus.Unicast.Transport.Msmq.MsmqTransport [] - Message has failed the maximum number of times allowed, ID=80cffd98-a5bd-43e0-a482-a2d96ca42b22\20677.
I have no indication why the message fails, and I don't know where to dig for more information/output.
The configuration 'endpoint' looks like this:
public void Init()
{
container = Windsor.Container;
NServiceBus.Configure.With().CastleWindsor251Builder(container).XmlSerializer().MsmqTransport().IsolationLevel(System.Transactions.IsolationLevel.Unspecified);
var masterInstaller = new NotificationServerInstaller();
masterInstaller.Install(container, null);
}
The message handler is, at this stage, really contrived, and looks like this:
public class NewUserMessageHandler : IHandleMessages<NotifyNewUserMessage>
{
private readonly IGetUserQuery _getUserQuery;
public NewUserMessageHandler(IGetUserQuery getUserQuery)
{
_getUserQuery = getUserQuery;
}
public void Handle(NotifyNewUserMessage message)
{
var result = _getUserQuery.Invoke(new GetUserRequest { Id = new Guid("C10D0684-D25F-4E5E-A347-16F85DB7BFBF") });
Console.WriteLine("New message received: {0}", message.UserSystemId);
}
}
If the first line in the handler method is commented out, the message is processed only once.
I have found some posts/threads on the web (including StackOverflow) which talk about similar issues, notably http://tech.groups.yahoo.com/group/nservicebus/message/5977 and Anyone using Ninject 2.0 as the nServiceBus ObjectBuilder? - but I haven't had any success in making my problem go away.
I'd be most obliged for any help. I'm a n00b at NServiceBus!

NServiceBus isn't handling it multiple times by default it will retry 5 times if an exception occurs, you can set this in a config file. Have you got distributed transactions turned on? Because you are committing to a database and you have an open transaction (the queue transaction) when you open another transaction it will try and upgrade it to a distributed transaction, I think that may be the issue. Have you run with the console app? You should see some out put on there.

I would recommend wrapping the body of the Handle method in a try/catch and add a break point to the catch and see what is wrong.
Once you work it out, remove the try/catch.

Related

How to handle "Unexpected EOF at target" error from API calls?

I'm creating a Forge application which needs to get version information from a BIM 360 hub. Sometimes it works, but sometimes (usually after the code has already been run once this session) I get the following error:
Exception thrown: 'Autodesk.Forge.Client.ApiException' in mscorlib.dll
Additional information: Error calling GetItem: {
"fault":{
"faultstring":"Unexpected EOF at target",
"detail": {
"errorcode":"messaging.adaptors.http.flow.UnexpectedEOFAtTarget"
}
}
}
The above error will be thrown from a call to an api, such as one of these:
dynamic item = await itemApi.GetItemAsync(projectId, itemId);
dynamic folder = await folderApi.GetFolderAsync(projectId, folderId);
var folders = await projectApi.GetProjectTopFoldersAsync(hubId, projectId);
Where the apis are initialized as follows:
ItemsApi itemApi = new ItemsApi();
itemApi.Configuration.AccessToken = Credentials.TokenInternal;
The Ids (such as 'projectId', 'itemId', etc.) don't seem to be any different when this error is thrown and when it isn't, so I'm not sure what is causing the error.
I based my application on the .Net version of this tutorial: http://learnforge.autodesk.io/#/datamanagement/hubs/net
But I adapted it so I can retrieve multiple nodes asynchronously (for example, all of the nodes a user has access to) without changing the jstree. I did this to allow extracting information in the background without disrupting the user's workflow. The main change I made was to add another Route on the server side that calls "GetTreeNodeAsync" (from the tutorial) asynchronously on the root of the tree and then calls it on each of the returned children, then each of their children, and so on. The function waits until all of the nodes are processed using Task.WhenAll, then returns data from each of the nodes to the client;
This means that there could be many api calls running asynchronously, and there might be duplicate api calls if a node was already opened in the jstree and then it's information is requested for the background extraction, or if the background extraction happens more than once. This seems to be when the error is most likely to happen.
I was wondering if anyone else has encountered this error, and if you know what I can do to avoid it, or how to recover when it is caught. Currently, after this error occurs, it seems that every other api call will throw this error as well, and the only way I've found to fix it is to rerun the code (I use Visual Studio so I just rerun the server and client, and my browser launches automatically)
Those are sporadic errors from our apigee router due to latency issues in the authorization process that we are currently looking into internally.
When they occur please cease all your upcoming requests, wait for a few minutes and retry again. Take a look at stuff like this or this to help you out.
And our existing reports calling out similar errors seem to point to concurrency as one of the factors leading up to the issue so you might also want to limit your concurrent requests and see if that mitigate the issue.

why DeferredResult ends on setResult() on trying to use SSE

i am trying to implement a Server Sent Events (SSE) webpage which is powered by Spring. My test code does the following:
Browser uses EventSource(url) to connect to server. Spring accepts the request with the following controller code:
#RequestMapping(value="myurl", method = RequestMethod.GET, produces = "text/event-stream")
#ResponseBody
public DeferredResult<String> subscribe() throws Exception {
final DeferredResult<String> deferredResult = new DeferredResult<>();
resultList.add(deferredResult);
deferredResult.onCompletion(() -> {
logTimer.info("deferedResult "+deferredResult+" completion");
resultList.remove(deferredResult);
});
return deferredResult;
}
So mainly it puts the DeferredResult in a List and register a completion callback so that i can remove this thing from the List in case of completion.
Now i have a timer method, that will periodically output current timestamp to all registered "browser" via their DeferredResults.
#Scheduled(fixedRate=10000)
public void processQueues() {
Date d = new Date();
log.info("outputting to "+ LoginController.resultList.size()+ " connections");
LoginController.resultList.forEach(deferredResult -> deferredResult.setResult("data: "+d.getTime()+"\n\n"));
}
The data is sent to the browser and the following client code works:
var source = new EventSource('/myurl');
source.addEventListener('message', function (e) {
console.log(e.data);
$("#content").append(e.data).append("<br>");
});
Now the problem:
The completion callback on the DeferredResult is called on every setResult() call in the timer thread. So for some reason the connection is closed after the setResult() call. SSE in the browser reconnects as per spec and then same thing again. So on client side i have a polling behavior, but i want an kept open request where i can push data on the same DeferredResult over and over again.
Do i miss something here? Is DeferredResult not capable of sending multiple results? i put in a 10 seconds delay in the timer thread to see if the request only terminates after setResult(). So in the browser the request is kept open until the timer pushes the data but then its closed.
Thanks for any hint on that. One more note: I added async-supported to all filters/servlets in tomcat.
Indeed DeferredResult can be set only once (notice that setResult returns a boolean). It completes processing with the full range of Spring MVC processing options, i.e. meaning that all you know about what happens during a Spring MVC request remains more or less the same, except for the asynchronously produced return value.
What you need for SSE is something more focused, i.e. write each value to the response using an HttpMessageConverter. I've created a ticket for that https://jira.spring.io/browse/SPR-12212.
Note that Spring's SockJS support does have an SSE transport which takes care of a few extras such as cross-domain requests with cookies (important for IE). It's also used on top of a WebSocket API and WebSocket-style messaging (even if WebSocket is not available on either the client or the server side) which fully abstracts the details of HTTP long polling.
As a workaround you can also write directly to the Servlet response using an HttpMessageConverter.

Perform multiple updates at once via Twitter4J on sincle account

The Java class here works fine for me. My problem is that I can't perform more than 1 updates at once. In fact, Java throws a try-catch exception every time it reaches the following lines for second time:
try {
// get request token.
// this will throw IllegalStateException if access token is already available
RequestToken requestToken = twitter.getOAuthRequestToken();
...
As you can imagine, the class is being called multiple times within a for loop. So, any ideas?
PS: Hope not having messed up my question. :D
This should be done like this (pseudo code)
GetAccessToken
AuthenticateUser
Loop
doUpdate
CleanUp
Twitter4J "holds" a session for the authenticated user, so starting with a new, different user needs a new instance of the Twitter4J.

WebAPI and HTML5 SSE

was trying to encapsulate a partial view to show feedback that i can push back to the client.
This Article shows a method of pushing back data using HTML5 Server-Sent events (SSE).
I noticed that if i opened up several browser tabs and then closed one i got exceptions as the logic didn't remove the respective stream from the ConcurrentQueue. I amended the code as below
private static void TimerCallback(object state)
{
StreamWriter data;
Random randNum = new Random();
// foreach (var data in _streammessage)
for (int x = 0; x < _streammessage.Count; x++)
{
_streammessage.TryDequeue(out data);
data.WriteLine("data:" + randNum.Next(30, 100) + "\n");
try
{
data.Flush();
_streammessage.Enqueue(data);
}
catch (Exception ex)
{
// dont re-add the stream as an error ocurred presumable the client has lost connection
}
}
//To set timer with random interval
_timer.Value.Change(TimeSpan.FromMilliseconds(randNum.Next(1, 3) * 500), TimeSpan.FromMilliseconds(-1));
}
I also had to amend the OnStreamAvailable member as the framework syntax had changed to the second parameter being a HttpContent rather than HttpContentHeaders
public static void OnStreamAvailable(Stream stream, HttpContent headers, TransportContext context)
The problem now is i am still getting inconsistant behaviour if i add or remove clients i.e it times out when trying to initialise a new client. Does anyone have any ideas or more examples of using SSE with WinAPI and the correct "framework of methods" to handle disconnected clients
Cheers
Tim
This article is actually an adaptation of my original article from May - http://www.strathweb.com/2012/05/native-html5-push-notifications-with-asp-net-web-api-and-knockout-js/ (notice even variable names and port numbers are the same :-).
It is a very valid point that you are raising, and detecting a broken connection is something that's not very easy with this setup. The main reason is that while ASP.NET (the host) allows you to check a broken connection, there is no notification mechanism between ASP.NET (host) and Web API informing about that.
That is why in order to detect a broken connection (disconnected client) you should really try writing to the stream, and catch any error - this would mean the client has been disconnected.
I asked the same question to Brad Wilson/Marcin Dobosz/Damien Edwards at aspconf, and Damien suggested using HttpContext.Current.Response.IsClientConnected - so basically bypassing Web API and obtaining the connectivity info from the underlying host directly (however there is still a race condition involved anyway). That is really .NET 4. He also pointed an interesting way in which this problem could be avoided in .NET 4.5 using an async cancellation token. Frankly, I have never got around to test it, but perhaps this is something you should explore.
You can see their response to this problem in this video - http://channel9.msdn.com/Events/aspConf/aspConf/Ask-The-Experts - fast forward to 48:00

Silverlight Exceptions on Server [duplicate]

I recently encountered a strange thing. On some of my company's servers when an exception message is printed out (yes, bad, I know. It's for debugging), the actual message isn't displayed. Instead it displays the key for an external string resource and says that "Debugging resource strings are unavailable"
After some research I've come up with the following:
In release mode, Silverlight does not package the xap with the dlls containing the actual error messages in order to save space.
I've found workarounds for OLD versions, but nothing for 4. It seems like there are Developer versions of the SL 2 and 3 runtime which will resolve the errors automatically, but I cannot find one for SL 4.
So my question is this:
What the heck do I need to do to my SL 4 app / computer to let me see the full, detailed errors when it's in release mode?
You can download the developer runtime (which contains the full exception strings) from the GetStarted page - http://www.silverlight.net/getstarted/ - search for "Developer Runtimes for Windows and OSX", it's near the bottom of the page.
Though it is too late to reply, it may help somebody else. We have a web application using Silverlight 4, installed in various test environments. This web application consumes more than one WCF services. All but one of the test environment sites consistently failed with message "Debugging Resource strings are unavailable". Agreeably the real exception was swallowed. Being a Silverlight application, there was no logging, and it always appeared that there was something failing in the Silverlight component. I connected the application in my development environment to that particular test environment, and found out that the problem was in fact in one of the WCF services. I fixed the problem at the service end and the SL component stopped having this problem.
Why was the WCF failing?
The WCF service had the following code in the constructor:
public MyService()
{
//Create an instance of Data Lookup service asycnchronously.
if (_dataLookupSrvc == null)
{
try
{
System.Threading.Tasks.Task.Factory.StartNew(() => _dataLookupSrvc = new LookupDataService.LookupDataService());
}
catch (Exception ex)
{
_log.Error<Exception>(ex);
}
}
}
Somebody moved the underlying LookupDataService.dll from the service folder causing the constructor to fail, but not right away. As the LookupDataService instance was created in anonymous method, the exception logging in this method never took place. Once the LookupDataService.dll was dropped in the service folder, the "Debugging Resource strings are unavailable" message went away.
It was a fun wild goose chase!
Have you already checked the event viewer on the machine where the application crashed? Start->Run. eventvwr