I've a fifo queue, and i'm using org.springframework.cloud.aws.messaging and here's what i'd like to do:
Consume the message
Try to handle it (inner logic)
If handles fails - write new attribute on the message (without sending the message again to the queue)
I don't want to send the messages to a new queue (i need to keep the order of the messages).
also, i don't want to use the deadletter queue for handling errors (same reason as above).
the reason i want to use message attributes is due to the fact that i need to implement in-house retry mechanism, meaning: when consuming the message i'll check the last_try_timestamp and if it's passed my validation then i'll try to handle it, else i'll throw an error.
(I know that the message will continue to be consumed until the MaxRetention and i'm fine with it)
Is something like that possible?
#SqsListener(value = "#{'${my.queue.fifo}'}", deletionPolicy = SqsMessageDeletionPolicy.ON_SUCCESS)
public void deadLetterQueueListener(#Headers Map<String, String> headers, String message) throws Exception {
log.info("consuming message");
if(!this.handleMessage(message)){
//Set message attributes (timestamp)
throw new Exception("Failed to handle message");
}
}
As far as I know there is no way to do it purely with SQS. If you modify the message attribute you would need to resend this message to propagate this change to SQS.
It can be implemented on the application side by storing metadata like last_try_timestamp in an external datasource like a DynamoDB where you could map message id to any metadata you need.
Related
Sorry if my question is dumb, I'm new to MassTransit.
My system consists of a server and multiple client devices.
I'd like to send a message from the server to a specific client or to a group of clients.
As far as I understand, IBusControl.Publish sends the message to all subscribers, and IBusControl.Send to the only one subscriber.
How can I achieve this using MassTransit?
My transports are RabbitMQ / Azure Service Bus.
Thanks!
MassTransit implements standard messaging patterns, which aren't MassTransit-specific. Point-to-point, publish-subscribe, invalid message channel, dead letter channel and so on:
You indeed have the choice between sending a message to one consumer using Send and to broadcast messages to all subscribers for that message type by using Publish.
Everything else can be easily done by adding code to consumers:
await bus.Publish(new MyMessage { ReceiverGroup = "group1", ... });
and
public async Task Consume(IContext<MyMessage> context)
{
if (context.Message.ReceiverGroup != myGroup) return;
...
}
I want my data access layer to handle exceptions thrown by DocumentDB API provided via Microsoft.Azure.Documents.Client.DocumentClient class. For example, the optimistic concurrency check implemented using AccessCondition class, but others as well.
By looking at the exception thrown, the best way to recognize different DocumentClient-specific exceptions seems to be something like this:
try { ... }
catch (DocumentClientException exception)
when (exception.Error.Code == "Some magic here")
{
//let the user know how to recover from this..
}
I don't like such magic strings as they are not verifiable compile-time. It may contract a typo, or it may change on random moment with DocumentDB client/server changes, etc. Also, it is not clear which such magic codes I could/should be handling since I don't see the Microsoft.Azure.DocumentDB .net API containing any ErrorCodes enum or constants, nor find any list in documentation.
Where can I find a list of possible Error.Code values DocumentClient API can throw?
To make it even more confusing, the XmlDoc for DocumentClient.CreateDocumentAsync method suggest working instead on http status codes.
UPDATE: This question is not about Http status codes but DocumentClientException.Error.Code field as I assume the latter is more precise.
Where can I find a list of possible error codes values DocumentClient API can throw?
It's hard to find the completely list of error code that DocumentClinet API throw. The exception is depend on what your request.
For example, the optimistic concurrency check
Azure Cosmos DB uses ETags for handling optimistic concurrency.
When we retrieve a document from Azure Cosmos DB, it always contains an ETag property as apart of our document.
When we then want to send our request to replace a document, we can specify an AccessCondition with the ETag we received when we fetched out our document.
If the ETag we send is not current, the server will return a 412 Precondition Failed status code. In our .NET SDK, this is wrapped up in a DocumentClientException.
Here is an example that show the possible problems when the concurrency occurred.
By decompile the version 1.22.0 client, the code is set as a HttpStatusCode enum. I think all the possible values can be found here https://msdn.microsoft.com/en-us/library/system.net.httpstatuscode(v=vs.110).aspx then.
However, what really contains richer information for debug is the Error.Message. Might need to decompile the whole library to figure out, or wait for Microsoft to release the source codes, which is unlikely to happen since the latest update in github was 2 or 3 years ago.
public Error Error
{
get
{
if (this.error == null)
{
this.error = new Error()
{
Code = this.StatusCode.ToString(),
Message = this.Message
};
}
return this.error;
}
}
There is a list of the HTTP Status Codes for Azure Cosmos DB
I use the following code in my catch blocks
catch (DocumentClientException e)
{
var resp = new HttpResponseMessage
{
StatusCode = (HttpStatusCode) e.StatusCode,
Content = new StringContent(e.Message)
};
return resp;
}
Letting the user know how to handle the exception should be done on the client application.
I recently got back into using GameMaker:Studio, and hoo boy have there been some massive updates since I last used it! In fact the last time I used it they only had Windows and HTML5 as export options...
Anyway, eager to try out some of the new stuff, I decided to take a shot at the native HTTP functions, since they looked very promising.
I did a test using http_post_string() to great effect, sending a JSON string to my server and getting a JSON string back. The returned string actually represented an object with a single property, "echo", which contained the HTTP request that had been made, just to see what GM:S was sending.
I didn't like that it sent Content-Type: application/x-www-form-urlencoded when it was quite clearly JSON, and I wanted the ability to set my own User Agent string so that the server could know which game was talking to it without having to pass an extra parameter.
So I re-created the same request using the lower-level http_request() function. Everything looked fine, so I tested it.
It crashed. Like, no error messages or anything, just a total crash and Windows had to force-close it.
So here I am with code that by all rights should work fine, but crashes when run...
///send_request(file,ds_map_data,callback_event_id)
var request = ds_map_create();
request[? "instance"] = id;
request[? "event"] = argument2;
if( !instance_exists(obj_ajax_callback)) {
instance_create(0,0,obj_ajax_callback);
}
var payload = json_encode(argument1);
var headers = ds_map_create();
headers[? "Content-Length"] = string_length(payload);
headers[? "Content-Type"] = "application/json";
headers[? "User-Agent"] = obj_ajax_callback.uastring;
var xhr = http_request("https://example.com/"+argument0,"POST",headers,payload);
with(obj_ajax_callback) {
active_callbacks[? xhr] = request;
}
ds_map_destroy(headers);
obj_ajax_callback is an object that maintains a ds_map of active requests, and in its HTTP event it listens for those requests' callbacks and reacts along the lines of with(request[? "instance"]) event_user(request[? "event"]) so that the calling object can handle the response. This hasn't changed from the fully working http_post_string() attempt.
Any idea what could be causing this crash?
The reason why this crashes is because you are sending the Content-Length header as a real instead of a string. If you change your line to
headers[? "Content-Length"] = string(string_length(payload));
It should work.
i am trying to implement a Server Sent Events (SSE) webpage which is powered by Spring. My test code does the following:
Browser uses EventSource(url) to connect to server. Spring accepts the request with the following controller code:
#RequestMapping(value="myurl", method = RequestMethod.GET, produces = "text/event-stream")
#ResponseBody
public DeferredResult<String> subscribe() throws Exception {
final DeferredResult<String> deferredResult = new DeferredResult<>();
resultList.add(deferredResult);
deferredResult.onCompletion(() -> {
logTimer.info("deferedResult "+deferredResult+" completion");
resultList.remove(deferredResult);
});
return deferredResult;
}
So mainly it puts the DeferredResult in a List and register a completion callback so that i can remove this thing from the List in case of completion.
Now i have a timer method, that will periodically output current timestamp to all registered "browser" via their DeferredResults.
#Scheduled(fixedRate=10000)
public void processQueues() {
Date d = new Date();
log.info("outputting to "+ LoginController.resultList.size()+ " connections");
LoginController.resultList.forEach(deferredResult -> deferredResult.setResult("data: "+d.getTime()+"\n\n"));
}
The data is sent to the browser and the following client code works:
var source = new EventSource('/myurl');
source.addEventListener('message', function (e) {
console.log(e.data);
$("#content").append(e.data).append("<br>");
});
Now the problem:
The completion callback on the DeferredResult is called on every setResult() call in the timer thread. So for some reason the connection is closed after the setResult() call. SSE in the browser reconnects as per spec and then same thing again. So on client side i have a polling behavior, but i want an kept open request where i can push data on the same DeferredResult over and over again.
Do i miss something here? Is DeferredResult not capable of sending multiple results? i put in a 10 seconds delay in the timer thread to see if the request only terminates after setResult(). So in the browser the request is kept open until the timer pushes the data but then its closed.
Thanks for any hint on that. One more note: I added async-supported to all filters/servlets in tomcat.
Indeed DeferredResult can be set only once (notice that setResult returns a boolean). It completes processing with the full range of Spring MVC processing options, i.e. meaning that all you know about what happens during a Spring MVC request remains more or less the same, except for the asynchronously produced return value.
What you need for SSE is something more focused, i.e. write each value to the response using an HttpMessageConverter. I've created a ticket for that https://jira.spring.io/browse/SPR-12212.
Note that Spring's SockJS support does have an SSE transport which takes care of a few extras such as cross-domain requests with cookies (important for IE). It's also used on top of a WebSocket API and WebSocket-style messaging (even if WebSocket is not available on either the client or the server side) which fully abstracts the details of HTTP long polling.
As a workaround you can also write directly to the Servlet response using an HttpMessageConverter.
was trying to encapsulate a partial view to show feedback that i can push back to the client.
This Article shows a method of pushing back data using HTML5 Server-Sent events (SSE).
I noticed that if i opened up several browser tabs and then closed one i got exceptions as the logic didn't remove the respective stream from the ConcurrentQueue. I amended the code as below
private static void TimerCallback(object state)
{
StreamWriter data;
Random randNum = new Random();
// foreach (var data in _streammessage)
for (int x = 0; x < _streammessage.Count; x++)
{
_streammessage.TryDequeue(out data);
data.WriteLine("data:" + randNum.Next(30, 100) + "\n");
try
{
data.Flush();
_streammessage.Enqueue(data);
}
catch (Exception ex)
{
// dont re-add the stream as an error ocurred presumable the client has lost connection
}
}
//To set timer with random interval
_timer.Value.Change(TimeSpan.FromMilliseconds(randNum.Next(1, 3) * 500), TimeSpan.FromMilliseconds(-1));
}
I also had to amend the OnStreamAvailable member as the framework syntax had changed to the second parameter being a HttpContent rather than HttpContentHeaders
public static void OnStreamAvailable(Stream stream, HttpContent headers, TransportContext context)
The problem now is i am still getting inconsistant behaviour if i add or remove clients i.e it times out when trying to initialise a new client. Does anyone have any ideas or more examples of using SSE with WinAPI and the correct "framework of methods" to handle disconnected clients
Cheers
Tim
This article is actually an adaptation of my original article from May - http://www.strathweb.com/2012/05/native-html5-push-notifications-with-asp-net-web-api-and-knockout-js/ (notice even variable names and port numbers are the same :-).
It is a very valid point that you are raising, and detecting a broken connection is something that's not very easy with this setup. The main reason is that while ASP.NET (the host) allows you to check a broken connection, there is no notification mechanism between ASP.NET (host) and Web API informing about that.
That is why in order to detect a broken connection (disconnected client) you should really try writing to the stream, and catch any error - this would mean the client has been disconnected.
I asked the same question to Brad Wilson/Marcin Dobosz/Damien Edwards at aspconf, and Damien suggested using HttpContext.Current.Response.IsClientConnected - so basically bypassing Web API and obtaining the connectivity info from the underlying host directly (however there is still a race condition involved anyway). That is really .NET 4. He also pointed an interesting way in which this problem could be avoided in .NET 4.5 using an async cancellation token. Frankly, I have never got around to test it, but perhaps this is something you should explore.
You can see their response to this problem in this video - http://channel9.msdn.com/Events/aspConf/aspConf/Ask-The-Experts - fast forward to 48:00