MassTransit - Azure Service Bus Consumer auto re-consumes the same message - message-queue

I have a worker service and an api service using Mass Transit - Azure Message Bus. The api acts as a sender, the worker acts as a receiver.
In the worker service, I created a consumer which consumes the message from the queue.
However, while the consumer is processing the message from Azure message queue, Mass Transit automatically re-initializes the consumer and triggers the Consume method to handle the same message again.
Are there any configurations on Mass Transit to prevent this from happening?
All I need is that for one specific message, only one consumer instance should pick up and process the message, the other consumer instances (on different servers) of the same message type should not pick it up while it is being processed by the other one.
On Azure there is a ReceiveAndDelete mode over the Peek-Mode, but I don't know how to enable this setting on top of Mass Transit.
The API configuration
public static IServiceCollection AddMessageBusConfig(this IServiceCollection services, IDictionary<string, string> messageBusOptions)
{
var queuePrefix = messageBusOptions["QueuePrefix"];
return services.AddMassTransit(x =>
{
EndpointConvention.Map<MigrateAccountancy.Command>(new Uri($"queue:{queuePrefix}-migrate-accountancies-event"));
x.UsingAzureServiceBus((context, cfg) =>
{
cfg.Host(messageBusOptions["ConnectionString"]);
cfg.ConfigureEndpoints(context);
});
});
}
The Worker service configuration
public static IServiceCollection AddMessageBus(this IServiceCollection services, IDictionary<string, string> messageBusOptions)
{
var queuePrefix = messageBusOptions["QueuePrefix"];
return services.AddMassTransit(x =>
{
x.AddConsumer<MigrateAccountancyConsumer>();
x.UsingAzureServiceBus((context, cfg) =>
{
cfg.Host(messageBusOptions["ConnectionString"]);
cfg.ReceiveEndpoint($"{queuePrefix}-migrate-accountancies-event", e =>
{
// skip move message to _error queue on exception since errors are logged in Application Insight
e.DiscardFaultedMessages();
e.ConfigureConsumer<MigrateAccountancyConsumer>(context, c =>
{
});
});
});
});
}
Consumer
public class MigrateAccountancyConsumer : IConsumer<MigrateAccountancy.Command>
{
private readonly IMediator _mediator;
public MigrateAccountancyConsumer(IMediator mediator)
{
_mediator = mediator;
}
public async Task Consume(ConsumeContext<MigrateAccountancy.Command> context)
{
await _mediator.Send(context.Message);
}
}
As you can see above, the Consume method automatically re-triggers during the processing of the same message.
Appreciate for your helps.

I don't believe you can configure the ReceiveMode with MassTransit. You can try setting the ConcurrentMessageLimit to 1 and see if that solves the issue.
x.UsingAzureServiceBus((context, cfg) =>
{
cfg.Host(messageBusOptions["ConnectionString"]);
cfg.ConcurrentMessageLimit = 1 //Broker level
cfg.ReceiveEndpoint($"{queuePrefix}-migrate-accountancies-event", e =>
{
//cfg.ConcurrentMessageLimit = 1 //Endpoint level
});
});

Related

Vert.x httpclient - 3.5.0 throws exception "Connection was closed" intermittently

I have vert.x app which is consuming api REST over json but intermittently I am seeing exception with reason "Connection was closed". Below are my details -
please share your inputs if anything wrong in the configuration. may be creating scheduler or instantiating httpclient ?
on a different note is it advisable to use same http client to call more than 1 different api's on the same host and port ?
Vert.x Version: 3.5.0
import io.vertx.core.http.HttpClient;
private static Scheduler scheduler =
Schedulers.from(Executors.newFixedThreadPool(8));
// http client instantiated at the time of verticle startup
HttpClient httpclient = vertx.createHttpClient(getHttpClientOptions());
public static HttpClientOptions getHttpClientOptions() {
return new HttpClientOptions()
.setKeepAlive(true)
.setMaxPoolSize(100)
.setPipelining(true)
.setDefaultHost(xxxx.xxxx.com)
.setDefaultPort(8084)
.setSsl(true);
}
// invoke api call
public static Single<Response> invokePOSTServiceAsync(String reqBodyStr, String endpointURI) throws Exception {
try{
return Single.create((SingleEmitter<Response> emitter) -> {
HttpClientRequest request = httpClient.post(endpointURI);
request.putHeader("Content-type","application/json")
request.exceptionHandler(error -> {
LOG.error("ExceptionHandler "+error.getMessage());
emitter.onError(new Throwable(" Failure"));
})
.handler(response -> {
int statusCode = response.statusCode();
if (statusCode == 200) {
response.bodyHandler(body -> {
StringBuilder responseData = new StringBuilder();
responseData.append(body);
emitter.onSuccess(new Response(statusCode,responseData.toString(),"","",null));
});
} else {
emitter.onError(new Throwable(" Failure"));
}
})
.putHeader(HttpHeaders.CONTENT_LENGTH, reqBodyStr.length() + "")
.setTimeout(6000)
.write(reqBodyStr)
.end();
}).subscribeOn(scheduler);
}catch(Exception exe){
exe.printStackTrace();
return null;
}
}
My guess is that this is not related to the client. Either your server is being overloaded, or your network is unreliable. If you're consuming service which doesn't belong to you, you also may get throttled, and that's the reason you're seeing this.
In any case, you need to circumvent those problems, as the network is unreliable anyway. Make your POST requests idempotent and introduce retries.

Spring Web Socket - notify the client from the MQ Listener

I am using web-sockets using Spring.
Here is my controller. A simple controller, which would accept a result object and return a result object with populated values. It would publish message to the STOMP topic subscribers "/topic/update".
#Controller
public class ReportController {
#MessageMapping("/charthandler")
#SendTo("/topic/update")
public Result pushMessage(Result r) throws Exception {
Thread.sleep(3000); // simulated delay
Result result = new Result();
result.setTitle("ChartsPage");
return result;
}
}
My Spring Configuration file:
#Configuration
#EnableWebSocketMessageBroker
public class WebSocketConfig extends AbstractWebSocketMessageBrokerConfigurer {
#Override
public void registerStompEndpoints(StompEndpointRegistry registry) {
registry.addEndpoint("/charthandler").withSockJS();
}
#Bean
public WebSocketHandler chartHandler() {
return new ChartHandler();
}
#Override
public void configureMessageBroker(MessageBrokerRegistry config) {
config.enableSimpleBroker("/topic");
config.setApplicationDestinationPrefixes("/app");
}
}
I have the following code in javascript, which creates a STOMP Web Socket Client. It is subscribing to the '/topic/update'
var socket = new SockJS('/reportapplication/charthandler/');
stompClient = Stomp.over(socket);
stompClient.connect({}, function(frame) {
console.log('Connected: ' + frame);
stompClient.subscribe('/topic/update', function(result) {
console.log(JSON.parse(result.body).title);
});
});
Now i am planning to add a listener(java and not in javascript) which would listen to the Rabbit MQ message, i want to pass the message object to my controller and push all the message to the Web Socket Clients.
I am not sure how to notify all my web-socket clients , when the message is arrived at my MQ listener.
How will i do that?
Is it a good way to create an instance of report controller and call the pushMessage to notify all my web socket clients.
ReportController controller = new ReportController();
controller.pushMessage(report);
Also i'm not sure, if this works. I will try that. I want to know if there is a better approach.
Is there a better approach or better way of doing this?
Maybe if you look at the response Artem Bilan has provided to the following question: Spring, how to broadcast message to connected clients using websockets?
So if your java listener to the Rabbit MQ message is in a service then you can do the following in the same service and call the sendTo marked WS notification endpoint and pass on expected message to go out to the WS clients listening.
#Autowired
private SimpMessagingTemplate brokerMessagingTemplate;
.......
this.brokerMessagingTemplate.convertAndSend("/topic/greetings", "foo");

Resubmit web requests on resuming from dormancy

I am using RestSharp (a REST client for .NET) in my Windows Phone 8 app, but I think my question also applies using HttpWebRequest or any other ways of running web requests.
I am trying to find a way to automatically resubmit the requests when app is resumed from dormant state. This is only from dormant and not from the tombstone state.
The idea I had was to create a wrapper object which subscribes to the Deactivated event before starting the request and rerunning the request in case it received the event.
I assume that since the deactivated event was received, the request failed.
public class RestClientEx
{
bool wasDeactivated = false;
public async Task<T> ExecuteTaskAsync<T>(RestClient client, RestRequest request) where T : new()
{
var phoneApplicationService = App.Current.ApplicationLifetimeObjects.OfType<PhoneApplicationService>().First();
phoneApplicationService.Deactivated += phoneApplicationService_Deactivated;
var t = await client.ExecuteTaskAsync<T>(request);
if (this.wasDeactivated)
{
// resubmit request
this.wasDeactivated = false;
t = await this.ExecuteTaskAsync<T>(client, request);
}
return t;
}
void phoneApplicationService_Deactivated(object sender, DeactivatedEventArgs e)
{
(sender as PhoneApplicationService).Deactivated -= phoneApplicationService_Deactivated;
this.wasDeactivated = true;
}
}
My question is, is there another way to achieve this?
Is it OK what I am doing?

GWT RPC Call onFailure

I created a Remote Procedure Call. The Server-Side connects to Webservices, to get Information, which it hands over to the Client-Side. This is the Client-Side Code.
public void statusFor(GwtLaneServiceAsync laneProxy){
AsyncCallback<LaneInformation> callback = new AsyncCallback<LaneInformation>()
{
#Override
public void onFailure(Throwable caught)
{
}
#Override
public void onSuccess(LaneInformation information)
{
doStatusForSuccess(information);
}
};
for (Lane lane : this.mainmenu.getSlidePanel().getLaneMenu().getProperLanes().values())
{
if (lane.isChecked().booleanValue())
laneProxy.statusFor("admin", "password", true, lane.getId(), callback);
else
laneProxy.statusFor("admin", "password", false, lane.getId(), callback);
this.laneIndex++;
}
}
Now i wanna do the following...
When the Server can't reach the Webservice, a WebServiceException is thrown. If that happens, I wanna type "Offline" on one of my Buttons of the GUI. BUT I need to tell on which button. It can't be hard coded, cause it depends on which "lane" the Webservice failed.
I need to catch the Exceptions
I need to tell the "onFailure"-Part, on which lane, the Service failed.
Can I somehow deliver the statusFor()-Parameters to that part?
There is no of ways to handle such case. you can throw any custom exception from server side while server can't reach the webservice. then it will come onFailure block. or you can return any message string in response variable. Here response variable you are using LaneInformation bean. so take new variable there like result, and set message as per your requirement.
OnFailure it comes only when any exception occurred or any wrong thing happens in RPC call.
Why not wrap your LaneInformation in a generic response object and add the exception/an error code to that response, to signal that something went wrong on the server side, eg.:
public class RemoteResult<T>
{
T payload;
String errorCode;
}
and
public abstract class AbstractAsyncCallBack<T> implements AsyncCallback<RemoteResult<T>>
{
public void onSuccess( RemoteResult<T> rr )
{
if ( rr.getErrrorCode() != null ) { failure( rr.getErrorCode() ); }
else { success( rr.getPayload() ); }
}
public abstract void success( T payload );
public void failure( String errorCode ) { /* Ignore by default */ }
}
To conclude, you shouldn't throw an exception on the server side when the server can't connect to some other service, you should communicate that nicely to the client, and that's not by (re)throwing the exception :-)
The onFailure() method is mostly for when things go wrong in the RPC communication proper.
Cheers,

Jade Agent Containers

Can anyone tell me how to find available agent containers through java code? I am using the JADE agent framework and I have figured out how to create new containers but not find existing containers (so that agents can be deployed in them).
There are two ways of doing this, depending on whether you want to receive the information via an ongoing service or the current snapshot in a message.
To get a snapshot of the IDs of the currently available agent containers, send a Request message to the Agent Management Service (AMS) and wait for its reply. Using the JADE Management Ontology and the QueryPlatformLocationsAction term, the sending and receiving methods would be:
private void queryAMS() throws CodecException, OntologyException {
QueryPlatformLocationsAction query = new QueryPlatformLocationsAction();
Action action = new Action(myAgent.getAID(), query);
ACLMessage message = new ACLMessage(ACLMessage.REQUEST);
message.addReceiver(myAgent.getAMS());
message.setLanguage(FIPANames.ContentLanguage.FIPA_SL);
message.setOntology(JADEManagementOntology.getInstance().getName());
myAgent.getContentManager().fillContent(message, action);
myAgent.send(message);
}
private void listenForAMSReply() throws UngroundedException, CodecException,
OntologyException {
ACLMessage receivedMessage = myAgent.blockingReceive(MessageTemplate
.MatchSender(myAgent.getAMS()));
ContentElement content = myAgent.getContentManager().extractContent(
receivedMessage);
// received message is a Result object, whose Value field is a List of
// ContainerIDs
Result result = (Result) content;
List listOfPlatforms = (List) result.getValue();
// use it
Iterator iter = listOfPlatforms.iterator();
while (iter.hasNext()) {
ContainerID next = (ContainerID) iter.next();
System.out.println(next.getID());
}
}
To get this information as an ongoing service, and to receive the ContainerID of each container as it registers with the AMS, create a Behaviour that extends the AMSSubscriber. Register a handler for the AddedContainer event and you will be able to access the ContainerID of the newly available container:
public class AMSListenerBehaviour extends AMSSubscriber {
#Override
public void installHandlers(Map handlersTable) {
handlersTable.put(AddedContainer.NAME, addedContainerHandler);
}
public final class AddedContainerHandler implements EventHandler {
#Override
public void handle(Event ev) {
AddedContainer event = (AddedContainer) ev;
ContainerID addedContainer = event.getContainer();
System.out.println(addedContainer.getID());
}
Hope this helps,
Russ