How to pause message processing for a certain period? - esb

We use Apache Camel in Talend ESB Studio v6.4
In an ESB route, we consume JMS messages, process them then send them to an HTTP server. But that target server is down for maintainance every saturday from 6pm to 10pm.
How can we "pause" message consuming or message processing during that period ? I think quartz only works with file/ftp endpoints.
We could use a Processor component to check in Java if we are in the down period, but what to do after that ?

There are several ways to do this. One camel specific way to do it is through CamelControlBus. It takes in a routeId and performs an action (start/stop/resume etc) on it - Read more here to get an understanding Camel ControlBus
However, there is another approach that you can take. You can create a POJO bean that has 3 methods
shouldRouteStop() : to check the current time and decide if it should stop your route.
startRoute() : Starts a route if it is suspended
stopRoute() : Suspends a route if it is started
A simple implementation can be as follows:
public class ManagementBean {
public boolean shouldRouteStop() {
// Mocking the decision here
return new Random().nextBoolean();
}
public void startRoute(org.apache.camel.CamelContext ctx) throws Exception {
if (ctx.getRouteStatus("GenerateInvoices") == ServiceStatus.Suspended)
// replace the argument with your route Id
ctx.resumeRoute("GenerateInvoices");
}
public void stopRoute(org.apache.camel.CamelContext ctx) throws Exception {
if (ctx.getRouteStatus("GenerateInvoices") == ServiceStatus.Started)
// replace the argument with your route Id
ctx.suspendRoute("GenerateInvoices");
}
}
Make sure that the jms-route that you wish to control has a routeId and add this bean to your base/default CamelContext like this
main.bind("manageRouteBean", new ManagementBean());
Create another timer based route, that checks on each tick, if the route should be stopped or not and then suspends or resumes the route by routeId. This route can be implemented like below:
public class MonitoringRoute extends RouteBuilder {
#Override
public void configure() throws Exception {
onException(Exception.class).log(exceptionMessage().toString());
from("timer:time?period=10000")
.choice()
.when().simple("${bean:manageRouteBean?method=shouldRouteStop}")
.log("Route Should Stop")
.bean(ManagementBean.class, "stopRoute(*)")
.otherwise()
.log("Route Should Start")
.bean(ManagementBean.class, "startRoute(*)")
.end();
}
}
Note that startRoute and stopRoute take the argument as *. This is camel way of automatically binding parameters based on type.
Finally, you can add this route to the main camel context like : main.addRouteBuilder(new MonitoringRoute());
For a complete implementation, have a look at this github repo

Related

ExceptionConsumeContext filter in MassTransit

in this post I found it that how to send my Serilog enrichment properties to consumers. when I logging my informations, then every thing is correct. but when throws an exception in the consumers, I don't see my Serilog properties in the log file. as you can see:
ERR - - -
2022-01-03 12:25:40.346 - myApp - MassTransit.ReceiveTransport - ERR - - - => R-FAULT "rabbitmq://localhost/MyConsumer" "c8100000-568d-0050-407a-08d9ce96c99c"
well, I guess the exception logging occurred in another context. I guess the exception logging occurred in the ExceptionConsumeContext class.
well I Implemented a filter for ExceptionConsumeContext:
public class IntegrationEventExceptionConsumeFilter<T> : IFilter<ExceptionConsumeContext <T>> where T : class {
public IntegrationEventPublishFilter()
{
}
public Task Send(ExceptionConsumeContext <T> context, IPipe<ExceptionConsumeContext <T>> next)
{
return next.Send(context);
}
public void Probe(ProbeContext context)
{
}}
now I need to add this filter to MassTransit configuration:
cfg.UseConsumeFilter(typeof(IntegrationEventExceptionConsumeFilter<>), context);
well, I got the exception:
The scoped filter must implement GreenPipes.IFilter<MassTransit.ConsumeContext<MyEvent>> (Parameter 'scopedType')
well, I couldn't find any conumer filter registeration for type of ExceptionConsumeContext.
and Since the I saw ExceptionConsumeContext inherited from ConsumeContext, I guess can be register this filter as a ConsumeFilter!. but don't work this way.
public interface ExceptionConsumeContext : ConsumeContext, PipeContext, MessageContext, IPublishEndpoint, IPublishObserverConnector, ISendEndpointProvider, ISendObserverConnector{}
now, I don't know what I do!
There is no scoped filter registration for ExceptionConsumeContext. You would need to add your additional filter to the receive pipeline.
cfg.ConfigureError(x =>
{
x.UseFilter(new GenerateFaultFilter());
x.UseFilter(new IntegrationEventExceptionConsumeFilter());
x.UseFilter(new ErrorTransportFilter());
});
Note that there is no message-specific generic version of ExceptionConsumeContext.

Should my Domain Exceptions be thrown from Application Layer?

I'm reading Vaughn Vernon Book - Implementing Domain Driven Design. There is an example of a Project Management Application. There are aggregates like BacklogItem, Sprint, etc. If I have BacklogItemNotFoundException defined in Domain layer. Should my Rest adapter catch it and transform into NotFoundHttpResult? Or any other broken invariant exceptions like: EmailPatternBrokenException or TooManyCharactersForNameException or whatever should be handled in Rest adapter(ports&adapters architecture) and re-transformed into rest responses? If yes, it means that RestAdapter should have a reference to Domain layer? This is what bothers me...
The question is a contradiction. If it is a Domain Exception, it means that it is thrown by the domain.
Anyway, exceptions thrown by the domain should be handled by the application layer.
I have an exception handler decorator for the command bus, that catch any domain exception and translates it into an Application Exception.
This application exception is thrown to the adapters.
Adapters know about application exceptions, not domain exceptions.
UPDATE
My domain exception is an abstract base class from which the concrte domain exceptions inherit
public abstract class DomainException extends RuntimeException {
private static final long serialVersionUID = 1L;
private ErrorMessage mainErrorMessage;
private List<ErrorMessage> detailErrorMessages;
protected DomainException ( List<ErrorMessage> aDetailMessages, Object... aMainMessageArgs ) {
this.mainErrorMessage = new ErrorMessage(this.getClass().getSimpleName(), aMainMessageArgs );
this.detailErrorMessages = ( (aDetailMessages==null) ? new ArrayList<ErrorMessage>() : aDetailMessages );
}
public ErrorMessage mainErrorMessage() {
return this.mainErrorMessage;
}
public List<ErrorMessage> detailErrorMessages() {
return this.detailErrorMessages;
}
}
ErrorMessage has a key and a list of args. The messages are in a property file where the key is the name of the concrete domain exception class.
Application exception is just one type, which holds the concrete text message.
public class ApplicationException extends Exception {
private static final long serialVersionUID = 1L;
private String mainMessage;
private String[] detailMessages = new String[0];
public ApplicationException ( String aMainMessage, Throwable aCause, String... aDetailMessages ) {
super ("Main Message = "+aMainMessage+" - DetailMessages = "+Utils.toString(aDetailMessages), aCause );
this.mainMessage = aMainMessage;
this.detailMessages = ( (aDetailMessages==null) ? (new String[0]) : aDetailMessages );
}
public String mainMessage() {
return this.mainMessage;
}
public boolean hasDetailMessages() {
return (this.detailMessages.length > 0);
}
public String[] detailMessages() {
return this.detailMessages;
}
}
I have a decorator (wraps the execution of every command) for handling domain exceptions:
public class DomainExceptionHandlerDecorator extends Decorator {
private final DomainExceptionHandler domainExceptionHandler;
public DomainExceptionHandlerDecorator (DomainExceptionHandler domainExceptionHandler) {
this.domainExceptionHandler = domainExceptionHandler;
}
#Override
public <C extends Command> void decorateCommand(Mediator mediator, C command) throws ApplicationException {
try {
mediator.executeCommand(command);
} catch ( DomainException de ) {
this.domainExceptionHandler.handle (de);
}
}
}
And I have a domain exception handler that takes a domain exception, translates it into an app exception by reading properties file (TextMessageService does the job) and throw the app exception.
public class TranslatorDomainExceptionHandler implements DomainExceptionHandler {
private final TextMessageService configurationService;
public TranslatorDomainExceptionHandler ( TextMessageService aConfigurationService ) {
this.configurationService = aConfigurationService;
}
#Override
public void handle ( DomainException de ) throws ApplicationException {
ErrorMessage mainErrorMessage = de.mainErrorMessage();
List<ErrorMessage> detailErrorMessages = de.detailErrorMessages();
String mainMessage = this.configurationService.mensajeDeError ( mainErrorMessage );
String[] detailMessages = new String [ detailErrorMessages.size() ];
int i = 0;
for ( ErrorMessage aDetailErrorMessage : detailErrorMessages ) {
detailMessages[i] = this.configurationService.mensajeDeError ( aDetailErrorMessage );
i++;
}
throw new ApplicationException ( mainMessage, de, detailMessages);
}
}
The adapter (an UI for example) will catch the app exception and show its message to the user. But it doesn't know about domain exceptions.
I try to avoid domain exceptions as much as I can and prefer to make invalid states unreachable instead. The first reason is that exceptions are for exceptional, unexpected things, the second that I don't like my code to be cluttered with fine-grained try/catches for every little business-ish thing that could go wrong.
BacklogItemNotFoundException
To me this is typically your Repository or query service returning null or an empty list. No need for a domain exception.
EmailPatternBrokenException
TooManyCharactersForNameException
I let the validation feature of my web framework handle these. You could also check it in the Domain but it will rarely reach that point and you don't really need to handle that kind of error specifically.
As a result, the two typical scenarios are:
+-----------------------+--------------------+-------------------------------------------------+
| Domain | Application | Presentation |
+-----------------------+--------------------+-------------------------------------------------+
| Expected failure case | Return Result.Fail | Clean error message |
+-----------------------+--------------------+-------------------------------------------------+
| Exception | - | Caught in catch-all clause > 500 error or other |
+-----------------------+--------------------+-------------------------------------------------+
I will add my 2 cents about error handling, not specifically related to DDD.
The exception are part of the contract you expose to the consumer. If you're expected to for example add an item to a shopping cart, the exception you may explicitly throw include itemNotAvailable, shoppingCartNotExisting, etc...
Technical exception on the other hand are not part of the contract, they may occurs but shouldn't be explicitly handled as no one can do anything about it, they must imply the operation interruption (and the rollback of the current unit of work).
A rest interface is a contract for an operation on a resource. When using rest over http the terms of the contract are related to the http protocol.
Typical operation described above (adding ie. post an item on a cart resource) would be translated to, for example, 404 for shoppingCartNotExisting and 409 for itemNotAvailable (conflict ie. the update on the resource is no more possible because some state has changed meantime).
So yes all "domain" exception (expected exceptions as part of the contract) should be explicitly mapped by the rest adapter, all unchecked ones should result in a 500 error.
TLDR; It is OK if the Application or Presentation layer has a dependency to the Domain layer, the other way is not recommended.
Idealy, there should not exist any dependency from one layer to another but that is impossible or the software would not be usable. Instead you should try to minimize the number and the direction of the dependencies. The general rule or best practice to a clean architecture is to keep the Domain layer agnostic of the infrastructure or the Application layer. The Domain objects (Aggregates, Value objects etc) should not care about a specific persistence or Rest or HTTP or MVC, just like the domain experts don't care about these things.
In real world, the Domain layer may be influenced by technology (like frameworks). For example we put annotations to mark some Domain objects as behaving in some specific way when persisted instead of using external XML or JSON files just because it is at hand, it is easier to maintain them. We need, however, to limit these influences to a minimum.
The application layer is business-specific domain itself. So your application layer should handle the domain exception based on what the application/business expects. The application(eg. client facing web application, mobile, an internal CRM app, or a backend-for-frontend API) is probably not the only client of the domain layer(eg. a rest api, a jar library). There might be certain domain exceptions that you don't want to expose to the end-user so the application has to wrap these exceptions specifically or handle exceptions globally.

Hystrix/Feign to solely react on HTTP status 429

I'm using Feign from the spring-cloud-starter-feign to send requests to a defined backend. I would like to use Hystrix as a circuit-breaker but for only one type of use-case: If the backend responds with a HTTP 429: Too many requests code, my Feign client should wait exactly one hour until it contacts the real backend again. Until then, a fallback method should be executed.
How would I have to configure my Spring Boot (1.5.10) application in order to accomplish that? I see many configuration possibilities but only few examples which are - in my opinion - unfortunately not resolved around use-cases.
This can be achieved by defining an ErrorDecoder and taking manual control of the Hystrix Circuit Breaker. You can inspect the response codes from the exceptions and provide your own fallback. In addition, if you wish to retry the request, wrap and throw your exception in a RetryException.
To meet your Retry requirement, also register a Retryer bean with the appropriate configuration. Keep in mind that using a Retryer will tie up a thread for the duration. The default implementation of Retryer does use an exponential backoff policy as well.
Here is an example ErrorDecoder taken from the OpenFeign documentation:
public class StashErrorDecoder implements ErrorDecoder {
#Override
public Exception decode(String methodKey, Response response) {
if (response.status() >= 400 && response.status() <= 499) {
return new StashClientException(
response.status(),
response.reason()
);
}
if (response.status() >= 500 && response.status() <= 599) {
return new StashServerException(
response.status(),
response.reason()
);
}
return errorStatus(methodKey, response);
}
}
In your case, you would react to 419 as desired.
You can forcibly open the Circuit Breaker setting this property at runtime
hystrix.command.HystrixCommandKey.circuitBreaker.forceOpen
ConfigurationManager.getConfigInstance()
.setProperty(
"hystrix.command.HystrixCommandKey.circuitBreaker.forceOpen", true);
Replace HystrixCommandKey with your own command. You will need to restore this circuit breaker back to closed after the desired time.
I could solve it with the following adjustments:
Properties in application.yml:
hystrix.command.commandKey:
execution.isolation.thread.timeoutInMilliseconds: 10_000
metrics.rollingStats.timeInMilliseconds: 10_000
circuitBreaker:
errorThresholdPercentage: 1
requestVolumeThreshold: 1
sleepWindowInMilliseconds: 3_600_000
Code in the respective Java class:
#HystrixCommand(fallbackMethod = "fallbackMethod", commandKey = COMMAND_KEY)
public void doCall(String parameter) {
try {
feignClient.doCall(parameter);
} catch (FeignException e) {
if (e.status() == 429) {
throw new TooManyRequestsException(e.getMessage());
}
}
}

How can I manually handle any subscribed to message type in NServiceBus?

I'm trying to build a layer over NServiceBus to make it simpler for other developers to use.
I'm trying to do without the config file and managed to get the publisher to work:
public class NServiceBusPublisher
{
private IBus _Bus { get; set; }
public void NServiceBusPublisher(string argInputQueue, string argErrorQueue)
{
Configure configure = NServiceBus.Configure.With().DefaultBuilder();
var transport = configure.Configurer.ConfigureComponent<MsmqTransport>(ComponentCallModelEnum.Singleton);
transport.ConfigureProperty(t => t.InputQueue, argInputQueue);
transport.ConfigureProperty(t => t.ErrorQueue, argErrorQueue);
transport.ConfigureProperty(t => t.NumberOfWorkerThreads, 1);
transport.ConfigureProperty(t => t.MaxRetries, 5);
_Bus =
configure
.XmlSerializer()
.MsmqTransport()
.IsTransactional(true)
.PurgeOnStartup(false)
.MsmqSubscriptionStorage()
.UnicastBus()
.ImpersonateSender(false)
.CreateBus()
.Start();
}
public void Publish(NServiceBus.IMessage argMessage)
{
_Bus.Publish(argMessage);
}
}
I also want to have an NServiceBus Subscriber and make it possible for developers to subscribe to any number of message types as long as the message inherits from NServiceBus.IMessage:
public class NServiceBusSubscriber
{
private IBus _Bus { get; set; }
public void NServiceBusSubscriber(string argInputQueue, string argOutputQueue, string argErrorQueue, string messagesAssembly)
{
Configure configure = NServiceBus.Configure.With().DefaultBuilder();
var transport = configure.Configurer.ConfigureComponent<MsmqTransport>(ComponentCallModelEnum.Singleton);
transport.ConfigureProperty(t => t.InputQueue, argInputQueue);
transport.ConfigureProperty(t => t.ErrorQueue, argErrorQueue);
transport.ConfigureProperty(t => t.NumberOfWorkerThreads, 1);
transport.ConfigureProperty(t => t.MaxRetries, 5);
var ucb = configure.Configurer.ConfigureComponent<NServiceBus.Unicast.UnicastBus>(ComponentCallModelEnum.Singleton);
ucb.ConfigureProperty(u => u.MessageOwners, new Dictionary<string,string>()
{
{messagesAssembly, argOutputQueue}
});
_Bus =
configure
.XmlSerializer()
.MsmqTransport()
.IsTransactional(true)
.PurgeOnStartup(false)
.MsmqSubscriptionStorage()
.UnicastBus()
.ImpersonateSender(false)
.DoNotAutoSubscribe()
.CreateBus()
.Start();
}
public void Subscribe<T>() where T : NServiceBus.IMessage
{
_Bus.Subscribe<T>();
}
}
The problem is that I couldn't find any way to attach an event handler to a particular message type.
Could you please help me figure this out?
Its been a while since the question has been asked, so I am not sure if the problem has been solved, but here's one way you can do it using Bus.Subscribe (although as has been said by other respondents this is not the prescribed way of doing it NServiceBus)
Subscribe the to the message type using the subscribe overload
void Subscribe(Type messageType, Predicate<IMessage> condition);
Then you can handle the message in the delegate
private bool Handle(NServiceBus.IMessage nsbMsg)
{
//you get the message instance that you can handle
//return true
}
So, your code would then be
class MySubscriber
{
public IBus Bus {get; set;}
public void Subscribe()
{
Bus.Subscribe(typeof(MyMessage), Handle);
}
public void Handle(NServiceBus.IMessage nsbMsg)
{
var msg = nsbMsg as MyMessage;
//your code
return true;
}
}
However please note that by doing this you have to manage the lifetime of the handler yourself, which otherwise would have been managed for you by NServiceBus using the IOC framework of your choice.
You will also have to pass the reference to IBus explicitly which would be injected for you automatically if you were just implementing the IHandleMessage interface.
An architectural point here is that NSB is a full fledged 'ESB', its not just a messaging layer. Adding another layer over your ESB is IMHO an abstraction too many.
I think you are missing the concept behind NServiceBus.
Based on the code you show I get the impression that you envision services that publish messages and others that process those messages. In my experience most processes do both: they subscribe to events or process incoming commands and in result publish new events and send new commands.
In your setup you would need to have publisher and subscriber instances for each of these message types.
NServiceBus is built for the situation I describe. You configure and start 1 bus instance and that orchestrates the complete application.
If you want to make it easier for developers to use NServiceBus I would concentrate on the configuration part only. In our company I have created a ServicebusConfigurator class that configures NServiceBus according our company standards and extracted that in a framework and a simple extension method for the .NET Core generic host. The only code our developers need to write to create a Windows Service that hosts an NServiceBus endpoint is something like this:
internal static class Program
{
private static int Main(string[] args)
{
return (int)Host.CreateDefaultBuilder(args) //.NET Core generic host
.WithNServiceBus() //configure NServiceBus according to our standards and start it.
.UseTopshelf<Worker>() // use Worker as the actual service doing the work.
.EnableNsbInstallersDuringInstall() // Execute any NServiceBus transport specific installation code during install of the service.
.Run(); // Run the thing.
}
}
Since you are not auto-subscribing the first thing you will need to do is subscribe to the message type via Bus.Subscribe(). Others could do this at the IWantToRunAtStartUp extension point(implement the interface in a class somewhere). From there, each subscriber will implement the IHandleMessages<T> interface. Implementing this interface wires you to a message where "T" is the message type.
When NSB starts up it will scan the local bin dir and find all your interface implementations and wire them up on your behalf internally. From there it will dispatch to the correct handler when a message of that type arrives.
NServiceBus automatically handles the subscription of messages. When you invoke Configure.With()....Start(); NServiceBus will scan to determine which assemblies implement IHandleMessages(SomeMessage) and it will send a subscription request to the publisher.
When you add "DoNotAutoSubscribe", you've got to manually get all messages being handled and do a Bus.Subscribe() for each of them.
Beyond that, NServiceBus will automatically handle the routing of an incoming message to the appropriate handler. In your subscriber code above, are you receiving an error message or are the messages disappearing from the queue?

How to register a uri dependency to return HttpContext.Current.Request.Url using Castle Windsor?

I'm new to Castle Windsor, so go easy!!
I am developing an MVC web app and one of my controllers has a dependency on knowing the current request Url.
So in my Application_Start I initialise a WindsorContainer (container below), register my controllers and then try the following...
container.AddFacility<FactorySupportFacility>();
container.Register(Component.For<Uri>().LifeStyle.PerWebRequest.UsingFactoryMethod(() => HttpContext.Current.Request.Url));
However when I run up my web app I get an exception that my controller...
is waiting for the following dependencies:
Keys (components with specific keys)
- uri which was not registered.
The controller it is trying to instantiate has the following signature:
public MyController(Uri uri)
For some reason it is not running my factory method?
However if I change the controller signature to:
public MyController(HttpContext httpContext)
and change the registration to:
container.Register(Component.For<HttpContext>().LifeStyle.PerWebRequest.UsingFactoryMethod(() => HttpContext.Current));
Then everything works a treat!!
What am I missing when trying to register a Uri type? Its seems exactly the same concept to me? I must be missing something!?
Updated:
I have done some more debugging and have registered both the Uri and the HttpContext using the factory methods shown above. I have added both types as parameters on my Controller constructor.
So to clarify I have a both Uri and HttpContext types registered and both using the FactoryMethods to return the relevant types from the current HttpContext at runtime. I also have registered my controller that has a dependency on these types.
I have then added a breakpoint after I have registration and have taken a look at the GraphNodes on the kernal as it looks like it stores all the dependencies. Here it is:
[0]: {EveryPage.Web.Controllers.BaseController} / {EveryPage.Web.Controllers.BaseController}
[1]: {EveryPage.Web.Controllers.WebpagesController} / {EveryPage.Web.Controllers.WebpagesController}
[2]: {System.Web.HttpContext} / {System.Web.HttpContext}
[3]: {Castle.MicroKernel.Registration.GenericFactory1[System.Web.HttpContext]} / {Castle.MicroKernel.Registration.GenericFactory1[System.Web.HttpContext]}
[4]: {System.Uri} / {System.Uri}
[5]: {Castle.MicroKernel.Registration.GenericFactory1[System.Uri]} / {Castle.MicroKernel.Registration.GenericFactory1[System.Uri]}
It looks as though it has registered my Controller and both the types, plus it has the Factories. Cool.
Now if I drill into the WebpagesController and take a look at its dependencies it only has 1 registered:
[0]: {System.Web.HttpContext} / {System.Web.HttpContext}
Now shouldn't this have 2 registered dependencies as it takes a HttpContext and Uri on its constructor??
Any ideas? Am I barking up the wrong tree?
UPDATE3:
There's new extension point in Windsor trunk now that you can use easily for that.
UPDATE2:
Turns out that I was right from the start (well kind of). Uri is a class, but Windsor treats it as a primitive. There are still at least two quick solutions to this:
Wrap the Uri in some kind of IHasUri or something and take dependency on that interface in your controller
public class FooController
{
public IHasUri CurrentUri { get; set; }
public void SomeAction()
{
var currentUri = CurrentUri.GetCurrentUri();
// do something with the uri
}
}
Tell the Windsor you don't want it to treat Uris like some primitive (but like a lady).
You need a IContributeComponentModelConstruction implementation for that:
public class UriIsAServiceNotAParameter:IContributeComponentModelConstruction
{
public void ProcessModel(IKernel kernel, ComponentModel model)
{
if (model.Service != typeof(UsesUri)) // your controller type here
return;
foreach (var constructor in model.Constructors)
{
foreach (var dependency in constructor.Dependencies)
{
if(dependency.TargetType ==typeof(Uri))
{
dependency.DependencyType = DependencyType.Service;
}
}
}
}
}
and add it to the container:
container.Kernel.ComponentModelBuilder.AddContributor(new UriIsAServiceNotAParameter());
There's also the most correct way of doing this, which means telling Windsor not to register Uris as primitives in the first place, rather than fixing this afterwards, but this would require reaching into the deepest guts of the kernel, and the result is far more code (though a straightforwad one) than the workarounds outlined above.