I want to be able to catch unretryable exceptions when sending a message through kafkaTemplate and then send the message to a different error topic where I can investigate the message.
How it's easier to implement such a behavior on consumer side than in producer side ?
I understand that for some exceptions like RecordTooLargeException the message could not be send to the new topic either. But for these kind of messages I will log it to the system as warning.
I don't know how to approach this problem and what Kafka actually offer as functionality for this kind of handling on the producer side.
Related
We have a requirement is to build spring boot command line applicarion where we have to send messages to queue.
Only request queue has been setup.
As there is no response queue setup, we are not getting any acknowledgement from client side if they receive a message or not.
Right now I am using Spring's JMSTemplate send() method to send message to request queue and SingleConnectionFactory to create one shared connection as this is commmand line application
As there is no acknowledgement/response to message we send to request queue, End to end testing is difficult.
If destination/request queue connection is obtained and message is sent without any exception, I consider it a successful test.
Is it a right to implement Spring JMS templates send() method only ? and not following jms template send/receive pattern
Note: It is not possible to setup a response queue and get any acknowledgement from client side.
In JMS (and in most other messaging systems) producers and consumers are logically separated (i.e. de-coupled). This is part of the fundamental design of the system to reduce complexity and increase scalability. With these constraints your producers shouldn't care whether or not the message is consumed. The producers simply send messages. Likewise, the consumers shouldn't care who sends the messages or how often, etc. Their job is simply to consume the messages.
Assuming your application is actually doing something with the message (i.e. there is some kind of functional output of message processing) then that is what your end-to-end test should measure. If you get the ultimate result you're looking for then you may deduce that the steps in between (e.g. sending a message, receiving a message, etc.) were completed successfully.
To be clear, it's perfectly fine to send a message with Spring's JMSTemplate without using a request/response pattern. Generally speaking, if you get no exceptions then that means the message was sent successfully. However, there are other caveats when using JMSTemplate. For example, Spring's JavaDoc says this:
The ConnectionFactory used with this template should return pooled Connections (or a single shared Connection) as well as pooled Sessions and MessageProducers. Otherwise, performance of ad-hoc JMS operations is going to suffer.
That said, it's important to understand the behavior of your specific JMS client implementation. Many implementations will send non-persistent JMS messages asynchronously (i.e. fire and forget) which means they may not make it to the broker and no exception will be thrown on the client. Sending persistent messages is generally sufficient to guarantee that the client will throw an exception in the event of any problem, but consult your client implementation documentation to confirm.
I am a newbie in grails and trying to find a best way for exception handling in Grails.
I am using Grails 2.4.2 with Spring Secxurity Core.
Following are the scenarios that needs to be considered.
Call to the Grails app could be Web Call/Ajax call/ RestFul call.
Response type expected could be a JSON/TEXT/XML/HTML
Exceptions could occur in any layer of the application(Filters, COntrollers, Service, Domain, TagLib, Database)
Proper HTTP Status code has to be set for the exception. (Could be same for all exceptions)
I want to have a least impact on the coding since grails reduces the boiler plate code a lot, i don't want to end up writing redundant codes in each controller.
Consider the above scenario's what could be the best possible approach on this.
Based on the format type( Content-Negotiation in Grails we could determine the Response Type that is to be provided). I was looking at the traits way of adding exception handling in the controller by implementing the traits for all controllers as defined in the Grails User Documentation. What would happen to the security exceptions that is being thrown by the Spring-Security.
When i make Ajax calls, I believe the error call back on the ajax client is determined by the status code of the HTTP Response code.
What I need is a way to forward any exceptions that occur to a generic error controller which should have access to the root cause, any messages associated with it and construct a HTML/TEXT/JSON/XML response out of it and render it to the client.
Is it possible to achieve this. Any inputs are highly appreciated.
Thanks in advance
I've been trying to read more about what to do properly catching / handling exceptions, but I don't think I've got it down. In fact, I think I'm getting much more confused and possibly implementing bad code. I don't want to do that.
An example setup that I have been using:
Mobile device makes a call to the WCF Service.
WCF Service retrieves the data from the database, and if any errors occur on the database level, they are logged and I am sent an e-mail.
WCF Service sends data (or a brief description of the exception) to the mobile device.
The mobile device processes the data, and if any error occurs, throws the error up to the UI layer.
For a few of the exceptions, I created custom ones - service exception, authorization exception, so I can properly notify the user. If the service encountered an error or an IOException occurs, the user will be notified that 'the data could not be retrieved.'
If, however, another error occurs - such as a JSON error, or anything like that 'just in case', the error is thrown to the UI layer and simply caught as Exception, since we don't really need to user to know what happened, but that an error occurred.
Is this appropriate exception handling?
Are you seeing any problems?
In general, it makes sense to have some sort of catch-all that allows the user to keep working. This should be combined with appropriate handling for any showstoppers, to let the user down gracefully, and catch anything else that would make proceeding dangerous.
"Appropriate exception handling" is always going to be a) application dependent and b) subjective - so there's no definitive answer.
In general I would say you need to do all of the following:
Specifically address and handle appropriately all likely exceptions.
Provided a catch all to prevent a non-graceful termination.
Notify the user of unexpected errors if there is potential it will effect
their data or usage (i.e. - don't mask errors that might impact user)
Sounds like you've done this so I believe you have a reasonable approach in place.
I have built several Mule processes that consume messages from jms queues (ActiveMQ). Whenever a Mule component throws an exception, the transaction that consumes the messages rollback and the message gets redelivered to the original queue. After a few tries, it will be sent to a dead letter queue (DLQ.queuName).
We have this working OK, but we are missing the exception thrown, either the first one or the last one, we don't care (it'll probably be the same). This is something that can be done on other brokers (like WebLogic JMS), but I've been struggling with this for a while to no avail.
Does anybody know if this is something that can be configured or do I need to build a specific Mule exception handler or policy for ActiveMQ.
TIA,
Martin
That exception is lost in ActiveMQ at the moment (don't know about Mule) but it is reported to the log as error.
It would make a good enhancement, remembering the string form of the exception in the ActiveMQConsumer and passing it back to the broker with the poison Ack that forces it to go to
the DLQ. In that way, it could be remembered as a message property in the resulting DLQ message.
How would you like to handle the exception, have it reported to a connection exception listener or have it recorded in the DLQ message?
What are the best practices for exceptions over remote methods?
I'm sure that you need to handle all exceptions at the level of a remote method implementation, because you need to log it on the server side. But what should you do afterwards?
Should you wrap the exception in a RemoteException (java) and throw it to the client? This would mean that the client would have to import all exceptions that could be thrown. Would it be better to throw a new custom exception with fewer details? Because the client won't need to know all the details of what went wrong. What should you log on the client? I've even heard of using return codes(for efficiency maybe?) to tell the caller about what happened.
The important thing to keep in mind, is that the client must be informed of what went wrong. A generic answer of "Something failed" or no notification at all is unacceptable. And what about runtime (unchecked) exceptions?
It seems like you want to be able to differentiate if the failure was due to a system failure (e.g. a service or machine is down) or a business logic failure (e.g. the user does not exist).
I'd recommend wrapping all system exceptions from the RMI call with your own custom exception. You can still maintain the information in the exception by passing it to your custom exception as the cause (this is possible in Java, not sure about other languages). That way client only need to know how to handle the one exception in the cause of system failure. Whether this custom exception is checked or runtime is up for debate (probably depends on your project standards). I would definitely log this type of failure.
Business type failures can be represented as either a separate exception or some type of default (or null) response object. I would attempt to recover (i.e. take some alternative action) from this type of failure and log only if the recovery fails.
In past projects we'd catch all service layer (tier) exceptions at the very top of the layer, passing the application specific error codes/information to the UI via DTO's/VO's. It's a simple approach in that there's an established pattern of all error handling happening in the same place for each service instead of scattered about the service and UI layers.
Then all the UI has to do is inspect the DTO/VO for a flag (hasError?) and display the error message(s), it doesn't have to know nor care what the actual exception was.
I would always log the exception within my application (at the server side as defined in your question).
I would then throw an exception, to be caught by the client. If the caller could take corrective action to prevent the exception then I would ensure that the exception contained this information (e.g. DateTime argName must not be in the past). If the error was caused by some outage of a third party system then I might pass this information up the call stack to the caller.
If, however, the exception was essentially caused by a bug in my system then I would structure my exception handling such that a non-informative exception message (e.g. General failure) was used.
Here's what I did. Every Remote Method implementation catches all Exceptions on the server side and logs them. Then they are wrapped in a Custom Exception, which will contain a description of the problem. This description must be useful to the client, so it won't contain all the details of the caught Exception, because the client doesn't need them. They have already been logged on the server side. Now, on the client, these Exceptions can be handled how the user wishes.
Why I chose using Exceptions and not return codes is because of one very important drawback of return codes: you can't throw them to higher levels without some effort. This means you have to check for an error right after the call and handle it there. But this may not be what I want.