Related
I understand, Assertions are used in tests to check if the programmer's pre- and post- conditions are true and if the assertions fail, there is/are bug/bugs that need to debugged and fixed.
An assertion is a software construct where the developer states ("asserts") a condition that he believes will always be true. If the condition evaluates to false in some languages an exception is thrown, in others a message is printed, and in others the program ceases to operate.
"A message is printed" highlighted in the above definition caught my attention.
My question is, can we extend the use of assert statements in the actual code itself and catch AssertionErrors to log the messages and not just tests?
Basically to avoid too much verbose with if-else statements(I hate if-else btw, personally).
For example, all of below
String response = aMethod(sendSomething);
if (response!=null && response!="") {
//blah blah
} else {
log.error("Response is null");
throw new NoDataFoundException();
}
could be replaced with the below that provides more detailed messages.
try{
assertThat(aMethod(sendSomething), notNullValue());
} catch (AssertionError e) {
log.error(e.getMessage());
}
You don't want to catch an assertion failure(*). By definition, the program is operating outside its design perimeter. The only safe thing to do is get out.
(*) OK, you can catch it if the point of catching it is to log whatever you anticipating you'll need to diagnose the problem, and then get out quickly.
In your example, it appears that after testing for a response and logging the failure, you'll carry on inline. In general, that would be a bad idea, but perhaps this was just a quick illustration on your part and should not be taken too literally.
As to what I take to be your overall point: yes, if you're checking pre/postconditions at all, leave the checks in shipping code.
My code would look like:
response = aMethod(sendSomething);
if (response == null || response.isEmpty())
throw new InternalError("....");
and there would be something catching the error at some outermost layer, a point of exception handling being that you can transfer control far away rather than having to deal with everything at the point it occurs -- especially if "it can't happen".
I am new to ReactiveX and reactive programming in general. I need to implement a retry mechanism for Couchbase CAS operations, but the example on the Couchbase website shows a retryWhen which seems to retry indefinitely. I need to have a retry limit and retry count somewhere in there.
The simple retry() would work, since it accepts a retryLimit, but I don't want it to retry on every exception, only on CASMismatchException.
Any ideas? I'm using the RxJava library.
In addition to what Simon Basle said, here is a quick version with linear backoff:
.retryWhen(notification ->
notification
.zipWith(Observable.range(1, 5), Tuple::create)
.flatMap(att ->
att.value2() == 3 ? Observable.error(att.value1()) : Observable.timer(att.value2(), TimeUnit.SECONDS)
)
)
Note that "att" here is a tuple which consists of both the throwable and the number of retries, so you can very specifically implement a return logic based on those two params.
If you want to learn even more, you can peek at the resilient doc I'm currently writing: https://gist.github.com/daschl/db9fcc9d2b932115b679#retry-with-delay
retryWhen is clearly a little bit more complicated than simple retry, but here's the gist of it:
you pass a notificationHandler function to retryWhen which takes an Observable<Throwable> and outputs an Observable<?>
the emission of this returned Observable determine when retry should occur or stop
so, for each occurring Exception in the original stream, if the handler's one emits 1 item, there'll be 1 retry. If it emits 2 items, there'll be 2...
as soon as the handler's stream emits an error, retry is aborted.
Using this, you can both:
work only on CasMismatchExceptions: just have your function return an Observable.error(t) in other cases
retry only for a specific number of times: for each exception, flatMap from an Observable.range representing the max number of retries, have it return an Observable.timer using the retry # if you need increasing delays.
Your use case is pretty close to the one in RxJava doc here
reviving this thread since in the Couchbase Java SDK 2.1.2 there's a new simpler way to do that: use the RetryBuilder:
Observable<Something> retryingObservable =
sourceObservable.retryWhen(
RetryBuilder
//will limit to the relevant exception
.anyOf(CASMismatchException.class)
//will retry only 5 times
.max(5)
//delay doubling each time, from 100ms to 2s
.delay(Delay.linear(TimeUnit.MILLISECONDS, 2000, 100, 2.0))
.build()
);
First of all, I know the standard answer will be that exceptions are never to be used for flow control. While I perfectly agree with this, I've been thinking a long time about something I sometimes did, which I'll describe with the following pseudo-code:
try
string keyboardInput = read()
int number = int.parse(keyboardInput)
//the conversion succeeds
if(number >= 1000)
//That's not what I asked for. The message to display to the user
//is already in the catch-block below.
throw new NumberFormatException() //well, there IS something wrong with the number...
catch(NumberFormatException ex) //the user entered text
print("Please enter a valid number below 1000.")
First of all, take this example in a very abstract way. This does not necessarily have to happen. The situation simply is:
A user input needs to be constrained and can go wrong in 2 ways,
either
by a thrown exception the language defines, or by a check. Both errors
are reported by the user in the same way, because they do not need to know
the technical difference of what caused it.
I have thought of several ways to solve it. To begin with, it would be better to throw a custom made exception. The problem I then face is, if I catch it locally, what to do with the other exception? In se, the custom exception would be cause for a second catch-block, in which the message would be copied into just as well. My solution:
//number is wrong
throw new MyException()
catch(NumberFormatException ex)
throw new MyException()
catch(MyException ex) {
print("Please enter...")
The meaning of the exceptions' names is everything here. This application of custom-made exceptions is widely accepted, but essentially I didn't do anything different from the first way: I forced to go into a catch-block, albeit by throwing a custom exception rather than a standard-library one.
The same way applied to throwing the exception on to the calling method (thus not having a catch block for the custom exception) seems to make more sense. My method can go wrong in what is technically two ways, but essentially one way: wrong user input. Therefore, one would write a UserInputException and make the method throw this. New problem: what if this is the main method of an application?
I'm not currently struggling with a specific application to implement this kind of behaviour, my question is purely theoretical and non-language specific.
What is the best way to approach this?
I would consider the first exception to be low-level, and I would handle it (by translation in this case) at the point of call. I find that this leads to code that is easier to maintain and refactor later, as you have less types of exceptions to handle.
try
string keyboardInput = read()
try
int number = int.parse(keyboardInput)
catch(NumberFormatException ex)
throw MyException("Input value was not a number")
//the conversion succeeds
if(number >= 1000)
throw MyException("Input value was out of range")
catch(MyException ex) //the user entered text
print( ex.ToString() )
print("Please enter a valid number below 1000.")
I think you have essentially a few ways to go about it with minimal code duplication in mind:
Use a boolean variable/store the exception: If there was an error anywhere in the the general logic of the specific task you are performing, you exit on the first sign of error and handle that in a separate error handling branch.
Advantages: only one place to handle the error; you can use any custom exception/error condition you like.
Disadvantages: the logic of what you are trying to achieve might be hard to discover.
Create a general function that you can use to inform the user about the error (pre-calculating/storing all information that describes the general error, e.g. the message to display the user), so you can just make one function call when an error condition happens.
Advantages: the logic of your intent might be clearer for readers of the code; you can use anu custom exception/error conditon you like.
Disadvantages: the error will have to be handled in separate places (although with the pre-computed/stored values, there is not much copy-paste, however complex the informing the user part).
If the intent is clear, I don't think throwing exceptions from within your try block explicitly is a bad idea. If you do not want to throw one of the system provided exceptions, you can always create your own that derives from one of them, so you only need a minimal number (preferably one) of catch blocks.
Advantages: only one place to handle error condition -- if there is essentially only one type of exception thrown in try-block.
Disadvantages: if more than one type of exception is thrown, you need nested try-catch blocks (to propagate the exceptions to the most outward one) or a very general (e.g. Exception) catch block to avoid having to duplicate error reporting.
The way I see it is this:
Assuming there's no other way to parse your int that doesn't throw an exception, your code as it is now, is correct and elegant.
The only issue would be if your code was in some kind of loop, in which case you might worry about the overhead of throwing and catching unnecessary exceptions. In that case, you will have to compromise some of your code's beauty in favor of only handling exceptions whenever necessary.
error=false;
try {
string keyboardInput = read();
int number = int.parse(keyboardInput);
//the conversion succeeds
if(number >= 1000) {
//That's not what I asked for. The message to display to the user
//is already in the catch-block below.
error=true;
} catch(NumberFormatException ex) { //the user entered text
error=true;
}
if (error)
print("Please enter a valid number below 1000.");
Also you can think about why you're trying to aggregate two errors into one.
Instead you could inform the user as to what error they did, which might be more helpful in some cases:
try {
string keyboardInput = read();
int number = int.parse(keyboardInput);
//the conversion succeeds
if(number >= 1000) {
//That's not what I asked for. The message to display to the user
//is already in the catch-block below.
print("Please enter a number below 1000.");
} catch(NumberFormatException ex) { //the user entered text
print("Please enter a valid number.");
}
You do not need any exceptions in this particular example.
int number;
if (int.TryParse(keyboardInput, out number) && number < 1000) // success
else // error
However, the situation you describe is common in business software, and throwing an exception to reach a uniform handler is quite common.
One such pattern is XML validation followed by XSLT. In some systems, invalid XML is handled through catching validation exceptions. In these systems, it is pretty natural to reuse the existing exception handling in XSLT (which can naturally detect some classes of data errors that a particular validation language cannot):
<xsl:if test="#required = 'yes' and #prohibited = 'yes'>
<xsl:message terminate='yes'>Error message</xsl:message>
</xsl:if>
It is important to see that if such conditions are extremely rare (expected to occur only during early integration testing, and disappear as defects in other modules get fixed), most of the typical concerns around not using exceptions for flow control do not really apply.
What about approaching this validation problem by writing several validator classes that take in an input and return errors, or no errors. As far as your struggle with exceptions: put that logic into each validator and deal with it there on a case by case basis.
after that you figure out the correct validators to use for your input, collect their errors and handle them.
the benefits of this are:
Validators do one thing, validate a single case
Its up to the validation function to decide how to handle the errors. Do you break on first validation error or do you collect them all and then deal with them?
You can write your code is such a way that the main validation function can validate different types of input using the same code, just picking the correct validators using your favorite technique.
and disadvantages:
You will end up writing more code (but if you are using java, this should be put into the 'benefits' bucket)
here is some example pseudo-code:
validate(input):
validators = Validator.for(input.type)
errors = []
for validator in validators:
errors.push(validator.validate(input))
if errors:
throw PoopException
and some validators:
MaxValidator extends IntValidator:
validate(input):
errors = []
errors.push(super.validate(input))
if input > 1000:
errors.push("bleee!!!! to big!")
return errors
IntValidator:
validate(input):
try:
int.parse(input)
catch NumberFormatException:
return ['not an int']
return []
of course you would need to do some trickery to make the parent validator possibly return you a valid version of the input, in this case string "123" converted to an int so the max validator can handle it, but this can be easily accomplished by making the validators statefull or some other magic.
I can't see this answer anywhere in here, so I'll just post it as another point of view.
As we all know, you can actually break the rules if you know them well enough, so you can use throwing an Exception for flow control if you know it's the best solution for your situation. From what I've seen, it happens usually with some dumb frameworks...
That said, before Java 7 (which brought us the mighty multicatch construct), this was my approach to avoid code repetition:
try {
someOffendingMethod();
} catch (Exception e) {
if (e instanceof NumberFormatException || e instanceof MyException) {
System.out.println("Please enter a valid number.");
}
}
It's a valid technique in C#, too.
//
// To Throw
void PrintType(object obj)
{
if(obj == null)
{
throw new ArgumentNullException("obj")
}
Console.WriteLine(obj.GetType().Name);
}
//
// Not to Throw
void PrintType(object obj)
{
if(obj != null)
{
Console.WriteLine(obj.GetType().Name);
}
}
What principle to keep?
Personally I prefer the first one its say developer-friendly(notified about each "anomaly").
The second one its say user-friendly(let user continue work even if "inside" not everything does right).
I think that is more complicated to find errors and bugs in the maintenance phase when you silently let the things to go on. If something goes wrong you are not notified at once, and sometimes have errors far away from the main error cause, and spend a lot of time to find it.
What do you think?
The second one is lethal. Failing silently is always the wrong thing to do. Suppose this were the banking system in the bank that holds your account. Would you like it if there was a problem paying in your salary and the system silently ignored it?
If the method body handles the null obj properly (in other words, obj != null is not a requirement), then there's no need to throw an exception.
In all other cases: Throw. Let the client take responsibility for their flawed input.
Throwing an exception (if null is an error) seems far better than silently ignoring an error.
There is a third option you can consider:
void PrintType(object obj)
{
Console.WriteLine(obj.GetType().Name);
}
This also throws an exception when obj is null. The advantage of this, is that less code is involved. The disadvantage of this approach is that it is more difficult to tell whether obj can be null.
Throw.
Let the caller of a function determine if it is important enough to throw an exception to the user on a null value, but the function itself should throw because of the invalid argument.
I'd say that it depends on your (developer) preference. From the user perspective, he should never see an unhandled exception, but it does not mean you cannot use exceptions.
I prefer the first one, because I find null to be a totally unnecessary (and annoying) construct, so I make effort to code without it. If there is a null somewhere, someone made a mistake, so the best thing is to just barf out instead of pretending everything is ok.
In the end it depends on what you consider to be the semantics of the method. If the method is supposed to accept nulls, then you should pick option number two. If the method is supposed to only accept real arguments (which I prefer), then you should pick option number one.
Always Throw, except in debugging/diagnostic code. It is most embarassing to have a NullPointerException that occurs in production code at a point where only a debugging message should be generated, e.g.
log.debug("id of object is " + obj.getId())
where the logger is turned off, and obj is null.
It is highly subjective, but I always prefer to just ignore non-fatal or recoverable errors. Put them in logs, if you must, but if you know how to continue - please do so.
Note, that when I say fatal, it actually depends on the function itself. Say, there's API function that gets ID and handful of other parameters. Suppose, that this ID also can be guessed from those other stuff that is passed in. API function should guess it if it can but the function somewhere inside that does all the work should get non-null ID and throw otherwise. Because for high level API function it is not fatal, it knows how to guess it, but for low level function it is fatal, it supposed to do something with that ID and with null value it can't continue.
All fatal errors should be noted, of course.
If you api if exposed outside, do always argument checking and throw a argument based exception so the api user can get the result.
Consider using the Null Object pattern is very useful to not clutter your code with try-catch, null checks (or god forbid swallowed errors).
In this particular example, giving nothing to a printer is like saying "print nothing", thus working as it should.
I do know this is an example, but it's just to clarify that this is relative.
If your code displays user-friendly messages on exceptions somehow, what difference does it make ? the first one would be both developer and user friendly.
It really depends on what your invariants are. If the parameter is optiona, then ignoring a null parameter is just fine, but if the parameter is required then that will hide a bug in your application. Also, and depending on the language, if the invariant is bad enough you may consider a third option: abort the application.
All discussions on whether to use or not exceptions can always be mapped to the decision on whether the situation is exceptional or not, and if it is exceptional, throwing or rather aborting the application depends on whether it is recoverable or not.
Id go for
void PrintType(object obj)
{
Console.WriteLine(obj.GetType().Name);
}
Third option, half in pseudocode:
// To Throw A Clean Error
void PrintType(object obj)
{
if(obj == null)
{
throw new ArgumentNullException(STANDARD_ERROR_MESSAGE, obj)
}
Console.WriteLine(obj.GetType().Name);
}
Either catch all errors and wrap them in a single place, so the user sees standard text:
There has been an error. If this error
persists, please contact an
administrator.
Or throw a select few errors, all of which are user-friendly, and display them directly to the user. "A connection error has occurred." "An authentication error has occurred." "A system error has occurred." And so on.
On the backend, have all errors and their stack trace logged, so you can use the debugging information that way.
It really depends on what the function is defined to do. The most important aspect is to have a clearly defined behavior and for the function to implement it correctly.
Now, if the question is whether is better to define the function to accept null and print it out, or to not accept it and throw an exception, I would say the latter, because it's probably less error prone for the user to check for null before calling the function, if that is a possibility.
What is an idempotent operation?
In computing, an idempotent operation is one that has no additional effect if it is called more than once with the same input parameters. For example, removing an item from a set can be considered an idempotent operation on the set.
In mathematics, an idempotent operation is one where f(f(x)) = f(x). For example, the abs() function is idempotent because abs(abs(x)) = abs(x) for all x.
These slightly different definitions can be reconciled by considering that x in the mathematical definition represents the state of an object, and f is an operation that may mutate that object. For example, consider the Python set and its discard method. The discard method removes an element from a set, and does nothing if the element does not exist. So:
my_set.discard(x)
has exactly the same effect as doing the same operation twice:
my_set.discard(x)
my_set.discard(x)
Idempotent operations are often used in the design of network protocols, where a request to perform an operation is guaranteed to happen at least once, but might also happen more than once. If the operation is idempotent, then there is no harm in performing the operation two or more times.
See the Wikipedia article on idempotence for more information.
The above answer previously had some incorrect and misleading examples. Comments below written before April 2014 refer to an older revision.
An idempotent operation can be repeated an arbitrary number of times and the result will be the same as if it had been done only once. In arithmetic, adding zero to a number is idempotent.
Idempotence is talked about a lot in the context of "RESTful" web services. REST seeks to maximally leverage HTTP to give programs access to web content, and is usually set in contrast to SOAP-based web services, which just tunnel remote procedure call style services inside HTTP requests and responses.
REST organizes a web application into "resources" (like a Twitter user, or a Flickr image) and then uses the HTTP verbs of POST, PUT, GET, and DELETE to create, update, read, and delete those resources.
Idempotence plays an important role in REST. If you GET a representation of a REST resource (eg, GET a jpeg image from Flickr), and the operation fails, you can just repeat the GET again and again until the operation succeeds. To the web service, it doesn't matter how many times the image is gotten. Likewise, if you use a RESTful web service to update your Twitter account information, you can PUT the new information as many times as it takes in order to get confirmation from the web service. PUT-ing it a thousand times is the same as PUT-ing it once. Similarly DELETE-ing a REST resource a thousand times is the same as deleting it once. Idempotence thus makes it a lot easier to construct a web service that's resilient to communication errors.
Further reading: RESTful Web Services, by Richardson and Ruby (idempotence is discussed on page 103-104), and Roy Fielding's PhD dissertation on REST. Fielding was one of the authors of HTTP 1.1, RFC-2616, which talks about idempotence in section 9.1.2.
No matter how many times you call the operation, the result will be the same.
Idempotence means that applying an operation once or applying it multiple times has the same effect.
Examples:
Multiplication by zero. No matter how many times you do it, the result is still zero.
Setting a boolean flag. No matter how many times you do it, the flag stays set.
Deleting a row from a database with a given ID. If you try it again, the row is still gone.
For pure functions (functions with no side effects) then idempotency implies that f(x) = f(f(x)) = f(f(f(x))) = f(f(f(f(x)))) = ...... for all values of x
For functions with side effects, idempotency furthermore implies that no additional side effects will be caused after the first application. You can consider the state of the world to be an additional "hidden" parameter to the function if you like.
Note that in a world where you have concurrent actions going on, you may find that operations you thought were idempotent cease to be so (for example, another thread could unset the value of the boolean flag in the example above). Basically whenever you have concurrency and mutable state, you need to think much more carefully about idempotency.
Idempotency is often a useful property in building robust systems. For example, if there is a risk that you may receive a duplicate message from a third party, it is helpful to have the message handler act as an idempotent operation so that the message effect only happens once.
A good example of understanding an idempotent operation might be locking a car with remote key.
log(Car.state) // unlocked
Remote.lock();
log(Car.state) // locked
Remote.lock();
Remote.lock();
Remote.lock();
log(Car.state) // locked
lock is an idempotent operation. Even if there are some side effect each time you run lock, like blinking, the car is still in the same locked state, no matter how many times you run lock operation.
An idempotent operation produces the result in the same state even if you call it more than once, provided you pass in the same parameters.
An idempotent operation is an operation, action, or request that can be applied multiple times without changing the result, i.e. the state of the system, beyond the initial application.
EXAMPLES (WEB APP CONTEXT):
IDEMPOTENT:
Making multiple identical requests has the same effect as making a single request. A message in an email messaging system is opened and marked as "opened" in the database. One can open the message many times but this repeated action will only ever result in that message being in the "opened" state. This is an idempotent operation. The first time one PUTs an update to a resource using information that does not match the resource (the state of the system), the state of the system will change as the resource is updated. If one PUTs the same update to a resource repeatedly then the information in the update will match the information already in the system upon every PUT, and no change to the state of the system will occur. Repeated PUTs with the same information are idempotent: the first PUT may change the state of the system, subsequent PUTs should not.
NON-IDEMPOTENT:
If an operation always causes a change in state, like POSTing the same message to a user over and over, resulting in a new message sent and stored in the database every time, we say that the operation is NON-IDEMPOTENT.
NULLIPOTENT:
If an operation has no side effects, like purely displaying information on a web page without any change in a database (in other words you are only reading the database), we say the operation is NULLIPOTENT. All GETs should be nullipotent.
When talking about the state of the system we are obviously ignoring hopefully harmless and inevitable effects like logging and diagnostics.
Just wanted to throw out a real use case that demonstrates idempotence. In JavaScript, say you are defining a bunch of model classes (as in MVC model). The way this is often implemented is functionally equivalent to something like this (basic example):
function model(name) {
function Model() {
this.name = name;
}
return Model;
}
You could then define new classes like this:
var User = model('user');
var Article = model('article');
But if you were to try to get the User class via model('user'), from somewhere else in the code, it would fail:
var User = model('user');
// ... then somewhere else in the code (in a different scope)
var User = model('user');
Those two User constructors would be different. That is,
model('user') !== model('user');
To make it idempotent, you would just add some sort of caching mechanism, like this:
var collection = {};
function model(name) {
if (collection[name])
return collection[name];
function Model() {
this.name = name;
}
collection[name] = Model;
return Model;
}
By adding caching, every time you did model('user') it will be the same object, and so it's idempotent. So:
model('user') === model('user');
Quite a detailed and technical answers. Just adding a simple definition.
Idempotent = Re-runnable
For example,
Create operation in itself is not guaranteed to run without error if executed more than once.
But if there is an operation CreateOrUpdate then it states re-runnability (Idempotency).
Idempotent Operations: Operations that have no side-effects if executed multiple times.
Example: An operation that retrieves values from a data resource and say, prints it
Non-Idempotent Operations: Operations that would cause some harm if executed multiple times. (As they change some values or states)
Example: An operation that withdraws from a bank account
It is any operation that every nth result will result in an output matching the value of the 1st result. For instance the absolute value of -1 is 1. The absolute value of the absolute value of -1 is 1. The absolute value of the absolute value of absolute value of -1 is 1. And so on. See also: When would be a really silly time to use recursion?
An idempotent operation over a set leaves its members unchanged when applied one or more times.
It can be a unary operation like absolute(x) where x belongs to a set of positive integers. Here absolute(absolute(x)) = x.
It can be a binary operation like union of a set with itself would always return the same set.
cheers
In short, Idempotent operations means that the operation will not result in different results no matter how many times you operate the idempotent operations.
For example, according to the definition of the spec of HTTP, GET, HEAD, PUT, and DELETE are idempotent operations; however POST and PATCH are not. That's why sometimes POST is replaced by PUT.
An operation is said to be idempotent if executing it multiple times is equivalent to executing it once.
For eg: setting volume to 20.
No matter how many times the volume of TV is set to 20, end result will be that volume is 20. Even if a process executes the operation 50/100 times or more, at the end of the process the volume will be 20.
Counter example: increasing the volume by 1. If a process executes this operation 50 times, at the end volume will be initial Volume + 50 and if a process executes the operation 100 times, at the end volume will be initial Volume + 100. As you can clearly see that the end result varies based upon how many times the operation was executed. Hence, we can conclude that this operation is NOT idempotent.
I have highlighted the end result in bold.
If you think in terms of programming, let's say that I have an operation in which a function f takes foo as the input and the output of f is set to foo back. If at the end of the process (that executes this operation 50/100 times or more), my foo variable holds the value that it did when the operation was executed only ONCE, then the operation is idempotent, otherwise NOT.
foo = <some random value here, let's say -2>
{ foo = f( foo ) } curly brackets outline the operation
if f returns the square of the input then the operation is NOT idempotent. Because foo at the end will be (-2) raised to the power (number of times operation is executed)
if f returns the absolute of the input then the operation is idempotent because no matter how many multiple times the operation is executed foo will be abs(-2).
Here, end result is defined as the final value of variable foo.
In mathematical sense, idempotence has a slightly different meaning of:
f(f(....f(x))) = f(x)
here output of f(x) is passed as input to f again which doesn't need to be the case always with programming.
my 5c:
In integration and networking the idempotency is very important.
Several examples from real-life:
Imagine, we deliver data to the target system. Data delivered by a sequence of messages.
1. What would happen if the sequence is mixed in channel? (As network packages always do :) ). If the target system is idempotent, the result will not be different. If the target system depends of the right order in the sequence, we have to implement resequencer on the target site, which would restore the right order.
2. What would happen if there are the message duplicates? If the channel of target system does not acknowledge timely, the source system (or channel itself) usually sends another copy of the message. As a result we can have duplicate message on the target system side.
If the target system is idempotent, it takes care of it and result will not be different.
If the target system is not idempotent, we have to implement deduplicator on the target system side of the channel.
For a workflow manager (as Apache Airflow) if an idempotency operation fails in your pipeline the system can retry the task automatically without affecting the system. Even if the logs change, that is good because you can see the incident.
The most important in this case is that your system can retry the task that failed and doesn't mess up the pipeline (e.g. appending the same data in a table each retry)
Let's say the client makes a request to "IstanceA" service which process the request, passes it to DB, and shuts down before sending the response. since the client does not see that it was processed and it will retry the same request. Load balancer will forward the request to another service instance, "InstanceB", which will make the same change on the same DB item.
We should use idempotent tokens. When a client sends a request to a service, it should have some kind of request-id that can be saved in DB to show that we have already executed the request. if the client retries the request, "InstanceB" will check the requestId. Since that particular request already has been executed, it will not make any change to the DB item. Those kinds of requests are called idempotent requests. So we send the same request multiple times, but we won't make any change