I have some async code that may throw an exception that JUnit misses (so the test passes).
I've created a TestRule to collect those exceptions into a list. After any test has finished, an assertion runs over the list and fails the test if the exception list is non-empty.
Instead of failing after the test finishes, I would like to fail the test immediately when the exception occurs. Is this possible to do this with a TestRule?
My TestRule
/**
* Coroutines can throw exceptions that can go unnoticed by the JUnit Test Runner which will pass
* a test that should have failed. This rule will ensure the test fails, provided that you use the
* [CoroutineContext] provided by [dispatcher].
*/
class CoroutineExceptionRule : TestWatcher(), TestRule {
private val exceptions = Collections.synchronizedList(mutableListOf<Throwable>())
val dispatcher: CoroutineContext
get() = Unconfined + CoroutineExceptionHandler { _, throwable ->
// I want to hook into test lifecycle and fail test immediately here
exceptions.add(throwable)
// this throw will not always fail the test. this does print the stacktrace at least
throw throwable
}
override fun starting(description: Description) {
exceptions.clear()
}
override fun finished(description: Description) {
// instead of waiting for test to finish to fail it
exceptions.forEach { throw AssertionError(it) }
}
}
Related
I was working on a unit test for the sample vert.x app. But could not understand the purpose of TestContext#verify() method. Here are two test cases one with TestContext#verify() and another without it. Both are running fine. So when do we need to use TestContext#verify()?
Test Case without TestContext#verify():
#Test
#DisplayName("Handler Test3")
void successTestCase1(VertxTestContext testContext) {
Checkpoint checkpoint = testContext.checkpoint(1);
new SampleVerticle().doWorkWithAsyncresult("hello",
testContext.succeeding(future -> {
assertThat(future).isEqualTo("test from");
testContext.completeNow();
})
);
}
Test Case with TestContext#verify():
#Test
#DisplayName("Handler Test4")
void successTestCase2(VertxTestContext testContext) {
Checkpoint checkpoint = testContext.checkpoint(1);
new SampleVerticle().doWorkWithAsyncresult("vivek",
testContext.succeeding(future -> {
testContext.verify(() -> {
assertThat(future).isEqualTo("test from");
testContext.completeNow();
});
})
);
}
Test Method:
public void doWorkWithAsyncresult(String input, Handler<AsyncResult<String>> handler) {
handler.handle(Future.succeededFuture("test from"));
}
Thanks,
Vivek Kumar
The verify method is useful when the test fails, not when it passes.
If one of the asynchronous assertions fails, verify will report it correctly to the TestContext. Then the test will fail immediately with the right error message.
Otherwise the test would timeout after an arbitrary amount of time, and the error may or may not be printed to the console.
You code is not async. So both methods are same now. And testContext.completeNow() does nothing.
When code is async you should add to the bottom of test (VertxExtension does this for you):
assertTrue(testContext.awaitCompletion(10, TimeUnit.SECONDS), "Timeout");
assertFalse(testContext.failed());
And if assert throws exception, the test without verify will end with timeout.
To fix that you should wrap your assert and call failNow:
try {
assert...
testContext.completeNow();
} catch (Exception e) {
textContext.failNow(e);
return;
}
TextContext::verify does this wrapping for you.
I have a simple unit test which asserts on an object instance of Try from the vavr library.
#Test
public void testFoo()
{
final Try<Foo> result = createMyFooInstance();
assertThat(result.isSuccess()).isTrue();
}
My question focuses on the formulation of the test assertion.
Semantically I want to have "if foo is success, all fine, otherwise, throw the encapsulated exception". The latter one is important so that I can see the error cause directly in the JUnit output.
Is there any convenient API that I can use to nicely formulate that semantics?
You could use
#Test
public void testFoo() {
final Try<Foo> result = createMyFooInstance();
result.get();
}
In case when result is a Failure, result.get() will throw the wrapped exception. In case when result is a Success, it will succeed.
Though this solution doesn't contain explicit assertions, it will implicitly fail the cases when the result is a Failure.
If you prefer to have an assertion failed instead of a test failed with exception, you could also use:
#Test
public void testFoo() {
final Try<Foo> result = createMyFooInstance();
assertThatCode(result::get).doesNotThrowAnyException();
}
I ran into a dilemma with making a test pass if it times out.
#Test(timeout=1, expected=Exception.class)
public void testMovesToSolveMaximum() {
PuzzleSolver pS = createSimplePuzzleSolver(maximumPuzzleStateA, maximumPuzzleStateB);
PuzzleState goal = new SimplePuzzleState();
goal.configureState(maximumPuzzleStateB);
checkThatComputedSolutionIsCorrect(pS, goal);
}
However, the test case fails due to timeout even though I specified that is the expected result.
If I understand the question correctly then you are observing the specific behavior due to the way that the default JUnit runner is evaluating the whole test:
After realizing that there is a timeout set on your test method it runs it in a different thread and is waiting for the result. As the timeout in your example is set to 1[ms] I believe that it reaches the timeout before the test actually finishes which makes the runner throw the timeout exception (that is indeed a java.lang.Exception) which you thought needed to be caught by the expected attribute in the Test annotation. But the attribute expected on the Test annotation is evaluating only the exceptions thrown from the test method and not from the timeout checking mechanism. In other words the expected exception mechanism is not working for the timeout exception throw by the f/w and not by a test.
You can explore this yourself starting in BlockJUnit4ClassRunner class in JUnit (relevant part to start from. NOTE: it is not so easy to go over the code and understand the flow...):
protected Statement methodBlock(FrameworkMethod method) {
Object test;
try {
test = new ReflectiveCallable() {
#Override
protected Object runReflectiveCall() throws Throwable {
return createTest();
}
}.run();
} catch (Throwable e) {
return new Fail(e);
}
Statement statement = methodInvoker(method, test);
statement = possiblyExpectingExceptions(method, test, statement);
statement = withPotentialTimeout(method, test, statement);
statement = withBefores(method, test, statement);
statement = withAfters(method, test, statement);
statement = withRules(method, test, statement);
return statement;
}
In my webdriver script I have the three methods
setup, test and tearDown
following the junit convention.
In the test method I have few asserts like this
#Test
public void testStudentHome() throws Exception {
String classCode = "I6OWW";
Utilities.studentSignin(driver, baseUrl);
assertEquals(true, sth.openNotification());
assertEquals("My Scores", sth.myScores(true));
}
The sth is the PageObject on which I am performing the tests and that I have created in the setup method.
I am calling all these three methods from a main method like this:
public static void main(String[] args) {
StudentHomeTest sht = new StudentHomeTest();
try {
sht.setup();
sht.testStudentHome();
sht.tearDown();
} catch (Exception ex) {
Logger.getLogger(StudentHomeTest.class.getName()).log(Level.SEVERE, null, ex);
sht.tearDown();
}
}
Now while running the test if some assertion fails the test method should (this is what I expect) throw an exception and the main method should call the tearDown method. But this does not happen. and the browser window continues to stay there.
I am using the netbeans ide for running the test.
following the junit convention
If you follow the jUnit convention, then you will know that teardown methods belong in the #After method as this method will always run after your tests.
create a new method with the #After jUnit annotation.
#After
public void tearDown() {
sht.tearDown();
}
Edit
You know what, I believe that you are running into a classic issue of assertEquals in jUnit.
Stolen from this answer...:
JUnit calls the .equals() method to determine equality in the method assertEquals(Object o1, Object o2).
So, you are definitely safe using assertEquals(string1, string2). (Because Strings are Objects)
--
Instead of using assertEquals on these calls, use assertTrue() instead.
assertTrue(sth.openNotification());
assertTrue("My Scores".equals(sth.myScores(true)));
AssertionError doesn't extend Exception - it's a Throwable.
But in any case, you should have
try {
sht.setup();
sht.testStudentHome();
} finally {
sht.tearDown();
}
No need for a catch block. main can throw Exception.
I have a method that throws exception. And i have a test like this.
#Rule
public ExpectedException expectedEx = ExpectedException.none();
#Test
public void shouldThrowExceptionIfValidationFails() throws Exception {
doThrow(new InvalidException("Invalid Token")).when(obj).foo(any());
expectedEx.expect(InvalidException.class);
expectedEx.expectMessage("Invalid Token");
// my method call
// verify DB save doesn't happens
assertTrue(false);
}
The test assert for exception, and since the exception is thrown the test passes. It doesn't care about the last line assertTrue(false)
How can i make sure that my other assertions are also satisfied.
This is the pattern I follow for this case. It uses ExpectedException as designed. I like the throw e rather than failing after method method call in the try because it will not result in a false-positive if someone decides to delete the fail (which people have a tendency to do when they see fail() or if a test is failing because it hits a fail()).
#Test
public void shouldThrowExceptionIfValidationFails() throws Exception {
doThrow(new InvalidException("Invalid Token")).when(obj).foo(any());
expectedEx.expect(InvalidException.class);
expectedEx.expectMessage("Invalid Token");
try{
// my method call
}catch(InvalidException e){
// verify DB save doesn't happens
assertTrue(false);
throw e;
}
}