I ran into a dilemma with making a test pass if it times out.
#Test(timeout=1, expected=Exception.class)
public void testMovesToSolveMaximum() {
PuzzleSolver pS = createSimplePuzzleSolver(maximumPuzzleStateA, maximumPuzzleStateB);
PuzzleState goal = new SimplePuzzleState();
goal.configureState(maximumPuzzleStateB);
checkThatComputedSolutionIsCorrect(pS, goal);
}
However, the test case fails due to timeout even though I specified that is the expected result.
If I understand the question correctly then you are observing the specific behavior due to the way that the default JUnit runner is evaluating the whole test:
After realizing that there is a timeout set on your test method it runs it in a different thread and is waiting for the result. As the timeout in your example is set to 1[ms] I believe that it reaches the timeout before the test actually finishes which makes the runner throw the timeout exception (that is indeed a java.lang.Exception) which you thought needed to be caught by the expected attribute in the Test annotation. But the attribute expected on the Test annotation is evaluating only the exceptions thrown from the test method and not from the timeout checking mechanism. In other words the expected exception mechanism is not working for the timeout exception throw by the f/w and not by a test.
You can explore this yourself starting in BlockJUnit4ClassRunner class in JUnit (relevant part to start from. NOTE: it is not so easy to go over the code and understand the flow...):
protected Statement methodBlock(FrameworkMethod method) {
Object test;
try {
test = new ReflectiveCallable() {
#Override
protected Object runReflectiveCall() throws Throwable {
return createTest();
}
}.run();
} catch (Throwable e) {
return new Fail(e);
}
Statement statement = methodInvoker(method, test);
statement = possiblyExpectingExceptions(method, test, statement);
statement = withPotentialTimeout(method, test, statement);
statement = withBefores(method, test, statement);
statement = withAfters(method, test, statement);
statement = withRules(method, test, statement);
return statement;
}
Related
I have a simple unit test which asserts on an object instance of Try from the vavr library.
#Test
public void testFoo()
{
final Try<Foo> result = createMyFooInstance();
assertThat(result.isSuccess()).isTrue();
}
My question focuses on the formulation of the test assertion.
Semantically I want to have "if foo is success, all fine, otherwise, throw the encapsulated exception". The latter one is important so that I can see the error cause directly in the JUnit output.
Is there any convenient API that I can use to nicely formulate that semantics?
You could use
#Test
public void testFoo() {
final Try<Foo> result = createMyFooInstance();
result.get();
}
In case when result is a Failure, result.get() will throw the wrapped exception. In case when result is a Success, it will succeed.
Though this solution doesn't contain explicit assertions, it will implicitly fail the cases when the result is a Failure.
If you prefer to have an assertion failed instead of a test failed with exception, you could also use:
#Test
public void testFoo() {
final Try<Foo> result = createMyFooInstance();
assertThatCode(result::get).doesNotThrowAnyException();
}
I have some async code that may throw an exception that JUnit misses (so the test passes).
I've created a TestRule to collect those exceptions into a list. After any test has finished, an assertion runs over the list and fails the test if the exception list is non-empty.
Instead of failing after the test finishes, I would like to fail the test immediately when the exception occurs. Is this possible to do this with a TestRule?
My TestRule
/**
* Coroutines can throw exceptions that can go unnoticed by the JUnit Test Runner which will pass
* a test that should have failed. This rule will ensure the test fails, provided that you use the
* [CoroutineContext] provided by [dispatcher].
*/
class CoroutineExceptionRule : TestWatcher(), TestRule {
private val exceptions = Collections.synchronizedList(mutableListOf<Throwable>())
val dispatcher: CoroutineContext
get() = Unconfined + CoroutineExceptionHandler { _, throwable ->
// I want to hook into test lifecycle and fail test immediately here
exceptions.add(throwable)
// this throw will not always fail the test. this does print the stacktrace at least
throw throwable
}
override fun starting(description: Description) {
exceptions.clear()
}
override fun finished(description: Description) {
// instead of waiting for test to finish to fail it
exceptions.forEach { throw AssertionError(it) }
}
}
Whenever a test function (a function annotated with test) contains assertions that fail, the assertion has the same effect as when trowing an exception: no further code lines in that function will be executed. Thus, assert statements in functions that are annotated with 'test' works just as ordinary assert statements in ordinary Ceylon functions. This runs contrary to the documentation, which states that ordinary assert statements can be used for making unit tests.
Thus, running the code below, I get to see the statement myTests1 but not ´myTests2`:
import ceylon.test {
test, TestRunner, createTestRunner
}
test
Anything myTests1 () {
// assert something true!
assert(40 + 2 == 42);
print("myTests1");
return null;
}
test
void myTests2 () {
// assert something false!
assert(2 + 2 == 54);
print("myTests2");
}
"Run the module `tests`."
shared void run() {
print("reached run function");
TestRunner myTestRunner = createTestRunner(
[`function myTests1`, `function myTests2`]);
myTestRunner.run();
}
This is the actual output:
"C:\Program Files\Java\jdk1.8.0_121\bin\java" -Dceylon.system.repo=C:\Users\Jon\.IdeaIC2017.2\config\plugins\CeylonIDEA\classes\embeddedDist\repo "-javaagent:C:\Program Files\JetBrains\IntelliJ IDEA Community Edition 2017.2.1\lib\idea_rt.jar=56393:C:\Program Files\JetBrains\IntelliJ IDEA Community Edition 2017.2.1\bin" -Dfile.encoding=windows-1252 -classpath C:\Users\Jon\.IdeaIC2017.2\config\plugins\CeylonIDEA\classes\embeddedDist\lib\ceylon-bootstrap.jar com.redhat.ceylon.launcher.Bootstrap run --run run tests/1.0.0
reached run function
myTests1
Process finished with exit code 0
This is working as intended – replacing those asserts with assertEquals’ has the same effect and prints the same output, because both do exactly the same thing: throw an exception if the assertion fails.
All test frameworks that I’m aware of behave the same way in this situation: an assertion failure results in an exception and thus immediately terminates execution of the test method. This is by design, since you don’t know what your program will do once one expectation will be violated – the rest of the method might depend on that assertion holding true, and might break in unpredictable and confusing ways.
If you’re writing tests like
test
shared void testTwoThings() {
assertEquals { expected = 42; actual = 40 + 2; };
assertEquals { expected = 42; actual = 6 * 9; };
}
you’re supposed to write two tests instead.
How do I test if an assertion is thrown by the method under test using junit? Heres the method I'm testing:
public int f(int i){
assert i > 0;
return i;
}
I'm using junit 4.12.
You can test it by providing parameter in #Test annotation:
#Test(expected = AssertionError.class)
public void shouldThrowExceptionWhenIncorrectInput() {
f(-3);
}
This will check if the AssertException is thrown.
However, if you want to ensure that this function wont be run with incorrect parameters, you have to be aware that assertions can be turned off by running java with -da parameters.
To ensure that exception is thrown I would suggest throwing IllegalArgumentException inside some validation method, and to provide it with proper message. Then you will be sure that this will always throw exception when incorrect parameters are provided.
I am new to JUnit, but this is what I am trying to do:
generate data with my DataGenerator class;
instantiate a test class MyTestClass
pass to MyTestClass the test data generated data (first step)
run the test
collect TestResult result
With the above, all works, but I cannot see any timing information (time it took to complete the test) at the TestResult object. Anything being done wrong here ?
The approach above is because I need to run this on other test classes using the same data.
DataGenerator testData = new DataGenerator();
MyTestClass myTestClass = new MyTestClass("mytestmethod");
myTestClass.setBaseLine(testData);
try {
TestResult testResult = myTestClass.run();
System.out.println(testResult.wasSuccessful());
} catch (Throwable ex) {
Logger.getLogger(TestSupervisor.class.getName()).log(Level.SEVERE, null, ex);
}
JUnit 4 has a timeout annotation that will fail the test if the test too long.
//this test will fail if it takes longer than 1 sec
#Test(timeout = 1000)
public void myTest(){
...
}
I think you can get around the DataGenerator issue by using a better TestFixture.
If all of your tests that use the DataGenerator are in the same test class, you can use the #BeforeClass and #AfterClass annotations to set up and tear down data that is used across tests. (those methods with those annotations are called only once before/after all the tests are run).