In the scenario below, I expect that the assertion failure in the step should make the test fail. What I am seeing is that the test continues and since a later assertion in the #Test method fails, the wrong exception is being reported, making it hard to debug.
Is there anyway I can get the test to stop when there is an assertion failure in a #Step?
#Test
public void test() {
....
steps.step1();
System.out.println("test should not reach here");
assertTrue(false);
}
#Step
public void step1() {
assertTrue(false);
}
Tried running the tests from my IDE and maven. (using ThucydidesRunner)
try to check this: http://thucydides.info/docs/thucydides/_creating_a_new_thucydides_project.html
in short words, in steps:
import net.thucydides.core.annotations.Step;
import static org.fest.assertions.Assertions.assertThat;
import static org.hamcrest.Matchers.is;
public class EndUserSteps extends Scenario Steps {
#Step
public void someStep() {
assertThat(true, is(false));
}
}
It turns out the tests don't stop, here is the reply I got from the creators of Thucydides:
"The test can't stop immediately, or it would be impossible to know what steps were not executed, so, yes, this is by design. They do record the first assertion error and then run the steps in "dry-run" mode (no WebDriver calls are made), but that's about the most it can guarantee."
Related
Is there any hook in Junit5 that reacts to fails in #Before* methods?
I use a #BeforeAll method to init environment for my tests. But this initialization may sometimes fail. I want to dump the environment, to find out whats wrong, but I need to do it before #After* methods are called, which will clear the environment and destroy all info.
We're talking about dozens of these #BeforeAll methods across the test suite, so doing it manually inside each method is not an option.
I've already tried these, but no luck:
TestWatcher does not work for this, since it only fires if an actual test is executed.
TestExecutionListener.executionFinished looks promising, but it fires after all the #After methods, which is too late for me.
I even tried to do it inside the #AfterAll cleanup methods, before the actual cleanup. But found no way to detect which tests were executed or if anything failed.
Any ideas?
I assume by "fails in #Before* methods" you mean exceptions? If that is the case, you could leverage the extension model as follows:
#ExtendWith(DumpEnvOnFailureExtension.class)
class SomeTest {
static class DumpEnvOnFailureExtension implements LifecycleMethodExecutionExceptionHandler {
#Override
public void handleBeforeAllMethodExecutionException(final ExtensionContext context, final Throwable ex)
throws Throwable {
System.out.println("handleBeforeAllMethodExecutionException()");
// dump env
throw ex;
}
}
#BeforeAll
static void setUpOnce() {
System.out.println("setUpOnce()");
throw new RuntimeException();
}
#Test
void test() throws Exception {
// some test
}
#AfterAll
static void tearDownOnce() {
System.out.println("tearDownOnce()");
}
}
The log will be:
setUpOnce()
handleBeforeAllMethodExecutionException()
tearDownOnce()
That is, the extension is notified if the #BeforeAll method fails with an exception. (Note that this is just an MWE, for the actual implementation you would extract DumpEnvOnFailureExtension and use it wherever needed.)
For further information, check out section "Exception Handling" in the user guide.
I have a Spring Batch Task running on our cloud platform that will launch with the provided command line parameters, and then skip over the execution of the first Step with the following error:
[OUT] The job execution id 992 was run within the task execution 1325
[OUT] Step already complete or not restartable, so no action to execute:
StepExecution: id=1071, version=3, name=OFileStep, status=COMPLETED, exitStatus=COMPLETED, readCount=0, filterCount=0, writeCount=0 readSkipCount=0,
writeSkipCount=0, processSkipCount=0, commitCount=1, rollbackCount=0, exitDescription=
I have investigated the metadata tables in the MySQL instance that Spring Batch uses to find that the JOB_INSTANCE_ID is the same between multiple executions, when it should increment by 1 each time.
The #Bean that I have defined for the Job Configuration is:
#Bean
public Job job() {
return jobBuilderFactory.get(OTaskConstants.JOB_NAME)
.listener(listener())
.incrementer(new RunIdIncrementer())
.start(dataTransferTaskStep())
.next(controlMTaskStep())
.build();
}
Is anyone aware of what could be causing this behavior?
Below line clearly says it all.
Step already complete or not restartable, so no action to execute:
Meaning the step/job already complete and can not be restarted. This is the behavior of Spring Batch. In order to by pass this we need to pass an unique argument.
In your case i see you already have RunIdIncrementer. Now question is why it is not working.
Can you see BATCH_JOB_PARMS table to see what arguments are getting passed to the job? May be you are missing something.
You can also use SimpleIncrementor. See below code for explanation.
https://docs.spring.io/spring-batch/docs/current/reference/html/index-single.html#JobParametersIncrementer
Remove #Bean annotation on Job.
It causes the Job to be launched with no parameters every time you launch/start application as spring tries to load the bean and which in-turn launches the batch job.
Remove the annotation and use spring scheduler to schedule the jobs.
I had the same issue. Below code helped me resolve it. By adding params in job launcher a new job_instance_id is created for every run.
#SpringBootApplication
public class App implements CommandLineRunner {
#Autowired
JobLauncher jobLauncher;
#Autowired
Job job;
public static void main(String[] args) {
SpringApplication.run(App.class, args);
}
#Override
public void run(String... args) throws Exception {
JobParameters params = new JobParametersBuilder()
.addString("JobID", String.valueOf(System.currentTimeMillis()))
.toJobParameters();
jobLauncher.run(job, params);
}
}
Solution
Refer error message above “If you want to run this job again, change the parameters.” The formula is JobInstance = JobParameters + Job. If you do not have any parameters for JobParameters, just pass a current time as parameter to create a new JobInstance. For example,
CustomJobLauncher.java
//...
#Component
public class CustomJobLauncher {
#Autowired
JobLauncher jobLauncher;
#Autowired
Job job;
public void run() {
try {
JobParameters jobParameters =
new JobParametersBuilder()
.addLong("time",System.currentTimeMillis()).toJobParameters();
JobExecution execution = jobLauncher.run(job, jobParameters);
System.out.println("Exit Status : " + execution.getStatus());
} catch (Exception e) {
e.printStackTrace();
}
}
}
Source : https://mkyong.com/spring-batch/spring-batch-a-job-instance-already-exists-and-is-complete-for-parameters/
I was working on a unit test for the sample vert.x app. But could not understand the purpose of TestContext#verify() method. Here are two test cases one with TestContext#verify() and another without it. Both are running fine. So when do we need to use TestContext#verify()?
Test Case without TestContext#verify():
#Test
#DisplayName("Handler Test3")
void successTestCase1(VertxTestContext testContext) {
Checkpoint checkpoint = testContext.checkpoint(1);
new SampleVerticle().doWorkWithAsyncresult("hello",
testContext.succeeding(future -> {
assertThat(future).isEqualTo("test from");
testContext.completeNow();
})
);
}
Test Case with TestContext#verify():
#Test
#DisplayName("Handler Test4")
void successTestCase2(VertxTestContext testContext) {
Checkpoint checkpoint = testContext.checkpoint(1);
new SampleVerticle().doWorkWithAsyncresult("vivek",
testContext.succeeding(future -> {
testContext.verify(() -> {
assertThat(future).isEqualTo("test from");
testContext.completeNow();
});
})
);
}
Test Method:
public void doWorkWithAsyncresult(String input, Handler<AsyncResult<String>> handler) {
handler.handle(Future.succeededFuture("test from"));
}
Thanks,
Vivek Kumar
The verify method is useful when the test fails, not when it passes.
If one of the asynchronous assertions fails, verify will report it correctly to the TestContext. Then the test will fail immediately with the right error message.
Otherwise the test would timeout after an arbitrary amount of time, and the error may or may not be printed to the console.
You code is not async. So both methods are same now. And testContext.completeNow() does nothing.
When code is async you should add to the bottom of test (VertxExtension does this for you):
assertTrue(testContext.awaitCompletion(10, TimeUnit.SECONDS), "Timeout");
assertFalse(testContext.failed());
And if assert throws exception, the test without verify will end with timeout.
To fix that you should wrap your assert and call failNow:
try {
assert...
testContext.completeNow();
} catch (Exception e) {
textContext.failNow(e);
return;
}
TextContext::verify does this wrapping for you.
I am getting InvalidUseOfMatchersException on a different test than the one using Matchers
The below two tests are running fine individually but when running together, after the first test passes successfully, second test is failing and throwing InvalidUseOfMatchersException pointing to first test
#Test(expected = InputException.class)
public void shouldThrowExceptionWhenInputNull() {
calculator.calculateA(any(), any(), any(),eq(null));
}
#Test
public void testCalculateB() {
assertTrue(BigDecimal.valueOf(8000).compareTo(calculator.calculateB(12)) == 0);
}
This is the exception in stack trace
org.mockito.exceptions.misusing.InvalidUseOfMatchersException:
Misplaced or misused argument matcher detected here:
TestClass.shouldThrowExceptionWhenInputNull
According to the exception, first test should fail but its passing and second test is failing. Individually both these tests are passing successfully
calculator.calculateA(any(), any(), any(), eq(null));
This isn't a valid use of Matchers. Mockito only uses any and eq when used with when or verify, as a means of matching invocations that tell Mockito what to return or what calls should have been recorded. You'll need to call calculateA with specific values, such as calculator.calculateA(1, 2, 3, null);.
Mockito matchers work via side effects, so the only time that Mockito can throw an exception is the next time you interact with Mockito. This might be another method, but you can help ensure that those are local by using MockitoRule, MockitoJUnitRunner, or by adding a call to validateMockitoUsage from an #After method:
#After public void validateMockito() {
Mockito.validateMockitoUsage();
}
In my webdriver script I have the three methods
setup, test and tearDown
following the junit convention.
In the test method I have few asserts like this
#Test
public void testStudentHome() throws Exception {
String classCode = "I6OWW";
Utilities.studentSignin(driver, baseUrl);
assertEquals(true, sth.openNotification());
assertEquals("My Scores", sth.myScores(true));
}
The sth is the PageObject on which I am performing the tests and that I have created in the setup method.
I am calling all these three methods from a main method like this:
public static void main(String[] args) {
StudentHomeTest sht = new StudentHomeTest();
try {
sht.setup();
sht.testStudentHome();
sht.tearDown();
} catch (Exception ex) {
Logger.getLogger(StudentHomeTest.class.getName()).log(Level.SEVERE, null, ex);
sht.tearDown();
}
}
Now while running the test if some assertion fails the test method should (this is what I expect) throw an exception and the main method should call the tearDown method. But this does not happen. and the browser window continues to stay there.
I am using the netbeans ide for running the test.
following the junit convention
If you follow the jUnit convention, then you will know that teardown methods belong in the #After method as this method will always run after your tests.
create a new method with the #After jUnit annotation.
#After
public void tearDown() {
sht.tearDown();
}
Edit
You know what, I believe that you are running into a classic issue of assertEquals in jUnit.
Stolen from this answer...:
JUnit calls the .equals() method to determine equality in the method assertEquals(Object o1, Object o2).
So, you are definitely safe using assertEquals(string1, string2). (Because Strings are Objects)
--
Instead of using assertEquals on these calls, use assertTrue() instead.
assertTrue(sth.openNotification());
assertTrue("My Scores".equals(sth.myScores(true)));
AssertionError doesn't extend Exception - it's a Throwable.
But in any case, you should have
try {
sht.setup();
sht.testStudentHome();
} finally {
sht.tearDown();
}
No need for a catch block. main can throw Exception.