Does JUnit has the possibility to add a description text to each test method so that the description text appears later in the surefire/failsave xml report!?
Background: i work in a regulated environment where a lot of documentation, test specifications and test reports must be written.
The JUnit test reports are part of the documentation and we would like to put the test description directly to the JUnit tests to have them in one place. Because tests evolves and appears and some are deleted, it is hard to have the test description in a different tool or place than the tests.
What i like to have is something like this:
public class MyTest {
#Test
#TestDescription("The test will do this and that. Preconditions are:
1. bla blubb
2. bla fasel"
public void testDoMyTestShouldCheckSomething {
[...]
}
}
I would look at a similar scheme to my answer to JUnit test report enrichment with JavaDoc.
Basically, use a RunListener which produces a report based on the outcome of the tests, using an annotation on the test to provide a description.
Related
My team is just getting started with X-Ray, and we are setting up our pipelines. However, while doing this I noticed that if I submit a Junit xml file to X-Ray via the REST api, it will create new tests for any test data that isn't already in the system.
Is there a way to have X-Ray ignore test results for tests that don't exist for the test execution? I don't want it constantly creating extra tests.
For example:
(Jira/X-Ray Server) TestExecution MyExecution has test testA
From client, I submit a Junit xml file containing results for testA and testB in the MyExecution TestExecution
testB now exist on the server under MyExecution
I would like to be able to submit the Junit xml file without it creating extra tests.
Whenever you import automation results using the REST API, or any of the available CI plugins, Xray will autoprovision ("generic") Test entities.
The flow is detailed here.
Xray tries to find a unique identifier for the automated test; in the case of JUnit, it's based on the full classname plus the name of the test method; this will become part of the Generic Definition field. The process for JUnit is described in more detail here.
How it works for a different test automation framework/report formats, is similar and is detailed on respective documentation pages.
If a "generic" Test is found, then the Test is reused and a Test Run is created against it. Otherwise, the Test will be auto-provisioned.
This process isn't configurable. However, in theory, if the user that you use for the submission of automation results isn't able to create Test issues, you may have what you need.
Things like these are usually not configurable because they are normally a consequence of applying good practices usually discussed internally with the team(s).
I'm looking for the best practice for following (simplified) scenario:
#Test
public void someTest() {
for(String someText : someTexts) {
Assert.true(checkForValidity(someText));
}
}
This test iterates through x-thousands of texts and in this case I don't want it to be stopped for each failure. I want the errors to be buffered and in case of error(s) to fail at the end. Has JUnit got something on board for for my aim?
First of all, it's not really the correct way to implement this. JUnit allows parametrizing tests by defining a collection of inputs/outputs with the Parametrized test runner. Doing it this way ensures that each test case becomes a unique instance, making test report clearly state which samples passed and which ones failed.
If you still insist on doing it your way you should have a look at AssertJ's Soft Assertions which allow "swallowing" individual assertion failures, accumulating them and only reporting after the test is finished. The linked documentation section uses a nice example and is definitely worth reading.
Can anyone explain the difference to me between Cucumber and Junit
From my understanding they are both used to test Java code although I am not sure of the difference?
Are they simply difference implementations of the same test suite or aimed at testing different things?
Cucumber and JUnit are different and solve different things.
Cucumber is a Behavior Driven Design (BDD) framework that takes "stories" or scenarios written in human readable languages such as English and turns those human readable text into a software test.
here's an Example cucumber story:
cucumber will then knows how to turn this text into a software test to make sure the software works as described. The output will tell you if the story is actually what the software does and if not, what was different:
Here's where the code is fixed to make the cucumber test pass:
This makes what is called an "Executable Specification" which is a nice way of documenting all of the features your software supports. This is different than normal documentation because without the corresponding test, someone reading the document doesn't know if the documentation is up to date.
Other Benefits of Executable Specifications:
Non-programmers can read and understand the tests
Non-programmers can write the tests since they are in plain English.
BDD results and Executable Specifications are very high level. They cover the overall features and perhaps a few edge cases as examples but don't test every possible condition or every code path. Also BDD tests are "integration tests" in that they test how all your code modules work together, but they don't test everything thoroughly.
This is where JUnit comes in.
JUnit is a lower level "Unit test" tool that allows developers to test every possible code path in their code. Each module of your code (or classes, or even methods) is tested in isolation. It is much more low level than a BDD framework. Using the same calculator story as the Cucumber example, JUnit tests would test lots of different calculation examples and invalid inputs to make sure the program responds correctly and computes the values correctly.
Hope that helps
I think Cucumber is more used for integration tests, while JUnit is more used in behaviour tests instead. Besides, Cucumber syntax is more accurate than JUnit, but much more complex. Here you can see a Cucumber test example:
package com.c0deattack.cucumberjvmtutorial;
import cucumber.annotation.en.Given;
import cucumber.annotation.en.Then;
import cucumber.annotation.en.When;
import cucumber.runtime.PendingException;
public class DepositStepDefinitions {
#Given("^a User has no money in their account$")
public void a_User_has_no_money_in_their_current_account() {
User user = new User();
Account account = new Account();
user.setAccount(account);
}
#When("^£(\\d+) is deposited in to the account$")
public void £_is_deposited_in_to_the_account(int arg1) {
// Express the Regexp above with the code you wish you had
throw new PendingException();
}
#Then("^the balance should be £(\\d+)$")
public void the_balance_should_be_£(int arg1) {
// Express the Regexp above with the code you wish you had
throw new PendingException();
}
private class User {
private Account account;
public void setAccount(Account account) {
this.account = account;
}
}
private class Account {
}
}
You can see that JUnit is more simple, but not necessarily less powerful:
import static org.junit.Assert.assertEquals;
import org.junit.AfterClass;
import org.junit.BeforeClass;
import org.junit.Test;
public class MyClassTest {
#Test(expected = IllegalArgumentException.class)
public void testExceptionIsThrown() {
MyClass tester = new MyClass();
tester.multiply(1000, 5);
}
#Test
public void testMultiply() {
MyClass tester = new MyClass();
assertEquals("10 x 5 must be 50", 50, tester.multiply(10, 5));
}
}
Hope it helps,
Clemencio Morales Lucas.
Cucumber is in which you can do BDD Behavior Driven Development. Something like you can convert your functional use case into Cucumber story. In that sense you can also take Cucumber to be DSL for Functional Use Case Document.
JUnit on other side is for unit testing which would be a method in Java. So a use can be unit test (rarely though) to inyegration test or a full system test, that's your first case Cucumber. Unit Test will be Unit Test only.
I do the following:
From the Package Explorer I select "New, Other, JUnit Test Case"
I write this code:
package dk.sample;
import org.junit.*;
import static org.junit.Assert.*;
public class TestCase {
#Test
public void alwaysTrue(){
assertTrue( true );
}
}
I then select "Run As, JUnit test"
Get this error: "Class not found dk.sample.TestCase
java.lang.ClassNotFoundException: ...."
What do I miss? Have tried with different Run Configurations - but it seems like I miss a classpath somewhere? But to what and where?
To make JUnit work within Domino Designer you need to perform few additional steps:
set up source control for your application
adjust the on-disk project to be recognized as Java application
run JUnit tests within your on-disk project
Please note that java agents have to be tested in a different way..
You can find more detailed explanation about enabling JUnit for both XPages and Agents in the following blog post: Unit Tests for Lotus Domino Applications
Here's also a great how-to on this topic.
Coundn't get JUnit to work inside the Domino Designer. Instead of running the tests from DDE, I now run the tests from a XPages. This works like a dream. Made my own 'JUnit runner' class - that is, I just call the JUnit runners but handles the result my self in order to display it as html on the XPage.
Code can be found here: http://xpages.dk/wp-content/uploads/2013/10/junitrunner.txt
Danish blog post here: http://xpages.dk/?p=1162
I have several Spock test classes grouped together in a package. I am using Junit 4.10. Each test class contains several feature test methods.
I want to perform some setup steps (such as loading data into a DB, starting up a web server) before I run any test case, but only once when the testing starts.
I want this "OneTimeSetup" method to be called only once whether:
I run all the test classes in the package (for example if they are grouped in a Test Suite)
I run a few test classes
I run only one test class
I run only a certain feature method within a test class
From reading other posts on SO, it seems that this is what TestNG's #BeforeSuite does.
I am aware of Spock's setupSpec() and cleanupSpec() methods, but they only work within a given test class. I am looking to do something like "setupTestSuite()." How can this be achieved in Spock?
You can write a global extension, use a JUnit test suite, call a static method in a helper class (say from setupSpec) that does its work just once, or let the build tool do the job.