when #beforeclass executes assertion failure , it is not causing test result to fail in jenkins. how can I change this to failure.
You need to add a post build step to check the junit test results. We use "Publish JUnit test result report" and add the location where the results output (xml files) are stored.
Related
As part of a testng automation test suite I would like to automatically push results from jenkins to testrail. I currently have this plugin installed on my jenkins server: https://github.com/jenkinsci/testrail-plugin
The read me states the output must comply with the junit schema: https://github.com/windyroad/JUnit-Schema/blob/master/JUnit.xsd
I have reference How do I get one junit report from TestNG for all my test cases? and added
<listeners>
<listener class-name="org.testng.reporters.JUnitXMLReporter"></listener>
</listeners>
to my listeners; however, this does not seem to create a file in the correct format as this causes jenkins to fail with the message :
Uploading results to TestRail.
Error pushing results to TestRail
Posting to index.php?/api/v2/add_results_for_cases/236 returned an error! Response from TestRail is:
{"error":"Field :results cannot be empty (one result is required)"}
Build step 'TestRail Plugin' marked build as failure
Finished: FAILURE
I am wondering if there is a different listener I should be using instead.
Thank you for the help.
I used the xsd file that was shared in the question to create a TestNG reporter that complies with the xsd.
To consume this reporter, please add a dependency as below
<dependency>
<groupId>com.rationaleemotions</groupId>
<artifactId>junitreport</artifactId>
<version>1.0.0</version>
</dependency>
This reporter makes use of the service loader approach to wire in itself. So it doesn't need to be added explicitly via the <listeners> tag (or) the #Listeners annotation.
Details can be found here
I am running Junit tests using Eclipse Luna. I have implemented #Test method. I loop within the #Test method for multiple records and use Assert.assertEquals for Non-XML messages and XMLAssert.assertXMLEqual for XML messages.
The problem is, when I run the Junit with single or multiple test cases, I do not get the proper result in the Junit View. It always shows "Runs: 1/1" and does not show the correct count of runs. Even the failures and success are not shown correctly. Am I missing something here?
If you only have a single #Test method, there is only one thing running, so the Runs: 1/1 is correct. If you want to have more show up, make each assertion in its own test.
At our organization we are following this DSL model Domain specific language and stuff where users can write tests from a spreadsheet and the underlying java code understands and executes those instructions.
Now here is the problem.
We have a single test method in our class which uses a data provider, reads all the test methods from the file and executes the instructions.
Naturally, when surefire executes and prints results it says:
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
Is there a way to manipulate this in TestNG such that each custom test metod from excel can be picked up by the system as a legitimate test method when the overall suite executes.
I actually made the group migrate from Junit to TestNG and they are questioning if the DataProvider feature can handle that and i have no response for it :(
So essentially we want to break bindings between java methods by using external data providers but at the same time preserve the number of test methods executed as provided in an excel spreadsheet.
If you can give me any direction it would be most helpful to me.
Attaching my spreadsheet here.
My java file has only 1 test method:
#test
RunSuite(){
// Read each test method from file, i want the build server to recognize them someway as a individual test methods
}
I have not found any related searches to my problem here and hence Im writing this one.
I have a JUnit Class
MyHandler Class with tests - testx,testy,testz
that calls other JUnit Classes
MyTestA, MyTestB classes each having tests
based on some logic
MyTestA has tests - testa1, testa2, testa3
MyTestB has tests - testb1, testb2, testb3
Everything works fine, except that the report shows only 3 tests executed (testx,testy,testz) - Though all the tests in MyTestA, MyTestB are also executed, they are not part of the report.
I see this when using Junit reporting using ant and also in Eclipse IDE.
Within MyHandler Class, I am calling the JUnit classes as ,
org.junit.runner.JUnitCore.runClasses(MyTestA)
org.junit.runner.JUnitCore.runClasses(MyTestB)
Is there anyway, we can include the individual called junit tests in the Junit report?
We are using ant to build along with junit reporting.
Appreciate any help on this,
Sudhakar
JUnit will only report on the structure it knows about, so use a JUnit test suite (Junit4 Test Suites). You can list all the test classes and JUnit will run them and also include them in the report.
I am executing a Junit test case that takes user inputs.
Junit is executed through ANT, but the script abruptly exits stating
java.lang.NumberFormatException: null
is it possible to pass dynamic input this way?
You could probably use Java System Properties for this. But I think that your unit test should not be dependent of any user input. It should executes all the time the same to ensure your code is working.