Need to run a job in Jenkins after successfully running the tests it needs to perform as post-build action for generating test reports
For this i have configured
Publish JUnit test result report
In the field
Test Report XMLs: continuum/*/target/surefire-reports/*TestSuite.xml
'continuum/*/target/surefire-reports/TestSuite.xml' doesn't match anything: 'continuum' exists but not 'continuum//target/surefire-reports/*TestSuite.xml'
Can you please help me out in resolving the error....??
I assume you have an 'Execute JUnit tests' Build step. This will produce a JUnit XML file to a location you specify, say, TestOutput/junitresults.xml.
In the 'Publish JUnit test result report' Post Build step you just need to specify TestOutput/junitresults.xml.
As long as your tests executed and produced the output file the Post Build step won't fail to publish it, whether the tests failed or not.
You shouldn't be trying to publish files in the surefire-reports directory unless that it where you told JUnit to write its output file. Normally you wouldn't.
If you want to make it even simpler just tell JUnit to write its output file to the Jenkins WORKSPACE root by removing the TestOutput/ and just specify junitresults.xml.
Related
We have .NET Core 3 codebase and the unit and integration tests are ran on the Gitlab CI.
Problem is, when one or more unit/int. tests fail, nothing specific is shown, you have to look at the entire pipeline dump and search to see for individual failed tests.
Looking at https://docs.gitlab.com/ee/ci/unit_test_reports.html, Junit report is exactly solving this issue.
Consulting with the How to capture structured xUnit test output in Gitlab CI?, I still wasn't able to find a proper solution.
Main problem is, there are multiple test projects that are executed with the dotnet test command:
current snapshot of gitlab.yaml file:
artifacts:
when: always
reports:
junit: ./Test.xml
script:
- for proj in $(dotnet sln MySolution.sln list | grep 'Test.csproj$'); do dotnet test --logger "junit;LogFilePath=Test.xml" $proj; done
Now the problematic is the script part, where we iterate through all the test assemblies and do the dotnet test for each project.
Is there a way to somehow produce a single junit xml log file out of each project and feed it to junit test report in the
reports:
junit: ./Test.xml
line?
You don't have to combine the reports. The artifacts:reports:junit key accepts multiple values, including glob patterns.
artifacts:
reports:
junit:
- "test.xml"
- "./tests/*.xml"
So, one solution would be to have all your XML output files in a particular directory and use a glob pattern in your .gitlab-ci.yml file that matches the many files.
If you really want to merge the xUnit XML files isntead, see this answer
I have a task to send reports of periodic execution of FitNesse tests to some specific endpoint in some specific JSON format.
I set periodic execution of tests in Jenkins properties and saving it in XML, but now I need to parse information about results of it.
It cannot be just step in "after build" property in Jenkins (or can, but I don't know a plugin for it), but what it would be and how I can do this?
Especially, I don't need information about the test, only general moments like date of the test, pass rate, status, name of the project, etc.
I think the best way to solve this is to make a script that parses the XML file, and creates the required JSON file. We normally use python scripts for this.
If you need certain generic information of the build in the script, like build number, you can pass this to your script using the Jenkins environments.
To call the script just add a batch or shell step, and place it below your fitnesse build step, to make sure the XML is generated before calling the script.
FitNesse comes with a jUnit runner which allows you to execute a test/suite. If you create a test class annotated with #RunWith(FitNesseRunner.class) and include its execution in a Jenkins Maven job (where the jUnit class is executed by either surefire or failsafe plugin), the outcome of the tests executed will be picked up automatically by Jenkins, just like it picks up other/regular jUnit tests (as surefire or failsafe will include them in their XML reports and Jenkins will pick these up).
You can find a sample Maven FitNesse project using (a slightly customised version of) this approach at https://github.com/fhoeben/sample-fitnesse-project. How to run the tests on Jenkins is described at https://github.com/fhoeben/hsac-fitnesse-fixtures#to-run-the-tests-on-a-build-server:
Have the build server checkout the project and execute mvn clean test-compile failsafe:integration-test. The result in JUnit XML results can be found in: target/failsafe-reports (Jenkins will pick these up automatically for a Maven job)
You indicate you don't need the HTML results, but they will be made available. They can be found in: target/fitnesse-results/index.html, and you could choose to use the 'HTML Publisher' Jenkins plugin to link to them from each build.
Can we have Junit test reports (not coverage) Test Pass / Fail Percentage report on Sonar Qube Dashboard (From my understanding, it should appear in the 'measures' tab) from the Junit report xml generated from Build ?
What is the step by step procedure to get Unit Test execution (Pass/Fail) reports.
You need to set sonar.tests path in sonar-project.properties file. It should be the directory where your test case report xml file is placed.
(https://docs.sonarqube.org/display/SONAR/Analysis+Parameters)
Once you will set the correct path, then you will be able to see the reports at Messures>Coverage.
Everything is clearly documented on the "Code Coverage by Unit Tests for Java Project" documentation page.
This will create metrics that you can indeed browse on the "Measures" page:
I have an ASP .NET MVC project that is built in a Jenkins project. We're using the nUnit plugin to fangle the output from our unit tests as a post-build step.
I've just added Jasmine tests for the javascript in the project and added a step in MSBuild to have Chutzpah run the Jasmine tests and output the results in jUnit format.
I added a post build step to process the jUnit results file and Jenkins runs the build and presents two 'Test Results' links in the Build result page...
However, when I click those links both go to the same result. When I left yesterday it was the nUnit results this morning it's the jUnit results so I'm guessing that it's just whichever finished last?
Is this possible? Do I need to do things a different way?
I would try just using the xUnit plugin, that way you can configure it to take bothe the junit and nunit test results all in a single publisher.
There is a bug in Jenkins related to this: NUnit Plugin fails to merge Unit test reports with unit tests split on basis of category.
To get around this, just put your NUnit post build action as the last that it works!
I'm stuck configuring tests for Jenkins. In the Publish JUnit test result report I have test-reports/*.xml but I'm getting the error:
'test-reports/*.xml' doesn't match anything: 'test-reports' exists but not 'test-reports/*.xml'
When I try */.xml I get:
Did not manage to validate **/*.xml (may be too slow)
When I throw an .xml file into test-reports folder manually it is deleted after the build.
What do?
It might actually be an issue with XCode. We figured out that it was actually skipping the application tests and it just wasn't evident anywhere on Jenkins' console output.