Im currently using QAF at my job and recently setup a few CICD tests using Azure Devops.
Azure Devops at the moment is only designed to read to Junit .XML results in their reporting dasbboard.
When running a Maven test, TestNG automatically creates a JunitReports folder that has a .xml report in it.
The issue is that when using QAF and using BDDStepFactory2 in my config.xml file, my junit report shows all tests as ignored.
Is there a way I can continue using BDDStepFactory2 and get the Junitresults to show either pass or fails?
EDIT:
The JunitResult is generating, however, this is what it looks like
<?xml version="1.0" encoding="UTF-8"?>
<!-- Generated by org.testng.reporters.JUnitReportReporter -->
<testsuite skipped="2" hostname=
name="com.qmetry.qaf.automation.step.client.DataDrivenScenario" tests="4" failures="0" timestamp="2021-12-27T16:21:34 EST" time="12.916" errors="0">
<testcase name="Validate the Home Page" time="7.287" classname="com.qmetry.qaf.automation.step.client.DataDrivenScenario"/>
<system-out/>
<testcase name="Validate the Home Page" time="5.629" classname="com.qmetry.qaf.automation.step.client.DataDrivenScenario"/>
<system-out/>
<testcase name="scenario" classname="com.qmetry.qaf.automation.step.client.DataDrivenScenario">
<skipped/>
</testcase> <!-- scenario -->
<system-out/>
<testcase name="scenario" classname="com.qmetry.qaf.automation.step.client.DataDrivenScenario">
<skipped/>
</testcase> <!-- scenario -->
<system-out/>
</testsuite> <!-- com.qmetry.qaf.automation.step.client.DataDrivenScenario-> ```
At the top, the xml is showing skipped="2". Since this is what Azure devops is reading, it is seeming like two of my tests have been skipped.
Most probably issue with TestNG version. Try latest (7.4.0) version of TestNG with latest (3.1.0-RC1) version of QAF.
<dependency>
<groupId>com.qmetry</groupId>
<artifactId>qaf</artifactId>
<version>3.1.0-RC1</version>
</dependency>
<!-- order is important -->
<dependency>
<groupId>org.testng</groupId>
<artifactId>testng</artifactId>
<version>7.4.0</version>
</dependency>
In order to write step information in junit report (just like testng reporter output) enable testng output by setting property.
report.log.testngoutput=true
Check your test-results directory, You may find junit report for each execution. For instance <project-root>\test-results\<timestamp>\junitreports. In case you don't found junit reports generated, you can enable testng default listeners.
Related
I've just migrated to JUnit 5 and I can't see any test cases in the XML test reports for my test suites. I've written my test suites in the new JUnit 5 syntax for junit-platform-suite and the tests in my suites all run successfully.
This generates JUnit XML reports via Surefire. One XML file is generated for each test class, plus one for the suite. The XML file for the suite has no <testcase> elements in it and shows that no tests have been run, even though they have.
Does anyone know why this might be? Are there perhaps some differences in XML reporting in JUnit 5?
Test suite class:
#Suite
#SelectClasses({ LoggedOutSupportPageTest.class,
LoggedInSupportPageTest.class
})
public class SingleTestSuite {
}
JUnit suite XML report structure:
<?xml version="1.0" encoding="UTF-8"?>
<testsuite xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="https://maven.apache.org/surefire/maven-surefire-plugin/xsd/surefire-test-report-3.0.xsd" version="3.0" name="com.myorganisation.suites.SingleTestSuite" time="17.231" tests="0" errors="0" skipped="0" failures="0">
<properties>
lots of properties are here
</properties>
</testsuite>
As part of a testng automation test suite I would like to automatically push results from jenkins to testrail. I currently have this plugin installed on my jenkins server: https://github.com/jenkinsci/testrail-plugin
The read me states the output must comply with the junit schema: https://github.com/windyroad/JUnit-Schema/blob/master/JUnit.xsd
I have reference How do I get one junit report from TestNG for all my test cases? and added
<listeners>
<listener class-name="org.testng.reporters.JUnitXMLReporter"></listener>
</listeners>
to my listeners; however, this does not seem to create a file in the correct format as this causes jenkins to fail with the message :
Uploading results to TestRail.
Error pushing results to TestRail
Posting to index.php?/api/v2/add_results_for_cases/236 returned an error! Response from TestRail is:
{"error":"Field :results cannot be empty (one result is required)"}
Build step 'TestRail Plugin' marked build as failure
Finished: FAILURE
I am wondering if there is a different listener I should be using instead.
Thank you for the help.
I used the xsd file that was shared in the question to create a TestNG reporter that complies with the xsd.
To consume this reporter, please add a dependency as below
<dependency>
<groupId>com.rationaleemotions</groupId>
<artifactId>junitreport</artifactId>
<version>1.0.0</version>
</dependency>
This reporter makes use of the service loader approach to wire in itself. So it doesn't need to be added explicitly via the <listeners> tag (or) the #Listeners annotation.
Details can be found here
HTML5 allows some tags to be written more liberally i.e without corresponding END tags. e.g. input need not be closed </input>.However if choose template mode HTML5 in Thymeleaf the Thymeleaf engine complains about this and does not parse the HTML template. I want to override this default Strict tag checking behavior. i.e Thymeleaf should parse an HTML template with meta and input (AND ALIKE) tags WITHOUT THEIR RESP. CLOSING TAGS. Pl. guide.
It also complains when you have something like this
<a href="/home/pic/image.png" download="/path/to/file" data-gallery></a>
It throws an exception when it encounters the data-gallery throws "should be followed by '=' " which is kind of annoying as it takes the flexibility out of HTML5.
All you have to do is run Thymeleaf in "LEGACYHTML5" mode and it works like a charm. Thanks to this and this post, I found the solution and am documenting in SO so others do not have to go through the same trouble in finding this answer.
To set the legacy mode you can define the bean in your Spring XML file:
<!-- View TemplateResolver -->
<bean id="templateResolver" class="org.thymeleaf.templateresolver.ServletContextTemplateResolver">
<property name="templateMode" value="LEGACYHTML5"/>
<property name="cacheable" value="false"/>
</bean>
or add the properties to the application.properties file:
spring.thymeleaf.mode=LEGACYHTML5
spring.thymeleaf.cache=false
And in both cases you have to add the nekohtml jar to your project or, if you are running maven, you can add its dependency to your pom.xml
<dependency>
<groupId>net.sourceforge.nekohtml</groupId>
<artifactId>nekohtml</artifactId>
<version>1.9.21</version>
</dependency>
Gradle
'net.sourceforge.nekohtml:nekohtml:1.9.21'
Here is how you can do it in a neat way
Step 1: Add thymeleaf mode to your application.properties file.
resources/application.properties
spring.thymeleaf.mode=LEGACYHTML5
Step 2: Add nekohtml dependency to your pom.xml file.
pom.xml
<dependency>
<groupId>net.sourceforge.nekohtml</groupId>
<artifactId>nekohtml</artifactId>
</dependency>
Using LEGACYHTML5 worked for me as well.
Also it is necessary to add
<dependency>
<groupId>net.sourceforge.nekohtml</groupId>
<artifactId>nekohtml</artifactId>
</dependency>
to pom.xml as stated above. But there is one more step which might occur. After doing those two steps I was getting:
java.lang.NoClassDefFoundError: org/w3c/dom/ElementTraversal
To avoid this it is necessary to add
<dependency>
<groupId>xml-apis</groupId>
<artifactId>xml-apis</artifactId>
<version>1.4.01</version>
</dependency>
to pom.xml
We have one xml configuration file that we use in production. We also have a little test app that has a couple of additional needs. What I'd like to do is create a second, testing-only xml config file that references the embedded production configuration file. Is there any way to do this?
I'm aware of the "include" element, but am not sure where in the file it is supposed to be placed--in the castle node? The components node?
I feel like the answer is here but I'm too dense to figure it out.
Thanks for any help you can provide.
UPDATE
This is how our production config file is set up:
<?xml version="1.0" encoding="utf-8"?>
<OurCompany>
<Framework>
<castle>
<installers>
<!-- some installers-->
<installers>
<components>
<!--some components-->
<components>
<castle>
<Framework>
<OurCompany>
My most recent attempt at a non-production config file looks like this:
<?xml version="1.0" encoding="utf-8"?>
<OurCompany>
<Framework>
<castle>
<include uri="assembly://AssemblyContainingEmbeddedXml/MyEmbeddedConfig.xml" />
<components>
<!--components I only want registered with container when running in non-production environment-->
<components>
<castle>
<Framework>
<OurCompany>
The exception I get reads:
Configuration parser encountered Framework, but it was expecting to find installers, facilities or components. There might be either a typo on or you might have forgotten to nest it properly.
(In the actual message, "Framework," "installers," "facilities," and "components" are enclosed in angle brackets.)
The bottom of the page you reference has an example of loading from an embedded resourced:
IResource resource = new AssemblyResource("assembly://Acme.Crm.Data/Configuration/services.xml");
container = new WindsorContainer(new XmlInterpreter(resource));
I have integrated Jenkins to C unit test library and I am parsing the output and generating XML in the schema that the jenkins understands.
while generating the xml output, I just show the gist of the errors reported, and I would like to create a link to where the actual logs are located so that the user can click on teh link in jenkins to view the actual file.
I am unable to add the HTML tags under this XML. The Jenkins turns them into encoded <.. and makes my link useless.
I even try to use . But I am still not able to get it.
Has anyone tried creating a link under jenkins report. Any help is much appreciated.
The following have not worked.
1.<failure> blah.. blah.. (my brief log summary)
ERROR
</failure>
2. <failure> blah.. blah.. (my brief log summary)
<a href=\quot;www.stackoverflow.com\quot;> ERROR </a>
</failure>
3. <failure>
<![CDATA[ ERROR]]>
</failure>
4. <failure>
//CDATA with encoded > <
Thanks in advance
Out testing framework has pluggable reporting modules - a call from a test generates output for all 'registered' modules. Among those are JUnit, HTML, and SQL.
The first one generates JUnit reports that can be immediately processed by Jenkins to decide if the build is stable or not, draw the nice trend graph, etc.
The second one generates the error log in a human-readable form. Over time it has been embellished with all kinds of bells and whistles - different colors for different kinds of status, shrinkable/expandable sections, and so on. A link to this log is displayed among the artifacts for a build.
SQL logger uploads test results into a database for eternal storage and heavy-duty analysis.
In my case I've just put the html address there and it was parsed autmatically in the page...
This works for me:
<failure message="test failure <a href="https://www.w3schools.com/">More info here!</a > " > Assertion failed </failure>
.
<?xml version="1.0" encoding="UTF-8"?>
<testsuites>
<testsuite name="JUnitXmlReporter" errors="0" tests="0" failures="0" time="0" timestamp="2013-05-24T10:23:58" />
<testsuite name="JUnitXmlReporter.constructor" errors="0" skipped="1" tests="3" failures="1" time="0.006" timestamp="2013-05-24T10:23:58">
<properties>
<property name="java.vendor" value="Sun Microsystems Inc." />
<property name="compiler.debug" value="on" />
<property name="project.jdk.classpath" value="jdk.classpath.1.6" />
</properties>
<testcase classname="JUnitXmlReporter.constructor" name="should default path to an empty string" time="0.006">
<failure message="test failure <a href="https://www.w3schools.com/">More info here!</a > " > Assertion failed </failure>
</testcase>
<testcase classname="JUnitXmlReporter.constructor" name="should default consolidate to true" time="0">
<skipped />
</testcase>
<testcase classname="JUnitXmlReporter.constructor" name="should default useDotNotation to true" time="0" />
</testsuite>
</testsuites>