JUnit optional/required tests - junit

I have Junit 4.8.2 and maven 3
Some tests in my application should fail the build in case of failure and some of them shouldn't (just report that the following optional tests failed)
How can I do it with junit and if I can't then maybe testng can?
E.g. I have two test cases:
First is not really important and can failed because of connection timeout, service unavailability and so on so on. So if it fail, I don't want to fail whole build, just to let user know about it and write to console
Second is really important one and if it fail - build should be failed as well
I know about #Ignore - it is not what I'm looking for, because I don't want to skip any tests.
I know about #Category so if you know how to configure surefire plugin to say: if category com.me.Required - build should be failed in case of failure and if category com.me.Optional - build should not be failed

Consider using the failsafe plugin for your tests that are allowed to fail and set the testFailureIgnore flag to true.
To use the failsafe plugin you have to add the plugin to you pom
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>foo.bar</groupId>
<artifactId>test</artifactId>
<version>0.0.1-SNAPSHOT</version>
<dependencies>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.10</version>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.12.4</version>
<executions>
<execution>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
</execution>
</executions>
<configuration>
<testFailureIgnore>true</testFailureIgnore>
</configuration>
</plugin>
</plugins>
</build>
</project>
The surefire plugin will per default execute test named like Test. The failsafe plugin will per default execute the test named like IT.
Given the tests
import static org.junit.Assert.*;
import org.junit.Test;
public class SurefireTest {
#Test
public void test() {
assertTrue(true);
}
}
and
import static org.junit.Assert.*;
import org.junit.Test;
public class FailsafeIT {
#Test
public void test() {
assertTrue(false);
}
}
Running mvn install will now result in
[INFO] Scanning for projects...
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building test 0.0.1-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO]
.
.
.
-------------------------------------------------------
T E S T S
-------------------------------------------------------
Running SurefireTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.062 sec
Results :
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
.
.
.
-------------------------------------------------------
T E S T S
-------------------------------------------------------
Running FailsafeIT
Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.072 sec <<< FA
ILURE!
...
Results :
Failed tests: test(FailsafeIT)
Tests run: 1, Failures: 1, Errors: 0, Skipped: 0
.
.
.
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 3.174s
[INFO] Finished at: Sat Sep 29 08:19:38 CEST 2012
[INFO] Final Memory: 9M/245M
[INFO] ------------------------------------------------------------------------

You can use #Ignore, see Is there a way to skip only a single test in maven?
You can skip tests in a certain package Is there a way to tell surefire to skip tests in a certain package? or http://maven.apache.org/plugins/maven-failsafe-plugin/examples/inclusion-exclusion.html or
You can use JUnit 4.8 Categories JUnit Categories http://maven.apache.org/plugins/maven-failsafe-plugin/examples/junit.html
You can use skipITs http://maven.apache.org/plugins/maven-failsafe-plugin/verify-mojo.html#skipITs
I think JUnit 4.8 Categories is actually what are you looking for.

Related

Upgrading maven-surefire-plugin from 2.19.1 to 2.22.1 causes tests to not run

I have JUnit 4 tests that run with JUnit Jupiter (JUnit 5) using the vintage engine and maven-surefire-plugin version 2.19.1.
[INFO] --- maven-surefire-plugin:2.19.1:test (default-test) # jon-snow ---
[INFO] Tests are skipped.
[INFO]
[INFO] --- maven-surefire-plugin:2.19.1:test (unit-tests) # jon-snow ---
-------------------------------------------------------
T E S T S
-------------------------------------------------------
Running com.whatever.WhateverTest
Tests run: 5, Failures: 0, Errors: 0, Skipped: 0 ...
When I upgrade the maven-surefire-plugin version to 2.22.1, no tests are detected.
[INFO] --- maven-surefire-plugin:2.22.1:test (default-test) # jon-snow ---
[INFO] Tests are skipped.
[INFO]
[INFO] --- maven-surefire-plugin:2.22.1:test (unit-tests) # jon-snow ---
[INFO]
[INFO] -------------------------------------------------------
[INFO] T E S T S
[INFO] -------------------------------------------------------
[INFO]
[INFO] Results:
[INFO]
[INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0
Why is this? I have 248 tests that should run. What dependencies or config changes do I need to add to make tests work again?
Similar to what #Marvin was stating, you will need to make sure you update your version of the jupiter engine plugin accordingly. Also if you're using an older jUnit version then utilize an updated dependency for it.
A small sample with each is provided below.
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>${maven-surefire-plugin.version}</version>
<dependencies>
<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter-engine</artifactId>
<version>5.5.2</version>
</dependency>
</dependencies>
...
</plugin>
If you're using older jUnit dependencies include this dependency
<dependency>
<groupId>org.junit.vintage</groupId>
<artifactId>junit-vintage-engine</artifactId>
<version>5.5.2</version>
</dependency>
In the sample above we are using maven-surefire-plugin version 2.22.2

With the shaded jar from dropwizard I got: Class path contains multiple SLF4J bindings

I probably made something stupid here. There is something (smallish) wrong with my dropwizard setup. Running the shaded jar works fine, but when executing integration tests I get this warning:
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/graphhopper/web/target/graphhopper-web-0.11-SNAPSHOT.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/user/.m2/repository/ch/qos/logback/logback-classic/1.2.3/logback-classic-1.2.3.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [ch.qos.logback.classic.util.ContextSelectorStaticBinder]
INFO [2018-04-08 18:44:27,653] org.eclipse.jetty.util.log: Logging initialized #1090ms to org.eclipse.jetty.util.log.Slf4jLog
Where graphhopper-web-0.11-SNAPSHOT.jar is the shaded jar with dropwizard (with logback).
This usually means that more than one slf4j binding is on the classpath but I could reject this theory and only slf4j-api is there, plus the logback dependency for dropwizard. I also have analyzed the dependency graph with Netbeans and
mvn dependency:tree -Dverbose -Dincludes=org.slf4j
(see output here)
but couldn't find something problematic.
Can it be that the shaded jar (with logback) is somehow put into the classpath with the other jars (including logback) for mvn clean install? How could I avoid this?
Reproduce it via:
git clone https://github.com/graphhopper/graphhopper
cd web
mvn clean install
See this issue.
imported your code into my intellij community edition, added exclusion as follows to all the dependency in pom.xml of web module. It resolve issues given issue. You may have do changes in respective module.
example:
<dependency>
<groupId>io.dropwizard</groupId>
<artifactId>dropwizard-core</artifactId>
<version>1.2.2</version>
<exclusions>
<exclusion>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
</exclusion>
</exclusions>
</dependency>
The issue is the jar logback-classic-1.2.3.jar has a actual file /org/slf4j/impl/StaticLoggerBinder.class, so jar exclusion won't work.
You need to change the configuration of the maven-shade-plugin
<configuration>
<createDependencyReducedPom>true</createDependencyReducedPom>
<filters>
<filter>
<artifact>ch.qos.logback:logback-classic</artifact>
<excludes>
<exclude>org/slf4j/impl/**</exclude>
</excludes>
</filter>
<filter>
<artifact>*:*</artifact>
<excludes>
<exclude>META-INF/*.SF</exclude>
<exclude>META-INF/*.DSA</exclude>
<exclude>META-INF/*.RSA</exclude>
</excludes>
</filter>
</filters>
</configuration>
Once you do that all is good and the warning is gone
[INFO] Scanning for projects...
[INFO]
[INFO] ------------------< com.graphhopper:graphhopper-web >-------------------
[INFO] Building GraphHopper Web 0.11-SNAPSHOT
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) # graphhopper-web ---
[INFO] Deleting /Users/tarun.lalwani/Desktop/tarunlalwani.com/tarunlalwani/workshop/ub16/so/graphhopper/web/target
[INFO]
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) # graphhopper-web ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 118 resources
[INFO]
[INFO] --- maven-compiler-plugin:3.5.1:compile (default-compile) # graphhopper-web ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 19 source files to /Users/tarun.lalwani/Desktop/tarunlalwani.com/tarunlalwani/workshop/ub16/so/graphhopper/web/target/classes
[INFO]
[INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) # graphhopper-web ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 1 resource
[INFO]
[INFO] --- maven-compiler-plugin:3.5.1:testCompile (default-testCompile) # graphhopper-web ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 9 source files to /Users/tarun.lalwani/Desktop/tarunlalwani.com/tarunlalwani/workshop/ub16/so/graphhopper/web/target/test-classes
[INFO]
[INFO] --- maven-surefire-plugin:2.19.1:test (default-test) # graphhopper-web ---
-------------------------------------------------------
T E S T S
-------------------------------------------------------
Running com.graphhopper.http.WebHelperTest
Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.151 sec - in com.graphhopper.http.WebHelperTest
Running com.graphhopper.http.IPFilterTest
20:29:46.762 [main] DEBUG com.graphhopper.http.IPFilter - whitelist:[4.5.67.1, 1.2.3.4]
20:29:46.765 [main] DEBUG com.graphhopper.http.IPFilter - blacklist:[8.9.7.3]
20:29:46.765 [main] DEBUG com.graphhopper.http.IPFilter - blacklist:[4.5.67.1, 1.2.3.4]
20:29:46.766 [main] DEBUG com.graphhopper.http.IPFilter - whitelist:[4.5.67.1, 1.2.3.4]
20:29:46.766 [main] DEBUG com.graphhopper.http.IPFilter - whitelist:[1.2.3*, 4.5.67.1, 7.8.*.3]
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.003 sec - in com.graphhopper.http.IPFilterTest
Results :
Tests run: 8, Failures: 0, Errors: 0, Skipped: 0
Edit-1: 26-Apr-2018
Updates from https://github.com/graphhopper/graphhopper/issues/1328
I believe the issue is that you run the Dropwizard app integration tests as integration test during the verify phase of the build. This phase gets invoked after the package phase where you create a shaded jar and replace the original jar with it.
[INFO] Replacing original artifact with shaded artifact.
[INFO] Replacing /home/travis/build/graphhopper/graphhopper/web/target/graphhopper-web-0.11-SNAPSHOT.jar with /home/travis/build/graphhopper/graphhopper/web/target/graphhopper-web-0.11-SNAPSHOT-shaded.jar
As a result you have two jars in the classpath with two slf4j bindings (graphhopper-web-0.11-SNAPSHOT and logback-classic-1.2.3.jar). There's no need to create a shaded jar for running an integration test against a Dropwizard application, it can be run as a unit test. So, if you configure graphhopper-web to run its integration tests during the test phase, there will be no warning:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.19.1</version>
<executions>
<execution>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
</execution>
</executions>
<configuration>
<excludes>
<exclude>com.graphhopper.http.**</exclude>
</excludes>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.19.1</version>
<configuration>
<argLine>-Xmx100m -Xms100m</argLine>
<includes>
<include>com.graphhopper.http.**</include>
</includes>
</configuration>
</plugin>
Another option is to rename them from *IT to *Test, so they are not automatically matched as integration tests.

How to deploy mysql datasource using jboss-as-maven-plugin

This question is here to give an answer to once i find a solution, I have found documentation on Postgresql and H2 databases from the jboss website and have seen how it is done manually through this website, however I cannot seem to find much information on how to deploy a mysql datasource using the jboss-as-maven-plugin.
What is the minimum configuration properties required to properly register a mysql datasource with jboss-as 7 server through their maven plugin?
I have this dependency:
<dependency>
<groupId>mysql</groupId>
<artifactId>mysql-connector-java</artifactId>
</dependency>
And this configuration for the maven plugin
<build>
<plugins>
...
<plugin>
<groupId>org.jboss.as.plugins</groupId>
<artifactId>jboss-as-maven-plugin</artifactId>
<configuration>
<execute-commands/>
<executeCommands/>
<properties>
<enable-welcome-root>false</enable-welcome-root>
</properties>
</configuration>
<executions>
...
<!-- deploy the mysql connectorj -->
<execution>
<id>deploy-mysql-driver</id>
<phase>install</phase>
<goals>
<goal>deploy-artifact</goal>
</goals>
<configuration>
<groupId>mysql</groupId>
<artifactId>mysql-connector-java</artifactId>
<name>mysql.jar</name>
</configuration>
</execution>
<execution>
<id>deploy</id>
<phase>install</phase>
<goals>
<goal>deploy</goal>
</goals>
</execution>
<execution>
<id>add-datasource</id>
<phase>deploy</phase>
<goals>
<goal>add-resource</goal>
</goals>
<configuration>
<address>subsystem=datasources</address>
<resources>
<resource>
<address>xa-data-source=java:global/datasources/tncDS</address>
<enable-resource>true</enable-resource>
<properties>
<jndi-name>java:jboss/datasources/tncDS</jndi-name>
<enabled>true</enabled>
<connection-url>jdbc:mysql://localhost:3306/tnc</connection-url>
<driver-class>com.mysql.jdbc.Driver</driver-class>
<driver-name>mysql.jar</driver-name>
</properties>
</resource>
</resources>
</configuration>
</execution>
</executions>
...
</plugin>
</plugins>
</build>
running the command mvn jboss-as:run causes this error:
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 7.241s
[INFO] Finished at: Tue Sep 24 21:37:28 EST 2013
[INFO] Final Memory: 16M/308M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.jboss.as.plugins:jboss-as-maven-plugin:7.4.Final:deploy-artifact (deploy-mysql-driver) on project ear: Could not execute goal deploy-artifact on null. Reason: I/O Error could not execute operation '{
[ERROR] "address" => [],
[ERROR] "operation" => "read-attribute",
[ERROR] "name" => "launch-type"
[ERROR] }': java.net.ConnectException: JBAS012144: Could not connect to remote://localhost:9999. The connection timed out
[ERROR] -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
UPDATE:
I have developed a plugin that injects the required file (META-INF/services/java.sql.Driver) into the jar before deployment:
<plugin>
<groupId>com.thenaglecode</groupId>
<artifactId>mysql-jdbc-compliance-maven-plugin</artifactId>
<version>1.0-SNAPSHOT</version>
<configuration>
<artifactId>mysql-connector-java</artifactId>
<version>5.1.26</version>
</configuration>
<executions>
<execution>
<goals>
<goal>modify-connector</goal>
</goals>
<phase>package</phase>
</execution>
</executions>
</plugin>
However i'm still getting could not connect to remote message. Is there a step that i'm missing or doing in the wrong sequence as i understand that the run command should start up the server.
UPDATE 2:
After some fiddling and reading of the jboss-as plugin website, I realized that the jboss-as:run goal also invokes the package phase. I was mostly receiving this error when I tried to run any of the deploying goals bound to the package phase.
Anything that needs to be deployed should be bound to the install phase.
I am now getting a seperate error regarding my persistence unit not existing
The problem is the MySQL driver is not JDBC 4 compliant. You need to add a META-INF/services/java.sql.Driver file to the JAR with the fully qualified JDBC driver class name or install it as a module. See https://community.jboss.org/wiki/DataSourceConfigurationInAS7 for more details.
solution here:
conf : jboss as7, maven 3.3.3, mysql java connector 5.1.29 28 27 26...,
starting from mysql java connector 5.1.30, the file Meta-INF/serivces/java.sql.driver contain this line "com.mysql.jdbc.Driver
com.mysql.fabric.jdbc.FabricMySQLDriver", if I change it to "com.mysql.jdbc.Driver" the datasource is created and the project deployed fine.
with mysql java connector 5.1.29 28 27 26 ..., that file contain only "com.mysql.jdbc.Driver", the deployement work fine.
check my pom.xml here
https://github.com/anouarattn/GestionAbsence

Failsafe html reports in Jenkins

I have some integration tests (with Selenium) which are run with failsafe maven plugin. Failsafe generates XML reports files only.
1) I want to generate HTML reports
2) I want to have a link in Jenkins to the html reports
For the 1) I installed the "maven-surefire-report-plugin" to use the failsafe-report-only goal.
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-report-plugin</artifactId>
<version>2.13</version>
<executions>
<execution>
<phase>post-integration-test</phase>
<goals>
<goal>failsafe-report-only</goal>
</goals>
</execution>
</executions>
</plugin>
But in the standard output, nothing seems to be generated :
[INFO]
[INFO] >>> maven-surefire-report-plugin:2.13:failsafe-report-only (default) # BaseContrats >>>
[INFO]
[INFO] <<< maven-surefire-report-plugin:2.13:failsafe-report-only (default) # BaseContrats <<<
[INFO]
[INFO] --- maven-surefire-report-plugin:2.13:failsafe-report-only (default) # BaseContrats ---
In my failsafe-reports directory, I have only XML report files but not the HTML ones.
Is it the good plugin to generate html reports for failsafe?
For the 2), I installed the Jenkins plugin "Selenium HTML report" and added the post build action "Publish Selenium HTML report" and configured it with "target/failsafe-reports" value for "Selenium tests results location" parameter, but nothing is displayed in Jenkins interface (surely because my html reports file are not generated...).
Could you help me for these 2 points?
Answer for the part 1) to generate HTML reports for Failsafe.
Add the following to pom.xml
<reporting>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-report-plugin</artifactId>
<version>2.18.1</version>
<configuration>
<skipSurefireReport>${skipSurefireReport}</skipSurefireReport>
<reportsDirectories>
<reportsDirectory>${basedir}/target/failsafe-reports</reportsDirectory>
</reportsDirectories>
</configuration>
</plugin>
</plugins>
</reporting>
<properties>
<skipSurefireReport>true</skipSurefireReport>
</properties>
I am assuming you are using mvn verify to run the integration tests using failsafe-plugin.
By default, this generates the (*.txt & *.xml) reports in {basedir}/target/failsafe-reports. This should be specified in <reportDirectories>. You can point to any directory which has TEST-*.xml reports.
And then run the cmd
mvn site
This would generate html reports for the Failsafe run integration tests. By default, The files would be in {basedir}/target/site/failsafe-report.html.
This location be changed.

Send test reports to different directory using maven-surefire-plugin

I want to generate my reports so I used maven-surefire-plugin.
This is my pom.xml:
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.10</version>
<executions>
<execution>
<id>test-reports</id>
<phase>install</phase>
<configuration>
<tasks>
<junitreport todir="./reports">
<fileset dir="./reports">
<include name="TEST-*.xml"/>
</fileset>
<report format="frames" todir="./report/html"/>
</junitreport>
</tasks>
</configuration>
<goals>
<goal>test </goal>
</goals>
</execution>
</executions>
</plugin>
But I'm getting this error when I run maven test:
Tests run: 6, Failures: 2, Errors: 2, Skipped: 0
[INFO] --------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] --------------------------------------------------------------------
[INFO] Total time: 10.403s
[INFO] Finished at: Wed Oct 12 08:35:10 CEST 2011
[INFO] Final Memory: 17M/126M
[INFO] --------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire- plugin:2.10:test (default-test) on project jebouquine: There are test failures.
[ERROR]
[ERROR] Please refer to C:\Users\Amira\workspace1\jebouquine\target\surefire-reports for the individual test results.
[ERROR] -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
When I open C:\Users\Amira\workspace1\jebouquine\target\surefire-reports I find repotrts but I want them under ./reports.
Any idea how to do it? Or is there any alternative to generate my tests reports?
Look, here's the bottom line: you need to read more about Maven because what you have written does not make sense. You have tried to stuff ant configuration elements into the surefire plugin config, which is not right at all. And you have tried to bind it to the install phase, but then you are running mvn test.
Finally, it might help if you explained why you want them in a different directory in the first place ... there might be a better way to achieve your real goals.
However, this will fix your problem for the next 5 seconds or so:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.10</version>
<configuration>
<reportsDirectory>${project.build.directory}/reports</reportsDirectory>
</configuration>
</plugin>