I have several projects in my organization that have a mix of unit tests.
Half of them use Arquillian, half use Mockito/CDIUnit.
In order to get complete code coverage, we have to run the Jacoco Maven Agent and the Arquillian Jacoco Exception.
It works fine but when we run both, both try to instrument all classes and the Arquillian tests will throw thousands of exceptions:
Caused by: java.lang.IllegalStateException: Class com/google/common/collect/RegularImmutableAsList is already instrumented.
at org.jacoco.agent.rt.internal_6da5971.core.internal.instr.InstrSupport.assertNotInstrumented(InstrSupport.java:89)
at org.jacoco.agent.rt.internal_6da5971.core.internal.instr.ClassInstrumenter.visitField(ClassInstrumenter.java:55)
at org.jacoco.agent.rt.internal_6da5971.asm.ClassVisitor.visitField(ClassVisitor.java:272)
at org.jacoco.agent.rt.internal_6da5971.asm.ClassReader.readField(ClassReader.java:768)
at org.jacoco.agent.rt.internal_6da5971.asm.ClassReader.accept(ClassReader.java:689)
at org.jacoco.agent.rt.internal_6da5971.asm.ClassReader.accept(ClassReader.java:506)
at org.jacoco.agent.rt.internal_6da5971.core.instr.Instrumenter.instrument(Instrumenter.java:84)
at org.jacoco.agent.rt.internal_6da5971.core.instr.Instrumenter.instrument(Instrumenter.java:108)
My question is, is there any way to include both the maven listener and the Arquillian Jacoco Exension? Can we configure the Arquillian extension so it doesn't try to re-instrument the classes, or at least doesn't print the stack trace?
I am anticipating someone will say to not mix Arquillian and Mockito, but if that is the only answer, my teams are going to vote to ditch Arquillian.
You can see a small project that is configured with both types of tests included here:
GitHub - teacurran/alwaysawake-server
And to see an example of the stack traces thrown during build, you can see that here:
teacurran-alwaysawake-server.master.ci #8 Console
update: it seems that the arquillian-jacoco isn't needed when running the container in embedded mode, this is where I am seeing the exception.
Seems that Arquillian JaCoCo Extension instruments classes that were already instrumented by the JaCoCo agent, or vise versa.
I don't know for what Arquillian JaCoCo Extension is used and why it needs to perform instrumentation instead of reliance on instrumentation performed by JaCoCo agent, but you can configure jacoco-maven-plugin goal "prepare-agent" that manages configuration of JaCoCo agent to exclude classes.
BTW as JaCoCo developer, I'd be interested to hear more about Arquillian JaCoCo Extension. Unfortunately there is not much information on page https://github.com/arquillian/arquillian-extension-jacoco
Related
Can someone tell me please: how to take a screenshot when test method fails (jUnit 5). I have a base test class with BeforeEach and AfterEach methods. Any other classes with #Test methods extends base class.
Well, it is possible to write java code that takes screenshots, see here for example.
But I am very much wondering about the real problem you are trying to solve this way. I am not sure if you figured that yet, but the main intention of JUnit is to provide you a framework that runs your tests in various environments.
Of course it is nice that you can run JUnit within your IDE, and maybe you would find it helpful to get a screenshot. But: "normally" unit tests also run during nightly builds and such - in environments where "taking a screenshot" might not make any sense!
Beyond that: screenshorts are an extremely ineffective way of collecting information! When you have a fail, you should be locking for textual log files, html/xml reports, whatever. You want that failing tests generate information that can be easily digested.
So, the real answer here is: step back from what you are doing right now, and re-consider non-screenshot solutions to the problem you actually want to solve!
You don't need to take screen shots for JUnit test failes/passes, rather the recommended way is to generate various reports (Tests Passed/Failed Report, Code coverage Report, Code complexity Report etc..) automatically using the below tools/plugins.
You can use Cobertura maven plugin or Sonarqube code quality tool so that these will automatically generate the reports for you.
You can look here for Cobertura-maven-plugin and here for Sonarqube for more details.
You need to integrate these tools with your CI (Continuous Integration) environments and ensure that if the code is NOT passing certain quality (in terms of tests coverage, code complexity, etc..) then the project build (war/ear) should fail automatically.
Can we use JUnit to test java batch jobs? Since Junit runs locally and java batch jobs run on the server, i am not sure how to start a job (i tried using using the JobOperator class) from JUnit test cases.
If JUnit is not the right tool, how can we unit test java batch code.
I am using using IBM's implementation of JSR 352 running on WAS Liberty
JUnit is first of all an automation and test monitor framework. Meaning: you can use it to drive all kinds of #Test methods.
From an conceptual point, the definition of unit tests is pretty vague; if you follow wikipedia, "everything you do to test something" can be seen as unit test. Following that perspective, of course, you can "unit test" batch code that runs on a batch framework.
But: most people think that "true", "helpful" unit tests do not require the presence of any external thing. Such tests can be run "locally" at build time. No need for servers, file systems, networking, ...
Keeping that in mind, I think there are two things you can work with:
You can use JUnit to drive "integration" or "functional tests". Meaning: you can define test suites that do the "full thing" - define batches, have them processed to check for expected results in the end. As said, that would be integration tests that make sure the end-to-end flow works as expected.
You look into"normal" JUnit unit-testing. Meaning: you focus on those aspects in your code that are "un-related" to the batch framework (in other words: look out for POJOs) and unit-test those. Locally; maybe with mocking frameworks; without relying on a real batch service running your code.
Building on the answer from #GhostCat, it seems you're asking how to drive the full job (his bullet 1.) in your tests. (Of course unit testing the reader/processor/writer components individually can also be useful.)
Your basic options are:
Use Arquillian (see here for a link on getting started with Arquillian and Liberty) to run your tests in the server but to let Arquillian handle the tasks of deploying the app to the server and collecting the results.
Write your own servlet harness driving your job through the JobOperator interface. See the answer by #aguibert to this question for a starting point. Note you'll probably want to write your own simple routine polling the JobExecution for one of the "finished" states (COMPLETED, FAILED, or STOPPED) unless your jobs have some other means of making the submitter aware.
Another technique to keep in mind is the startup bean. You can run your jobs simply by starting the server with a startup bean like:
#Startup
#Singleton
public class StartupBean {
JobOperator jobOp = BatchRuntime.getJobOperator();
// Drive job(s) on startup.
jobOp.start(...);
This can be useful if you have a way to check the job results separate from using the JobOperator interface (for which you need to be in the server). Your tests can simply poll and check for the job results. You don't even have to open an HTTP port, and the server startup overhead is only a few seconds.
I am having a issue with generating code coverage report and getting an analysis of unit tests covering the project code. I have used Sonar to get a report on project but the unit test coverage section do not show anything. I have done some research and found that sonar needs to have a junit report generated by emma/clover/corbetura/jacoco. The project is non-maven based. Please tell me how can I analyse code coverage for a non-maven based multi-module project. I am using sonar runner.
Any link or reference will be helpful.
Please do not give links of documentation pages. Apparently, I have browsed them all.
Thanks in advance.
I'm giving the links to the documentation pages to help others who have not seen the documentation or examples. Take the maven property settings and translate them into sonar-runner.properties. For example, depending on how you set up your multi-module project you might put this setting in the parent level or inside each module: sonar.jacoco.reportPath=reports/coverage/jacoco.exec
Here is the link to the multi-module sonar runner project:
http://docs.codehaus.org/display/SONAR/Analyzing+with+SonarQube+Runner
To configure unit test code coverage apply the same properties in maven to the sonar-project.properties file. Here is the example from the Code Coverage example:
https://github.com/SonarSource/sonar-examples/tree/master/projects/code-coverage
Note you need to set the unit and coverage properties as well as the source encoding properties.
sonar.projectKey=org.codehaus.sonar:example-ut-sonarRunner-jacoco-reuseReports
sonar.projectName=UT coverage with SonarQube Runner reusing JUnit and JaCoCo reports
sonar.projectVersion=1.0
sonar.sources=src
sonar.binaries=classes
sonar.language=java
# Tells SonarQube to reuse existing reports for unit tests execution and coverage reports
sonar.dynamicAnalysis=reuseReports
# Tells SonarQube where the unit tests execution reports are
sonar.junit.reportsPath=reports/junit
# Tells SonarQube that the code coverage tool by unit tests is JaCoCo
sonar.java.coveragePlugin=jacoco
# Tells SonarQube where the unit tests code coverage report is
sonar.jacoco.reportPath=reports/coverage/jacoco.exec
# Encoding of the source files
sonar.sourceEncoding=UTF-8
I want to add some hints to my build, to run certain tests "first" without re-running them later.
Simply add Class names to a "priority" string in an input parameter to my test task, or
Have JUnit's testers smart enough to remember/persist failing test class names, so that the next time around the builder runs those first.
What is the most idiomatic way of doing this in Ant?
The following tools might help you to achieve the desired JUnit test execution order, but they depend on Eclipse usage:
Continuous Testing for Eclipse (CT-Eclipse)
JUnit Max
infinitest
I have not used any of those tools, and I have no Ant-only solution.
You might consider these related posts:
Run JUnit automatically when building Eclipse project
Starting unit tests automatically after saving a file
Is there a way to cause hudson to report a build as failed, rather than unstable, if only a single unit test fails? thanks.
Hudson actually enables the ignoring of test failures. It just needs to be put as a property in hudson.
-Dmaven.test.failure.ignore=false
It's actually not a good idea to fail the build if tests failed when using hudson. Problem is hudson will not report the state of test pass/fail if the build fails. If the build fails, hudson deems it to not have completed properly and thus does not act on the result.
There are two properties to the junit task
errorProperty="maven.test.error"
failureProperty="maven.test.failure"
After the junit tag you should be able to do something like this
<fail message="Test failed!!!" if="maven.test.error" />
<fail message="Test failed!!!" if="maven.test.failure" />
But don't nail me on this
If you're using Ant to drive the build, you can configure the JUnit task to halt on failure. Is that what you mean?
Look through your job configuration, I believe there is a property (check box) that says fail on test failure, or something of the sort. We use this on some of our projects at my work.
Otherwise if you want to use the Ant method as suggested maven can run ant tasks...