On our project we try to collect all tests results into reportportal.
For unit tests we applied java agent for junit and api tests are executed over gauge framework. Since our project has several tests environments before the application is shipped to production i would like to display the results aggregated per environment. Like local, development, staging, e2e, production and group the results accordingly. Is there such feature available?
Since reportportal v5.1 supports attributes that can be passed over api
rp.attributes = key:value; value;
Hence passing environment:foo.
On these attributes filters can be applied.
Related
I have a problem: I used to use SonarQube version 5.1 where I had JUnit and Integration tests' code coverage set up. Both of these were represented by three items in SonarQube (coverage, it_coverage, overall_coverage - the previous two merged).
Since the 6.2 version, the coverage and it_coverage shouldn't be available anymore, there is supposed to have just an item called coverage which is the value of the item overall_coverage of previous versions (if report paths to jaccoco are set up correctly for jUnit reports and integration tests reports), everything should work well.
My question is, how can I recognize that the integration tests are included in the coverage cause, I don't have any indicator which makes me know that these tests were part of the analysis, cause I still see just the number of unit tests where the Jameleon ones are not listed in.
There is no visual way in SonarQube to be sure that your IT coverage has been taken into account.
You can probably look at the analysis logs to verify that the reports are found (if not, you will get a warning message).
If you want to be sure, you can also run an analysis without specifying the IT report and see if there's a difference or not.
Can we use JUnit to test java batch jobs? Since Junit runs locally and java batch jobs run on the server, i am not sure how to start a job (i tried using using the JobOperator class) from JUnit test cases.
If JUnit is not the right tool, how can we unit test java batch code.
I am using using IBM's implementation of JSR 352 running on WAS Liberty
JUnit is first of all an automation and test monitor framework. Meaning: you can use it to drive all kinds of #Test methods.
From an conceptual point, the definition of unit tests is pretty vague; if you follow wikipedia, "everything you do to test something" can be seen as unit test. Following that perspective, of course, you can "unit test" batch code that runs on a batch framework.
But: most people think that "true", "helpful" unit tests do not require the presence of any external thing. Such tests can be run "locally" at build time. No need for servers, file systems, networking, ...
Keeping that in mind, I think there are two things you can work with:
You can use JUnit to drive "integration" or "functional tests". Meaning: you can define test suites that do the "full thing" - define batches, have them processed to check for expected results in the end. As said, that would be integration tests that make sure the end-to-end flow works as expected.
You look into"normal" JUnit unit-testing. Meaning: you focus on those aspects in your code that are "un-related" to the batch framework (in other words: look out for POJOs) and unit-test those. Locally; maybe with mocking frameworks; without relying on a real batch service running your code.
Building on the answer from #GhostCat, it seems you're asking how to drive the full job (his bullet 1.) in your tests. (Of course unit testing the reader/processor/writer components individually can also be useful.)
Your basic options are:
Use Arquillian (see here for a link on getting started with Arquillian and Liberty) to run your tests in the server but to let Arquillian handle the tasks of deploying the app to the server and collecting the results.
Write your own servlet harness driving your job through the JobOperator interface. See the answer by #aguibert to this question for a starting point. Note you'll probably want to write your own simple routine polling the JobExecution for one of the "finished" states (COMPLETED, FAILED, or STOPPED) unless your jobs have some other means of making the submitter aware.
Another technique to keep in mind is the startup bean. You can run your jobs simply by starting the server with a startup bean like:
#Startup
#Singleton
public class StartupBean {
JobOperator jobOp = BatchRuntime.getJobOperator();
// Drive job(s) on startup.
jobOp.start(...);
This can be useful if you have a way to check the job results separate from using the JobOperator interface (for which you need to be in the server). Your tests can simply poll and check for the job results. You don't even have to open an HTTP port, and the server startup overhead is only a few seconds.
currently working with a very large application to test (several custom programs, running in a distributed environment), and has built up a very large set of automated test cases for regression and feature testing. These tests are large and there are a lot, so full test runs are dispatched across many machines, the results gathered, and then imported into a custom web app.
technologies: java/selenium/ant/testng/jenkins
reports: testng,reportng,xslt
how to store results in database(eg: mysql)?
Create a custom Reporter Listener by extending the org.testng.TestListenerAdapter and override the onTestSuccess, onTestFailure and onTestSkipped methods and log the results of the tests there to mySQL. After that you have to add your custom Reporter as a Listener.
You can find on the TestNG's website, how can you define a custom Listener:
http://testng.org/doc/documentation-main.html#testng-listeners
And here you can find how can you override the TestListenerAdapter:
http://testng.org/doc/documentation-main.html#logging-listeners
In the Cloudbees wiki, this page explains how to add configuration parameter for an app deployment, using cloudbees-web.xml.
But, is the content of:
<appid>APP_ID</appid>
injected as a well ? How can I retrieve this value from my application's code ?
My preference is to avoid coding an application to contain explicit references to the container within which it runs. So I would favour using techniques that do not tie your code to CloudBees (a.k.a. us).
Thus I would use a container specific descriptor file that configures a context parameter, then your application just reads the context parameter and uses that parameter directly.
There are two techniques for doing this:
Application Environments personally I love this way... though if you want to deploy the application to your own test environment that you have just spun up yourself, your cloudbees-web.xml will likely be missing the required environment definition... so better is to use the newer
Configuration Parameters so that when you need your own test instance you just define the configuration parameters for that test environment and then deploy the exact same artifact to that instance... it also prevents the issue of deploying to the test instance with the production environment turned on.
I am sure one of the RUN# team may well have some other trick such as a System property that tells you the app id... but keep in mind that when running locally, e.g. using a local jetty/tomcat/bees:run container your code will then blow up!
I want to add some hints to my build, to run certain tests "first" without re-running them later.
Simply add Class names to a "priority" string in an input parameter to my test task, or
Have JUnit's testers smart enough to remember/persist failing test class names, so that the next time around the builder runs those first.
What is the most idiomatic way of doing this in Ant?
The following tools might help you to achieve the desired JUnit test execution order, but they depend on Eclipse usage:
Continuous Testing for Eclipse (CT-Eclipse)
JUnit Max
infinitest
I have not used any of those tools, and I have no Ant-only solution.
You might consider these related posts:
Run JUnit automatically when building Eclipse project
Starting unit tests automatically after saving a file