Logs in Junit test methods printed after all tests are executed [duplicate] - junit

This question already has an answer here:
Maven parallel test output
(1 answer)
Closed 5 years ago.
I am using Java util logging (JUL) in a Junit test class with JUL working in default config (i.e. printing to console).
Logs statements from methods annotated with #BeforeClass and #AfterClass are getting printed when they are executed but logs in '#Test' methods are printed only after all test execution is finished.
Not sure what exactly is wrong because the same were working earlier.
Logger instantiation:
private static Logger logger = Logger.getLogger(MainIntegrationTest.class.getName());
Logger use:
logger.info("start test");
The test cases are run using maven. I noticed that this started happening only after I started running the test classes in parallel using this surefire configuration:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.20</version>
<configuration>
<parallel>classes</parallel>
<useUnlimitedThreads>true</useUnlimitedThreads>
</configuration>
</plugin>
The logs are printed at test execution time if I run the test classes in serial fashion.

The test cases are run using maven. I noticed that this started happening only after I started running the test classes in parallel using this surefire configuration
The output would become scrambled if all tests were writing to console concurrently. For concurrent execution the results have to be buffered for the whole run and then written to the console as one chunk. The only safe time to write to the console is when the test is complete.
Even if the buffering was performed line by line this could lead to interleaved messages in the console which could be hard to read or even misdirect you as you don't know the source of the line.

Related

TestCafe tests pass but job randomly fails [error]Process completed with exit code 1

The scenario is very similar to this one https://github.com/DevExpress/testcafe/issues/6804
Scenario:
I'm running a set of tests using testcafe, in a folder, with concurrency 4:
UI clicks
UI navigation
File upload scenario
Assertions
The job runs in github actions, with self-runners (docker container with no limits set) in AWS, using a properly sized m5.2xlarge box.
All goes well, tests pass, but randomly the job fails with an elusive error:
2022-06-11T08:10:53.9599698Z 30 passed (4m 34s)
2022-06-11T08:10:54.2336015Z Error:
2022-06-11T08:10:56.2617307Z ##[error]Process completed with exit code 1.
testcafe version 1.17.1
node version 14.17.18
I've tested changing/resizing the EC2 runner but didn't help. There are no errors on infra side, just the exit code 1 w/ no other details.
Is anyone else having this issue or have any advice on troubleshooting it?

How to resolve below error-ERROR: Child Process Error: Test runner sel (JUnitTester) has failed with retcode 1

When I ran the selenium script using Taurus and want to convert in JMX then got the error-
10:55:54 ERROR: Child Process Error: Test runner sel (JUnitTester) has failed with retcode 1
10:55:54 ERROR: JUnitTester STDERR:
May 02, 2022 10:55:50 AM com.blazemeter.taurus.junit.CustomRunner main
INFO: Starting: [C:\Users\sss\2022-05-02_10-55-36.435651\runner.properties]
Exception in thread "main" com.blazemeter.taurus.junit.exception.CustomRunnerException: Nothing to test
at com.blazemeter.taurus.junit.CustomRunner.main(CustomRunner.java:54)
How it resolved?
It means that Taurus JUnit Executor failed somewhere somehow, inspect logs in the Artifacts Directory and look for Java/JUnit-related errors there.
Most probably JUnit executor wasn't able to detect any test cases to run, it's impossible to tell for sure what is the root cause without seeing the full logs and your automation code.
Be aware that you can use another approach not involving Taurus, i.e.
Kick off BlazeMeter Proxy Recorder
Configure your tests to use the above recorder as the proxy
Run your tests from IDE or console
The BlazeMeter proxy recorder will capture the requests and convert them into .jmx file (you can also export it as "SmartJMX" with automatic detection and correlation of the dynamic parameters)

Unable to run individual Junit Test class in a Webapplication

My project is a webapplication, currently I have deployed it my local machine. I have some testcases written in Mockito, Junit.
When I run the Junit test on the whole project, by right clicking on project , all these test cases run successfully but when I pick up an individual test case and run it I get the below error and test case wont run.
Error occurred during initialization of VM
java/lang/NoClassDefFoundError: java/lang/Object
I have all the required jars of mockito and junit.

JUnit are failing when upgraded from JDK 1.6 to JDK 1.8 in maven, but its works in eclipse

I am using the JUnit for code coverage in my project. For db i am using the dbunit as like mock DB. When i am running JUnit from Eclipse UI its getting passed, but its getting failed when run through maven.
Above set up is running fine in JDK 1.6.25 by using maven and its started failing when upgraded to 1.8.51. I had updated the maven compiler plugin, its doesn't work. I am used below versions, junit - 4.7 2.dbunit - 2.4.8 hsqldb - 2.0.0 maven - 2.2.1.
Issue:
-> All test cases which ran fine in JAVA 1.6, started failing on migrating to JDK 1.8.51.
-> Due this we faced build failure issue and also code coverage reduction.
Root Cause:
-> JUnit uses Java reflection to get the test methods from Test classes. In JAVA 1.6 test method order returned as same as declaration in source file.
-> But from JAVA 7 onwards the methods order returned the by JVM is not same as the source file, it will be returned randomly.
-> Since our test cases are dependent on each other, due to order change it started failing.
For Example below test cases are using the same data (Mock DB) for execution,
-> AddOperationTestCase()
-> EditOperationTestCase()
-> DeleteOperationTestCase()
If delete run first due JVM random order, for Add and Edit data won't be available it will fail.
Solution :
-> I had tried to find options in JUnit and Sure Fire plugin to maintain same order as like source file, but I could not find feasibility there.
-> I have identified the class which will returns the order of execution in JUnit library and override that accordingly to run it source file order.
-> As of now I had added this annotation wrapper to failed classes, now build is running successfully.
Link for Wrapper class:
https://somethingididnotknow.wordpress.com/2014/03/07/run-junit-tests-in-order/

Junit test cases running some test cases parallel and some sequentially [duplicate]

I'm using JUnit 4.4 and Maven and I have a large number of long-running integration tests.
When it comes to parallelizing test suites there are a few solutions that allow me to run each test method in a single test-class in parallel. But all of these require that I change the tests in one way or another.
I really think it would be a much cleaner solution to run X different test classes in X threads in parallel. I have hundreds of tests so I don't really care about threading individual test-classes.
Is there any way to do this?
Use maven plugin:
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.7.1</version>
<configuration>
<parallel>classes</parallel>
<threadCount>5</threadCount>
</configuration>
</plugin>
</plugins>
</build>
From junit 4.7 it's now possible to run tests in parallel without using TestNG. Actually it has been possible since 4.6, but there are a number of fixes being made in 4.7 that will make it a viable option. You may also run parallel tests with spring, which you can read about here
Inspired by JUnit's experimental ParallelComputer runner I've built my own ParallelSuite and ParallelParameterized runners. Using these runners one can easily parallelize test suites and parameterized tests.
ParallelSuite.java
public class ParallelSuite extends Suite {
public ParallelSuite(Class<?> klass, RunnerBuilder builder) throws InitializationError {
super(klass, builder);
setScheduler(new RunnerScheduler() {
private final ExecutorService service = Executors.newFixedThreadPool(4);
public void schedule(Runnable childStatement) {
service.submit(childStatement);
}
public void finished() {
try {
service.shutdown();
service.awaitTermination(Long.MAX_VALUE, TimeUnit.NANOSECONDS);
} catch (InterruptedException e) {
e.printStackTrace(System.err);
}
}
});
}
}
ParallelParameterized.java
public class ParallelParameterized extends Parameterized {
public ParallelParameterized(Class<?> arg0) throws Throwable {
super(arg0);
setScheduler(new RunnerScheduler() {
private final ExecutorService service = Executors.newFixedThreadPool(8);
public void schedule(Runnable childStatement) {
service.submit(childStatement);
}
public void finished() {
try {
service.shutdown();
service.awaitTermination(Long.MAX_VALUE, TimeUnit.NANOSECONDS);
} catch (InterruptedException e) {
e.printStackTrace(System.err);
}
}
});
}
}
Usage is simple. Just change #RunWith annotations value to one of these Parallel* classes.
#RunWith(ParallelSuite.class)
#SuiteClasses({ATest.class, BTest.class, CTest.class})
public class ABCSuite {}
tempus-fugit offers something similar, check the docs for details. It relies on JUnit 4.7 and you just mark your test to #RunWith(ConcurrentTestRunner).
Cheers
You can check out the open source library - Test Load Balancer. It does exactly what you ask for - run different test classes in parallel. This integrates at the ant-junit level so that you do not have to change your tests in anyway. I am one of the authors of the library.
Also, think about not running them in threads as you may need a process level sandbox. For example, if you are hitting a DB in your integration tests, you do not want one test to fail because another test added some data in a different thread. Most of the times, tests are not written with this in mind.
Finally, how have solved this problem till now?
You can run the tests in parallel using ParallelComputer provided by Junit itself. Here's a small snippet to get you started.
Class[] cls = { TestCase1.class, TestCase2.class };
Result result = JUnitCore.runClasses(ParallelComputer.classes(), cls);
List<Failure> failures = result.getFailures();
This will help when you need to run tests from code as it has no dependencies on Maven or any other build management tools.
Please note that, this will run all test cases in parallel, if you have any dependencies between different test cases it might result in false positives. You SHOULD NOT have interdependent tests anyway.
TestNG can do that (this was my first reflex - then I saw you're already having a lot of testcases).
For JUnit, look at parallel-junit.
Another choice: Punner, a new parallel junit runner and maven plugin. You don't have to change your code, copy it to your pom.xml:
<!-- Disable default surefire based testing -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.20</version>
<configuration>
<skip>true</skip>
</configuration>
</plugin>
<plugin>
<groupId>com.github.marks-yag</groupId>
<artifactId>punner-maven-plugin</artifactId>
<version>${version}</version>
<configuration>
</configuration>
<executions>
<execution>
<id>test</id>
<phase>test</phase>
<goals>
<goal>test</goal>
</goals>
</execution>
</executions>
</plugin>
Punner can run test methods in parallel, can keep test outputs separately and clean.
Punner will reduce your mvn console outputs, like this:
[INFO] --- punner-maven-plugin:0.9.13:test (test) # ipc ---
[INFO] Punner report directory: /Users/guile/workspace/ipc/target/punner-reports
[INFO]
[INFO] com.github.yag.ipc.IPCTest.testConnectionHandler.............. PASSED
[INFO] com.github.yag.ipc.IPCTest.testSequence....................... PASSED
[INFO] com.github.yag.ipc.IPCTest.testPartialContent................. PASSED
[INFO] com.github.yag.ipc.IPCTest.testResponseContent................ PASSED
[INFO] com.github.yag.ipc.IPCTest.testPingPong....................... PASSED
[INFO] com.github.yag.ipc.IPCTest.testServerClose.................... PASSED
[INFO] com.github.yag.ipc.IPCTest.testServerSideHeartbeatTimeout..... PASSED
[INFO] com.github.yag.ipc.IPCTest.testClientSideHeartbeatTimeout..... PASSED
[INFO] com.github.yag.ipc.IPCTest.testClientSideHeartbeat............ PASSED
[INFO] com.github.yag.ipc.IPCTest.testClientReconnect................ PASSED
[INFO]
[INFO] Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 10.952 sec, Time saved: 25.919 sec.
Punner produce surefire compatible outputs, you can also get raw log data and a markdown format report from reports directory:
➜ ipc git:(develop) ll target/punner-reports
total 104
-rw-r--r-- 1 guile staff 11K Oct 15 23:07 TEST-com.github.yag.ipc.IPCTest.xml
-rw-r--r-- 1 guile staff 298B Oct 15 23:07 com.github.yag.ipc.IPCTest.txt
drwxr-xr-x 12 guile staff 384B Oct 8 00:50 logs
-rw-r--r-- 1 guile staff 33K Oct 15 23:07 report.md
Punner is my personal project, I written Punner to speed up unit test phase of some other projects such as IPC framework, fine-grained locking, journal service, distributed workflow engine, etc. It saved a lot of my waiting time.
Punner don't support some advanced feature yet. I'm very glad if you could try it and give me some feedback.
You can change your test to be TestNg test in a minute (you just need to change imports), TestNG is the best in parallel testing.
You could try Gridgain that lets you run distribute your tests across a compute grid.