Running unit tests with mpirun using ant - junit

Question

I'm trying to run my unit tests through mpirun using ant. I have specified the task as:
<target name="unitTest" depends="buildUnitTest">
<mkdir dir="reports"/>
<junit fork="yes" jvm="mpirun java" printsummary="yes" haltonfailure="yes">
<classpath>
<pathelement location="./bin"/>
<pathelement location="/usr/share/java/junit4.jar"/>
</classpath>
<jvmarg value="-DDIM=3"/>
<jvmarg value="-ea"/>
<formatter type="plain"/>
<batchtest todir="reports">
<fileset dir="test">
<include name="haparanda/utils/*Test.java"/>
<include name="haparanda/iterators/*Test.java"/>
<exclude name="haparanda/iterators/FieldIteratorTest.java"/>
<include name="haparanda/grid/*Test.java"/>
</fileset>
</batchtest>
</junit>
</target>
Running eg:
mpirun java -ea -DDIM=3 -cp ./bin:/usr/share/java/junit4.jar org.junit.runner.JUnitCore haparanda.grid.ComputationalComposedBlockTest
from command line works fine. However, when I run:
ant unitTest
I get the following error:
BUILD FAILED
.../build.xml:28: Process fork failed.
Running ant with the verbose flag I get told that I got an IOException with the error message:
Cannot run program "mpirun java": error=2, No such file or directory
This is the case also when I specify the full path to mpirun and Java:
<junit fork="yes" jvm="/home/malin/bin/openmpi/bin/mpirun /usr/bin/java" printsummary="yes" haltonfailure="yes">
gives me:
.../build.xml:28: Process fork failed.
at ...
Caused by: java.io.IOException: Cannot run program "/home/malin/bin/openmpi/bin/mpirun /usr/bin/java": error=2, No such file or directory
How can i make this work?

Answer 1

This question is quite old and seems to have been successfully addressed in the comments by Gilles Gouaillardet. In the academic setting I am working in, I also attempted to use Junit with Java and MPI. I was unable to successfully use the trick proposed by Gilles Gouaillardet and I ended up with a quite different solution.
Custom Junit4 runner - general idea
An other way of running Junit tests with MPI consists in implementing a custom Junit runner.
In this custom Junit runner, instead of calling the Test methods "directly", you can launch your custom command using a ProcessLauncher. In my implementation, I made every MPI process use the normal Junit4 runtime to run the test methods. However, instead of using the normal RunNotifier of the Junit runtime, the MPI processes use my custom RunNotifier which writes the calls it receives to a file. A file with the calls of
Back in my custom runner, once the mpirun processes have finished, I aggregate the results of each test method of each MPI process and transmit those to the normal RunNotifier.
Benefits
With this system, you are staying within the "normal" Junit4 framework. In my case I was trying to run Junit tests from Maven. I can also integrate the test results with the Eclipse Junit view successfully (this required a few tricks that are not shown in the code excerpt below).
Here is a capture of my Eclipse environment after running the tests (the class names are slightly different than those presented in the excerpts below due to some additional complications for my particular environment).
Some selected code details
Only the most important parts of the custom MpiRunner and MpiTestLauncher are shown. Imports, try/catch structures and numerous details have been removed. I will eventually make the whole code available on GitHub but it isn't quite ready yet.
/** A test class using the custom "MpiRunner" */
#RunWith(MpiRunner.class)
public class TestUsingMpi {
#Test
public void test() {
assertTrue("Should run with multiple processes", MPI.COMM_WORLD.Size() > 1);
}
}
/** Curstom Junit4 Runner */
public class MpiRunner extends Runner {
// some methods skipped, try/catch blocks have been removed
#Override
public void run(RunNotifier notifier) {
// Build the command
final ArrayList<String> command = new ArrayList<>();
command.add("mpirun");
command.add("-np");
command.add(String.valueOf(processCount));
command.add("java");
// Classpath, UserDirectory, JavaLibraryPath ...
command.add("MpiTestLauncher "); // Class with main
command.add(testClass.getCanonicalName()); // Class under test as argument
ProcessBuilder pb = new ProcessBuilder(command);
File mpirunOutFile = new File("MpirunCommandOutput.txt");
pb.redirectOutput(Redirect.appendTo(mpirunOutFile));
pb.redirectError(Redirect.appendTo(mpirunOutFile));
Process p = pb.start(); // Launch the mpirun command
p.waitFor(); // Wait for termination
// Parse the notifications of each MPI process
for (int i = o; i < NbProcesses; i++) {
List<Notification> mpiRankNotifs = parse(i);
//Re-run those notifications on the parameter "notifier" of this method
for (Notification n : notifications) {
//Reconstitute the method call made in the mpi process
Class<?> paramClass = n.parameters[0].getClass();
Method m = RunNotifier.class.getDeclaredMethod(n.method, paramClass);
m.invoke(notifier, n.parameters);
}
}
}
}
/** Main class of the Mpirun java processes */
public class MpiTestLauncher {
public static void main(String[] args) throws Exception {
MPI.Init(args);
commRank = MPI.COMM_WORLD.Rank();
commSize = MPI.COMM_WORLD.Size();
Class<?> testClass = Class.forName(args[0]); // Class that contains the tests
String notificationFileName = testClass.getCanonicalName() + "_" +
commRank;
File f = new File(notificationFileName);
CustomNotifier notifier = new MpiApgasRunNotifier(f);
BlockJUnit4ClassRunner junitDefaultRunner = new BlockJUnit4ClassRunner(testClass);
junitDefaultRunner.run(notifier);
notifier.close(); //Flushes the underlying buffer
MPI.Finalize();
}
}