Is there a way to cause hudson to report a build as failed, rather than unstable, if only a single unit test fails? thanks.
Hudson actually enables the ignoring of test failures. It just needs to be put as a property in hudson.
-Dmaven.test.failure.ignore=false
It's actually not a good idea to fail the build if tests failed when using hudson. Problem is hudson will not report the state of test pass/fail if the build fails. If the build fails, hudson deems it to not have completed properly and thus does not act on the result.
There are two properties to the junit task
errorProperty="maven.test.error"
failureProperty="maven.test.failure"
After the junit tag you should be able to do something like this
<fail message="Test failed!!!" if="maven.test.error" />
<fail message="Test failed!!!" if="maven.test.failure" />
But don't nail me on this
If you're using Ant to drive the build, you can configure the JUnit task to halt on failure. Is that what you mean?
Look through your job configuration, I believe there is a property (check box) that says fail on test failure, or something of the sort. We use this on some of our projects at my work.
Otherwise if you want to use the Ant method as suggested maven can run ant tasks...
Related
I'm new to Junit. I've created a test suite with 50 test cases. If I run it passes only 30 test cases and 20 test cases are failing. How I can achieve running only those 20 failed test cases again with the help of Junit? Is it possible? Can someone guide on this?
Deciding which tests to run, and in which order, is the job of the test runner. So, to rerun only failed tests you need to use a test runner that records which tests passed or failed, and which can be instructed to run only tests it knows failed.
The tested runner in the Eclipse IDE can do this. Typically, and IDE is the only place you need this functionality, because an IDE is interactive and when using it you want fast feedback on whether a fix works. This functionality is not so useful elsewhere because in other contexts we typically want to check that the program passes all its tests.
Is there any easy way to configure log4j so that in unit tests I only print the stacktrace of the failing tests without having noise in the output on the shell?
Right now when I execute tests I can see the output printed by the classes I am testing. I want to avoid that since it's not bringing any value for passing tests
I would recommend to have log4j.properties file under src/test/resources with content:
log4j.rootLogger=OFF
As result all your production classes will not log but the jUnit or testNG will print the failed tests as you expect.
In your unit test, set the log level to the level at which stacktraces are being logged:
Logger.getRootLogger().setLevel(Level.WARNING);
I want to add some hints to my build, to run certain tests "first" without re-running them later.
Simply add Class names to a "priority" string in an input parameter to my test task, or
Have JUnit's testers smart enough to remember/persist failing test class names, so that the next time around the builder runs those first.
What is the most idiomatic way of doing this in Ant?
The following tools might help you to achieve the desired JUnit test execution order, but they depend on Eclipse usage:
Continuous Testing for Eclipse (CT-Eclipse)
JUnit Max
infinitest
I have not used any of those tools, and I have no Ant-only solution.
You might consider these related posts:
Run JUnit automatically when building Eclipse project
Starting unit tests automatically after saving a file
Hudson-CI shows a build as broken despite if the build was successful and only the unit tests are failing.
Could I configure it to show the red circle only on really broken builds, and show another color (let's say yellow) when the build is successful and only other condition is broken?
Hudson will report the build as failing if the exit/error code on the last step is non-zero. Our testing tool (NUnit) returns a zero only if all the tests passed, and I suspect your tests are doing similar.
To get round this, configure Hudson to have two build steps per job. The first step builds the code. If this step fails, the build fails and appears as a RED circle.
The second step runs the tests - but is set to always return a zero exit code. e.g. I have a Execute Windows batch command step with the following command:
NUnit-console.exe /options as required...
exit 0
The exit 0 forces Hudson to think that the tests have run OK. Thus if both steps are run, the job will be labelled as sucessful.
Finally, if you use the "Publish NUnit test result report" option, Hudson can inspect the test results and make the build either unstable (YELLOW) if some tests failed or successful (GREEN) if all passed.
(There are other options for JUNit, MSTest etc. but I've only got experience with NUnit and Hudson)
failonerrors="on" on task tag. Isn't it you're looking for ? Apache Ant - task
I am new at this and I was wondering how I can setup that I save the artifacts, only if less than 90% of the tests have passed.
Any idea how I can do this?
thanks
This is not currently possible with Hudson. What is the motivation to avoid archiving artifacts on every build?
How about a rather simple workaround. You create a post build step (or additional build step) that calls your tests from the command line. Be sure to capture all errors so Hudson don't count it as a failure. Than you evaluate your condition and set the error level accordingly. In addition you need to save reports (probably outside hudson) before you set the error level, so they are available even or only when the build fails.
My assumption here is, that it is OK, not to run the tests when building the app fails. However, you can separate the building and testing in two jobs. See here.