| Issue 10: | Actual error display in html reports | |
| 1 person starred this issue and may be notified of changes. | Back to list |
It would be very useful if the html reports could show on demand the actual error that made a scenario fail. E.g. the error text could be shown by clicking the red "Failed" icon, as Hudson/Jenkins do for the failing tests. So far the only way to see what went wrong is by reading the logs and the files generated in the target/surfire-reports directory.
May 27, 2011
Project Member
#1
lipinski...@gmail.com
Status:
Accepted
Jun 1, 2011
I came across a similar need, and I use JUnit MethodRule as a work around
something like this as a MethodRule implemtention
{{{
package tumbler.experimental;
import org.junit.rules.MethodRule;
import org.junit.runners.model.FrameworkMethod;
import org.junit.runners.model.Statement;
public class BddRule implements MethodRule {
@Override
public Statement apply(final Statement base, FrameworkMethod method, Object target) {
return new Statement() {
public void evaluate() throws Throwable {
try {
base.evaluate();
} catch (Throwable t) {
unexpectedThen(t.getMessage());
throw t;
}
}
};
}
private void unexpectedThen(String message) {
tumbler.Tumbler.Then(message);
};
}
}}}
then my scenario looks like this (here, I'm testing a 'bugged' add function which substracts instead of adding)
{{{
package tumbler.experimental.sample;
import static tumbler.Tumbler.Given;
import static tumbler.Tumbler.Then;
import static tumbler.Tumbler.When;
import junit.framework.Assert;
import org.junit.Rule;
import org.junit.rules.MethodRule;
import org.junit.runner.RunWith;
import tumbler.Scenario;
import tumbler.Story;
import tumbler.TumblerRunner;
import tumbler.experimental.BddRule;
@RunWith(TumblerRunner.class)
@Story("Basic Operations")
public class SomeScenario {
@Rule
public MethodRule tumblerFailureRule = new BddRule();
@Scenario("Addition validation")
public void shouldDoSomething() {
int a = 1;
int b = 2;
Given("operands a=" + a + " and b=" + b);
When("I add a and b");
int actual = add(a, b);
int expected = 3;
Assert.assertEquals(expected, actual);
Then("expected result is=" + expected);
}
static int add(int a, int b) {
// system under test is bugged : substract instead of add
return a - b;
}
}
}}}
as a result, i can see directly, in the *html report*, at the *Then* place, the Exception message raised by Junit *assertEquals*, namely _expected:<3> but was:<-1>_
Jun 13, 2011
Ok, this was a nice way of getting that exception, now you have a normal one ;-) Exception object is stored in the scenario status. So in your report template you can take it by calling scenario.status().details() Currently I've set it as icon (stop sign) title (when mouse is over it).
Status:
Fixed
Jun 14, 2011
Great, thanks !
Jun 15, 2011
Great! Is that showing the full stack trace? Shall we be able to see it in the upcoming 0.3.1 release?
Jun 15, 2011
This is exactly the Exception object. Scenario.status().details() returns Exception object for failed scenarios. It's already in 0.3.1-SNAPSHOT in sonatype's snapshot repo. I'll release a full-numbered version still this week with all the stuff inside.
Jun 16, 2011
It might not be enough. The exception and the stack trace say what has happened, but not where. When JUnit tests fail, it's necessary to have a look at JUnit output to see where the failure occurred. Not all failures are due to exceptions. They are mostly due to failing assertions. Is this going to show that as well?
Jun 16, 2011
When an assertion fails, you get AssertionException. Everything is in the exception. |