My favorites | Sign in
Project Home Downloads Wiki Issues Source
READ-ONLY: This project has been archived. For more information see this post.
Search
for
  Advanced search   Search tips   Subscriptions
Issue 10: Actual error display in html reports
1 person starred this issue and may be notified of changes. Back to list
Status:  Fixed
Owner:  ----
Closed:  Jun 2011


 
Reported by al...@fastmail.net, May 19, 2011
It would be very useful if the html reports could show on demand the actual error that made a scenario fail. E.g. the error text could be shown by clicking the red "Failed" icon, as Hudson/Jenkins do for the failing tests.

So far the only way to see what went wrong is by reading the logs and the files generated in the target/surfire-reports directory.
May 27, 2011
Project Member #1 lipinski...@gmail.com
Valid comment. Will think how to best solve it.
Status: Accepted
Jun 1, 2011
#2 pascal.b...@gmail.com
I came across a similar need, and I use JUnit MethodRule as a work around

something like this as a MethodRule implemtention

{{{

package tumbler.experimental;

import org.junit.rules.MethodRule;
import org.junit.runners.model.FrameworkMethod;
import org.junit.runners.model.Statement;

public class BddRule implements MethodRule {

  @Override
  public Statement apply(final Statement base, FrameworkMethod method, Object target) {
    return new Statement() {
      public void evaluate() throws Throwable {
        try {
          base.evaluate();
        } catch (Throwable t) {
          unexpectedThen(t.getMessage());
          throw t;
        }
      }
    };
  }

  private void unexpectedThen(String message) {
    tumbler.Tumbler.Then(message);
  };

}


}}}



then my scenario looks like this (here, I'm testing a 'bugged' add function which substracts instead of adding) 


{{{

package tumbler.experimental.sample;

import static tumbler.Tumbler.Given;
import static tumbler.Tumbler.Then;
import static tumbler.Tumbler.When;
import junit.framework.Assert;

import org.junit.Rule;
import org.junit.rules.MethodRule;
import org.junit.runner.RunWith;

import tumbler.Scenario;
import tumbler.Story;
import tumbler.TumblerRunner;
import tumbler.experimental.BddRule;

@RunWith(TumblerRunner.class)
@Story("Basic Operations")
public class SomeScenario {

  @Rule
  public MethodRule tumblerFailureRule = new BddRule();

  @Scenario("Addition validation")
  public void shouldDoSomething() {
    int a = 1;
    int b = 2;
    Given("operands a=" + a + " and b=" + b);

    When("I add a and b");
    int actual = add(a, b);

    int expected = 3;
    Assert.assertEquals(expected, actual);
    Then("expected result is=" + expected);
  }

  static int add(int a, int b) {

    // system under test is bugged : substract instead of add
    return a - b;
  }
}


}}}



as a result, i can see directly, in the *html report*, at the *Then* place, the Exception message raised by Junit *assertEquals*, namely _expected:<3> but was:<-1>_  









Jun 13, 2011
Project Member #3 lipinski...@gmail.com
Ok, this was a nice way of getting that exception, now you have a normal one ;-)
Exception object is stored in the scenario status. So in your report template you can take it by calling scenario.status().details()

Currently I've set it as icon (stop sign) title (when mouse is over it).
Status: Fixed
Jun 14, 2011
#4 pascal.b...@gmail.com
Great, thanks !
Jun 15, 2011
#5 al...@fastmail.net
Great! Is that showing the full stack trace?

Shall we be able to see it in the upcoming 0.3.1 release?
Jun 15, 2011
Project Member #6 lipinski...@gmail.com
This is exactly the Exception object. Scenario.status().details() returns Exception object for failed scenarios.
It's already in 0.3.1-SNAPSHOT in sonatype's snapshot repo. I'll release a full-numbered version still this week with all the stuff inside.
Jun 16, 2011
#7 al...@fastmail.net
It might not be enough. The exception and the stack trace say what has happened, but not where.

When JUnit tests fail, it's necessary to have a look at JUnit output to see where the failure occurred. Not all failures are due to exceptions. They are mostly due to failing assertions.

Is this going to show that as well?
Jun 16, 2011
Project Member #8 lipinski...@gmail.com
When an assertion fails, you get AssertionException. Everything is in the exception.

Powered by Google Project Hosting