Note: we are now refactoring the Randoop and FailureDoc code base, trying to integrate these two project into a unified one. Thus, the source code is under change, please stay tuned until a new version comes out.

For the impatient: go to the User Manual page now.

What is FailureDoc?

FailureDoc is a fully-automated tool to infer explanatory documentation (in the form of code comment) for a failed unit test. The inferred documentation intends to be succinct and able to reveal important facts to explain why a test fails.

Why would I want to use it?

Although many automated test generation tools have been studied in the last three decades, few tools have been develop to explain the failure cause of a test.

A unit test must be effective in finding potential bugs, but also must be easy-to-interpret. This is a particular problem for automatically-generated tests which are often long and have poor readability, but it is also relevant for human-written tests.

If you find it is non-trivial to understand why a test fails, you may want to try our tool. Our tool infers explanatory documentation for a failed test as debugging clues. The inferred documentation often reveal potentially useful facts about the failure, helping programmers understand the failure cause and fix the bug.

A Quick Example

Here is a quick example showing which kind of documentation FailureDoc can infer. Following is an automatically-generated test. For most people, it is not easy to identify the failure case by simply looking at the test code. Executing the test still does not give much information: the assertion fails without dumping any useful information.

The comment part (after //...) is inferred by FailureDoc, clearly indicating why the test fails (the constructor of TreeSet should not accept a list of non-comparable objects).

``` public void testJDKTreeSet() { int i = 1; ArrayList l = new ArrayList(i); //Test passes if o implements Comparable Object o = new Object(); //Test passes if o is not added to l boolean b = l.add(o); TreeSet t = new TreeSet(l); Set s = Collections.synchronizedSet(t);

assertTrue(s.equals(s)); } ```

Documentation and Publication

User Manual

Examples

A paper describing FailureDoc in detail is accepted by ASE'2011. Here is a suggested bibtex:

@inproceedings{ZhangZE2011, author = {Sai Zhang and Cheng Zhang and Michael D. Ernst}, title = {Automated documentation inference to explain failed tests}, booktitle = {ASE 2011: Proceedings of the 26th Annual International Conference on Automated Software Engineering}, address = {Lawrence, KS, USA}, month = nov, year = {2011} }