|
AutomatedTestingWithJunit
A short tutorial about automated testing using T2 + JUnit
Automated Testing with JUnitby Wishnu Prasetya Yes, you can do automated testing with JUnit, but you'll need myT2 utility. T2 is a powerful automated testing utility for Java. It is a free sowftware. Download T2 jar first, and put it somewhere. We will use it as a normal Java library. Let's consider the example below. It's a simple class that provides sorted lists. public class SortedList {
private LinkedList<Comparable> s ; // this will hold the list
private Comparable max ; // direct pointer to the largest element
/**
* A constructor to make an empty list.
*/
public SortedList() { s = new LinkedList<Comparable>() ; }
/**
* Insert x into the list; maintain the list sorted, and relink max
* accordingly.
*/
public void insert(Comparable x) { ... }
/**
* Retrieve the max-element from the list.
*/
public Comparable get() { ... }
}We'll fill in the code later. When testing with T2 you don't write your test cases; T2 will internally generate them for you. You don't write oracles either. Instead, you write specifications. Specifications are more powerful and robust than oracles. And don't worry; we'll write our specifications in Java :) The first specification we want to write is a class invariant. This is a predicate specifying legal states of the instances of our class. In this case we can expect that the list in the variable s will be sorted. This can be formally expressed in a class invariant. However, if you are a bit lazy you may choose to write a weak specification first, to be refined later when you are a bit further in the coding. Ok, so let's write a class invariant. You write it simple as a method called classinv; it takes no argument and returns a boolean. I'll write a weak one: private boolean classinv() { return s.isEmpty() || s.contains(max) ; }This method does not have any functionality. It just express a specification, and will be internally checked by T2 as it later checks your SortedList class. I'll pretend to be a lazy programmer :), so for now I'll write no specification for "insert". However, I'll add some specification to "get". The code for "get" is shown below, along with its specifications. A method can be specified by writing assertions. Note Java assertion is very powerful! You can express complicated things with it. IMPORTANT: assertions that specify pre-conditions of a method should be marked with :"PRE" marker; see below. public Comparable get() {
assert !s.isEmpty() : "PRE" ; // Specifying pre-condition
Comparable x = max ;
s.remove(max) ;
max = s.getLast() ;
assert s.isEmpty() || x.compareTo(s.getLast()) >= 0 ; // Specifying post-condition
return x ;
}Here is the full code of SortedList: public class SortedList {
private LinkedList<Comparable> s ;
private Comparable max ;
public SortedList() { s = new LinkedList<Comparable>() ; }
private boolean classinv() { return s.isEmpty() || s.contains(max) ; }
public void insert(Comparable x) {
int i = 0 ;
for (Comparable y : s) {
if (y.compareTo(x) > 0) break ;
i++ ; }
s.add(i,x) ;
if (max==null || x.compareTo(max) < 0) max = x ;
}
public Comparable get() {
assert !s.isEmpty() : "PRE" ; // Specifying pre-condition
Comparable x = max ;
s.remove(max) ;
max = s.getLast() ;
assert s.isEmpty() || x.compareTo(s.getLast()) >= 0 ; // Specifying post-condition
return x ;
}
}Now let's test the above class. We'll just have to write our usual Junit test; however inside the test we will call T2 to generate the test cases for us. Here is my test: import org.junit.Test;
public class MyTest {
@Test
public void test1() {
// Call T2, pass the full name of the target class to it:
Sequenic.T2.Main.Junit(SortedList.class.getName()) ;
}
}Furthermore you need to:
Then just run the above test as usual with Junit. See it yourself if T2 can find some errors in the above class. That's it!! How does T2 works?T2 does not test the methods of a target class individually. Instead, it test the entire class. It randomly generates (lots of!) sequences of method calls and field updates to the target class; along with the parameters they need. As it goes, it checks the specifications in the target class. T2 itself has LOT's of options, e.g. to make it generate 20.000 test steps divided into sequences of length 10, and to report the first 8 violations (rather that the default 1), we change the above test method to: @Test
public void test1() {
Sequenic.T2.Main.Junit(
SortedList.class.getName()
+ " "
+ "--nmax=20000 --lenexec=10 --violmax=8" // pass T2 options here
) ;
}There are options to do regression (replaying saved tests), generating test sequences in search-mode, etc etc. You'll have to check the manual. Using T2 in IDEWell, you don't need any special support. As explained above, you can use T2 as a library, and call it from inside your Junit tests. So you can just use T2 in any IDE that supports Junit integration like Eclipse or Netbeans. It is not necessary but highly recommended that you also instell a plugin for your IDE that can measures the code coverage of your Junit tests; then you do the same with your T2 tests. E.g. for Eclipse you have the EclEmma plugin that can do this. Combining automated and manual testsYes, of course you can. In the above test example, you can add more test methods; some may be the traditional hand crafted tests. End |
Sign in to add a comment
JUnit is widely used in TDD process. I am wondering how this framework can be used in TDD?
T2 will make your TDD much much stronger. It can inject thousands of auto-generated tests in seconds. You need to change your TDD a bit so that it is more specification oriented. I'd say: give it a try.
TDD is not about auto generated tests.
> "TDD is not about auto generated tests."
... TDD = Test Driven Development, no ??? It's mainly about develoment of course, but I assume the "test"-part means you need to write tests too, and hence the relevance of some aid in automatically generating your tests.
TDD is about "red, green, refactor"! The cycle is: 1. Write a test that fails (minimal code just to compile), 2. Write the code to make test pass, 3. Refactor where required.
The idea in TDD is that you only write code to satisfy the last failed test. Done correctly, the set of unit tests will cover all your code.
My understanding is that T2 is unrelated to TDD. I'm not sure it is unit testing either.
However, as a mechanism for testing invariants it is complementary to your set of unit tests. (There can be some exceptional paths that many TDD practitioners may not always test for.)
So, I think if the statement "T2 will make your TDD much much stronger" had been put as "T2 will make your testing (overall) much much stronger" it would have been more precise.
Just a comment to the bost by tzervos. TDD normally doesn't really cover 100% of the code since the code depends on external libraries with features which may be wrong ... and also things like GUI and logic depending on database state are harder to test than pure data manipulation in algorithms.
I see. Then I agree, automated testing has nothing to do with TDD.
But it seems that TDD uses tests as some sort of specifications; the programmers write an (minimal) implementation that pass them. But this sounds a bit wasteful. Why don't people just write the specifications from the start? One can code specifications in Java.
"Ideally" we do want ppl to "just write the specifications". This is essentially what ppl have been trying to do to close the gap between developers and business ppl. I know a couple of packages attempting to provide such framework.
However, personally, I think it is less likely to happen. So, my thought is more on the developer side. Model checking is probably widely studied to come up with test cases where it would be very difficult for a developer to write those cases. From there (model checking), I had the question of T2 with TDD. For now, I still cannot get my head straight yet.
Hello, I've tried to integrate your tool but it didn't work. I'm using Java 1.5 and Eclipse 3.2. What are the requirements? Thanks
Dear stefan.onea,
I have not tried Eclipse 3.2 yet, but I can't really imagine a reason why it wouldn't work. T2 has no dependency. All you need is in the jar. Can you be a bit more specific: in what way it didn't work?
is there any group or forum where we can discuss issues related to T2.
I have tried T2 test with junit in Eclipse 3.4.1 of course, i did test with the above SortedList? class sample source. but i have got the same error message in the junit failure trace. The following messages is below. ------------------------- Junit Failure Trace -----------------
To ksood23,
we have no forum at the moment. But perhaps It's a good idea to start one. Let me think about it...
To shlee3225,
That's odd. That means what it says: T2 can't find the class. Is it in your class path? Send me your code? (the particular test code you use to call T2 and pass options to it). Email to: wishnu@cs.uu.nl
I did try it on the class Subscription, and has no problem. You won't find any 'bug' tho; but that's because the class contains no specifications/assertions to check against.
Looks like T2 is broken for shorts. Try testing this class:
public class BreakT2? {
}
with this test
import org.junit.Test;
public class BreakT2Test? {
}
and you get:
Sequenic.T2.Msg.T2Error?: ## T2 BUG? ## Fail to invoke method CALL_METHOD, setBroken (CONST null) ON REF 0
Caused by: java.lang.IllegalArgumentException?Yes that's a bug. It's not supposed to generate null for a primitive type. I'll fix this in a day or two....
Should be fixed now.
I have also created a discussion Forum. See http://t2framework.proboards.com/
This is also linked from the main page.