Why are there no test cases to verify expected behavior of the various collections classes or is that part of the codebase that doesnt get synced during the integration with google's internal repo?

When Kevin and Jared open sourced the collections, it required much effort to clean up the code and to decouple it from from Google's other libraries. The effort to do likewise for the tests is underway.

When it does eventually come out, you'll find the test coverage to be very cool. There's testers that can test an arbitrary implementation of say, Set, plus customizations of those testers for special cases (ie. ImmutableSet). It makes the test infrastructure very reusable - if we write a new implementation of Set, we get a lot of test coverage for free.

Plus, we can also check subtle consistency issues with Sun's Java APIs. One example - what happens if you call remove(null) on Collections.emptyList()? Does it fail with an UnsupportedOperationException? Or does it return 'false' to indicate that no change was made. Either answer could be correct, depending on perspective. The collections tests cover several subtle details like this so we can keep track of the subtlest of behaviour changes.

Tests are there now. They are not passing cleanly yet, so I'll leave this open.

I did a fresh checkout, and then "ant test" which has a few problems. First, test needs to depend on testfw. After testfw has run successfully, test still fails with this message:

$ ant test Buildfile: build.xml

compile:

test.compile:

test: [junit] Usage: TestRunner [-wait] testCaseName, where name is the name of the TestCase class [junit] Picked up JAVA_TOOL_OPTIONS:
[junit] Picked up _JAVA_OPTIONS:

BUILD FAILED c:\users\projects\google-collections\build.xml:65: Java returned: 2

I've added the missing Ant dependency pointed out in comment 4 (thanks), included the required cglib dependency for EasyMock classextension, and introduced a test-suite class. The tests now run, but some do not pass:

1) NullPointerTester failures. I'm not sure what the problem here is. 2) Ordering.reverse().reverse(). The code and tests are not in sync. This should be fixed in the next snapshot.

I should also verify that all tests are being run.

The problem with the NullPointerTester tests was that jsr305.jar was not in the runtime dependencies of the tests. I should have realized this earlier, but:

1) "foo instanceof Nullable" silently returns false if the class Nullable is unknown. This makes a little sense: If Nullable hasn't been loaded, it's impossible for an existing object to be an instance. But it's not the fail-fast behavior I'd subconsciously assumed.

2) Not all NullPointerTester tests are failing. JoinerTest, for example, passes. The tester examines Joiner.join(@Nullable Object, @Nullable Object, Object...) and doesn't see the @Nullable annotations. But in fact Joiner.on(","),join(null, null, new Object[0]) does throw NPE, so the test passes. This is because the @Nullable annotations indicate that, while the main Joiner implementation does not accept null, subclasses may do so. I think there may be more advanced annotations to indicate this, but annotations or not, it's admittedly a bit confusing.

In any case, this is now fixed.

The only remaining issue is OrderingTest. The public version of Ordering no longer ensures that comparator.reverse().reverse().equals(comparator), but the corresponding test change didn't come over along with it. (Luckily, problems like this will be easier to avoid in the future when we have a runnable test suite :))

Assigning to Kevin for him to close when he merges in the next round of OrderingTest and other changes.

Test suite has been passing cleanly for a while.