From jmmv@google.com on December 09, 2010 04:48:18

Specifying the test case timeouts is fragile: a test case may run faster or slower depending on the platform, so hardcoding an upper bound per test case is generally a bad idea.

The way we should be doing this is by specifying the test case size declaratively: i.e. allow a test case to define itself as "small", "large" or whatever. Then, the runtime engine can apply different deadlines for every test case size (e.g. 1 minute for small tests, 1 hour for large tests, etc.). We need to have a hard deadline for all tests even if they are set to be of the largest possible type.

Once we have this, the deadlines per size can be either automatically tuned depending on the speed of the running platform and/or we can allow the end user to specify them as part of the configuration of Kyua.

Original issue: http://code.google.com/p/kyua/issues/detail?id=5