The Theory Behind Testing

At some point your code is going to break and the scary thing is it might not even be your fault. Upgrades, feature enhancements, bug fixes are all part of the development lifecycle and with deadlines and screaming managers you don't have the time to test the functionality of your entire application with every change you make.

The problem is that today's fix could be tomorrow's bug. What if there was an automated way of seeing if that change you're making is going to break something? That's where writing tests for your application can be invaluable.

In the past writing test against your application meant downloading, configuring and learning a completely separate framework. Often times this caused more headaches then it was worth and was the reason why most developers didn't bother writing tests. With Wheels we've included a customized version of the RocketUnit Testing Framework to help address just this issue.

With Wheels, writing tests for your application is part of the development lifecycle itself and running the test is as simple as clicking a link.

Touring the Testing Framework

Like everything else in Wheels, the testing framework is very simple yet powerful. You're not going to having to remember a hundred different commands and methods since Wheels' testing framework has only three commands and four methods (you can't get any simpler than that). Let's take a tour and go over the conventions, commands and methods that make up the different parts of the Testing Framework:

Conventions

All testing code MUST reside in the tests directory off the root of your Wheels application or within a subdirectory thereof. If you do not have a tests directory, simply create one. When you run the tests for your application Wheels recursively scans your application's tests directory and looks for any available valid tests to run, so feel free to organize the subdirectories and place whatever files needed to run your tests under the tests directory any way you like.

Any components that will contain tests MUST extend the wheels.test component:

<cfcomponent extends="wheels.test">

If the testing engine sees that a component DOES NOT extend wheels.test, that component skipped. This let's you create and store any mock components you might want to use in your tests directory and keep everything together.

Any methods that will be used as tests must begin their name with test:

<cffunction name="test_a_equals_b">

If a method DOES NOT begin with test it is NOT considered a test and is skipped. This let's you create as many helper methods for your testing components as you want.

DO NOT VAR ANY VARIABLES used in your tests. In order for the testing framework to get access to the variables within the tests that you're writing, all variables need to be within the components variables scope. The easy way to do this is to just not "var" variables within your tests and ColdFusion will automatically assign these variables into the variables scope of the component for you. The only gotcha is that this could cause some variables to overwrite each other and gets kinda of confusing of what's being thrown into the variables scope.

Because of this, we've taken the liberty to create for you two scopes that you can use to store variables when writing your tests:

global: Place any variables that will be used within THROUGH OUT a group of tests in here.

loc: Place any variables that will be used within a SINGLE test in here. This scope is recreated for each test to make sure that variables created during tests don't overwrite each other.

NOTE: Any variables that you have declared within your global scope will be automatically duplicated and available to you within your loc scope. We do this so that you only need to worry about accessing the loc scope when writing your tests.

Commands

assert(): This is the main command that you will be using when developing tests. To use, all you have to do is provide a quoted expression. The power of this is that ANY expression can be used.

An example test that checks that two values equal each other:

<cffunction name="test_a_equals_b">
  <cfset loc.a = 1>
  <cfset loc.b = 1>
  <cfset assert("loc.a eq loc.b")>
</cffunction>

An example test that checks that the first value is less then the second value:

<cffunction name="test_a_is_less_than_b">
  <cfset loc.a = 1>
  <cfset loc.b = 5>
  <cfset assert("loc.a lt loc.b")>
</cffunction>

You get the idea since you've used these kinds of expressions a thousand times. If you think of the assert() command as another way of using evaluate(), it will all make sense. Remember that you can use ANY expression, so you can write assertions against structures, arrays, objects, you name it, you can test it!

An example test that checks that a key exists in a structure:

<cffunction name="test_key_exists_in_structure">
  <cfset loc.a = {a=1, b=2, c=3}>
  <cfset loc.b = "d">
  <cfset assert("structkeyexists(loc.a, loc.b)")>
</cffunction>

raised(): Used when you want to test that an exception will be thrown. Raised() will raise and catch the exception and return to you the exception type (think cfcatch.type). Just like assert(), raise() takes a quoted expression as it's argument.

An example of raising the Wheels.TableNotFound error when you specify an invalid model name:

<cffunction name="test_table_not_found">
  <cfset loc.controller = createobject("component", "wheels.tests.ControllerBlank")>
  <cfset loc.e = raised("loc.controller.model('InvalidModel')")>
  <cfset loc.r = "Wheels.TableNotFound">
  <cfset assert("loc.e eq loc.r")>
</cffunction>

halt(): Think of this as doing 'cfdump/cfabort` only having some control over it. Often times when testing you want to check the results of an expression so that you can write proper tests against it. The old way of doing this is creating a test.cfm template, writing the code you want to check in it, saving it and then finally running it. Tell me that doesn't slow you down. By using halt() you get that same benefit only you don't have to go through all that extra stuff.

Halt() also takes a quoted expression as a second argument, but it's the first argument that we need to take a moment to talk about. The first argument in halt() is a boolean (true/false) value that tells halt() whether it should stop or halt (see where it gets it name from) the testing process and dump the contents of the quoted expression to the screen. By setting this first argument to true, halt() will do just that, but what if you set this first argument to false? In that case halt() will not only NOT display the results of the quoted expression to the screen and continue on with the testing process, it will also not even bother trying to evaluate the quoted expression. Because of that, it's safe it keep as many halt() calls with the first argument set to false inside your tests for later reference since there's no performance hit.

An example of halting the testing process and dumping an expression to the screen:

<cffunction name="test_key_exists_in_structure">
  <cfset loc.a = {a=1, b=2, c=3}>
  <cfset loc.b = "d">
  <cfset halt(true, "loc.a")>
  <cfset assert("structkeyexists(loc.a, loc.b)")>
</cffunction>

An example of keeping halt in a test for future reference:

<cffunction name="test_key_exists_in_structure">
  <cfset loc.a = {a=1, b=2, c=3}>
  <cfset loc.b = "d">
  <cfset halt(false, "loc.a")>
  <cfset assert("structkeyexists(loc.a, loc.b)")>
</cffunction>

Methods

There are four reserved methods that Wheels's testing framework provides for you:

_setup(): This method is run ONCE for each test component BEFORE ANY TESTS have been ran. You can use this for setting up any variables that will be used through all the tests in that component.

_teardown(): This method is ran ONCE for each test component AFTER ALL TESTS have been ran. You can use this for cleaning up any variables that you might have set within the setup() method.

setup(): Used to initialize or override any variables or execute any code that needs to be run BEFORE EACH test.

teardown(): Used to cleanup any variables or execute any code that needs to be ran AFTER EACH test.

Remember that you don't have to use these methods when creating your tests, however they do help to keep thing DRY (Don't Repeat Yourself) and compact.

Example:

<cfcomponent extends="wheels.test">

	<cfset global.controller = createobject("component", "wheels.Controller")>
	<cfset global.f = global.controller.distanceOfTimeInWords>
	<cfset global.args = {}>
	<cfset global.args.fromTime = now()>
	<cfset global.args.includeSeconds = true>

	<cffunction name="test_with_seconds_below_5_seconds">
		<cfset loc.c = 5 - 1>
		<cfset loc.args.toTime = dateadd('s', loc.c, loc.args.fromTime)>
		<cfinvoke method="loc.f" argumentcollection="#loc.args#" returnvariable="loc.e">
		<cfset loc.r = "less than 5 seconds">
		<cfset assert("loc.e eq loc.r")>
	</cffunction>

	<cffunction name="test_with_seconds_below_10_seconds">
		<cfset loc.c = 10 - 1>
		<cfset loc.args.toTime = dateadd('s', loc.c, loc.args.fromTime)>
		<cfset halt(false, "loc.controller.distanceOfTimeInWords(argumentcollection=loc.args)")>
		<cfinvoke method="loc.f" argumentcollection="#loc.args#" returnvariable="loc.e">
		<cfset loc.r = "less than 10 seconds">
		<cfset assert("loc.e eq loc.r")>
	</cffunction>

</cfcomponent>

Running Tests

Down in the debug area of your Wheels application (that grey area at the bottom of the page), you will notice a Run Tests link in the following areas:

Application Name, Framework and Plugins. Each of these links runs the following suite of tests:

Application Name: Runs any tests that you have created for your application. You should run these tests before deploying your application.

Framework: Runs any test that the Wheels Team has created for the Wheels framework itself. You should run these tests whenever you update your Wheels installation or are switching platforms.

Plugins: A Run Tests link will be next to each installed plugin you have in your wheels installation. You should run these tests whenever you install or update a plugin.

Running Selected Test Packages

Very often you might want to only run a package of tests (a group of tests within a single directory). In order to do this, just append the package argument to the url with the name of the package you want ran. For instance, say you had a test package called UserVerifcation. To run only that test package, add &package=UserVerifcation to the end of the test url.