|
RunningSpecs
How to execute specifications
DependenciesThe following libraries are necessary to run specifications. Please check carefully the version numbers! The junit library also allows you to run the specifications as JUnit suites by mixing-in the org.specs.runner.JUnit trait. The junit library is a requirement for specs because the @RunWith annotations on the JUnit is unfortunately not inherited by the Specification class. If you use ScalaCheck you must add If you use mock objects with jMock, add And if you mock classes or traits with attributes you will need If you use mock objects with Mockito, add If you use mock objects with EasyMock, add If you want to run the specifications as ScalaTest suites, use If you want to use Textile markup in Literate Specifications, you need to add Run your specification in the ConsoleYou can run your specifications in the console just by invoking scala on the Specification class: package org.hw
import org.specs._
object helloWorld extends Specification { ... }java -cp ... org.hw.helloWorld Getting some helpYou can display a help message with all available options by passing the -h or --help options on the command line. Removing stacktraces from the reportIf you have several exceptions being thrown and if stacktraces are obscuring the Console output you can turn them off by passing the -ns or --nostacktrace option. Showing only failures and errorsYou can also choose to display only the failed and error examples by using either the -xonly or the --failedonly option ("x" like the symbol showing a failure or an error in the console). Don't show statisticsFor big specifications, if you want to remove the display of statistics, you can use the -nostats or --nostatistics flag to prevent statistics to be displayed. Filter systems and examplesThe following flags can be used to run only some systems or some examples: -sus or --system regexp runs only the systems which description matches the regular expression -ex or --example regexp runs only the examples which description matches the regular expression Report with colorsIf your console supports it, you can display success, failures and skipped examples in color with the -c or --color flag. Run a Specification classIf a Specification is declared as an object you can run it directly on the command line. However, if you declare the Specification as a class (to make it a JUnit test for example), you will have to use the run class to execute the Specification on the command line: java -cp <classpath> run org.specs.samples.mySpec If the class can't be instantiated a stacktrace will be printed. Run several Specification classesYou can also run many specifications at once with the -k flag (or --classes): java -cp <classpath> run -k org.specs.samples.mySpec,org.specs.samples.mySpec2 And the package names can be factored with -p (or --packages): java -cp <classpath> run -p org.specs.samples -k mySpec,mySpec2 Additionally you can display instantiations issues with the -v flag. Display the helpThe help flag: -h or --help can be used to display the available flags and their description: usage java classpath package.mySpecification [-h|--help]
[-ns|--nostacktrace]
[-finalstats|--finalstatistics]
[-nostats|--nostatistics]
[-xonly | -failedonly]
[[-acc | --accept] tag1,tag2,...] [[-rej | --reject] tag1,tag2,...]
[-sus | --system]
[-ex | --example]
[-plan | --planonly]
[-c | --color]
-h, --help print this message and doesn't execute the specification
-config, --configuration class name of an object extending the org.specs.util.Configuration trait
-ns, --nostacktrace remove the stacktraces from the reporting
-nostats, --nostatistics remove the statistics from the reporting
-finalstats, --finalstatistics print the final statistics only
-xonly, --failedonly report only failures and errors
-acc, --accept tags accept only the specified tags (comma-separated names)
-rej, --reject tags reject the specified tags (comma-separated names)
-sus, --system only the systems under specifications matching this regular expression will be executed
-ex, --example only the examples matching this regular expression will be executed
-plan, --planOnly only display the sus and first level descriptions without executing the examples
-c, --color report with colors
Note that the specification is not executed in that case. Override specs default behaviorYou can override specs behavior by providing a configuration (see below). This configuration file allows you to set default reporting options for:
But it also allows to control the behavior of specs for:
This how you declare a specs configuration:
/** this value controls if the errors stacktrace should be printed. */ def stacktrace = true /** this value controls if ok examples should be printed. */ def failedAndErrorsOnly = false /** this value controls if the statistics should be printed. */ def statistics = true /** this value controls if the final statistics should be printed. */ def finalStatisticsOnly = false /** this value controls if the ANSI color sequences should be used to colorize output */ def colorize = false /** this value controls if examples without expectations should be marked as PENDING examples */ def examplesWithoutExpectationsMustBePending = true
Find specifications in a directory pathYou can use the SpecsFinder class to find specifications in a given path: object displaySpecifications extends SpecsFinder with Application {
// print all specifications contained in subdirectories of the project directory
// whose names are matching "all.*"
specificationNames("project/**/*.scala", "all.*") foreach { println(_) }
}To be more specific, the name of the specification will be retrieved if the scala file contains: \\s*object\\s*(" + pattern + ")\\s*extends\\s*.*Spec.*\\s*\\{"Execute specifications in a directory pathYou simply use the SpecsFileRunner, with the parameters required for the SpecsFinder: import org.specs.runner.SpecsFileRunner
object allSpecsRunner extends SpecsFileRunner("project/**/*.scala", "all.*")The SpecsFileRunner will find possible specification names with the SpecsFinder and will try to instantiate them, keeping them only if the resulting object is an instance of Specification. How to redirect the results on a different outputIf you want to redirect the result of the execution to a file for example, you have to create a new trait extending the Output trait: trait FileOutput extends Output {
def println(m: Any) = {...}
def printf(format: String, args: Any*) = {...}
def flush() = {...}
}And then you "mix" it with the ConsoleRunner: object mySpecRunner extends ConsoleRunner(mySpec) with FileOutput
object mySpec extends Specification { ... }On the command line, you can then invoke: java -cp ... mySpecRunner Run your specification with JUnit4To execute your specification with JUnit, you need to use the JUnit4 class: import org.specs.runner.JUnit4
class mySpecTest extends JUnit4(mySpec)
object mySpec extends Specification { ... }The name of the test class will be <package name>.mySpecTest Note: we should declare the JUnit runner as a class and not an object, otherwise Ant and Maven test tasks won't be able to instantiate the class properly. Moreover, following the convention of having 'Test' at the end of the name should be make it being picked up by default. On the command line, you can run your suite by executing: java -cp ... org.junit.runner.JUnitCore mySpecTest Note that subexamples of an example won't be reported as separate tests but their failure messages will be added to the parent example failure messages. Alternate method of declaring a specification runnable with JUnitBefore version 1.5.1, you can directly mix in the JUnit trait to your Specification: import org.specs.runner.JUnit
class mySpec extends Specification with JUnit { ... }Starting from version 1.5.1, this feature has been replaced with a direct inheritance of the SpecificationWithJUnit class: class mySpec extends SpecificationWithJUnit { ... }This was necessary in order to remove a dependency between the Specification class and the junit libraries when JUnit is not required. Run your specification with JUnit4 in EclipseIn order to be able to select your JUnit4 classes, you need to add the output directory of your project to your build path:
You should now be able to select the JUnit4 classes of your Scala project. Run your specifications with JUnit4 and MavenBy default, you need to respect the surefire plugin naming convention: // all JUnit4 tests must end with "Test" // it must be a class, not an object, otherwise the class name would be mySpecTest$ class mySpecTest extends JUnit4(mySpec) object mySpec extends Spec Then you run the usual mvn test to compile main code, test code and execute the tests. If you want another naming convention to be used with Maven, you can add this section to your pom file: <plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.4.3</version>
<configuration>
<useSystemClassLoader>false</useSystemClassLoader>
<argLine>-Xmx512m</argLine>
<includes>
<include>**/*Unit.java</include>
<include>**/*Spec.java</include>
</includes>
</configuration>
</plugin>
<plugin>This section above will execute all specification classes whose name end with Unit or Spec (note the .java at the end of the include name). Run your specification with JUnit4 and AntYou can use the following Ant build file as a starter to compile and execute your specs as JUnit tests: <project name="MyFirstSpecsProject" default="test" basedir=".">
<description>sample build file</description>
<!-- 1. Define common properties. Change the paths according to your installation -->
<property name="src.dir" value="src/main/scala" />
<property name="src.test.dir" value="src/test/scala" />
<property name="build.dir" value="target/classes" />
<property name="lib.dir" value="lib" />
<property name="repository.home" value="c:/local_repository" />
<property name="scala-compiler.jar"
value="${repository.home}/org/scala-lang/scala-compiler/2.7.7/scala-compiler-2.7.7.jar" />
<property name="scala-library.jar"
value="${repository.home}/org/scala-lang/scala-library/2.7.7/scala-library-2.7.7.jar" />
<!-- 2. Define Scala CLASSPATH. -->
<path id="scala.classpath">
<pathelement location="${scala-compiler.jar}" />
<pathelement location="${scala-library.jar}" />
</path>
<!-- 3. Define project CLASSPATH. -->
<path id="project.classpath">
<path refid="scala.classpath" />
<pathelement location="${build.dir}" />
<pathelement location="${repository.home}/junit/junit/4.4/junit-4.4.jar" />
<pathelement location="${repository.home}/org/scala-tools/testing/specs/1.6.1/specs-1.6.1.jar" />
<pathelement location="http://scala-tools.org/repo-snapshots/org/scala-tools/testing/scalacheck/1.6/scalacheck-1.6.jar" />
</path>
<!-- 4. Define scala compiler command. -->
<taskdef resource="scala/tools/ant/antlib.xml">
<classpath refid="scala.classpath" />
</taskdef>
<!-- 5. Compiles sources by using "scalac" command. -->
<target name="compile">
<mkdir dir="${build.dir}" />
<scalac srcdir="${src.dir}" destdir="${build.dir}" classpathref="project.classpath" force="changed">
<include name="**/*.scala" />
</scalac>
</target>
<target name="test-compile">
<mkdir dir="${build.dir}" />
<scalac srcdir="${src.test.dir}" destdir="${build.dir}" classpathref="project.classpath" force="changed">
<include name="**/*.scala" />
</scalac>
</target>
<!-- 6. Execute the specs as junit tests. -->
<target name="test" description="execute the tests">
<junit haltonfailure="true" showoutput="true">
<classpath refid="project.classpath" />
<formatter type="brief" usefile="false" />
<batchtest fork="yes">
<fileset dir="${build.dir}">
<include name="**/*Test.class" />
<exclude name="**/All*Test.class" />
</fileset>
</batchtest>
</junit>
</target>
<taskdef name="junit" classname="org.apache.tools.ant.taskdefs.optional.junit.JUnitTask" />
</project>(the initial compile script comes from this blog post) Run your specifications and get the results as an XML fileYou can use the XmlRunner class to export the results of your specifications as an xml file: object specResults extends XmlRunner(extendedThrowableUnit) This will create a file named extendedThrowableUnit.xml in the current directory: <spec errors="0" description="extendedThrowableUnit" failures="0" assertions="1" name="extendedThrowableUnit">
<sut errors="0" description="an extended Throwable" failures="0" assertions="1">
<example errors="0" description="provide a location method extracting the name of the file and the line from an exception" failures="0" assertions="1"></example>
</sut>
</spec>and display: Specification "extendedThrowableUnit" an extended Throwable should + provide a location method extracting the name of the file and the line from an exception Total for specification "extendedThrowableUnit": Finished in 0 second, 16 ms 1 example, 1 assertion, 0 failure, 0 error It is also possible to specify the output directory with: object specResults extends XmlRunner(extendedThrowableUnit, "./target/reports/specs") Run your specification with Scala TestDeclare a runner extending the ScalaTestSuite runner: import org.specs.runner._ class mySpecSuite extends ScalaTestSuite(mySpec) Then use the ScalaTest gui runner: java -cp ... org.scalatest.Runner -g -s mySpecSuite Run your specification with Scala Test in IDEA IntelliJJust add the ScalaTest trait to your specification (declared as a class, see below) and you're done! Run your specification with either the Console, JUnit or ScalaTestThe following specification: package org.specs.samples
import org.specs._
import org.specs.runner._
class helloWorldTest extends Specification with JUnit with ScalaTest { ... }
Run your specification with Team CityYou can use the TeamCityRunner to get an output parsable by TeamCity4: import org.specs._
import org.specs.runner._
object mySpec extends Specification { ... }
object tcRunner extends TeamCityRunner(mySpec)(see some screenshots: here and there) Declare several spec runners at onceYou may want to execute a specification with several runners. In that case you can declare: import org.specs.runner._ class mySpecRunner extends Runner(mySpec) with JUnit with ScalaTest with Console // or alternately if you also require an xml output class mySpecRunner extends Runner(mySpec) with JUnit with ScalaTest with Xml The execution of each runner can be done via: JUnit4 -> java -cp ... org.junit.runner.JUnitCore mySpecRunner ScalaTest -> java -cp ... org.scalatest.Runner -g -s mySpecRunner XmlRunner -> scala -cp ... -e "(new mySpecRunner).reportSpecs" ConsoleRunner -> scala -cp ... -e "(new mySpecRunner).reportSpecs" Skip examplesYou can skip several examples by using the skip method: object mySkippedSpecification extends Specification {
"These examples" should {
skip("those examples don't pass yet")
"be skipped" in {...}
"be skipped2" in {...}
}
}You can also skip an individual example if a matcher is not satisfied: object mySpecification extends Specification {
"my web framework" should {
"work with DB2" in {
// skip the example if the DB2 connection is not available locally
DB2Connection.start must not(throwA(ConnectionException(""))).orSkipExample // alias orSkip
// use DB2Connection
}
...
}
}Skipped examples will appear in the Console runner with a small 'o' my web framework should
o work with DB2
skipped because DB2 cannot connect was thrownOtherwise, the skipped examples will be reported in the JUnit reports if using a JUnit4 runner to execute the specs. Include or exclude examplesMost of the time examples are naturally classified and grouped according to the Specification they belong to. However, sometimes you may want to group examples differently in order to execute them separately for instance. One very common use of this is when you want to run one example only inside a Specification containing a lot of other examples. Instead of commenting out the other examples you can:
"this example is tagged" in {
// assertions
} tag("only this")
// other examples will be marked as skipped
mySpec.accept("only this")It is possible also to reject tags in order to exclude some examples: // returns the Specification object so that
// class SpecTest extends JUnit4(mySpec.accept("tag1")) is valid
mySpec.accept("only this").reject("still failing") Tags are also applicable to whole systems under specification so that all included examples will also be tagged: "this system" should {
// lots of examples
} tag("basic functionalities")From the command line, the tags can be accepted or rejected with the following options:
For example: scala mySpecification --accept tag1,tag2 --reject tag3 Specification planA Specification can be executed so as to display its systems and first-level examples descriptions without executing them. In order to activate this functionality, you need to:
Then sus and examples descriptions are displayed like this: - This system should
- do this
- do that
|
Sign in to add a comment
I'm kicking myself for my stupidity, but I just spent about an hour trying to get junit & ant working because I went straight to http://code.google.com/p/specs/wiki/RunningSpecs#Run_your_specification_with_JUnit4_and_Ant
Perhaps it should be nested under, or make reference to the all-important http://code.google.com/p/specs/wiki/RunningSpecs#Run_your_specification_with_JUnit4
The 1.6.0-SNAPSHOT worked well with:
class helloWorldTest extends SpecificationWithJUnit{...}
but my:
project> mvn test -Drun.mode=test
only gave me:
Running com.test.helloWorldTest Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.157 sec Results : Tests run: 2, Failures: 0, Errors: 0, Skipped: 0
... and none of the goodies, like which tests were run, colorization, and keeping the runner going.
How can I get those things? Thanks for a fine product!
Oh yes, I neglected to mention it, but I'm running this through the Lift framework...
http://code.google.com/p/specs/wiki/RunningSpecs#Run_your_specification_with_JUnit4_in_Eclipse
is not possible, due to the error (@ Step 6): "The folder 'bin' already exists."
That leads to the problem that the tests cannot be executed with a right-click and I received the message: "No tests found with test runner 'JUnit 4'.
I am using:
Best Regards, Nicolas Gramlich
It would be super-awesome if you could ping me then: ( nicolasgramlich {at} gmail dot? com }
Ok @ Running it in Eclipse with JUnit:
You would add sth like: E:\Java\MyBool?\bin\org\anddev\scala\dsl\mybool
not just E:\Java\MyBool?\bin
Works perfectyl now =)
Hi gregor007,
Can you please create an issue in the issue tracker or on the mailing-list because there's no way for me to be notified of comments in the wiki?
Concerning what you describe:
Thanks for clarifying those points,
Eric.