org.scalatest.PropSpec.scala Maven / Gradle / Ivy
Show all versions of scalatest_2.11.0-M3 Show documentation
/*
* Copyright 2001-2008 Artima, Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.scalatest
import scala.collection.immutable.ListSet
import java.util.ConcurrentModificationException
import java.util.concurrent.atomic.AtomicReference
import org.scalatest.exceptions.StackDepthExceptionHelper.getStackDepth
import org.scalatest.events._
import Suite.anErrorThatShouldCauseAnAbort
import Suite.checkRunTestParamsForNull
/**
* A suite of property-based tests.
*
*
* This trait facilitates a style of testing in which each test is composed
* of one property check. Tests are registered via a "property" method, and given a name and a body.
* (A PropSpec behaves just like a FunSuite, except test is replaced with
* property.) You can do anything in the body of the test, but the intention is that you'd check
* one property in each test. To write properties in the ScalaCheck style, mix Checkers into
* your PropSpec. To write them in the ScalaTest style, mix in PropertyChecks.
*
*
*
* For example, given this Fraction class:
*
*
*
* class Fraction(n: Int, d: Int) {
* require(d != 0)
* require(d != Integer.MIN_VALUE)
* require(n != Integer.MIN_VALUE)
*
* val numer = if (d < 0) -1 * n else n
* val denom = d.abs
*
* override def toString = numer + " / " + denom
* }
*
*
*
* You could write a PropSpec in the ScalaTest property style that specifies the Fraction behavior like this:
*
*
*
* import org.scalatest.PropSpec
* import org.scalatest.prop.PropertyChecks
* import org.scalatest.matchers.ShouldMatchers
*
* class FractionSpec extends PropSpec with PropertyChecks with ShouldMatchers {
*
* property("Fraction constructor normalizes numerator and denominator") {
*
* forAll { (n: Int, d: Int) =>
* whenever (d != 0 && d != Integer.MIN_VALUE
* && n != Integer.MIN_VALUE) {
*
* val f = new Fraction(n, d)
*
* if (n < 0 && d < 0 || n > 0 && d > 0)
* f.numer should be > 0
* else if (n != 0)
* f.numer should be < 0
* else
* f.numer should be === 0
*
* f.denom should be > 0
* }
* }
* }
*
* property("Fraction constructor throws IAE on bad data.") {
*
* val invalidCombos =
* Table(
* ("n", "d"),
* (Integer.MIN_VALUE, Integer.MIN_VALUE),
* (1, Integer.MIN_VALUE),
* (Integer.MIN_VALUE, 1),
* (Integer.MIN_VALUE, 0),
* (1, 0)
* )
*
* forAll (invalidCombos) { (n: Int, d: Int) =>
* evaluating {
* new Fraction(n, d)
* } should produce [IllegalArgumentException]
* }
* }
* }
*
*
*
* “property” is a method, defined in PropSpec, which will be invoked
* by the primary constructor of MathSpec. You specify the name of the property as
* a string between the parentheses, and the test code containing the property check between curly braces.
* The test code is a function passed as a by-name parameter to property, which registers
* it for later execution.
*
*
*
* A PropSpec's lifecycle has two phases: the registration phase and the
* ready phase. It starts in registration phase and enters ready phase the first time
* run is called on it. It then remains in ready phase for the remainder of its lifetime.
*
*
*
* Properties can only be registered with the property method while the PropSpec is
* in its registration phase. Any attempt to register a property after the PropSpec has
* entered its ready phase, i.e., after run has been invoked on the PropSpec,
* will be met with a thrown TestRegistrationClosedException. The recommended style
* of using PropSpec is to register properties during object construction as is done in all
* the examples shown here. If you keep to the recommended style, you should never see a
* TestRegistrationClosedException.
*
*
*
* Note: Trait PropSpec is in part inspired by class org.scalacheck.Properties, designed by
* Rickard Nilsson for the ScalaCheck test framework.
*
*
* Ignored tests
*
*
* To support the common use case of “temporarily” disabling a test, with the
* good intention of resurrecting the test at a later time, PropSpec provides registration
* methods that start with ignore instead of property. For example, to temporarily
* disable the test named addition, just change “property” into “ignore,” like this:
*
*
*
* import org.scalatest.PropSpec
* import org.scalatest.prop.PropertyChecks
* import org.scalatest.matchers.ShouldMatchers
*
* class MathSpec extends PropSpec with PropertyChecks with ShouldMatchers {
*
* ignore("addition", SlowTest) {
* forAll { (i: Int) => i + i should equal (2 * i) }
* }
*
* property("subtraction", SlowTest, DbTest) {
* forAll { (i: Int) => i - i should equal (0) }
* }
* }
*
*
*
* If you run this version of MathSpec with:
*
*
*
* scala> (new MathSpec).execute()
*
*
*
* It will run only subtraction and report that addition was ignored:
*
*
*
* MathSpec:
* - addition !!! IGNORED !!!
* - subtraction
*
*
* Informers
*
*
* One of the parameters to the run method is a Reporter, which
* will collect and report information about the running suite of tests.
* Information about suites and tests that were run, whether tests succeeded or failed,
* and tests that were ignored will be passed to the Reporter as the suite runs.
* Most often the reporting done by default by PropSpec's methods will be sufficient, but
* occasionally you may wish to provide custom information to the Reporter from a test.
* For this purpose, an Informer that will forward information to the current Reporter
* is provided via the info parameterless method.
* You can pass the extra information to the Informer via one of its apply methods.
* The Informer will then pass the information to the Reporter via an InfoProvided event.
* Here's an example:
*
*
*
* import org.scalatest.PropSpec
* import org.scalatest.prop.PropertyChecks
* import org.scalatest.matchers.ShouldMatchers
*
* class MathSpec extends PropSpec with PropertyChecks with ShouldMatchers {
*
* property("addition", SlowTest) {
* forAll { (i: Int) => i + i should equal (2 * i) }
* info("Addition seems to work")
* }
*
* property("subtraction", SlowTest, DbTest) {
* forAll { (i: Int) => i - i should equal (0) }
* }
* }
*
*
* If you run this PropSpec from the interpreter, you will see the following message
* included in the printed report:
*
*
* MathSpec:
* - addition
* + Addition seems to work
*
*
* Pending tests
*
*
* A pending test is one that has been given a name but is not yet implemented. The purpose of
* pending tests is to facilitate a style of testing in which documentation of behavior is sketched
* out before tests are written to verify that behavior (and often, before the behavior of
* the system being tested is itself implemented). Such sketches form a kind of specification of
* what tests and functionality to implement later.
*
*
*
* To support this style of testing, a test can be given a name that specifies one
* bit of behavior required by the system being tested. The test can also include some code that
* sends more information about the behavior to the reporter when the tests run. At the end of the test,
* it can call method pending, which will cause it to complete abruptly with TestPendingException.
*
*
*
* Because tests in ScalaTest can be designated as pending with TestPendingException, both the test name and any information
* sent to the reporter when running the test can appear in the report of a test run. (In other words,
* the code of a pending test is executed just like any other test.) However, because the test completes abruptly
* with TestPendingException, the test will be reported as pending, to indicate
* the actual test, and possibly the functionality, has not yet been implemented.
*
*
*
* Although pending tests may be used more often in specification-style suites, such as
* org.scalatest.FunSpec, you can also use it in PropSpec, like this:
*
*
*
* import org.scalatest.PropSpec
* import org.scalatest.prop.PropertyChecks
* import org.scalatest.matchers.ShouldMatchers
*
* class MathSpec extends PropSpec with PropertyChecks with ShouldMatchers {
*
* ignore("addition") {
* forAll { (i: Int) => i + i should equal (2 * i) }
* }
*
* property("subtraction") (pending)
* }
*
*
*
* (Note: "(pending)" is the body of the test. Thus the test contains just one statement, an invocation
* of the pending method, which throws TestPendingException.)
* If you run this version of MathSpec with:
*
*
*
* scala> (new MathSpec).execute()
*
*
*
* It will run both tests, but report that subtraction is pending. You'll see:
*
*
*
* MathSpec:
* - addition
* - subtraction (pending)
*
*
* Tagging tests
*
*
* A PropSpec's tests may be classified into groups by tagging them with string names.
* As with any suite, when executing a PropSpec, groups of tests can
* optionally be included and/or excluded. To tag a PropSpec's tests,
* you pass objects that extend abstract class org.scalatest.Tag to methods
* that register tests, test and ignore. Class Tag takes one parameter, a string name. If you have
* created Java annotation interfaces for use as group names in direct subclasses of org.scalatest.Suite,
* then you will probably want to use group names on your PropSpecs that match. To do so, simply
* pass the fully qualified names of the Java interfaces to the Tag constructor. For example, if you've
* defined Java annotation interfaces with fully qualified names, com.mycompany.tags.SlowTest and
* com.mycompany.tags.DbTest, then you could
* create matching groups for PropSpecs like this:
*
*
*
* import org.scalatest.Tag
*
* object SlowTest extends Tag("com.mycompany.tags.SlowTest")
* object DbTest extends Tag("com.mycompany.tags.DbTest")
*
*
*
* Given these definitions, you could tag a PropSpec's tests like this:
*
*
*
* import org.scalatest.PropSpec
* import org.scalatest.prop.PropertyChecks
* import org.scalatest.matchers.ShouldMatchers
*
* class MathSpec extends PropSpec with PropertyChecks with ShouldMatchers {
*
* property("addition", SlowTest) {
* forAll { (i: Int) => i + i should equal (2 * i) }
* }
*
* property("subtraction", SlowTest, DbTest) {
* forAll { (i: Int) => i - i should equal (0) }
* }
* }
*
*
*
* This code marks both tests, "addition" and "subtraction," with the com.mycompany.tags.SlowTest tag,
* and test "subtraction" with the com.mycompany.tags.DbTest tag.
*
*
*
* The run method takes a Filter, whose constructor takes an optional
* Set[String] called tagsToInclude and a Set[String] called
* tagsToExclude. If tagsToInclude is None, all tests will be run
* except those those belonging to tags listed in the
* tagsToExclude Set. If tagsToInclude is defined, only tests
* belonging to tags mentioned in the tagsToInclude set, and not mentioned in tagsToExclude,
* will be run.
*
*
* Shared fixtures
*
*
* A test fixture is objects or other artifacts (such as files, sockets, database
* connections, etc.) used by tests to do their work. You can use fixtures in
* PropSpecs with the same approaches suggested for FunSuite in
* its documentation. For more information, see the Shared fixtures section of FunSuite's
* documentation (and substitute property for test).
*
*
* Shared tests
*
*
* Sometimes you may want to run the same test code on different fixture objects. In other words, you may want to write tests that are "shared"
* by different fixture objects.
* You accomplish this in a PropSpec in the same way you would do it in a FunSuite, exception instead of test
* you say property, and instead of testsFor you say propertiesFor.
* For more information, see the Shared tests section of FunSuite's
* documentation.
*
*
* @author Bill Venners
*/
trait PropSpec extends Suite { thisSuite =>
private final val engine = new Engine("concurrentPropSpecMod", "PropSpec")
import engine._
/**
* Returns an Informer that during test execution will forward strings (and other objects) passed to its
* apply method to the current reporter. If invoked in a constructor, it
* will register the passed string for forwarding later during test execution. If invoked while this
* PropSpec is being executed, such as from inside a test function, it will forward the information to
* the current reporter immediately. If invoked at any other time, it will
* throw an exception. This method can be called safely by any thread.
*/
implicit protected def info: Informer = atomicInformer.get
/**
* Register a property-based test with the specified name, optional tags, and function value that takes no arguments.
* This method will register the test for later execution via an invocation of one of the run
* methods. The passed test name must not have been registered previously on
* this PropSpec instance.
*
* @param testName the name of the property
* @param testTags the optional list of tags for this property
* @param testFun the property function
* @throws TestRegistrationClosedException if invoked after run has been invoked on this suite
* @throws DuplicateTestNameException if a test with the same name has been registered previously
* @throws NotAllowedException if testName had been registered previously
* @throws NullPointerException if testName or any passed test tag is null
*/
protected def property(testName: String, testTags: Tag*)(testFun: => Unit) {
registerTest(testName, testFun _, "propertyCannotAppearInsideAnotherProperty", "PropSpec.scala", "property", 2, None, None, testTags: _*)
}
/**
* Register a property-based test to ignore, which has the specified name, optional tags, and function value that takes no arguments.
* This method will register the test for later ignoring via an invocation of one of the run
* methods. This method exists to make it easy to ignore an existing test by changing the call to test
* to ignore without deleting or commenting out the actual test code. The test will not be run, but a
* report will be sent that indicates the test was ignored. The passed test name must not have been registered previously on
* this PropSpec instance.
*
* @param testName the name of the test
* @param testTags the optional list of tags for this test
* @param testFun the test function
* @throws TestRegistrationClosedException if invoked after run has been invoked on this suite
* @throws DuplicateTestNameException if a test with the same name has been registered previously
* @throws NotAllowedException if testName had been registered previously
*/
protected def ignore(testName: String, testTags: Tag*)(testFun: => Unit) {
registerIgnoredTest(testName, testFun _, "ignoreCannotAppearInsideAProperty", "PropSpec.scala", "ignore", 1, testTags: _*)
}
/**
* An immutable Set of test names. If this PropSpec contains no tests, this method returns an empty Set.
*
*
* This trait's implementation of this method will return a set that contains the names of all registered tests. The set's iterator will
* return those names in the order in which the tests were registered.
*
*/
override def testNames: Set[String] = {
// I'm returning a ListSet here so that they tests will be run in registration order
ListSet(atomic.get.testNamesList.toArray: _*)
}
/**
* Run a test. This trait's implementation runs the test registered with the name specified by testName.
*
* @param testName the name of one test to run.
* @param reporter the Reporter to which results will be reported
* @param stopper the Stopper that will be consulted to determine whether to stop execution early.
* @param configMap a Map of properties that can be used by the executing Suite of tests.
* @throws IllegalArgumentException if testName is defined but a test with that name does not exist on this PropSpec
* @throws NullPointerException if any of testName, reporter, stopper, or configMap
* is null.
*/
protected override def runTest(testName: String, reporter: Reporter, stopper: Stopper, configMap: Map[String, Any], tracker: Tracker) {
def invokeWithFixture(theTest: TestLeaf) {
val theConfigMap = configMap
withFixture(
new NoArgTest {
def name = testName
def apply() { theTest.testFun() }
def configMap = theConfigMap
}
)
}
runTestImpl(thisSuite, testName, reporter, stopper, configMap, tracker, true, invokeWithFixture)
}
/**
* A Map whose keys are String tag names to which tests in this PropSpec belong, and values
* the Set of test names that belong to each tag. If this PropSpec contains no tags, this method returns an empty Map.
*
*
* This trait's implementation returns tags that were passed as strings contained in Tag objects passed to
* methods test and ignore.
*
*/
override def tags: Map[String, Set[String]] = atomic.get.tagsMap
/**
* Run zero to many of this PropSpec's tests.
*
* @param testName an optional name of one test to run. If None, all relevant tests should be run.
* I.e., None acts like a wildcard that means run all relevant tests in this Suite.
* @param reporter the Reporter to which results will be reported
* @param stopper the Stopper that will be consulted to determine whether to stop execution early.
* @param filter a Filter with which to filter tests based on their tags
* @param configMap a Map of key-value pairs that can be used by the executing Suite of tests.
* @param distributor an optional Distributor, into which to put nested Suites to be run
* by another entity, such as concurrently by a pool of threads. If None, nested Suites will be run sequentially.
* @param tracker a Tracker tracking Ordinals being fired by the current thread.
* @throws NullPointerException if any of the passed parameters is null.
* @throws IllegalArgumentException if testName is defined, but no test with the specified test name
* exists in this Suite
*/
protected override def runTests(testName: Option[String], reporter: Reporter, stopper: Stopper, filter: Filter,
configMap: Map[String, Any], distributor: Option[Distributor], tracker: Tracker) {
runTestsImpl(thisSuite, testName, reporter, stopper, filter, configMap, distributor, tracker, info, true, runTest)
}
override def run(testName: Option[String], reporter: Reporter, stopper: Stopper, filter: Filter,
configMap: Map[String, Any], distributor: Option[Distributor], tracker: Tracker) {
runImpl(thisSuite, testName, reporter, stopper, filter, configMap, distributor, tracker, super.run)
}
/**
* Registers shared tests.
*
*
* This method enables the following syntax for shared tests in a PropSpec:
*
*
*
* propertiesFor(nonEmptyStack(lastValuePushed))
*
*
*
* This method just provides syntax sugar intended to make the intent of the code clearer.
* Because the parameter passed to it is
* type Unit, the expression will be evaluated before being passed, which
* is sufficient to register the shared tests. For examples of shared tests, see the
* Shared tests section in the main documentation for this trait.
*
*/
protected def propertiesFor(unit: Unit) {}
/**
* Suite style name.
*/
final override val styleName: String = "org.scalatest.PropSpec"
}