org.scalatest.FeatureSpec.scala Maven / Gradle / Ivy
Show all versions of scalatest_2.8.0 Show documentation
/*
* Copyright 2001-2008 Artima, Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.scalatest
import NodeFamily._
import scala.collection.immutable.ListSet
import org.scalatest.StackDepthExceptionHelper.getStackDepth
import java.util.concurrent.atomic.AtomicReference
import java.util.ConcurrentModificationException
import org.scalatest.events._
import Suite.anErrorThatShouldCauseAnAbort
/**
* A suite of tests in which each test represents one scenario of a feature.
* FeatureSpec is intended for writing tests that are "higher level" than unit tests, for example, integration
* tests, functional tests, and acceptance tests. You can use FeatureSpec for unit testing if you prefer, however.
* Here's an example:
*
*
* import org.scalatest.FeatureSpec
* import org.scalatest.GivenWhenThen
* import scala.collection.mutable.Stack
*
* class StackFeatureSpec extends FeatureSpec with GivenWhenThen {
*
* feature("The user can pop an element off the top of the stack") {
*
* info("As a programmer")
* info("I want to be able to pop items off the stack")
* info("So that I can get them in last-in-first-out order")
*
* scenario("pop is invoked on a non-empty stack") {
*
* given("a non-empty stack")
* val stack = new Stack[Int]
* stack.push(1)
* stack.push(2)
* val oldSize = stack.size
*
* when("when pop is invoked on the stack")
* val result = stack.pop()
*
* then("the most recently pushed element should be returned")
* assert(result === 2)
*
* and("the stack should have one less item than before")
* assert(stack.size === oldSize - 1)
* }
*
* scenario("pop is invoked on an empty stack") {
*
* given("an empty stack")
* val emptyStack = new Stack[String]
*
* when("when pop is invoked on the stack")
* then("NoSuchElementException should be thrown")
* intercept[NoSuchElementException] {
* emptyStack.pop()
* }
*
* and("the stack should still be empty")
* assert(emptyStack.isEmpty)
* }
* }
* }
*
*
*
* A FeatureSpec contains feature clauses and scenarios. You define a feature clause
* with feature, and a scenario with scenario. Both
* feature and scenario are methods, defined in
* FeatureSpec, which will be invoked
* by the primary constructor of StackFeatureSpec.
* A feature clause describes a feature of the subject (class or other entity) you are specifying
* and testing. In the previous example,
* the subject under specification and test is a stack. The feature being specified and tested is
* the ability for a user (a programmer in this case) to pop an element off the top of the stack. With each scenario you provide a
* string (the spec text) that specifies the behavior of the subject for
* one scenario in which the feature may be used, and a block of code that tests that behavior.
* You place the spec text between the parentheses, followed by the test code between curly
* braces. The test code will be wrapped up as a function passed as a by-name parameter to
* scenario, which will register the test for later execution.
*
*
*
* A FeatureSpec's lifecycle has two phases: the registration phase and the
* ready phase. It starts in registration phase and enters ready phase the first time
* run is called on it. It then remains in ready phase for the remainder of its lifetime.
*
*
*
* Scenarios can only be registered with the scenario method while the FeatureSpec is
* in its registration phase. Any attempt to register a scenario after the FeatureSpec has
* entered its ready phase, i.e., after run has been invoked on the FeatureSpec,
* will be met with a thrown TestRegistrationClosedException. The recommended style
* of using FeatureSpec is to register tests during object construction as is done in all
* the examples shown here. If you keep to the recommended style, you should never see a
* TestRegistrationClosedException.
*
*
*
* Each scenario represents one test. The name of the test is the spec text passed to the scenario method.
* The feature name does not appear as part of the test name. In a FeatureSpec, therefore, you must take care
* to ensure that each test has a unique name (in other words, that each scenario has unique spec text).
*
*
*
* When you run a FeatureSpec, it will send Formatters in the events it sends to the
* Reporter. ScalaTest's built-in reporters will report these events in such a way
* that the output is easy to read as an informal specification of the subject being tested.
* For example, if you ran StackFeatureSpec from within the Scala interpreter:
*
*
*
* scala> (new StackFeatureSpec).execute()
*
*
*
* You would see:
*
*
*
* Feature: The user can pop an element off the top of the stack
* As a programmer
* I want to be able to pop items off the stack
* So that I can get them in last-in-first-out order
* Scenario: pop is invoked on a non-empty stack
* Given a non-empty stack
* When when pop is invoked on the stack
* Then the most recently pushed element should be returned
* And the stack should have one less item than before
* Scenario: pop is invoked on an empty stack
* Given an empty stack
* When when pop is invoked on the stack
* Then NoSuchElementException should be thrown
* And the stack should still be empty
*
*
* Shared fixtures
*
*
* A test fixture is objects or other artifacts (such as files, sockets, database
* connections, etc.) used by tests to do their work. You can use fixtures in
* FeatureSpecs with the same approaches suggested for Suite in
* its documentation. The same text that appears in the test fixture
* section of Suite's documentation is repeated here, with examples changed from
* Suite to FeatureSpec.
*
*
*
* If a fixture is used by only one test, then the definitions of the fixture objects can
* be local to the test function, such as the objects assigned to stack and emptyStack in the
* previous StackFeatureSpec examples. If multiple tests need to share a fixture, the best approach
* is to assign them to instance variables. Here's a (very contrived) example, in which the object assigned
* to shared is used by multiple test functions:
*
*
*
* import org.scalatest.FeatureSpec
*
* class ArithmeticFeatureSpec extends FeatureSpec {
*
* // Sharing immutable fixture objects via instance variables
* val shared = 5
*
* feature("Integer arithmetic") {
*
* scenario("addition") {
* val sum = 2 + 3
* assert(sum === shared)
* }
*
* scenario("subtraction") {
* val diff = 7 - 2
* assert(diff === shared)
* }
* }
* }
*
*
*
* In some cases, however, shared mutable fixture objects may be changed by tests such that
* they need to be recreated or reinitialized before each test. Shared resources such
* as files or database connections may also need to be created and initialized before,
* and cleaned up after, each test. JUnit offers methods setUp and
* tearDown for this purpose. In ScalaTest, you can use the BeforeAndAfterEach trait,
* which will be described later, to implement an approach similar to JUnit's setUp
* and tearDown, however, this approach often involves reassigning vars
* between tests. Before going that route, you should consider some approaches that
* avoid vars. One approach is to write one or more create-fixture methods
* that return a new instance of a needed object (or a tuple or case class holding new instances of
* multiple objects) each time it is called. You can then call a create-fixture method at the beginning of each
* test that needs the fixture, storing the fixture object or objects in local variables. Here's an example:
*
*
*
* import org.scalatest.FeatureSpec
* import scala.collection.mutable.ListBuffer
*
* class MyFeatureSpec extends FeatureSpec {
*
* // create objects needed by tests and return as a tuple
* def createFixture = (
* new StringBuilder("ScalaTest is "),
* new ListBuffer[String]
* )
*
* feature("The create-fixture approach") {
*
* scenario("shared fixture objects are mutated by a test") {
* val (builder, lbuf) = createFixture
* builder.append("easy!")
* assert(builder.toString === "ScalaTest is easy!")
* assert(lbuf.isEmpty)
* lbuf += "sweet"
* }
*
* scenario("test gets a fresh copy of the shared fixture") {
* val (builder, lbuf) = createFixture
* builder.append("fun!")
* assert(builder.toString === "ScalaTest is fun!")
* assert(lbuf.isEmpty)
* }
* }
* }
*
*
*
* If different tests in the same FeatureSpec require different fixtures, you can create multiple create-fixture methods and
* call the method (or methods) needed by each test at the begining of the test. If every test requires the same set of
* mutable fixture objects, one other approach you can take is make them simply vals and mix in trait
* OneInstancePerTest. If you mix in OneInstancePerTest, each test
* will be run in its own instance of the FeatureSpec, similar to the way JUnit tests are executed.
*
*
*
* Although the create-fixture and OneInstancePerTest approaches take care of setting up a fixture before each
* test, they don't address the problem of cleaning up a fixture after the test completes. In this situation,
* one option is to mix in the BeforeAndAfterEach trait.
* BeforeAndAfterEach's beforeEach method will be run before, and its afterEach
* method after, each test (like JUnit's setUp and tearDown
* methods, respectively).
* For example, you could create a temporary file before each test, and delete it afterwords, like this:
*
*
*
* import org.scalatest.FeatureSpec
* import org.scalatest.BeforeAndAfterEach
* import java.io.FileReader
* import java.io.FileWriter
* import java.io.File
*
* class FileIoFeatureSpec extends FeatureSpec with BeforeAndAfterEach {
*
* private val FileName = "TempFile.txt"
* private var reader: FileReader = _
*
* // Set up the temp file needed by the test
* override def beforeEach() {
* val writer = new FileWriter(FileName)
* try {
* writer.write("Hello, test!")
* }
* finally {
* writer.close()
* }
*
* // Create the reader needed by the test
* reader = new FileReader(FileName)
* }
*
* // Close and delete the temp file
* override def afterEach() {
* reader.close()
* val file = new File(FileName)
* file.delete()
* }
*
* feature("Reading and writing files") {
*
* scenario("reading from a temp file") {
* var builder = new StringBuilder
* var c = reader.read()
* while (c != -1) {
* builder.append(c.toChar)
* c = reader.read()
* }
* assert(builder.toString === "Hello, test!")
* }
*
* scenario("reading first char of a temp file") {
* assert(reader.read() === 'H')
* }
*
* scenario("no fixture is passed") {
* assert(1 + 1 === 2)
* }
* }
* }
*
*
*
* In this example, the instance variable reader is a var, so
* it can be reinitialized between tests by the beforeEach method.
*
*
*
* Although the BeforeAndAfterEach approach should be familiar to the users of most
* test other frameworks, ScalaTest provides another alternative that also allows you to perform cleanup
* after each test: overriding withFixture(NoArgTest).
* To execute each test, Suite's implementation of the runTest method wraps an invocation
* of the appropriate test method in a no-arg function. runTest passes that test function to the withFixture(NoArgTest)
* method, which is responsible for actually running the test by invoking the function. Suite's
* implementation of withFixture(NoArgTest) simply invokes the function, like this:
*
*
*
* // Default implementation
* protected def withFixture(test: NoArgTest) {
* test()
* }
*
*
*
* The withFixture(NoArgTest) method exists so that you can override it and set a fixture up before, and clean it up after, each test.
* Thus, the previous temp file example could also be implemented without mixing in BeforeAndAfterEach, like this:
*
*
*
* import org.scalatest.FeatureSpec
* import org.scalatest.BeforeAndAfterEach
* import java.io.FileReader
* import java.io.FileWriter
* import java.io.File
*
* class FileIoFeatureSpec extends FeatureSpec {
*
* private var reader: FileReader = _
*
* override def withFixture(test: NoArgTest) {
*
* val FileName = "TempFile.txt"
*
* // Set up the temp file needed by the test
* val writer = new FileWriter(FileName)
* try {
* writer.write("Hello, test!")
* }
* finally {
* writer.close()
* }
*
* // Create the reader needed by the test
* reader = new FileReader(FileName)
*
* try {
* test() // Invoke the test function
* }
* finally {
* // Close and delete the temp file
* reader.close()
* val file = new File(FileName)
* file.delete()
* }
* }
*
* feature("Reading and writing files") {
*
* scenario("reading from a temp file") {
* var builder = new StringBuilder
* var c = reader.read()
* while (c != -1) {
* builder.append(c.toChar)
* c = reader.read()
* }
* assert(builder.toString === "Hello, test!")
* }
*
* scenario("reading first char of a temp file") {
* assert(reader.read() === 'H')
* }
*
* scenario("no fixture is passed") {
* assert(1 + 1 === 2)
* }
* }
* }
*
*
*
* If you prefer to keep your test classes immutable, one final variation is to use the
* FixtureFeatureSpec trait from the
* org.scalatest.fixture package. Tests in an org.scalatest.fixture.FixtureFeatureSpec can have a fixture
* object passed in as a parameter. You must indicate the type of the fixture object
* by defining the Fixture type member and define a withFixture method that takes a one-arg test function.
* (A FixtureFeatureSpec has two overloaded withFixture methods, therefore, one that takes a OneArgTest
* and the other, inherited from Suite, that takes a NoArgTest.)
* Inside the withFixture(OneArgTest) method, you create the fixture, pass it into the test function, then perform any
* necessary cleanup after the test function returns. Instead of invoking each test directly, a FixtureFeatureSpec will
* pass a function that invokes the code of a test to withFixture(OneArgTest). Your withFixture(OneArgTest) method, therefore,
* is responsible for actually running the code of the test by invoking the test function.
* For example, you could pass the temp file reader fixture to each test that needs it
* by overriding the withFixture(OneArgTest) method of a FixtureFeatureSpec, like this:
*
*
*
* import org.scalatest.fixture.FixtureFeatureSpec
* import java.io.FileReader
* import java.io.FileWriter
* import java.io.File
*
* class MySuite extends FixtureFeatureSpec {
*
* type FixtureParam = FileReader
*
* def withFixture(test: OneArgTest) {
*
* val FileName = "TempFile.txt"
*
* // Set up the temp file needed by the test
* val writer = new FileWriter(FileName)
* try {
* writer.write("Hello, test!")
* }
* finally {
* writer.close()
* }
*
* // Create the reader needed by the test
* val reader = new FileReader(FileName)
*
* try {
* // Run the test using the temp file
* test(reader)
* }
* finally {
* // Close and delete the temp file
* reader.close()
* val file = new File(FileName)
* file.delete()
* }
* }
*
* feature("Reading and writing files") {
*
* scenario("reading from a temp file") { reader =>
* var builder = new StringBuilder
* var c = reader.read()
* while (c != -1) {
* builder.append(c.toChar)
* c = reader.read()
* }
* assert(builder.toString === "Hello, test!")
* }
*
* scenario("reading first char of a temp file") { reader =>
* assert(reader.read() === 'H')
* }
*
* scenario("no fixture is passed") { () =>
* assert(1 + 1 === 2)
* }
* }
* }
*
*
*
* It is worth noting that the only difference in the test code between the mutable
* BeforeAndAfterEach approach shown here and the immutable FixtureFeatureSpec
* approach shown previously is that two of the FixtureFeatureSpec's test functions take a FileReader as
* a parameter via the "reader =>" at the beginning of the function. Otherwise the test code is identical.
* One benefit of the explicit parameter is that, as demonstrated
* by the "no fixture passed" scenario, a FixtureFeatureSpec
* test need not take the fixture. So you can have some tests that take a fixture, and others that don't.
* In this case, the FixtureFeatureSpec provides documentation indicating which
* tests use the fixture and which don't, whereas the BeforeAndAfterEach approach does not.
* (If you have want to combine tests that take different fixture types in the same FeatureSpec, you can
* use MultipleFixtureFeatureSpec.)
*
*
*
* If you want to execute code before and after all tests (and nested suites) in a suite, such
* as you could do with @BeforeClass and @AfterClass
* annotations in JUnit 4, you can use the beforeAll and afterAll
* methods of BeforeAndAfterAll. See the documentation for BeforeAndAfterAll for
* an example.
*
*
* Shared scenarios
*
*
* Sometimes you may want to run the same test code on different fixture objects. In other words, you may want to write tests that are "shared"
* by different fixture objects.
* To accomplish this in a FeatureSpec, you first place shared tests (i.e., shared scenarios) in
* behavior functions. These behavior functions will be
* invoked during the construction phase of any FeatureSpec that uses them, so that the scenarios they contain will
* be registered as scenarios in that FeatureSpec.
* For example, given this stack class:
*
*
*
* import scala.collection.mutable.ListBuffer
*
* class Stack[T] {
*
* val MAX = 10
* private var buf = new ListBuffer[T]
*
* def push(o: T) {
* if (!full)
* o +: buf
* else
* throw new IllegalStateException("can't push onto a full stack")
* }
*
* def pop(): T = {
* if (!empty)
* buf.remove(0)
* else
* throw new IllegalStateException("can't pop an empty stack")
* }
*
* def peek: T = {
* if (!empty)
* buf(0)
* else
* throw new IllegalStateException("can't pop an empty stack")
* }
*
* def full: Boolean = buf.size == MAX
* def empty: Boolean = buf.size == 0
* def size = buf.size
*
* override def toString = buf.mkString("Stack(", ", ", ")")
* }
*
*
*
* You may want to test the Stack class in different states: empty, full, with one item, with one item less than capacity,
* etc. You may find you have several scenarios that make sense any time the stack is non-empty. Thus you'd ideally want to run
* those same scenarios for three stack fixture objects: a full stack, a stack with a one item, and a stack with one item less than
* capacity. With shared tests, you can factor these scenarios out into a behavior function, into which you pass the
* stack fixture to use when running the tests. So in your FeatureSpec for stack, you'd invoke the
* behavior function three times, passing in each of the three stack fixtures so that the shared scenarios are run for all three fixtures.
*
*
*
* You can define a behavior function that encapsulates these shared scenarios inside the FeatureSpec that uses them. If they are shared
* between different FeatureSpecs, however, you could also define them in a separate trait that is mixed into
* each FeatureSpec that uses them.
* For example, here the nonEmptyStack behavior function (in this case, a
* behavior method) is defined in a trait along with another
* method containing shared scenarios for non-full stacks:
*
*
*
* import org.scalatest.FeatureSpec
* import org.scalatest.GivenWhenThen
* import org.scalatestexamples.helpers.Stack
*
* trait FeatureSpecStackBehaviors { this: FeatureSpec with GivenWhenThen =>
*
* def nonEmptyStack(createNonEmptyStack: => Stack[Int], lastItemAdded: Int) {
*
* scenario("empty is invoked on this non-empty stack: " + createNonEmptyStack.toString) {
*
* given("a non-empty stack")
* val stack = createNonEmptyStack
*
* when("empty is invoked on the stack")
* then("empty returns false")
* assert(!stack.empty)
* }
*
* scenario("peek is invoked on this non-empty stack: " + createNonEmptyStack.toString) {
*
* given("a non-empty stack")
* val stack = createNonEmptyStack
* val size = stack.size
*
* when("peek is invoked on the stack")
* then("peek returns the last item added")
* assert(stack.peek === lastItemAdded)
*
* and("the size of the stack is the same as before")
* assert(stack.size === size)
* }
*
* scenario("pop is invoked on this non-empty stack: " + createNonEmptyStack.toString) {
*
* given("a non-empty stack")
* val stack = createNonEmptyStack
* val size = stack.size
*
* when("pop is invoked on the stack")
* then("pop returns the last item added")
* assert(stack.pop === lastItemAdded)
*
* and("the size of the stack one less than before")
* assert(stack.size === size - 1)
* }
* }
*
* def nonFullStack(createNonFullStack: => Stack[Int]) {
*
* scenario("full is invoked on this non-full stack: " + createNonFullStack.toString) {
*
* given("a non-full stack")
* val stack = createNonFullStack
*
* when("full is invoked on the stack")
* then("full returns false")
* assert(!stack.full)
* }
*
* scenario("push is invoked on this non-full stack: " + createNonFullStack.toString) {
*
* given("a non-full stack")
* val stack = createNonFullStack
* val size = stack.size
*
* when("push is invoked on the stack")
* stack.push(7)
*
* then("the size of the stack is one greater than before")
* assert(stack.size === size + 1)
*
* and("the top of the stack contains the pushed value")
* assert(stack.peek === 7)
* }
* }
* }
*
*
*
* Given these behavior functions, you could invoke them directly, but FeatureSpec offers a DSL for the purpose,
* which looks like this:
*
*
*
* scenariosFor(nonEmptyStack(stackWithOneItem, lastValuePushed))
* scenariosFor(nonFullStack(stackWithOneItem))
*
*
*
* If you prefer to use an imperative style to change fixtures, for example by mixing in BeforeAndAfterEach and
* reassigning a stack var in beforeEach, you could write your behavior functions
* in the context of that var, which means you wouldn't need to pass in the stack fixture because it would be
* in scope already inside the behavior function. In that case, your code would look like this:
*
*
*
* scenariosFor(nonEmptyStack) // assuming lastValuePushed is also in scope inside nonEmptyStack
* scenariosFor(nonFullStack)
*
*
*
* The recommended style, however, is the functional, pass-all-the-needed-values-in style. Here's an example:
*
*
*
* import org.scalatest.FeatureSpec
* import org.scalatest.GivenWhenThen
* import org.scalatestexamples.helpers.Stack
*
* class StackFeatureSpec extends FeatureSpec with GivenWhenThen with FeatureSpecStackBehaviors {
*
* // Stack fixture creation methods
* def emptyStack = new Stack[Int]
*
* def fullStack = {
* val stack = new Stack[Int]
* for (i <- 0 until stack.MAX)
* stack.push(i)
* stack
* }
*
* def stackWithOneItem = {
* val stack = new Stack[Int]
* stack.push(9)
* stack
* }
*
* def stackWithOneItemLessThanCapacity = {
* val stack = new Stack[Int]
* for (i <- 1 to 9)
* stack.push(i)
* stack
* }
*
* val lastValuePushed = 9
*
* feature("A Stack is pushed and popped") {
*
* scenario("empty is invoked on an empty stack") {
*
* given("an empty stack")
* val stack = emptyStack
*
* when("empty is invoked on the stack")
* then("empty returns true")
* assert(stack.empty)
* }
*
* scenario("peek is invoked on an empty stack") {
*
* given("an empty stack")
* val stack = emptyStack
*
* when("peek is invoked on the stack")
* then("peek throws IllegalStateException")
* intercept[IllegalStateException] {
* stack.peek
* }
* }
*
* scenario("pop is invoked on an empty stack") {
*
* given("an empty stack")
* val stack = emptyStack
*
* when("pop is invoked on the stack")
* then("pop throws IllegalStateException")
* intercept[IllegalStateException] {
* emptyStack.pop
* }
* }
*
* scenariosFor(nonEmptyStack(stackWithOneItem, lastValuePushed))
* scenariosFor(nonFullStack(stackWithOneItem))
*
* scenariosFor(nonEmptyStack(stackWithOneItemLessThanCapacity, lastValuePushed))
* scenariosFor(nonFullStack(stackWithOneItemLessThanCapacity))
*
* scenario("full is invoked on a full stack") {
*
* given("an full stack")
* val stack = fullStack
*
* when("full is invoked on the stack")
* then("full returns true")
* assert(stack.full)
* }
*
* scenariosFor(nonEmptyStack(fullStack, lastValuePushed))
*
* scenario("push is invoked on a full stack") {
*
* given("an full stack")
* val stack = fullStack
*
* when("push is invoked on the stack")
* then("push throws IllegalStateException")
* intercept[IllegalStateException] {
* stack.push(10)
* }
* }
* }
* }
*
*
*
* If you load these classes into the Scala interpreter (with scalatest's JAR file on the class path), and execute it,
* you'll see:
*
*
*
* scala> (new StackFeatureSpec).execute()
* Feature: A Stack is pushed and popped
* Scenario: empty is invoked on an empty stack
* Given an empty stack
* When empty is invoked on the stack
* Then empty returns true
* Scenario: peek is invoked on an empty stack
* Given an empty stack
* When peek is invoked on the stack
* Then peek throws IllegalStateException
* Scenario: pop is invoked on an empty stack
* Given an empty stack
* When pop is invoked on the stack
* Then pop throws IllegalStateException
* Scenario: empty is invoked on this non-empty stack: Stack(9)
* Given a non-empty stack
* When empty is invoked on the stack
* Then empty returns false
* Scenario: peek is invoked on this non-empty stack: Stack(9)
* Given a non-empty stack
* When peek is invoked on the stack
* Then peek returns the last item added
* And the size of the stack is the same as before
* Scenario: pop is invoked on this non-empty stack: Stack(9)
* Given a non-empty stack
* When pop is invoked on the stack
* Then pop returns the last item added
* And the size of the stack one less than before
* Scenario: full is invoked on this non-full stack: Stack(9)
* Given a non-full stack
* When full is invoked on the stack
* Then full returns false
* Scenario: push is invoked on this non-full stack: Stack(9)
* Given a non-full stack
* When push is invoked on the stack
* Then the size of the stack is one greater than before
* And the top of the stack contains the pushed value
* Scenario: empty is invoked on this non-empty stack: Stack(9, 8, 7, 6, 5, 4, 3, 2, 1)
* Given a non-empty stack
* When empty is invoked on the stack
* Then empty returns false
* Scenario: peek is invoked on this non-empty stack: Stack(9, 8, 7, 6, 5, 4, 3, 2, 1)
* Given a non-empty stack
* When peek is invoked on the stack
* Then peek returns the last item added
* And the size of the stack is the same as before
* Scenario: pop is invoked on this non-empty stack: Stack(9, 8, 7, 6, 5, 4, 3, 2, 1)
* Given a non-empty stack
* When pop is invoked on the stack
* Then pop returns the last item added
* And the size of the stack one less than before
* Scenario: full is invoked on this non-full stack: Stack(9, 8, 7, 6, 5, 4, 3, 2, 1)
* Given a non-full stack
* When full is invoked on the stack
* Then full returns false
* Scenario: push is invoked on this non-full stack: Stack(9, 8, 7, 6, 5, 4, 3, 2, 1)
* Given a non-full stack
* When push is invoked on the stack
* Then the size of the stack is one greater than before
* And the top of the stack contains the pushed value
* Scenario: full is invoked on a full stack
* Given an full stack
* When full is invoked on the stack
* Then full returns true
* Scenario: empty is invoked on this non-empty stack: Stack(9, 8, 7, 6, 5, 4, 3, 2, 1, 0)
* Given a non-empty stack
* When empty is invoked on the stack
* Then empty returns false
* Scenario: peek is invoked on this non-empty stack: Stack(9, 8, 7, 6, 5, 4, 3, 2, 1, 0)
* Given a non-empty stack
* When peek is invoked on the stack
* Then peek returns the last item added
* And the size of the stack is the same as before
* Scenario: pop is invoked on this non-empty stack: Stack(9, 8, 7, 6, 5, 4, 3, 2, 1, 0)
* Given a non-empty stack
* When pop is invoked on the stack
* Then pop returns the last item added
* And the size of the stack one less than before
* Scenario: push is invoked on a full stack
* Given an full stack
* When push is invoked on the stack
* Then push throws IllegalStateException
*
*
*
* One thing to keep in mind when using shared tests is that in ScalaTest, each test in a suite must have a unique name.
* If you register the same tests repeatedly in the same suite, one problem you may encounter is an exception at runtime
* complaining that multiple tests are being registered with the same test name.
* In a FeatureSpec there is no nesting construct analogous to Spec's describe clause.
* Therefore, you need to do a bit of
* extra work to ensure that the test names are unique. If a duplicate test name problem shows up in a
* FeatureSpec, you'll need to pass in a prefix or suffix string to add to each test name. You can pass this string
* the same way you pass any other data needed by the shared tests, or just call toString on the shared fixture object.
* This is the approach taken by the previous FeatureSpecStackBehaviors example.
*
*
*
* Given this FeatureSpecStackBehaviors trait, calling it with the stackWithOneItem fixture, like this:
*
*
*
* scenariosFor(nonEmptyStack(stackWithOneItem, lastValuePushed))
*
*
*
* yields test names:
*
*
*
* empty is invoked on this non-empty stack: Stack(9)peek is invoked on this non-empty stack: Stack(9)pop is invoked on this non-empty stack: Stack(9)
*
*
* Whereas calling it with the stackWithOneItemLessThanCapacity fixture, like this:
*
*
*
* scenariosFor(nonEmptyStack(stackWithOneItemLessThanCapacity, lastValuePushed))
*
*
*
* yields different test names:
*
*
*
* empty is invoked on this non-empty stack: Stack(9, 8, 7, 6, 5, 4, 3, 2, 1)peek is invoked on this non-empty stack: Stack(9, 8, 7, 6, 5, 4, 3, 2, 1)pop is invoked on this non-empty stack: Stack(9, 8, 7, 6, 5, 4, 3, 2, 1)
*
* Tagging tests
*
*
* A FeatureSpec's tests may be classified into groups by tagging them with string names.
* As with any suite, when executing a FeatureSpec, groups of tests can
* optionally be included and/or excluded. To tag a FeatureSpec's tests,
* you pass objects that extend abstract class org.scalatest.Tag to methods
* that register tests, test and ignore. Class Tag takes one parameter, a string name. If you have
* created Java annotation interfaces for use as group names in direct subclasses of org.scalatest.Suite,
* then you will probably want to use group names on your FeatureSpecs that match. To do so, simply
* pass the fully qualified names of the Java interfaces to the Tag constructor. For example, if you've
* defined Java annotation interfaces with fully qualified names, com.mycompany.groups.SlowTest and
* com.mycompany.groups.DbTest, then you could
* create matching groups for FeatureSpecs like this:
*
*
*
* import org.scalatest.Tag
*
* object SlowTest extends Tag("com.mycompany.groups.SlowTest")
* object DbTest extends Tag("com.mycompany.groups.DbTest")
*
*
*
* Given these definitions, you could place FeatureSpec tests into groups like this:
*
*
*
* import org.scalatest.FeatureSpec
*
* class ArithmeticFeatureSpec extends FeatureSpec {
*
* // Sharing fixture objects via instance variables
* val shared = 5
*
* feature("Integer arithmetic") {
*
* scenario("addition", SlowTest) {
* val sum = 2 + 3
* assert(sum === shared)
* }
*
* scenario("subtraction", SlowTest, DbTest) {
* val diff = 7 - 2
* assert(diff === shared)
* }
* }
* }
*
*
*
* This code marks both tests, "addition" and "subtraction," with the com.mycompany.groups.SlowTest tag,
* and test "subtraction" with the com.mycompany.groups.DbTest tag.
*
*
*
* The primary run method takes a Filter, whose constructor takes an optional
* Set[String]s called tagsToInclude and a Set[String] called
* tagsToExclude. If tagsToInclude is None, all tests will be run
* except those those belonging to tags listed in the
* tagsToExclude Set. If tagsToInclude is defined, only tests
* belonging to tags mentioned in the tagsToInclude set, and not mentioned in tagsToExclude,
* will be run.
*
*
* Ignored tests
*
*
* To support the common use case of “temporarily” disabling a test, with the
* good intention of resurrecting the test at a later time, FeatureSpec provides registration
* methods that start with ignore instead of scenario. For example, to temporarily
* disable the test named addition, just change “scenario” into “ignore,” like this:
*
*
*
* import org.scalatest.FeatureSpec
*
* class ArithmeticFeatureSpec extends FeatureSpec {
*
* // Sharing fixture objects via instance variables
* val shared = 5
*
* feature("Integer arithmetic") {
*
* ignore("addition") {
* val sum = 2 + 3
* assert(sum === shared)
* }
*
* scenario("subtraction") {
* val diff = 7 - 2
* assert(diff === shared)
* }
* }
* }
*
*
*
* If you run this version of ArithmeticFeatureSpec with:
*
*
*
* scala> (new ArithmeticFeatureSpec).execute()
*
*
*
* It will run only subtraction and report that addition was ignored:
*
*
*
* Feature: Integer arithmetic
* Scenario: addition !!! IGNORED !!!
* Scenario: subtraction
*
*
* Informers
*
*
* One of the parameters to the primary run method is a Reporter, which
* will collect and report information about the running suite of tests.
* Information about suites and tests that were run, whether tests succeeded or failed,
* and tests that were ignored will be passed to the Reporter as the suite runs.
* Most often the reporting done by default by FeatureSpec's methods will be sufficient, but
* occasionally you may wish to provide custom information to the Reporter from a test.
* For this purpose, an Informer that will forward information to the current Reporter
* is provided via the info parameterless method.
* You can pass the extra information to the Informer via its apply method.
* The Informer will then pass the information to the Reporter via an InfoProvided event.
* Here's an example:
*
*
*
* import org.scalatest.FeatureSpec
*
* class ArithmeticFeatureSpec extends FeatureSpec {
*
* feature("Integer arithmetic") {
*
* scenario("addition") {
* val sum = 2 + 3
* assert(sum === 5)
* info("Addition seems to work")
* }
*
* scenario("subtraction") {
* val diff = 7 - 2
* assert(diff === 5)
* }
* }
* }
*
*
* If you run this ArithmeticFeatureSpec from the interpreter, you will see the following message
* included in the printed report:
*
*
* Feature: Integer arithmetic
* Scenario: addition
* Addition seems to work
*
*
*
* One use case for the Informer is to pass more information about a scenario to the reporter. For example,
* the GivenWhenThen trait provides methods that use the implicit info provided by FeatureSpec
* to pass such information to the reporter. Here's an example:
*
*
*
* import org.scalatest.FeatureSpec
* import org.scalatest.GivenWhenThen
*
* class ArithmeticSpec extends FeatureSpec with GivenWhenThen {
*
* feature("Integer arithmetic") {
*
* scenario("addition") {
*
* given("two integers")
* val x = 2
* val y = 3
*
* when("they are added")
* val sum = x + y
*
* then("the result is the sum of the two numbers")
* assert(sum === 5)
* }
*
* scenario("subtraction") {
*
* given("two integers")
* val x = 7
* val y = 2
*
* when("one is subtracted from the other")
* val diff = x - y
*
* then("the result is the difference of the two numbers")
* assert(diff === 5)
* }
* }
* }
*
*
*
* If you run this FeatureSpec from the interpreter, you will see the following messages
* included in the printed report:
*
*
*
* scala> (new ArithmeticFeatureSpec).execute()
* Feature: Integer arithmetic
* Scenario: addition
* Given two integers
* When they are added
* Then the result is the sum of the two numbers
* Scenario: subtraction
* Given two integers
* When one is subtracted from the other
* Then the result is the difference of the two numbers
*
*
* Pending tests
*
*
* A pending test is one that has been given a name but is not yet implemented. The purpose of
* pending tests is to facilitate a style of testing in which documentation of behavior is sketched
* out before tests are written to verify that behavior (and often, the before the behavior of
* the system being tested is itself implemented). Such sketches form a kind of specification of
* what tests and functionality to implement later.
*
*
*
* To support this style of testing, a test can be given a name that specifies one
* bit of behavior required by the system being tested. The test can also include some code that
* sends more information about the behavior to the reporter when the tests run. At the end of the test,
* it can call method pending, which will cause it to complete abruptly with TestPendingException.
* Because tests in ScalaTest can be designated as pending with TestPendingException, both the test name and any information
* sent to the reporter when running the test can appear in the report of a test run. (In other words,
* the code of a pending test is executed just like any other test.) However, because the test completes abruptly
* with TestPendingException, the test will be reported as pending, to indicate
* the actual test, and possibly the functionality, has not yet been implemented.
* You can mark tests as pending in a FeatureSpec like this:
*
*
*
* import org.scalatest.FeatureSpec
*
* class ArithmeticFeatureSpec extends FeatureSpec {
*
* // Sharing fixture objects via instance variables
* val shared = 5
*
* feature("Integer arithmetic") {
*
* scenario("addition") {
* val sum = 2 + 3
* assert(sum === shared)
* }
*
* scenario("subtraction") (pending)
* }
* }
*
*
*
* (Note: "(pending)" is the body of the test. Thus the test contains just one statement, an invocation
* of the pending method, which throws TestPendingException.)
* If you run this version of ArithmeticFeatureSpec with:
*
*
*
* scala> (new ArithmeticFeatureSpec).execute()
*
*
*
* It will run both tests, but report that subtraction is pending. You'll see:
*
*
*
* Feature: Integer arithmetic
* Scenario: addition
* Scenario: subtraction (pending)
*
*
*
* One difference between an ignored test and a pending one is that an ignored test is intended to be used during a
* significant refactorings of the code under test, when tests break and you don't want to spend the time to fix
* all of them immediately. You can mark some of those broken tests as ignored temporarily, so that you can focus the red
* bar on just failing tests you actually want to fix immediately. Later you can go back and fix the ignored tests.
* In other words, by ignoring some failing tests temporarily, you can more easily notice failed tests that you actually
* want to fix. By contrast, a pending test is intended to be used before a test and/or the code under test is written.
* Pending indicates you've decided to write a test for a bit of behavior, but either you haven't written the test yet, or
* have only written part of it, or perhaps you've written the test but don't want to implement the behavior it tests
* until after you've implemented a different bit of behavior you realized you need first. Thus ignored tests are designed
* to facilitate refactoring of existing code whereas pending tests are designed to facilitate the creation of new code.
*
*
*
* One other difference between ignored and pending tests is that ignored tests are implemented as a test tag that is
* excluded by default. Thus an ignored test is never executed. By contrast, a pending test is implemented as a
* test that throws TestPendingException (which is what calling the pending method does). Thus
* the body of pending tests are executed up until they throw TestPendingException. The reason for this difference
* is that it enables your unfinished test to send InfoProvided messages to the reporter before it completes
* abruptly with TestPendingException, as shown in the previous example on Informers
* that used the GivenWhenThen trait. For example, the following snippet in a FeatureSpec:
*
*
*
* feature("Integer arithmetic") {
*
* scenario("addition") {
* given("two integers")
* when("they are added")
* then("the result is the sum of the two numbers")
* pending
* }
* // ...
*
*
*
* Would yield the following output when run in the interpreter:
*
*
*
* Feature: Integer arithmetic
* Scenario: addition (pending)
* Given two integers
* When they are added
* Then the result is the sum of the two numbers
*
*
* @author Bill Venners
*/
trait FeatureSpec extends Suite { thisSuite =>
private val IgnoreTagName = "org.scalatest.Ignore"
private class Bundle private(
val trunk: Trunk,
val currentBranch: Branch,
val tagsMap: Map[String, Set[String]],
// All tests, in reverse order of registration
val testsList: List[TestLeaf],
// Used to detect at runtime that they've stuck a describe or an it inside an it,
// which should result in a TestRegistrationClosedException
val registrationClosed: Boolean
) {
def unpack = (trunk, currentBranch, tagsMap, testsList, registrationClosed)
}
private object Bundle {
def apply(
trunk: Trunk,
currentBranch: Branch,
tagsMap: Map[String, Set[String]],
testsList: List[TestLeaf],
registrationClosed: Boolean
): Bundle =
new Bundle(trunk, currentBranch, tagsMap, testsList, registrationClosed)
def initialize(
trunk: Trunk,
tagsMap: Map[String, Set[String]],
testsList: List[TestLeaf],
registrationClosed: Boolean
): Bundle =
new Bundle(trunk, trunk, tagsMap, testsList, registrationClosed)
}
private val atomic =
new AtomicReference[Bundle](
Bundle.initialize(new Trunk, Map(), List[TestLeaf](), false)
)
private def updateAtomic(oldBundle: Bundle, newBundle: Bundle) {
val shouldBeOldBundle = atomic.getAndSet(newBundle)
if (!(shouldBeOldBundle eq oldBundle))
throw new ConcurrentModificationException(Resources("concurrentFeatureSpecBundleMod"))
}
private def registerTest(specText: String, f: => Unit) = {
val oldBundle = atomic.get
var (trunk, currentBranch, tagsMap, testsList, registrationClosed) = oldBundle.unpack
val testName = getTestName(specText, currentBranch)
if (testsList.exists(_.testName == testName)) {
throw new DuplicateTestNameException(testName, getStackDepth("Spec.scala", "it"))
}
val testShortName = specText
val test = TestLeaf(currentBranch, testName, specText, f _)
currentBranch.subNodes ::= test
testsList ::= test
updateAtomic(oldBundle, Bundle(trunk, currentBranch, tagsMap, testsList, registrationClosed))
testName
}
private class RegistrationInformer extends Informer {
def apply(message: String) {
if (message == null)
throw new NullPointerException
val oldBundle = atomic.get
var (trunk, currentBranch, tagsMap, testsList, registrationClosed) = oldBundle.unpack
currentBranch.subNodes ::= InfoLeaf(currentBranch, message)
updateAtomic(oldBundle, Bundle(trunk, currentBranch, tagsMap, testsList, registrationClosed))
}
}
// The informer will be a registration informer until run is called for the first time. (This
// is the registration phase of a FeatureSpec's lifecycle.)
private final val atomicInformer = new AtomicReference[Informer](new RegistrationInformer)
/**
* Returns an Informer that during test execution will forward strings (and other objects) passed to its
* apply method to the current reporter. If invoked in a constructor, it
* will register the passed string for forwarding later during test execution. If invoked while this
* FeatureSpec is being executed, such as from inside a test function, it will forward the information to
* the current reporter immediately. If invoked at any other time, it will
* throw an exception. This method can be called safely by any thread.
*/
implicit protected def info: Informer = atomicInformer.get
private val zombieInformer =
new Informer {
private val complaint = Resources("cantCallInfoNow", "FeatureSpec")
def apply(message: String) {
if (message == null)
throw new NullPointerException
throw new IllegalStateException(complaint)
}
}
/**
* Register a test with the given spec text, optional tags, and test function value that takes no arguments.
* An invocation of this method is called an “example.”
*
* This method will register the test for later execution via an invocation of one of the execute
* methods. The name of the test will be a concatenation of the text of all surrounding describers,
* from outside in, and the passed spec text, with one space placed between each item. (See the documenation
* for testNames for an example.) The resulting test name must not have been registered previously on
* this Spec instance.
*
* @param specText the specification text, which will be combined with the descText of any surrounding describers
* to form the test name
* @param testTags the optional list of tags for this test
* @param testFun the test function
* @throws DuplicateTestNameException if a test with the same name has been registered previously
* @throws TestRegistrationClosedException if invoked after run has been invoked on this suite
* @throws NullPointerException if specText or any passed test tag is null
*/
protected def scenario(specText: String, testTags: Tag*)(testFun: => Unit) {
if (atomic.get.registrationClosed)
throw new TestRegistrationClosedException(Resources("scenarioCannotAppearInsideAnotherScenario"), getStackDepth("FeatureSpec.scala", "scenario"))
if (specText == null)
throw new NullPointerException("specText was null")
if (testTags.exists(_ == null))
throw new NullPointerException("a test tag was null")
val testName = registerTest(specText, testFun)
val oldBundle = atomic.get
var (trunk, currentBranch, tagsMap, testsList, registrationClosed2) = oldBundle.unpack
val tagNames = Set[String]() ++ testTags.map(_.name)
if (!tagNames.isEmpty)
tagsMap += (testName -> tagNames)
updateAtomic(oldBundle, Bundle(trunk, currentBranch, tagsMap, testsList, registrationClosed2))
}
/**
* Register a test with the given spec text and test function value that takes no arguments.
*
* This method will register the test for later execution via an invocation of one of the execute
* methods. The name of the test will be a concatenation of the text of all surrounding describers,
* from outside in, and the passed spec text, with one space placed between each item. (See the documenation
* for testNames for an example.) The resulting test name must not have been registered previously on
* this Spec instance.
*
* @param specText the specification text, which will be combined with the descText of any surrounding describers
* to form the test name
* @param testFun the test function
* @throws DuplicateTestNameException if a test with the same name has been registered previously
* @throws TestRegistrationClosedException if invoked after run has been invoked on this suite
* @throws NullPointerException if specText or any passed test tag is null
*/
protected def scenario(specText: String)(testFun: => Unit) {
if (atomic.get.registrationClosed)
throw new TestRegistrationClosedException(Resources("scenarioCannotAppearInsideAnotherScenario"), getStackDepth("FeatureSpec.scala", "scenario"))
scenario(specText, Array[Tag](): _*)(testFun)
}
/**
* Register a test to ignore, which has the given spec text, optional tags, and test function value that takes no arguments.
* This method will register the test for later ignoring via an invocation of one of the execute
* methods. This method exists to make it easy to ignore an existing test by changing the call to it
* to ignore without deleting or commenting out the actual test code. The test will not be executed, but a
* report will be sent that indicates the test was ignored. The name of the test will be a concatenation of the text of all surrounding describers,
* from outside in, and the passed spec text, with one space placed between each item. (See the documenation
* for testNames for an example.) The resulting test name must not have been registered previously on
* this Spec instance.
*
* @param specText the specification text, which will be combined with the descText of any surrounding describers
* to form the test name
* @param testTags the optional list of tags for this test
* @param testFun the test function
* @throws DuplicateTestNameException if a test with the same name has been registered previously
* @throws TestRegistrationClosedException if invoked after run has been invoked on this suite
* @throws NullPointerException if specText or any passed test tag is null
*/
protected def ignore(specText: String, testTags: Tag*)(testFun: => Unit) {
if (atomic.get.registrationClosed)
throw new TestRegistrationClosedException(Resources("ignoreCannotAppearInsideAScenario"), getStackDepth("FeatureSpec.scala", "ignore"))
if (specText == null)
throw new NullPointerException("specText was null")
if (testTags.exists(_ == null))
throw new NullPointerException("a test tag was null")
val testName = registerTest(specText, testFun)
val tagNames = Set[String]() ++ testTags.map(_.name)
val oldBundle = atomic.get
var (trunk, currentBranch, tagsMap, testsList, registrationClosed) = oldBundle.unpack
tagsMap += (testName -> (tagNames + IgnoreTagName))
updateAtomic(oldBundle, Bundle(trunk, currentBranch, tagsMap, testsList, registrationClosed))
}
/**
* Register a test to ignore, which has the given spec text and test function value that takes no arguments.
* This method will register the test for later ignoring via an invocation of one of the execute
* methods. This method exists to make it easy to ignore an existing test by changing the call to it
* to ignore without deleting or commenting out the actual test code. The test will not be executed, but a
* report will be sent that indicates the test was ignored. The name of the test will be a concatenation of the text of all surrounding describers,
* from outside in, and the passed spec text, with one space placed between each item. (See the documenation
* for testNames for an example.) The resulting test name must not have been registered previously on
* this Spec instance.
*
* @param specText the specification text, which will be combined with the descText of any surrounding describers
* to form the test name
* @param testFun the test function
* @throws DuplicateTestNameException if a test with the same name has been registered previously
* @throws TestRegistrationClosedException if invoked after run has been invoked on this suite
* @throws NullPointerException if specText or any passed test tag is null
*/
protected def ignore(specText: String)(testFun: => Unit) {
if (atomic.get.registrationClosed)
throw new TestRegistrationClosedException(Resources("ignoreCannotAppearInsideAScenario"), getStackDepth("FeatureSpec.scala", "ignore"))
ignore(specText, Array[Tag](): _*)(testFun)
}
/**
* Describe a “subject” being specified and tested by the passed function value. The
* passed function value may contain more describers (defined with describe) and/or tests
* (defined with it). This trait's implementation of this method will register the
* description string and immediately invoke the passed function.
*/
protected def feature(description: String)(f: => Unit) {
if (atomic.get.registrationClosed)
throw new TestRegistrationClosedException(Resources("featureCannotAppearInsideAScenario"), getStackDepth("FeatureSpec.scala", "feature"))
def createNewBranch() = {
val oldBundle = atomic.get
var (trunk, currentBranch, tagsMap, testsList, registrationClosed) = oldBundle.unpack
// features cannot nest
if (currentBranch != trunk)
throw new NotAllowedException(Resources("cantNestFeatureClauses"), getStackDepth("FeatureSpec.scala", "feature"))
val newBranch = DescriptionBranch(currentBranch, Resources("feature", description))
val oldBranch = currentBranch
currentBranch.subNodes ::= newBranch
currentBranch = newBranch
updateAtomic(oldBundle, Bundle(trunk, currentBranch, tagsMap, testsList, registrationClosed))
oldBranch
}
val oldBranch = createNewBranch()
f
val oldBundle = atomic.get
val (trunk, currentBranch, tagsMap, testsList, registrationClosed) = oldBundle.unpack
updateAtomic(oldBundle, Bundle(trunk, oldBranch, tagsMap, testsList, registrationClosed))
}
/**
* A Map whose keys are String tag names to which tests in this Spec belong, and values
* the Set of test names that belong to each tag. If this FeatureSpec contains no tags, this method returns an empty Map.
*
*
* This trait's implementation returns tags that were passed as strings contained in Tag objects passed to
* methods test and ignore.
*
*/
override def tags: Map[String, Set[String]] = atomic.get.tagsMap
private def runTestsInBranch(branch: Branch, reporter: Reporter, stopper: Stopper, filter: Filter, configMap: Map[String, Any], tracker: Tracker) {
val stopRequested = stopper
// Wrap any non-DispatchReporter, non-CatchReporter in a CatchReporter,
// so that exceptions are caught and transformed
// into error messages on the standard error stream.
val report = wrapReporterIfNecessary(reporter)
branch match {
case desc @ DescriptionBranch(_, descriptionName) =>
def sendInfoProvidedMessage() {
// Need to use the full name of the description, which includes all the descriptions it is nested inside
// Call getPrefix and pass in this Desc, to get the full name
val descriptionFullName = getPrefix(desc).trim
report(InfoProvided(tracker.nextOrdinal(), descriptionFullName, Some(NameInfo(thisSuite.suiteName, Some(thisSuite.getClass.getName), None)), None, None, Some(IndentedText(descriptionFullName, descriptionFullName, 0))))
}
sendInfoProvidedMessage()
case _ =>
}
branch.subNodes.reverse.foreach(
_ match {
case TestLeaf(_, tn, specText, _) =>
if (!stopRequested()) { // TODO: Seems odd to me to check for stop here but still fire infos
val (filterTest, ignoreTest) = filter(tn, tags)
if (!filterTest)
if (ignoreTest) {
val formattedSpecText = " " + Resources("scenario", specText)
report(TestIgnored(tracker.nextOrdinal(), thisSuite.suiteName, Some(thisSuite.getClass.getName), tn, Some(IndentedText(formattedSpecText, specText, 1))))
}
else
runTest(tn, report, stopRequested, configMap, tracker)
}
case InfoLeaf(_, message) =>
val formattedText = " " + message
report(InfoProvided(tracker.nextOrdinal(), message,
Some(NameInfo(thisSuite.suiteName, Some(thisSuite.getClass.getName), None)), None,
None, Some(IndentedText(formattedText, message, 1))))
case branch: Branch => runTestsInBranch(branch, reporter, stopRequested, filter, configMap, tracker)
}
)
}
/**
* Run a test. This trait's implementation runs the test registered with the name specified by
* testName. Each test's name is a concatenation of the text of all describers surrounding a test,
* from outside in, and the test's spec text, with one space placed between each item. (See the documenation
* for testNames for an example.)
*
* @param testName the name of one test to execute.
* @param reporter the Reporter to which results will be reported
* @param stopper the Stopper that will be consulted to determine whether to stop execution early.
* @param configMap a Map of properties that can be used by this Spec's executing tests.
* @throws NullPointerException if any of testName, reporter, stopper, or configMap
* is null.
*/
protected override def runTest(testName: String, reporter: Reporter, stopper: Stopper, configMap: Map[String, Any], tracker: Tracker) {
if (testName == null || reporter == null || stopper == null || configMap == null)
throw new NullPointerException
atomic.get.testsList.find(_.testName == testName) match {
case None => throw new IllegalArgumentException("Requested test doesn't exist: " + testName)
case Some(test) => {
val report = wrapReporterIfNecessary(reporter)
val scenarioSpecText = Resources("scenario", test.specText)
val formattedSpecText = " " + scenarioSpecText
// Create a Rerunner if the Spec has a no-arg constructor
val hasPublicNoArgConstructor = Suite.checkForPublicNoArgConstructor(getClass)
val rerunnable =
if (hasPublicNoArgConstructor)
Some(new TestRerunner(getClass.getName, testName))
else
None
val testStartTime = System.currentTimeMillis
// A TestStarting event won't normally show up in a specification-style output, but
// will show up in a test-style output.
report(TestStarting(tracker.nextOrdinal(), thisSuite.suiteName, Some(thisSuite.getClass.getName), test.testName, Some(MotionToSuppress), rerunnable))
val formatter = IndentedText(formattedSpecText, scenarioSpecText, 1)
val informerForThisTest =
new MessageRecordingInformer(NameInfo(thisSuite.suiteName, Some(thisSuite.getClass.getName), Some(testName))) {
def apply(message: String) {
if (message == null)
throw new NullPointerException
if (shouldRecord)
record(message)
else {
val formattedText = " " + message
report(InfoProvided(tracker.nextOrdinal(), message, nameInfoForCurrentThread, None, None, Some(IndentedText(formattedText, message, 2))))
}
}
}
val oldInformer = atomicInformer.getAndSet(informerForThisTest)
var testWasPending = false
var swapAndCompareSucceeded = false
try {
val theConfigMap = configMap
withFixture(
new NoArgTest {
def name = testName
def apply() { test.f() }
def configMap = theConfigMap
}
)
val duration = System.currentTimeMillis - testStartTime
report(TestSucceeded(tracker.nextOrdinal(), thisSuite.suiteName, Some(thisSuite.getClass.getName), test.testName, Some(duration), Some(formatter), rerunnable))
}
catch {
case _: TestPendingException =>
report(TestPending(tracker.nextOrdinal(), thisSuite.suiteName, Some(thisSuite.getClass.getName), test.testName, Some(formatter)))
testWasPending = true
case e if !anErrorThatShouldCauseAnAbort(e) =>
val duration = System.currentTimeMillis - testStartTime
handleFailedTest(e, false, test.testName, test.specText, formattedSpecText, rerunnable, report, tracker, duration)
case e => throw e
}
finally {
// send out any recorded messages
for (message <- informerForThisTest.recordedMessages) {
val formattedText = " " + message
report(InfoProvided(tracker.nextOrdinal(), message, informerForThisTest.nameInfoForCurrentThread, Some(testWasPending), None, Some(IndentedText(formattedText, message, 2))))
}
val shouldBeInformerForThisTest = atomicInformer.getAndSet(oldInformer)
swapAndCompareSucceeded = shouldBeInformerForThisTest eq informerForThisTest
}
if (!swapAndCompareSucceeded) // Do outside finally to workaround Scala compiler bug
throw new ConcurrentModificationException(Resources("concurrentInformerMod", thisSuite.getClass.getName))
}
}
}
private def handleFailedTest(throwable: Throwable, hasPublicNoArgConstructor: Boolean, testName: String,
specText: String, formattedSpecText: String, rerunnable: Option[Rerunner], report: Reporter, tracker: Tracker, duration: Long) {
val message =
if (throwable.getMessage != null) // [bv: this could be factored out into a helper method]
throwable.getMessage
else
throwable.toString
val formatter = IndentedText(formattedSpecText, specText, 1)
report(TestFailed(tracker.nextOrdinal(), message, thisSuite.suiteName, Some(thisSuite.getClass.getName), testName, Some(throwable), Some(duration), Some(formatter), rerunnable))
}
/**
* Run zero to many of this FeatureSpec's tests.
*
*
* This method takes a testName parameter that optionally specifies a test to invoke.
* If testName is Some, this trait's implementation of this method
* invokes runTest on this object, passing in:
*
*
*
* testName - the String value of the testName Option passed
* to this methodreporter - the Reporter passed to this method, or one that wraps and delegates to itstopper - the Stopper passed to this method, or one that wraps and delegates to itconfigMap - the configMap passed to this method, or one that wraps and delegates to it
*
*
* This method takes a Set of tag names that should be included (tagsToInclude), and a Set
* that should be excluded (tagsToExclude), when deciding which of this Suite's tests to execute.
* If tagsToInclude is empty, all tests will be executed
* except those those belonging to tags listed in the tagsToExclude Set. If tagsToInclude is non-empty, only tests
* belonging to tags mentioned in tagsToInclude, and not mentioned in tagsToExclude
* will be executed. However, if testName is Some, tagsToInclude and tagsToExclude are essentially ignored.
* Only if testName is None will tagsToInclude and tagsToExclude be consulted to
* determine which of the tests named in the testNames Set should be run. For more information on trait tags, see the main documentation for this trait.
*
*
*
* If testName is None, this trait's implementation of this method
* invokes testNames on this Suite to get a Set of names of tests to potentially execute.
* (A testNames value of None essentially acts as a wildcard that means all tests in
* this Suite that are selected by tagsToInclude and tagsToExclude should be executed.)
* For each test in the testName Set, in the order
* they appear in the iterator obtained by invoking the elements method on the Set, this trait's implementation
* of this method checks whether the test should be run based on the tagsToInclude and tagsToExclude Sets.
* If so, this implementation invokes runTest, passing in:
*
*
*
* testName - the String name of the test to run (which will be one of the names in the testNames Set)reporter - the Reporter passed to this method, or one that wraps and delegates to itstopper - the Stopper passed to this method, or one that wraps and delegates to itconfigMap - the configMap passed to this method, or one that wraps and delegates to it
*
* @param testName an optional name of one test to run. If None, all relevant tests should be run.
* I.e., None acts like a wildcard that means run all relevant tests in this Suite.
* @param reporter the Reporter to which results will be reported
* @param stopper the Stopper that will be consulted to determine whether to stop execution early.
* @param filter a Filter with which to filter tests based on their tags
* @param configMap a Map of key-value pairs that can be used by the executing Suite of tests.
* @param distributor an optional Distributor, into which to put nested Suites to be run
* by another entity, such as concurrently by a pool of threads. If None, nested Suites will be run sequentially.
* @param tracker a Tracker tracking Ordinals being fired by the current thread.
* @throws NullPointerException if any of the passed parameters is null.
* @throws IllegalArgumentException if testName is defined, but no test with the specified test name
* exists in this Suite
*/
protected override def runTests(testName: Option[String], reporter: Reporter, stopper: Stopper, filter: Filter,
configMap: Map[String, Any], distributor: Option[Distributor], tracker: Tracker) {
if (testName == null)
throw new NullPointerException("testName was null")
if (reporter == null)
throw new NullPointerException("reporter was null")
if (stopper == null)
throw new NullPointerException("stopper was null")
if (filter == null)
throw new NullPointerException("filter was null")
if (configMap == null)
throw new NullPointerException("configMap was null")
if (distributor == null)
throw new NullPointerException("distributor was null")
if (tracker == null)
throw new NullPointerException("tracker was null")
val stopRequested = stopper
testName match {
case None => runTestsInBranch(atomic.get.trunk, reporter, stopRequested, filter, configMap, tracker)
case Some(tn) => runTest(tn, reporter, stopRequested, configMap, tracker)
}
}
/**
* An immutable Set of test names. If this FeatureSpec contains no tests, this method returns an
* empty Set.
*
*
* This trait's implementation of this method will return a set that contains the names of all registered tests. The set's
* iterator will return those names in the order in which the tests were registered. Each test's name is composed
* of the concatenation of the text of each surrounding describer, in order from outside in, and the text of the
* example itself, with all components separated by a space. For example, consider this FeatureSpec:
*
*
*
* import org.scalatest.FeatureSpec
*
* class StackSpec extends FeatureSpec {
* feature("A Stack") {
* scenario("(when not empty) must allow me to pop") {}
* scenario("(when not full) must allow me to push") {}
* }
* }
*
*
*
* Invoking testNames on this Spec will yield a set that contains the following
* two test name strings:
*
*
*
* "A Stack (when not empty) must allow me to pop"
* "A Stack (when not full) must allow me to push"
*
*/
override def testNames: Set[String] = ListSet(atomic.get.testsList.map(_.testName): _*)
override def run(testName: Option[String], reporter: Reporter, stopper: Stopper, filter: Filter,
configMap: Map[String, Any], distributor: Option[Distributor], tracker: Tracker) {
val stopRequested = stopper
// Set the flag that indicates registration is closed (because run has now been invoked),
// which will disallow any further invocations of "describe", it", or "ignore" with
// an RegistrationClosedException.
val oldBundle = atomic.get
var (trunk, currentBranch, tagsMap, testsList, registrationClosed) = oldBundle.unpack
if (!registrationClosed)
updateAtomic(oldBundle, Bundle(trunk, currentBranch, tagsMap, testsList, true))
val report = wrapReporterIfNecessary(reporter)
val informerForThisSuite =
new ConcurrentInformer(NameInfo(thisSuite.suiteName, Some(thisSuite.getClass.getName), None)) {
def apply(message: String) {
if (message == null)
throw new NullPointerException
report(InfoProvided(tracker.nextOrdinal(), message, nameInfoForCurrentThread))
}
}
atomicInformer.set(informerForThisSuite)
var swapAndCompareSucceeded = false
try {
super.run(testName, report, stopRequested, filter, configMap, distributor, tracker)
}
finally {
val shouldBeInformerForThisSuite = atomicInformer.getAndSet(zombieInformer)
swapAndCompareSucceeded = shouldBeInformerForThisSuite eq informerForThisSuite
}
if (!swapAndCompareSucceeded) // Do outside finally to workaround Scala compiler bug
throw new ConcurrentModificationException(Resources("concurrentInformerMod", thisSuite.getClass.getName))
}
/**
* Registers shared scenarios.
*
*
* This method enables the following syntax for shared scenarios in a FeatureSpec:
*
*
*
* scenariosFor(nonEmptyStack(lastValuePushed))
*
*
*
* This method just provides syntax sugar intended to make the intent of the code clearer.
* Because the parameter passed to it is
* type Unit, the expression will be evaluated before being passed, which
* is sufficient to register the shared scenarios. For examples of shared scenarios, see the
* Shared scenarios section in the main documentation for this trait.
*
*/
protected def scenariosFor(unit: Unit) {}
}