org.scalatest.AsyncTestSuite.scala Maven / Gradle / Ivy
/*
* Copyright 2001-2014 Artima, Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.scalatest
import scala.concurrent.Future
import scala.concurrent.ExecutionContext
import scala.language.implicitConversions // To convert Assertion to Future[Assertion]
import enablers.Futuristic
/*
* TODO: Fill in here and also add a lifecycle-methods to Suite, which is linked to
* from SuiteMixin.
*
*
* This trait provides a final override of withFixture(NoArgTest), declared in
* supertrait Suite, because the withFixture(NoArgTest) lifecycle
* method assumes synchronous testing. Here is its signature:
*
*
* * def withFixture(test: NoArgTest): Outcome
*
*
*
* The test function interface, NoArgTest, offers an apply method
* that also returns Outcome:
*
*
* * // In trait NoArgTest:
* def apply(): Outcome
*
*
*
* Because the result of a test is an Outcome, when the test function returns, the test body must have determined an outcome already. It
* will already be one of Succeeded, Failed, Canceled, or Pending. This is
* also true when withFixture(NoArgTest) returns: because the result type of withFixture(NoArgTest) is Outcome,
* the test has by definition already finished execution.
*
*
*/
/**
* The base trait of ScalaTest's asynchronous testing styles, which defines a
* withFixture lifecycle method that accepts as its parameter a test function
* that returns a FutureOutcome.
*
*
* The withFixture method add by this trait has the
* following signature and implementation:
*
*
* * def withFixture(test: NoArgAsyncTest): FutureOutcome = {
* test()
* }
*
*
*
* This trait enables testing of asynchronous code without blocking. Instead of returning
* Outcome like TestSuite's
* withFixture, this trait's withFixture method returns a
* FutureOutcome. Similarly, the apply method of test function interface,
* NoArgAsyncTest, returns FutureOutcome:
*
*
* * // In trait NoArgAsyncTest:
* def apply(): FutureOutcome
*
*
*
* The withFixture method supports async testing, because when the test function returns,
* the test body has not necessarily finished execution.
*
*
*
* The recommended way to ensure cleanup is performed after a test body finishes execution is
* to use a complete-lastly clause, syntax that is defined in trait
* CompleteLastly, which this trait extends.
* Using cleanup-lastly will ensure that cleanup will occur whether
* FutureOutcome-producing code completes abruptly by throwing an exception, or returns
* normally yielding a FutureOutcome. In the latter case,
* complete-lastly will
* register the cleanup code to execute asynchronously when the FutureOutcome completes.
*
*
*
* The withFixture method is designed to be stacked, and to enable this, you should always call the super implementation
* of withFixture, and let it invoke the test function rather than invoking the test function directly. In other words, instead of writing
* “test()”, you should write “super.withFixture(test)”. Thus, the recommended
* structure of a withFixture implementation that performs cleanup looks like this:
*
*
* * // Your implementation
* override def withFixture(test: NoArgAsyncTest) = {
* // Perform setup here
* complete {
* super.withFixture(test) // Invoke the test function
* } lastly {
* // Perform cleanup here
* }
* }
*
*
*
* If you have no cleanup to perform, you can write withFixture like this instead:
*
*
* * // Your implementation
* override def withFixture(test: NoArgAsyncTest) = {
* // Perform setup here
* super.withFixture(test) // Invoke the test function
* }
*
*
*
* The test function and withFixture method returns a
* FutureOutcome,
* a ScalaTest class that wraps a Scala Future[Outcome] and offers methods
* more specific to asynchronous test outcomes. In a Scala Future, any exception
* results in a scala.util.Failure. In a FutureOutcome, a
* thrown TestPendingException always results in a Pending,
* a thrown TestCanceledException always results in a Canceled,
* and any other exception, so long as it isn't suite-aborting, results in a
* Failed. This is true of the asynchronous test code itself that's represented by
* the FutureOutcome and any transformation or callback registered on the
* FutureOutcome in withFixture.
*
*
*
* If you want to perform an action only for certain outcomes, you'll need to
* register code performing that action on the FutureOutcome using
* one of FutureOutcome's callback registration methods:
*
*
* -
*
onSucceededThen- executed if theOutcomeis aSucceeded. *onFailedThen- executed if theOutcomeis aFailed. *onCanceledThen- executed if theOutcomeis aCanceled. *onPendingThen- executed if theOutcomeis aPending. *onOutcomeThen- executed on anyOutcome(i.e., no * suite-aborting exception is thrown). *onAbortedThen- executed if a suite-aborting exception is thrown. *onCompletedThen- executed whether the result is anOutcome* or a thrown suite-aborting exception. *
* For example, if you want to perform an action if a test fails, you'd register the
* callback using onFailedThen, like this:
*
* // Your implementation
* override def withFixture(test: NoArgAsyncTest) = {
*
* // Perform setup here
*
* val futureOutcome = super.withFixture(test) // Invoke the test function
*
* futureOutcome onFailedThen { ex =>
* // perform action that you want to occur
* // only if a test fails here
* }
* }
*
*
*
* Note that all callback registration methods, such as onFailedThen used in the
* previous example, return a new FutureOutcome that won't complete until the
* the original FutureOutcome and the callback has completed. If the callback
* throws an exception, the resulting FutureOutcome will represent that exception.
* For example, if a FutureOutcome results in Failed, but a callback
* registered on that FutureOutcome with onFailedThen throws TestPendingException, the
* result of the FutureOutcome returned by onFailedThen will
* be Pending.
*
* Lastly, if you want to change the outcome in some way in withFixture, you'll need to use
* the change method of FutureOutcome, like this:
*
* // Your implementation
* override def withFixture(test: NoArgAsyncTest) = {
*
* // Perform setup here
*
* val futureOutcome = super.withFixture(test) // Invoke the test function
*
* futureOutcome change { outcome =>
* // transform the outcome into a new outcome here
* }
* }
*
*/
trait AsyncTestSuite extends Suite with RecoverMethods with CompleteLastly { thisAsyncTestSuite =>
/**
* An implicit execution context used by async styles to transform Future[Assertion] values
* returned by tests into FutureOutcome values, and can be used within the async tests themselves,
* for example, when mapping assertions onto futures.
*/
private final val serialExecutionContext: ExecutionContext = new concurrent.SerialExecutionContext
implicit def executionContext: ExecutionContext = serialExecutionContext
private def anAsyncExceptionThatShouldCauseAnAbort(ex: Throwable): Boolean =
ex match {
// Not sure why a thrown OutOfMemoryError in our test is showing up nested inside
// an ExecutionException, but since it is, look inside.
case ee: java.util.concurrent.ExecutionException if ex.getCause != null => Suite.anExceptionThatShouldCauseAnAbort(ex.getCause)
case other => Suite.anExceptionThatShouldCauseAnAbort(other)
}
/**
* Transform the test outcome, `Registration` type to `AsyncOutcome`.
*
* @param testFun test function
* @return function that returns `AsyncOutcome`
*/
private[scalatest] def transformToOutcome(testFun: => Future[compatible.Assertion]): () => AsyncOutcome =
() => {
val futureSucceeded: Future[Succeeded.type] = testFun.map(_ => Succeeded)
InternalFutureOutcome(
futureSucceeded.recover {
case ex: exceptions.TestCanceledException => Canceled(ex)
case _: exceptions.TestPendingException => Pending
case tfe: exceptions.TestFailedException => Failed(tfe)
case ex: Throwable if !anAsyncExceptionThatShouldCauseAnAbort(ex) => Failed(ex)
}
)/* fills in executionContext here */
}
/**
* Implicitly converts an Assertion to a Future[Assertion].
*
*
* This implicit conversion is used to allow synchronous tests to be included along with
* asynchronous tests in an AsyncTestSuite. It will be
*
Assertion to convert
* @return a Future[Assertion] that has already completed successfully
* (containing the Succeeded singleton).
*/
implicit def convertAssertionToFutureAssertion(assertion: compatible.Assertion): Future[compatible.Assertion] = Future.successful(assertion)
protected[scalatest] def parallelAsyncTestExecution: Boolean = thisAsyncTestSuite.isInstanceOf[org.scalatest.ParallelTestExecution] ||
thisAsyncTestSuite.isInstanceOf[org.scalatest.RandomTestOrder]
// TODO: Document how exceptions are treated. I.e., that TestConceledException becomes Success(Canceled),
// TestPendingException becomes Success(Pending), non-test-fatal exceptions become Success(Failed), and
// test-fatal exceptions become Failure(ex)
/**
* A test function taking no arguments and returning a FutureOutcome.
*
*
* For more detail and examples, see the relevant section in the
* documentation for trait AsyncFlatSpec.
*
FutureOutcome.
*/
def apply(): FutureOutcome
}
/**
* Run the passed test function in the context of a fixture established by this method.
*
*
* This method should set up the fixture needed by the tests of the
* current suite, invoke the test function, and if needed, register a callback
* on the resulting FutureOutcome to perform any clean
* up needed after the test completes. Because the NoArgAsyncTest function
* passed to this method takes no parameters, preparing the fixture will require
* side effects, such as reassigning instance vars in this Suite or initializing
* a globally accessible external database. If you want to avoid reassigning instance vars
* you can use fixture.AsyncTestSuite.
*
* This trait's implementation of runTest invokes this method for each test, passing
* in a NoArgAsyncTest whose apply method will execute the code of the test
* and returns its result.
*
* This trait's implementation of this method simply invokes the passed NoArgAsyncTest function.
*
* If the by-name passed as the first parameter, future, completes abruptly with an exception
* this method will catch that exception, invoke the cleanup function passed as the second parameter,
* cleanup, then rethrow the exception. Otherwise, this method will register the cleanup
* function to be invoked after the resulting future completes.
*
* This method is redefine in this trait solely to narrow its contract. Subclasses must implement
* this method to call the withFixture(NoArgAsyncTest) method, which is defined in
* this trait.
*
* This trait's implementation of this method simply returns SucceededStatus
* and has no other effect.
*
Args for this run
* @return a Status object that indicates when the test started by this method has completed, and whether or not it failed.
*
* @throws NullArgumentException if either testName or args
* is null.
*/
protected override def runTest(testName: String, args: Args): Status = SucceededStatus
}