All Downloads are FREE. Search and download functionalities are using the official Maven repository.

org.scalatest.tools.Runner.scala Maven / Gradle / Ivy

Go to download

ScalaTest is a free, open-source testing toolkit for Scala and Java programmers.

There is a newer version: 2.0.M5
Show newest version
/*
 * Copyright 2001-2008 Artima, Inc.
 *
 * Licensed under the Apache License, Version 2.0 (the "License");
 * you may not use this file except in compliance with the License.
 * You may obtain a copy of the License at
 *
 *     http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */
package org.scalatest.tools

import org.scalatest._
import scala.collection.mutable.ListBuffer
import java.lang.reflect.Constructor
import java.lang.reflect.Modifier
import java.net.URL
import java.net.MalformedURLException
import java.net.URLClassLoader
import java.io.File
import java.io.IOException
import javax.swing.SwingUtilities
import java.util.concurrent.ArrayBlockingQueue
import java.util.regex.Pattern
import org.scalatest.testng.TestNGWrapperSuite
import java.util.concurrent.Semaphore
import org.scalatest.events._
import org.scalatest.junit.JUnitWrapperSuite
import java.util.concurrent.Executors
import java.util.concurrent.ExecutorService
import SuiteDiscoveryHelper._

/**
 * 

 * Application that runs a suite of tests.
 * The application accepts command line arguments that specify optional config map (key-value pairs), an optional 
 * runpath, zero to many Reporters, optional lists of tags to include and/or exclude, zero to many
 * Suite class names, zero to many "members-only" Suite paths, zero to many "wildcard" Suite paths,
 * and zero to many TestNG XML config file paths.
 * All of these arguments are described in more detail below. Here's a summary:
 * 

 *
 * 
 * scala [-classpath scalatest-<version>.jar:...] org.scalatest.tools.Runner
 * [-D<key>=<value> [...]] [-R <runpath>] [reporter [...]]
 * [-n <includes>] [-l <excludes>] [-P] [-s <suite class name>
 * [...]] [-j <junit class name> [...]] [-m <members-only suite path>
 * [...]] [-w <wildcard suite path> [...]] [-q <suffixes>] [-Q] [-y <chosen styles>]
 * [-b <TestNG config file path> [...]]
 * 

 *
 * 

 * The simplest way to start Runner is to specify the directory containing your compiled tests as the sole element of the runpath, for example:
 * 

 *
 * 
scala -classpath scalatest-<version>.jar org.scalatest.tools.Runner -R compiled_tests

 *
 * 

 * Given the previous command, Runner will discover and execute all Suites in the compiled_tests directory and its subdirectories,
 * and show results in graphical user interface (GUI).
 * 

 *
 * 
 * 
Specifying the config map

 *
 * 

 * A config map contains pairs consisting of a string key and a value that may be of any type. (Keys that start with
 * "org.scalatest." are reserved for ScalaTest. Configuration values that are themselves strings may be specified on the
 * Runner command line.
 * Each configuration pair is denoted with a "-D", followed immediately by the key string, an "=", and the value string.
 * For example:
 * 

 *
 * 
-Ddbname=testdb -Dserver=192.168.1.188

 *
 * 
 * 
Specifying a runpath

 *
 * 

 * A runpath is the list of filenames, directory paths, and/or URLs that Runner
 * uses to load classes for the running test. If runpath is specified, Runner creates
 * a custom class loader to load classes available on the runpath.
 * The graphical user interface reloads the test classes anew for each run
 * by creating and using a new instance of the custom class loader for each run.
 * The classes that comprise the test may also be made available on
 * the classpath, in which case no runpath need be specified.
 * 

 *
 * 

 * The runpath is specified with the -R option. The -R must be followed by a space,
 * a double quote ("), a white-space-separated list of
 * paths and URLs, and a double quote. If specifying only one element in the runpath, you can leave off
 * the double quotes, which only serve to combine a white-space separated list of strings into one
 * command line argument. If you have path elements that themselves have a space in them, you must
 * place a backslash (\) in front of the space. Here's an example:
 * 

 *
 * 
-R "serviceuitest-1.1beta4.jar myjini http://myhost:9998/myfile.jar target/class\ files"

 *
 * 
 * 
Specifying reporters

 *
 * 

 * Reporters can be specified  on the command line in any of the following
 * ways:
 * 

 *
 * 
    
 * 
  •  -g[configs...] - causes display of a graphical user interface that allows
 *    tests to be run and results to be investigated
    • 
 * 
  •  -f[configs...] <filename> - causes test results to be written to
 *     the named file
    • 
 * 
  •  -u <directory> - causes test results to be written to
 *      xml files in the named directory
    • 
 * 
  •  -o[configs...] - causes test results to be written to
 *     the standard output
    • 
 * 
  •  -e[configs...] - causes test results to be written to
 *     the standard error
    • 
 * 
  •  -C[configs...] <reporterclass> - causes test results to be reported to
 *     an instance of the specified fully qualified Reporter class name
    • 
 * 

 *
 * 

 * The [configs...] parameter, which is used to configure reporters, is described in the next section.
 * 

 *
 * 

 * The -C option causes the reporter specified in
 * <reporterclass> to be
 * instantiated.
 * Each reporter class specified with a -C option must be public, implement
 * org.scalatest.Reporter, and have a public no-arg constructor.
 * Reporter classes must be specified with fully qualified names. 
 * The specified reporter classes may be
 * deployed on the classpath. If a runpath is specified with the
 * -R option, the specified reporter classes may also be loaded from the runpath.
 * All specified reporter classes will be loaded and instantiated via their no-arg constructor.
 * 

 *
 * 

 * For example, to run a suite named MySuite from the mydir directory
 * using two reporters, the graphical reporter and a file reporter
 * writing to a file named "test.out", you would type:
 * 

 *
 * 
java -jar scalatest.jar -R mydir -g -f test.out -s MySuite

 *
 * 

 * The -g, -o, or -e options can
 * appear at most once each in any single command line.
 * Multiple appearances of -f and -C result in multiple reporters
 * unless the specified <filename> or <reporterclass> is
 * repeated. If any of -g, -o, -e,
 * <filename> or <reporterclass> are repeated on
 * the command line, the Runner will print an error message and not run the tests.
 * 

 *
 * 

 * Runner adds the reporters specified on the command line to a dispatch reporter,
 * which will dispatch each method invocation to each contained reporter. Runner will pass
 * the dispatch reporter to executed suites. As a result, every
 * specified reporter will receive every report generated by the running suite of tests.
 * If no reporters are specified, a graphical
 * runner will be displayed that provides a graphical report of
 * executed suites.
 * 

 *
 * 
 * 
Configuring Reporters

 * 
 *
 * 

 * Each reporter option on the command line can include configuration characters. Configuration characters
 * are specified immediately following the -g, -o,
 * -e, -f, or -C. The following configuration
 * characters, which cause reports to be dropped, are valid for any reporter:
 * 

 *
 * 
    
 * 
  •  N - drop TestStarting events
    • 
 * 
  •  C - drop TestSucceeded events
    • 
 * 
  •  X - drop TestIgnored events
    • 
 * 
  •  E - drop TestPending events
    • 
 * 
  •  H - drop SuiteStarting events
    • 
 * 
  •  L - drop SuiteCompleted events
    • 
 * 
  •  O - drop InfoProvided events
    • 
 * 

 *
 * 

 * A dropped event will not be delivered to the reporter at all. So the reporter will not know about it and therefore not
 * present information about the event in its report. For example, if you specify -oN, the standard output reporter
 * will never receive any TestStarting events and will therefore never report them. The purpose of these
 * configuration parameters is to allow users to selectively remove events they find add clutter to the report without
 * providing essential information.
 * 

 *
 * 

 * The following three reporter configuration parameters may additionally be used on standard output (-o), standard error (-e),
 * and file (-f) reporters: 
 * 

 *
 * 
    
 * 
  •  W - without color
    • 
 * 
  •  D - show all durations
    • 
 * 
  •  S - show short stack traces
    • 
 * 
  •  F - show full stack traces
    • 
 * 

 *
 * 

 * If you specify a W, D, S, or F for any reporter other than standard output, standard error, or file reporters, Runner
 * will complain with an error message and not perform the run.
 * 

 *
 * 

 * Configuring a standard output, error, or file reporter with D will cause that reporter to
 * print a duration for each test and suite.  When running in the default mode, a duration will only be printed for
 * the entire run.
 * 

 *
 * 

 * Configuring a standard output, error, or file reporter with F will cause that reporter to print full stack traces for all exceptions,
 * including TestFailedExceptions. Every TestFailedException contains a stack depth of the
 * line of test code that failed so that users won't need to search through a stack trace to find it. When running in the default,
 * mode, these reporters will only show full stack traces when other exceptions are thrown, such as an exception thrown
 * by production code. When a TestFailedException is thrown in default mode, only the source filename and
 * line number of the line of test code that caused the test to fail are printed along with the error message, not the full stack
 * trace. 
 * 

 *
 * 

 * By default, a standard output, error, or file reporter inserts ansi escape codes into the output printed to change and later reset
 * terminal colors. Information printed as a result of run starting, completed, and stopped events
 * is printed in cyan. Information printed as a result of ignored or pending test events is shown in yellow. Information printed
 * as a result of test failed, suite aborted, or run aborted events is printed in red. All other information is printed in green.
 * The purpose of these colors is to facilitate speedy reading of the output, especially the finding of failed tests, which can
 * get lost in a sea of passing tests. Configuring a standard output, error, or file reporter into without-color mode ('W') will
 * turn off this behavior. No ansi codes will be inserted.
 * 

 *
 * 

 * For example, to run a suite using two reporters, the graphical reporter configured to present every reported event
 * and a standard error reporter configured to present everything but test starting, test succeeded, test ignored, test
 * pending, suite starting, suite completed, and info provided events, you would type:
 * 

 *
 * 

 * scala -classpath scalatest-<version>.jar -R mydir -g -eNDXEHLO -s MySuite
 * 

 *
 * 

 * Note that no white space is allowed between the reporter option and the initial configuration
 * parameters. So "-e NDXEHLO" will not work,
 * "-eNDXEHLO" will work.
 * 

 *
 * 
 * 
Specifying tags to include and exclude

 * 
 *
 * 

 * You can specify tag names of tests to include or exclude from a run. To specify tags to include,
 * use -n followed by a white-space-separated list of tag names to include, surrounded by
 * double quotes. (The double quotes are not needed if specifying just one tag.)  Similarly, to specify tags
 * to exclude, use -l followed by a white-space-separated
 * list of tag names to exclude, surrounded by double quotes. (As before, the double quotes are not needed
 * if specifying just one tag.) If tags to include is not specified, then all tests
 * except those mentioned in the tags to exclude (and in the org.scalatest.Ignore tag), will be executed.
 * (In other words, the absence of a -n option is like a wildcard, indicating all tests be included.)
 * If tags to include is specified, then only those tests whose tags are mentioned in the argument following -n
 * and not mentioned in the tags to exclude, will be executed. For more information on test tags, see
 * the documentation for Suite. Here are some examples:
 * 

 *
 * 

 * 
    
 * 
  • -n CheckinTests
    • 
 * 
  • -n FunctionalTests -l SlowTests
    • 
 * 
  • -n "CheckinTests FunctionalTests" -l "SlowTests NetworkTests"
    • 
 * 

 * 

 *
 * 
 * 
Specifying suffixes to discover

 * 
 *
 * 

 * You can specify suffixes of Suite names to discover. To specify suffixes to discover,
 * use -q followed by a vertical-bar-separated list of suffixes to discover, surrounded by
 * double quotes. (The double quotes are not needed if specifying just one suffix.)  Or you can specify
 * them individually using multiple -q's.
 * If suffixes to discover is not specified, then all suffixes are considered.
 * If suffixes is specified, then only those Suites whose class names end in one of the specified suffixes
 * will be considered during discovery. Here are some examples:
 * 

 *
 * 

 * 
    
 * 
  • -q Spec
    • 
 * 
  • -q "Spec|Suite"
    • 
 * 
  • -q Spec -q Suite
    • 
 * 

 * 

 *
 * 

 * Option -Q can be used to specify a default set of suffixes "Spec|Suite". If you specify both -Q and -q, you'll get Spec
 * and Suite in addition to the other suffix or suffixes you specify with -q.
 * 

 *
 * 

 * Specifying suffixes can speed up the discovery process because class files with names not ending the specified suffixes
 * can be immediately disqualified, without needing to load and inspect them to see if they either extend Suite
 * and declare a public, no-arg constructor, or are annotated with WrapWith. 
 * 

 *
 * 
 * 
Executing Suites in parallel

 * 
 *
 * 

 * With the proliferation of multi-core architectures, and the often parallelizable nature of tests, it is useful to be able to run
 * tests in parallel. If you include -P on the command line, Runner will pass a Distributor to
 * the Suites you specify with -s. Runner will set up a thread pool to execute any Suites
 * passed to the Distributor's put method in parallel. Trait Suite's implementation of
 * runNestedSuites will place any nested Suites into this Distributor. Thus, if you have a Suite
 * of tests that must be executed sequentially, you should override runNestedSuites as described in the documentation for Distributor.
 * 

 *
 * 

 * The -P option may optionally be appended with a number (e.g.
 * "-P10" -- no intervening space) to specify the number of
 * threads to be created in the thread pool.  If no number (or 0) is
 * specified, the number of threads will be decided based on the number of
 * processors available.
 * 

 *
 * 
 * 
Specifying Suites

 * 
 *
 * 

 * Suites are specified on the command line with a -s followed by the fully qualified
 * name of a Suite subclass, as in:
 * 

 *
 * 
-s com.artima.serviceuitest.ServiceUITestkit

 *
 * 

 * Each specified suite class must be public, a subclass of
 * org.scalatest.Suite, and contain a public no-arg constructor.
 * Suite classes must be specified with fully qualified names. 
 * The specified Suite classes may be
 * loaded from the classpath. If a runpath is specified with the
 * -R option, specified Suite classes may also be loaded from the runpath.
 * All specified Suite classes will be loaded and instantiated via their no-arg constructor.
 * 

 *
 * 

 * The runner will invoke execute on each instantiated org.scalatest.Suite,
 * passing in the dispatch reporter to each execute method.
 * 

 *
 * 

 * Runner is intended to be used from the command line. It is included in org.scalatest
 * package as a convenience for the user. If this package is incorporated into tools, such as IDEs, which take
 * over the role of runner, object org.scalatest.tools.Runner may be excluded from that implementation of the package.
 * All other public types declared in package org.scalatest.tools.Runner should be included in any such usage, however,
 * so client software can count on them being available.
 * 

 *
 * 
 * 
Specifying "members-only" and "wildcard" Suite paths

 * 
 *
 * 

 * If you specify Suite path names with -m or -w, Runner will automatically
 * discover and execute accessible Suites in the runpath that are either a member of (in the case of -m)
 * or enclosed by (in the case of -w) the specified path. As used in this context, a path is a portion of a fully qualified name.
 * For example, the fully qualifed name com.example.webapp.MySuite contains paths com, com.example, and com.example.webapp.
 * The fully qualifed name com.example.webapp.MyObject.NestedSuite contains paths com, com.example,
 * com.example.webapp, and com.example.webapp.MyObject.
 * An accessible Suite is a public class that extends org.scalatest.Suite
 * and defines a public no-arg constructor. Note that Suites defined inside classes and traits do not have no-arg constructors,
 * and therefore won't be discovered. Suites defined inside singleton objects, however, do get a no-arg constructor by default, thus
 * they can be discovered.
 * 

 *
 * 

 * For example, if you specify -m com.example.webapp
 * on the command line, and you've placed com.example.webapp.RedSuite and com.example.webapp.BlueSuite
 * on the runpath, then Runner will instantiate and execute both of those Suites. The difference
 * between -m and -w is that for -m, only Suites that are direct members of the named path
 * will be discovered. For -w, any Suites whose fully qualified
 * name begins with the specified path will be discovered. Thus, if com.example.webapp.controllers.GreenSuite
 * exists on the runpath, invoking Runner with -w com.example.webapp will cause GreenSuite
 * to be discovered, because its fully qualifed name begins with "com.example.webapp". But if you invoke Runner
 * with -m com.example.webapp, GreenSuite will not be discovered because it is directly
 * a member of com.example.webapp.controllers, not com.example.webapp.
 * 

 *
 * 

 * If you specify no -s, -m, or -w arguments on the command line to Runner, it will discover and execute all accessible Suites
 * in the runpath.
 * 

 *
 * 
 * 
Specifying chosen styles

 * 
 *
 * 

 * You can optionally specify chosen styles for a ScalaTest run. ScalaTest supports different styles of
 * testing so that different teams can use the style or styles that best suits their situation and culture. But
 * in any one project, it is recommended you decide on one main style for unit testing, and
 * consistently use only that style for unit testing throughout the project. If you also have integration
 * tests in your project, you may wish to pick a different style for them than you are using for unit testing.
 * You may want to allow certain styles to be used in special testing situations on a project, but in general,
 * it is best to minimize the styles used in any given project to a few, or one.
 * 

 *
 * 

 * To facilitate the communication and enforcement of a team's style choices for a project, you can
 * specify the chosen styles in your project build. If chosen styles is defined, ScalaTest style traits that are
 * not among the chosen list will abort with a message complaining that the style trait is not one of the
 * chosen styles. The style name for each ScalaTest style trait is its fully qualified name. For example,
 * to specify that org.scalatest.FunSpec as your chosen style you'd pass this to
 * Runner:
 * 

 *
 * 
-y org.scalatest.FunSpec

 *
 * 

 * If you wanted org.scalatest.FunSpec as your main unit testing style, but also wanted to
 * allow PropSpec for test matrixes and FeatureSpec for
 * integration tests, you would write:
 * 

 *
 * 
-y org.scalatest.FunSpec -y org.scalatest.PropSpec -y org.scalatest.FeatureSpec

 *
 * 

 * To select org.scalatest.FlatSpec as your main unit testing style, but allow
 * org.scalatest.fixture.FlatSpec for multi-threaded unit tests, you'd write:
 * 

 *
 * 
-y org.scalatest.FlatSpec -y org.scalatest.fixture.FlatSpec

 *
 * 

 * The style name for a suite is obtained by invoking its styleName method. Custom style
 * traits can override this method so that a custom style can participate in the chosen styles list.
 * 

 *
 * 

 * Because ScalaTest is so customizable, a determined programmer could circumvent
 * the chosen styles check, but in practice -y should be persuasive enough tool
 * to keep most team members in line.
 * 

 *
 * 
 * 
Specifying TestNG XML config file paths

 *
 * 

 * If you specify one or more file paths with -b (b for Beust, the last name of TestNG's creator), Runner will create a org.scalatest.testng.TestNGWrapperSuite,
 * passing in a List of the specified paths. When executed, the TestNGWrapperSuite will create one TestNG instance
 * and pass each specified file path to it for running. If you include -b arguments, you must include TestNG's jar file on the class path or runpath.
 * The -b argument will enable you to run existing TestNG tests, including tests written in Java, as part of a ScalaTest run.
 * You need not use -b to run suites written in Scala that extend TestNGSuite. You can simply run such suites with 
 * -s, -m, or -w parameters.
 * 

 *
 * 
 * 
Specifying JUnit tests

 *
 * 

 * JUnit tests, including ones written in Java, may be run by specifying
 * -j classname, where the classname is a valid JUnit class
 * such as a TestCase, TestSuite, or a class implementing a static suite()
 * method returning a TestSuite. 

 * 

 * To use this option you must include a JUnit jar file on your classpath.
 * 

 *
 * @author Bill Venners
 * @author George Berger
 * @author Josh Cough
 */
object Runner {

  private val RUNNER_JFRAME_START_X: Int = 150
  private val RUNNER_JFRAME_START_Y: Int = 100
  
  @volatile private[scalatest] var spanScaleFactor: Double = 1.0

  //                     TO
  // We always include a PassFailReporter on runs in order to determine
  // whether or not all tests passed.
  //
  // The thread that calls Runner.run() will either start a GUI, if a graphic
  // reporter was requested, or just run the tests itself. If a GUI is started,
  // an event handler thread will get going, and it will start a RunnerThread,
  // which will actually do the running. The GUI can repeatedly start RunnerThreads
  // and RerunnerThreads, until the GUI is closed. If -P is specified, that means
  // the tests should be run concurrently, which in turn means a Distributor will
  // be passed to the execute method of the Suites, which will in turn populate
  // it with its nested suites instead of executing them directly in the same
  // thread. The Distributor works in conjunction with a pool of threads that
  // will take suites out of the distributor queue and execute them. The DispatchReporter
  // will serialize all reports via an actor, which because that actor uses receive
  // not react, will have its own thread. So the DispatchReporter actor's thread will
  // be the one that actually invokes TestFailed, RunAborted, etc., on this PassFailReporter.
  // The thread that invoked Runner.run(), will be the one that calls allTestsPassed.
  //
  // The thread that invoked Runner.run() will be the one to instantiate the PassFailReporter
  // and in its primary constructor acquire the single semaphore permit. This permit will
  // only be released by the DispatchReporter's actor thread when a runAborted, runStopped,
  // or runCompleted is invoked. allTestsPassed will block until it can reacquire the lone
  // semaphore permit. Thus, a PassFailReporter can just be used for one run, then it is
  // spent. A new PassFailReporter is therefore created each time the Runner.run() method is invoked.
  //
  private class PassFailReporter extends Reporter {

    @volatile private var failedAbortedOrStopped = false
    private val runDoneSemaphore = new Semaphore(1)
    runDoneSemaphore.acquire()

    override def apply(event: Event) {
      event match {
        case _: TestFailed =>
          failedAbortedOrStopped = true

        case _: RunAborted =>
          failedAbortedOrStopped = true
          runDoneSemaphore.release()

        case _: SuiteAborted =>
          failedAbortedOrStopped = true

        case _: RunStopped =>
          failedAbortedOrStopped = true
          runDoneSemaphore.release() 

        case _: RunCompleted =>
          runDoneSemaphore.release()

        case _ =>
      }
    }

    def allTestsPassed = {
      runDoneSemaphore.acquire()
      !failedAbortedOrStopped
    }
  }

  // TODO: I don't think I'm enforcing that properties can't start with "org.scalatest"
  // TODO: I don't think I'm handling rejecting multiple -f/-C with the same arg. -f fred.txt -f fred.txt should
  // fail, as should -C MyReporter -C MyReporter. I'm failing on -o -o, -g -g, and -e -e, but the error messages
  // could indeed be nicer.
  /**
   * Runs a suite of tests, with optional GUI. See the main documentation for this singleton object for the details.
   */
  def main(args: Array[String]) {
    val result = 
      if (args.contains("-v") || args.contains("--version")) {
        val version = Resources("AppVersion")
        val scalaVersion = Resources("ScalaVersion")
        println("ScalaTest " + version + " (Built for Scala " + scalaVersion + ")")
        runOptionallyWithPassFailReporter(args.filter(arg => arg != "-v" && arg != "--version"), true)
      }
      else
        runOptionallyWithPassFailReporter(args, true)

    if (result)
      exit(0)
    else
      exit(1)
  }

  /**
   * Runs a suite of tests, with optional GUI. See the main documentation for this singleton object for the details.
   * The difference between this method and main is simply that this method will block until the run
   * has completed, aborted, or been stopped, and return true if all tests executed and passed. In other
   * words, if any test fails, or if any suite aborts, or if the run aborts or is stopped, this method will
   * return false. This value is used, for example, by the ScalaTest ant task to determine whether
   * to continue the build if haltOnFailure is set to true.
   *
   * @return true if all tests were executed and passed.
   */
  def run(args: Array[String]): Boolean = {
    runOptionallyWithPassFailReporter(args, true)
  }

  private def runOptionallyWithPassFailReporter(args: Array[String], runWithPassFailReporter: Boolean): Boolean = {

    checkArgsForValidity(args) match {
      case Some(s) => {
        println(s)
        exit(1)
      }
      case None =>
    }

    val (
      runpathArgsList,
      reporterArgsList,
      suiteArgsList,
      junitArgsList,
      propertiesArgsList,
      includesArgsList,
      excludesArgsList,
      concurrentList,
      membersOnlyArgsList,
      wildcardArgsList,
      testNGArgsList,
      suffixes, 
      chosenStyles, 
      spanScaleFactors
    ) = parseArgs(args)

    val fullReporterConfigurations: ReporterConfigurations =
      if (reporterArgsList.isEmpty)
        // If no reporters specified, just give them a graphic reporter
        new ReporterConfigurations(Some(GraphicReporterConfiguration(Set())), Nil, Nil, None, None, Nil, Nil)
      else
        parseReporterArgsIntoConfigurations(reporterArgsList)

    val suitesList: List[String] = parseSuiteArgsIntoNameStrings(suiteArgsList, "-s")
    val junitsList: List[String] = parseSuiteArgsIntoNameStrings(junitArgsList, "-j")
    val runpathList: List[String] = parseRunpathArgIntoList(runpathArgsList)
    val propertiesMap: Map[String, Object] = parsePropertiesArgsIntoMap(propertiesArgsList)
    val tagsToInclude: Set[String] = parseCompoundArgIntoSet(includesArgsList, "-n")
    val tagsToExclude: Set[String] = parseCompoundArgIntoSet(excludesArgsList, "-l")
    val concurrent: Boolean = !concurrentList.isEmpty
    val numThreads: Int = parseConcurrentNumArg(concurrentList)
    val membersOnlyList: List[String] = parseSuiteArgsIntoNameStrings(membersOnlyArgsList, "-m")
    val wildcardList: List[String] = parseSuiteArgsIntoNameStrings(wildcardArgsList, "-w")
    val testNGList: List[String] = parseSuiteArgsIntoNameStrings(testNGArgsList, "-b")
    val chosenStyleSet: Set[String] = parseChosenStylesIntoChosenStyleSet(chosenStyles, "-y")
    spanScaleFactor = parseSpanScaleFactor(spanScaleFactors, "-F")

    val filter = Filter(if (tagsToInclude.isEmpty) None else Some(tagsToInclude), tagsToExclude)

    // If there's a graphic reporter, we need to leave it out of
    // reporterSpecs, because we want to pass all reporterSpecs except
    // the graphic reporter's to the RunnerJFrame (because RunnerJFrame *is*
    // the graphic reporter).
    val reporterConfigs: ReporterConfigurations =
      fullReporterConfigurations.graphicReporterConfiguration match {
        case None => fullReporterConfigurations
        case Some(grs) => {
          new ReporterConfigurations(
            None,
            fullReporterConfigurations.fileReporterConfigurationList,
            fullReporterConfigurations.xmlReporterConfigurationList,
            fullReporterConfigurations.standardOutReporterConfiguration,
            fullReporterConfigurations.standardErrReporterConfiguration,
            fullReporterConfigurations.htmlReporterConfigurationList,
            fullReporterConfigurations.customReporterConfigurationList
          )
        }
      }

    val passFailReporter = if (runWithPassFailReporter) Some(new PassFailReporter) else None
    
    if (propertiesMap.isDefinedAt("org.scalatest.ChosenStyles"))
      throw new IllegalArgumentException("Property name 'org.scalatest.ChosenStyles' is used by ScalaTest, please choose other property name.")
    val configMap = 
      if (chosenStyleSet.isEmpty)
        propertiesMap
      else
        propertiesMap + ("org.scalatest.ChosenStyles" -> chosenStyleSet)

    fullReporterConfigurations.graphicReporterConfiguration match {
      case Some(GraphicReporterConfiguration(configSet)) => {
        val graphicEventsToPresent: Set[EventToPresent] = EventToPresent.allEventsToPresent filter
          (if (configSet.contains(FilterTestStarting)) {_ != PresentTestStarting} else etp => true) filter
          (if (configSet.contains(FilterTestSucceeded)) {_ != PresentTestSucceeded} else etp => true) filter
          (if (configSet.contains(FilterTestIgnored)) {_ != PresentTestIgnored} else etp => true) filter
          (if (configSet.contains(FilterTestPending)) {_ != PresentTestPending} else etp => true) filter
          (if (configSet.contains(FilterSuiteStarting)) {_ != PresentSuiteStarting} else etp => true) filter
          (if (configSet.contains(FilterSuiteCompleted)) {_ != PresentSuiteCompleted} else etp => true) filter
          (if (configSet.contains(FilterInfoProvided)) {_ != PresentInfoProvided} else etp => true) 

        val abq = new ArrayBlockingQueue[RunnerJFrame](1)
        usingEventDispatchThread {
          val rjf = new RunnerJFrame(graphicEventsToPresent, reporterConfigs, suitesList, junitsList, runpathList,
            filter, configMap, concurrent, membersOnlyList, wildcardList, testNGList, passFailReporter, numThreads,
            suffixes)
          rjf.setLocation(RUNNER_JFRAME_START_X, RUNNER_JFRAME_START_Y)
          rjf.setVisible(true)
          rjf.prepUIForRunning()
          rjf.runFromGUI()
          abq.put(rjf)
        }
        // To get the Ant task to work, the main thread needs to block until
        // The GUI window exits.
        val rjf = abq.take()
        rjf.blockUntilWindowClosed()
      }
      case None => { // Run the test without a GUI
        withClassLoaderAndDispatchReporter(runpathList, reporterConfigs, None, passFailReporter) {
          (loader, dispatchReporter) => {
            doRunRunRunDaDoRunRun(dispatchReporter, suitesList, junitsList, new Stopper {}, filter,
                configMap, concurrent, membersOnlyList, wildcardList, testNGList, runpathList, loader, new RunDoneListener {}, 1, numThreads, suffixes) 
          }
        }
      }
    }
    
    passFailReporter match {
      case Some(pfr) => pfr.allTestsPassed
      case None => false
    }
  }

  // Returns an Option[String]. Some is an error message. None means no error.
  private[scalatest] def checkArgsForValidity(args: Array[String]) = {

    val lb = new ListBuffer[String]
    val it = args.iterator
    while (it.hasNext) {
      val s = it.next
      // Style advice
      // If it is multiple else ifs, then make it symetrical. If one needs an open curly brace, put it on all
      // If an if just has another if, a compound statement, go ahead and put the open curly brace's around the outer one
      if (s.startsWith("-p") || s.startsWith("-R") || s.startsWith("-f") || s.startsWith("-u") || s.startsWith("-h") || s.startsWith("-r") || s.startsWith("-C") || s.startsWith("-n") || s.startsWith("-x") || s.startsWith("-l") || s.startsWith("-q") || s.startsWith("-Q") || s.startsWith("-s") || s.startsWith("-j") || s.startsWith("-m") || s.startsWith("-w") || s.startsWith("-b") || s.startsWith("-y") || s.startsWith("-F")) {
        if (it.hasNext)
          it.next
      }
      else if (!s.startsWith("-D") && !s.startsWith("-g") && !s.startsWith("-o") && !s.startsWith("-e") && !s.startsWith("-c") && !s.startsWith("-P")) {
        lb += s
      }
    }
    val argsList = lb.toList
    if (argsList.length != 0)
      Some("Unrecognized argument" + (if (argsList.isEmpty) ": " else "s: ") + argsList.mkString("", ", ", "."))
    else
      None
  }

  //
  // Examines concurrent option arg to see if it contains an optional numeric
  // value representing the number of threads to use, e.g. -P10 for 10 threads.
  //
  // It's possible for user to specify the -P option multiple times on the
  // command line, although it isn't particularly useful.  This method scans
  // through multiples until it finds one with a number appended and uses
  // that.  If none have a number it just returns 0.
  //
  private[scalatest] def parseConcurrentNumArg(concurrentList: List[String]):
  Int = {
    val opt = concurrentList.find(_.matches("-c\\d+"))

    opt match {
      case Some(arg) => arg.replace("-c", "").toInt
      case None      => 0
    }
  }

  //
  // Generates a Pattern based on suffixes passed in by user.  Pattern
  // matches class names that end with one of the specified suffixes.
  //
  private def genSuffixesPattern(suffixesList: List[String]): Option[Pattern] = {
    if (suffixesList.isEmpty)
      None
    else
      Some(Pattern.compile(".*(" + suffixesList.mkString("|") + ")$"))
  }

  private[scalatest] def parseArgs(args: Array[String]) = {

    val runpath = new ListBuffer[String]()
    val reporters = new ListBuffer[String]()
    val suites = new ListBuffer[String]()
    val junits = new ListBuffer[String]()
    val props = new ListBuffer[String]()
    val includes = new ListBuffer[String]()
    val excludes = new ListBuffer[String]()
    val concurrent = new ListBuffer[String]()
    val membersOnly = new ListBuffer[String]()
    val wildcard = new ListBuffer[String]()
    val testNGXMLFiles = new ListBuffer[String]()
    val suffixes = new ListBuffer[String]()
    val chosenStyles = new ListBuffer[String]()
    val spanScaleFactor = new ListBuffer[String]()

    val it = args.iterator
    while (it.hasNext) {

      val s = it.next

      if (s.startsWith("-D")) {
         props += s
      }
      else if (s.startsWith("-p")) {
        println("WARNING: -p has been deprecated and will be reused for a different (but still very cool) purpose in ScalaTest 2.0. Please change all uses of -p to -R.")
        runpath += s
        if (it.hasNext)
          runpath += it.next
      }
      else if (s.startsWith("-R")) {
        runpath += "-p" + s.substring(2)
        if (it.hasNext)
          runpath += it.next
      }
      else if (s.startsWith("-g")) {
        reporters += s
      }
      else if (s.startsWith("-o")) {
        reporters += s
      }
      else if (s.startsWith("-e")) {
        reporters += s
      }
      else if (s.startsWith("-f")) {
        reporters += s
        if (it.hasNext)
          reporters += it.next
      }
      else if (s.startsWith("-u")) {
        reporters += s
        if (it.hasNext)
          reporters += it.next
      }
      else if (s.startsWith("-h")) {
        reporters += s
        if (it.hasNext)
          reporters += it.next
      }
      else if (s.startsWith("-n")) {
        includes += s
        if (it.hasNext)
          includes += it.next
      }
      else if (s.startsWith("-x")) {
        System.err.println(Resources("dashXDeprecated"))
        excludes += s.replace("-x", "-l")
        if (it.hasNext)
          excludes += it.next
      }
      else if (s.startsWith("-l")) {
        excludes += s
        if (it.hasNext)
          excludes += it.next
      }
      else if (s.startsWith("-r")) {
        println("WARNING: -r has been deprecated and will be reused for a different (but still very cool) purpose in ScalaTest 2.0. Please change all uses of -r to -C.")
        reporters += s
        if (it.hasNext)
          reporters += it.next
      }
      else if (s.startsWith("-C")) {

        reporters += "-r" + s.substring(2)
        if (it.hasNext)
          reporters += it.next
      }
      else if (s.startsWith("-s")) {

        suites += s
        if (it.hasNext)
          suites += it.next
      }
      else if (s.startsWith("-j")) {

        junits += s
        if (it.hasNext)
          junits += it.next
      }
      else if (s.startsWith("-m")) {

        membersOnly += s
        if (it.hasNext)
          membersOnly += it.next
      }
      else if (s.startsWith("-w")) {

        wildcard += s
        if (it.hasNext)
          wildcard += it.next
      }
      else if (s.startsWith("-c")) {
        println("WARNING: -c has been deprecated and will be reused for a different (but still very cool) purpose in ScalaTest 2.0. Please change all uses of -c to -P.")
        concurrent += s
      }
      else if (s.startsWith("-P")) {

        concurrent += "-c" + s.substring(2)
      }
      else if (s.startsWith("-b")) {

        testNGXMLFiles += s
        if (it.hasNext)
          testNGXMLFiles += it.next
      }
      else if (s.startsWith("-q")) {
        if (it.hasNext)
          suffixes += it.next()
      }
      else if (s.startsWith("-Q")) {
        suffixes += "Spec|Suite"
      }
      else if (s.startsWith("-y")) {
        chosenStyles += s
        if (it.hasNext)
          chosenStyles += it.next()
      }
      else if (s.startsWith("-F")) {
        spanScaleFactor += s
        if (it.hasNext)
          spanScaleFactor += it.next()
      }
      else {
        throw new IllegalArgumentException("Unrecognized argument: " + s)
      }
    }

    (
      runpath.toList,
      reporters.toList,
      suites.toList,
      junits.toList,
      props.toList,
      includes.toList,
      excludes.toList,
      concurrent.toList,
      membersOnly.toList,
      wildcard.toList,
      testNGXMLFiles.toList,
      genSuffixesPattern(suffixes.toList), 
      chosenStyles.toList, 
      spanScaleFactor.toList
    )
  }


  /**
   * Returns a possibly empty ConfigSet containing configuration
   * objects specified in the passed reporterArg. Configuration
   * options are specified immediately following
   * the reporter option, as in:
   *
   * -oFA
   *
   * If no configuration options are specified, this method returns an
   * empty ConfigSet. This method never returns null.
   */
  private def parseConfigSet(reporterArg: String): Set[ReporterConfigParam] = {

    if (reporterArg == null)
      throw new NullPointerException("reporterArg was null")

    if (reporterArg.length < 2)
      throw new IllegalArgumentException("reporterArg < 2")

    // The reporterArg passed includes the initial -, as in "-oFI",
    // so the first config param will be at index 2
    val configString = reporterArg.substring(2)
    val it = configString.iterator
    var set = Set[ReporterConfigParam]()
    while (it.hasNext) 
      it.next match {
        case 'Y' =>  throw new IllegalArgumentException("Use of Y was deprecated in ScalaTest 1.0 and removed in 1.5. Please check the Scaladoc documentation of org.scalatest.Runner for information on valid Reporter config parameters.")
        case 'Z' => throw new IllegalArgumentException("Use of Z was deprecated in ScalaTest 1.0 and removed in 1.5. Please check the Scaladoc documentation of org.scalatest.Runner for information on valid Reporter config parameters.")
        case 'T' => // Use for Dots
        case 'U' => throw new IllegalArgumentException("Use of U was deprecated in ScalaTest 1.0 and removed in 1.5. Please check the Scaladoc documentation of org.scalatest.Runner for information on valid Reporter config parameters.")
        case 'P' =>throw new IllegalArgumentException("Use of P was deprecated in ScalaTest 1.0 and removed in 1.5. Please check the Scaladoc documentation of org.scalatest.Runner for information on valid Reporter config parameters.")
        case 'B' =>throw new IllegalArgumentException("Use of B was deprecated in ScalaTest 1.0 and removed in 1.5. Please check the Scaladoc documentation of org.scalatest.Runner for information on valid Reporter config parameters.")
        case 'I' =>throw new IllegalArgumentException("Use of I was deprecated in ScalaTest 1.0 and removed in 1.5. Please check the Scaladoc documentation of org.scalatest.Runner for information on valid Reporter config parameters.")
        // case 'S' => // Use for Short Stack Traces
        case 'A' =>throw new IllegalArgumentException("Use of A was deprecated in ScalaTest 1.0 and removed in 1.5. Please check the Scaladoc documentation of org.scalatest.Runner for information on valid Reporter config parameters.")
        case 'R' =>throw new IllegalArgumentException("Use of R was deprecated in ScalaTest 1.0 and removed in 1.5. Please check the Scaladoc documentation of org.scalatest.Runner for information on valid Reporter config parameters.")
        case 'G' =>throw new IllegalArgumentException("Use of G was deprecated in ScalaTest 1.0 and removed in 1.5. Please check the Scaladoc documentation of org.scalatest.Runner for information on valid Reporter config parameters.")
        case 'N' => set += FilterTestStarting
        case 'C' => set += FilterTestSucceeded
        case 'X' => set += FilterTestIgnored
        case 'E' => set += FilterTestPending
        case 'H' => set += FilterSuiteStarting
        case 'L' => set += FilterSuiteCompleted
        case 'O' => set += FilterInfoProvided
        case 'W' => set += PresentWithoutColor
        case 'F' => set += PresentFullStackTraces
        case 'S' => set += PresentShortStackTraces
        case 'D' => set += PresentAllDurations
        case c: Char => { 

          // this should be moved to the checker, and just throw an exception here with a debug message. Or allow a MatchError.
          val msg1 = Resources("invalidConfigOption", String.valueOf(c)) + '\n'
          val msg2 =  Resources("probarg", reporterArg) + '\n'

          throw new IllegalArgumentException(msg1 + msg2)
        }
      }
    set
  }

  private[scalatest] def parseReporterArgsIntoConfigurations(args: List[String]) = {
    //
    // Checks to see if any args are smaller than two characters in length.
    // Allows a one-character arg if it's a directory-name parameter, to
    // permit use of "." for example.
    //
    def argTooShort(args: List[String]): Boolean = {
      args match {
        case Nil => false

        case "-u" :: directory :: list => argTooShort(list)

        case x :: list =>
          if (x.length < 2) true
          else              argTooShort(list)
      }
    }

    if (args == null)
      throw new NullPointerException("args was null")

    if (args.exists(_ == null))
      throw new NullPointerException("an arg String was null")

    if (argTooShort(args)) // TODO: check and print out a user friendly message for this
      throw new IllegalArgumentException("an arg String was less than 2 in length: " + args)

    for (dashX <- List("-g", "-o", "-e")) {
      if (args.toList.count(_.startsWith(dashX)) > 1) // TODO: also check and print a user friendly message for this
        throw new IllegalArgumentException("Only one " + dashX + " allowed")
    }

    // TODO: also check and print a user friendly message for this
    // again here, i had to skip some things, so I had to use an iterator.
    val it = args.iterator
    while (it.hasNext)
      it.next.take(2).toString match {
        case "-g" =>
        case "-o" =>
        case "-e" =>
        case "-f" =>
          if (it.hasNext)
            it.next // scroll past the filename
          else
            throw new IllegalArgumentException("-f needs to be followed by a file name arg: ")
        case "-u" =>
          if (it.hasNext) {
            val directoryName = it.next
            val directory = new File(directoryName)
            if (!directory.isDirectory) {
              try {
                directory.mkdirs()
                if (!directory.exists)
                  throw new IllegalArgumentException("Unable to create directory: " + directory.getAbsolutePath)
              }
              catch {
                case se: SecurityException => 
                  throw new IllegalArgumentException("Unable to create directory: " + directory.getAbsolutePath)
              }
            }
            else if (directory.isFile)
              throw new IllegalArgumentException(directory.getAbsolutePath + " is a file, directory expected.")
          }
          else {
            throw new IllegalArgumentException("-u needs to be followed by a directory name arg: ")
          }
        case "-h" =>
          if (it.hasNext)
            it.next // scroll past the filename
          else
            throw new IllegalArgumentException("-h needs to be followed by a file name arg: ")
        case "-r" =>
          if (it.hasNext)
            it.next // scroll past the reporter class
          else
            throw new IllegalArgumentException("-r needs to be followed by a reporter class name arg: ")
        case "-C" =>
          if (it.hasNext)
            it.next // scroll past the reporter class
          else
            throw new IllegalArgumentException("-C needs to be followed by a reporter class name arg: ")
        case arg: String =>
          throw new IllegalArgumentException("An arg started with an invalid character string: " + arg)
      }

    val graphicReporterConfigurationOption =
      args.find(arg => arg.startsWith("-g")) match {
        case Some(dashGString) =>
          val configSet = parseConfigSet(dashGString)
          if (configSet.contains(PresentShortStackTraces))
            throw new IllegalArgumentException("Cannot specify an S (present short stack traces) configuration parameter for the graphic reporter (because it shows them anyway): " + dashGString)
          if (configSet.contains(PresentFullStackTraces))
            throw new IllegalArgumentException("Cannot specify an F (present full stack traces) configuration parameter for the graphic reporter (because it shows them anyway): " + dashGString)
          if (configSet.contains(PresentWithoutColor))
            throw new IllegalArgumentException("Cannot specify a W (present without color) configuration parameter for the graphic reporter: " + dashGString)
          if (configSet.contains(PresentAllDurations))
            throw new IllegalArgumentException("Cannot specify a D (present all durations) configuration parameter for the graphic reporter (because it shows them all anyway): " + dashGString)
          Some(new GraphicReporterConfiguration(configSet))
        case None => None
      }

    def buildFileReporterConfigurationList(args: List[String]) = {
      val it = args.iterator
      val lb = new ListBuffer[FileReporterConfiguration]
      while (it.hasNext) {
        val arg = it.next
        if (arg.startsWith("-f"))
          lb += new FileReporterConfiguration(parseConfigSet(arg), it.next)
      }
      lb.toList
    }
    val fileReporterConfigurationList = buildFileReporterConfigurationList(args)

    def buildXmlReporterConfigurationList(args: List[String]) = {
      val it = args.iterator
      val lb = new ListBuffer[XmlReporterConfiguration]
      while (it.hasNext) {
        val arg = it.next
        if (arg.startsWith("-u"))
          lb += new XmlReporterConfiguration(Set[ReporterConfigParam](),
                                             it.next)
      }
      lb.toList
    }
    val xmlReporterConfigurationList = buildXmlReporterConfigurationList(args)

    def buildHtmlReporterConfigurationList(args: List[String]) = {
      val it = args.iterator
      val lb = new ListBuffer[HtmlReporterConfiguration]
      while (it.hasNext) {
        val arg = it.next
        if (arg.startsWith("-h"))
          lb += new HtmlReporterConfiguration(parseConfigSet(arg), it.next)
      }
      lb.toList
    }
    val htmlReporterConfigurationList = buildHtmlReporterConfigurationList(args)

    val standardOutReporterConfigurationOption =
      args.find(arg => arg.startsWith("-o")) match {
        case Some(dashOString) => Some(new StandardOutReporterConfiguration(parseConfigSet(dashOString)))
        case None => None
      }

    val standardErrReporterConfigurationOption =
      args.find(arg => arg.startsWith("-e")) match {
        case Some(dashEString) => Some(new StandardErrReporterConfiguration(parseConfigSet(dashEString)))
        case None => None
      }

    def buildCustomReporterConfigurationList(args: List[String]) = {
      val it = args.iterator
      val lb = new ListBuffer[CustomReporterConfiguration]
      while (it.hasNext) {
        val arg = it.next
        if (arg.startsWith("-r")) {
          val dashRString = arg
          val customReporterClassName = it.next
          val configSet = parseConfigSet(dashRString)
          if (configSet.contains(PresentShortStackTraces))
            throw new IllegalArgumentException("Cannot specify an S (present short stack traces) configuration parameter for a custom reporter: " + dashRString + " " + customReporterClassName)
          if (configSet.contains(PresentFullStackTraces))
            throw new IllegalArgumentException("Cannot specify an F (present full stack traces) configuration parameter for a custom reporter: " + dashRString + " " + customReporterClassName)
          if (configSet.contains(PresentWithoutColor))
            throw new IllegalArgumentException("Cannot specify a W (without color) configuration parameter for a custom reporter: " + dashRString + " " + customReporterClassName)
          if (configSet.contains(PresentAllDurations))
            throw new IllegalArgumentException("Cannot specify a D (present all durations) configuration parameter for a custom reporter: " + dashRString + " " + customReporterClassName)
          lb += new CustomReporterConfiguration(configSet, customReporterClassName)
        }
      }
      lb.toList
    }
    val customReporterConfigurationList = buildCustomReporterConfigurationList(args)

    // Here instead of one loop, i go through the loop several times.
    new ReporterConfigurations(
      graphicReporterConfigurationOption,
      fileReporterConfigurationList,
      xmlReporterConfigurationList,
      standardOutReporterConfigurationOption,
      standardErrReporterConfigurationOption,
      htmlReporterConfigurationList,
      customReporterConfigurationList
    )
  }

  // Used to parse -s, -j, -m, and -w args, one of which will be passed as a String as dashArg
  private[scalatest] def parseSuiteArgsIntoNameStrings(args: List[String], dashArg: String) = {

    if (args == null)
      throw new NullPointerException("args was null")

    if (args.exists(_ == null))
      throw new NullPointerException("an arg String was null")

    if (dashArg != "-j" && dashArg != "-s" && dashArg != "-w" && dashArg != "-m" && dashArg != "-b")
      throw new NullPointerException("dashArg invalid: " + dashArg)

    val lb = new ListBuffer[String]
    val it = args.iterator
    while (it.hasNext) {
      val dashS = it.next
      if (dashS != dashArg)
        throw new IllegalArgumentException("Every other element, starting with the first, must be " + dashArg)
      if (it.hasNext) {
        val suiteName = it.next
        if (!suiteName.startsWith("-"))
          lb += suiteName
        else
          throw new IllegalArgumentException("Expecting a Suite class name or package name to follow " + dashArg + ", but got: " + suiteName)
      }
      else
        throw new IllegalArgumentException("Last element must be a Suite class name or package name, not a " + dashArg + ".")
    }
    lb.toList
  }

  private[scalatest] def parseCompoundArgIntoSet(args: List[String], expectedDashArg: String): Set[String] = 
      Set() ++ parseCompoundArgIntoList(args, expectedDashArg)

  private[scalatest] def parseRunpathArgIntoList(args: List[String]): List[String] = parseCompoundArgIntoList(args, "-p")

  private[scalatest] def parseCompoundArgIntoList(args: List[String], expectedDashArg: String): List[String] = {

    if (args == null)
      throw new NullPointerException("args was null")

    if (args.exists(_ == null))
      throw new NullPointerException("an arg String was null")

    if (args.length == 0) {
      List()
    }
    else if (args.length == 2) {
      val dashArg = args(0)
      val runpathArg = args(1)

      if (dashArg != expectedDashArg)
        throw new IllegalArgumentException("First arg must be " + expectedDashArg + ", but was: " + dashArg)

      if (runpathArg.trim.isEmpty)
        throw new IllegalArgumentException("The runpath string must actually include some non-whitespace characters.")

      splitPath(runpathArg)
    }
    else {
      throw new IllegalArgumentException("Runpath must be either zero or two args: " + args)
    }
  }
  
  private[scalatest] def parseChosenStylesIntoChosenStyleSet(args: List[String], dashArg: String) = {
    val it = args.iterator
    val lb = new ListBuffer[String]()
    while (it.hasNext) {
      val dash = it.next
      if (dash != dashArg) 
        throw new IllegalArgumentException("Every other element, starting with the first, must be " + dashArg)
      if (it.hasNext) {
        lb += it.next
      }
      else
        throw new IllegalArgumentException("Last element must be a style name, not a " + dashArg + ".")
    }
    lb.toSet
  }
  
  private[scalatest] def parseSpanScaleFactor(args: List[String], dashArg: String) = {
    val it = args.iterator
    val lb = new ListBuffer[Double]()
    while (it.hasNext) {
      val dash = it.next
      if (dash != dashArg)
        throw new IllegalArgumentException("Every other element, starting with the first, must be " + dashArg)
      if (it.hasNext) {
        val spanString = it.next
        try {
          lb += spanString.toDouble
        }
        catch {
          case e: NumberFormatException =>
            throw new IllegalArgumentException(dashArg + " must be followed by a number, but '" + spanString + "' is not a number.")
        }
      }
      else
        throw new IllegalArgumentException("Last element must be a number, not a " + dashArg + ".")
    }
    if (lb.size == 0)
      1.0
    else if (lb.size == 1)
      lb(0)
    else
      throw new IllegalArgumentException("Only one -F can be specified.")
  }

  //
  // Splits a space-delimited path into its component parts.
  //
  // Spaces within path elements may be escaped with backslashes, e.g.
  // "c:\Documents\ And\ Settings c:\Program\ Files"
  //
  // See comments for isCompleteToken() below for exceptions.
  //
  private val START_TOKEN_PATTERN = Pattern.compile("""^\s*(.*?)(\s|$)""")
  private val FULL_TOKEN_PATTERN  = Pattern.compile("""^\s*(.+?)(((?<=[^\\])\s)|$)""")
  private def splitPath(pathArg: String): List[String] = {
    val path = pathArg.trim

    if (path.isEmpty) Nil
    else {
      val startMatcher = START_TOKEN_PATTERN.matcher(path)

      if (!startMatcher.find())
        throw new RuntimeException("unexpected startMatcher path [" +
                                   path + "]")
      val token = startMatcher.group(1)

      if (isCompleteToken(token)) {
        token :: splitPath(path.substring(startMatcher.end))
      }
      else {
        val fullMatcher = FULL_TOKEN_PATTERN.matcher(path)

        if (!fullMatcher.find())
          throw new RuntimeException("unexpected fullMatcher path [" +
                                     path + "]")
        val fullToken = fullMatcher.group(1).replaceAll("""\\(\s)""", "$1")

        fullToken :: splitPath(path.substring(fullMatcher.end))
      }
    }
  }

  //
  // Determines whether specified token is complete or partial.
  //
  // Tokens are considered partial if they end with a backslash, since
  // backslash is used to escape spaces that would otherwise be
  // treated as delimiters within the path string.
  //
  // Exceptions are cases where the token ends in a backslash
  // but is still considered a complete token because it constitutes
  // a valid representation of a root directory on a windows system,
  // e.g. "c:\" or just "\".
  //
  private val ROOT_DIR_PATTERN = Pattern.compile("""(?i)\\|[a-z]:\\""")
  private def isCompleteToken(token: String): Boolean = {
    val matcher = ROOT_DIR_PATTERN.matcher(token)

    matcher.matches() || (token(token.length - 1) != '\\')
  }

  private[scalatest] def parsePropertiesArgsIntoMap(args: List[String]) = {

    if (args == null)
      throw new NullPointerException("args was null")

    if (args.exists(_ == null))
      throw new NullPointerException("an arg String was null")

    if (args.exists(_.indexOf('=') == -1))
      throw new IllegalArgumentException("A -D arg does not contain an equals sign.")

    if (args.exists(!_.startsWith("-D")))
      throw new IllegalArgumentException("A spice arg does not start with -D.")

    if (args.exists(_.indexOf('=') == 2))
      throw new IllegalArgumentException("A spice arg does not have a key to the left of the equals sign.")

    if (args.exists(arg => arg.indexOf('=') == arg.length - 1))
      throw new IllegalArgumentException("A spice arg does not have a value to the right of the equals sign.")

    val tuples = for (arg <- args) yield {
      val keyValue = arg.substring(2) // Cut off the -D at the beginning
      val equalsPos = keyValue.indexOf('=')
      val key = keyValue.substring(0, equalsPos)
      val value = keyValue.substring(equalsPos + 1)
      (key, value)
    }

    scala.collection.immutable.Map() ++ tuples
  }

  // For debugging.
/*
  private[scalatest] def printOpts(opt: EventToPresent.Set32) {
    if (opt.contains(EventToPresent.PresentRunStarting))
      println("PresentRunStarting")
    if (opt.contains(EventToPresent.PresentTestStarting))
      println("PresentTestStarting")
    if (opt.contains(EventToPresent.PresentTestSucceeded))
      println("PresentTestSucceeded")
    if (opt.contains(EventToPresent.PresentTestFailed))
      println("PresentTestFailed")
    if (opt.contains(EventToPresent.PresentTestIgnored))
      println("PresentTestIgnored")
    if (opt.contains(EventToPresent.PresentSuiteStarting))
      println("PresentSuiteStarting")
    if (opt.contains(EventToPresent.PresentSuiteCompleted))
      println("PresentSuiteCompleted")
    if (opt.contains(EventToPresent.PresentSuiteAborted))
      println("PresentSuiteAborted")
    if (opt.contains(EventToPresent.PresentInfoProvided))
      println("PresentInfoProvided")
    if (opt.contains(EventToPresent.PresentRunStopped))
      println("PresentRunStopped")
    if (opt.contains(EventToPresent.PresentRunCompleted))
      println("PresentRunCompleted")
    if (opt.contains(EventToPresent.PresentRunAborted))
      println("PresentRunAborted")
  }
*/

  private[scalatest] def doRunRunRunDaDoRunRun(
    dispatch: DispatchReporter,
    suitesList: List[String],
    junitsList: List[String],
    stopRequested: Stopper,
    filter: Filter,
    configMap: Map[String, Object],
    concurrent: Boolean,
    membersOnlyList: List[String],
    wildcardList: List[String],
    testNGList: List[String],
    runpath: List[String],
    loader: ClassLoader,
    doneListener: RunDoneListener,
    runStamp: Int,
    numThreads: Int,
    suffixes: Option[Pattern]
  ) = {

    // TODO: add more, and to RunnerThread too
    if (dispatch == null)
      throw new NullPointerException
    if (suitesList == null)
      throw new NullPointerException
    if (junitsList == null)
      throw new NullPointerException
    if (stopRequested == null)
      throw new NullPointerException
    if (filter == null)
      throw new NullPointerException
    if (configMap == null)
      throw new NullPointerException
    if (membersOnlyList == null)
      throw new NullPointerException
    if (wildcardList == null)
      throw new NullPointerException
    if (runpath == null)
      throw new NullPointerException
    if (loader == null)
      throw new NullPointerException
    if (doneListener == null)
      throw new NullPointerException

    val tagsToInclude =
      filter.tagsToInclude match {
        case None => Set[String]()
        case Some(tti) => tti
      }
    val tagsToExclude = filter.tagsToExclude

    var tracker = new Tracker(new Ordinal(runStamp))

    val runStartTime = System.currentTimeMillis

    try {
      val loadProblemsExist =
        try {
          val unrunnableList = suitesList.filter{ className => 
            loader.loadClass(className) // Check if the class exist, so if not we get the nice cannot load suite error message.
            !isAccessibleSuite(className, loader) && !isRunnable(className, loader)
          }
          if (!unrunnableList.isEmpty) {
            val names = for (className <- unrunnableList) yield " " + className
            dispatch(RunAborted(tracker.nextOrdinal(), Resources("nonSuite") + names, None))
            true
          }
          else {
            false
          }
        }
        catch {
          case e: ClassNotFoundException => {
            dispatch(RunAborted(tracker.nextOrdinal(), Resources("cannotLoadSuite", e.getMessage), Some(e)))
            true
          }
        }
  
      if (!loadProblemsExist) {
        try {
          val namedSuiteInstances: List[Suite] =
            for (suiteClassName <- suitesList)
              yield {
                val clazz = loader.loadClass(suiteClassName)
                val wrapWithAnnotation = clazz.getAnnotation(classOf[WrapWith])
                if (wrapWithAnnotation == null)
                  clazz.newInstance.asInstanceOf[Suite]
                else {
                  val suiteClazz = wrapWithAnnotation.value
                  val constructorList = suiteClazz.getDeclaredConstructors()
                  val constructor = constructorList.find { c => 
                    val types = c.getParameterTypes
                    types.length == 1 && types(0) == classOf[java.lang.Class[_]]
                  }
                  constructor.get.newInstance(clazz).asInstanceOf[Suite]
                } 
              }

          val junitSuiteInstances: List[Suite] =
            for (junitClassName <- junitsList)
              yield new JUnitWrapperSuite(junitClassName, loader)

          val testNGWrapperSuiteList: List[TestNGWrapperSuite] =
            if (!testNGList.isEmpty)
              List(new TestNGWrapperSuite(testNGList))
            else
              Nil

          val (membersOnlySuiteInstances, wildcardSuiteInstances) = {

            val membersOnlyAndBeginsWithListsAreEmpty = membersOnlyList.isEmpty && wildcardList.isEmpty // They didn't specify any -m's or -w's on the command line


            // TODO: rename the 'BeginsWith' variables to 'Wildcard' to match the terminology that
            // we ended up with on the outside
            // TODO: Should SuiteDiscoverHelper be a singleton object?
            if (membersOnlyAndBeginsWithListsAreEmpty && (!suitesList.isEmpty || !junitsList.isEmpty)) {
              (Nil, Nil) // No DiscoverySuites in this case. Just run Suites named with -s or -j
            }
            else {
              val accessibleSuites = discoverSuiteNames(runpath, loader, suffixes)

              if (membersOnlyAndBeginsWithListsAreEmpty && suitesList.isEmpty && junitsList.isEmpty) {
                // In this case, they didn't specify any -w, -m, -s, or -j on the command line, so the default
                // is to run any accessible Suites discovered on the runpath
                (Nil, List(new DiscoverySuite("", accessibleSuites, true, loader)))
              }
              else {
                val membersOnlyInstances =
                  for (membersOnlyName <- membersOnlyList)
                    yield new DiscoverySuite(membersOnlyName, accessibleSuites, false, loader)

                val wildcardInstances =
                  for (wildcardName <- wildcardList)
                    yield new DiscoverySuite(wildcardName, accessibleSuites, true, loader)

                (membersOnlyInstances, wildcardInstances)
              }
            }
          }

          val suiteInstances: List[Suite] = namedSuiteInstances ::: junitSuiteInstances ::: membersOnlySuiteInstances ::: wildcardSuiteInstances ::: testNGWrapperSuiteList

          val testCountList =
            for (suite <- suiteInstances)
              yield suite.expectedTestCount(filter)
  
          def sumInts(list: List[Int]): Int =
            list match {
              case Nil => 0
              case x :: xs => x + sumInts(xs)
            }

          val expectedTestCount = sumInts(testCountList)

          dispatch(RunStarting(tracker.nextOrdinal(), expectedTestCount, configMap))
          
          if (concurrent) {

            // Because some tests may do IO, will create a pool of 2 times the number of processors reported
            // by the Runtime's availableProcessors method.
            val poolSize =
              if (numThreads > 0) numThreads
              else Runtime.getRuntime.availableProcessors * 2

            val execSvc: ExecutorService = Executors.newFixedThreadPool(poolSize)
            try {

            if (System.getProperty("org.scalatest.tools.Runner.forever", "false") == "true") {
              val distributor = new ConcurrentDistributor(dispatch, stopRequested, filter, configMap, execSvc)
              while (true) {
                for (suite <- suiteInstances) {
                  distributor.apply(suite, tracker.nextTracker())
                }
                distributor.waitUntilDone()
              }
            }
            else {
              val distributor = new ConcurrentDistributor(dispatch, stopRequested, filter, configMap, execSvc)
              for (suite <- suiteInstances) {
                distributor.apply(suite, tracker.nextTracker())
              }
              distributor.waitUntilDone()
            }
            }
            finally {
              execSvc.shutdown()
            }
          }
          else {
            for (suite <- suiteInstances) {
              val suiteRunner = new SuiteRunner(suite, dispatch, stopRequested, filter,
                  configMap, None, tracker)
              suiteRunner.run()
            }
          }

          val duration = System.currentTimeMillis - runStartTime
          if (stopRequested()) {
            dispatch(RunStopped(tracker.nextOrdinal(), Some(duration)))
          }
          else {
            dispatch(RunCompleted(tracker.nextOrdinal(), Some(duration)))
          }
        }
        catch {
          case e: InstantiationException =>
            dispatch(RunAborted(tracker.nextOrdinal(), Resources("cannotInstantiateSuite", e.getMessage), Some(e), Some(System.currentTimeMillis - runStartTime)))
          case e: IllegalAccessException =>
            dispatch(RunAborted(tracker.nextOrdinal(), Resources("cannotInstantiateSuite", e.getMessage), Some(e), Some(System.currentTimeMillis - runStartTime)))
          case e: NoClassDefFoundError =>
            dispatch(RunAborted(tracker.nextOrdinal(), Resources("cannotLoadClass", e.getMessage), Some(e), Some(System.currentTimeMillis - runStartTime)))
          case e: Throwable =>
            dispatch(RunAborted(tracker.nextOrdinal(), Resources.bigProblems(e), Some(e), Some(System.currentTimeMillis - runStartTime)))
        }
      }
    }
    finally {
      dispatch.dispatchDisposeAndWaitUntilDone()
      doneListener.done()
    }
  }

  private[scalatest] def excludesWithIgnore(excludes: Set[String]) = excludes + "org.scalatest.Ignore"

  private[scalatest] def withClassLoaderAndDispatchReporter(runpathList: List[String], reporterSpecs: ReporterConfigurations,
      graphicReporter: Option[Reporter], passFailReporter: Option[Reporter])(f: (ClassLoader, DispatchReporter) => Unit): Unit = {

    val loader: ClassLoader = getRunpathClassLoader(runpathList)
    try {
      Thread.currentThread.setContextClassLoader(loader)
      try {
        val dispatchReporter = ReporterFactory.getDispatchReporter(reporterSpecs, graphicReporter, passFailReporter, loader)
        try {
          f(loader, dispatchReporter)
        }
        finally {
          dispatchReporter.dispatchDisposeAndWaitUntilDone()
        }
      }
      catch {
        // getDispatchReporter may complete abruptly with an exception, if there is an problem trying to load
        // or instantiate a custom reporter class.
        case ex: Throwable => {
          System.err.println(Resources("bigProblemsMaybeCustomReporter"))
          ex.printStackTrace(System.err)
        }
      }
    }
    finally {
      // eventually call close on the RunpathClassLoader
    }
  }

  private[scalatest] def getRunpathClassLoader(runpathList: List[String]): ClassLoader = {

    if (runpathList == null)
      throw new NullPointerException
    if (runpathList.isEmpty) {
      classOf[Suite].getClassLoader // Could this be null technically?
    }
    else {
      val urlsList: List[URL] =
        for (raw <- runpathList) yield {
          try {
            new URL(raw)
          }
          catch {
            case murle: MalformedURLException => {
  
              // Assume they tried to just pass in a file name
              val file: File = new File(raw)
  
              // file.toURL may throw MalformedURLException too, but for now
              // just let that propagate up.
              file.toURI.toURL // If a dir, comes back terminated by a slash
            }
          }
        }
  
      // Here is where the Jini preferred class loader stuff went.

      // Tell the URLConnections to not use caching, so that repeated runs and reruns actually work
      // on the latest binaries.
      for (url <- urlsList) {
        try {
          url.openConnection.setDefaultUseCaches(false)
        }
        catch {
          case e: IOException => // just ignore these
        }
      }

      new URLClassLoader(urlsList.toArray, classOf[Suite].getClassLoader)
    }
  }

  private[scalatest] def usingEventDispatchThread(f: => Unit): Unit = {
    SwingUtilities.invokeLater(
      new Runnable() {
        def run() {
          f
        }
      }
    )
  }
}




© 2015 - 2024 Weber Informatics LLC | Privacy Policy