site.apt.examples.muto-process-example.apt.vm Maven / Gradle / Ivy
Go to download
Show more of this group Show more artifacts with this name
Show all versions of muto Show documentation
Show all versions of muto Show documentation
Automated Mutation Testing Framework
The newest version!
------
Muto Process Examples
------
Myk Kolisnyk
------
2014-06-04
Overview
The <> goal performs mutation testing of the current project code. During execution it goes through the following stages:
* <> - creates folder where current project is copied to. This is done to minimize probability of committing mutation after testing.
* <> - based on rules specified the Muto applies changes to the code in the workspace.
* <> - normally that should be a command running tests against mutated code.
* <> - after testing is done and reports are generated the initial state is to be restored.
* <> - once all runs are done the workspace is cleaned up.
The final result is the report containing information about all mutations applied and results collected.
Parameters to specify
Running Muto Processor
* Maven Configuration
The sample <> fragment shows how to generate such report:
+------+
...
com.github.mkolisnyk
muto
${project.version}
.
target/muto/workspace
mvn compile test
target/muto/workspace/target/surfire
target/muto
...
...
+------+
Customize Muto Processing
* Restrict initial file set to process
By default the engine copies all current project content into separate working directory. But project folder may contain some auxiliary folders like <<.git>> or
<<.svn>> or some temporary stuff like <>, <> etc. These resources are not really needed in the workspace so it would be good to avoid such directories
from being copied into the workspace. For such optimization there are <> and <> parameters. They specify the list of inclusion/exclusion patterns.
The below configuration will copy all <<*.java>> files into workspace folder while <<.git>>, <> and <> folders will be excluded. The <> fragment looks like:
+------+
...
com.github.mkolisnyk
muto
${project.version}
.
target/muto/workspace
\\.git
target
src/site
(.*).java
mvn compile test
target/muto/workspace/target/surfire
target/muto
...
...
+------+
* Restricting the set of tests to run
In some cases we're interested just in some specific tests to run against the mutated version. Actually, this is the same as if we run those tests from command line.
So, normally we can restrict out test suite by:
* groups/categories
* File/Class names
* File patterns
So, actually, we need to run specific command line for the build engine which we use to run our tests. Something like:
+------+
...
com.github.mkolisnyk
muto
${project.version}
...
mvn compile test -Dcategory=regression
...
...
...
+------+
* Identifying the lists of mutations to apply
By default all built-in mutation types are applied during mutation testing suite execution. But in some cases we need some specific rules to be used.
Also, since Muto is extensible engine, there may be a need to create some custom mutation rules/strategies. Naturally, we'd like to see them running as well.
For this purpose the plugin configuration has entries like:
* mutationRules - contains the list of class names corresponding to mutation rules
* mutationStrategies - contains the list of class names representing mutation insertion strategies
* fileStrategies - contains the list of class names representing file processing strategies
Each of them can be customized to fit exact mutation approach you really need. Here is the example of Maven configuration defining mutation related attributes:
+------+
...
com.github.mkolisnyk
muto
${project.version}
...
com.github.mkolisnyk.muto.generator.filestrategies.OneByOneFileProcessingStrategy
com.github.mkolisnyk.muto.generator.strategies.OneByOneMutationStrategy
com.github.mkolisnyk.muto.generator.rules.BlockCleanMutationRule
com.github.mkolisnyk.muto.generator.rules.NumberSignMutationRule
...
...
...
+------+
<> if there's a need to apply some custom mutation attributes which are out of scope of standard set of classes we should make sure
those classes are included into classpath.
* Identifying the set of files to apply mutations to
In some cases we need to apply mutations to specific set of files, not the entire suite. This can be done by identifying <> attribute.
When engine runs it takes <> information to identify the list of files to process.
+------+
...
com.github.mkolisnyk
muto
${project.version}
...
(.*)Strategy.java
...
...
...
+------+
After such settings all mutations will be applied only to files matchine the <<(.*)Strategy.java>> pattern
* Customizing results output
By default no output is done. But all reporting functionality is concentrated inside listeners. If we want to
make XML output or just simply write results into the console we should define <> attribute. Here is
an example of it:
+------+
...
com.github.mkolisnyk
muto
${project.version}
...
com.github.mkolisnyk.muto.reporter.listeners.ConsoleListener
com.github.mkolisnyk.muto.reporter.listeners.XmlListener
...
...
...
+------+
Such instruction will produce both console and XML output.