au.com.dius.pact.provider.junit5.PactJUnit5VerificationProvider.kt Maven / Gradle / Ivy

Go to download

Show more of this group Show more artifacts with this name
Show all versions of pact-jvm-provider-junit5 Show documentation

# Pact Junit 5 Extension ## Overview For writing Pact verification tests with JUnit 5, there is an JUnit 5 Invocation Context Provider that you can use with the `@TestTemplate` annotation. This will generate a test for each interaction found for the pact files for the provider. To use it, add the `@Provider` and one of the pact source annotations to your test class (as per a JUnit 4 test), then add a method annotated with `@TestTemplate` and `@ExtendWith(PactVerificationInvocationContextProvider.class)` that takes a `PactVerificationContext` parameter. You will need to call `verifyInteraction()` on the context parameter in your test template method. For example: ```java @Provider("myAwesomeService") @PactFolder("pacts") public class ContractVerificationTest { @TestTemplate @ExtendWith(PactVerificationInvocationContextProvider.class) void pactVerificationTestTemplate(PactVerificationContext context) { context.verifyInteraction(); } } ``` For details on the provider and pact source annotations, refer to the [Pact junit runner](../pact-jvm-provider-junit/README.md) docs. ## Test target You can set the test target (the object that defines the target of the test, which should point to your provider) on the `PactVerificationContext`, but you need to do this in a before test method (annotated with `@BeforeEach`). There are three different test targets you can use: `HttpTestTarget`, `HttpsTestTarget` and `AmpqTestTarget`. For example: ```java @BeforeEach void before(PactVerificationContext context) { context.setTarget(HttpTestTarget.fromUrl(new URL(myProviderUrl))); // or something like // context.setTarget(new HttpTestTarget("localhost", myProviderPort, "/")); } ``` **Note for Maven users:** If you use Maven to run your tests, you will have to make sure that the Maven Surefire plugin is at least version 2.22.1 uses an isolated classpath. For example, configure it by adding the following to your POM: ```xml <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-surefire-plugin</artifactId> <version>2.22.1</version> <configuration> <useSystemClassLoader>false</useSystemClassLoader> </configuration> </plugin> ``` ## Provider State Methods Provider State Methods work in the same way as with JUnit 4 tests, refer to the [Pact junit runner](../pact-jvm-provider-junit/README.md) docs. ### Using multiple classes for the state change methods If you have a large number of state change methods, you can split things up by moving them to other classes. You will need to specify the additional classes on the test context in a `Before` method. Do this with the `withStateHandler` or `setStateHandlers` methods. See [StateAnnotationsOnAdditionalClassTest](src/test/java/au/com/dius/pact/provider/junit5/StateAnnotationsOnAdditionalClassTest.java) for an example. ## Modifying the requests before they are sent **Important Note:** You should only use this feature for things that can not be persisted in the pact file. By modifying the request, you are potentially modifying the contract from the consumer tests! Sometimes you may need to add things to the requests that can't be persisted in a pact file. Examples of these would be authentication tokens, which have a small life span. The Http and Https test targets support injecting the request that will executed into the test template method. You can then add things to the request before calling the `verifyInteraction()` method. For example to add a header: ```java @TestTemplate @ExtendWith(PactVerificationInvocationContextProvider.class) void testTemplate(PactVerificationContext context, HttpRequest request) { // This will add a header to the request request.addHeader("X-Auth-Token", "1234"); context.verifyInteraction(); } ``` ## Objects that can be injected into the test methods You can inject the following objects into your test methods (just like the `PactVerificationContext`). They will be null if injected before the supported phase. | Object | Can be injected from phase | Description | | ------ | --------------- | ----------- | | PactVerificationContext | @BeforeEach | The context to use to execute the interaction test | | Pact | any | The Pact model for the test | | Interaction | any | The Interaction model for the test | | HttpRequest | @TestTemplate | The request that is going to be executed (only for HTTP and HTTPS targets) | | ProviderVerifier | @TestTemplate | The verifier instance that is used to verify the interaction | ## Allowing the test to pass when no pacts are found to verify (version 4.0.7+) By default, the test will fail with an exception if no pacts were found to verify. This can be overridden by adding the `@IgnoreNoPactsToVerify` annotation to the test class. For this to work, you test class will need to be able to receive null values for any of the injected parameters.

The newest version!

package au.com.dius.pact.provider.junit5

import au.com.dius.pact.core.model.Interaction
import au.com.dius.pact.core.model.Pact
import au.com.dius.pact.core.model.ProviderState
import au.com.dius.pact.core.model.RequestResponseInteraction
import au.com.dius.pact.core.pactbroker.TestResult
import au.com.dius.pact.core.support.expressions.SystemPropertyResolver
import au.com.dius.pact.core.support.expressions.ValueResolver
import au.com.dius.pact.core.support.isNotEmpty
import au.com.dius.pact.provider.ConsumerInfo
import au.com.dius.pact.provider.DefaultTestResultAccumulator
import au.com.dius.pact.provider.IProviderVerifier
import au.com.dius.pact.provider.PactVerification
import au.com.dius.pact.provider.ProviderInfo
import au.com.dius.pact.provider.ProviderVerifier
import au.com.dius.pact.provider.TestResultAccumulator
import au.com.dius.pact.provider.junit.AllowOverridePactUrl
import au.com.dius.pact.provider.junit.Consumer
import au.com.dius.pact.provider.junit.IgnoreNoPactsToVerify
import au.com.dius.pact.provider.junit.JUnitProviderTestSupport
import au.com.dius.pact.provider.junit.JUnitProviderTestSupport.checkForOverriddenPactUrl
import au.com.dius.pact.provider.junit.JUnitProviderTestSupport.filterPactsByAnnotations
import au.com.dius.pact.provider.junit.MissingStateChangeMethod
import au.com.dius.pact.provider.junit.Provider
import au.com.dius.pact.provider.junit.State
import au.com.dius.pact.provider.junit.StateChangeAction
import au.com.dius.pact.provider.junit.VerificationReports
import au.com.dius.pact.provider.junit.loader.NoPactsFoundException
import au.com.dius.pact.provider.junit.loader.PactLoader
import au.com.dius.pact.provider.junit.loader.PactSource
import au.com.dius.pact.provider.reporters.ReporterManager
import mu.KLogging
import org.apache.http.HttpRequest
import org.junit.jupiter.api.extension.AfterTestExecutionCallback
import org.junit.jupiter.api.extension.BeforeEachCallback
import org.junit.jupiter.api.extension.BeforeTestExecutionCallback
import org.junit.jupiter.api.extension.Extension
import org.junit.jupiter.api.extension.ExtensionContext
import org.junit.jupiter.api.extension.ParameterContext
import org.junit.jupiter.api.extension.ParameterResolver
import org.junit.jupiter.api.extension.TestTemplateInvocationContext
import org.junit.jupiter.api.extension.TestTemplateInvocationContextProvider
import org.junit.platform.commons.support.AnnotationSupport
import org.junit.platform.commons.support.HierarchyTraversalMode
import org.junit.platform.commons.support.ReflectionSupport
import java.io.File
import java.lang.reflect.Method
import java.util.stream.Stream
import kotlin.reflect.full.createInstance
import kotlin.reflect.full.findAnnotation

/**
 * The instance that holds the context for the test of an interaction. The test target will need to be set on it in
 * the before each phase of the test, and the verifyInteraction method must be called in the test template method.
 */
data class PactVerificationContext @JvmOverloads constructor(
  private val store: ExtensionContext.Store,
  private val context: ExtensionContext,
  var target: TestTarget = HttpTestTarget(port = 8080),
  var verifier: IProviderVerifier? = null,
  var valueResolver: ValueResolver = SystemPropertyResolver(),
  var providerInfo: ProviderInfo = ProviderInfo(),
  val consumerName: String,
  val interaction: Interaction,
  var testExecutionResult: TestResult = TestResult.Ok
) {
  val stateChangeHandlers: MutableList = mutableListOf()
  var executionContext: Map? = null

  /**
   * Called to verify the interaction from the test template method.
   *
   * @throws AssertionError Throws an assertion error if the verification fails.
   */
  fun verifyInteraction() {
    val store = context.getStore(ExtensionContext.Namespace.create("pact-jvm"))
    val client = store.get("client")
    val request = store.get("request")
    val testContext = store.get("interactionContext") as PactVerificationContext
    val failures = mutableMapOf()
    try {
      this.testExecutionResult = validateTestExecution(client, request, failures, testContext.executionContext ?: emptyMap())
      if (testExecutionResult is TestResult.Failed) {
        verifier!!.displayFailures(failures)
        throw AssertionError(JUnitProviderTestSupport.generateErrorStringFromMismatches(failures))
      }
    } finally {
      verifier!!.finaliseReports()
    }
  }

  private fun validateTestExecution(
    client: Any?,
    request: Any?,
    failures: MutableMap,
    context: Map
  ): TestResult {
    if (providerInfo.verificationType == null || providerInfo.verificationType == PactVerification.REQUEST_RESPONSE) {
      val interactionMessage = "Verifying a pact between $consumerName and ${providerInfo.name}" +
        " - ${interaction.description}"
      return try {
        val reqResInteraction = interaction as RequestResponseInteraction
        val expectedResponse = reqResInteraction.response.generatedResponse(context)
        val actualResponse = target.executeInteraction(client, request)

        verifier!!.verifyRequestResponsePact(expectedResponse, actualResponse, interactionMessage, failures,
          reqResInteraction.interactionId.orEmpty())
      } catch (e: Exception) {
        failures[interactionMessage] = e
        verifier!!.reporters.forEach {
          it.requestFailed(providerInfo, interaction, interactionMessage, e,
            verifier!!.projectHasProperty.apply(ProviderVerifier.PACT_SHOW_STACKTRACE))
        }
        TestResult.Failed(listOf(mapOf("message" to "Request to provider failed with an exception",
          "exception" to e, "interactionId" to interaction.interactionId)),
          "Request to provider failed with an exception")
      }
    } else {
      return verifier!!.verifyResponseByInvokingProviderMethods(providerInfo, ConsumerInfo(consumerName), interaction,
        interaction.description, failures)
    }
  }

  fun withStateChangeHandlers(vararg stateClasses: Any): PactVerificationContext {
    stateChangeHandlers.addAll(stateClasses)
    return this
  }

  fun addStateChangeHandlers(vararg stateClasses: Any) {
    stateChangeHandlers.addAll(stateClasses)
  }
}

/**
 * JUnit 5 test extension class used to inject parameters and execute the test for a Pact interaction.
 */
class PactVerificationExtension(
  private val pact: Pact,
  private val pactSource: au.com.dius.pact.core.model.PactSource,
  private val interaction: Interaction,
  private val serviceName: String,
  private val consumerName: String?
) : TestTemplateInvocationContext, ParameterResolver, BeforeEachCallback, BeforeTestExecutionCallback,
  AfterTestExecutionCallback {

  private val testResultAccumulator = DefaultTestResultAccumulator

  override fun getDisplayName(invocationIndex: Int): String {
    return "${pact.consumer.name} - ${interaction.description}"
  }

  override fun getAdditionalExtensions(): MutableList {
    return mutableListOf(PactVerificationStateChangeExtension(pact, interaction, testResultAccumulator), this)
  }

  override fun supportsParameter(parameterContext: ParameterContext, extensionContext: ExtensionContext): Boolean {
    val store = extensionContext.getStore(ExtensionContext.Namespace.create("pact-jvm"))
    val testContext = store.get("interactionContext") as PactVerificationContext
    return when (parameterContext.parameter.type) {
      Pact::class.java -> true
      Interaction::class.java -> true
      HttpRequest::class.java -> testContext.target is HttpTestTarget || testContext.target is HttpsTestTarget
      PactVerificationContext::class.java -> true
      ProviderVerifier::class.java -> true
      else -> false
    }
  }

  override fun resolveParameter(parameterContext: ParameterContext, extensionContext: ExtensionContext): Any? {
    val store = extensionContext.getStore(ExtensionContext.Namespace.create("pact-jvm"))
    return when (parameterContext.parameter.type) {
      Pact::class.java -> pact
      Interaction::class.java -> interaction
      HttpRequest::class.java -> store.get("httpRequest")
      PactVerificationContext::class.java -> store.get("interactionContext")
      ProviderVerifier::class.java -> store.get("verifier")
      else -> null
    }
  }

  override fun beforeEach(context: ExtensionContext) {
    val store = context.getStore(ExtensionContext.Namespace.create("pact-jvm"))
    store.put("interactionContext", PactVerificationContext(store, context, consumerName = pact.consumer.name,
      interaction = interaction))
  }

  override fun beforeTestExecution(context: ExtensionContext) {
    val store = context.getStore(ExtensionContext.Namespace.create("pact-jvm"))
    val testContext = store.get("interactionContext") as PactVerificationContext

    val providerInfo = testContext.target.getProviderInfo(serviceName, pactSource)
    testContext.providerInfo = providerInfo

    prepareVerifier(testContext, context, pactSource)
    store.put("verifier", testContext.verifier)

    val requestAndClient = testContext.target.prepareRequest(interaction, testContext.executionContext ?: emptyMap())
    if (requestAndClient != null) {
      val (request, client) = requestAndClient
      store.put("request", request)
      store.put("client", client)
      if (testContext.target.isHttpTarget()) {
        store.put("httpRequest", request)
      }
    }
  }

  private fun prepareVerifier(testContext: PactVerificationContext, extContext: ExtensionContext, pactSource: au.com.dius.pact.core.model.PactSource) {
    val consumer = ConsumerInfo(consumerName ?: pact.consumer.name)

    val verifier = ProviderVerifier()
    testContext.target.prepareVerifier(verifier, extContext.requiredTestInstance)

    setupReporters(verifier, serviceName, interaction.description, extContext, testContext.valueResolver)

    verifier.initialiseReporters(testContext.providerInfo)
    verifier.reportVerificationForConsumer(consumer, testContext.providerInfo, pactSource)

    if (interaction.providerStates.isNotEmpty()) {
      for ((name) in interaction.providerStates) {
        verifier.reportStateForInteraction(name.toString(), testContext.providerInfo, consumer, true)
      }
    }

    verifier.reportInteractionDescription(interaction)

    testContext.verifier = verifier
  }

  private fun setupReporters(
    verifier: IProviderVerifier,
    name: String,
    description: String,
    extContext: ExtensionContext,
    valueResolver: ValueResolver
  ) {
    var reportDirectory = "target/pact/reports"
    val reports = mutableListOf()
    var reportingEnabled = false

    val verificationReports = AnnotationSupport.findAnnotation(extContext.requiredTestClass, VerificationReports::class.java)
    if (verificationReports.isPresent) {
      reportingEnabled = true
      reportDirectory = verificationReports.get().reportDir
      reports.addAll(verificationReports.get().value)
    } else if (valueResolver.propertyDefined("pact.verification.reports")) {
      reportingEnabled = true
      reportDirectory = valueResolver.resolveValue("pact.verification.reportDir:$reportDirectory")!!
      reports.addAll(valueResolver.resolveValue("pact.verification.reports:")!!.split(","))
    }

    if (reportingEnabled) {
      val reportDir = File(reportDirectory)
      reportDir.mkdirs()
      verifier.reporters = reports
        .filter { r -> r.isNotEmpty() }
        .map { r ->
          val reporter = ReporterManager.createReporter(r.trim(), reportDir)
          reporter.reportFile = File(reportDir, "$name - $description${reporter.ext}")
          reporter
        }
    }
  }

  override fun afterTestExecution(context: ExtensionContext) {
    val store = context.getStore(ExtensionContext.Namespace.create("pact-jvm"))
    val testContext = store.get("interactionContext") as PactVerificationContext
    testResultAccumulator.updateTestResult(pact, interaction, testContext.testExecutionResult)
  }

  companion object : KLogging()
}

/**
 * JUnit 5 test extension class for executing state change callbacks
 */
class PactVerificationStateChangeExtension(
  private val pact: Pact,
  private val interaction: Interaction,
  private val testResultAccumulator: TestResultAccumulator
) : BeforeTestExecutionCallback, AfterTestExecutionCallback {
  override fun beforeTestExecution(extensionContext: ExtensionContext) {
    logger.debug { "beforeEach for interaction '${interaction.description}'" }
    val store = extensionContext.getStore(ExtensionContext.Namespace.create("pact-jvm"))
    val testContext = store.get("interactionContext") as PactVerificationContext

    try {
      val providerStateContext = invokeStateChangeMethods(extensionContext, testContext,
        interaction.providerStates, StateChangeAction.SETUP)
      testContext.executionContext = mapOf("providerState" to providerStateContext)
    } catch (e: Exception) {
      logger.error(e) { "Provider state change callback failed" }
      testContext.testExecutionResult = TestResult.Failed(description = "Provider state change callback failed",
        results = listOf(mapOf("exception" to e)))
      throw AssertionError("Provider state change callback failed", e)
    }
  }

  override fun afterTestExecution(context: ExtensionContext) {
    logger.debug { "afterEach for interaction '${interaction.description}'" }
    val store = context.getStore(ExtensionContext.Namespace.create("pact-jvm"))
    val testContext = store.get("interactionContext") as PactVerificationContext

    invokeStateChangeMethods(context, testContext, interaction.providerStates, StateChangeAction.TEARDOWN)
  }

  private fun invokeStateChangeMethods(
    context: ExtensionContext,
    testContext: PactVerificationContext,
    providerStates: List,
    action: StateChangeAction
  ): Map {
    val errors = mutableListOf()

    val providerStateContext = mutableMapOf()
    providerStates.forEach { state ->
      val stateChangeMethods = findStateChangeMethods(context.requiredTestInstance,
        testContext.stateChangeHandlers, state)
      if (stateChangeMethods.isEmpty()) {
        errors.add("Did not find a test class method annotated with @State(\"${state.name}\")")
      } else {
        stateChangeMethods.filter { it.second.action == action }.forEach { (method, _, instance) ->
          logger.debug { "Invoking state change method ${method.name} for state '${state.name}' on $instance" }
          val stateChangeValue = if (method.parameterCount > 0) {
            ReflectionSupport.invokeMethod(method, instance, state.params)
          } else {
            ReflectionSupport.invokeMethod(method, instance)
          }

          if (stateChangeValue is Map<*, *>) {
            providerStateContext.putAll(stateChangeValue as Map)
          }
        }
      }
    }

    if (errors.isNotEmpty()) {
      throw MissingStateChangeMethod(errors.joinToString("\n"))
    }

    return providerStateContext
  }

  private fun findStateChangeMethods(
    testClass: Any,
    stateChangeHandlers: List,
    state: ProviderState
  ): List> {
    val stateChangeClasses =
      AnnotationSupport.findAnnotatedMethods(testClass.javaClass, State::class.java, HierarchyTraversalMode.TOP_DOWN)
        .map { it to testClass }
        .plus(stateChangeHandlers.flatMap { handler ->
          AnnotationSupport.findAnnotatedMethods(handler.javaClass, State::class.java, HierarchyTraversalMode.TOP_DOWN)
            .map { it to handler }
        })
    return stateChangeClasses
      .map { Triple(it.first, it.first.getAnnotation(State::class.java), it.second) }
      .filter { it.second.value.any { s -> state.name == s } }
  }

  companion object : KLogging()
}

/**
 * Main TestTemplateInvocationContextProvider for JUnit 5 Pact verification tests. This class needs to be applied to
 * a test template method on a test class annotated with a @Provider annotation.
 */
open class PactVerificationInvocationContextProvider : TestTemplateInvocationContextProvider {

  override fun provideTestTemplateInvocationContexts(context: ExtensionContext): Stream {
    logger.debug { "provideTestTemplateInvocationContexts called" }
    val tests = resolvePactSources(context)
    return when {
      tests.first.isNotEmpty() -> tests.first.stream() as Stream
      AnnotationSupport.isAnnotated(context.requiredTestClass, IgnoreNoPactsToVerify::class.java) ->
        listOf(DummyTestTemplate).stream() as Stream
      else -> throw NoPactsFoundException("No Pact files were found to verify\n${tests.second}")
    }
  }

  private fun resolvePactSources(context: ExtensionContext): Pair, String> {
    var description = ""
    val providerInfo = AnnotationSupport.findAnnotation(context.requiredTestClass, Provider::class.java)
    if (!providerInfo.isPresent) {
      throw UnsupportedOperationException("Provider name should be specified by using @${Provider::class.java.name} annotation")
    }
    val serviceName = providerInfo.get().value
    description += "Provider: $serviceName"

    val consumerInfo = AnnotationSupport.findAnnotation(context.requiredTestClass, Consumer::class.java)
    val consumerName = consumerInfo.orElse(null)?.value
    if (consumerName.isNotEmpty()) {
      description += "\nConsumer: $consumerName"
    }

    validateStateChangeMethods(context.requiredTestClass)

    logger.debug { "Verifying pacts for provider '$serviceName' and consumer '$consumerName'" }

    val pactSources = findPactSources(context).flatMap {
      val valueResolver = getValueResolver(context)
      if (valueResolver != null) {
        it.setValueResolver(valueResolver)
      }
      description += "\nSource: ${it.description()}"
      val pacts = it.load(serviceName)
      filterPactsByAnnotations(pacts, context.requiredTestClass).map { pact -> pact to it.pactSource }
    }.filter { p -> consumerName == null || p.first.consumer.name == consumerName }

    return Pair(pactSources.flatMap { pact ->
      pact.first.interactions.map { PactVerificationExtension(pact.first, pact.second, it, serviceName, consumerName) }
    }, description)
  }

  protected open fun getValueResolver(context: ExtensionContext): ValueResolver? = null

  private fun validateStateChangeMethods(testClass: Class<*>) {
    val errors = mutableListOf()
    AnnotationSupport.findAnnotatedMethods(testClass, State::class.java, HierarchyTraversalMode.TOP_DOWN).forEach {
      if (it.parameterCount > 1) {
        errors.add("State change method ${it.name} should either take no parameters or a single Map parameter")
      } else if (it.parameterCount == 1 && !Map::class.java.isAssignableFrom(it.parameterTypes[0])) {
        errors.add("State change method ${it.name} should take only a single Map parameter")
      }
    }

    if (errors.isNotEmpty()) {
      throw UnsupportedOperationException(errors.joinToString("\n"))
    }
  }

  private fun findPactSources(context: ExtensionContext): List {
    val pactSource = context.requiredTestClass.getAnnotation(PactSource::class.java)
    logger.debug { "Pact source on test class: $pactSource" }
    val pactLoaders = context.requiredTestClass.annotations.filter { annotation ->
      annotation.annotationClass.findAnnotation() != null
    }
    logger.debug { "Pact loaders on test class: $pactLoaders" }

    if (pactSource == null && pactLoaders.isEmpty()) {
      throw UnsupportedOperationException("At least one pact source must be present on the test class")
    }

    return pactLoaders.plus(pactSource).filterNotNull().map {
      if (it is PactSource) {
        val pactLoaderClass = pactSource.value
        try {
          // Checks if there is a constructor with one argument of type Class.
          val constructorWithClass = pactLoaderClass.java.getDeclaredConstructor(Class::class.java)
          if (constructorWithClass != null) {
            constructorWithClass.isAccessible = true
            constructorWithClass.newInstance(context.requiredTestClass)
          } else {
            pactLoaderClass.createInstance()
          }
        } catch (e: NoSuchMethodException) {
          logger.error(e) { e.message }
          pactLoaderClass.createInstance()
        }
      } else {
        it.annotationClass.findAnnotation()!!.value.java
          .getConstructor(it.annotationClass.java).newInstance(it)
      }
    }.map {
      checkForOverriddenPactUrl(it,
        context.requiredTestClass.getAnnotation(AllowOverridePactUrl::class.java),
        context.requiredTestClass.getAnnotation(Consumer::class.java))
      it
    }
  }

  override fun supportsTestTemplate(context: ExtensionContext): Boolean {
    return AnnotationSupport.isAnnotated(context.requiredTestClass, Provider::class.java)
  }

  companion object : KLogging()
}