org.igniterealtime.smack.inttest.package-info Maven / Gradle / Ivy
/**
*
* Copyright 2015-2023 Florian Schmaus
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
/**
* The Smack Integration Test Framework (sinttest) used to run a set of tests against a real XMPP service. The framework
* discovers on start-up the available tests by reflection.
* Quickstart
*
* You can run the framework against an XMPP service with
*
*
*
* $ gradle integrationTest -Dsinttest.service=my.xmppservice.org
*
*
* Note that the service needs to have In-Band Registration (IBR) enabled.
*
*
* A better alternative to IBR is using XEP-0133: Service Administration to create the throw away accounts used by the
* integration test framework. Simply use
*
*
*
* $ gradle integrationTest -Dsinttest.service=my.xmppservice.org \
* -Dsinttest.adminAccountUsername=admin \
* -Dsinttest.adminAccountPassword=aeR0Wuub
*
*
* to run Smack's integration test framework against my.xmppservice.org with an admin account named 'admin' and
* 'aeR0Wuub' as password.
*
* Configuration
*
* The framework is configured with a standard Java properties file. This file simply contains key/value pairs, which
* are separated by an equals sign ("="). The most important configuration value is the `service` value, it's also the
* only required setting.
*
*
* The file properties can be overridden with Java system properties. The name of a system property that is used by the
* framework needs to be prefixed with 'sinttest.' (*S*mack *Int*egration *Test* Framework). For example the `service`
* property becomes `sinttest.service`.
*
* Minimal example 'properties' file
*
*
* service = example.org
*
*
* Another example 'properties' file
*
*
* service=example.org
* serviceTlsPin=CERTSHA256:2F:92:C9:4D:30:58:E1:05:21:9A:57:59:5F:6E:25:9A:0F:BF:FF:64:1A:C3:4B:EC:06:7D:4A:6F:0A:D5:21:85
* debugger=console
*
*
* Framework properties
*
* Properties of the Smack Integration Test Framework
*
* Name
* Description
*
*
* service
* XMPP service to run the tests on
*
*
* host
* IP address or DNS name of the XMPP service to run the tests on
*
*
* serviceTlsPin
* TLS Pin (used by java-pinning)
*
*
* securityMode
* Either ‘required’ or ‘disabled’
*
*
* replyTimeout
* In milliseconds
*
*
* adminAccountUsername
* Username of the XEP-0133 Admin account
*
*
* adminAccountPassword
* Password of the XEP-0133 Admin account
*
*
* accountOneUsername
* Username of the first XMPP account
*
*
* accountOnePassword
* Password of the first XMPP account
*
*
* accountTwoUsername
* Username of the second XMPP account
*
*
* accountTwoPassword
* Password of the second XMPP account
*
*
* accountThreeUsername
* Username of the third XMPP account
*
*
* accountThreePassword
* Password of the third XMPP account
*
*
* debugger
* ‘console’ for console debugger, ‘enhanced’ for the enhanced debugger, or the name of a class that implements SmackDebuggerFactory for a custom debugger
*
*
* enabledTests
* List of enabled tests
*
*
* disabledTests
* List of disabled tests
*
*
* enabledSpecifications
* List of specifications for which to enable tests
*
*
* disabledSpecifications
* List of specifications for which to disable tests
*
*
* defaultConnection
* Nickname of the default connection
*
*
* enabledConnections
* List of enabled connection’s nicknames
*
*
* disabledConnections
* List of disabled connection’s nicknames
*
*
* testPackages
* List of packages with tests
*
*
* verbose
* If true set output to verbose
*
*
* dnsResolver
* One of ‘minidns’, ‘javax’ or ‘dnsjava’. Defaults to ‘minidns’.
*
*
* testRunResultProcessors
* List of class names for generating test run output. Defaults to 'org.igniterealtime.smack.inttest.SmackIntegrationTestFramework$ConsoleTestRunResultProcessor'
*
*
* Where to place the properties file
*
* The framework will first load the properties file from ~/.config/smack-integration-test/properties.
*
* Running selected tests only
*
* Using enabledTests is is possible to run only selected tests. The tests can be selected on a per class
* base or by specifying concrete test methods. In the latter case, the methods must be qualified by a (simple) class
* name.
*
*
* For example:
*
*
*
* $ gradle integrationTest -Dsinttest.enabledTests=SoftwareInfoIntegrationTest.test
*
*
* will only run the test() method of SoftwareInfoIntegrationTest, whereas
*
*
*
* $ gradle integrationTest -Dsinttest.enabledTests=SoftwareInfoIntegrationTest
*
*
* would run all tests defined in the SoftwareInfoIntegrationTest class.
*
*
* Use enabledSpecifications to run all tests that assert implementation of functionality that is described
* in standards identified by the provided specification-reference.
*
*
* For example:
*
*
*
* $ gradle integrationTest -Dsinttest.enabledSpecifications=XEP-0045
*
*
* would run all tests that are annotated to verify functionality specified in XEP-0045: "Multi-User Chat".
*
* Overview of the components
*
* Package org.igniterealtime.smack.inttest
*
* SmackIntegrationTestFramework
*
* Contains public static void main method, i.e. the entry point for the framework. Here the available
* integration tests are discovered by means of reflection, the configuration is read and a
* IntegrationTestEnvironment instance created, which includes the XMPP connections.
*
* AbstractSmackIntegrationTest
*
* The base class that integration tests need to subclass.
*
* AbstractSmackLowLevelIntegrationTest
*
* Allows low level integration test, i.e. every test method will have its own exclusive XMPPTCPConnection instances.
*
* AbstractSmackSpecificLowLevelIntegrationTest
*
* Operates, like AbstractSmackLowLevelIntegrationTest, on its own XMPPConnection instances,
* but is limited to a particular type of XMPPConnection.
*
* IntegrationTestEnvironment
*
* The environment, e.g. the `XMPPConnections` provided to the integration tests by the framework. Note that for
* convenience `AbstractSmackIntegrationTest` contains some of those as protected members.
*
* SmackIntegrationTest
*
* An annotation that needs to be added to all methods that represent a single integration test. Annotated integration
* test methods must not take any arguments, i.e., their parameter count is 0, and should return void as it's not
* evaluated in any way. The methods are supposed to throw an exception if their integration test fails.
*
* TestNotPossibleException
*
* Can be thrown by test methods or constructors to signal that their test is not possible, e.g. because the service
* does not support the required feature.
*
* Running the integration tests
*
* Smack's Gradle build system is configured with a special task called `integrationTest`, which means you can run the
* tests simply with
*
*
* {@code
* $ gradle integrationTest -Dsinttest.service=my.xmppservice.org
* }
*
* If one of accountOneUsername, accountOnePassword, accountTwoUsername or
* accountTwoPassword is not configured, then the framework will automatically create the accounts on the
* service. Of course this requires account registration (IBR) to be enabled. If the accounts got created automatically
* by the framework, then they will also be deleted at the end of the test.
*
* Implementing Integration Tests
*
* Create a new class which extends AbstractSmackIntegrationTest. Every non-static method, including the
* constructor, of this class will have two XMPPConnections available to perform the integration tests with:
* conOne and conTwo. You can use the constructor to check if the XMPP service does provide
* the required XMPP feature. If it does not, simply throw a TestNotPossibleException.
*
*
* Test methods must be public, take zero arguments i.e. declare no parameters and be annotated with
* @SmackIntegrationTest. If the test method is not able to perform a test then it should throw a
* TestNotPossibleException.
*
* Rules for integration tests
*
* Tests should not leave any traces on the service if they are finished, i.e. the service state at the end of the test
* must be equal to the state of the beginning. It must be possible to run the tests in parallel.
*
* Why are there two mechanisms to signal that the test is not possible?
*
* Because the XMPP service may provide a component that is required to perform a certain integration test, but that
* component may not support all features. For example, the XMPP service may provides a PubSub (XEP-0060) component, but
* this component may not support all features of XEP-0060.
*
* Low-Level Integration Tests
*
* Classes that implement low-level integration tests need to subclass
* {@link org.igniterealtime.smack.inttest.AbstractSmackLowLevelIntegrationTest}. The test methods can declare as many
* parameters as they need to, but every parameter must be of type XMPPTCPConnection. The framework will
* automatically create, register and login the connections. After the test is finished, the connections will be
* unregistered with the XMPP service and terminated.
*
* Debugging Integration Tests
*
* A test, like any other code, may not be perfect on the first attempt, and you may require more information in order
* to ascertain quite what's wrong.
*
* Smack Debugger Options
*
* As described in the package documentation of org.jivesoftware.smack.debugger, there are two built-in debuggers that
* could surface you more information. Using the 'enhanced' debugger config option listed above, you'll get the Smack
* Debug Window launching when your tests launch, and you'll get a stanza-by-stanza account of what happened on each
* connection, hopefully enough to diagnose what went wrong.
*
*
* Lastly, you can provide a custom debugger, by providing the fully qualified name of a class that implements
* {@link org.jivesoftware.smack.debugger.SmackDebuggerFactory}. The provided factory must declare a public constructor
* that takes no arguments.
*
*
* Example:
*
*
* {@code
* $ gradle integrationTest -Dsinttest.service=my.xmppservice.org -Dsinttest.debugger="org.example.MyDebugger$Factory"
* }
* Debugging in the IDE
*
* If the output isn't enough, you may need to debug and inspect running code within the IDE. Depending on the IDE, in
* order to get execution to pause at your breakpoints, you may need to switch your configuration. Instead of running
* `gradle integrationTest`, instead run the `SmackIntegrationTestFramework` class directly with the same command-line
* options.
*
* Running Your Own Integration Tests
*
* The framework can be used to run your own tests residing outside of the framework's default package scope. Simply set
* the testPackages property to a comma separated list of package names where the framework should look for
* integration tests.
*
*
* Example:
*
*
* {@code
* $ gradle integrationTest -Dsinttest.service=my.xmppservice.org -Dsinttest.testPackages=org.mypackage,org.otherpackage
* }
* Generating test run reports
*
* By default, the results of the test run is printed to standard-error. You can, however, provide your own processing
* for the test run results. To do so, create an implementation of
* {@link org.igniterealtime.smack.inttest.SmackIntegrationTestFramework.TestRunResultProcessor} and provide its class
* name to the testRunResultProcessor property. This property takes a comma separated list of class names.
*
*
* Example:
*
* {@code
* $ gradle integrationTest -Dsinttest.service=my.xmppservice.org -Dsinttest.testRunResultProcessor=org.igniterealtime.smack.inttest.SmackIntegrationTestFramework$ConsoleTestRunResultProcessor
* }
*/
package org.igniterealtime.smack.inttest;