-
-
Notifications
You must be signed in to change notification settings - Fork 1.6k
Description
Overview
As requested in issue #681 (Introduce junit-platform-suite-api module
), the annotations used for test discovery and selection in the JUnitPlatform
runner have been refactored into the junit-platform-suite-api
. This allows a purely JUnit 5 test suite to be defined declaratively, though it currently requires an alternate engine (as opposed to an extension).
This issue describes a generalized JUnit platform suite that's suitable for the Jupiter test engine as well as any others that might be chosen. It's my opinion that unit tests rarely require test suites, but it becomes far more common when integration, system and acceptance tests are being executed due to the very common integration with external systems that should only be initialized before the entire test run. Examples of resources that can be safely shared among tests are database connection pools (or really any connection pool), test frameworks that require significant set-up time (e.g. GWT test suites eliminate 20s of HtmlUnit set-up time for each test - it's only incurred once at the beginning of each suite).
The examples shown below indicate how the @Suite
annotation can be used in conjunction with those already in the junit-platform-suite-api
module. Originally I'd also defined the @BeforeSuite
and @AfterSuite
annotations but after further consideration decided that since suites are really just an arbitrary collection of indeterminately nested test containers, it would be better if the @BeforeAll
and @AfterAll
annotations were contextual (nested in the same way the hierarchy would be presented in the test tree). I've also assumed that these annotations no longer require the static modifier as discussed elsewhere.
Imagine if you will a set of acceptance tests that have a custom reporting mechanism that should be configured for the entire test run. Included within this outer suite is a set of tests that require a database connection pool and a different set of tests that require an LDAP connection pool. The top-level suite definition for the acceptance tests might look something like:
@Suite
@IncludeEngines("jupiter-engine") // not required if there's only one engine on the classpath
@IncludeClassNamePatterns(".*AcceptanceTestSuite$")
public class AcceptanceTestSuite {
@BeforeAll
void setUp() {
... // code to set-up custom test reporting
}
@AfterAll() {
void tearDown() {
... // code to tear-down custom test reporting
}
}
There are two notable characteristics of this test suite:
- It specifies which engine will be used for the entire container/test hierarchy (assuming there is more than one).
- It includes potentially numerous other test suites as sub-containers which can each have their own contextual set-up and tear-down.
A child suite for tests that require a database connection pool might look something like this:
@Suite
@IncludeTags({"integration", "database"})
@IncludeClassNamePatterns(".*IT$")
public class DatabaseAcceptanceTestSuite {
@BeforeAll
void setUp() {
... // code to set-up the database connection pool
}
@AfterAll() {
void tearDown() {
... // code to tear-down the database connection pool
}
}
If run under Maven, test classes that end in "IT" would be run using the maven-failsafe-plugin
. If there are other classes of integration tests not run during acceptance testing it would also be necessary to make sure the acceptance test classes were not run (as they'd fail without the suite's set-up methods).
Feature Request
Provide the means for arbitrarily nested, declarative tests suites in JUnit 5 including the execution of set-up and tear-down methods
Related Issues
- Introduce support for programmatic test suites #330
- Introduce
@BeforeSuite
and@AfterSuite
annotations #456 - Improve JavaDoc for junit-platform-suite-api annotations #746
- Introduce an extension point to allow test container creation #687 (perhaps superseded by this issue)
- Introduce support for parallel test execution in declarative test suites #964
- Eclipse IDE issue
- Introduce file-based declarative test suite support #1243
Deliverables
- TBD