Skip to content

Introduce support for declarative test suites #744

@smoyer64

Description

@smoyer64

Overview

As requested in issue #681 (Introduce junit-platform-suite-api module), the annotations used for test discovery and selection in the JUnitPlatform runner have been refactored into the junit-platform-suite-api. This allows a purely JUnit 5 test suite to be defined declaratively, though it currently requires an alternate engine (as opposed to an extension).

This issue describes a generalized JUnit platform suite that's suitable for the Jupiter test engine as well as any others that might be chosen. It's my opinion that unit tests rarely require test suites, but it becomes far more common when integration, system and acceptance tests are being executed due to the very common integration with external systems that should only be initialized before the entire test run. Examples of resources that can be safely shared among tests are database connection pools (or really any connection pool), test frameworks that require significant set-up time (e.g. GWT test suites eliminate 20s of HtmlUnit set-up time for each test - it's only incurred once at the beginning of each suite).

The examples shown below indicate how the @Suite annotation can be used in conjunction with those already in the junit-platform-suite-api module. Originally I'd also defined the @BeforeSuite and @AfterSuite annotations but after further consideration decided that since suites are really just an arbitrary collection of indeterminately nested test containers, it would be better if the @BeforeAll and @AfterAll annotations were contextual (nested in the same way the hierarchy would be presented in the test tree). I've also assumed that these annotations no longer require the static modifier as discussed elsewhere.

Imagine if you will a set of acceptance tests that have a custom reporting mechanism that should be configured for the entire test run. Included within this outer suite is a set of tests that require a database connection pool and a different set of tests that require an LDAP connection pool. The top-level suite definition for the acceptance tests might look something like:

@Suite
@IncludeEngines("jupiter-engine") // not required if there's only one engine on the classpath
@IncludeClassNamePatterns(".*AcceptanceTestSuite$")
public class AcceptanceTestSuite {

    @BeforeAll
    void setUp() {
        ... // code to set-up custom test reporting
    }

    @AfterAll() {
    void tearDown() {
        ... // code to tear-down custom test reporting
    }

}

There are two notable characteristics of this test suite:

  1. It specifies which engine will be used for the entire container/test hierarchy (assuming there is more than one).
  2. It includes potentially numerous other test suites as sub-containers which can each have their own contextual set-up and tear-down.

A child suite for tests that require a database connection pool might look something like this:

@Suite
@IncludeTags({"integration", "database"})
@IncludeClassNamePatterns(".*IT$")
public class DatabaseAcceptanceTestSuite {

    @BeforeAll
    void setUp() {
        ... // code to set-up the database connection pool
    }

    @AfterAll() {
    void tearDown() {
        ... // code to tear-down the database connection pool
    }

}

If run under Maven, test classes that end in "IT" would be run using the maven-failsafe-plugin. If there are other classes of integration tests not run during acceptance testing it would also be necessary to make sure the acceptance test classes were not run (as they'd fail without the suite's set-up methods).

Feature Request

Provide the means for arbitrarily nested, declarative tests suites in JUnit 5 including the execution of set-up and tear-down methods

Related Issues

Deliverables

  • TBD

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions