Static @BeforeEach method causes entire test plan to abort

[SqAutoTestTeam]

Overview

It is probable that I've encountered a bug related to execution of batch of tests.

If one of these test classes has a static method annotated with @BeforeEach, the whole batch is skipped.

Although the case with static method annotated with @BeforeEach is an error, it must not
ruin the rest of test plan.

Steps to Reproduce

The details are here: https://github.com/SqAutoTestTeam/possiblejunit5bug.git

Current behavior

• Error is written to log
• No tests in batch are executed

Expected behavior

• Error is still written to log
• Execute those test classes that are well-formed (have annotations over mwthods with right modifiers) and
• skip execution for those that are not.

Related Issues

• Introduce error handling mechanism for validation and discovery errors #242

Deliverables

• Ensure that configuration errors related to lifecycle methods (i.e., @BeforeAll, @AfterAll, @BeforeEach, @AfterEach) do not cause the entire test plan to abort prematurely.

[sbrannen]

Thanks for the bug report!

I have confirmed that this is in fact a bug: a configuration error like that should not abort the entire test plan.

[sbrannen]

As an aside, I had to repair your dependency configuration in your example project.

The following works for me:

dependencies {
    compile 'org.junit.platform:junit-platform-engine:1.0.0-M6'
    compile 'org.junit.platform:junit-platform-launcher:1.0.0-M6'

    compile 'org.junit.jupiter:junit-jupiter-api:5.0.0-M6'
    compile 'org.junit.jupiter:junit-jupiter-engine:5.0.0-M6'
}

[sbrannen]

Added Deliverables section

[sbrannen]

Please note that this is in many ways a duplicate of #242.

Consequently, the real solution to the problem will be addressed in #242; however, we can still introduce an interim solution in RC1 snapshots that does not cause the entire test plan to abort.

[sbrannen]

in progress