Using with JUnit 5 @RunWith(JUnitPlatform.class) causes multiple deployment.

[charleech]

Using with JUnit 5 @RunWith(JUnitPlatform.class) causes multiple deployment.

Since the JUnit5 does not have the Test Suite and provide the interim solution by the JUnitPlatform which requires the junit:junit:4.13.2 in the classpath.

• Using JUnit 4 to run the JUnit Platform
• Using JUnit 4 to run the JUnit Platform: Test Suite
• Are test suites considered deprecated in JUnit5?

Then the test suite will be like the following: -

import org.junit.platform.runner.JUnitPlatform;
import org.junit.platform.suite.api.IncludeClassNamePatterns;
import org.junit.platform.suite.api.SelectClasses;
import org.junit.platform.suite.api.SuiteDisplayName;
import org.junit.runner.RunWith;

/**
 * This is a first test suite.
 */
@RunWith(JUnitPlatform.class)
@SuiteDisplayName("JUnit Platform Suite Demo")
@SelectClasses({
    MyCdiJunit5IntgrtnTester.class
})
@IncludeClassNamePatterns("^(Test.*|.+[.$]Test.*|.*Tests?|.*Tester)$")
public class MyCdiJunit5IntgrtnTestSuite {

}

/**
 * This is a second test suite.
 */
@RunWith(JUnitPlatform.class)
@SuiteDisplayName("JUnit Platform Suite Demo")
@SelectClasses({
    MyCdiJunit5IntgrtnTester.class
})
@IncludeClassNamePatterns("^(Test.*|.+[.$]Test.*|.*Tests?|.*Tester)$")
public class MyCdiJunit5Round2IntgrtnTestSuite {

}

By using it, it causes the Arquillian-Suite-Extension to deploy multiple times (each time for each test suite) !

#
# Only first executed test suite success, all the rest are failed
#
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 28.145 s - in it.test.app.github.charleech.my.cdi.MyCdiJunit5IntgrtnTestSuite

#
# The second test suite causes re-deployment !
#
[INFO] Running it.test.app.github.charleech.my.cdi.MyCdiJunit5Round2IntgrtnTestSuite
May 08, 2021 1:47:43 PM org.eu.ingwar.tools.arquillian.extension.suite.DeploymentClassFinder getDeploymentClassFromAnnotation
INFO: arquillian-suite-deployment: Found class annotated with @ArquillianSuiteDeployment: it.test.app.github.charleech.my.cdi.ArquillianSuiteDeployments
[ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.016 s <<< FAILURE! - in it.test.app.github.charleech.my.cdi.MyCdiJunit5Round2IntgrtnTestSuite
[ERROR] MyCdiJunit5IntgrtnTester  Time elapsed: 0.015 s  <<< ERROR!
java.lang.RuntimeException: Could not setup GlassFish Embedded Bootstrap
Caused by: org.glassfish.embeddable.GlassFishException: Already bootstrapped

[ERROR] Picked up JAVA_TOOL_OPTIONS: -Dfile.encoding=UTF-8
[ERROR] OpenJDK 64-Bit Server VM warning: ignoring option MaxPermSize=256m; support was removed in 8.0
[INFO]
[INFO] Results:
[INFO]
[ERROR] Errors:
[ERROR]   Could not setup GlassFish Embedded Bootstrap
[INFO]
[ERROR] Tests run: 3, Failures: 0, Errors: 1, Skipped: 0

The reproducer

I've create the reproducer at https://github.com/charleech/arquillian-junit5-payara

How to execute

1.1 The multiple tester without test suite

This is a base line which use pure JUnit 5 without any dependency to JUnit4 and JUnit 5 Platform.

cd path/to/arquillian-junit5-payara

mvn clean install -N

cd path/to/arquillian-junit5-payara/my-cdi-junit5-multiple-tester

mvn clean install -Dpayara-embedded=true

#
# Everything works fine !
#

1.2 The multiple test suite

This is a reproducer which use JUnit 5 with dependency to JUnit4 and JUnit 5 Platform.

cd path/to/arquillian-junit5-payara

mvn clean install -N

cd path/to/arquillian-junit5-payara/my-cdi-junit5-multiple-testsuite

mvn clean install -Dpayara-embedded=true

#
# Only first executed test suite success, all the rest are failed
#
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 28.145 s - in it.test.app.github.charleech.my.cdi.MyCdiJunit5IntgrtnTestSuite

#
# The second test suite causes re-deployment !
#
# ... Could not setup GlassFish Embedded Bootstrap
# ... Already bootstrapped
# ...
# ... Could not setup GlassFish Embedded Bootstrap
#

1.3 My Environment

• OS: Windows 10 Home Edition 64 Bits
• JDK: AdoptOpenJDK 1.8.0_292

openjdk version "1.8.0_292"
OpenJDK Runtime Environment (AdoptOpenJDK)(build 1.8.0_292-b10)
OpenJDK 64-Bit Server VM (AdoptOpenJDK)(build 25.292-b10, mixed mode)

• Build Tools: Apache Maven 3.8.1

Apache Maven 3.8.1 (05c21c65bdfed0f71a2f2ada8b84da59348c4c5d)
Maven home: C:\Java.Application\Apache\apache-maven\bin\..
Java version: 1.8.0_292, vendor: AdoptOpenJDK, runtime: C:\Java.Application\Sun\Java\jdk8u292-b10\jre
Default locale: en_US, platform encoding: UTF-8
OS name: "windows 10", version: "10.0", arch: "amd64", family: "windows"

• Dependencies
  • Arquillian version 1.7.0.Alpha9
  • Arquillian Suite Extension version 1.2.2
  • JUnit4 version 4.13.2
  • JUnit5 version 5.8.0-M1
  • JUnit5 Platform version 1.8.0-M1
  • Payara version 5.2021.3
  • Payara Aqrquillian Container version 2.4.1

[ingwarsw]

@charleech Plugin works with juint5 (just need few beta dependencies).
Here you have fully working solution https://github.com/ingwarsw/arquillian-suite-extension/tree/master/test-suites/junit5

Im not sure how suites works in junit5.. but seems arquillian and plugin dont use them..

[charleech]

Hi @ingwarsw, I've followed this repository example at https://github.com/ingwarsw/arquillian-suite-extension/tree/master/test-suites/junit5 and implemented as https://github.com/charleech/arquillian-junit5-payara/tree/main/my-cdi-junit5-multiple-tester as the following example: -

@ArquillianSuiteDeployment
public class ArquillianSuiteDeployments {
    @Deployment(name = "payara-evaluation-base")
    public static WebArchive createDeployment() throws IOException {
        return ShrinkWrap.create()....
    }
}
@ExtendWith(ArquillianExtension.class)
public class MyCdiJunit5IntgrtnTester {
    @Inject
    private Greetable greeter;

    @Test
    public void whenGreet() {
        // use the injected greeter and assert
    }
}
@ExtendWith(ArquillianExtension.class)
public class MyCdiJunit5Round2IntgrtnTester {
    @Inject
    private Greetable greeter;

    @Test
    public void whenGreet() {
        // use the injected greeter and assert
    }
}

Everything works like a charm and is able to confirm that our arquillian-suite-extension works perfectly with latest Arquillian , latest Surefire/Failsafe and latest JUnit5 .

[charleech]

Anyhow when I've tried to use the workaround test suite as mentioned at the formal JUnit5 document as the following: -

• https://junit.org/junit5/docs/current/user-guide/#running-tests-junit-platform-runner
• https://junit.org/junit5/docs/current/user-guide/#running-tests-junit-platform-runner-test-suite
• https://stackoverflow.com/questions/50565724/are-test-suites-considered-deprecated-in-junit5

I also have tried to implement the test suite as https://github.com/charleech/arquillian-junit5-payara/tree/main/my-cdi-junit5-multiple-testsuite as the following example: -

import org.junit.platform.runner.JUnitPlatform;
import org.junit.platform.suite.api.IncludeClassNamePatterns;
import org.junit.platform.suite.api.SelectClasses;
import org.junit.platform.suite.api.SuiteDisplayName;
import org.junit.runner.RunWith;

/**
 * This is a first test suite.
 */
@RunWith(JUnitPlatform.class)
@SuiteDisplayName("JUnit Platform Suite Demo")
@SelectClasses({
    MyCdiJunit5IntgrtnTester.class
})
@IncludeClassNamePatterns("^(Test.*|.+[.$]Test.*|.*Tests?|.*Tester)$")
public class MyCdiJunit5IntgrtnTestSuite {

}

/**
 * This is a second test suite.
 */
@RunWith(JUnitPlatform.class)
@SuiteDisplayName("JUnit Platform Suite Demo")
@SelectClasses({
    MyCdiJunit5IntgrtnTester.class
})
@IncludeClassNamePatterns("^(Test.*|.+[.$]Test.*|.*Tests?|.*Tester)$")
public class MyCdiJunit5Round2IntgrtnTestSuite {

}

Sadly, it makes our arquillian-suite-extension to do a multiple deployments.

[charleech]

I'm not sure if it is possible to enhance the arquillian-suite-extension so that it may support the JUnit5 test suite or not.

Could you please help to consider and advise?

[charleech]

I've just found that the JUnit 5 version 5.8.0-M1 and org.junit.platform:junit-platform-suite:1.0.8-M1 introduces @Suite natively (without any dependency to JUnit 4 anymore) as the following :-

• https://junit.org/junit5/docs/snapshot/user-guide/#junit-platform-suite-engine
• Introduce junit-platform-suite-engine junit-team/junit-framework#2416

Then the pure JUnit 5 test suite will be look like the following: -

import org.junit.platform.suite.api.IncludeClassNamePatterns;
import org.junit.platform.suite.api.SelectClasses;
import org.junit.platform.suite.api.Suite;
import org.junit.platform.suite.api.SuiteDisplayName;

/**
 * This is a first test suite.
 */
@Suite
@SuiteDisplayName("JUnit Platform Suite Demo")
@SelectClasses({
    MyCdiJunit5IntgrtnTester.class
})
@IncludeClassNamePatterns("^(Test.*|.+[.$]Test.*|.*Tests?|.*Tester)$")
public class MyCdiJunit5IntgrtnTestSuite {

}

/**
 * This is a second test suite.
 */
@Suite
@SuiteDisplayName("JUnit Platform Suite Demo")
@SelectClasses({
    MyCdiJunit5IntgrtnTester.class
})
@IncludeClassNamePatterns("^(Test.*|.+[.$]Test.*|.*Tests?|.*Tester)$")
public class MyCdiJunit5Round2IntgrtnTestSuite {

}

I also have a chance to update the https://github.com/charleech/arquillian-junit5-payara/tree/main/my-cdi-junit5-multiple-testsuite to use the @Suite and also still face the multiple deployment as the following log :-

[INFO] --- maven-failsafe-plugin:3.0.0-M5:integration-test (integration-test) @ my-cdi-junit5-multiple-testsuite ---
[INFO]
[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------

#
# The first test suite is executed.
#

[INFO] Running it.test.app.github.charleech.my.cdi.MyCdiJunit5IntgrtnTestSuite
[INFO] Running it.test.app.github.charleech.my.cdi.MyCdiJunit5IntgrtnTester
2021-05-10 08:19:23.552 [main] [] INFO  org.reflections.Reflections                   - 232 scan Reflections took 95 ms to scan 1 urls, producing 7 keys and 16 values
May 10, 2021 8:19:23 AM org.eu.ingwar.tools.arquillian.extension.suite.DeploymentClassFinder getDeploymentClassFromAnnotation
INFO: arquillian-suite-deployment: Found class annotated with @ArquillianSuiteDeployment: it.test.app.github.charleech.my.cdi.ArquillianSuiteDeployments
May 10, 2021 8:19:25 AM com.sun.enterprise.v3.server.CommonClassLoaderServiceImpl findH2Client
...
INFO: Clustered CDI Event bus initialized
May 10, 2021 8:19:47 AM org.glassfish.soteria.servlet.SamRegistrationInstaller onStartup
INFO: Initializing Soteria null for context '/myweb'
May 10, 2021 8:19:47 AM com.sun.enterprise.web.WebApplication start
INFO: Loading application [myweb] at [/myweb]
May 10, 2021 8:19:47 AM org.glassfish.deployment.admin.DeployCommand execute
INFO: myweb was successfully deployed in 7,342 milliseconds.
2021-05-10 08:19:48.626 [http-thread-pool::https-listener(5)] [] INFO  app.github.charleech.my.cdi.Greeter           - 31 greet The parameter=JUnit5_1
2021-05-10 08:19:49.281 [http-thread-pool::https-listener(4)] [] INFO  app.github.charleech.my.cdi.Greeter           - 31 greet The parameter=JUnit5_2
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 26.462 s - in it.test.app.github.charleech.my.cdi.MyCdiJunit5IntgrtnTester

#
# The first test suite ends and causes the payara-embedded-all to shutdown
#

May 10, 2021 8:19:50 AM com.sun.enterprise.admin.cli.embeddable.DeployerImpl undeploy
INFO: myweb was successfully undeployed
...
INFO: Shutdown procedure finished
[INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 27.767 s - in it.test.app.github.charleech.my.cdi.MyCdiJunit5IntgrtnTestSuite

#
# The second test suite is executed.
#

[INFO] Running it.test.app.github.charleech.my.cdi.MyCdiJunit5Round2IntgrtnTestSuite
[INFO] Running it.test.app.github.charleech.my.cdi.MyCdiJunit5IntgrtnTester

#
# The multiple deployments occurs.
#

May 10, 2021 8:19:50 AM org.eu.ingwar.tools.arquillian.extension.suite.DeploymentClassFinder getDeploymentClassFromAnnotation
INFO: arquillian-suite-deployment: Found class annotated with @ArquillianSuiteDeployment: it.test.app.github.charleech.my.cdi.ArquillianSuiteDeployments
[ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.079 s <<< FAILURE! - in it.test.app.github.charleech.my.cdi.MyCdiJunit5IntgrtnTester
[ERROR] it.test.app.github.charleech.my.cdi.MyCdiJunit5IntgrtnTester  Time elapsed: 0.079 s  <<< ERROR!
java.lang.RuntimeException: Could not setup GlassFish Embedded Bootstrap
Caused by: org.glassfish.embeddable.GlassFishException: Already bootstrapped

[INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.103 s - in it.test.app.github.charleech.my.cdi.MyCdiJunit5Round2IntgrtnTestSuite
[ERROR] Picked up JAVA_TOOL_OPTIONS: -Dfile.encoding=UTF-8
[ERROR] OpenJDK 64-Bit Server VM warning: ignoring option MaxPermSize=256m; support was removed in 8.0
[INFO]
[INFO] Results:
[INFO]
[ERROR] Errors:
[ERROR]   MyCdiJunit5IntgrtnTester ? Runtime Could not setup GlassFish Embedded Bootstra...
[INFO]
[ERROR] Tests run: 3, Failures: 0, Errors: 1, Skipped: 0
[INFO]