`ClassOrderer.Random` and `MethodOrderer.Random` do not use same default seed

[kurellajunior]

The documentation is very misleading. It states, that the seed can be set individually for classes and methods, but neither configuration key is actually used:

• ClassOrderer#RANDOM_SEED_PROPERTY_NAME https://github.com/junit-team/junit5/blob/28869631bcecb6e6589ebc249b8e5f9c750b26e8/junit-jupiter-api/src/main/java/org/junit/jupiter/api/ClassOrderer.java#L78
• MethodOrderer#DEFAULT_ORDER_PROPERTY_NAME https://github.com/junit-team/junit5/blob/413ccc0a699fdcfde4d0cef4620ae557e5c10bd5/junit-jupiter-api/src/main/java/org/junit/jupiter/api/MethodOrderer.java#L75

instead for both random seeds is used MethodOrderer.Random#RANDOM_SEED_PROPERTY_NAME https://github.com/junit-team/junit5/blob/413ccc0a699fdcfde4d0cef4620ae557e5c10bd5/junit-jupiter-api/src/main/java/org/junit/jupiter/api/MethodOrderer.java#L297

This is also very inconsistent with the output of the tests which clearly suggests to set the respective seed to a specific value.

I do not know what is tha actual desired behaviour, so cannot provide a PR

[marcphilipp]

The default order can be configured separately for classes and methods using the junit.jupiter.testclass.order.default and junit.jupiter.testmethod.order.default configuration parameters, respectively. Both ClassOrderer and MethodOrderer use the same junit.jupiter.execution.order.random.seed configuration parameter for customizing the seed used with their random orderers, though.

The documentation is very misleading. It states, that the seed can be set individually for classes and methods

Where does it state that?

neither configuration key is actually used

By "both" are you referring to the setting of the default class/method orderer or the custom seed?

This is also very inconsistent with the output of the tests which clearly suggests to set the respective seed to a specific value.

Which tests are you referring to?

[kurellajunior]

The default order can be configured separately for classes and methods using the junit.jupiter.testclass.order.default and junit.jupiter.testmethod.order.default configuration parameters, respectively. Both ClassOrderer and MethodOrderer use the same junit.jupiter.execution.order.random.seed configuration parameters for customizing the seed used with their random orderers, though.

that is understood

This is also very inconsistent with the output of the tests which clearly suggests setting the respective seed to a specific value.

Which tests are you referring to?

When I set the ClassOrderer$Random and MethodOrderer$Random as the test class and testmethod order default respectively, the tests log on the CONFIG level the (different) random seeds used. The assumption is they do so to replay a test run with the defined seeds. This is impossible, as both classes use the same configuration key as a custom seed

The documentation is very misleading. It states, that the seed can be set individually for classes and methods

Where does it state that?

Indeed now I see, it states here the opposite: https://github.com/junit-team/junit5/blob/28869631bcecb6e6589ebc249b8e5f9c750b26e8/junit-jupiter-api/src/main/java/org/junit/jupiter/api/ClassOrderer.java#L220-L221

But actually uses two different seeds when running on fallback. This is then the inconsistency

Neither configuration key is actually used

By "both" are you referring to the setting of the default class/method orderer or the custom seed?

At least I could not find any documentation on how to use ClassOrderer#RANDOM_SEED_PROPERTY_NAME or MethodOrderer#DEFAULT_ORDER_PROPERTY_NAME. Luckily GitHub Copilot knew how to set those in the properties…

[sbrannen]

Thanks for the explanation, @kurellajunior. 👍

But actually uses two different seeds when running on fallback. This is then the inconsistency

That's an actual bug.

I've changed the title of this issue to reflect that, and I've assigned this to the 5.11 M2 milestone.