Tracking Issue for shuffle and shuffle-seed

[smoelius]

This is a tracking issue for unstable libtest options --shuffle and --shuffle-seed.

These options allow tests to be run in random order, as opposed to the default alphabetical order.

Steps / History

• Implementation: Implement #85440 (Random test ordering) #89082
• Final comment period (FCP)
• Stabilization PR

Unresolved Questions

• None yet.

[smoelius]

Following up on #89082 (comment) (cc: @joshtriplett).

This appears to be the commit that introduced lld's --shuffle-sections: llvm/llvm-project@d48d339

Here are some differences I noticed between their implementation and ours:

• They use a 32-bit seed, we use 64 bits. Using a 64-bit seed seems reasonable since, e.g., that is what Hasher::finish outputs. Note that Go also uses a 64-bit seed.
• They disallow 0 as a seed; we allow it. It is conceivable that a developer might test with a fixed set of seeds, and that 0 might be one of them. Hence, it seems preferable to allow 0 as a seed.
• They use Mersenne Twister for their random number generator; we use a kind of iterated SipHash13. Using a standard random number generator would be preferable, but we are limited by what code is available to a core library. See this comment for some discussion.