Skip to content

Random order for tests #7089

@nohwnd

Description

@nohwnd

Summary

Run tests in random order. By randomizing the classes and the the tests within them, as well as the non-parallelizable tests.

Allow reporting and consuming the random seed that is used for the run, to allow repeatable order of tests.

Consider the impact on dotnet test, where we probably want to define one seed for whole solution rather than random seed per project.

Background and Motivation

Running tests in random order can be useful tool to figure out hidden dependencies between tests. Also race conditions in code that are not normally found might be made visible by randomizing the tests.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Area: MSTestIssues with MSTest that are not specific to more refined area (e.g. analyzers or assertions)

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions