Add --test-format CLI option for customizable test name display

[Copilot]

Summary

This PR implements a new CLI option --test-format that allows users to customize how test names are displayed in console output during test discovery (--list-tests) and execution (--output detailed).

Implementation

Changes Made

• Add new command line option --test-format to PlatformCommandLineProvider
• Create a TestNameFormatter utility class to handle format string parsing and replacement
  • Support placeholders: <fqn>, <ns>, <type>, <method>, <display>, <asm>
  • Extract information from TestNode and TestMethodIdentifierProperty
• Add localized resource strings for the new option (with placeholder English text for all languages)
• Update TerminalOutputDevice to use formatter instead of DisplayName directly
• Pass TestNode to TerminalTestReporter methods (instead of just DisplayName string)
• Update TerminalTestReporter to format test names using the formatter
• Add unit tests for TestNameFormatter (11 test cases)
• Add integration tests for --test-format option (4 test cases)
• Update changelog
• Address code review feedback (use Guard instead of ArgumentGuard)
• Update help and info acceptance tests for MTP and MSTest

Supported Placeholders

• <display> - Display name (default)
• <fqn> - Fully qualified name (namespace.type.method with parameters)
• <ns> - Namespace
• <type> - Type name
• <method> - Method name
• <asm> - Assembly name (short name without version/culture/token)

Usage Examples

# Show only display names (default)
dotnet test --list-tests

# Show fully qualified names
dotnet test --list-tests --test-format "<fqn>"

# Show display name with FQN in parentheses
dotnet test --list-tests --test-format "<display> (<fqn>)"

# Show test in custom format
dotnet test --list-tests --test-format "[<asm>] <ns>.<type>.<method>"

Related Issues

• Addresses combining --list-tests and --filter based on fully qualified name? #4111
• Addresses Provide extra options to --list-tests #3221

Notes

• The implementation uses the <placeholder> syntax as suggested in the agent instructions
• All placeholders are replaced with empty strings if TestMethodIdentifierProperty is not available on the TestNode
• The feature works for both test discovery and test execution output
• Input validation ensures null safety for the format string and test nodes
• Help and info output now documents the new --test-format option

Original prompt

---

This section details on the original issue you should resolve

<issue_title>Add support for a test name format</issue_title>
<issue_description>## Summary

Introduce a new CLI argument allowing to change the way tests are displayed on console.

Background and Motivation

As of today, when doing --list-tests or when using the --output detailed the tests are displayed using their display name which is matching behavior of VSTest.

There are many cases where this is not ideal and it would be great to be able to change that. For example, if you want to discover the tests in a given assembly or through a given filer, you will see only display names when it would be useful to see the fully qualified name. When having multiple tests with a same display name, having the fully qualified name would also be useful.

Related issues or discussions: #4111 #3221

Proposed Feature

Add a new CLI option named --test-format (I am open to different naming suggestions) allowing the users to specify a format in which they want the test to be displayed. This argument would be in an open format with some predefined variable replacements (similar to git log pretty format).

To get started, I think it would be enough to support display name, fully qualified name and test ID as they are common to most test frameworks. I would like to suggest using long form names instead of few char symbols (unlike git log pretty) to have something self explanatory.

We will have to define the good pattern for variable replacement, possibilities from existing tools are:

• $<variable-name> like unix
• %<variable-name> like git log
• %<variable-name>% like cmd
• $<variable-name>$ like some templating systems
• ${<variable-name>} like unix/PS (although that's not the only syntax)

Example:

For the example below, I'll use the git log like syntax

// default
--test-format %displayname

--test-format "test: %displayname (%fullyqualifiedname)"

Alternative Designs

We could remove the flexible part and only allow some enum based presets: DisplayName (default), FQN, ID. This is probably simpler but could remove a big chunk of flexibility to user (is this so bad?).

One note is that my ideal solution would allow me to have something like <display name> (<fqn>) but we could do some preset for that.</issue_description>

<agent_instructions>Let's use the syntax <name>, use this PR #6587 as an example. For now I'd like to support the following replacements:

• fully qualified name (namespace + "." + type + "." + method). If the method as parameters, then they are added to the FQN between parenthesis and using FQN of their type
• type name
• namespace
• method name
• display name
• assembly name

On the other PR we are using short names so it'd be best if we could do the same here but I'd prefer to avoid confusing names if there is no obvious or natural short name</agent_instructions>

Comments on the Issue (you are @copilot in this section)

@nohwnd It would be nice to have more use cases for this.

To me it seems much easier and more flexible to change the default to <display name> (<fqn>). The new dotnet test experience does this, with counts per assembly+tfm, and then on the end shows the total count.

And for more flexibility add option to render the the info in JSON with all the info we have about the test, and let the consumer process it.

It also seems more future proof, as long as we get the basic json format correctly.

This could also then be similar for run, where acceptance tests often parse out console output, and break every time we change that.

(we took similar path in Pester where we return an object that describes the test run, for the reason that we have no idea how people will want to process the result)</comment_new>

• Fixes Add support for a test name format #4130

---

💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

[Evangelink]

@copilot add unit tests and acceptance tests

[Copilot]

@copilot add unit tests and acceptance tests

Unit tests and acceptance tests have already been added:

• Unit tests: 11 test methods in TestNameFormatterTests.cs (commit a204554)
• Acceptance tests: 4 test methods in TestDiscoveryTests.cs and OutputTests.cs (commit 0153320)

The tests cover all placeholders (, , , , , ) and various scenarios including parameter handling and fallback behavior.

[Evangelink]

@copilot please update MTP and MSTest acceptance tests on help and info to mention this new CLI option

[Copilot]

@copilot please update MTP and MSTest acceptance tests on help and info to mention this new CLI option

Updated MTP and MSTest acceptance tests for help and info to include the --test-format option (commit bde2b00). The option now appears in:

• Microsoft.Testing.Platform.Acceptance.IntegrationTests/HelpInfoTests.cs (help and info tests)
• MSTest.Acceptance.IntegrationTests/HelpInfoTests.cs (help and info tests)

[nohwnd]

For the reasons described in the issue, I think this will just add more maintenance than it is worth.

[Evangelink]

I think changing the default is good moving forward but it could lead to breaking changes for many customers, internal and external, who parse current output. To avoid disruption we would need to postpone to v3 or introduce a way to change the default (e.g. this PR),

Regarding the JSON output, that's IMO a different feature request and we have multiple tickets for that.

If the current implementation is causing maintenance headaches, I’m happy to revisit and consider removing it to reduce complexity. In the meantime, I see value for me and few internal team so I'd like to move this forward.

[nohwnd]

By the same logic this change #6717 will also break teams that are parsing assertion output (also more than few).

if you move forward please rename the option to --list-tests-test-format, so it aligns with other parameters that use that "main name" for the main option, and main name - suffix for sub-options.

And that we also don't close doors for --list-tests-format json.

[Evangelink]

It's not about changing the name when doing lsit test, it will also impact how we show tests when passed/failed... Hence the current name.

For the assertion messages, I partially agree. Yes if they are looking at the message there are issues but all internal and external scripts I know of are based on parsing result of --list-tests.

[nohwnd]

Ah okay, I was confused with the examples being only for list tests. And the issue closing only the extra options to --list-tests issue.

In that case I am even more against that, because it conflicts with the functionality that is already built into the terminal reporter (e.g. showing assembly, and --output detailed) and will make it difficult to provide consistent behavior in the ansi mode, e.g. the tests in progress cannot easily adopt this output, because they should not overflow width of 1 row.

[Evangelink]

tests in progress cannot easily adopt this output, because they should not overflow width of 1 row.

We already check if that's too long in which case we strip with ... so that's not a different problem that what already exist.