Subtests as Symbols

[deiga]

Is your feature request related to a problem? Please describe.
I'm always frustrated when trying to navigate my tests, especially in large files, as there is no way to list and access Subtests (t.Run(..)).

Describe the solution you'd like
I would love it if Subtests could be configured to be listed as Symbols, so that I could search and navigate to them with my keyboard.

Describe alternatives you've considered
Honestly, I haven't considered alternatives.

[firelizzard18]

What do you want to type into the search bar? If you have t.Run("foobar", ...) do you want that to match "foobar"? What if you have t.Run(<expr>, ...) where the expression is a variable or sprintf or some other non-trivial expression?

@h9jiang @adonovan For subtests with names that can be statically determined, I think it might be feasible to could borrow the testfuncs (test discovery) logic to present subtests as seekable symbols (depending on how the seek to symbol functionality works, which is not something I've looked at).

For subtests with names that cannot be statically determined (e.g. subtests dynamically discovered by the test runner) I do not think there is any sane way to do this at the moment. Seeking to a dynamic subtest would require knowing where it is, and currently go test gives us zero information. I would love it if the JSON events emitted by test2json would include the line number of the t.Run call, but currently I don't see any way (certainly not any sane way) to retrieve that information.

Clarification: when I say no sane way, I mean, I am not discounting the possibility that there's some radioactive-level hack to extract this, but I don't see any sane pathway by which this information could be extracted without a change similar to CL 601535 (i.e. updating t.Run to capture the caller's PC and emitting line info in the JSON event).

[deiga]

That's an excellent question.
I hadn't even thought of dynamic test names.

What I'd like to be able to type for static names: just the contents of the string. And I guess it could be lovely if typing the parent name would list all it's subtests.

I wonder if there would be any reasonable way to show hierarchy (indentation?)

Maybe dynamic names could just get placeholders (dynamic test #1) or a string representation of the regex/variable 🤔

[firelizzard18]

I wonder if there would be any reasonable way to show hierarchy (indentation?)

I don' think there is. We're talking about "Go to symbol in Editor/Workspace..." (@ or # in the command palette, right? AFAIK the UX is constrained by VSCode; vscode sends a document symbols request/workspace symbols request to the language server (gopls), which responds with a list of symbols. Looking at gopls's source, it is already returning the richer/hierarchical version (returning DocumentSymbol[]) so I don't think there's anything more gopls can do. It's up to VSCode to implement something like that.

What I'd like to be able to type for static names: just the contents of the string. And I guess it could be lovely if typing the parent name would list all it's subtests.

gopls can return the subtests as a sub-symbol of the test functions (because those will already be included in document symbols). Beyond that, VSCode controls the UX.

[deiga]

I don' think there is. We're talking about "Go to symbol in Editor/Workspace..." (@ or # in the command palette, right?

Yes, exactly. Too bad, but understandable.

Instead of hierarchy, would it be possible to have each Sub-test be <test_function>/<sub_test> similar to how go test outputs test names? Any chance that we could "mark" test_function differently, so that it would show up visually less prominently than sub-test?

[firelizzard18]

The name is whatever gopls reports it to be. The structure is as follows:

export interface DocumentSymbol {
	name: string;
	detail?: string;
	kind: SymbolKind;
	tags?: SymbolTag[];
	deprecated?: boolean;
	range: Range;
	selectionRange: Range;
	children?: DocumentSymbol[];
}

The only defined symbol tag is "deprecated" (which supersedes the boolean field). SymbolKind is what you'd expect; class, method, function, string, number, object, etc. That's all we have to work with (unless the spec is updated).

[h9jiang]

For subtests with names that can be statically determined, I think it might be feasible to could borrow the testfuncs (test discovery) logic to present subtests as seekable symbols (depending on how the seek to symbol functionality works, which is not something I've looked at).

I don't have any objection to make such change, but the limitation is pretty clear. As static analysis tool, we can only support t.Run("ast.Expr") that resolves to constant value.

We could move this to x/tools/gopls to collect other people's opinion.

[adonovan]

The symbol indexer uses only syntax information, allowing it to run on every file in the workspace in parallel. This is important for efficiency, however it does mean it cannot use type information to compute its result. It would be quite feasible to detect a subtest TestA/B in simple cases such as func TestA(t *testing.T) { t.Run("B", ...) }, where the Run call has a literal name and is nested lexically within the main TestA FuncDecl with no intervening FuncLits, but that's about the limit. I don't know how useful that is: in x/tools, only 35% of Run call sites (and presumably a much smaller fraction of dynamic calls) has a literal name.