Allow running specific test case of a test table

[ofabricio]

Related issues

Issue	Work to be done
This issue	Statically detect simple table-driven tests
#1602	Support "run at cursor" for entries in a table
#1734	Let the user manually enter a test or subtest name to execute

---

I'd like to suggest a feature that allows running a specific test case in a test table.

When the cursor is in a test case line, a run button gutter could appear allowing you to run that specific case. Or a new option on top of the test function (see image below).

I have tests with a lot of cases and when I have to focus in one to debug it I have either to comment all the other cases or separate that one in a new test function.

[adonovan]

How would such a thing work? It seems like an impossible static analysis problem in all but the most trivial cases.

[ofabricio]

Maybe name convention? Identify tt or testTable or testCases¹ with a []struct{ ... }. Then scan through its items until a { ... } that contains the cursor position and create a new test just with that item or comment out the other items behind scenes?

Footnotes

1. this one is used by the tdt (table driven test) snippet ↩

[hyangah]

Behind the codelens, the extension simply invokes go test or dlv test commands. Even with name convention around tt, testTable, testCases, I don't know how to wire go test and dlv test commands. There is a concept of subtest (testing.T.Run) that go test and dlv test recognizes, but it seems like you aren't using it in the code example.

If users use the subtest, the extension currently has subtest support through the test explorer view.
It does by running all tests and discover the list of dynamically found subtests.

cc @firelizzard18

[firelizzard18]

@ofabricio In your example, it is not possible to run a specific test case without changing tt. In my opinion it is not reasonable for the extension to modify your code (e.g. commenting out the other test cases) or to create a copy of your test with only one case. Not to mention that would have to interpret your code and decide what part of it constituted a test case. If you instead define subtests by making calls to (*testing.T).Run(name string, fn func(*testing.T)), determining what block of code is a subtest is straight forward: the code inside the callback is the subtest. For example:

func TestExample(t *testing.T) {
    tt := []struct{
        give int
        then int
    }{
        {give: 1, then: 3},
        {give: 2, then: 4},
        {give: 3, then: 5},
    }

    for i, tc := range tt {
        t.Run(fmt.Sprintf("Case %d", i+1), func(t *testing.T) {
            v := tc.give + 2
            assertEqual(t, v, tc.then, tc)
        })
    }
}

If you know the name of the sub test you want to run, passing the right arguments to go test is relatively simple. As @hyangah said the test explorer uses the output of go test to dynamically discover subtests. Discovered subtests are listed in the test explorer and can be individually run from there. However, statically discovering what values should be considered a test case is non trivial in any but the simplest situations.

Statically discovering test cases is a complex problem. Finding a static subtest such as t.Run("name", subtest) within TestXxx(t *testing.T) is somewhat feasible, but even then the call might be within a conditional or loop. Any more complex static analysis is not feasible to do on the extension side unless the team decides they want the extension to include a Go test static analyzer written in JavaScript (and someone volunteers to write it).

If we move test discovery out of the extension into gopls, it is more feasible to statically detect subtests. Though that will still be limited in scope as the halting problem makes it (literally) impossible to build a static analyzer that is guaranteed to run in finite time and find every subtest for an arbitrary input. @hyangah is moving test discovery to gopls an option?

[adonovan]

A simpler implementation would be to comment out all of the table except the line you want to execute. :)

[ofabricio]

I didn't know the subtests appear in that view, as I don't use subtests often. I made a few tests with it and, although not so convenient, it kinda meet my needs tbh.

So I don't think there is a need to implement my suggestion unless you think it worth it -- it still looks like a cool, handy feature :p

What if instead of running a case, we add a "Extract test cases" option similar to "Extract function"? It would create a new test with just the selected cases. That would serve two purposes: I could debug one case separately by isolating it (as I currently manually do); or split a big test table (currently I have a test table with 600+ cases)

[firelizzard18]

Test manipulation utilities like "Extract test cases" sounds like a candidate for a separate extension. You might be able to use an approach similar to how I implemented a Go test explorer for the Test Explorer UI: https://gitlab.com/firelizzard/vscode-go-test-adapter/-/blob/5fe67440e8dfa2be14a840e084df165d59eb3c4e/src/model.ts#L111.

[tkgalk]

How would such a thing work? It seems like an impossible static analysis problem in all but the most trivial cases.

I wouldn't say it's impossible, given that GoLand team has done it fairly reliably. Sure, we might not have the manpower, resources and the backend tooling - but it is doable.

[piavgh]

Is there any update on this issue?
This is definitely a great to have feature