Add a button to run a subset of tests

[rob-miller]

Issue Type: Feature Request

For a large project with many unit tests, it would be nice if a subset of tests could be pre-selected and then run by clicking a button. Exactly like the current 'Run Tests' button in the bottom status bar (or perhaps a configuration option for it), but so that it only runs the tests (my chosen subset) relevant to the section of the project that I am currently working on.

Extension version: 2019.8.30787
VS Code version: Code 1.37.1 (f06011ac164ae4dc8e753a3fe7f9549844d15e35, 2019-08-15T16:16:34.800Z)
OS version: Darwin x64 18.7.0

[kimadeline]

Hello @rob-miller, thank you for the suggestion (I will also link the original ticket here: #7176)! We have marked this issue as "needs decision" to make sure we have a conversation about your idea. We plan to leave this feature request open for at least a month to see how many 👍 votes your idea gets to help us make our decision.

[DonJayamanne]

@rob-miller have you used the test explorer for running of tests, this allows you to run specific tests.
I feel that adding a ui to select specific tests is too specific. Generally one structures tests into suites (folders/files) and then run a specific suites or the like. The test explorer supports this.
Please do check it out, if you have done so, them please could you let is know how/why this doesn't meet your requirements so we can learn and improve on the current solution.

[rob-miller]

@DonJayamanne Thanks, yes and the #7176 ticket raises an issue where (in my hands) running a file of tests from the test explorer does not appear to be running all the tests in the file. Leaving that bug out, the issue here is that I am collaborating on a large project (biopython) with 287 executable test files in the Tests subdirectory. Twelve of these are specific to the directory I'm working in and perhaps three directly hit code I'm likely to break. Despite the advice that 'tests should run quickly,' several of the 287 (and 1 of the 12) access web services so I've never managed to wait through running the complete set. This has been discussed on the group lists but isn't something I can fix (or I don't know how to do a simple local-only solution).

So for me the point of the 'run tests' button in the status bar is that I will hit it frequently - and I preach in #7176 why I think making unit testing super easy in VS Code is such a great thing to do. For one file of tests (mine for the module I'm developing) test explorer is fine, but the activation barrier rises exponentially with each additional out-of-list-sequence test file that I probably should run, so I fall back to never doing testing. Now the too-slow 'Run Tests' button becomes just a to-do sign listing another waiting task, when it could/should be an instant gratification hit when I break from coding.

I agree the selection UI is where the work is, for me I'm happy with a list of test file names in Settings but I expect normal people would prefer a right-click menu option or a checkbox next to the current run/play/open icons. If I've missed some kind of 'virtual folder' layer or other mechanism in test explorer to help with this I apologise.

[luabud]

Thanks for the suggestion 👍 but we decided not to move forward with this idea for now.