Support multiple test environments

[brettcannon]

Discussed in #16507

^{Originally posted by brettcannon June 16, 2021}
When I use test explorer in Python it allows me to run tests just in one way, but I cannot believe I'm the only one who need to run tests with different pytest args or with different env variables. PyCharm for example allows to create different test runners with specific arguments and environment variables in each one.

Originally posted by @cperezabo in a discussion

[brettcannon]

Thanks for the feature request! We are going to give the community 60 days from when this issue was created to provide 5 👍 upvotes on the opening comment to gauge general interest in this idea. If there's enough upvotes then we will consider this feature request in our future planning. If there's unfortunately not enough upvotes then we will close this issue.

[brettcannon]

Thank you to everyone who upvoted this issue! Since the community showed interest in this feature request we will leave this issue open as something to consider implementing at some point in the future.

We do encourage people to continue 👍 this issue as it helps us prioritize our work based on what the community seems to want the most.

[cperezabo]

The community seems to have been showing interest for years, I've found this old issue with the same request. #12075

I would like to know how @brettcannon or others are running tests with vscode-python... I think they don't even use the extension, because it's impossible to use python professionally in vscode without these type of features.

It only works for students who are using just one test suite while learning.

[brettcannon]

Closing as a duplicate of #12075 .

@cperezabo that issue is the 17th most upvoted issue, 12th if I merge in the upvotes here with the other issue (assuming no duplicate votes). So it's up there, but it doesn't break the top 10. We are also in the process of completely rewriting the test code right now for more reliable testing support, so new features are not a priority while we are doing the rewrite.

As for running tests using the extension, I just run tests w/o issue. I typically don't need multiple environments while I'm developing, only as a final step before publishing to something like PyPI. And in that instance most projects I use has set up Nox or tox to handle multi-Python version testing as that's what CI will run in the end.

And please take care with how you state things to others. Your comments come of as unnecessarily negative which is demotivating to want to prioritize this work. The insinuation that the extension is no good because it doesn't work for you personally isn't needed. I for one use the extension to develop Python itself as well as critical parts of the packaging stack that you are very likely using.

[cperezabo]

@brettcannon if you want to do something as easy as mark some tests as slow and run them once in a while you can't, because you need 2 different runner configurations, one for fast unit tests, and then another that include the slower ones, or running mocked tests most of times and then running them with final implementations that consume services from your docker-compose setup. I think it's quite normal and not a "personal" scenario. That's why I asked you how were you working without such feature. Maybe I am missing the point.