[pytest] If present, "python.testing.cwd" expected to change `pytest --rootdir` arg value.

[chrisgervang]

Environment data

• VS Code version: 1.41.0
• Extension version (available under the Extensions sidebar): 2020.1.57204
• OS and version: MacOS 10.14.6
• Python version (& distribution if applicable, e.g. Anaconda): 3.7
• Type of virtual environment used (N/A | venv | virtualenv | conda | ...): venv
• Relevant/affected Python packages and their versions: pytest-5.2.2
• Jedi or Language Server? (i.e. what is "python.jediEnabled" set to; more info How to update the language server to the latest stable version #3977): Language Server

Expected behaviour

When running a test with a settings.json that contains the "python.testing.cwd": "root/dir" option should result in a pytest command that uses "root/dir" as the --rootdir argument. If it is null, then fallback to existing behavior: --rootdir ${workspaceFolder}.

This is necessary for my use case since my code is in a monorepo and loads test fixtures from disk using paths relative to a project root rather than the repo root.

For example,

python [~]/.vscode/extensions/ms-python.python-2020.1.57204/pythonFiles/testing_tools/run_adapter.py discover pytest -- --rootdir root/dir -s --cache-clear [...testfolders]

jfyi, I've anonymized [~] and [...testfolders].

Actual behaviour

The "python.testing.cwd" setting has no visible effect on --rootdir. It is always the "workspace folder", which makes sense given the code I linked to below.

Relevant Code

vscode-python/src/client/testing/pytest/services/discoveryService.ts

Line 53 in 2b6a8f2

args.splice(0, 0, '--rootdir', options.workspaceFolder.fsPath);

vscode-python/src/client/testing/pytest/main.ts

Lines 50 to 52 in 2b6a8f2

	const options: TestRunOptions = {
	workspaceFolder: this.workspaceFolder,
	cwd: this.rootDirectory,

vscode-python/src/client/testing/pytest/main.ts

Lines 19 to 20 in 2b6a8f2

	constructor(workspaceFolder: Uri, rootDirectory: string, serviceContainer: IServiceContainer) {
	super(PYTEST_PROVIDER, Product.pytest, workspaceFolder, rootDirectory, serviceContainer);

Steps to reproduce:

[NOTE: Self-contained, minimal reproducing code samples are extremely helpful and will expedite addressing your issue]

1. Open a workspace folder containing python tests that are not in the root directory (i.e. a monorepo).
2. Configure pytest to test a project directory.
   2a. configure "python.testing.cwd" in settings.json to the project root.
3. Discover tests and observe that all tests are discovered.
4. Run all tests.
   4a. Observe the test session starts and all tests pass that do not need to load fixtures from disk.
   4b. Observe the output logs look like this:

python [~]/.vscode/extensions/ms-python.python-2020.1.57204/pythonFiles/testing_tools/run_adapter.py discover pytest -- --rootdir [workspaceFolder] -s --cache-clear [project dir]
============================= test session starts ==============================
platform darwin -- Python 3.7.6, pytest-5.2.2, py-1.8.1, pluggy-0.13.1
rootdir: [workspaceFolder]
plugins: doubles-1.5.3, cov-2.8.1

Logs

Output for Python in the Output panel (View→Output, change the drop-down the upper-right of the Output panel to Python)

XXX

Output from Console under the Developer Tools panel (toggle Developer Tools on under Help; turn on source maps to make any tracebacks be useful by running Enable source map support for extension debugging)

XXX

[chrisgervang]

Somewhat related to #9349 and #6881, but I don't believe I'd expect any auto detection.

[ericsnowcurrently]

@chrisgervang, thanks for getting in touch with us. We'll take a look as soon as possible.

[ericsnowcurrently]

Having looked this over, did you mean to file this as a bug or as a feature request? FWIW, I consider this a feature request. Just to be sure I understood, what you are asking for is that the "python.testing.cwd" setting be treated as the "root" directory against which the tests will be discovered/run. Is that correct?

Assuming yes, I see three options:

1. the extension starts using "python.testing.cwd" as the test root dir (--rootdir for pytest, --top-directory for unittest)
2. the extension adds a new setting to use as the test root
3. the docs/help text are updated to be more clear
4. nothing changes (not great)

If we do something about this then I expect it would be option 2. The reason is, "cwd" already has a specific meaning (essentially "the working directory of the executing program"). In the context of testing this often relates closely with the test root, which is part of the confusion here. (Sorry the help text for the setting, Optional working directory for tests. isn't clear.) The semantics of CWD for program execution need to remain, but may be distinct from the test root, so the best solution will likely be to add a new setting to explicitly define an alternate test root, something like "python.testing.testroot".

Oh, and what in particular connected the "python.testing.cwd" setting to pytest's --rootdir from your perspective?

[ericsnowcurrently]

Also, have you tried the multi-root workspace functionality of VS Code. You create a workspace and then "Add Folder to Workspace...". Consider adding each project root as a workspace root to see if that solves the problem for you.

[chrisgervang]

Gotcha, I agree with you that this is a feature request and #2 sounds like a good addition given there is an existing, different semantic for CWD. It was the quoted language in docs that led to my confusion - I thought this was a bug and the expected behavior was to use CWD for —rootdir. I haven’t tried the multiple workspace feature yet, but will give it a go. Thanks for the suggestion. Looking at the plugin code, I’m not sure how it’ll resolve multiple workspace folders down to one. Does it use the “closest” (down the tree) folder of all the workspace folders added? I’ll be interested to find out. Thanks for looking into this!

…

On Tue, Jan 14, 2020 at 11:16 AM Eric Snow ***@***.***> wrote: Also, have you tried the multi-root workspace functionality of VS Code. You create a workspace and then "Add Folder to Workspace...". Consider adding each project root as a workspace root to see if that solves the problem for you. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#9553?email_source=notifications&email_token=AASY6273ODGS5PDTLSLV2OLQ5YFQFA5CNFSM4KFUZ4YKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEI5Y7FA#issuecomment-574328724>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AASY62ZLCINBWJ6HGMBGRI3Q5YFQFANCNFSM4KFUZ4YA> .

[ericsnowcurrently]

Sounds good. We'll make a decision on the request.

As to multi-root workspace, the extension determines the current workspace based on the file in the active editor. Really it just calls the workspace.getWorkspaceFolder() function in VS Code's API. I'm not sure how it decides it several roots could match for a file.