custom pytest wrapper in python.testing.pytestPath not working

[TallJimbo]

Issue Type: Bug

Behaviour

Expected vs. Actual

I have some Python dependencies satisfied by a conda environment (which provides my Python interpreter) and some in directories in PYTHONPATH, and have created a .env file accordingly.

Pytest discovery fails because modules in PYTHONPATH are not importable, and as a workaround I attempted to create a wrapper around pytest. Setting python.testing.pytestPath to this wrapper has no effect; my custom wrapper does not appear in the "Output: Python" logs at all.

Steps to reproduce:

1. Configure the Python interpreter to a conda-provided one.
2. Override python.testing.pytestPath to a custom script.
3. Attempt to run pytest test discovery, and look for that custom script in the Python Output.

Diagnostic data

• Python version (& distribution if applicable, e.g. Anaconda): 3.8.12
• Type of virtual environment used (e.g. conda, venv, virtualenv, etc.): Conda
• Value of the python.languageServer setting: Default

User Settings

defaultLS: {"defaultLSType":"Pylance"}

downloadLanguageServer: true

envFile: "<placeholder>"

venvPath: "<placeholder>"

venvFolders: "<placeholder>"

condaPath: "<placeholder>"

pipenvPath: "<placeholder>"

poetryPath: "<placeholder>"

languageServer: "Pylance"

linting
• enabled: true
• cwd: "<placeholder>"
• Flake8Args: "<placeholder>"
• flake8Enabled: true
• flake8Path: "<placeholder>"
• lintOnSave: true
• banditArgs: "<placeholder>"
• banditEnabled: false
• banditPath: "<placeholder>"
• mypyArgs: "<placeholder>"
• mypyEnabled: false
• mypyPath: "<placeholder>"
• pycodestyleArgs: "<placeholder>"
• pycodestyleEnabled: false
• pycodestylePath: "<placeholder>"
• prospectorArgs: "<placeholder>"
• prospectorEnabled: false
• prospectorPath: "<placeholder>"
• pydocstyleArgs: "<placeholder>"
• pydocstyleEnabled: false
• pydocstylePath: "<placeholder>"
• pylamaArgs: "<placeholder>"
• pylamaEnabled: false
• pylamaPath: "<placeholder>"
• pylintArgs: "<placeholder>"
• pylintPath: "<placeholder>"

sortImports
• args: "<placeholder>"
• path: "<placeholder>"

formatting
• autopep8Args: "<placeholder>"
• autopep8Path: "<placeholder>"
• provider: "black"
• blackArgs: "<placeholder>"
• blackPath: "<placeholder>"
• yapfArgs: "<placeholder>"
• yapfPath: "<placeholder>"

testing
• cwd: "<placeholder>"
• debugPort: 3000
• nosetestArgs: "<placeholder>"
• nosetestsEnabled: undefined
• nosetestPath: "<placeholder>"
• promptToConfigure: true
• pytestArgs: "<placeholder>"
• pytestEnabled: true
• pytestPath: "<placeholder>"
• unittestArgs: "<placeholder>"
• unittestEnabled: false
• autoTestDiscoverOnSaveEnabled: true

terminal
• activateEnvironment: false
• executeInFileDir: "<placeholder>"
• launchArgs: "<placeholder>"

experiments
• enabled: true
• optInto: []
• optOutFrom: []

tensorBoard
• logDirectory: "<placeholder>"

Extension version: 2022.4.0
VS Code version: Code 1.65.2 (c722ca6c7eed3d7987c0d5c3df5c45f6b15e77d1, 2022-03-10T15:36:26.048Z)
OS version: Linux x64 5.10.0-1057-oem
Restricted Mode: No

System Info

Item	Value
CPUs	11th Gen Intel(R) Core(TM) i7-1165G7 @ 2.80GHz (8 x 3598)
GPU Status	2d_canvas: enabled gpu_compositing: enabled multiple_raster_threads: enabled_on oop_rasterization: disabled_off opengl: enabled_on rasterization: disabled_software skia_renderer: enabled_on video_decode: disabled_software vulkan: disabled_off webgl: enabled webgl2: enabled
Load (avg)	3, 3, 2
Memory (System)	15.36GB (1.43GB free)
Process Argv	../DM-34247.code-workspace --crash-reporter-id 2908cf08-efb5-48bb-a135-4322661e19b2
Screen Reader	no
VM	0%
DESKTOP_SESSION	cinnamon
XDG_CURRENT_DESKTOP	X-Cinnamon
XDG_SESSION_DESKTOP	cinnamon
XDG_SESSION_TYPE	x11

A/B Experiments

vsliv368:30146709
vsreu685:30147344
python383:30185418
vspor879:30202332
vspor708:30202333
vspor363:30204092
pythonvspyl392:30443607
pythontb:30283811
pythonptprofiler:30281270
vshan820:30294714
vstes263cf:30335440
vscorecescf:30445987
pythondataviewer:30285071
vscod805cf:30301675
pythonvspyt200:30340761
binariesv615:30325510
bridge0708:30335490
bridge0723:30353136
vsaa593cf:30376535
testflight:30433670
vsc1dst:30438360
pythonvs932:30410667
wslgetstarted:30449410
vsclayoutctrt:30451275
dsvsc009:30452663
pythonvsnew555:30457759
vscscmwlcmc:30438804
vscgsvid2:30447481
pynewfile477:30463512

[kimadeline]

Hi @TallJimbo, thank you for reaching out.

Instead of using a wrapper, is it possible for you to add your paths to PYTHONPATH in the terminal, and launch VS Code from there?

Alternatively, can you expand PYTHONPATH in an environment file, and pass it to pytest?

[TallJimbo]

I'm actually doing both already, and neither is working for pytest, and that's the only reason I resorted to the wrapper. They do work for the language server and flake8 (specifically, the .env file alone is sufficient, because it's all I do when I work remotely).

I'm not sure at all why pytest is different; works fine from the command-line, and it looks like flake8 is also being invoked by vscode via conda run, which was otherwise going to be my guess at the culprit.

[TallJimbo]

Ah, I read

Alternatively, can you expand PYTHONPATH in an environment file, and pass it to pytest?

as meaning a VSCode .env file, but looking more closely at the pytest docs, I see that you can pass it a PYTHONPATH directly somehow. I'll look into that.

[TallJimbo]

Passing a pythonpath = ... entry in via a pytest.ini does work per se, but it just revealed that I also need LD_LIBRARY_PATH for some of the compiled extension modules that I get through PYTHONPATH, and I don't see a way to pass arbitrary envvars into pytest.

I think that means I need either the .env method or the pytest-wrapper-script method to work for me in general, or at least the start-VSCode-from-terminal method if I'm not developing remotely, and none of those work at present.

[kimadeline]

According to this StackOverflow answer you can do the following:

If you put os.environ['BUCKET'] = 'test-bucket' at the top of conftest.py you will set the environment variable for the rest of the test session. Then as long as the first import of the module happens afterwards you won't have a key error. The big downside to this approach is that unless you know to look in conftest.py or grep the code it will be difficult to determine where the environment variable is getting set when troubleshooting.

Would that work?

Other than that, the python.envFile setting should work. If it doesn't, please provide a sample repo (and repro steps) that I could use to replicate your issue.

We also have an open feature request to set env variables specific to the test runner, feel free to upvote it to help us prioritize it: #17171

[TallJimbo]

Thanks for digging it up, but I think the conftest.py approach is getting into territory where it'd be easier for me to keep running pytest outside the editor anyway.

So, with that in mind, I set out trying to create a minimal how-to-reproduce and (as is often the case) discovered that my original diagnosis was pretty inaccurate: python.envFile does work fine, but this was masked by two separate problems:

• I had mine in the wrong place in my multi-folder workspace configuration;
• Starting VSCode from a terminal that already has the environment variables I need somehow breaks this in a way that also leads to failures in pytest discovery that look related to imports. I haven't tracked it down exactly which variables cause the problem, but I wouldn't be surprised if my organization is using conda in a way that is at least unusual, if not actually unsupported.

I'll keep digging into this - I do prefer to launch from a terminal with the right environment in order to get another extension working. But I won't be doing it quickly, so I'm closing this on the assumption that if I do encounter a real problem in VSCode later, it'll be substantially different from what I originally reported here and a new issue would make sense. Thanks!