Virtual environment not activated before running pytest (or other tools) via the extension

[Smattacus]

Issue Type: Bug

Notes

I have a Windows machine with a remote SSH to a linux machine that is running my Python and tests. It appears that sometimes the virtual environment isn't set up before a debug or run with pytest through VSCode. If I run pytest using the integrated terminal, everything runs fine. I have another situation where the test is failing in both normal and debug run mode, but I haven't been able to recreate it with my simple testcase above yet.

Thank you for all the work on this great extension and on VSCode, using it has been very enjoyable!!

Environment data

• VS Code version: 1.55.2 (3c4e3df9e89829dce27b7b5c24508306b151f30d, 2021-04-13T09:35:57.887Z)
• Extension version (available under the Extensions sidebar): python|ms-|2021.4.765268190
• OS version: Windows_NT x64 10.0.18363
• Remote OS version: Linux x64 4.12.14-122.60-default
• Python version (& distribution if applicable, e.g. Anaconda): python3.7.4
• Type of virtual environment used (N/A | venv | virtualenv | conda | ...): venv
• Relevant/affected Python packages and their versions: pytest 6.2.3
• Relevant/affected Python-related VS Code extensions and their versions: python|ms-|2021.4.765268190
• Value of the python.languageServer setting: Jedi

Expected Behavior

Returncode should be 0, and stdout should have flake8 --help information.

Actual Behavior

Test fails in debug mode with stderr capture: "/bin/sh: flake8: command not found". Test succeeds in normal run mode.

Steps to Reproduce

1. Windows VSCode with remote-ssh connection to a linux box.
2. Set up a virtual environment. Point VSCode to it.

My settings.json for this testcase:

{
    "python.pythonPath": "venv/bin/python",
    "python.testing.pytestArgs": [
        "."
    ],
    "python.testing.unittestEnabled": false,
    "python.testing.nosetestsEnabled": false,
    "python.testing.pytestEnabled": true
}

3. Install some cli tool in that virtual environment (my case: "pip install flake8")
4. Test a function using pytest that calls this cli with subprocess.Popen():

import subprocess

def run_flake():
    proc = subprocess.Popen("flake8 --help", 
        universal_newlines = True, 
        shell=True, 
        stdin=subprocess.PIPE, 
        stdout=subprocess.PIPE, 
        stderr=subprocess.PIPE)
    (stdout_data, stderr_data) = proc.communicate()
    print("Standard output capture:" + stdout_data)
    print("Standard error capture:" + stderr_data)
    return proc

def test_run_flake():
    p = run_flake()
    assert p.returncode == 0

Logs

Output for Python in the Output panel (View→Output, change the drop-down the upper-right of the Output panel to Python)

collected 1 item

hello_world_test.py F                                                    [100%]

=================================== FAILURES ===================================
________________________________ test_run_flake ________________________________

    def test_run_flake():
        p = run_flake()
>       assert p.returncode == 0, "ERROR: Child Process failed."
E       AssertionError: ERROR: Child Process failed.
E       assert 127 == 0
E        +  where 127 = <subprocess.Popen object at 0x7fffe8a145d0>.returncode

hello_world_test.py:21: AssertionError
----------------------------- Captured stdout call -----------------------------
Standard output capture:
Standard error capture:/bin/sh: flake8: command not found

Information from VSCode reporting tool

VS Code version: Code 1.55.2 (3c4e3df9e89829dce27b7b5c24508306b151f30d, 2021-04-13T09:35:57.887Z)
OS version: Windows_NT x64 10.0.18363
Remote OS version: Linux x64 4.12.14-122.60-default
Remote OS version: Linux x64 4.12.14-122.60-default

System Info

Item	Value
CPUs	Intel(R) Core(TM) i5-8350U CPU @ 1.70GHz (8 x 1896)
GPU Status	2d_canvas: enabled gpu_compositing: enabled multiple_raster_threads: enabled_on oop_rasterization: enabled opengl: enabled_on protected_video_decode: enabled rasterization: enabled skia_renderer: enabled_on video_decode: enabled vulkan: disabled_off webgl: enabled webgl2: enabled
Load (avg)	undefined
Memory (System)	7.84GB (1.05GB free)
Process Argv	--crash-reporter-id 1ba6190d-2b0c-4453-bddf-b5ec814f80ab
Screen Reader	no
VM	0%

Item	Value
Remote	SSH: scc919193
OS	Linux x64 4.12.14-122.60-default
CPUs	Intel(R) Xeon(R) E-2236 CPU @ 3.40GHz (6 x 3400)
Memory (System)	125.54GB (22.56GB free)
VM	0%

Item	Value
Remote	SSH: scc919193
OS	Linux x64 4.12.14-122.60-default
CPUs	Intel(R) Xeon(R) E-2236 CPU @ 3.40GHz (6 x 3400)
Memory (System)	125.54GB (22.56GB free)
VM	0%

Extensions (8)

Extension	Author (truncated)	Version
remote-containers	ms-	0.166.1
remote-ssh	ms-	0.65.4
remote-ssh-edit	ms-	0.65.4
remote-wsl	ms-	0.54.6
vscode-remote-extensionpack	ms-	0.20.0
gitlens	eam	11.4.1
python	ms-	2021.4.765268190
jupyter	ms-	2021.5.745244803

A/B Experiments

vsliv368cf:30146710
vsreu685:30147344
python383:30185418
vspyt653:30270858
vspor879:30202332
vspor708:30202333
vspor363:30204092
vstry244:30276681
pythonvsdeb440:30248342
pythonvsded773:30248341
pythonvspyt875:30259475
pythonvspyt639:30291489
pythontb:30283811
pythonvspyt551cf:30291415
vspre833:30267464
pythonptprofiler:30281270
vshan820:30276952
vscoreces:30290705
pythondataviewer:30285071
vscus158cf:30286554

[kimadeline]

Hi @Smattacus, thank you for reaching out!

It appears that sometimes the virtual environment isn't set up before a debug or run with pytest through VSCode.

You got it there, it's actually a long-standing issue: #4300 (feel free to upvote it!).

I reworded the title of the issue you created to make it more generic, since it's a problem that we have with all the tools being run by the extension behind the scenes.

[Smattacus]

Thanks for the follow up @kimadeline! I'll upvote it there. Again, thanks for this extension - I find it very useful.