VS Code unable to find test when running single unit test

[ahyerman]

Issue Type: Bug

Python: 3.6.6
Pytest: 3.8.2

When trying to run an individual test, vs code seems to fail with ERROR: file not found: <file>::<class>::<method>. It seems VS Coder can find the test since the command to run all tests works, also all tests appear to be discovered when running either all or a single test. However when running a single test I am seeing the above failure. I've tried to enforce the path to ensure the pytest.ini and rootdir are set correctly and that also fails.

Seems this working directory issue can be solved by including multiple paths in the folders option on the workspace. It'd be great if the python.unittest.cwd option could be used for pytest as well!

Repro:

workspace settings:

       "folders": [
	{
		"path": "C:\\proj"
	}
],
	"python.unitTest.pyTestArgs": [
		"--show-capture=no",
		"-ra",
		"-Wignore",
	],
	// Whether to enable or disable unit testing using pytest.
	"python.unitTest.unittestEnabled": false,
	"python.unitTest.pyTestEnabled": true,
	"python.unitTest.nosetestsEnabled": false,

folder structure:

/proj
|___/tests
|pytest.ini
|/scenario1
|_________test_1_s1.py
|___test_2_s1.py
|/scenario2
|_________test_1_s2.py
|_________test_2_s2.py

It seems this error only appears when trying to run or debug a single unit test, as running all tests seems to succeed.

VS Code version: Code 1.28.2 (7f3ce96ff4729c91352ae6def877e59c561f4850, 2018-10-17T00:23:51.859Z)
OS version: Windows_NT x64 10.0.17763

System Info

Item	Value
CPUs	Intel(R) Core(TM) i7-7700 CPU @ 3.60GHz (8 x 3600)
GPU Status	2d_canvas: enabled checker_imaging: disabled_off flash_3d: enabled flash_stage3d: enabled flash_stage3d_baseline: enabled gpu_compositing: enabled multiple_raster_threads: enabled_on native_gpu_memory_buffers: disabled_software rasterization: enabled video_decode: enabled video_encode: enabled webgl: enabled webgl2: enabled
Memory (System)	47.79GB (29.28GB free)
Process Argv
Screen Reader	no
VM	0%

Extensions (5)

Extension	Author (truncated)	Version
gitlens	eam	8.5.6
nbpreviewer	jit	1.0.0
python	ms-	2018.9.2
PowerShell	ms-	1.9.0
team	ms-	1.142.0

[maciejmatczak]

@ahyerman, can you share more info how your tests look like? Are those I am observing similar behavior but only on pytest tests with parametrize decorator and multiline arguments with triple quotes... whenever there is a newline character in argument value, it stops being "foundable".

If that's not the case I will create new issue.

[ahyerman]

@maciejmatczak most of my tests do not have decorators. They generally look like the following:

class TestClassOne(MyTestBase):
    def setup_class(self):
        <do setup work>

    def test_some_function(self):
          <do test work>

Tests have become discoverable but only when I change the root directory as described above.

It seems the following setting is not honored properly when using Pytest: Python.unitTest.cwd

[d3r3kk]

I am indeed seeing some trouble here as well.

I can see that we aren't recognizing tests on load-up (discover tests):

Also, we are not discovering tests when we issue the 'Discover Tests' command, nor when we attempt to run a single test, but we are recognizing when we have finished running all tests:

This is going to have to be refactored at minimum.

[DonJayamanne]

Dup of #3911