Stderr output causes test discovery to show as failed

[alexkirsz]

Environment data

• VS Code version: 1.37.1
• Extension version (available under the Extensions sidebar): 2019.8.30787
• OS and version: macOS 10.14.6
• Python version (& distribution if applicable, e.g. Anaconda): 3.6.8
• Type of virtual environment used (N/A | venv | virtualenv | conda | ...): N/A
• Relevant/affected Python packages and their versions: pytest 4.3.1
• Jedi or Language Server? (i.e. what is "python.jediEnabled" set to; more info How to update the language server to the latest stable version #3977): Jedi

Running inside of a Docker container

Expected behaviour

Test discovery completes successfully.

Actual behaviour

The extension logs the following message:

python /root/.vscode-server/extensions/ms-python.python-2019.8.30787/pythonFiles/testing_tools/run_adapter.py discover pytest -- -s --cache-clear
Test Discovery failed: 
Error: You are using pip version 19.0.3, however version 19.2.2 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.

Upgrading pip to remove the warning fixes the issue. However, this is but a warning and should not cause test discovery to fail (or, as I suspect in this case, to just display as failed).

Steps to reproduce:

Have running pytest output some kind of warning to stderr.

[kimadeline]

Hello @alexkirsz 👋 Thank you for reaching out. I tried reproducing your issue by writing to stderr inside a test in multiple ways, but pytest discovered and ran the tests without issues.

📋 Here's the content of my test file:

import logging
import sys
import warnings

def test_write_stderr():
    sys.stderr.write("Write something to stderr")

def test_warnings():
    warnings.warn("Warn something")

def test_logger_warning():
    logger = logging.getLogger(__name__)
    logger.warning("Logger warning")

def test_logger_error():
    logger.error("Logger error")

pip uses logger.warning to display their message (see their repo), it shouldn't have any influence on test discovery.

Could you please provide repro steps and/or sample code?

[brettcannon]

Because we have not heard back with the information we requested, we are closing this issue for now. If you are able to provide the info later on then we will be happy to re-open this issue to pick up where we left off.