Skip to content

Python self.subTest decorators show false passes #25546

@JGSweets

Description

@JGSweets

Type: Bug

Behaviour

When using a decorator that creates subtests, VSCode reports back false passes depending on the order of subtests.

Additionally, it cannot locate tests or improperly combines tests when running the run at cursor.

Steps to reproduce:

  1. Configure for pytests.
    2a. Run Tests in file.
import unittest


def cases_subtests(*items):
    """
    Decorate a unittest method so it runs once per case using self.subTest.
    Each item can be:
      - tuple/list  -> passed as *args
      - dict        -> passed as **kwargs
    """

    def deco(func):
        def wrapper(self):
            for i, item in enumerate(items):
                if isinstance(item, dict):
                    if "msg" in item:
                        raise ValueError(
                            "Cannot have case with an input of `msg`"
                        )
                    with self.subTest(i=i, **item, msg=f"case={item!r}"):
                        func(self, **item)
                else:
                    args = (
                        tuple(item)
                        if isinstance(item, (tuple, list))
                        else (item,)
                    )
                    with self.subTest(i=i, args=args, msg=f"case={item!r}"):
                        func(self, *args)

        return wrapper

    return deco


class MyTest(unittest.TestCase):

    @cases_subtests(
        (None, None),
        (None, []),
    )
    def test_equal_false_pass(self, value, expected):
        self.assertEqual(expected, value)

    @cases_subtests(
        ([], None),
        (None, None),
    )
    def test_equal_true_fail(self, value, expected):
        self.assertEqual(expected, value)

Diagnostic data

Output for Python in the Output panel (ViewOutput, change the drop-down the upper-right of the Output panel to Python)

2025-10-24 15:21:39.548 [info] Python interpreter path: ./.venv/bin/python
2025-10-24 15:21:41.028 [info] arg: --rootdir already exists in args, not adding.
2025-10-24 15:21:41.028 [info] Attempting to use temp directory for test ids file, file name: test-ids-XXX.txt
2025-10-24 15:21:41.030 [info] Environment variables set for pytest execution: PYTHONPATH=.../python_files, TEST_RUN_PIPE=/var/folders/.../python-test-results-XXX, RUN_TEST_IDS_PIPE=/var/folders/.../test-ids-XXX.txt
2025-10-24 15:21:41.030 [info] Running pytest with arguments: /.../ms-python.python-2025.16.0-darwin-arm64/python_files/vscode_pytest/run_pytest_script.py --rootdir=/XXX for workspace /XXX

2025-10-24 15:21:41.030 [info] > ~/XXX/.venv/bin/python ~/.vscode/extensions/ms-python.python-2025.16.0-darwin-arm64/python_files/vscode_pytest/run_pytest_script.py --rootdir=~/XXX
2025-10-24 15:21:41.030 [info] cwd: ~/XXX
2025-10-24 15:21:41.205 [info] Test Result named pipe /var/folders/.../python-test-results-YYY  cancelled
2025-10-24 15:21:41.717 [info] Python interpreter path: ~/XXX.venv/bin/python

Extension version: 2025.16.0
VS Code version: Code 1.104.3 (Universal)
Modes:

Metadata

Metadata

Assignees

Labels

info-neededIssue requires more information from postertriage-neededNeeds assignment to the proper sub-team

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions