1.14.1: pytest fails in `test_async_fixture_module_scope.py::test_second` unit

[kloczek]

I'm packaging your module as an rpm package so I'm using the typical PEP517 based build, install and test cycle used on building packages from non-root account.

• python3 -sBm build -w --no-isolation
• because I'm calling build with --no-isolation I'm using during all processes only locally installed modules
• install .whl file in </install/prefix> using installer module
• run pytest with $PYTHONPATH pointing to sitearch and sitelib inside </install/prefix>
• build is performed in env which is cut off from access to the public network (pytest is executed with -m "not network")

Here is pytest output:

+ PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-pytest-twisted-1.14.1-2.fc37.x86_64/usr/lib64/python3.10/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-pytest-twisted-1.14.1-2.fc37.x86_64/usr/lib/python3.10/site-packages
+ /usr/bin/pytest -ra -m 'not network' -q
============================= test session starts ==============================
platform linux -- Python 3.10.14, pytest-8.2.1, pluggy-1.5.0
rootdir: /home/tkloczko/rpmbuild/BUILD/pytest-twisted-1.14.1
configfile: pytest.ini
plugins: twisted-1.14.1, hypothesis-6.100.0
collected 60 items

testing/test_basic.py .....................................ss.ssF....... [ 83%]
..........                                                               [100%]

=================================== FAILURES ===================================
_______________________ test_async_fixture_module_scope ________________________

testdir = <Testdir local('/tmp/pytest-of-tkloczko/pytest-33/test_async_fixture_module_scope0')>
cmd_opts = ('/usr/bin/python3', '-m', 'pytest', '-v', '--reactor=default')

    @skip_if_no_async_generators()
    def test_async_fixture_module_scope(testdir, cmd_opts):
        test_file = """
        from twisted.internet import reactor, defer
        import pytest
        import pytest_twisted

        check_me = 0

        @pytest_twisted.async_yield_fixture(scope="module")
        async def foo():
            global check_me

            if check_me != 0:
                raise Exception('check_me already modified before fixture run')

            check_me = 1

            yield 42

            if check_me != 3:
                raise Exception(
                    'check_me not updated properly: {}'.format(check_me),
                )

            check_me = 0

        def test_first(foo):
            global check_me

            assert check_me == 1
            assert foo == 42

            check_me = 2

        def test_second(foo):
            global check_me

            assert check_me == 2
            assert foo == 42

            check_me = 3
        """
        testdir.makepyfile(test_file)
        rr = testdir.run(*cmd_opts, timeout=timeout)
>       assert_outcomes(rr, {"passed": 2})

/home/tkloczko/rpmbuild/BUILD/pytest-twisted-1.14.1/testing/test_basic.py:1055:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

run_result = <RunResult ret=ExitCode.TESTS_FAILED len(stdout.lines)=35 len(stderr.lines)=0 duration=1.22s>
outcomes = {'passed': 2}

    def assert_outcomes(run_result, outcomes):
        formatted_output = format_run_result_output_for_assert(run_result)

        try:
            result_outcomes = run_result.parseoutcomes()
        except ValueError:
            assert False, formatted_output

        normalized_result_outcomes = {
            force_plural(name): outcome
            for name, outcome in result_outcomes.items()
            if name != "seconds"
        }

>       assert normalized_result_outcomes == outcomes, formatted_output
E       AssertionError:
E         ---- stdout
E         ============================= test session starts ==============================
E         platform linux -- Python 3.10.14, pytest-8.2.1, pluggy-1.5.0 -- /usr/bin/python3
E         cachedir: .pytest_cache
E         hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase(PosixPath('/tmp/pytest-of-tkloczko/pytest-33/test_async_fixture_module_scope0/.hypothesis/examples'))
E         rootdir: /tmp/pytest-of-tkloczko/pytest-33/test_async_fixture_module_scope0
E         plugins: twisted-1.14.1, hypothesis-6.100.0
E         collecting ... collected 2 items
E
E         test_async_fixture_module_scope.py::test_first PASSED                    [ 50%]
E         test_async_fixture_module_scope.py::test_second ERROR                    [100%]
E
E         ==================================== ERRORS ====================================
E         ________________________ ERROR at setup of test_second _________________________
E
E             @pytest_twisted.async_yield_fixture(scope="module")
E             async def foo():
E                 global check_me
E
E                 if check_me != 0:
E                     raise Exception('check_me already modified before fixture run')
E
E                 check_me = 1
E
E                 yield 42
E
E                 if check_me != 3:
E         >           raise Exception(
E                         'check_me not updated properly: {}'.format(check_me),
E                     )
E         E           Exception: check_me not updated properly: 2
E
E         test_async_fixture_module_scope.py:19: Exception
E         =========================== short test summary info ============================
E         ERROR test_async_fixture_module_scope.py::test_second - Exception: check_me n...
E         ========================== 1 passed, 1 error in 0.74s ==========================
E         ---- stderr
E
E         ----
E
E       assert {'errors': 1, 'passed': 1} == {'passed': 2}
E
E         Differing items:
E         {'passed': 1} != {'passed': 2}
E         Left contains 1 more item:
E         {'errors': 1}
E         Use -v to get more diff

/home/tkloczko/rpmbuild/BUILD/pytest-twisted-1.14.1/testing/test_basic.py:45: AssertionError
----------------------------- Captured stdout call -----------------------------
running: /usr/bin/python3 -m pytest -v --reactor=default
     in: /tmp/pytest-of-tkloczko/pytest-33/test_async_fixture_module_scope0
============================= test session starts ==============================
platform linux -- Python 3.10.14, pytest-8.2.1, pluggy-1.5.0 -- /usr/bin/python3
cachedir: .pytest_cache
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase(PosixPath('/tmp/pytest-of-tkloczko/pytest-33/test_async_fixture_module_scope0/.hypothesis/examples'))
rootdir: /tmp/pytest-of-tkloczko/pytest-33/test_async_fixture_module_scope0
plugins: twisted-1.14.1, hypothesis-6.100.0
collecting ... collected 2 items

test_async_fixture_module_scope.py::test_first PASSED                    [ 50%]
test_async_fixture_module_scope.py::test_second ERROR                    [100%]

==================================== ERRORS ====================================
________________________ ERROR at setup of test_second _________________________

    @pytest_twisted.async_yield_fixture(scope="module")
    async def foo():
        global check_me

        if check_me != 0:
            raise Exception('check_me already modified before fixture run')

        check_me = 1

        yield 42

        if check_me != 3:
>           raise Exception(
                'check_me not updated properly: {}'.format(check_me),
            )
E           Exception: check_me not updated properly: 2

test_async_fixture_module_scope.py:19: Exception
=========================== short test summary info ============================
ERROR test_async_fixture_module_scope.py::test_second - Exception: check_me n...
========================== 1 passed, 1 error in 0.74s ==========================
=========================== short test summary info ============================
SKIPPED [2] testing/test_basic.py:77: reactor is default not qt5reactor
SKIPPED [2] testing/test_basic.py:77: reactor is default not asyncio
FAILED testing/test_basic.py::test_async_fixture_module_scope - AssertionError:
============== 1 failed, 55 passed, 4 skipped in 79.06s (0:01:19) ==============
/usr/lib/python3.10/site-packages/_pytest/pathlib.py:98: PytestWarning: (rm_rf) error removing /tmp/pytest-of-tkloczko/garbage-aca48b6b-259c-4520-8811-f9d7aedc6826/test_safe_get_no_perms0
<class 'OSError'>: [Errno 39] Directory not empty: 'test_safe_get_no_perms0'
  warnings.warn(
/usr/lib/python3.10/site-packages/_pytest/pathlib.py:98: PytestWarning: (rm_rf) error removing /tmp/pytest-of-tkloczko/garbage-aca48b6b-259c-4520-8811-f9d7aedc6826/test_safe_set_no_perms0
<class 'OSError'>: [Errno 39] Directory not empty: 'test_safe_set_no_perms0'
  warnings.warn(
/usr/lib/python3.10/site-packages/_pytest/pathlib.py:98: PytestWarning: (rm_rf) error removing /tmp/pytest-of-tkloczko/garbage-aca48b6b-259c-4520-8811-f9d7aedc6826/test_safe_delete_no_perms0
<class 'OSError'>: [Errno 39] Directory not empty: 'test_safe_delete_no_perms0'
  warnings.warn(
/usr/lib/python3.10/site-packages/_pytest/pathlib.py:98: PytestWarning: (rm_rf) error removing /tmp/pytest-of-tkloczko/garbage-aca48b6b-259c-4520-8811-f9d7aedc6826/test_rmtree_errorhandler_rerai0
<class 'OSError'>: [Errno 39] Directory not empty: 'test_rmtree_errorhandler_rerai0'
  warnings.warn(
/usr/lib/python3.10/site-packages/_pytest/pathlib.py:98: PytestWarning: (rm_rf) error removing /tmp/pytest-of-tkloczko/garbage-aca48b6b-259c-4520-8811-f9d7aedc6826
<class 'OSError'>: [Errno 39] Directory not empty: '/tmp/pytest-of-tkloczko/garbage-aca48b6b-259c-4520-8811-f9d7aedc6826'
  warnings.warn(

List of installed modules in build env:

Package            Version
------------------ -----------
attrs              23.2.0
Automat            22.10.0
build              1.2.1
constantly         23.10.4
decorator          5.1.1
exceptiongroup     1.1.3
greenlet           3.0.3
hyperlink          21.0.0
hypothesis         6.100.0
idna               3.7
importlib_metadata 7.1.0
incremental        22.10.0
iniconfig          2.0.0
installer          0.7.0
packaging          24.0
pluggy             1.5.0
pyproject_hooks    1.0.0
pytest             8.2.1
python-dateutil    2.9.0.post0
setuptools         69.4.0
sortedcontainers   2.4.0
tokenize_rt        5.2.0
tomli              2.0.1
Twisted            24.3.0
typing_extensions  4.12.1
wheel              0.43.0
zipp               3.19.1
zope.event         5.0
zope.interface     6.4.post2

Please let me know if you need more details or want me to perform some diagnostics.

[mgorny]

This is caused by pytest-8.2, and can be reproduced using tox.

@kloczek, could you update the summary to point the cause?

[altendky]

Looks like this is present in the last (awhile ago) scheduled CI run. https://github.com/pytest-dev/pytest-twisted/actions/runs/9460305008/job/26058879367#step:10:252 I'll see what I can do. Thanks for pointing this out.

[RonnyPfannschmidt]

pytest-twisted/pytest_twisted.py

Line 336 in ba5eb88

fixturedef.cached_result = (arg_value, request.param_index, None)

needs to use the cache key

[altendky]

https://pypi.org/project/pytest-twisted/1.14.2/

v1.14.2 is available. @kloczek, let me know if that works for you or not. I'm sorry for leaving CI broken and making you find this issue instead.

[mgorny]

If that's worth anything, it works for me now. Thanks!