`tests/test_xdist_handling.py` fails with PyTest 8.0.2 and Python 3.12 in Debian CI environment: tries stat on `/tmp/systemd-private-xxx`

[juliangilbey]

A bizarre error, which you may have some insight into. I obtain the same error if I use 1.2.0 or the current HEAD (commit a568bb1):

=================================== FAILURES ===================================
_____________________________ test_xdist_ordering ______________________________

tmpdir = local('/tmp/pytest-of-test/pytest-0/test_xdist_ordering0')

    def test_xdist_ordering(tmpdir):
        testname = str(tmpdir.join("first_test.py"))
[... rest of function snipped ...]

        # With `loadfile`, the tests should pass
        args = ["-n3", "--dist=loadfile", str(tmpdir)]
        ret = pytest.main(args, [pytest_order])
>       assert ret == 0
E       assert <ExitCode.TESTS_FAILED: 1> == 0

tests/test_xdist_handling.py:64: AssertionError
----------------------------- Captured stdout call -----------------------------
============================= test session starts ==============================
platform linux -- Python 3.12.2, pytest-8.0.2, pluggy-1.4.0
rootdir: /tmp
plugins: xdist-3.4.0, dependency-0.5.1, mock-3.12.0
created: 3/3 workers
3 workers [0 items]


==================================== ERRORS ====================================
______________________________ ERROR collecting . ______________________________
/usr/lib/python3/dist-packages/pluggy/_hooks.py:501: in __call__
    return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
/usr/lib/python3/dist-packages/pluggy/_manager.py:119: in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
/usr/lib/python3/dist-packages/_pytest/python.py:211: in pytest_collect_directory
    if pkginit.is_file():
/usr/lib/python3.12/pathlib.py:894: in is_file
    return S_ISREG(self.stat().st_mode)
/usr/lib/python3.12/pathlib.py:842: in stat
    return os.stat(self, follow_symlinks=follow_symlinks)
E   PermissionError: [Errno 13] Permission denied: '/tmp/systemd-private-2dd08cf217854d6285343c302b696375-systemd-logind.service-qOgrXp/__init__.py'
[... a similar error is repeated twice more ...]
=========================== short test summary info ============================
ERROR ../../.. - PermissionError: [Errno 13] Permission denied: '/tmp/systemd...
ERROR ../../.. - PermissionError: [Errno 13] Permission denied: '/tmp/systemd...
ERROR ../../.. - PermissionError: [Errno 13] Permission denied: '/tmp/systemd...
============================== 3 errors in 0.42s ===============================
=========================== short test summary info ============================
FAILED tests/test_xdist_handling.py::test_xdist_ordering - assert <ExitCode.T...
======================== 1 failed, 134 passed in 8.16s =========================

(A full log is attached, in case it is helpful.)

The tests are called as python3.12 -m pytest tests perf_tests in a directory only containing copies of perf_tests and tests.

If you have any idea what might be causing this, I'd be very grateful! It doesn't seem to make any sense that something should be looking in this systemd-private directory.

[juliangilbey]

Oops, I forgot the attachment (the full autopkgtest log).
pytest-order.log.txt

[mrbean-bremen]

Can you please check if you get the same problem with pytest 8.1? And if yes, with pytest 7?

[mrbean-bremen]

There was a problem in pytest 8.0.2 with tests in the CI, but that should have only affected tests under Windows... but the effect was similar - tests have been collected from other locations that where symlinked to the test dir.

[juliangilbey]

Yes, we get the same problem with pytest 8.1.1, but not with pytest 7.4.4; that's the only change I made to the test environment. Hmm.

[mrbean-bremen]

I suggest you pin pytest to 7 until the issue has been understood and fixed.
I assume it's something in the collection process of pytest, or maybe I'm doing something wrong in the tests that is triggered by the pytest change. I won't probably be able to investigate this soon (I will have have next to no free time this week, and will be away the week after), but I'll certainly have a look.

CC @nicoddemus

[juliangilbey]

Thanks @mrbean-bremen! We can't pin to pytest 7, unfortunately, but we can skip the test for now.

[juliangilbey]

It seems that it's this bug in pytest: pytest-dev/pytest#11781
I haven't yet had a chance to find a way to work around this; I'll let you know when I have.

[mrbean-bremen]

A new bugfix release is out.