BUG: pytest.raises(HTTPError, match="...") breaks on Python 3.9 + pytest 8.0

[neutrinoceros]

import pytest
from urllib.error import HTTPError

def test_foo():
    with pytest.raises(HTTPError, match="Not Found"):
        raise HTTPError(code=404, msg="Not Found", fp=None, hdrs=None, url="")

this test pass with Python 3.9 + pytest 7.4.4, but fails if I upgrade to pytest 8.0.0
Here's the short traceback (simply running pytest t.py)

========================================= test session starts ==========================================
platform darwin -- Python 3.9.18, pytest-8.0.0, pluggy-1.4.0
rootdir: /private/tmp
collected 1 item

t.py F                                                                                           [100%]

=============================================== FAILURES ===============================================
_______________________________________________ test_foo _______________________________________________

    def test_foo():
        with pytest.raises(HTTPError, match="Not Found"):
>           raise HTTPError(code=404, msg="Not Found", fp=None, hdrs=None, url="")
E           urllib.error.HTTPError: HTTP Error 404: Not Found

t.py:5: HTTPError

During handling of the above exception, another exception occurred:

    def test_foo():
        with pytest.raises(HTTPError, match="Not Found"):
>           raise HTTPError(code=404, msg="Not Found", fp=None, hdrs=None, url="")

t.py:5:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
.venv/lib/python3.9/site-packages/_pytest/_code/code.py:704: in _stringify_exception
    *getattr(exc, "__notes__", []),
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <HTTPError 404: 'Not Found'>, name = '__notes__'

    def __getattr__(self, name):
        # Attribute lookups are delegated to the underlying file
        # and cached for non-numeric results
        # (i.e. methods are cached, closed and friends are not)
>       file = self.__dict__['file']
E       KeyError: 'file'

/Users/clm/.pyenv/versions/3.9.18/lib/python3.9/tempfile.py:473: KeyError
======================================= short test summary info ========================================
FAILED t.py::test_foo - KeyError: 'file'
========================================== 1 failed in 0.03s ===========================================

And attached, the full-trace (pytest t.py --full-trace)

I haven't bisected yet, but git blame seems to indicate that #11424 is probably the breaking patch

• a detailed description of the bug or problem you are having
• output of pip list from the virtual environment you are using

Package        Version
-------------- -------
exceptiongroup 1.2.0
iniconfig      2.0.0
packaging      23.2
pip            23.0.1
pluggy         1.4.0
pytest         8.0.0
setuptools     58.1.0
tomli          2.0.1

• pytest and operating system versions:
  see on MacOS 14.3 and Ubuntu 22.04.3, but I don't think that's relevant
• minimal example if possible

[neutrinoceros]

given that the exception is raised from tempfile, I suspect this could be an actual CPython bug.

[bluetech]

Looks like getattr(exc, "__notes__", []) is causing issues on this particular exception on Python 3.9 (I verified that this doesn't happen on 3.10 and 3.11). I haven't haven't checked how tempfile is involved yet, probably a urllib thing. The most likely fix is to change this getattr to a safe getattr, but if you're able to analyze the cause more precisely that'd be helpful :)

[neutrinoceros]

what do you mean by "safe getattr" ?

[bluetech]

pytest/src/_pytest/compat.py

Line 277 in 8853a57

def safe_getattr(object: Any, name: str, default: Any) -> Any:

[glemaitre]

Our CIs tried to bump pytest and this failure happen on 3.9 only for what it is worth.

[glemaitre]

and using safe_getattr is solving the issue.

[glemaitre]

The root cause was probably this bug: python/cpython#98778

[tmct]

I'm seeing very similar behaviour for python 3.10 + pytest 8 - it's failing to register that a ValueError has been thrown. Only occurs when I run the whole test suite at once, but then one of my tests (one that uses mocking - not sure if relevant) fails every time. Fine on pytest 7.x.

I'll try to put together a minimal, shareable example if I can.