Merge branch 'main' into drop-python3.8

Author: Pierre-Sassoulas

.github/workflows/test.yml

@@ -261,7 +261,7 @@ jobs:
 
     - name: Upload coverage to Codecov
       if: "matrix.use_coverage"
-      uses: codecov/codecov-action@v4
+      uses: codecov/codecov-action@v5
       with:
         fail_ci_if_error: false
         files: ./coverage.xml

changelog/10839.deprecation.rst

@@ -0,0 +1 @@
+Requesting an asynchronous fixture without a `pytest_fixture_setup` hook that resolves it will now give a DeprecationWarning. This most commonly happens if a sync test requests an async fixture. This should have no effect on a majority of users with async tests or fixtures using async pytest plugins, but may affect non-standard hook setups or ``autouse=True``. For guidance on how to work around this warning see :ref:`sync-test-async-fixture`.

changelog/12535.doc.rst

@@ -0,0 +1,4 @@
+`This
+example`<https://docs.pytest.org/en/latest/example/simple.html#making-test-result-information-available-in-fixtures>
+showed ``print`` statements that do not exactly reflect what the
+different branches actually do.  The fix makes the example more precise.

changelog/12960.breaking.rst

@@ -0,0 +1,3 @@
+Test functions containing a yield now cause an explicit error. They have not been run since pytest 4.0, and were previously marked as an expected failure and deprecation warning.
+
+See :ref:`the docs <yield tests deprecated>` for more information.

changelog/12966.doc.rst

@@ -0,0 +1 @@
+Clarify :ref:`filterwarnings` docs on filter precedence/order when using multiple :ref:`@pytest.mark.filterwarnings <pytest.mark.filterwarnings ref>` marks.

doc/en/deprecations.rst

@@ -15,6 +15,76 @@ Below is a complete list of all pytest features which are considered deprecated.
 :class:`~pytest.PytestWarning` or subclasses, which can be filtered using :ref:`standard warning filters <warnings>`.
 
 
+.. _sync-test-async-fixture:
+
+sync test depending on async fixture
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+.. deprecated:: 8.4
+
+Pytest has for a long time given an error when encountering an asynchronous test function, prompting the user to install
+a plugin that can handle it. It has not given any errors if you have an asynchronous fixture that's depended on by a
+synchronous test. If the fixture was an async function you did get an "unawaited coroutine" warning, but for async yield fixtures you didn't even get that.
+This is a problem even if you do have a plugin installed for handling async tests, as they may require
+special decorators for async fixtures to be handled, and some may not robustly handle if a user accidentally requests an
+async fixture from their sync tests. Fixture values being cached can make this even more unintuitive, where everything will
+"work" if the fixture is first requested by an async test, and then requested by a synchronous test.
+
+Unfortunately there is no 100% reliable method of identifying when a user has made a mistake, versus when they expect an
+unawaited object from their fixture that they will handle on their own. To suppress this warning
+when you in fact did intend to handle this you can wrap your async fixture in a synchronous fixture:
+
+.. code-block:: python
+
+    import asyncio
+    import pytest
+
+
+    @pytest.fixture
+    async def unawaited_fixture():
+        return 1
+
+
+    def test_foo(unawaited_fixture):
+        assert 1 == asyncio.run(unawaited_fixture)
+
+should be changed to
+
+
+.. code-block:: python
+
+    import asyncio
+    import pytest
+
+
+    @pytest.fixture
+    def unawaited_fixture():
+        async def inner_fixture():
+            return 1
+
+        return inner_fixture()
+
+
+    def test_foo(unawaited_fixture):
+        assert 1 == asyncio.run(unawaited_fixture)
+
+
+You can also make use of `pytest_fixture_setup` to handle the coroutine/asyncgen before pytest sees it - this is the way current async pytest plugins handle it.
+
+If a user has an async fixture with ``autouse=True`` in their ``conftest.py``, or in a file
+containing both synchronous tests and the fixture, they will receive this warning.
+Unless you're using a plugin that specifically handles async fixtures
+with synchronous tests, we strongly recommend against this practice.
+It can lead to unpredictable behavior (with larger scopes, it may appear to "work" if an async
+test is the first to request the fixture, due to value caching) and will generate
+unawaited-coroutine runtime warnings (but only for non-yield fixtures).
+Additionally, it creates ambiguity for other developers about whether the fixture is intended to perform
+setup for synchronous tests.
+
+The `anyio pytest plugin <https://anyio.readthedocs.io/en/stable/testing.html>`_ supports
+synchronous tests with async fixtures, though certain limitations apply.
+
+
 .. _import-or-skip-import-error:
 
 ``pytest.importorskip`` default behavior regarding :class:`ImportError`
@@ -304,6 +374,42 @@ an appropriate period of deprecation has passed.
 
 Some breaking changes which could not be deprecated are also listed.
 
+.. _yield tests deprecated:
+
+``yield`` tests
+~~~~~~~~~~~~~~~
+
+.. versionremoved:: 4.0
+
+    ``yield`` tests ``xfail``.
+
+.. versionremoved:: 8.4
+
+    ``yield`` tests raise a collection error.
+
+pytest no longer supports ``yield``-style tests, where a test function actually ``yield`` functions and values
+that are then turned into proper test methods. Example:
+
+.. code-block:: python
+
+    def check(x, y):
+        assert x**x == y
+
+
+    def test_squared():
+        yield check, 2, 4
+        yield check, 3, 9
+
+This would result in two actual test functions being generated.
+
+This form of test function doesn't support fixtures properly, and users should switch to ``pytest.mark.parametrize``:
+
+.. code-block:: python
+
+    @pytest.mark.parametrize("x, y", [(2, 4), (3, 9)])
+    def test_squared(x, y):
+        assert x**x == y
+
 .. _nose-deprecation:
 
 Support for tests written for nose
@@ -1200,36 +1306,6 @@ with the ``name`` parameter:
         return cell()
 
 
-.. _yield tests deprecated:
-
-``yield`` tests
-~~~~~~~~~~~~~~~
-
-.. versionremoved:: 4.0
-
-pytest supported ``yield``-style tests, where a test function actually ``yield`` functions and values
-that are then turned into proper test methods. Example:
-
-.. code-block:: python
-
-    def check(x, y):
-        assert x**x == y
-
-
-    def test_squared():
-        yield check, 2, 4
-        yield check, 3, 9
-
-This would result into two actual test functions being generated.
-
-This form of test function doesn't support fixtures properly, and users should switch to ``pytest.mark.parametrize``:
-
-.. code-block:: python
-
-    @pytest.mark.parametrize("x, y", [(2, 4), (3, 9)])
-    def test_squared(x, y):
-        assert x**x == y
-
 .. _internal classes accessed through node deprecated:
 
 Internal classes accessed through ``Node``

doc/en/example/simple.rst

@@ -904,7 +904,9 @@ here is a little example implemented via a local plugin:
         # "function" scope
         report = request.node.stash[phase_report_key]
         if report["setup"].failed:
-            print("setting up a test failed or skipped", request.node.nodeid)
+            print("setting up a test failed", request.node.nodeid)
+        elif report["setup"].skipped:
+            print("setting up a test skipped", request.node.nodeid)
         elif ("call" not in report) or report["call"].failed:
             print("executing test failed or skipped", request.node.nodeid)
 

doc/en/how-to/capture-warnings.rst

@@ -128,7 +128,7 @@ is performed.
 
 
 
-You can use the ``@pytest.mark.filterwarnings`` to add warning filters to specific test items,
+You can use the :ref:`@pytest.mark.filterwarnings <pytest.mark.filterwarnings ref>` mark to add warning filters to specific test items,
 allowing you to have finer control of which warnings should be captured at test, class or
 even module level:
 
@@ -147,10 +147,30 @@ even module level:
         assert api_v1() == 1
 
 
+You can specify multiple filters with separate decorators:
+
+.. code-block:: python
+
+    # Ignore "api v1" warnings, but fail on all other warnings
+    @pytest.mark.filterwarnings("ignore:api v1")
+    @pytest.mark.filterwarnings("error")
+    def test_one():
+        assert api_v1() == 1
+
+.. important::
+
+    Regarding decorator order and filter precedence:
+    it's important to remember that decorators are evaluated in reverse order,
+    so you have to list the warning filters in the reverse order
+    compared to traditional :py:func:`warnings.filterwarnings` and :option:`-W option <python:-W>` usage.
+    This means in practice that filters from earlier :ref:`@pytest.mark.filterwarnings <pytest.mark.filterwarnings ref>` decorators
+    take precedence over filters from later decorators, as illustrated in the example above.
+
+
 Filters applied using a mark take precedence over filters passed on the command line or configured
-by the ``filterwarnings`` ini option.
+by the :confval:`filterwarnings` ini option.
 
-You may apply a filter to all tests of a class by using the ``filterwarnings`` mark as a class
+You may apply a filter to all tests of a class by using the :ref:`filterwarnings <pytest.mark.filterwarnings ref>` mark as a class
 decorator or to all tests in a module by setting the :globalvar:`pytestmark` variable:
 
 .. code-block:: python
@@ -159,6 +179,13 @@ decorator or to all tests in a module by setting the :globalvar:`pytestmark` var
     pytestmark = pytest.mark.filterwarnings("error")
 
 
+.. note::
+
+    If you want to apply multiple filters
+    (by assigning a list of :ref:`filterwarnings <pytest.mark.filterwarnings ref>` mark to :globalvar:`pytestmark`),
+    you must use the traditional :py:func:`warnings.filterwarnings` ordering approach (later filters take precedence),
+    which is the reverse of the decorator approach mentioned above.
+
 
 *Credits go to Florian Schulze for the reference implementation in the* `pytest-warnings`_
 *plugin.*