Merge pull request #11646 from bluetech/pkg-collect

Rework Session and Package collection

Author: bluetech

changelog/7777.breaking.rst

@@ -0,0 +1,90 @@
+Added a new :class:`pytest.Directory` base collection node, which all collector nodes for filesystem directories are expected to subclass.
+This is analogous to the existing :class:`pytest.File` for file nodes.
+
+Changed :class:`pytest.Package` to be a subclass of :class:`pytest.Directory`.
+A ``Package`` represents a filesystem directory which is a Python package,
+i.e. contains an ``__init__.py`` file.
+
+:class:`pytest.Package` now only collects files in its own directory; previously it collected recursively.
+Sub-directories are collected as sub-collector nodes, thus creating a collection tree which mirrors the filesystem hierarchy.
+
+Added a new :class:`pytest.Dir` concrete collection node, a subclass of :class:`pytest.Directory`.
+This node represents a filesystem directory, which is not a :class:`pytest.Package`,
+i.e. does not contain an ``__init__.py`` file.
+Similarly to ``Package``, it only collects the files in its own directory,
+while collecting sub-directories as sub-collector nodes.
+
+Added a new hook :hook:`pytest_collect_directory`,
+which is called by filesystem-traversing collector nodes,
+such as :class:`pytest.Session`, :class:`pytest.Dir` and :class:`pytest.Package`,
+to create a collector node for a sub-directory.
+It is expected to return a subclass of :class:`pytest.Directory`.
+This hook allows plugins to :ref:`customize the collection of directories <custom directory collectors>`.
+
+:class:`pytest.Session` now only collects the initial arguments, without recursing into directories.
+This work is now done by the :func:`recursive expansion process <pytest.Collector.collect>` of directory collector nodes.
+
+:attr:`session.name <pytest.Session.name>` is now ``""``; previously it was the rootdir directory name.
+This matches :attr:`session.nodeid <_pytest.nodes.Node.nodeid>` which has always been `""`.
+
+Files and directories are now collected in alphabetical order jointly, unless changed by a plugin.
+Previously, files were collected before directories.
+
+The collection tree now contains directories/packages up to the :ref:`rootdir <rootdir>`,
+for initial arguments that are found within the rootdir.
+For files outside the rootdir, only the immediate directory/package is collected --
+note however that collecting from outside the rootdir is discouraged.
+
+As an example, given the following filesystem tree::
+
+    myroot/
+        pytest.ini
+        top/
+        ├── aaa
+        │   └── test_aaa.py
+        ├── test_a.py
+        ├── test_b
+        │   ├── __init__.py
+        │   └── test_b.py
+        ├── test_c.py
+        └── zzz
+            ├── __init__.py
+            └── test_zzz.py
+
+the collection tree, as shown by `pytest --collect-only top/` but with the otherwise-hidden :class:`~pytest.Session` node added for clarity,
+is now the following::
+
+    <Session>
+      <Dir myroot>
+        <Dir top>
+          <Dir aaa>
+            <Module test_aaa.py>
+              <Function test_it>
+          <Module test_a.py>
+            <Function test_it>
+          <Package test_b>
+            <Module test_b.py>
+              <Function test_it>
+          <Module test_c.py>
+            <Function test_it>
+          <Package zzz>
+            <Module test_zzz.py>
+              <Function test_it>
+
+Previously, it was::
+
+    <Session>
+      <Module top/test_a.py>
+        <Function test_it>
+      <Module top/test_c.py>
+        <Function test_it>
+      <Module top/aaa/test_aaa.py>
+        <Function test_it>
+      <Package test_b>
+        <Module test_b.py>
+          <Function test_it>
+      <Package zzz>
+        <Module test_zzz.py>
+          <Function test_it>
+
+Code/plugins which rely on a specific shape of the collection tree might need to update.

doc/en/deprecations.rst

@@ -495,6 +495,91 @@ an appropriate period of deprecation has passed.
 Some breaking changes which could not be deprecated are also listed.
 
 
+Collection changes in pytest 8
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Added a new :class:`pytest.Directory` base collection node, which all collector nodes for filesystem directories are expected to subclass.
+This is analogous to the existing :class:`pytest.File` for file nodes.
+
+Changed :class:`pytest.Package` to be a subclass of :class:`pytest.Directory`.
+A ``Package`` represents a filesystem directory which is a Python package,
+i.e. contains an ``__init__.py`` file.
+
+:class:`pytest.Package` now only collects files in its own directory; previously it collected recursively.
+Sub-directories are collected as sub-collector nodes, thus creating a collection tree which mirrors the filesystem hierarchy.
+
+:attr:`session.name <pytest.Session.name>` is now ``""``; previously it was the rootdir directory name.
+This matches :attr:`session.nodeid <_pytest.nodes.Node.nodeid>` which has always been `""`.
+
+Added a new :class:`pytest.Dir` concrete collection node, a subclass of :class:`pytest.Directory`.
+This node represents a filesystem directory, which is not a :class:`pytest.Package`,
+i.e. does not contain an ``__init__.py`` file.
+Similarly to ``Package``, it only collects the files in its own directory,
+while collecting sub-directories as sub-collector nodes.
+
+Files and directories are now collected in alphabetical order jointly, unless changed by a plugin.
+Previously, files were collected before directories.
+
+The collection tree now contains directories/packages up to the :ref:`rootdir <rootdir>`,
+for initial arguments that are found within the rootdir.
+For files outside the rootdir, only the immediate directory/package is collected --
+note however that collecting from outside the rootdir is discouraged.
+
+As an example, given the following filesystem tree::
+
+    myroot/
+        pytest.ini
+        top/
+        ├── aaa
+        │   └── test_aaa.py
+        ├── test_a.py
+        ├── test_b
+        │   ├── __init__.py
+        │   └── test_b.py
+        ├── test_c.py
+        └── zzz
+            ├── __init__.py
+            └── test_zzz.py
+
+the collection tree, as shown by `pytest --collect-only top/` but with the otherwise-hidden :class:`~pytest.Session` node added for clarity,
+is now the following::
+
+    <Session>
+      <Dir myroot>
+        <Dir top>
+          <Dir aaa>
+            <Module test_aaa.py>
+              <Function test_it>
+          <Module test_a.py>
+            <Function test_it>
+          <Package test_b>
+            <Module test_b.py>
+              <Function test_it>
+          <Module test_c.py>
+            <Function test_it>
+          <Package zzz>
+            <Module test_zzz.py>
+              <Function test_it>
+
+Previously, it was::
+
+    <Session>
+      <Module top/test_a.py>
+        <Function test_it>
+      <Module top/test_c.py>
+        <Function test_it>
+      <Module top/aaa/test_aaa.py>
+        <Function test_it>
+      <Package test_b>
+        <Module test_b.py>
+          <Function test_it>
+      <Package zzz>
+        <Module test_zzz.py>
+          <Function test_it>
+
+Code/plugins which rely on a specific shape of the collection tree might need to update.
+
+
 :class:`pytest.Package` is no longer a :class:`pytest.Module` or :class:`pytest.File`
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 

doc/en/example/conftest.py

@@ -1 +1 @@
-collect_ignore = ["nonpython"]
+collect_ignore = ["nonpython", "customdirectory"]

doc/en/example/customdirectory.rst

@@ -0,0 +1,77 @@
+.. _`custom directory collectors`:
+
+Using a custom directory collector
+====================================================
+
+By default, pytest collects directories using :class:`pytest.Package`, for directories with ``__init__.py`` files,
+and :class:`pytest.Dir` for other directories.
+If you want to customize how a directory is collected, you can write your own :class:`pytest.Directory` collector,
+and use :hook:`pytest_collect_directory` to hook it up.
+
+.. _`directory manifest plugin`:
+
+A basic example for a directory manifest file
+--------------------------------------------------------------
+
+Suppose you want to customize how collection is done on a per-directory basis.
+Here is an example ``conftest.py`` plugin that allows directories to contain a ``manifest.json`` file,
+which defines how the collection should be done for the directory.
+In this example, only a simple list of files is supported,
+however you can imagine adding other keys, such as exclusions and globs.
+
+.. include:: customdirectory/conftest.py
+    :literal:
+
+You can create a ``manifest.json`` file and some test files:
+
+.. include:: customdirectory/tests/manifest.json
+    :literal:
+
+.. include:: customdirectory/tests/test_first.py
+    :literal:
+
+.. include:: customdirectory/tests/test_second.py
+    :literal:
+
+.. include:: customdirectory/tests/test_third.py
+    :literal:
+
+An you can now execute the test specification:
+
+.. code-block:: pytest
+
+    customdirectory $ pytest
+    =========================== test session starts ============================
+    platform linux -- Python 3.x.y, pytest-8.x.y, pluggy-1.x.y
+    rootdir: /home/sweet/project/customdirectory
+    configfile: pytest.ini
+    collected 2 items
+
+    tests/test_first.py .                                                [ 50%]
+    tests/test_second.py .                                               [100%]
+
+    ============================ 2 passed in 0.12s =============================
+
+.. regendoc:wipe
+
+Notice how ``test_three.py`` was not executed, because it is not listed in the manifest.
+
+You can verify that your custom collector appears in the collection tree:
+
+.. code-block:: pytest
+
+    customdirectory $ pytest --collect-only
+    =========================== test session starts ============================
+    platform linux -- Python 3.x.y, pytest-8.x.y, pluggy-1.x.y
+    rootdir: /home/sweet/project/customdirectory
+    configfile: pytest.ini
+    collected 2 items
+
+    <Dir customdirectory>
+      <ManifestDirectory tests>
+        <Module test_first.py>
+          <Function test_1>
+        <Module test_second.py>
+          <Function test_2>
+
+    ======================== 2 tests collected in 0.12s ========================

doc/en/example/customdirectory/conftest.py

@@ -0,0 +1,28 @@
+# content of conftest.py
+import json
+
+import pytest
+
+
+class ManifestDirectory(pytest.Directory):
+    def collect(self):
+        # The standard pytest behavior is to loop over all `test_*.py` files and
+        # call `pytest_collect_file` on each file. This collector instead reads
+        # the `manifest.json` file and only calls `pytest_collect_file` for the
+        # files defined there.
+        manifest_path = self.path / "manifest.json"
+        manifest = json.loads(manifest_path.read_text(encoding="utf-8"))
+        ihook = self.ihook
+        for file in manifest["files"]:
+            yield from ihook.pytest_collect_file(
+                file_path=self.path / file, parent=self
+            )
+
+
+@pytest.hookimpl
+def pytest_collect_directory(path, parent):
+    # Use our custom collector for directories containing a `mainfest.json` file.
+    if path.joinpath("manifest.json").is_file():
+        return ManifestDirectory.from_parent(parent=parent, path=path)
+    # Otherwise fallback to the standard behavior.
+    return None

doc/en/example/customdirectory/pytest.ini

@@ -0,0 +1,6 @@
+{
+    "files": [
+        "test_first.py",
+        "test_second.py"
+    ]
+}

doc/en/example/customdirectory/tests/manifest.json

@@ -0,0 +1,3 @@
+# content of test_first.py
+def test_1():
+    pass

doc/en/example/customdirectory/tests/test_first.py

@@ -0,0 +1,3 @@
+# content of test_second.py
+def test_2():
+    pass

doc/en/example/customdirectory/tests/test_second.py

@@ -0,0 +1,3 @@
+# content of test_third.py
+def test_3():
+    pass

doc/en/example/customdirectory/tests/test_third.py

@@ -32,3 +32,4 @@ The following examples aim at various use cases you might encounter.
    special
    pythoncollection
    nonpython
+   customdirectory

doc/en/example/index.rst

@@ -682,6 +682,8 @@ Collection hooks
 .. autofunction:: pytest_collection
 .. hook:: pytest_ignore_collect
 .. autofunction:: pytest_ignore_collect
+.. hook:: pytest_collect_directory
+.. autofunction:: pytest_collect_directory
 .. hook:: pytest_collect_file
 .. autofunction:: pytest_collect_file
 .. hook:: pytest_pycollect_makemodule
@@ -921,6 +923,18 @@ Config
 .. autoclass:: pytest.Config()
     :members:
 
+Dir
+~~~
+
+.. autoclass:: pytest.Dir()
+    :members:
+
+Directory
+~~~~~~~~~
+
+.. autoclass:: pytest.Directory()
+    :members:
+
 ExceptionInfo
 ~~~~~~~~~~~~~
 

doc/en/reference/reference.rst

@@ -27,8 +27,8 @@
 from _pytest.fixtures import fixture
 from _pytest.fixtures import FixtureRequest
 from _pytest.main import Session
+from _pytest.nodes import Directory
 from _pytest.nodes import File
-from _pytest.python import Package
 from _pytest.reports import TestReport
 
 README_CONTENT = """\
@@ -222,7 +222,7 @@ def pytest_make_collect_report(
         self, collector: nodes.Collector
     ) -> Generator[None, CollectReport, CollectReport]:
         res = yield
-        if isinstance(collector, (Session, Package)):
+        if isinstance(collector, (Session, Directory)):
             # Sort any lf-paths to the beginning.
             lf_paths = self.lfplugin._last_failed_paths
 

src/_pytest/cacheprovider.py

@@ -415,8 +415,6 @@ def __init__(self) -> None:
         # session (#9478), often with the same path, so cache it.
         self._get_directory = lru_cache(256)(_get_directory)
 
-        self._duplicatepaths: Set[Path] = set()
-
         # plugins that were explicitly skipped with pytest.skip
         # list of (module name, skip reason)
         # previously we would issue a warning when a plugin was skipped, but