pytest broken with: assert type in (None, "pathlist", "args", "linelist", "bool")

[saper]

Things to check first

• I have searched the existing issues and didn't find my bug already reported there

• I have checked that my bug is still present in the latest release

AnyIO version

4.11.0

Python version

3.9

What happened?

When having anyio==4.11.0 as a dependency, running old pytest gives AssertionError.

How can we reproduce the bug?

1. Create python 3.9 virtual environment (we are using AlmaLinux 9 or Red Hat Enterprise Linux 9)

2. Create the following requirements file repro.txt with the following list

    anyio==4.11.0
      exceptiongroup==1.3.0
        typing_extensions==4.15.0
      idna==3.11
      sniffio==1.3.1
      typing_extensions==4.15.0
pipdeptree==2.28.0
  packaging==25.0
  pip==25.3
pytest==6.1.2
  attrs==25.4.0
  iniconfig==2.1.0
  packaging==25.0
  pluggy==0.13.1
  py==1.11.0
  toml==0.10.2

3. Install the dependencies with

   pip install -r repro.txt

4. Run the following command:

   pytest .

We are getting the following exception:

Traceback (most recent call last):
  File "/run/user/503/bug-1028-repro/bin/pytest", line 8, in <module>
    sys.exit(console_main())
  File "/run/user/503/bug-1028-repro/lib/python3.9/site-packages/_pytest/config/__init__.py", line 187, in console_main
    code = main()
  File "/run/user/503/bug-1028-repro/lib/python3.9/site-packages/_pytest/config/__init__.py", line 143, in main
    config = _prepareconfig(args, plugins)
  File "/run/user/503/bug-1028-repro/lib/python3.9/site-packages/_pytest/config/__init__.py", line 318, in _prepareconfig
    config = pluginmanager.hook.pytest_cmdline_parse(
  File "/run/user/503/bug-1028-repro/lib/python3.9/site-packages/pluggy/hooks.py", line 286, in __call__
    return self._hookexec(self, self.get_hookimpls(), kwargs)
  File "/run/user/503/bug-1028-repro/lib/python3.9/site-packages/pluggy/manager.py", line 93, in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
  File "/run/user/503/bug-1028-repro/lib/python3.9/site-packages/pluggy/manager.py", line 84, in <lambda>
    self._inner_hookexec = lambda hook, methods, kwargs: hook.multicall(
  File "/run/user/503/bug-1028-repro/lib/python3.9/site-packages/pluggy/callers.py", line 203, in _multicall
    gen.send(outcome)
  File "/run/user/503/bug-1028-repro/lib/python3.9/site-packages/_pytest/helpconfig.py", line 100, in pytest_cmdline_parse
    config = outcome.get_result()  # type: Config
  File "/run/user/503/bug-1028-repro/lib/python3.9/site-packages/pluggy/callers.py", line 80, in get_result
    raise ex[1].with_traceback(ex[2])
  File "/run/user/503/bug-1028-repro/lib/python3.9/site-packages/pluggy/callers.py", line 187, in _multicall
    res = hook_impl.function(*args)
  File "/run/user/503/bug-1028-repro/lib/python3.9/site-packages/_pytest/config/__init__.py", line 1003, in pytest_cmdline_parse
    self.parse(args)
  File "/run/user/503/bug-1028-repro/lib/python3.9/site-packages/_pytest/config/__init__.py", line 1280, in parse
    self._preparse(args, addopts=addopts)
  File "/run/user/503/bug-1028-repro/lib/python3.9/site-packages/_pytest/config/__init__.py", line 1172, in _preparse
    self.pluginmanager.load_setuptools_entrypoints("pytest11")
  File "/run/user/503/bug-1028-repro/lib/python3.9/site-packages/pluggy/manager.py", line 300, in load_setuptools_entrypoints
    self.register(plugin, name=ep.name)
  File "/run/user/503/bug-1028-repro/lib/python3.9/site-packages/_pytest/config/__init__.py", line 436, in register
    ret = super().register(plugin, name)  # type: Optional[str]
  File "/run/user/503/bug-1028-repro/lib/python3.9/site-packages/pluggy/manager.py", line 127, in register
    hook._maybe_apply_history(hookimpl)
  File "/run/user/503/bug-1028-repro/lib/python3.9/site-packages/pluggy/hooks.py", line 333, in _maybe_apply_history
    res = self._hookexec(self, [method], kwargs)
  File "/run/user/503/bug-1028-repro/lib/python3.9/site-packages/pluggy/manager.py", line 93, in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
  File "/run/user/503/bug-1028-repro/lib/python3.9/site-packages/pluggy/manager.py", line 84, in <lambda>
    self._inner_hookexec = lambda hook, methods, kwargs: hook.multicall(
  File "/run/user/503/bug-1028-repro/lib/python3.9/site-packages/pluggy/callers.py", line 208, in _multicall
    return outcome.get_result()
  File "/run/user/503/bug-1028-repro/lib/python3.9/site-packages/pluggy/callers.py", line 80, in get_result
    raise ex[1].with_traceback(ex[2])
  File "/run/user/503/bug-1028-repro/lib/python3.9/site-packages/pluggy/callers.py", line 187, in _multicall
    res = hook_impl.function(*args)
  File "/run/user/503/bug-1028-repro/lib/python3.9/site-packages/anyio/pytest_plugin.py", line 68, in pytest_addoption
    parser.addini(
  File "/run/user/503/bug-1028-repro/lib/python3.9/site-packages/_pytest/config/argparsing.py", line 176, in addini
    assert type in (None, "pathlist", "args", "linelist", "bool")
AssertionError

If we use anyio==4.9.0 with the following requirements

anyio==4.9.0
  exceptiongroup==1.3.0
    typing_extensions==4.15.0
  idna==3.11
  sniffio==1.3.1
  typing_extensions==4.15.0
pipdeptree==2.28.0
  packaging==25.0
  pip==25.3
pytest==6.1.2
  attrs==25.4.0
  iniconfig==2.1.0
  packaging==25.0
  pluggy==0.13.1
  py==1.11.0
  toml==0.10.2

I am getting pytest to run (with an expected test collecting error):

============================= test session starts ==============================
platform linux -- Python 3.9.21, pytest-6.1.2, py-1.11.0, pluggy-0.13.1
rootdir: /run/user/503
plugins: anyio-4.9.0
collected 0 items / 1 error

==================================== ERRORS ====================================
________________________ ERROR collecting test session _________________________
bug-1028-repro-olderanyio/lib/python3.9/site-packages/_pytest/runner.py:310: in from_call
    result = func()  # type: Optional[TResult]
bug-1028-repro-olderanyio/lib/python3.9/site-packages/_pytest/runner.py:340: in <lambda>
    call = CallInfo.from_call(lambda: list(collector.collect()), "collect")
bug-1028-repro-olderanyio/lib/python3.9/site-packages/_pytest/main.py:684: in collect
    for direntry in visit(str(argpath), self._recurse):
bug-1028-repro-olderanyio/lib/python3.9/site-packages/_pytest/pathlib.py:570: in visit
    yield from visit(entry.path, recurse)
bug-1028-repro-olderanyio/lib/python3.9/site-packages/_pytest/pathlib.py:570: in visit
    yield from visit(entry.path, recurse)
bug-1028-repro-olderanyio/lib/python3.9/site-packages/_pytest/pathlib.py:570: in visit
    yield from visit(entry.path, recurse)
bug-1028-repro-olderanyio/lib/python3.9/site-packages/_pytest/pathlib.py:566: in visit
    entries = sorted(os.scandir(path), key=lambda entry: entry.name)
E   PermissionError: [Errno 13] Permission denied: '/run/user/503/systemd/inaccessible/dir'
=========================== short test summary info ============================
ERROR  - PermissionError: [Errno 13] Permission denied: '/run/user/503/system...
!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!
=============================== 1 error in 0.14s ===============================

[saper]

As mentioned in #667, the behaviour to load pytest plugin automatically is very surprising.

We have this library (which I have never heard before of) as a dependency of something else and there was no intention to use this plugin.

The particular problem is caused by this line https://github.com/agronholm/anyio/pull/971/files#diff-91193a2cf8ab1b4f54ebe54e80a6d2e3d198b998b9ee58d5a77af306107054fdR72 which requires https://github.com/pytest-dev/pytest/pull/7872/files which got included in pytest 6.2.0 (well, not just documentation change).

[altendky]

so i guess one perspective on the error here is that anyio does not specify pytest 6.2.0 as a minimum compatibilty. another is your point about being unable to install anyio without installing the pytest plugin. trio, for example, provides a separate pytest-trio package.

just to note it, pytest 6.2.0 is ancient and python 3.9 is EOL. but... that doesn't make the above two points invalid.

[saper]

This trouble has been caused by a tiny omission that could not be easily tested. The real problem is that we bundle the plugin, see #1029

[agronholm]

By default, pytest automatically loads all plugins it can find. This is the first time anybody has complained about the AnyIO plugin doing something unwanted simply by configuring itself. If you wish to disable the plugin, you can add -p no:anyio to pytest options. Alternatively, you can disable pytest's autoloading of plugins (if indeed you don't want any plugins to be autoloaded) by setting PYTEST_DISABLE_PLUGIN_AUTOLOAD=1.

[saper]

We are using various pytest plugins by adding them as a test dependency to the project that needs them.

I don't think anybody wants to enable anyio pytest plugin just because they depend on fastapi or something else.

A quick fix for this particular assertion is in #1030, basically a one-liner.

[agronholm]

so i guess one perspective on the error here is that anyio does not specify pytest 6.2.0 as a minimum compatibilty.

Is there a way to actually do that?

[altendky]

hmm, yeah... kinda. but not really really. i've often done stuff along the lines of an extra testing with the requirement so that it is opt-in. but your question is mostly pointing out the lack of ability for a project to specify a constraint instead of a requirement which i indeed did not think through. sorry...

i'm pretty sure we could make the plugin not do anything much unless it were explicitly enabled.

pytest-dev/pytest#7675 (comment)
https://docs.pytest.org/en/6.2.x/reference.html#pytest.config.PytestPluginManager.unregister

though, of course, anything in this direction of defaulting to some form of 'disabled' carries the cost of losing backwards compatibility for those that want the plugin as well as being an outlier compared to other plugins. a separate package like pytest-anyio may be the least surprising option. (that i am aware of so far) it could be released pre-emptively and then the in-anyio package plugin later removed. or the plugin straight up moved but with a temporary anyio -> pytest-anyio dependency that would later be removed as a breaking change.

good:

• similar to other projects
• having a pytest dependency with minimum version would be accurate
• plugin only present when wanted

bad:

• breaking change
• more projects to release
• inter-compatibility between the two packages

i'll note that i'm only aware of this issue due to my having authored the pytest pr that exposed the option type that's now used. i am not a significant contribute here. also, i would guess that the balance that will be achieved here will at least temporarily involve any projects running into this issue explicitly disabling the plugin themselves. that just seems like the right tradeoff between a breaking change for the projects that want the plugin vs. the ones that don't and for which it causes issues. i'm sorry for the hassle here, this is just my pragmatic take on reality following my theoretical correctness exploration above.

[agronholm]

I designed the AnyIO pytest plugin to be as unintrusive as possible squarely because it was included with AnyIO itself. But I didn't foresee the scenario of people updating only AnyIO and not other dependencies too. I'm now trying to figure out how to use uv to install the oldest possible dependencies in a separate test job to check for this.