Error in pytest possibly relating to pytest-benchmark in version `5.0.0`

[dpanici]

Hello,

Unsure if this is the main source of. the error or if it is an upstream bug with a package that depends on pytest-benchmark, but we use pytest-benchmark in our CI, in addition to other plugins to run jupyter notebooks as part of the CI.

Upon the update from 4.0.0 to 5.0.0 we've found our CI failing, with the exact cause unclear, but it seems to relate to pytest-benchmark from the error logs.
The CI runner is on Linux, the details of the specifics and the environment are in the aforementioned linked error logs.

With 4.0.0 our tests pass. Again, unsure if this is a specific issue with pytest-benchmark or related to another package we use which depends on pytest-benchmark that broke when 5.0.0 was released, but if you have any insight into the error we are getting that would be greatly appreciated!

If it helps, our CI yml that is failing is https://github.com/PlasmaControl/DESC/blob/master/.github/workflows/notebook_tests.yml and line 79 of said file has the specific pytest command that results in the error.

[ionelmc]

Yeah I added a type check so that the "benchmark" fixture is not defined to something else. The goal was to give a clear error message for situations like #265.

In your project apparently that is not a problem (I guess you do define a "benchmark" fixture but it's scoping prevents the bad interaction in #265). I'll convert that to a warning....

[ionelmc]

So it's still an error but I fixed the check. Released a 5.0.1.