How to intercept setup_module and setup_class when it fails from own plugin file

[shreyashah]

If a setup_module() fails, then the error is propagated to all the test cases and all those test cases go into 'ERROR' state showing the same propagated error. This is understood.

However, I want to intercept using a hook or a method when setup_module() fails. I tried pytest_exception_interact() hook, but that seems to come into picture if a test case fails/errors.

How do I intercept in my plugin file when setup_module() fails to execute some custom things?
Similarly, how do I iintercept when setup_class fails?

[pytestbot]

GitMate.io thinks possibly related issues are #1164 (test_something fails), #74 (test_cmdline_python_package() fails), #747 (flakes test fail), #1777 (sigalrm fails test), and #1845 (Some tests fail on FreeBSD).

[nicoddemus]

Unfortunately I don't think this is currently possible, setup_* methods are called outside of the fixture mechanism and can't be intersected I think.

[shreyashah]

@nicoddemus : Ok, got that. However, is there a way when a test case goes into an 'ERROR' state, we can determine was it because of setup_module() failure or setup_class() failure. Anywhere in pytest_exception_interact() hook or other item hook we can determine this?

I hv seen for a node, node.session.exitstatus=0 when setup_module() fails or setup_class() fails. But, can't tell from this exit status that who failed- setup_module() or setup_class()

[nicoddemus]

Anywhere in pytest_exception_interact() hook or other item hook we can determine this?

It might be possible to write a hook wrapper around pytest_runtest_setup, but I don't know a way to distinguish between a setup_module or setup_class failure.

Can you describe what you are trying to accomplish? Perhaps by understanding your use case we might suggest alternatives.

Also have you considered switching to fixtures? That would make things much easier to deal with.

[shreyashah]

Ok.So what I am trying to do is invoke our own debug infra whenever an exception is encountered, wherever that exception is coming from - be it from setup_module(), setup_class or from the test case itself. In current scenario, I have implemented pytest_exception_interact() hook where is check for report.outcome==failed and process the call.excinfo for further debug infra stuff.

Now assume setup_module() fails, then all the test cases (say I have 50 tests) inside that script will land up in pytest_exception_interact() hook because all these test's report.outcome is = failed. I don't want to execute the code in pytest_exception_interact() hook 50 times because it adds to my run time and its obvious that all these 50 tests are going to be in "ERROR" state, but executing the code in pytest_exception_interact() is redundant.

[nicoddemus]

I see @shreyashah thanks for the clarification. Unfortunately I'm not sure it is possible to know if the failure at the point where pytest_exception_interact happens originated in a setup_module or setup_class function. 😕

[nicoddemus]

@RonnyPfannschmidt do you have any pointers here perhaps?

[RonnyPfannschmidt]

other than walking the stack, nope

[nicoddemus]

What if we passed the request object and/or the report object to pytest_exception_interact? 🤔

EDIT: report is already passed to pytest_exception_interact. 😅

[RonnyPfannschmidt]

Afair We don't have those there

[nicoddemus]

Yeah we would need some refactor to access the request object (which can provide the scope), or add it to some other object.