Description
Originally reported by: Jurko Gospodnetić (BitBucket: jurko, GitHub: jurko)
Sometimes, when refactoring a project's test code, it becomes useful to extract some useful project specific testing functions into a separate testing utility module. A function in that module may then perform an assertion while actual test functions just call that utility function as needed.
Problem in this case is that such assertions are ignored when running the tests using the '-O' command-line option. However, assertions used directly inside test modules are not, since pytest's assertion rewriting preserves them.
This could be resolved by allowing a project to list extra modules pytest's assertion rewriting should be applied to. A new configuration option seems like a natural way to allow this.
Some alternative solution ideas:
- It is possible to refactor the project code so that the utility function just returns a True/False boolean indicator and then have the test function assert that the function returned True, but having the function perform the actual assertion often provides better error reports in case of test failure. For example, if you have a utility function comparing two XML structures, it seems much better if the test fails and reports that node X on one side and node X on the other side have different content, then if it just says 'the two XML documents do not match'.
- Utility functions could raise some sort of a project specific exception instead of an assertion failure - a bit smelly, would not directly provide a clean test failure explanation error message but would allow the user to drop into the debugger at the location where the exception was raised.
Best regards,
Jurko Gospodnetić