pytest.parametrize fails on test function with non-signature-preserving decorator on Python 3 #3435
Comments
|
GitMate.io thinks possibly related issues are #1007 (pytest-xdist appears to fail on Python 3.5), #1111 (pytest.mark.parametrize fails with lambdas), #18 (pytest-2.0.0 fails testing/test_nose.py on python-3.1.3), #2132 (pytest tests explicitly check for ImportError, fails on Python 3.6), and #2975 (Pytest 3.3: tests being skipped without failing). |
|
as far as i can tell the decorator library is completely wrong at preserving python signatures adhering to the for sanity/safety pytest unpacks wrappers unless they declare a valid signature |
nicoddemus
added
status: needs information
|
Note that the decorator package is used only by our existing approach, which works on py2 and py3. Our new approach which works on py2 but not on py3 uses functools.update_wrapper(). The pytest version used in both approaches on both py2 and p3 is the same: 3.5.1. |
|
@andy-maier functools.update_wrapper is broken on python2 |
|
ok, so can you tell me whether we can fix this, and if so, how? |
|
@andy-maier attaching a |
|
Ok, great. We will try that. |
|
Because the So this returns exactly whatz I would expect, but then maybe my expectation is incorrect ;-) When following the wrapper, it returns the signature of the original function (which would be incorrect), and when not following the wrapper, it returns the signature of the outer (wrapper) function (which would be correct). So if I understand your statement above right:
then pytest does not trust the (correct) result of |
|
i believe this might have been an oversight when switching to using signature objects CC @nicoddemus historically pytest used various forms of unpacking to deal with different kinds of decorators so its entirely possible that this is a unclear interaction and we need to find a way to shift toward consistency |
|
@RonnyPfannschmidt Hi Ronny, I first did not want to believe your recommendation because it basically said that exactly in all environments where it worked, some function would be flawed. However, it turned out you were exactly right. So thanks a lot!! I now have it running on all Python versions we support (2.6+, 3.4+) for both our minimum package levels and the latest package levels. I had to inrease the minimum package levels of pytest to 3.3.2, and of "py" to 1.5.1. The key parts of our signature-changing Setting the signature for Python 2 was not necessary to make it work, but I am setting it nevertheless, in case the flawed behavior on Python 2 gets fixed one day. Let me know if you have further comments. Andy |
|
@andy-maier the main problem is that on python2 the that issue has been fixed on python3, which in turn makes things fall apart as they should |
|
Thanks for the info. My original problem is solved now. I guess what remains to be done is for pytest to figure out whether to follow up on what Ronny pointed out 4 comments above. |
|
thanks for your follow-up, i created a tracking issue |
In the pywbem project, we are using
pytest.parametrizeon top of our own decoratorpytest_extension.test_functionthat handles expected exceptions, expected warnings, and some more things, in an attempt to simplify the test function that is coded. That decorator is signature-preserving. An example of such a test function is here. This runs fine on Python 2 and Python 3 since a while now. There is no issue with that, and I'm just mentioning this as a basis.Now we have made an attempt to further simplify the test function that is coded, with a new variant
pytest_extension.test_function_newof our decorator, which eliminates the explicit unpacking of kwargs passed to the test function, and avoids having unused arguments. That new decorator is signature-changing. Its wrapper function provides exactly the signature that is needed by thepytest.parametrizedecorator. The accordingly modified example test function is here. This runs fine on Python 2 but not on Python 3.On Python 3.6, it fails while collecting testcases (see this Travis job):
In that Travis job, the packages in the venv were:
It also fails on Python 3.4 with older package versions, see this Travis job. The job log shows the
pip list.From the error that is raised, my conclusion was that pytest looks at the signature of the original test function when checking the signature of the function to be called by parametrize, instead of looking at the signature of its wrapper function. At least on Python 3.
I have debugged into pytest and was found that conclusion to be confirmed, but when trying to find the root cause, I gave up at some point.
In case you want to reproduce the issue, create a Python 3 venv, clone the pywbem repo, and issue
make allin its work directory.Some interesting questions may be:
-> Why does it succeed on Python 2 but fail on Python 3?
-> Is the use of a signature-changing decorator ok in combination with
pytest.parametrize?The text was updated successfully, but these errors were encountered: