Skip to content

Give warning when test function return other than None #9956

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 16 commits into from
May 25, 2022
Merged
Show file tree
Hide file tree
Changes from 10 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions AUTHORS
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,7 @@ Ceridwen
Charles Cloud
Charles Machalow
Charnjit SiNGH (CCSJ)
Cheuk Ting Ho
Chris Lamb
Chris NeJame
Chris Rose
Expand Down
1 change: 1 addition & 0 deletions changelog/7337.improvement.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Warns for test functions that return non-None.
10 changes: 10 additions & 0 deletions doc/en/deprecations.rst
Original file line number Diff line number Diff line change
Expand Up @@ -252,6 +252,16 @@ or ``pytest.warns(Warning)``.

See :ref:`warns use cases` for examples.


Returning non-None value in test functions
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. deprecated:: 7.0

:class:`pytest.PytestReturnNotNoneWarning` will be warn if a test function return values other than None.
It is to avoid typo of an `assert` statement as `return` in a test function makes it passes the test without warning or failing, regardless of if the test should have pass or not.


The ``--strict`` command-line option
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Expand Down
3 changes: 3 additions & 0 deletions doc/en/reference/reference.rst
Original file line number Diff line number Diff line change
Expand Up @@ -1130,6 +1130,9 @@ Custom warnings generated in some situations such as improper usage or deprecate
.. autoclass:: pytest.PytestExperimentalApiWarning
:show-inheritance:

.. autoclass:: pytest.PytestReturnNotNoneWarning
:show-inheritance:

.. autoclass:: pytest.PytestUnhandledCoroutineWarning
:show-inheritance:

Expand Down
9 changes: 9 additions & 0 deletions src/_pytest/python.py
Original file line number Diff line number Diff line change
Expand Up @@ -77,10 +77,12 @@
from _pytest.pathlib import visit
from _pytest.scope import Scope
from _pytest.warning_types import PytestCollectionWarning
from _pytest.warning_types import PytestReturnNotNoneWarning
from _pytest.warning_types import PytestUnhandledCoroutineWarning

if TYPE_CHECKING:
from typing_extensions import Literal

from _pytest.scope import _ScopeName


Expand Down Expand Up @@ -192,6 +194,13 @@ def pytest_pyfunc_call(pyfuncitem: "Function") -> Optional[object]:
result = testfunction(**testargs)
if hasattr(result, "__await__") or hasattr(result, "__aiter__"):
async_warn_and_skip(pyfuncitem.nodeid)
elif result is not None:
warnings.warn(
PytestReturnNotNoneWarning(
f"Expected None, but the test returned {result!r}, which will be an error in a "
"future version of pytest. Did you mean to use `assert` instead of `return`?"
)
)
return True


Expand Down
7 changes: 7 additions & 0 deletions src/_pytest/warning_types.py
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,13 @@ class PytestRemovedIn8Warning(PytestDeprecationWarning):
__module__ = "pytest"


@final
class PytestReturnNotNoneWarning(PytestDeprecationWarning):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be imported in src/pytest/__init__.py and added to __all__ there, since we want e.g. people to be able to ignore it via their config without having to import private API.

It should also be added to doc/en/reference/reference.rst with the other warnings, and probably get a new entry in doc/en/deprecations.rst.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the review, just want to be clear about the deprecations. Since we are already in 7, shall it be deprecated in 8? Or should I say 7?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be a warning, not a deprecation warning

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should it, if we plan to turn it into an error in the future?

Copy link
Member

@nicoddemus nicoddemus May 25, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Deprecation warning is correct if we want to turn it into an error in the future, which seems the plan for unittest in the stdlib too: #7337 (comment).

python/cpython#27748

"""Warning emitted when a test function is returning value other than None."""

__module__ = "pytest"


@final
class PytestExperimentalApiWarning(PytestWarning, FutureWarning):
"""Warning category used to denote experiments in pytest.
Expand Down
2 changes: 2 additions & 0 deletions src/pytest/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -69,6 +69,7 @@
from _pytest.warning_types import PytestDeprecationWarning
from _pytest.warning_types import PytestExperimentalApiWarning
from _pytest.warning_types import PytestRemovedIn8Warning
from _pytest.warning_types import PytestReturnNotNoneWarning
from _pytest.warning_types import PytestUnhandledCoroutineWarning
from _pytest.warning_types import PytestUnhandledThreadExceptionWarning
from _pytest.warning_types import PytestUnknownMarkWarning
Expand Down Expand Up @@ -127,6 +128,7 @@
"PytestDeprecationWarning",
"PytestExperimentalApiWarning",
"PytestRemovedIn8Warning",
"PytestReturnNotNoneWarning",
"Pytester",
"PytestPluginManager",
"PytestUnhandledCoroutineWarning",
Expand Down
13 changes: 13 additions & 0 deletions testing/acceptance_test.py
Original file line number Diff line number Diff line change
Expand Up @@ -1292,3 +1292,16 @@ def test_no_brokenpipeerror_message(pytester: Pytester) -> None:

# Cleanup.
popen.stderr.close()


def test_function_return_non_none_warning(testdir) -> None:
testdir.makepyfile(
"""
def test_stuff():
return "something"
"""
)
res = testdir.runpytest()
res.stdout.fnmatch_lines(
["*PytestReturnNotNoneWarning: Expected None, but the test returned*"]
)