Skip to content

stubtest: import submodules mentioned in __all__ #9943

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Jul 26, 2021
Merged

stubtest: import submodules mentioned in __all__ #9943

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
67d1b26
stubtest: import submodules mentioned in __all__
Jan 22, 2021
f4ee795
fix
Jan 23, 2021
1a17841
Merge branch 'master' into stubt
hauntsaninja Jul 26, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 13 additions & 6 deletions mypy/stubtest.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -161,6 +161,10 @@ def test_module(module_name: str) -> Iterator[Error]:
with warnings.catch_warnings():
warnings.simplefilter("ignore")
runtime = importlib.import_module(module_name)
# Also run the equivalent of `from module import *`
# This could have the additional effect of loading not-yet-loaded submodules
# mentioned in __all__
__import__(module_name, fromlist=["*"])
except Exception as e:
yield Error([module_name], "failed to import: {}".format(e), stub, MISSING)
return
Expand Down Expand Up @@ -200,26 +204,29 @@ def verify_mypyfile(
to_check = set(
m
for m, o in stub.names.items()
# TODO: change `o.module_public` to `not o.module_hidden`
if o.module_public and (not m.startswith("_") or hasattr(runtime, m))
)
runtime_public_contents = [
m
for m in dir(runtime)
if not m.startswith("_")
# Ensure that the object's module is `runtime`, e.g. so that we don't pick up reexported
# modules and infinitely recurse. Unfortunately, there's no way to detect an explicit
# reexport missing from the stubs (that isn't specified in __all__)
# Ensure that the object's module is `runtime`, since in the absence of __all__ we don't
# have a good way to detect re-exports at runtime.
and getattr(getattr(runtime, m), "__module__", None) == runtime.__name__
]
# Check all things declared in module's __all__, falling back to runtime_public_contents
to_check.update(getattr(runtime, "__all__", runtime_public_contents))
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe this is incorrect, you just have to updated with both __all__ and runtime_public_contents.

__all__ is not about things being public or private, it is just about from pgk import *. For example, you could have tons of public stuff in a package, but just list just the most useful things in __all__.

However, consider that __all__ can list names starting with understcore (_), not sure if those should be considered public or private in mypy, but from pgk import * do import them.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Type checkers use __all__ as a way to determine the public API:

https://mail.python.org/archives/list/[email protected]/message/FPTLFORIBPKY36UO7PLO47HJDCLG2KD5/

mypy/mypy/semanal.py

Line 483 in 5aa707d

def adjust_public_exports(self) -> None:

Note that due to the logic on L218 (on the right), there's a class of stuff that stubtest will check if it's present in the stub and runtime, but won't complain about if it's missing from the stub.

to_check.difference_update({"__file__", "__doc__", "__name__", "__builtins__", "__package__"})

for entry in sorted(to_check):
stub_to_verify = stub.names[entry].node if entry in stub.names else MISSING
assert stub_to_verify is not None
stub_entry = stub.names[entry].node if entry in stub.names else MISSING
if isinstance(stub_entry, nodes.MypyFile):
# Don't recursively check exported modules, since that leads to infinite recursion
continue
assert stub_entry is not None
yield from verify(
stub_to_verify,
stub_entry,
getattr(runtime, entry, MISSING),
object_path + [entry],
)
Expand Down