Skip to content

Improve stub files generated by stubgenc for pybind11 modules #5850

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 9 commits into from
Nov 22, 2018

Conversation

wiktorn
Copy link
Contributor

@wiktorn wiktorn commented Oct 28, 2018

Following changes fixes stubgenc that generates invalid *pyi files in following cases:

  1. Class constructors may sometimes get Any type as return value. After this change - for init None is the default
  2. For modules using pybind11, base class is pybind11_object which is non-existent (see Improve stub generation so it works better with pybind11 wrappers. #5814). After this change pybind11_object is removed
  3. Stubgenc doesn't add any import statements other than referencing some typing.* objects. If module references other types in different modules/packages no import statement is added. This pull requests adds import statements to referenced types

I'm not sure about 2, maybe pybind11 library should provide dummy pybind11_object in its module and annotate types, so dummy object will be assumed as superclass.

Strip module name from type information if its the same as annotated module or
add import statement.
@gvanrossum
Copy link
Member

gvanrossum commented Oct 28, 2018 via email

Copy link
Member

@gvanrossum gvanrossum left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for the late review -- this got caught in a busy period. I have a few requests for changes, but in general I think this is fine. (I do wonder if maybe we should switch to a more robust approach to handling argument lists rather than repeated splitting and stripping. But that's for a follow-up PR.)

mypy/stubgenc.py Outdated

if sig:
sig_types = []
for arg in (arg.split(':', 1) for arg in sig.split(',')):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm sorry, but this code is inscrutable, and I have no easy example in the stdlib to even test it (no C function in the stdlib has type annotations in its docstring). At the very least please don't use the same name (arg) for the two different variables -- the inner arg takes on string values like x: int or foo: C while the outer arg would take on list values like ['x', ' int'] and ['foo', ' C']. You might also want to add some comments explaining the different cases (len(arg) being 1 or not).

Copy link
Member

@gvanrossum gvanrossum left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(Sorry, marking as "changes requested".)

@wiktorn
Copy link
Contributor Author

wiktorn commented Nov 21, 2018

I hope that I correctly identified changes that you've requested. The only thing that may require further discussion is handling of pybind11_object.

Copy link
Member

@gvanrossum gvanrossum left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Everything looks good except I'd still like to see some improvements on the "inscrutable" for-loop. :-)

Thinking it through I believe the special case for pybind11_object is acceptable -- I assume you don't have a say in what pybind11 generates anyway...

mypy/stubgenc.py Outdated
if len(arg) == 1:
sig_types.append(arg[0].strip())
# convert signature in form of "self: TestClass, arg0: str" to list [[self, TestClass], [arg0, str]]
for arg_type in (arg.split(':', 1) for arg in sig.split(',')):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please rewrite this as follows:

# Convert signature in form of "self: testmod.TestClass, arg0: str" to list ["self: testmod.TestClass", "arg0: str"]
# and update imports with relevant modules (e.g. "testmod").
for arg in sig.split(','):
    arg_type = arg.split(':', 1)
    if len(arg_type) == 1:
    # etc.

wiktorn and others added 2 commits November 22, 2018 22:12
* Fix PEP8 line length
* simplify logic
* introduce testcase for generate_c_function_stub that verifies docstring parsing
@gvanrossum
Copy link
Member

LGTM, waiting for tests (I fixed one typo).

@gvanrossum gvanrossum merged commit 93c6e2c into python:master Nov 22, 2018
@gvanrossum
Copy link
Member

Thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants