You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
stubgen: unify C extension and pure python stub generators with object oriented design (#15770)
This MR is a major overhaul to `stubgen`. It has been tested extensively
in the process of creating stubs for multiple large and varied libraries
(detailed below).
## User story
The impetus of this change is as follows: as a maintainer of third-party
stubs I do _not_ want to use `stubgen` as a starting point for
hand-editing stub files, I want a framework to regenerate stubs against
upstream changes to a library.
## Summary of Changes
- Introduces an object-oriented design for C extension stub generation,
including a common base class that is shared between inspection-based
and parsing-based stub generation.
- Generally unifies and harmonizes the behavior between inspection and
parsing approaches. For example, function formatting, import tracking,
signature generators, and attribute filtering are now handled with the
same code.
- Adds support for `--include-private` and `--export-less` to
c-extensions (inspection-based generation).
- Adds support for force enabling inspection-based stub generation (the
approach used for C extensions) on pure python code using a new
`--inspect-mode` flag. Useful for packages that employ dynamic function
or class factories. Also makes it possible to generate stubs for
pyc-only modules (yes, this is a real use case)
- Adds an alias `--no-analysis` for `--parse-only` to clarify the
purpose of this option.
- Removes filtering of `__version__` attribute from modules: I've
encountered a number of cases in real-world code that utilize this
attribute.
- Adds a number of tests for inspection mode. Even though these run on
pure python code they increase coverage of the C extension code since it
shares much of hte same code base.
Below I've compiled some basic information about each stub library that
I've created using my changes, and a link to the specialized code for
procedurally generating the stubs.
| Library | code type | other notes |
| --- | --- | --- |
|
[USD](https://github.com/LumaPictures/cg-stubs/blob/master/usd/stubgen_usd.py)
| boost-python | integrates types from doxygen |
|
[katana](https://github.com/LumaPictures/cg-stubs/blob/master/katana/stubgen_katana.py)
| pyc and C extensions | uses epydoc docstrings. has pyi-only packages |
|
[mari](https://github.com/LumaPictures/cg-stubs/blob/master/mari/stubgen_mari.py)
| pure python and C extensions | uses epydoc docstrings |
|
[opencolorio](https://github.com/LumaPictures/cg-stubs/blob/master/ocio/stubgen_ocio.py)
| pybind11 | |
|
[pyside2](https://github.com/LumaPictures/cg-stubs/blob/master/pyside/stubgen_pyside.py)
| shiboken | |
| substance_painter | pure python | basic / non-custom. reads types from
annotations |
| pymel | pure python | integrates types parsed from custom docs |
I know that this is a pretty big PR, and I know it's a lot to go
through, but I've spent a huge amount of time on it and I believe this
makes mypy's stubgen tool the absolute best available. If it helps, I
also have 13 merged mypy PRs under my belt and I'll be around to fix any
issues if they come up.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Jelle Zijlstra <[email protected]>
0 commit comments