Skip to content

[review wanted] add support for locally installed typehints #1895

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 5 commits into from
Closed

[review wanted] add support for locally installed typehints #1895

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
7350d5f
add support for locally installed typehints
tharvik Jul 18, 2016
ce70756
fix path, add comments
tharvik Jul 22, 2016
20ae3b5
document how to package stubs
tharvik Jul 25, 2016
4474e8a
extract default pkg paths in function
tharvik Jul 25, 2016
b7f6b9e
move subdir generation outside of loop
tharvik Jul 25, 2016
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 19 additions & 0 deletions docs/source/basics.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -176,6 +176,25 @@ There is more information about creating stubs in the
The following sections explain the kinds of type annotations you can use
in your programs and stub files.

Mypy will also look in some default install directory to try to find annotations.
For example, for UNIX based, with python 3.4, it will try to find it in
- ``${HOME}/.local/shared/typehints/python3.4/``
- ``/usr/shared/typehints/python3.4/``
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The standard directory is /usr/share/ (no d). And I think it would be better to put python* before typehints in the path, to mirror /usr/lib/pythonX.Y/. I'd also rather use /usr/lib/python* for this rather than introduce a second python tree under /usr/share (and maybe put it under site-packages?).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is actually based on the example in PEP 484 in chapter store and distribute stub files on how to add it to a setup.py.

...
data_files=[
    (
        'shared/typehints/python{}.{}'.format(*sys.version_info[:2]),
        pathlib.Path(SRC_PATH).glob('**/*.pyi'),
    ),
],
...

The standard directory is /usr/share/ (no d).

From the example, it is indeed 'shared', the '/usr' is because setup.py will install data_files under sys.prefix on system-wide install (based on install info on docs.python.org)

And I think it would be better to put python* before typehints in the path, to mirror /usr/lib/pythonX.Y/.

That's taken from the example, so that would require a PEP change, but I guess it can easily be introduce as it isn't really implemented for now.

I'd also rather use /usr/lib/python* for this rather than introduce a second python tree under /usr/share (and maybe put it under site-packages?).

/usr/lib/python* is for modules, not really for related data; IMO both options are valid, because typehints are closely related to code, but in itself, it isn't necessary for the code to run, thus not going into site-packages. /usr/ is also the standard place to store modules' related data.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm referring to the filesystem hierarchy standard: http://www.pathname.com/fhs/pub/fhs-2.3.html#USRSHAREARCHITECTUREINDEPENDENTDATA

If the PEP says /usr/shared, then the PEP should be corrected before this part of it is implemented.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hum, I see, and that makes sense, it's nicer. Then that would be a design choice coming from @gvanrossum or @JukkaL, and a separated issue IMO.

But I disagreed on 'before this part of it is implemented', we can merge it for now, and update mypy when the path changes are accepted in the PEP.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor Author

@tharvik tharvik Aug 2, 2016 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll try to rephrase/intepret what was said, correct me if I'm wrong

  • not having a separated python tree outside of /usr/lib/python*/site-packages
  • at least, using UNIX standard install location (not /usr/shared but /usr/share for example)

I guess, a way to do it would be to change the PEP to not use data_files but rather use package_data, as these are closer to the implementation than having a separated tree.

@bdarnell is that okay for you?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

package_data is fine with me.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've opened peps#67 about it, we can move the discussion there.

- ``/usr/local/shared/typehints/python3.4/``

If you want to package stubs for your library, write the ``.pyi`` alongside the
``.py`` files, and add the next line to your ``setup.py``,

.. code-block:: python

data_files=[
(
'shared/typehints/python{}.{}'.format(*sys.version_info[:2]),
pathlib.Path(SRC_PATH).glob('**/*.pyi'),
),
],


.. note::

You may be tempted to point ``MYPYPATH`` to the standard library or
Expand Down
21 changes: 21 additions & 0 deletions mypy/build.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@
import json
import os
import os.path
import site
import sys
import time
from os.path import dirname, basename
Expand Down Expand Up @@ -244,6 +245,16 @@ def mypy_path() -> List[str]:
return path_env.split(os.pathsep)


def get_pkg_locations():
# The paths are based on https://docs.python.org/3/install
yield site.USER_BASE # default user pkg dir
yield sys.prefix # default system pkg dir

if sys.platform != 'win32':
yield '/usr'
yield '/usr/local'


def default_lib_path(data_dir: str, pyversion: Tuple[int, int]) -> List[str]:
"""Return default standard library search paths."""
# IDEA: Make this more portable.
Expand All @@ -260,6 +271,16 @@ def default_lib_path(data_dir: str, pyversion: Tuple[int, int]) -> List[str]:
# E.g. for Python 3.2, try 3.2/, 3.1/, 3.0/, 3/, 2and3/.
# (Note that 3.1 and 3.0 aren't really supported, but we don't care.)
for v in versions + [str(pyversion[0]), '2and3']:
# Add package installed annotations.
# The idea is to implement the example in PEP 484, where the annotations
# are installed under shared/typehints/python.
# TODO it would be nicer to find typehints directly via pkg_resources
pkgsubdir = os.path.join('shared', 'typehints', 'python' + v)
for pkginstalldir in get_pkg_locations():
pkgstubdir = os.path.join(pkginstalldir, pkgsubdir)
if os.path.isdir(pkgstubdir):
path.append(pkgstubdir)

for lib_type in ['stdlib', 'third_party']:
stubdir = os.path.join(data_dir, 'typeshed', lib_type, v)
if os.path.isdir(stubdir):
Expand Down