Stop suggesting MYPYPATH?

[gvanrossum]

Currently whenever mypy fails to find an import it prints a note suggesting to set MYPYPATH, e.g.:

main:1: error: Cannot find module named 'a.b.c'
main:1: note: (Perhaps setting MYPYPATH or using the "--ignore-missing-imports" flag would help)

In most cases setting MYPYPATH is an anti-pattern, since the issue is likely that this is a third-party module for which no stubs exist in typeshed (nor anywhere else). We used to see a lot of problems when people added site-packages to MYPYPATH.

Maybe we can add a more nuanced explanation to the docs and link to the docs from the note instead?

[JukkaL]

Yeah, that suggestion is often not good. It may be the correct thing to do if the user has multiple package roots, however. Also, --ignore-missing-imports is kind of a big hammer to suggest, but at least it generally solves the problem. Adding a link to the doc with a longer explanation seems like the right course of action. We might not have a good discussion of this topic in the docs yet, though.

Here are the various things that the user might do if a module can't be found, in order of desirability:

• If it's a third party module, install a stub package (or package with inline annotations) to the environment used to run mypy. [We might want to maintain a list of such packages in a central location (in typeshed?).]
• If no package with type information is available, consider writing a stub (it doesn't need to complete). If the user uses only a few functions from a module, this may be the most reasonable thing to do.
• If the module is part of the user's codebase, point MYPYPATH to it.
• If writing a stub is too much work, either use # type: ignore on the import or use the config file to ignore only imports to that module.
• If there are too many such imports to ignore, use --ignore-missing-imports.

[ilevkivskyi]

We might not have a good discussion of this topic in the docs yet, though.

As @msullivan suggested there are few docs on --ignore-missing-imports, I am going to add some as a part of #5228, but an entry in "Common issues" would be a separate PR.

Here are the various things that the user might do if a module can't be found, in order of desirability:

This is a good list, but too long to show as a note (and sometimes it is hard to diagnose which case applies). Just showing a link as a note would be better.

We might want to maintain a list of such packages in a central location (in typeshed?).

PEP 561 currently says that the name of stub packages on PyPI is fixed as "The name of the stub package MUST follow the scheme foopkg-stubs for type stubs for the package named foopkg". It means that it can be simply automated. Btw maybe pypi.org maintainers will be interested in having a dedicated (search-)page for stubs. Then typeshed can just have link to that page.

Also there is something missing in PEP 561 I think, it doesn't say whether the name foo-stubs is reserved for owners of foo. I think it would make to have such rule. @ethanhs what do you think?

[JukkaL]

Just showing a link as a note would be better.

Agreed.

PEP 561 currently says that the name of stub packages on PyPI is fixed as "The name of the stub package MUST follow the scheme foopkg-stubs for type stubs for the package named foopkg".

There is some gray area here. For example, what if we have stubs for package "foo" that rely on a mypy plugin to be useful, or some mypy type system features that aren't in a PEP yet. These might be in competition with the "official" stubs, which might only rely on features available in the released typing module. Is it impossible to distribute such stubs on PyPI? For example, we might want to publish a package called mypy-foo that contains a version of "foo" stubs that only work with mypy.

Another case where this could be a problem if foo-stubs is very slow to update and is way behind the official package, and the owners don't have capacity to review PRs. Does this mean that a fork that makes the stubs up to date would not be accepted on PyPI? Other packages can be forked; this is FOSS after all.

it doesn't say whether the name foo-stubs is reserved for owners of foo

I think that this could be problematic with the above rule, since some owners might not care about static typing enough to upload such a package, even if there is a lot of demand for the stubs. Also, many owners are only doing minimal maintenance for their package and creating stubs could be a lot of work, as we've learned.

[ilevkivskyi]

There is some gray area here.

I think PEP 561 should be more clear here and accommodate such use cases. It may be not important right now, but I could imagine possible problems in future with "competing" stub packages for the same library. A potential solution could be to allow names like numpy-stubs-more-words.

There is a problem however, what to do if mypy finds more than one installed package that matches numpy-stubs-*?

For example, we might want to publish a package called mypy-foo that contains a version of "foo" stubs that only work with mypy.

Yes, but with current version of the PEP, and more importantly its implementation, such package is not recognised by mypy as containing stubs for "foo".

[emmatyping]

Also there is something missing in PEP 561 I think, it doesn't say whether the name foo-stubs is reserved for owners of foo. I think it would make to have such rule. @ethanhs what do you think?

I suppose this could be included in the PEP, but I feel like it is more a decision to make by the PyPi folks.

I think PEP 561 should be more clear here and accommodate such use cases. It may be not important right now, but I could imagine possible problems in future with "competing" stub packages for the same library. A potential solution could be to allow names like numpy-stubs-more-words.

This was brought up in the initial discussion of the PEP. I don't think it is an issue. If you don't want to use the numpy-stubs distribution, for example, you can still have a distribution under another name which installs a package named numpy-stubs and everything should "just work". That way there can be multiple distributions on PyPi which can install the stub package, it is up to the user to manage their packaging environment.

[gvanrossum]

I'm with Ethan, competing stub packages can have different Distribution names but all install the same Package name (see PEP 561 for term definitions), so the user can choose.

[emmatyping]

Also FWIW it seems there is pypi/warehouse#4164 for reserving the names of stub packages on warehouse.

[ilevkivskyi]

I'm with Ethan, competing stub packages can have different Distribution names but all install the same Package name (see PEP 561 for term definitions), so the user can choose.

Ah yes, I am totally fine with this solution.

[emmatyping]

Also related to this we should probably stop recommending typeshed for missing stubs as well.

[edwardcwang]

As a user, I would like to add that getting this message was useful when libraries were in $PYTHONPATH but not in $MYPYPATH. Perhaps that's included in the point below, but having the current suggestion (perhaps in conjunction with something else for other scenarios) would be useful.

• If the module is part of the user's codebase, point MYPYPATH to it.

[gvanrossum]

If the module is part of the user's codebase, point MYPYPATH to it.

I like that phrasing. Or perhaps "if it's part of the user's code base but not in the current directory, ..."