Using mypy to test/compare stub package/distribution?

[wcooley]

• Are you reporting a bug, or opening a feature request?
  Uncertain; it's a bug if I'm trying to do something that should work or a feature request if I'm trying to do something unexpected. Or maybe it's a support request, to tell me what I'm doing wrong or if it's even possible to do what I am trying to do?
• What are the versions of mypy and Python you are using?
  mypy: 0.641
  Python: 3.6.5 built with pyenv on macOS 10.13.6
  Do you see the same issue after installing mypy from Git master?
  Yes, I installed "0.650+dev.2b1bb6e..." and tried cases 1.i and 1.ii below. Same result.
• Please insert below the code you are checking with mypy,
  or a mock-up repro if the source is private. We would appreciate
  if you try to simplify your case to a minimal repro.
  The library and stub repos are linked below; I am using the current master of each.
• What is the actual behavior/output?
• What is the behavior/output you expect?
• What are the mypy flags you are using? (For example --strict-optional)

I am attempting to make a stub package & distribution for the boltons library. It's working for regular client code that uses a module from boltons (see below), but I would like to use mypy to audit the boltons package itself to tell me what stubs are lacking. Should I be able to do this?

Baseline (what works)

I can build and install the dist as boltons-stubs as package boltons-stubs.

I can write a test script with the wrong type for a class from a boltons module:

from boltons.cacheutils import LRU
LRU(max_size=10, values={'one': 1, 'two': 2},
        on_miss=False)  # on_miss declared Optional[Callable[[str], int]]

And mypy produces the error I expect:

$ mypy client-test/test.py
client-test/test.py:3: error: Argument "on_miss" to "LRU" has incompatible type "bool"; expected "Optional[Callable[[str], int]]"

And then fix it and get no error, as expected.

What I have tried that does not work

It's a little hard to enumerate exactly every combination of parameters and setups that I've tried; if there is one that should and does work that I have not tried, then maybe it should be documented more clearly or the CLI flags should be more obvious?

In general, I've tried CLI parameters:

• with or without --warn-incomplete-stub.
• with either --strict or --disallow-untyped-defs.
• with --show-error-context (so I can see if my representative case of "Function is missing a type annotation" for LRU.__init__ shows up or not without having to remember line numbers or check the source).
• with -v or -vvvv or none.
• with or without --no-silence-site-packages.

I get either no output or errors, including the following:

boltons/cacheutils.py: note: In member "__init__" of class "LRU":
boltons/cacheutils.py:107: error: Function is missing a type annotation

1. Installing both boltons-stubs and boltons into a virtualenv and running mypy on boltons (from a clean directory, so I am not accidentally picking up either in '.'):
   1. I get "Can't find package 'boltons'" if I run mypy ... -p boltons:

      $ cd /tmp; mypy --no-silence-site-packages --show-error-context --disallow-untyped-defs --warn-incomplete-stub -p boltons
      Can't find package 'boltons'
      

   2. I get no output when I run mypy ... -m boltons.typeutils. The stub for boltons.typeutils is empty, so I know it is missing some declarations. If I examine the cache files .mypy_cache/3.6/boltons/typeutils.{data,meta}.json, it refers to the stub file in path, so I know it's seeing it:

      $ cd /tmp; mypy --no-silence-site-packages --show-error-context --disallow-untyped-defs --warn-incomplete-stub -m boltons.typeutils
      

   3. I get the following if I attempt to refer directly to the installed boltons/cacheutils.py:

      $ cd /tmp; mypy --no-silence-site-packages --show-error-context --disallow-untyped-defs --warn-incomplete-stub /my/virtualenv/python-boltons-stubs/lib/python3.6/site-packages/boltons/cacheutils.py
      /my/virtualenv/python-boltons-stubs/lib/python3.6/site-packages is in the PYTHONPATH. Please change directory so it is not.
      

2. Installing only boltons-stubs and pointing mypy at a working copy of the boltons repo (I have to remove the latter from the former's install_requires):
   1. mypy ... -m boltons.cacheutils:

   $ cd ~/src/boltons; mypy --no-silence-site-packages --show-error-context --disallow-untyped-defs --warn-incomplete-stub -m boltons.cacheutils
   ...
   boltons/cacheutils.py: note: In member "__init__" of class "LRU":
   boltons/cacheutils.py:107: error: Function is missing a type annotation
   ...
   

   2. mypy ... -p boltons: Same as 2.i.
   3. mypy ... boltons/cacheutils.py: Same as 2.i.
3. Uninstall boltons-stubs and boltons and manually copy the .pyi files into the boltons source: Same as 2.i.
4. Symlink the boltons-stubs package directory as boltons and set MYPYPATH to the path: Same as 2.i.

[davidism]

I have been typing Werkzeug, and ran into this issue as well. I had to use a stub pyi file for one module, since Generic makes instantiation slow until Python 3.9. However, mypy now ignores the code and only considers the stub file. So I can't both type check the Werkzeug code and provide the benefits of generic types to users.

[hauntsaninja]

I think the original issue is somewhat covered by stubtest and davidism's issue is better represented by #5028, so closing