Type substitutions in Callable ignore TypeVar bounds

[mthuurne]

When I run mypy on the following code:

from typing import Callable, TypeVar

class Base:
    pass

class Sub(Base):
    pass

class Other:
    pass

BT = TypeVar('BT', bound=Base)

def check(obj: BT) -> None:
    pass

check(Base())
check(Sub())
check(Other())

Factory = Callable[[], BT]

def f() -> Other:
    pass

fAny:   Factory        = f
fBase:  Factory[Base]  = f
fSub:   Factory[Sub]   = f
fOther: Factory[Other] = f

It outputs:

19: error: Value of type variable "BT" of "check" cannot be "Other"  [type-var]
    check(Other())
    ^
27: error: Incompatible types in assignment (expression has type "Callable[[], Other]",
variable has type "Callable[[], Base]")  [assignment]
    fBase:  Factory[Base]  = f
                             ^
28: error: Incompatible types in assignment (expression has type "Callable[[], Other]",
variable has type "Callable[[], Sub]")  [assignment]
    fSub:   Factory[Sub]   = f
                             ^

All of the things mypy warns about are valid. But there are also statements it doesn't warn about that are incorrect code.

The most clear one is the type annotation of fOther (last line of the test case): while the function f does indeed return an object of type Other, that type gets substituted for BT in the definition of Factory and BT has Base as its bound, which is a type unrelated to Other.

A similar but more complex case can be made for fAny, which doesn't provide a type argument. However, there cannot exist a type argument that would satisfy both the bound of BT and the signature of f.

I have limited knowledge of the internals of mypy, but it seems feasible that the fOther case could be reported. If the fAny case can be reported as well that would be great, but I can imagine that determining that no valid type argument can exist is a lot harder than validating a given type argument.

I'm using mypy 0.770 on Python 3.8.2.

[JukkaL]

When substituting values for type variables in generic type aliases, mypy doesn't check that the value is compatible with the bound. Here's a simplified repro:

from typing import Callable, TypeVar, List

N = TypeVar('N', bound='int')

F = Callable[[], N]
L = List[N]

x: F[str]  # No error, but should be error
y: L[str]  # No error, but should be error

[JukkaL]

The fAny case is working as designed. If a type variable is omitted, it defaults to Any. It's pretty confusing, and I'd suggest using --disallow-any-generics.

[mthuurne]

The fAny case is working as designed. If a type variable is omitted, it defaults to Any.

What does Any mean exactly? From the name, I would assume it means "any type": a unique anonymous TypeVar. But in how it is implemented, it seems to mean "we don't know anything about this type".

In your simplified repro, y: L[T] implies:

• isinstance(a, T) holds for every element a in y (property of List)
• issubclass(T, int) holds (bound of N)

Combining both implications, this means that for any type T that can be substituted for N, isinstance(a, int) holds for every element a in y.

So even if we don't know which concrete type is passed to L, we do know that the elements of the list must all be integers. But this information is lost if L[Any] is evaluated as List[Any].

Note that the fAny case isn't important to me as a programmer, but I think it's interesting from a theoretical point of view.

[mthuurne]

I opened a separate issue about the handling of Any: #8981, since that is not about checking the substitution, but about remembering the bound in the substitution result.

[BvB93]

The fAny case is working as designed. If a type variable is omitted, it defaults to Any.

Should this be the case though?
The fact that the bound parameter has been specified implies that the BT typevar cannot adopt any arbitrary value;
yet defaulting to Any suggests the opposite.

Would it not make more sense to default to whichever value is specified as bound?

[EdwardBlair]

When substituting values for type variables in generic type aliases, mypy doesn't check that the value is compatible with the bound. Here's a simplified repro

Deserves separate issue.

[posita]

Still happens in 0.931.

[ilevkivskyi]

Mypy now checks for type variable bounds in type aliases.