Union[Generic[T], T]] does not work as expected as a parameter in the signature of a generic function

[haikuginger]

• Are you reporting a bug, or opening a feature request?

Bug

• Please insert below the code you are checking with mypy,
  or a mock-up repro if the source is private. We would appreciate
  if you try to simplify your case to a minimal repro.

from typing import Generic, TypeVar, Union

F = TypeVar("F")

class Z(Generic[F]):
    y: F
    def __init__(self, y: F):
        self.y = y


def q2(x: Union[Z[F], F]) -> F:
    if isinstance(x, Z):
        return x.y
    else:
        return x

def q3(x: Z[F]) -> F:
    return x.y

p = Z('hi')
reveal_type(p) # Z[builtins.str*]
reveal_type(q2)  # def [F] (x: Union[Z[F`-1], F`-1]) -> F`-1
q2(p)  # error: Argument 1 to "q2" has incompatible type "Z[str]"; expected "Z[<nothing>]"
q3(p)  # No error

• What is the actual behavior/output?

When creating a generic function that has a signature where one parameter is a union on a generic of a typevar and that typevar, calling with the generic causes an error.

• What is the behavior/output you expect?

Z[F] satisfies Union[Z[F], F]

• What are the versions of mypy and Python you are using?
  Do you see the same issue after installing mypy from Git master?

Python 3.6.6, using mypy 0.660; same issue happens with master.

• What are the mypy flags you are using? (For example --strict-optional)

No flags

[ilevkivskyi]

Yeah, inferring against unions is tricky. We have couple similar issues, but there is no simple solution.

Just to illustrate why it is tricky, in your case F = str and F = Z[str] are both valid solutions for F in that function call. Instead of guessing which one to take, mypy computes a meet of them (common subtype), which is <nothing> (bottom type) in this case.

Adding an upper bound to F might be a possible solution, but it looks like mypy checks the bound after computing the meet, which is probably a bug.

So the only option for you is to use overloads:

@overload
def q2(x: Z[F]) -> F: ...
@overload
def q2(x: F) -> F: ...
def q2(x):
    if isinstance(x, Z):
        return x.y
    else:
        return x

reveal_type(q2(p)) # Revealed type is "str"

which is a bit verbose but removes the ambiguity I explained above, because for overloads the variants are chosen in the order of appearance.

[haikuginger]

Thank you so much for the rapid response!

I think I understand.

If I'm reading this correctly, because F itself is satisfied by both str and Z[str] when the function is called as q2(x: Z[str]), an unambiguous fully-qualified type signature that includes an unambiguous return value can't be created - the return value has a definite relation to what F is, but the value of F is ambiguous.

To prevent the issue wholesale, then, when constructing Z we would need to make it generic on a typevar that would prevent Z[Z[F]] - "if the function is called with any Z[T], then necessarily, F=T", but it sounds like that might not work right now due to the bug around order of operations.

Is that an accurate understanding of the situation?

[ilevkivskyi]

Is that an accurate understanding of the situation?

Yes.

[glyph]

Adding an upper bound to F might be a possible solution, but it looks like mypy checks the bound after computing the meet, which is probably a bug.

For what it's worth, this is exactly the scenario that we originally encountered this bug in, since intuitively it did make sense that the inference for an unbound type var would be "hard" (even though it's definitely computable given the inputs here and mypy should totally do it!)

[ilevkivskyi]

Adding an upper bound to F might be a possible solution, but it looks like mypy checks the bound after computing the meet, which is probably a bug.

Hm, I just looked at this again, and it looks like adding an upper bound fixes the issue on master. I will add a regression test for this, but otherwise it looks like there are no action items here.

[ilevkivskyi]

Hm, I probably messed up something, I am trying it again and it doesn't work. It looks like there however might be a simple solution.