IO[Any] is not inferred as the supertype of IO[str] and IO[bytes] in condional expression v = iostr if x else iobytes

[MestreLion]

Bug Report

Consider the following snippet, adapted from argparse.FileType:

import sys

def opener(path: str, mode: str = "r") -> None:
    if path and path != "-":
        fh = open(path, mode)
    else:
        if "r" in mode:
            fh = sys.stdin.buffer if 'b' in mode else sys.stdin
        elif any(c in mode for c in 'wax'):
            fh = sys.stdout.buffer if 'b' in mode else sys.stdout
        else:
            # intentionally not using custom exception
            raise ValueError(f"Tried to open '-' (stdin/stdout) with mode {mode!r}")
    ...

mypy gives errors for line 8 and 10 (the fh assignments using sys.stdin and sys.stdout):

error: Incompatible types in assignment (expression has type "object", variable has type "IO[Any]")  [assignment]
error: Incompatible types in assignment (expression has type "object", variable has type "IO[Any]")  [assignment]

How come the expressions sys.stdin.buffer if 'b' in mode else sys.stdin and sys.stdout.buffer if 'b' in mode else sys.stdout has type evaluated as object? Both sys.stdin and sys.stdout are strictly IO[str], and their buffers are IO[bytes], hence I expected both expressions to actually be IO[Any], matching the inferred type of open().

If I break the x = a if y else b into proper if/else statements, mypy does not complain:

    if "r" in mode:
        if 'b' in mode:   
            fh = sys.stdin.buffer
        else:
            fh = sys.stdin
    else:
        if 'b' in mode:   
            fh = sys.stdout,buffer
        else:
            fh = sys.stdout

Your Environment

Python 3.8 from apt repository and pip-installed mypy 1.4.1 (compiled: yes) in a venv in Ubuntu 18.04

[ikonst]

The first assignment determines the implicit type. In the ternary case, there's just one.

As for object, I think it's one of https://github.com/python/mypy/issues?q=is%3Aissue+is%3Aopen+label%3Atopic-join-v-union

[MestreLion]

The first assignment determines the implicit type. In the ternary case, there's just one.

Yes, first assignment determines the implicit type, in this case, fh = open(path, mode), which is inferred as IO[Any], as expected. The ternary can result in either IO[str] and IO[bytes], which I assume are subtypes of IO[Any], so I expected the ternary expression to evaluate as their supertype IO[Any], not object

What do you mean by "In the ternary case, there's just one"?

As for object, I think it's one of https://github.com/python/mypy/issues?q=is%3Aissue+is%3Aopen+label%3Atopic-join-v-union

Is there any workaround, besides breaking the ternary one-liners into multiple if / else?

[A5rocks]

I don't think mypy combines two non-Any types to become Any.

You can type-annotate the fh variable and mypy will backtrack from "fh is IO[Any]" to "OK, each part of the if can be IO[Any]" to "OK this looks good" err well now that it's been 15 minutes and I'm rereading this issue, I'm not sure that'll work. In fact that's... probably a bug? IDK the type of sys.stdout though.

[erictraut]

The supertype of IO[str] and IO[bytes] is not IO[Any]. Any doesn't participate in supertype or subtype relationships, (which are defined in terms of sets). Any is a special case that exists outside of supertype and subtype relationships for purposes of gradual typing. For details, refer to PEP 483. For this reason, it wouldn't be appropriate for a type checker to evaluate the type as IO[Any] in this situation.

The supertype of IO[str] and IO[bytes] is either a union of the two types (IO[str] | IO[bytes]) or the join of the two types (object). Mypy generally uses join instead of union, although there are a few special cases where it uses the latter. In this case, it's apparently using a join operation. For comparison, pyright always uses unions, and it generates no errors for the code posted at the top of this issue.

Maintainers may want to add the "topic-join-v-union" label to this issue.

[MestreLion]

IDK the type of sys.stdout though.

sys.stdout is type IO[str], sys.stdout.buffer is IO[bytes], so I was hoping fh = sys.stdout if x else sys.stdout.buffer would be the same (or compatible) type as fh = open(path, mode), which is IO[Any]. But, alas, they're not.

So I'm not sure how to annotate fh in order to make mypy happy (and possibly keep the ternary).

The supertype of IO[str] and IO[bytes] is either a union of the two types (IO[str] | IO[bytes]) or the join of the two types (object).

Ohh, thanks! So in this particular case the proper way to annotate fh (and the function's return value) would be IO[Any] | IO[str] | IO[bytes] ? I always assumed IO[Any] was created to cover that.

[hauntsaninja]

Fixed in #17427