Type checking of None values

[mvcisback]

A common pattern in alot of functions is to use None to signify something alternative behavior.
It might be beneficial to include a convenience Maybe type that is a partial application of Union with None.

I'm still familiarizing myself with the code and am not 100% what to do yet

[JukkaL]

I'd like to have Optional[x] which means Union[x, None]. As long as None is compatible with every type (as it is now), Optional[x] would be equivalent with just x and would basically be just for documentation, which is not optimal.

More strict type checking of None values (i.e., turning None to an ordinary type) probably requires at least better type inference of conditional None checks. For example, consider this code:

def f(x: Optional[str]) -> bool:
    return x and x.startswith('a')

The code is valid, but inferring that the second reference to x is never None is not entirely trivial, and there are more complex cases that will probably have to be handled as well. The current implementation of union types supports some common idioms but not all of them.

[mvcisback]

In addition to isinstance guards, could we utilize something like single dispatch (https://docs.python.org/3/library/functools.html#functools.singledispatch) to handle these kind of assertions?

Ultimately this comes down to a pattern matching issue. I think using a dispatch decorator, we could shoehorn pattern matching in.

[JukkaL]

Mypy already has something like single dispatch (@overload -- it's actually multiple dispatch), but the problem is that most existing Python code doesn't use anything like it. Also, often you have None values in lists, for example, and I don't see how to easily dispatch on items in lists using a decorator-based approach without some pretty ugly code. I want to support common Python idioms without complex refactoring, and splitting a function into multiple variants feels pretty complex.

Of course, for new code, single/multiple dispatch function decorators may often make a lot of sense.

General pattern matching is tricky without new syntax. Repeated isinstance tests + type inference that knows about them is about as good as I've been able to come up with.

[ambv]

Nullable types is one of the most important features we'd like in a type checker for Python at Facebook. For Hack (our typed PHP variant), assuming that arguments don't accept "null" by default catches many simple bugs before they get committed.

So, all in all this is a crucial feature. That's not enough, though. Nullable types will be common in the wild. There are thousands of cases of optional ints in just the Facebook codebase. What that means is, optionality must be easy to express. Arguably Optional[int] is quite verbose. Hack uses ?int, pytypedecl uses int or None, which is still shorter and reads like English.

As a side note: single dispatch was intentionally kept simple. The worry was that making it too powerful would possibly skew how future code is structured in Python to a form that is harder to debug and read. This is why we didn't even add method support (e.g. "skip the first argument"), let alone multiple dispatch. By just introducing single dispatch we single-handedly got rid of most problems with conflict resolution. I had to anyway re-implement C3 linearization to support abstract base classes, which is in my opinion the biggest win from the PEP.

[JukkaL]

I agree that nullable types are important for static type checking. Also, even though supporting them is non-trivial, it seems quite feasible. Much of the needed machinery is already implemented for union types.

I have some ideas for making syntax verbosity less of a problem. First, for arguments with a default value of None, nullability could be inferred:

def f(x: int = None) -> None: ...   # type of x is implicitly Optional[int]

Also, we could infer an optional type from assigning a None value:

x = None
if c:
    x = 1
# type of x is Optional[int]

Currently the type of x above would be inferred from the first assignment only, and thus an explicit annotation would be needed. The above idiom would only work for straightforward cases where the value of the variable cannot be read between two assignments (i.e., there are conceptually two or more initializers for the variable), and it would probably not work for two unrelated types, in order to catch programming errors early. The following code would thus be invalid:

x = 1
if c:
    x = 'x'  # Won't infer type Union[int, str] for x

Verbosity could still be an issue. int or None has the issue that the runtime value of the type would just be int, and we could not use it for runtime dispatch.

Most of the syntax decisions in mypy are still up for debate. I'm not completely against the syntax int or None, for example.

About your side note: Overloading in in mypy is actually pretty simple -- the variants are always checked in the order they are introduced in a source file, and the first match is chosen. There is no attempt to pick the most precise one. Actually, many forms of overlapping overload variants are disallowed by the type checker.

[mvcisback]

I'm confused how Union[int, str] would be technically any different than Optional[int].

Isn't Optional[int] just sugar for Union[NoneType] ?

I feel that without a proof that bool(c) != False, the type inference engine should always union the types.

As for syntax, int or None is nice!
In fact rewriting it as int | None makes it look like Sum Types from ML Style languages.

[ghost]

Is the type None the same as the type Union[] i.e. Union of no types?

[gvanrossum]

No, when used as a type, None is a shorthand for type(None). Union [] is
invalid , and so is Union [()].
On Apr 7, 2015 11:06 AM, "Dedoig" [email protected] wrote:

Is the type None the same as the type Union[] i.e. Union of no types?

—
Reply to this email directly or view it on GitHub.

[ghost]

Thanks.

It seems Union[…] is already implemented and working, but type None (as disjoint from every other type) is not yet implemented. I was wondering of treating type None as Union[] might make it easier to implement…

On Apr 7, 2015, at 2:36 PM, Guido van Rossum [email protected] wrote:

No, when used as a type, None is a shorthand for type(None). Union [] is
invalid , and so is Union [()].
On Apr 7, 2015 11:06 AM, "Dedoig" [email protected] wrote:

Is the type None the same as the type Union[] i.e. Union of no types?

—
Reply to this email directly or view it on GitHub.
—
Reply to this email directly or view it on GitHub.

[gvanrossum]

Type None does not mean "not any type". It means "must be the value None". This is primarily useful to be explicit about the return value of a function that has no result (because it still returns None):

def greeting() -> None:
    print('Hello, world')

Without the -> None the body of the function won't be type-checked and its return value will be assumed to be Any.

[NYKevin]

Technically, Union[()] could be a thing, but I'm not sure how useful it would be in practice. Maybe if you want to indicate that a function can raise exceptions? E.g. sys.exit() could hypothetically look something like this:

def exit(value: Any) -> Union[()]:
    raise SystemExit(value)

[JukkaL]

See the dupe issue #875 for more discussion.

[rwbarton]

The flag --strict-optional now exists. Is it useful to keep this ticket open?

[gvanrossum]

IMO the ticket should remain open until strict optional checking is the default.