Declaring type of variable without initializing with a value

[JukkaL]

Mypy has the Undefined helper that serves two purposes.

First, it can be used as a dummy value when declaring the types of variables:

from typing import Undefined, List
x = Undefined # type: List[int]

This declares x as a list of int, but it doesn't give x a valid value. Any operation on Undefined (other than passing it around and operations that can't be overloaded) will raise an exception.

Second, Undefined can be used to declare the type of a variable without using a comment:

from typing import Undefined, List
x = Undefined(List[int])

The second example is semantically equivalent to the first example.

In my experience, being able to declare the type of a variable without initializing it with a proper value is often useful. Undefined is better than None for this purpose since None is a valid value. If we see Undefined values in tracebacks then we know that somebody forgot to initialize a variable, but if we see a None value things are murkier. Also, None are valid for certain operations such as if statements and equality checks (and dict lookups), and thus an unexpected None value can easily go undetected. This is less likely with an Undefined value.

Also, we'd like to be able to type check uses of None values (for example, by requiring an Optional[...] type), but Undefined is compatible with all types.

Some tools/implementations (jython, for example, according to Jim Baker) can't easily access types declared as comments, and the Undefined(...) syntax could be used as a workaround.

If we decide to support both of the above variants (+ just using a # type: comment without Undefined), we have the problem that there is no longer a single obvious way to declare the type of a variable. I don't think that the inconsistency is a major issue, since all of the variants serve useful purposes.

[JukkaL]

This is related to #9.

[ambv]

In Python 3.6 this could become:

var x : List[int]

or

x : List[int]

In case of Undefined(), would the following still be true in vanilla Python 3.5?

l = Undefined(List[int])
l    # raises NameError: name 'l' is not defined

[gvanrossum]

Łukasz: I don't think we should strive to raise a NameError in that case (the only ways I can think of are truly horrible). I don't even think that printing Undefined(...) should raise an exception -- it should just print 'Undefined(...)' (with the actual argument type instead of the dots). But it should define no other attributes or operations, and it should be unhashable (so you can't put it in a set or use it as a dict key without an error). It should also be runtime introspectable (e.g. x.__type__ would be List[int]). The runtime implemenation should be pretty straightforward.

The type checker, of course, should flag all uses of Undefined as an error (even printing it or passing it as an argument to a function or assigning it to another variable).

Jukka: I prefer Undefined(<type>) over Undefined # type: <type>.

[JukkaL]

There are two reasons why mypy supports Undefined # type:. First, it's much more efficient than Undefined(...) in CPython (and even more so if Undefined(...) would keep track of the argument type; currently it isn't stored). Second, it makes it possible to use # type: consistently for declaring variable types (outside function signatures). # type: is the only way to declare the type of a variable when there is an initializer, so always using Undefined(...) would be awkward, though possible by introducing additional assignment statements.

However, # type: comments are not very pretty, and it's arguable whether being able to be consistently ugly is very useful :-)

[ambv]

@gvanrossum, my worry is exactly that people will have to start expecting "undefined" as a possible edge-case value returned from third-party library functions, next to None and raised exceptions.

That's isset() from PHP and undefined from JavaScript all over again.

By the time Python 3.6 comes with the var syntax (hopefully raising NameErrors when no assignment was made?), it might be too late.

I'm all ears for final decision and we can move on.

[gvanrossum]

Why would you ever have to expect Undefined? The PEP should make it very clear that such a value should never be used or returned or stored, and even though the runtime cannot check this, the static checker can (mypy currently apparently doesn't but I filed python/mypy#551 for that). And why would people use Undefined except to make the type checker happy?

[gvanrossum]

OK, final decision: The forms below are both allowed. The type checker should verify that no use is made of an undefined (or conditionally undefined) value, and at runtime most uses of undefined values except printing it will raise an exception.

x = Undefined  # type: List[int]
x = Undefined(List[int])

These forms should mean the same thing to the type checker (but the former is faster, since it doesn't have to evaluate List[int]).

[ambv]

@gvanrossum, my worry comes from the fact that Undefined will bubble up in case of bugs. This might be an unreasonable worry but I think that the following scenario is possible:

1. a library author might do a "quick change" without re-running the type checker, ship this broken thing to PyPI
2. suddenly people not interested in type hinting that are also users of said library will start seeing "Undefined" returned from a routine

Even if you say this is not something we should design against, Undefined will show up during breakpointing in debuggers, etc.

Anyway, fixed in e2e6fc4.

[gvanrossum]

Yeah, I understand it will sometimes show up in the debugger. (That's why I proposed the exception for printing it and probably for eq.) But that doesn't mean it's something that non-debugger code should ever test for -- no API should be documented as returning Undefined in certain circumstances, and that's different from the status of undefined in JavaScript (I don't know about PHP).

The only remaining fear would be about people coming from JavaScript or PHP designing unPythonic APIs. I think our community is strong enough to stop that.

[JukkaL]

Another use case where programmers may be tempted to use Undefined as a real value is where an attribute hasn't been explicitly initialized. Normal Python code could use hasattr, but for attributes with an Undefined initial value, they may try something like this:

class A:
    x = Undefined # type: int

    def foo(self):
        if self.x is not Undefined:
            bar(self.x)
        ...

Note that using None for the initial value can be inconvenient, since a type checker would require all uses of the attribute to be guarded (as the type would be a union type). When using Undefined in an attribute initializer, a type checker will likely allow the attribute to used even in contexts where it might be uninitialized, since tracking whether there is always a proper value is impractical to do in general (due to overriding, etc.).

If Undefined(type) keeps track of the type, the above idiom will break since there can be any number of Undefined instances. A more complex check may work, but programmers would probably be less likely to bother with it in that case.

[gvanrossum]

I think mypy should actually protest when it sees the "is not Undefined"
part -- in fact it should complain about any use of Undefined in an
expression except when assigning exactly "Undefined" to a variable (perhaps
even requiring a "# type: ..." comment). Even "x = (Undefined,)" should be
disallowed. And any use of a variable that it can see is Undefined (or
potentially Undefined) should also be rejected.

We really should nip this in the bud.

On Wed, Jan 14, 2015 at 10:10 PM, Jukka Lehtosalo notifications@github.com
wrote:

Another use case where programmers may be tempted to use Undefined as a
real value is where an attribute hasn't been explicitly initialized. Normal
Python code could use hasattr, but for attributes with an Undefined
initial value, they may try something like this:

class A:
x = Undefined # type: int

def foo(self):
    if self.x is not Undefined:
        bar(self.x)
    ...

Note that using None for the initial value can be inconvenient, since a
type checker would require all uses of the attribute to be guarded (as the
type would be a union type). When using Undefined in an attribute
initializer, a type checker will likely allow the attribute to used even in
contexts where it might be uninitialized, since tracking whether there is
always a proper value is impractical to do in general (due to overriding,
etc.).

If Undefined(type) keeps track of the type, the above idiom will break
since there can be any number of Undefined instances. A more complex
check may work, but programmers would probably be less likely to bother
with it in that case.

—
Reply to this email directly or view it on GitHub
#20 (comment).

--Guido van Rossum (python.org/~guido)

[JukkaL]

Added a mypy issue for restricting where Undefined is allowed: python/mypy#555

[gvanrossum]

@ambv still doesn't like Undefined, and presents the following reasoning:

4.a. Undefined changes the runtime in ugly ways (hiding UnboundLocalErrors).
4.b. There is a danger of people starting to use Undefined as guard values or “unset optional argument” in functions.
4.c. Proposed syntax:

# var a:Tuple[float, float]

4.d. No support for multiple variables on the same line or reusing the same type. This might be introduced in a future syntax revision.
4.e. Python 3.6 will likely go with this syntax for variable typing.
4.f. If Python 3.6 decides to go with angle brackets, double colons and braces instead, the syntax of this comment doesn’t block anything: it’s just a comment.
4.g Undefined will maybe stay in stubs for optional keyword arguments when @overload is used. (Jukka: did I get this right?)

My (Guido's) response is that I'm not yet convinced by this, and the easiest way is certainly to keep Undefined (since mypy already implements it). Hopefully we can decide this in person at PyCon.