Temporary moratorium on literal constants (until the end of 2022)

[JukkaL]

I've encountered cases where the changes from int to a literal type in stubs causes regressions for mypy users. Here's one (simplified) example:

import logging

class C:
    def __init__(self) -> None:
        self.level = logging.INFO

    def enable_debug(self) -> None:
        # Error: expression has type "Literal[10]", variable has type "Literal[20]"
        self.level = logging.DEBUG

Here's another example:

import logging

class Base:
    level = logging.INFO

class Derived(Base):
    level = logging.DEBUG  # Error -- incompatible with base class

Previously the types were int, so no errors were generated. The code in the above examples doesn't look buggy to me, and the behavior of mypy also seems as expected.

There could well be other problematic cases like this. Having more precise types is generally useful, but in the case of literal types, the regressions such as the above may make them not worth the potential false positives (which are hard to predict, since in most use cases literal types are fine). I wonder if there are concrete examples where the literal types bring concrete benefits that could offset the false positives.

There were several false positives like the above in a big internal codebase when using a recent typeshed. Working around the issues is simple by annotating the attributes as int, but this could be quite non-obvious for non-expert users who aren't familiar with literal types (a somewhat advanced typing feature).

Thoughts? I'm leaning towards preferring to only use literal types when there is a concrete use cases where they help, since the fallout from using them seems difficult to predict.

[srittau]

At least in the sort term we should revert in cases where literals cause problems and in the future only use literals where it's unlikely that the literal gets assigned to a mutable variable.

In the long term we should rethink how type checkers handle literals. For example,

FOO: Literal[1] = 1
BAR: Literal[2] = 2

x = FOO  # `x` should probably have the type `int`, not `Literal[1]`

[srittau]

Actually, at least in mypy the following already works:

FOO: Final = 1
BAR: Final = 2

x = FOO

reveal_type(FOO)  # type: Literal[1]
reveal_type(x)  # type: int

So maybe we should allow X: Final = 1 in stubs?

[srittau]

Cc @rchen152 @erictraut @pradeep90 for feedback on using X: Final = 3 in stubs.

[srittau]

Also Cc @sobolevn as "Literal master".

[srittau]

In #7259 pytype currently fails with "ParseError: Default value for CRITICAL: $external$typing_extensions.Final can only be '...', got LITERAL(50)", so we'd at least need to fix that and of course flake8-pyi.

[AlexWaygood]

So maybe we should allow X: Final = 1 in stubs?

This to me feels much more expressive than writing X: Literal[1], so I'd be in favour of allowing this in stubs whatever the case.

I worry, however, that there's not much point in annotating a variable as Literal or Final if it's automatically inferred as something less precise as soon as it's assigned to a different variable.

[JukkaL]

I worry, however, that there's not much point in annotating a variable as Literal or Final if it's automatically inferred as something less precise as soon as it's assigned to a different variable.

Using Final in the assignment will propagate the literal value:

Y: Final = X

Since a variable with a literal type is effectively a constant, using Final seems logical to me. In Python all variables are mutable by default, so making something effectively final (i.e. literal) but not quite seems a bit unexpected to me. Using literal types is fairly rare, in my experience, so requiring an explicit annotation when something has a literal type seems more usable than requiring an int annotation when something is not a literal (int seems about 100x as common as all literal types, in one big codebase at least). "Make the common case easy and the uncommon case possible" seems to apply here, and it was an important principle when literal types were designed.

PEP 586 has some relevant discussion:

• https://www.python.org/dev/peps/pep-0586/#backwards-compatibility
• https://www.python.org/dev/peps/pep-0586/#interactions-with-final

A common use case for explicit literal types is as function arguments, and using a Final variable works for this purpose:

X: Final = 5
f(X)  # X treated as Literal[5] as needed

Similarly, this is fine:

X: Final = 5
a: Literal[5, 6] = X

[sobolevn]

Thanks for letting me know.

I think this is a perfect example of "soft literal" type that I've mentioned several times in different discussions.

So, basically we have two cases:

1. When we have a "strict literal", like x: Literal[1] = 1. You can't reassign x to anything other than 1
2. When we have a "soft literal", when value can be changed. Like x: Literal[1]; self.value = x; self.value = 2. I think, that this should not be an error at all

I know that @Akuli was interested in this as well.

[rchen152]

Just double-checking that I understand correctly: x: Final = 3 would mean that the type of x is Literal[3], but if we write y = x, the type of y will be int. This is to prevent false positives in mypy caused by assigning a name to multiple Literal values. Is that right?

If so:

• No objections to allowing x: Final = 3 in stubs; I think it reads nicely. Changing pytype to not complain about this should be pretty straightforward, too.
• However, I don't love the fact that it means something slightly different from x: Literal[3] = 3, and it's not obvious from the annotations which one is stricter than the other, so to speak. If anything, mypy's behavior with the Final annotation makes more sense to me, and I would want it to behave that way for Literal, too.

[erictraut]

I've encountered cases where the changes from int to a literal type in stubs causes regressions for mypy users.

I'm not seeing any errors when I run mypy on these code samples. Are you saying that mypy infers that the type of self.level is Literal[20] in this case? That's not what I'm seeing.

Type stub files should be as unambiguous as possible. The statement x: Final = 3 tells a type checker that x is read-only, but it doesn't specify its type. A type checker must infer its type in this case. Since type inference rules are not specified in any PEP, this is ambiguous. It's probably safe to say that most (all?) Python type checkers will infer a type of Literal[3] in this case, but why take a chance? Explicit is better than implicit.

So I would recommend against using the x: Final = 3 form in any typeshed stubs.

x: Literal[3] # x is a read/write variable that can be assigned only a value of 3

x: Final[Literal[3]] # x is a read-only variable that has a value of 3

x: Final = 3 # x is a read-only variable that most (all?) type checkers will infer to have the type Final[Literal[3]]

Edit: reading this section of PEP 586 makes it pretty clear that x: Final = 3 should be interpreted the same as x: Final[Literal[3]], so I guess it is safe to assume that all type checkers will infer the same type in this case.

[erictraut]

Pyright follows these rules consistently:

1. If a symbol has a declared (annotated) type, its type is dictated by the annotation, and all assignments to that symbol will be validated against the declared type
2. If a symbol doesn't have a declared type, its type is inferred based on the union of all assignments, but literals are widened to their corresponding type; (note: for class and instance variables, only assignments within the class implementation are considered)
3. Within a local execution scope, the type of a symbol can be further narrowed based on assignments or type guards

Here's how these rules apply to the examples above:

import logging
class C:
    def __init__(self) -> None:
        reveal_type(self.level) # int
        self.level = logging.INFO
        reveal_type(self.level) # Literal[20] (narrowed via assignment)

    def enable_debug(self) -> None:
        reveal_type(self.level) # int
        self.level = logging.DEBUG
        reveal_type(self.level) # Literal[10] (narrowed via assignment)

c = C()
reveal_type(c.level) # int
c.level = 1
reveal_type(c.level) # Literal[1] (narrowed via assignment)
c.level = "hi" # Error because inferred type of `C.level` is `int`

FOO: Literal[1] = 1
x = FOO
reveal_type(x) # Literal[1] (narrowed via assignment)
x = 3 # This is not an error because the type of x was not declared
reveal_type(x) # Literal[3] (narrowed via assignment)
x = "hi" # This is not an error because the type of x was not declared
reveal_type(x) # Literal["hi"] (narrowed via assignment)

@sobolevn, I don't understand what you mean by a "soft literal". If you declare a variable with the type Literal[1], that means it can accept only values of Literal[1]. If it is meant to accept any int value, then it should be annotation as int.