non-final frozen attributes (was: Improved read-only attributes of Protocol)

[ariebovenberg]

---

☝️ NOTE

I have since refined this issue, see #922 (comment)

Below I have kept the original text

---

Are there any ideas about making read-only attributes in Protocol easier to define? Immutable objects (e.g. datetime) and frozen dataclasses are great, but they are painful to integrate with Protocol. Which is a shame, because immutability and structural typing are both great!

The problem

class SomeProto(Protocol):
    a: int
    b: str

@dataclass(frozen=True)
class SomeDClass:
    a: int = 1
    b: str = ""
    c: bool = False

# not allowed as attributes must also be settable to satisfy protocol
x: SomeProto = SomeDClass()

# to allow non-settable attributes, we have to define this.
# It's 3 times as long, and less readable
class SomeProto2(Protocol):

    @property
    def a(self) -> int:
        ...

    @property
    def b(self) -> str:
        ...

# ...but it does work
y: SomeProto2 = SomeDClass()

Why do something about it

In my experience, the use case for purely read-only attributes is far more common than settable attributes. It feels strange that this requires such a more cumbersome and less readable workaround. This can be easily fixed without breaking existing behavior.

Proposed solution: allow using Final or another annotation

class AnotherProto(Protocol):
    d: Final[int]  # read-only attribute
    e: str  # read and writeable attribute

Currently this gives an explicit error (error: Protocol member cannot be final). Understandable, because the semantics don't quite match up.

However, PEP591 does seem to leave the door open a bit:

The current typing module lacks a way to indicate that a variable will not be assigned to. This is a useful feature in several situations:
[...]
Creating a read-only attribute that may not be overridden by subclasses. (@property can make an attribute read-only but does not prevent overriding)

An alternative would be a separate ReadOnly[...] annotation or similar
What are your thoughts?

(possibly related: #920)

[ariebovenberg]

---

☝️ NOTE

I have since refined this issue, see #922 (comment)

Below I have kept the original text

---

One usecase that complicates things is adding mutability through inheritance:

class Address(Protocol):
    street: Final[str]  # only readable
    zipcode: Final[int]


class MutableAddress(Address):
    street: str  # readable and writable
    zipcode: int

What are your thoughts on this?
Going from 'frozen' to non-'frozen' through inheritance
is something Python often avoids (frozenset/set, dataclass)

If we want to support this, Final would be the wrong choice.
Other options I can think of:

• ReadOnly
• Read
• Frozen
• Gettable
• Get
• Immutable

[ariebovenberg]

It seems the problem is more general: could we have a dedicated way to prevent reassignment of attributes without also preventing overriding (which Final does)?

The problem

The goals of Final as stated in PEP 591 are to support:

1. Declaring that a method should not be overridden
2. Declaring that a class should not be subclassed
3. Declaring that a variable or attribute should not be reassigned

The problem is that (1) limits (3). Consider the following:

class Measurement:
    timestamp: Final[date]  # final used here to prevent reassigning this attribute
    value: int

    def __init__(self, value: int):
        self.timestamp = date.today()
        self.value = value


m = Measurement(0)
m.timestamp = date(2020, 1, 1)  # this is prevented, as it should be

class AccurateMeasurement(Measurement):
    # Declaring this attribute is currently not allowed because Final forbids overriding `timestamp`.
    # If we're only interested in preventing attribute reassignment, it should be possible though.
    timestamp: Final[datetime]  # (note: datetime is a subclass of date)

    def __init__(self, value: int):
        self.timestamp = datetime.now()
        self.value = value

Proposal

An annotation specifically for preventing reassignment of attributes, but that does not prevent overriding.
It could be called something like Frozen.

An additional advantage is that readonly members of protocols become way easier to define:

class MeasurementLike(Protocol):
    timestamp: Frozen[date]

instead of

class MeasurementLike(Protocol):
    @property
    def timestamp(self) -> date:
        ...

[sobolevn]

Well, technically you override timestamp in Measurement with AccurateMeasurement:

class AccurateMeasurement(Measurement):
    # Declaring this attribute is currently not allowed because Final forbids overriding `timestamp`
    # If we're only interested in preventing attribute reassignment, it should be possible though.
    timestamp: Final[datetime]  # note: datetime is a subclass of date

    def __init__(self, value: int):
        self.timestamp = datetime.now()
        self.value = value

It is just made via subclassing that reassignes timestamp in a different way.

This will be especially visiable when super().__init__() is called.

[ariebovenberg]

@sobolevn you could indeed interpret it that way 🤔. Maybe it's better to talk of 'frozen' attributes, like with dataclasses:

(the following code checks out)

@dataclass(frozen=True)
class Measurement:
    value: int
    timestamp: date = field(default_factory=date.today, init=False)


@dataclass(frozen=True)
class AccurateMeasurement(Measurement):
    timestamp: datetime = field(default_factory=datetime.now, init=False)

m = Measurement(1)
a: Measurement = AccurateMeasurement(2)

it'd be useful to express 'frozenness' of attributes outside of dataclasses:

class Measurement:
    value: Frozen[int]
    timestamp: Frozen[date]

class AccurateMeasurement(Measurement):
    timestamp: Frozen[datetime]

[ariebovenberg]

It is just made via subclassing that reassignes timestamp in a different way.

Note that PEP591 does seem to imply 'reassignment' of attributes means: assignment outside __init__. (Would subclassing with a different __init__ count as 'reassignment' 🤔 ?)

In any case, the rule for Frozen attributes would be: can only be set in __init__.

[ariebovenberg]

additionally, a @frozen decorator may be useful to indicate a class is fully frozen:

class Measurement:
    value: Frozen[int]
    timestamp: Frozen[date]

# equivalent to
@frozen
class Measurement:
    value: int
    timestamp: date

A lot more readable as the number of attributes increases!

[ariebovenberg]

Found a related discussion about read-only protocol members: #903 (conclusion was use the @property syntax)

[ariebovenberg]

@ilevkivskyi I was wondering if you had any thoughts on the matter -- since you co-authored PEP591 and wrote the reference implementation for Final in mypy?

[ilevkivskyi]

IMO the convenience of saving one line of code (by not using a property) is not worth introducing some special rules/concepts. IIRC final things were developed explicitly to not overlap with existing solutions (like properties).

[srittau]

What use case isn't covered by @property?

[ariebovenberg]

Thanks @srittau @ilevkivskyi for the remarks! I can definitely understand not wanting to add
an extra overlapping concept for the use case of specifying read-only attributes.

To answer your question, I'd say there are 2 limitations of property (for specifying a read-only attribute)

Limitation 1: properties in Protocols and ABCs are so verbose that they obscure intention

class MeasurementLike(Protocol):
    @property
    def timestamp(self) -> date:
        ...

class MeasurementBase(ABC):
    @property
    @abstractmethod
    def timestamp(self) -> date:
        ...

# compare with the much less noisy
class MeasurementLike(Protocol):
    timestamp: Final[date]

Does this occur often enough to warrant a shorthand?
A quick search of the typeshed repo shows circa 1 in 6 (!) defs are a property.
A good shorthand would thus cut out a lot of noise! Readability counts 👀

Potential solution to limitation 1: a type qualifier for property

A simple solution would be a Property type qualifier:

class MeasurementLike(Protocol):
    timestamp: Property[date]

class MeasurementBase(ABC):
    timestamp: Property[date]  # no implementation means it's abstract

We could also simply allow property itself to function as a type qualifier,
to cut down on imports.

Limitation 2: In concrete classes, property requires a lot of extra moving parts.

The example below shows the noise needed for one read-only attribute:

class Measurement:
    _timestamp: date

    def __init__(self):
        self._timestamp = ...

    @property
    def timestamp(self) -> date:
        return self._timestamp

It takes a while to understand "aha! All this is to make timestamp read-only"

Potential solution to limitation 2: a helper for "assign once" properties

In addition to the type qualifier, a new descriptor could implement the common 'assign once' pattern:

class Measurement:
    timestamp: Property[date] = property.assigned_once

    def __init__(self):
        self.timestamp = ...  # assignment allowed here

m = Measurement()
m.timestamp = ...  # a subsequent assignment is blocked at runtime

Do we need an extra helper? In 3.8 cached_property was added to functools. The 'assign once' behavior is IMO at least as common as caching, if not more.

Of course in this case you could use Final, but you wouldn't be able to set the attribute in a subclass __init__.
Also, an advantage of descriptors could be allowing other behaviors such as allowing
reassignment only privately.

tl;dr

• property is the preferred way to model read-only attributes,
  but it can be clunky to use in the simplest and most common cases, hindering readability.
• A Property type qualifier and a new descriptor can give us a readable shorthand
  without having to resort to new concepts.

[ariebovenberg]

FYI I created a small mypy plugin for fixing the initial problem of read-only protocols: link

Nevertheless I'd be interested to hear any thoughts on the Property type qualifier described above.
If it's not deemed a good idea, we can close this issue ✅

[ariebovenberg]

I see that this particular Property idea been proposed before: #594

I will proceed to close this issue 🔒