From a26fa39123ceb85b49d9077c5df8400de5a4449a Mon Sep 17 00:00:00 2001 From: Eric Snow Date: Fri, 24 Jun 2016 15:45:25 -0600 Subject: [PATCH] PEP 520 updates after feedback. --- pep-0520.txt | 169 +++++++++++++++++++++++++++++++++++++-------------- 1 file changed, 123 insertions(+), 46 deletions(-) diff --git a/pep-0520.txt b/pep-0520.txt index a011d634b38..eb7be43b975 100644 --- a/pep-0520.txt +++ b/pep-0520.txt @@ -8,24 +8,28 @@ Type: Standards Track Content-Type: text/x-rst Created: 7-Jun-2016 Python-Version: 3.6 -Post-History: 7-Jun-2016, 11-Jun-2016, 20-Jun-2016 +Post-History: 7-Jun-2016, 11-Jun-2016, 20-Jun-2016, 24-Jun-2016 Abstract ======== -When a class is defined using a ``class`` statement, the class body is -executed within a namespace. After the execution completes, that -namespace is copied into new ``dict`` and the original definition -namespace is discarded. The new copy is stored away as the class's -namespace and is exposed as ``__dict__`` through a read-only proxy. - -This PEP preserves the order in which the attributes in the definition -namespace were added to it, before that namespace is discarded. This -means it reflects the definition order of the class body. That order -will now be preserved in the ``__definition_order__`` attribute of the -class. This allows introspection of the original definition order, -e.g. by class decorators. +The class definition syntax is ordered by its very nature. Class +attributes defined there are thus ordered. Aside from helping with +readability, that ordering is sometimes significant. If it were +automatically available outside the class definition then the +attribute order could be used without the need for extra boilerplate +(such as metaclasses or manually enumerating the attribute order). +Given that this information already exists, access to the definition +order of attributes is a reasonable expectation. However, currently +Python does not preserve the attribute order from the class +definition. + +This PEP changes that by preserving the order in which attributes +are introduced in the class definition body. That order will now be +preserved in the ``__definition_order__`` attribute of the class. +This allows introspection of the original definition order, e.g. by +class decorators. Additionally, this PEP changes the default class definition namespace to ``OrderedDict``. The long-lived class namespace (``__dict__``) will @@ -35,14 +39,51 @@ remain a ``dict``. Motivation ========== -Currently Python does not preserve the order in which attributes are -added to the class definition namespace. The namespace used during -execution of a class body defaults to ``dict``. If the metaclass -defines ``__prepare__()`` then the result of calling it is used. Thus, -before this PEP, to access your class definition namespace you must -use ``OrderedDict`` along with a metaclass. Then you must preserve the -definition order (from the ``OrderedDict``) yourself. This has a -couple of problems. +The attribute order from a class definition may be useful to tools +that rely on name order. However, without the automatic availability +of the definition order, those tools must impose extra requirements on +users. For example, use of such a tool may require that your class use +a particular metaclass. Such requirements are often enough to +discourage use of the tool. + +Some tools that could make use of this PEP include: + +* documentation generators +* testing frameworks +* CLI frameworks +* web frameworks +* config generators +* data serializers +* enum factories (my original motivation) + + +Background +========== + +When a class is defined using a ``class`` statement, the class body +is executed within a namespace. Currently that namespace defaults to +``dict``. If the metaclass defines ``__prepare__()`` then the result +of calling it is used for the class definition namespace. + +After the execution completes, the definition namespace namespace is +copied into new ``dict``. Then the original definition namespace is +discarded. The new copy is stored away as the class's namespace and +is exposed as ``__dict__`` through a read-only proxy. + +The class attribute definition order is represented by the insertion +order of names in the *definition* namespace. Thus, we can have +access to the definition order by switching the definition namespace +to an ordered mapping, such as ``collections.OrderedDict``. This is +feasible using a metaclass and ``__prepare__``, as described above. +In fact, exactly this is by far the most common use case for using +``__prepare__`` (see PEP 487). + +At that point, the only missing thing for later access to the +definition order is storing it on the class before the definition +namespace is thrown away. Again, this may be done using a metaclass. +However, this means that the definition order is preserved only for +classes that use such a metaclass. There are two practical problems +with that: First, it requires the use of a metaclass. Metaclasses introduce an extra level of complexity to code and in some cases (e.g. conflicts) @@ -55,8 +96,6 @@ we have such an opportunity by defaulting to ``OrderedDict``. Second, only classes that opt in to using the ``OrderedDict``-based metaclass will have access to the definition order. This is problematic for cases where universal access to the definition order is important. -One of the original motivating use cases for this PEP is generic class -decorators that make use of the definition order. Specification @@ -64,21 +103,21 @@ Specification Part 1: -* the order in which class attributes are defined is preserved in the - new ``__definition_order__`` attribute on each class -* "dunder" attributes (e.g. ``__init__``, ``__module__``) are ignored -* ``__definition_order__`` is a ``tuple`` (or ``None``) +* all classes have a ``__definition_order__`` attribute +* ``__definition_order__`` is a ``tuple`` of identifiers (or ``None``) * ``__definition_order__`` is a read-only attribute * ``__definition_order__`` is always set: - 1. if ``__definition_order__`` is defined in the class body then it + 1. during execution of the class body, the insertion order of names + into the class *definition* namespace is stored in a tuple + 2. if ``__definition_order__`` is defined in the class body then it must be a ``tuple`` of identifiers or ``None``; any other value will result in ``TypeError`` - 2. classes that do not have a class definition (e.g. builtins) have + 3. classes that do not have a class definition (e.g. builtins) have their ``__definition_order__`` set to ``None`` - 3. classes for which `__prepare__()`` returned something other than + 4. classes for which `__prepare__()`` returned something other than ``OrderedDict`` (or a subclass) have their ``__definition_order__`` - set to ``None`` (except where #1 applies) + set to ``None`` (except where #2 applies) Part 2: @@ -95,12 +134,7 @@ default behavior:: class Spam(metaclass=Meta): ham = None eggs = 5 - __definition_order__ = tuple(k for k in locals() - if not (k.startswith('__') and - k.endswith('__'))) - -Note that [pep487_] proposes a similar solution, albeit as part of a -broader proposal. + __definition_order__ = tuple(locals()) Why a tuple? ------------ @@ -125,22 +159,36 @@ If a use case for a writable (or mutable) ``__definition_order__`` arises, the restriction may be loosened later. Presently this seems unlikely and furthermore it is usually best to go immutable-by-default. -Note that ``__definition_order__`` is centered on the class definition +Note that the ability to set ``__definition_order__`` manually allows +a dynamically created class (e.g. Cython, ``type()``) to still have +``__definition_order__`` properly set. + +Why not "__attribute_order__"? +------------------------------ + +``__definition_order__`` is centered on the class definition body. The use cases for dealing with the class namespace (``__dict__``) post-definition are a separate matter. ``__definition_order__`` would be a significantly misleading name for a feature focused on more than class definition. -See [nick_concern_] for more discussion. - -Why ignore "dunder" names? --------------------------- +Why not ignore "dunder" names? +------------------------------ Names starting and ending with "__" are reserved for use by the interpreter. In practice they should not be relevant to the users of ``__definition_order__``. Instead, for nearly everyone they would only be clutter, causing the same extra work for everyone. +However, dropping dunder names by default may inadvertantly cause +problems for classes that use dunder names unconventionally. In this +case it's better to play it safe and preserve *all* the names from +the class definition. + +Note that a couple of dunder names (``__name__`` and ``__qualname__``) +are injected by default by the compiler. So they will be included even +though they are not strictly part of the class definition body. + Why None instead of an empty tuple? ----------------------------------- @@ -192,6 +240,12 @@ have a roughly equivalent concept of a definition order. So conceivably PEP does not introduce any such support. However, it does not prohibit it either. +The specific cases: + +* builtin types +* PyType_Ready +* PyType_FromSpec + Compatibility ============= @@ -221,6 +275,22 @@ be minimal. If a Python implementation cannot support switching to to ``None``. +Open Questions +============== + +* What about `__slots__`? + +* Drop the "read-only attribute" requirement? + +Per Guido: + + I don't see why it needs to be a read-only attribute. There are + very few of those -- in general we let users play around with + things unless we have a hard reason to restrict assignment (e.g. + the interpreter's internal state could be compromised). I don't + see such a hard reason here. + + Implementation ============== @@ -230,8 +300,8 @@ The implementation is found in the tracker. [impl_] Alternatives ============ -cls.__dict__ as OrderedDict -------------------------------- +An Order-preserving cls.__dict__ +-------------------------------- Instead of storing the definition order in ``__definition_order__``, the now-ordered definition namespace could be copied into a new @@ -240,8 +310,15 @@ the now-ordered definition namespace could be copied into a new However, using ``OrderedDict`` for ``__dict__`` would obscure the relationship with the definition namespace, making it less useful. -Additionally, doing this would require significant changes to the -semantics of the concrete ``dict`` C-API. + +Additionally, (in the case of ``OrderedDict`` specifically) doing +this would require significant changes to the semantics of the +concrete ``dict`` C-API. + +There has been some discussion about moving to a compact dict +implementation which would (mostly) preserve insertion order. However +the lack of an explicit ``__definition_order__`` would still remain +as a pain point. A "namespace" Keyword Arg for Class Definition ----------------------------------------------