PyEval_GetLocals() to return a borrowed reference. Minor fixes to examples. (#2057)

Author: markshannon

pep-0667.rst

@@ -92,8 +92,9 @@ for function scopes it will be a custom class.
 ``locals()`` will be defined as::
 
     def locals():
-        f_locals = sys._getframe(1).f_locals
-        if not isinstance(f_locals, dict):
+        frame = sys._getframe(1)
+        f_locals = frame.f_locals
+        if frame.is_function():
             f_locals = dict(f_locals)
         return f_locals
 
@@ -109,6 +110,7 @@ For example::
         return sys._getframe(1).f_locals
 
     def test():
+        if 0: y = 1 # Make 'y' a local variable
         x = 1
         l()['x'] = 2
         l()['y'] = 4
@@ -141,20 +143,17 @@ Both functions will return a new reference.
 Changes to existing APIs
 ''''''''''''''''''''''''
 
-The existing  C-API function ``PyEval_GetLocals()`` will always raise an
-exception with a message like::
+The C-API function ``PyEval_GetLocals()`` will be deprecated.
+``PyEval_Locals()`` should be used instead.
 
-    PyEval_GetLocals() is unsafe. Please use PyEval_Locals() instead.
-
-This is necessary as ``PyEval_GetLocals()`` 
-returns a borrowed reference which cannot be made safe.
-
-The following functions will be retained, but will become no-ops::
+The following three functions will become no-ops, and will be deprecated::
 
     PyFrame_FastToLocalsWithError()
     PyFrame_FastToLocals()
     PyFrame_LocalsToFast()
 
+The above four deprecated functions will be removed in 3.13.
+
 Behavior of f_locals for optimized functions
 --------------------------------------------
 
@@ -185,8 +184,12 @@ C-API
 PyEval_GetLocals
 ''''''''''''''''
 
-Code that uses  ``PyEval_GetLocals()`` will continue to operate safely, but
-will need to be changed to use ``PyEval_Locals()`` to restore functionality.
+Because ``PyEval_GetLocals()`` returns a borrowed reference, it requires
+the dictionary to be cached on the frame, extending its lifetime and
+forces memory to be allocated for the frame object on the heap as well.
+
+Using ``PyEval_Locals()`` will be much more efficient
+than ``PyEval_GetLocals()``.
 
 This code::
 
@@ -209,8 +212,9 @@ PyFrame_FastToLocals, etc.
 These functions were designed to convert the internal "fast" representation 
 of the locals variables of a function to a dictionary, and vice versa.
 
-Calls to them are no longer required. C code that directly accesses the ``f_locals``
-field of a frame should be modified to call ``PyFrame_GetLocals()`` instead::
+Calls to them are no longer required. C code that directly accesses the
+``f_locals`` field of a frame should be modified to call
+``PyFrame_GetLocals()`` instead::
 
     PyFrame_FastToLocals(frame);
     PyObject *locals = frame.f_locals;
@@ -246,14 +250,17 @@ They serve only to illustrate the proposed design.
 
         def __init__(self, ...):
             self._name_to_offset_mapping_impl = NULL
+            self._variable_names = deduplicate(
+                self.co_varnames + self.co_cellvars + self.co_freevars
+            )
             ...
 
         @property
         def _name_to_offset_mapping(self):
             "Mapping of names to offsets in local variable array."
             if self._name_to_offset_mapping_impl is NULL:
                 self._name_to_offset_mapping_impl = {
-                    name: index for (index, name) in enumerate(self.co_varnames)
+                    name: index for (index, name) in enumerate(self._variable_names)
                 }
             return self._name_to_offset_mapping_impl
 
@@ -315,7 +322,7 @@ They serve only to illustrate the proposed design.
             f = self._frame
             co = f.f_code
             yield from iter(f._extra_locals)
-            for index, name in enumerate(co._varnames):
+            for index, name in enumerate(co._variable_names):
                 val = f._locals[index]
                 if val is NULL:
                     continue
@@ -330,7 +337,7 @@ They serve only to illustrate the proposed design.
             co = f.f_code
             if f._extra_locals:
                 return f._extra_locals.pop()
-            for index, _ in enumerate(co._varnames):
+            for index, _ in enumerate(co._variable_names):
                 val = f._locals[index]
                 if val is NULL:
                     continue
@@ -348,7 +355,7 @@ They serve only to illustrate the proposed design.
             f = self._frame
             co = f.f_code
             res = 0
-            for index, _ in enumerate(co._varnames):
+            for index, _ in enumerate(co._variable_names):
                 val = f._locals[index]
                 if val is NULL:
                     continue
@@ -358,6 +365,20 @@ They serve only to illustrate the proposed design.
                 res += 1
             return len(self._extra_locals) + res
 
+C API
+-----
+
+``PyEval_GetLocals()`` will be implemented roughly as follows::
+
+    PyObject *PyEval_GetLocals(void) {
+        PyFrameObject * = ...; // Get the current frame.
+        Py_CLEAR(frame->_locals_cache);
+        frame->_locals_cache = PyEval_Locals();
+        return frame->_locals_cache;
+    }
+
+As with all functions that return a borrowed reference, care must be taken to
+ensure that the reference is not used beyond the lifetime of the object.
 
 Comparison with PEP 558
 =======================
@@ -372,8 +393,8 @@ complex, and has many corner cases which will lead to bugs.
 The key difference between this PEP and PEP 558 is that
 PEP 558 requires an internal copy of the local variables,
 whereas this PEP does not.
-Maintaining a copy would add considerably to the complexity of both
-the specification and implementation, and bring no real benefits.
+Maintaining a copy adds considerably to the complexity of both
+the specification and implementation, and brings no real benefits.
 
 The semantics of ``frame.f_locals``
 -----------------------------------
@@ -412,7 +433,7 @@ An alternative way to define ``locals()`` would be simply as::
 
 This would be simpler and easier to understand. However,
 there would be backwards compatibility issues when ``locals`` is assigned
-to a local variable or when passed to ``eval``.
+to a local variable or when passed to ``eval`` or ``exec``.
 
 References
 ==========