Skip to content

bpo-39481: Implementation for PEP 585 #18239

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 78 commits into from
Apr 7, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
78 commits
Select commit Hold shift + click to select a range
d557a64
PEP 585 step 2: Write a Py_GenericAlias() function that returns origin
gvanrossum Jan 27, 2020
da33267
PEP 585 step 3: Add Py_GenericAlias() as __class_getitem__ to tuple, …
gvanrossum Jan 27, 2020
447fbe9
PEP 585 step 4: Write some tests
gvanrossum Jan 27, 2020
83e2ba2
Define a GenericAlias (proxy) object (with lots of TODOs still)
gvanrossum Jan 27, 2020
fb3e8d3
More tests
gvanrossum Jan 28, 2020
235180b
PEP 585 steps 5-6: Finish basic GenericAlias object and test
gvanrossum Jan 28, 2020
70f3abc
Fix bug with class methods
gvanrossum Jan 28, 2020
e4d1356
Define tp_new instead of tp_init
gvanrossum Jan 28, 2020
8f7a55e
Test that list[int][int] fails
gvanrossum Jan 28, 2020
f2ba2e5
Test that class L(list): pass; L[int] works
gvanrossum Jan 28, 2020
0af74d1
Implement PEP 585 for collections.deque and ... (#2)
emmatyping Jan 28, 2020
b264294
Respond to Serhiy's code review
gvanrossum Jan 28, 2020
0ddea56
📜🤖 Added by blurb_it.
blurb-it[bot] Jan 28, 2020
91edf1c
PEP 585 steps 7-8: Implement repr for GenericAlias (#1)
emmatyping Jan 29, 2020
6b1a218
Add static keyword to ga_repr_item (#3)
emmatyping Jan 29, 2020
ce21a96
Add GenericAlias to types.py
gvanrossum Jan 29, 2020
dbe67d0
Add test for types.py
gvanrossum Jan 29, 2020
15697f0
Fix tests that were sensitive to the presence of __class_getitem__
gvanrossum Jan 29, 2020
da15d43
Fix crash in repr(tuple[...]) -- call PyErr_Clear()
gvanrossum Jan 29, 2020
647cda8
Make re.Match, re.Pattern, and io.IOBase generic (#4)
emmatyping Jan 29, 2020
3824954
Restore parity between _io._IOBase and _pyio.IOBase
gvanrossum Jan 29, 2020
8aec6ae
Make contextlib.Abstract*ContextManager generic, with tests
gvanrossum Jan 29, 2020
58af183
WIP: Tweak test_typing.py to pass with the new definition of __orig_c…
gvanrossum Jan 29, 2020
8da0a08
Rename __parameters__ to __args__
gvanrossum Jan 29, 2020
eaecacb
Rename parameters to args
gvanrossum Jan 29, 2020
98d479f
Properly implement __parameters__ as the tuple of args that are type …
gvanrossum Jan 29, 2020
82bc806
Set __orig_class__ if possible when instantiating
gvanrossum Jan 29, 2020
2567ce3
WIP: Implement ga_getitem so list[T][int] -> list[int]
gvanrossum Jan 30, 2020
4f2ced6
Exclude duplicates from __parameters__
gvanrossum Jan 30, 2020
2c25fb0
Implement ga_getitem properly, with tests
gvanrossum Jan 31, 2020
9ed17ef
Implement == and != for GA
gvanrossum Jan 31, 2020
bb9a11f
Make collections.abc.* generic (#5)
emmatyping Jan 31, 2020
6d3f535
Make isinstance and issubclass error for GenericAlias (#6)
emmatyping Jan 31, 2020
4b3ecf2
Fix two more AssertIs calls in test_typing.py
gvanrossum Jan 31, 2020
2df602b
Admit defeat -- comment out two expected TypeErrors in test_extended_…
gvanrossum Jan 31, 2020
eede65a
Fix whitespace to satisfy patchcheck.py
gvanrossum Jan 31, 2020
2b22f6e
Fix test_site.py
gvanrossum Jan 31, 2020
065a032
Better way to fix test_site.py: don't import types from collections
gvanrossum Jan 31, 2020
baf9b32
Use Py_ssize_t instead of int for indexing (#8)
emmatyping Jan 31, 2020
9915ba2
GenericAlias pickle support (#9)
emmatyping Feb 5, 2020
4e1da19
Make type generic (#7)
emmatyping Feb 5, 2020
8926f7a
Remove a completed TODO comment
gvanrossum Feb 5, 2020
7b1ad20
Remove extraneous whitespace. (#10)
emmatyping Feb 5, 2020
a05d9ae
Do not make IOBase generic (#11)
emmatyping Feb 7, 2020
d4a372b
Remove __class_getitem__ from _pyio.py too; and don't test for IOBase
gvanrossum Feb 7, 2020
c534f67
Merge branch 'master' into pep585
gvanrossum Feb 10, 2020
2f94a36
Make types.MappingProxyType generic (#13)
emmatyping Feb 25, 2020
3da2797
Reword isinstance/subclass checks (#12)
emmatyping Feb 25, 2020
13d8d6d
Fix test_types (mappingproxy now has a __class_getitem__ method)
gvanrossum Feb 25, 2020
43a97b9
Merge remote-tracking branch 'upstream/master' into pep585
gvanrossum Feb 25, 2020
4783c2e
Convert some already generic things to GenericAlias to be consistent …
emmatyping Feb 25, 2020
e50136d
Make __parameters__ lazy (#15)
emmatyping Feb 25, 2020
0192986
Fix failing test_doctest -- we now find 821 tests
gvanrossum Feb 25, 2020
b80b077
Don't import types in os.py -- it breaks test_site.py
gvanrossum Feb 25, 2020
3bc4e12
Move GenericAlias definition to its own file
gvanrossum Mar 14, 2020
f658a64
Add a hash function to GenericAlias
gvanrossum Mar 14, 2020
bfd6342
Make Union[list[T], int] work
gvanrossum Mar 15, 2020
7919fa2
Add GenericAlias to Windows build (#16)
emmatyping Mar 16, 2020
854fa99
Merge remote-tracking branch 'upstream/master' into pep585
gvanrossum Mar 16, 2020
728084a
Run CI on PRs for pep585 branch
gvanrossum Mar 18, 2020
29130ab
Merge branch 'master' into pep585
gvanrossum Mar 29, 2020
ddbce27
Add docstring for GenericAlias (#17)
emmatyping Mar 30, 2020
4db065d
Remove unneeded _GenericAlias definition from collections/__init__.py
gvanrossum Mar 31, 2020
e14625f
Make is_typevar() harder to fool
gvanrossum Apr 5, 2020
76c0805
Rewrite ga_repr_item() to honor exceptions
gvanrossum Apr 5, 2020
128aacb
Use __class_getitem__ = classmethod(GenericAlias)
gvanrossum Apr 5, 2020
94f4095
Update Objects/genericaliasobject.c
gvanrossum Apr 5, 2020
79f4170
Apply suggestions from code review
gvanrossum Apr 5, 2020
f6f5b81
Use PyTuple_GET_SIZE
gvanrossum Apr 5, 2020
99cac83
Use PyErr_SetString correctly
gvanrossum Apr 5, 2020
6044934
Fix test failure (setattr may raise TypeError)
gvanrossum Apr 5, 2020
355d267
Propagate errors from is_typevar()
gvanrossum Apr 6, 2020
95b3c2a
Use PyObject_Str(qualname)
gvanrossum Apr 6, 2020
13d3742
Merge remote-tracking branch 'upstream/master' into pep585
gvanrossum Apr 6, 2020
d3839ea
Delete mention of pep585 branch in GitHub workflow
gvanrossum Apr 6, 2020
31fd842
Move Py_GenericAlias* to genericaliasobject.h
gvanrossum Apr 7, 2020
6bad315
Merge remote-tracking branch 'upstream/master' into pep585
gvanrossum Apr 7, 2020
cdb46c3
Add Py_GenericAlias[Type] to PC/python3.ref
gvanrossum Apr 7, 2020
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions Include/Python.h
Original file line number Diff line number Diff line change
Expand Up @@ -120,6 +120,7 @@
#include "iterobject.h"
#include "genobject.h"
#include "descrobject.h"
#include "genericaliasobject.h"
#include "warnings.h"
#include "weakrefobject.h"
#include "structseq.h"
Expand Down
14 changes: 14 additions & 0 deletions Include/genericaliasobject.h
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
// Implementation of PEP 585: support list[int] etc.
#ifndef Py_GENERICALIASOBJECT_H
#define Py_GENERICALIASOBJECT_H
#ifdef __cplusplus
extern "C" {
#endif

PyAPI_FUNC(PyObject *) Py_GenericAlias(PyObject *, PyObject *);
PyAPI_DATA(PyTypeObject) Py_GenericAliasType;

#ifdef __cplusplus
}
#endif
#endif /* !Py_GENERICALIASOBJECT_H */
27 changes: 27 additions & 0 deletions Lib/_collections_abc.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,8 @@
from abc import ABCMeta, abstractmethod
import sys

GenericAlias = type(list[int])

__all__ = ["Awaitable", "Coroutine",
"AsyncIterable", "AsyncIterator", "AsyncGenerator",
"Hashable", "Iterable", "Iterator", "Generator", "Reversible",
Expand Down Expand Up @@ -110,6 +112,8 @@ def __subclasshook__(cls, C):
return _check_methods(C, "__await__")
return NotImplemented

__class_getitem__ = classmethod(GenericAlias)


class Coroutine(Awaitable):

Expand Down Expand Up @@ -169,6 +173,8 @@ def __subclasshook__(cls, C):
return _check_methods(C, "__aiter__")
return NotImplemented

__class_getitem__ = classmethod(GenericAlias)


class AsyncIterator(AsyncIterable):

Expand Down Expand Up @@ -255,6 +261,8 @@ def __subclasshook__(cls, C):
return _check_methods(C, "__iter__")
return NotImplemented

__class_getitem__ = classmethod(GenericAlias)


class Iterator(Iterable):

Expand All @@ -274,6 +282,7 @@ def __subclasshook__(cls, C):
return _check_methods(C, '__iter__', '__next__')
return NotImplemented


Iterator.register(bytes_iterator)
Iterator.register(bytearray_iterator)
#Iterator.register(callable_iterator)
Expand Down Expand Up @@ -353,6 +362,7 @@ def __subclasshook__(cls, C):
'send', 'throw', 'close')
return NotImplemented


Generator.register(generator)


Expand Down Expand Up @@ -385,6 +395,9 @@ def __subclasshook__(cls, C):
return _check_methods(C, "__contains__")
return NotImplemented

__class_getitem__ = classmethod(GenericAlias)


class Collection(Sized, Iterable, Container):

__slots__ = ()
Expand All @@ -395,6 +408,7 @@ def __subclasshook__(cls, C):
return _check_methods(C, "__len__", "__iter__", "__contains__")
return NotImplemented


class Callable(metaclass=ABCMeta):

__slots__ = ()
Expand All @@ -409,6 +423,8 @@ def __subclasshook__(cls, C):
return _check_methods(C, "__call__")
return NotImplemented

__class_getitem__ = classmethod(GenericAlias)


### SETS ###

Expand Down Expand Up @@ -550,6 +566,7 @@ def _hash(self):
h = 590923713
return h


Set.register(frozenset)


Expand Down Expand Up @@ -632,6 +649,7 @@ def __isub__(self, it):
self.discard(value)
return self


MutableSet.register(set)


Expand Down Expand Up @@ -688,6 +706,7 @@ def __eq__(self, other):

__reversed__ = None


Mapping.register(mappingproxy)


Expand All @@ -704,6 +723,8 @@ def __len__(self):
def __repr__(self):
return '{0.__class__.__name__}({0._mapping!r})'.format(self)

__class_getitem__ = classmethod(GenericAlias)


class KeysView(MappingView, Set):

Expand All @@ -719,6 +740,7 @@ def __contains__(self, key):
def __iter__(self):
yield from self._mapping


KeysView.register(dict_keys)


Expand All @@ -743,6 +765,7 @@ def __iter__(self):
for key in self._mapping:
yield (key, self._mapping[key])


ItemsView.register(dict_items)


Expand All @@ -761,6 +784,7 @@ def __iter__(self):
for key in self._mapping:
yield self._mapping[key]


ValuesView.register(dict_values)


Expand Down Expand Up @@ -847,6 +871,7 @@ def setdefault(self, key, default=None):
self[key] = default
return default


MutableMapping.register(dict)


Expand Down Expand Up @@ -914,6 +939,7 @@ def count(self, value):
'S.count(value) -> integer -- return number of occurrences of value'
return sum(1 for v in self if v is value or v == value)


Sequence.register(tuple)
Sequence.register(str)
Sequence.register(range)
Expand Down Expand Up @@ -1000,5 +1026,6 @@ def __iadd__(self, values):
self.extend(values)
return self


MutableSequence.register(list)
MutableSequence.register(bytearray) # Multiply inheriting, see ByteString
6 changes: 5 additions & 1 deletion Lib/contextlib.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
import _collections_abc
from collections import deque
from functools import wraps
from types import MethodType
from types import MethodType, GenericAlias

__all__ = ["asynccontextmanager", "contextmanager", "closing", "nullcontext",
"AbstractContextManager", "AbstractAsyncContextManager",
Expand All @@ -16,6 +16,8 @@ class AbstractContextManager(abc.ABC):

"""An abstract base class for context managers."""

__class_getitem__ = classmethod(GenericAlias)

def __enter__(self):
"""Return `self` upon entering the runtime context."""
return self
Expand All @@ -36,6 +38,8 @@ class AbstractAsyncContextManager(abc.ABC):

"""An abstract base class for asynchronous context managers."""

__class_getitem__ = classmethod(GenericAlias)

async def __aenter__(self):
"""Return `self` upon entering the runtime context."""
return self
Expand Down
5 changes: 3 additions & 2 deletions Lib/os.py
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,8 @@

from _collections_abc import _check_methods

GenericAlias = type(list[int])

_names = sys.builtin_module_names

# Note: more names are added to __all__ later.
Expand Down Expand Up @@ -1074,8 +1076,7 @@ def __subclasshook__(cls, subclass):
return _check_methods(subclass, '__fspath__')
return NotImplemented

def __class_getitem__(cls, type):
return cls
__class_getitem__ = classmethod(GenericAlias)


if name == 'nt':
Expand Down
24 changes: 3 additions & 21 deletions Lib/subprocess.py
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,7 @@
import warnings
import contextlib
from time import monotonic as _time
import types

try:
import pwd
Expand Down Expand Up @@ -446,17 +447,7 @@ def __repr__(self):
args.append('stderr={!r}'.format(self.stderr))
return "{}({})".format(type(self).__name__, ', '.join(args))

def __class_getitem__(cls, type):
"""Provide minimal support for using this class as generic
(for example in type annotations).

See PEP 484 and PEP 560 for more details. For example,
`CompletedProcess[bytes]` is a valid expression at runtime
(type argument `bytes` indicates the type used for stdout).
Note, no type checking happens at runtime, but a static type
checker can be used.
"""
return cls
__class_getitem__ = classmethod(types.GenericAlias)


def check_returncode(self):
Expand Down Expand Up @@ -1000,16 +991,7 @@ def __repr__(self):
obj_repr = obj_repr[:76] + "...>"
return obj_repr

def __class_getitem__(cls, type):
"""Provide minimal support for using this class as generic
(for example in type annotations).

See PEP 484 and PEP 560 for more details. For example, `Popen[bytes]`
is a valid expression at runtime (type argument `bytes` indicates the
type used for stdout). Note, no type checking happens at runtime, but
a static type checker can be used.
"""
return cls
__class_getitem__ = classmethod(types.GenericAlias)

@property
def universal_newlines(self):
Expand Down
13 changes: 2 additions & 11 deletions Lib/tempfile.py
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,7 @@
import errno as _errno
from random import Random as _Random
import sys as _sys
import types as _types
import weakref as _weakref
import _thread
_allocate_lock = _thread.allocate_lock
Expand Down Expand Up @@ -643,17 +644,7 @@ def __init__(self, max_size=0, mode='w+b', buffering=-1,
'encoding': encoding, 'newline': newline,
'dir': dir, 'errors': errors}

def __class_getitem__(cls, type):
"""Provide minimal support for using this class as generic
(for example in type annotations).

See PEP 484 and PEP 560 for more details. For example,
`SpooledTemporaryFile[str]` is a valid expression at runtime (type
argument `str` indicates whether the file is open in bytes or text
mode). Note, no type checking happens at runtime, but a static type
checker can be used.
"""
return cls
__class_getitem__ = classmethod(_types.GenericAlias)

def _check(self, file):
if self._rolled: return
Expand Down
1 change: 1 addition & 0 deletions Lib/test/test_descrtut.py
Original file line number Diff line number Diff line change
Expand Up @@ -167,6 +167,7 @@ def merge(self, other):
>>> pprint.pprint(dir(list)) # like list.__dict__.keys(), but sorted
['__add__',
'__class__',
'__class_getitem__',
'__contains__',
'__delattr__',
'__delitem__',
Expand Down
2 changes: 1 addition & 1 deletion Lib/test/test_doctest.py
Original file line number Diff line number Diff line change
Expand Up @@ -665,7 +665,7 @@ def non_Python_modules(): r"""

>>> import builtins
>>> tests = doctest.DocTestFinder().find(builtins)
>>> 800 < len(tests) < 820 # approximate number of objects with docstrings
>>> 810 < len(tests) < 830 # approximate number of objects with docstrings
True
>>> real_tests = [t for t in tests if len(t.examples) > 0]
>>> len(real_tests) # objects that actually have doctests
Expand Down
Loading