bpo-39491: Merge PEP 593 (typing.Annotated) support #18260

jstasiak · 2020-01-29T21:59:53Z

PEP 593 has been accepted some time ago. I got a green light for merging
this from Till, so I went ahead and combined the code contributed to
typing_extensions[1] and the documentation from the PEP 593 text[2].

My changes were limited to:

removing code designed for typing_extensions to run on older Python
versions
removing some irrelevant parts of the PEP text when copying it over as
documentation and otherwise changing few small bits to better serve
the purpose
changing the get_type_hints signature to match reality (parameter
names)

I wasn't entirely sure how to go about crediting the authors but I used
my best judgment, let me know if something needs changing in this
regard.

[1] https://github.com/python/typing/blob/8280de241fd8c8afe727c7860254b753e383b360/typing_extensions/src_py3/typing_extensions.py
[2] https://github.com/python/peps/blob/17710b879882454d55f82c2d44596e8e9f8e4bff/pep-0593.rst

https://bugs.python.org/issue39491

PEP 593 has been accepted some time ago. I got a green light for merging this from Till, so I went ahead and combined the code contributed to typing_extensions[1] and the documentation from the PEP 593 text[2]. My changes were limited to: * removing code designed for typing_extensions to run on older Python versions * removing some irrelevant parts of the PEP text when copying it over as documentation and otherwise changing few small bits to better serve the purpose * changing the get_type_hints signature to match reality (parameter names) I wasn't entirely sure how to go about crediting the authors but I used my best judgment, let me know if something needs changing in this regard. [1] https://github.com/python/typing/blob/8280de241fd8c8afe727c7860254b753e383b360/typing_extensions/src_py3/typing_extensions.py [2] https://github.com/python/peps/blob/17710b879882454d55f82c2d44596e8e9f8e4bff/pep-0593.rst

jstasiak · 2020-01-30T14:48:35Z

I also modified get_origin() and get_args() to support extracting information from Annotated[...] at runtime, since the conclusion of the discussion on typing-sig was that get_origin() and get_args() are the public API to do that.

gvanrossum

I didn't audit that the changes to typing.py and test_typing.py are exactly what's in the typing repo, I skimmed it and beyond that I trust you.

gvanrossum · 2020-02-05T00:14:00Z

Doc/library/typing.rst

+   more information). For example::
+
+       class Student(NamedTuple):
+           name: Annotated[str, struct.ctype("<10s")]


The builtin module struct doesn't have an attribute ctype. Maybe pick a different example?

gvanrossum · 2020-02-05T00:20:22Z

Doc/library/typing.rst

@@ -1372,3 +1388,87 @@ The module defines the following classes, functions and decorators:
   evaluated, so the second annotation does not need to be enclosed in quotes.

   .. versionadded:: 3.5.2
+
+.. data:: Annotated


Can I ask that you hint at the possibility of passing multiple extras (Annotated[T, x, y, z]) early in the description?

gvanrossum · 2020-02-05T00:21:44Z

Doc/library/typing.rst

+   It's up to the tool consuming the annotations
+   to decide whether the client is allowed to have several annotations on
+   one type and how to merge those annotations.


Reflow this paragraph.

gvanrossum · 2020-02-05T00:23:11Z

Doc/library/typing.rst

+       T1 = Annotated[int, ValueRange(-10, 5)]
+       T2 = Annotated[T1, ValueRange(-20, 3)]
+
+   A new ``include_extras`` parameter to :func:`get_type_hints` has been added,


It will stop to be "new" in the next version. You can just state that the functionality exists without describing that it's new and has been added (that would be text for "what's new" though).

gvanrossum · 2020-02-05T00:24:11Z

Doc/library/typing.rst

+
+   * ``Annotated`` can be used with nested and generic aliases::
+
+       Typevar T = ...


Please use correct syntax for type variables.

gvanrossum · 2020-02-05T00:24:47Z

Doc/whatsnew/3.9.rst

+:pep:`593` introduced an :data:`typing.Annotated` type to decorate existing
+types with context-specific metadata. (Contributed by Till Varoquaux and
+Konstantin Kashin.)


Also mention the new argument to get_type_hints() here.

gvanrossum · 2020-02-05T00:27:37Z

Lib/test/test_typing.py

+
+

Only one blank line here.

jstasiak · 2020-02-05T00:50:20Z

If there's a difference between this and what's in the typing repo other than I listed above it's a mistake, but I don't believe anything significant got changed when transferring the code here. As for your comments – they should be addressed now.

gvanrossum

Thanks! I'll land this once the tests pass.

bedevere-bot · 2020-02-05T01:10:23Z

@gvanrossum: Please replace # with GH- in the commit message next time. Thanks!

jstasiak · 2020-02-05T14:17:24Z

Woo!

I'm wondering now – what should happen in the typing repository? Should the typing module import (from typing_extensions) and reexport Annotated and get_type_hints?

jstasiak · 2020-02-05T14:30:46Z

I'm thinking typing.Annotated needs to be the same object as typing_extensions.Annotated to avoid multiple implementations present at runtime (I'm sure it'd cause issues) which should be easy enough with a conditional import in typing_extensions, correct me if this is not making sense.

gvanrossum · 2020-02-05T16:18:40Z

I can't speak for @ilevkivskyi (who seems to be too busy to follow this, alas) but I think the dependency clearly goes from typing_extension to typing -- typing doesn't use anything from typing_extensions, while typing_extensions already depends on typing.

So to make typing_extensions work with Python 3.9, typing_extensions should try to import Annotated from typing before defining its own versions (and refraining from defining its own versions if it can import Annotated from typing).

I don't think it's worth your while to copy Annotated and the improved get_type_info into typing -- that would only benefit Python versions so old that they don't have typing in the stdlib at all. Probably the only relevant version would be 2.7, whose end of life has come. Let those users continue to import this feature from typing_extensions.

There's also something similar to be done for typeshed, otherwise 3.9 users who import Annotated from typing will get complaints. Here I think the definition should strictly check on sys.version_info: in 3.9 and later, the definition is in typing, and typing_extensions imports from there; in earlier versions, the definition is only in typing_extensions.

The implementations come from CPython commit 427c84f13f77 with one small change – the get_origin's docstring mentions Annotated as it's also supported. get_origin() and get_args() introduced in [1] and modified in [2] to support Annotated. [1] python/cpython#13685 [2] python/cpython#18260

Following [1] this prevents multiple runtime implementations of Annotated and get_type_hints from existing on Python 3.9 (which has recently merged PEP 593 changes[2]). Reexporting allows code targetting both Python pre-3.9 and 3.9+ to be able to import from typing_extensions and to keep working without changes. [1] python/cpython#18260 (comment) [2] python/cpython#18260

I forgot to do it in python#18260.

This is to handle PEP 593 support recently merged into CPython[1]. [1] python/cpython#18260

I forgot to do it in #18260.

The implementations come from CPython commit 427c84f13f77 with one small change – the get_origin's docstring mentions Annotated as it's also supported. get_origin() and get_args() introduced in [1] and modified in [2] to support Annotated. [1] python/cpython#13685 [2] python/cpython#18260 * Define our own get_origin()/get_args() in typing_extensions on Python 3.8 Otherwise typing_extensions.get_origin() would not recognize typing_extensions.Annotated on 3.8.

…699) Following [1] this prevents multiple runtime implementations of Annotated and get_type_hints from existing on Python 3.9 (which has recently merged PEP 593 changes[2]). Reexporting allows code targetting both Python pre-3.9 and 3.9+ to be able to import from typing_extensions and to keep working without changes. [1] python/cpython#18260 (comment) [2] python/cpython#18260

The implementations come from CPython commit 427c84f13f77 with one small change – the get_origin's docstring mentions Annotated as it's also supported. get_origin() and get_args() introduced in [1] and modified in [2] to support Annotated. [1] python/cpython#13685 [2] python/cpython#18260 * Define our own get_origin()/get_args() in typing_extensions on Python 3.8 Otherwise typing_extensions.get_origin() would not recognize typing_extensions.Annotated on 3.8.

…#699) Following [1] this prevents multiple runtime implementations of Annotated and get_type_hints from existing on Python 3.9 (which has recently merged PEP 593 changes[2]). Reexporting allows code targetting both Python pre-3.9 and 3.9+ to be able to import from typing_extensions and to keep working without changes. [1] python/cpython#18260 (comment) [2] python/cpython#18260

The implementations come from CPython commit 427c84f13f77 with one small change – the get_origin's docstring mentions Annotated as it's also supported. get_origin() and get_args() introduced in [1] and modified in [2] to support Annotated. [1] python/cpython#13685 [2] python/cpython#18260 * Define our own get_origin()/get_args() in typing_extensions on Python 3.8 Otherwise typing_extensions.get_origin() would not recognize typing_extensions.Annotated on 3.8.

…#699) Following [1] this prevents multiple runtime implementations of Annotated and get_type_hints from existing on Python 3.9 (which has recently merged PEP 593 changes[2]). Reexporting allows code targetting both Python pre-3.9 and 3.9+ to be able to import from typing_extensions and to keep working without changes. [1] python/cpython#18260 (comment) [2] python/cpython#18260

jstasiak requested review from gvanrossum and ilevkivskyi as code owners January 29, 2020 21:59

the-knights-who-say-ni added the CLA signed label Jan 29, 2020

bedevere-bot added the awaiting review label Jan 29, 2020

Make typing.get_origin() and get_args() aware of Annotated

21bde41

gvanrossum reviewed Feb 5, 2020

View reviewed changes

Implement changes requested in code review

8ce4053

gvanrossum approved these changes Feb 5, 2020

View reviewed changes

bedevere-bot added awaiting merge and removed awaiting review labels Feb 5, 2020

gvanrossum merged commit cf5b109 into python:master Feb 5, 2020

bedevere-bot removed the awaiting merge label Feb 5, 2020

jstasiak deleted the pep593 branch February 5, 2020 13:39

jstasiak mentioned this pull request Feb 5, 2020

Support typing.Annotated on top of typing_extensions.Annotated python/mypy#8371

Merged

jstasiak mentioned this pull request Feb 6, 2020

Backport get_origin() and get_args() python/typing#698

Merged

jstasiak mentioned this pull request Feb 6, 2020

Reuse stdlib PEP 593 implementation in typing_extensions if present python/typing#699

Merged

jstasiak added a commit to jstasiak/cpython that referenced this pull request Feb 6, 2020

Mention Annotated in get_origin() docstring

ddc50ba

I forgot to do it in python#18260.

jstasiak mentioned this pull request Feb 6, 2020

bpo-39491: Mention Annotated in get_origin() docstring #18379

Merged

msullivan pushed a commit to python/mypy that referenced this pull request Feb 6, 2020

Support typing.Annotated on top of typing_extensions.Annotated (#8371)

8888b1a

This is to handle PEP 593 support recently merged into CPython[1]. [1] python/cpython#18260

gvanrossum pushed a commit that referenced this pull request Feb 7, 2020

bpo-39491: Mention Annotated in get_origin() docstring (GH-18379)

38aaaaa

I forgot to do it in #18260.

sm-Fifteen mentioned this pull request Jun 23, 2020

[Feature Request] Provide a discriminated union type (OpenAPI 3) pydantic/pydantic#619

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

bpo-39491: Merge PEP 593 (typing.Annotated) support #18260

bpo-39491: Merge PEP 593 (typing.Annotated) support #18260

jstasiak commented Jan 29, 2020 •

edited by bedevere-bot

Loading

jstasiak commented Jan 30, 2020

gvanrossum left a comment

gvanrossum Feb 5, 2020

gvanrossum Feb 5, 2020

gvanrossum Feb 5, 2020

gvanrossum Feb 5, 2020

gvanrossum Feb 5, 2020

gvanrossum Feb 5, 2020

gvanrossum Feb 5, 2020

jstasiak commented Feb 5, 2020

gvanrossum left a comment

bedevere-bot commented Feb 5, 2020

jstasiak commented Feb 5, 2020

jstasiak commented Feb 5, 2020

gvanrossum commented Feb 5, 2020


		* ``Annotated`` can be used with nested and generic aliases::

		Typevar T = ...

bpo-39491: Merge PEP 593 (typing.Annotated) support #18260

bpo-39491: Merge PEP 593 (typing.Annotated) support #18260

Conversation

jstasiak commented Jan 29, 2020 • edited by bedevere-bot Loading

jstasiak commented Jan 30, 2020

gvanrossum left a comment

Choose a reason for hiding this comment

gvanrossum Feb 5, 2020

Choose a reason for hiding this comment

gvanrossum Feb 5, 2020

Choose a reason for hiding this comment

gvanrossum Feb 5, 2020

Choose a reason for hiding this comment

gvanrossum Feb 5, 2020

Choose a reason for hiding this comment

gvanrossum Feb 5, 2020

Choose a reason for hiding this comment

gvanrossum Feb 5, 2020

Choose a reason for hiding this comment

gvanrossum Feb 5, 2020

Choose a reason for hiding this comment

jstasiak commented Feb 5, 2020

gvanrossum left a comment

Choose a reason for hiding this comment

bedevere-bot commented Feb 5, 2020

jstasiak commented Feb 5, 2020

jstasiak commented Feb 5, 2020

gvanrossum commented Feb 5, 2020

jstasiak commented Jan 29, 2020 •

edited by bedevere-bot

Loading