Document PEP 585, 563, 604 and more (#9763)

Fixes #8629, fixes #8523.

This creates a new page to document issues arising from discrepancies
between the runtime and annotations. I felt this was better, rather than
force-fitting things into existing pages and "common issues", for
instance, it prevents us from having to explain PEP 563 in several
different places.

I do still list the runtime errors you'd get in the "common issues" page
to preserve SEO :-)

"String literal types", "Class name forward references", and "Import
cycles" are basically the same as where they were copied over from.

This also factors out the documentation of PEP 604 that I promised when
merging that PR (it seemed pretty verbose, particularly for the "kinds
of types" page). It's also a good place to document PEP 613, when we get
around to supporting that.

Resolves #9856.

Co-authored-by: hauntsaninja <>

Author: hauntsaninja

docs/source/builtin_types.rst

@@ -38,3 +38,8 @@ though they are similar to abstract base classes defined in
 :py:mod:`collections.abc` (formerly ``collections``), they are not identical. In
 particular, prior to Python 3.9, the built-in collection type objects do not
 support indexing.
+
+In Python 3.9 and later, built-in collection type objects support indexing. This
+means that you can use built-in classes or those from :py:mod:`collections.abc`
+instead of importing from :py:mod:`typing`. See :ref:`generic-builtins` for more
+details.

docs/source/common_issues.rst

@@ -210,6 +210,21 @@ checking would require a large number of ``assert foo is not None``
 checks to be inserted, and you want to minimize the number
 of code changes required to get a clean mypy run.
 
+Issues with code at runtime
+---------------------------
+
+Idiomatic use of type annotations can sometimes run up against what a given
+version of Python considers legal code. These can result in some of the
+following errors when trying to run your code:
+
+* ``ImportError`` from circular imports
+* ``NameError: name 'X' is not defined`` from forward references
+* ``TypeError: 'type' object is not subscriptable`` from types that are not generic at runtime
+* ``ImportError`` or ``ModuleNotFoundError`` from use of stub definitions not available at runtime
+* ``TypeError: unsupported operand type(s) for |: 'type' and 'type'`` from use of new syntax
+
+For dealing with these, see :ref:`runtime_troubles`.
+
 Mypy runs are slow
 ------------------
 
@@ -499,112 +514,6 @@ to see the types of all local variables at once. Example:
    run your code. Both are always available and you don't need to import
    them.
 
-
-.. _import-cycles:
-
-Import cycles
--------------
-
-An import cycle occurs where module A imports module B and module B
-imports module A (perhaps indirectly, e.g. ``A -> B -> C -> A``).
-Sometimes in order to add type annotations you have to add extra
-imports to a module and those imports cause cycles that didn't exist
-before.  If those cycles become a problem when running your program,
-there's a trick: if the import is only needed for type annotations in
-forward references (string literals) or comments, you can write the
-imports inside ``if TYPE_CHECKING:`` so that they are not executed at runtime.
-Example:
-
-File ``foo.py``:
-
-.. code-block:: python
-
-   from typing import List, TYPE_CHECKING
-
-   if TYPE_CHECKING:
-       import bar
-
-   def listify(arg: 'bar.BarClass') -> 'List[bar.BarClass]':
-       return [arg]
-
-File ``bar.py``:
-
-.. code-block:: python
-
-   from typing import List
-   from foo import listify
-
-   class BarClass:
-       def listifyme(self) -> 'List[BarClass]':
-           return listify(self)
-
-.. note::
-
-   The :py:data:`~typing.TYPE_CHECKING` constant defined by the :py:mod:`typing` module
-   is ``False`` at runtime but ``True`` while type checking.
-
-Python 3.5.1 doesn't have :py:data:`~typing.TYPE_CHECKING`. An alternative is
-to define a constant named ``MYPY`` that has the value ``False``
-at runtime. Mypy considers it to be ``True`` when type checking.
-Here's the above example modified to use ``MYPY``:
-
-.. code-block:: python
-
-   from typing import List
-
-   MYPY = False
-   if MYPY:
-       import bar
-
-   def listify(arg: 'bar.BarClass') -> 'List[bar.BarClass]':
-       return [arg]
-
-.. _not-generic-runtime:
-
-Using classes that are generic in stubs but not at runtime
-----------------------------------------------------------
-
-Some classes are declared as generic in stubs, but not at runtime. Examples
-in the standard library include :py:class:`os.PathLike` and :py:class:`queue.Queue`.
-Subscripting such a class will result in a runtime error:
-
-.. code-block:: python
-
-   from queue import Queue
-
-   class Tasks(Queue[str]):  # TypeError: 'type' object is not subscriptable
-       ...
-
-   results: Queue[int] = Queue()  # TypeError: 'type' object is not subscriptable
-
-To avoid these errors while still having precise types you can either use
-string literal types or :py:data:`~typing.TYPE_CHECKING`:
-
-.. code-block:: python
-
-   from queue import Queue
-   from typing import TYPE_CHECKING
-
-   if TYPE_CHECKING:
-       BaseQueue = Queue[str]  # this is only processed by mypy
-   else:
-       BaseQueue = Queue  # this is not seen by mypy but will be executed at runtime.
-
-   class Tasks(BaseQueue):  # OK
-       ...
-
-   results: 'Queue[int]' = Queue()  # OK
-
-If you are running Python 3.7+ you can use ``from __future__ import annotations``
-as a (nicer) alternative to string quotes, read more in :pep:`563`.  For example:
-
-.. code-block:: python
-
-   from __future__ import annotations
-   from queue import Queue
-
-   results: Queue[int] = Queue()  # This works at runtime
-
 .. _silencing-linters:
 
 Silencing linters

docs/source/index.rst

@@ -35,6 +35,7 @@ Mypy is a static type checker for Python 3 and Python 2.7.
    type_inference_and_annotations
    kinds_of_types
    class_basics
+   runtime_troubles
    protocols
    python2
    dynamic_typing

docs/source/kinds_of_types.rst

@@ -243,93 +243,24 @@ more specific type:
 
 .. _alternative_union_syntax:
 
-Alternative union syntax
-------------------------
+X | Y syntax for Unions
+-----------------------
 
-`PEP 604 <https://www.python.org/dev/peps/pep-0604/>`_ introduced an alternative way
-for writing union types. Starting with **Python 3.10** it is possible to write
-``Union[int, str]`` as ``int | str``. Any of the following options is possible
+:pep:`604` introduced an alternative way for spelling union types. In Python
+3.10 and later, you can write ``Union[int, str]`` as ``int | str``. It is
+possible to use this syntax in versions of Python where it isn't supported by
+the runtime with some limitations, see :ref:`runtime_troubles`.
 
 .. code-block:: python
 
     from typing import List
 
-    # Use as Union
     t1: int | str  # equivalent to Union[int, str]
 
-    # Use as Optional
     t2: int | None  # equivalent to Optional[int]
 
-    # Use in generics
-    t3: List[int | str]  # equivalent to List[Union[int, str]]
-
-    # Use in type aliases
-    T4 = int | None
-    x: T4
-
-    # Quoted variable annotations
-    t5: "int | str"
-
-    # Quoted function annotations
-    def f(t6: "int | str") -> None: ...
-
-    # Type comments
-    t6 = 42  # type: int | str
-
-It is possible to use most of these even for earlier versions. However there are some
-limitations to be aware of.
-
-.. _alternative_union_syntax_stub_files:
-
-Stub files
-""""""""""
-
-All options are supported, regardless of the Python version the project uses.
-
-.. _alternative_union_syntax_37:
-
-Python 3.7 - 3.9
-""""""""""""""""
-
-It is necessary to add ``from __future__ import annotations`` to delay the evaluation
-of type annotations. Not using it would result in a ``TypeError``.
-This does not apply for **type comments**, **quoted function** and **quoted variable** annotations,
-as those also work for earlier versions, see :ref:`below <alternative_union_syntax_older_version>`.
-
-.. warning::
-
-    Type aliases are **NOT** supported! Those result in a ``TypeError`` regardless
-    if the evaluation of type annotations is delayed.
-
-    Dynamic evaluation of annotations is **NOT** possible (e.g. ``typing.get_type_hints`` and ``eval``).
-    See `note PEP 604 <https://www.python.org/dev/peps/pep-0604/#change-only-pep-484-type-hints-to-accept-the-syntax-type1-type2>`_.
-    Use ``typing.Union`` or **Python 3.10** instead if you need those!
-
-.. code-block:: python
-
-    from __future__ import annotations
-
-    t1: int | None
-
-    # Type aliases
-    T2 = int | None  # TypeError!
-
-.. _alternative_union_syntax_older_version:
-
-Older versions
-""""""""""""""
-
-+------------------------------------------+-----------+-----------+-----------+
-| Python Version                           | 3.6       | 3.0 - 3.5 | 2.7       |
-+==========================================+===========+===========+===========+
-| Type comments                            | yes       | yes       | yes       |
-+------------------------------------------+-----------+-----------+-----------+
-| Quoted function annotations              | yes       | yes       |           |
-+------------------------------------------+-----------+-----------+-----------+
-| Quoted variable annotations              | yes       |           |           |
-+------------------------------------------+-----------+-----------+-----------+
-| Everything else                          |           |           |           |
-+------------------------------------------+-----------+-----------+-----------+
+    # Usable in type comments
+    t3 = 42  # type: int | str
 
 .. _strict_optional:
 
@@ -565,82 +496,6 @@ valid for any type, but it's much more
 useful for a programmer who is reading the code. This also makes
 it easier to migrate to strict ``None`` checking in the future.
 
-Class name forward references
-*****************************
-
-Python does not allow references to a class object before the class is
-defined. Thus this code does not work as expected:
-
-.. code-block:: python
-
-   def f(x: A) -> None:  # Error: Name A not defined
-       ...
-
-   class A:
-       ...
-
-In cases like these you can enter the type as a string literal — this
-is a *forward reference*:
-
-.. code-block:: python
-
-   def f(x: 'A') -> None:  # OK
-       ...
-
-   class A:
-       ...
-
-Starting from Python 3.7 (:pep:`563`), you can add the special import ``from __future__ import annotations``,
-which makes the use of string literals in annotations unnecessary:
-
-.. code-block:: python
-
-   from __future__ import annotations
-
-   def f(x: A) -> None:  # OK
-       ...
-
-   class A:
-       ...
-
-.. note::
-
-    Even with the ``__future__`` import, there are some scenarios that could still
-    require string literals, typically involving use of forward references or generics in:
-
-    * :ref:`type aliases <type-aliases>`;
-    * :ref:`casts <casts>`;
-    * type definitions (see :py:class:`~typing.TypeVar`, :py:func:`~typing.NewType`, :py:class:`~typing.NamedTuple`);
-    * base classes.
-
-    .. code-block:: python
-
-        # base class example
-        class A(Tuple['B', 'C']): ... # OK
-        class B: ...
-        class C: ...
-
-Of course, instead of using a string literal type or special import, you could move the
-function definition after the class definition. This is not always
-desirable or even possible, though.
-
-Any type can be entered as a string literal, and you can combine
-string-literal types with non-string-literal types freely:
-
-.. code-block:: python
-
-   def f(a: List['A']) -> None: ...  # OK
-   def g(n: 'int') -> None: ...      # OK, though not useful
-
-   class A: pass
-
-String literal types are never needed in ``# type:`` comments and :ref:`stub files <stub-files>`.
-
-String literal types must be defined (or imported) later *in the same
-module*.  They cannot be used to leave cross-module references
-unresolved.  (For dealing with import cycles, see
-:ref:`import-cycles`.)
-
 .. _type-aliases:
 
 Type aliases