Skip to content

Document PEP 585, 563, 604 and more. #9763

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
JukkaL merged 9 commits into from
Jan 6, 2021
Merged

Document PEP 585, 563, 604 and more. #9763

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
8142da0
Document PEP 585, 563, 604 and more.
Nov 27, 2020
5bc7167
more edits
Nov 28, 2020
8e66ca3
more edits
Nov 28, 2020
218a46e
make string literal types use with future annotations more explicit
Dec 11, 2020
39e5ff9
cross reference generic classes
Dec 11, 2020
c0c14b4
fix nits
Dec 11, 2020
fb16a12
reference previous discussion in forward references
Dec 11, 2020
e5d098d
mention py39 in runtime generics section
Dec 11, 2020
b3d6644
Mention conditional import of typing_extensions
Dec 29, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions docs/source/builtin_types.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,3 +38,8 @@ though they are similar to abstract base classes defined in
:py:mod:`collections.abc` (formerly ``collections``), they are not identical. In
particular, prior to Python 3.9, the built-in collection type objects do not
support indexing.

In Python 3.9 and later, built-in collection type objects support indexing. This
means that you can use built-in classes or those from :py:mod:`collections.abc`
instead of importing from :py:mod:`typing`. See :ref:`generic-builtins` for more
details.
121 changes: 15 additions & 106 deletions docs/source/common_issues.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -210,6 +210,21 @@ checking would require a large number of ``assert foo is not None``
checks to be inserted, and you want to minimize the number
of code changes required to get a clean mypy run.

Issues with code at runtime
---------------------------

Idiomatic use of type annotations can sometimes run up against what a given
version of Python considers legal code. These can result in some of the
following errors when trying to run your code:

* ``ImportError`` from circular imports
* ``NameError: name 'X' is not defined`` from forward references
* ``TypeError: 'type' object is not subscriptable`` from types that are not generic at runtime
* ``ImportError`` or ``ModuleNotFoundError`` from use of stub definitions not available at runtime
* ``TypeError: unsupported operand type(s) for |: 'type' and 'type'`` from use of new syntax

For dealing with these, see :ref:`runtime_troubles`.

Mypy runs are slow
------------------

Expand Down Expand Up @@ -499,112 +514,6 @@ to see the types of all local variables at once. Example:
run your code. Both are always available and you don't need to import
them.


.. _import-cycles:

Import cycles
-------------

An import cycle occurs where module A imports module B and module B
imports module A (perhaps indirectly, e.g. ``A -> B -> C -> A``).
Sometimes in order to add type annotations you have to add extra
imports to a module and those imports cause cycles that didn't exist
before. If those cycles become a problem when running your program,
there's a trick: if the import is only needed for type annotations in
forward references (string literals) or comments, you can write the
imports inside ``if TYPE_CHECKING:`` so that they are not executed at runtime.
Example:

File ``foo.py``:

.. code-block:: python

from typing import List, TYPE_CHECKING

if TYPE_CHECKING:
import bar

def listify(arg: 'bar.BarClass') -> 'List[bar.BarClass]':
return [arg]

File ``bar.py``:

.. code-block:: python

from typing import List
from foo import listify

class BarClass:
def listifyme(self) -> 'List[BarClass]':
return listify(self)

.. note::

The :py:data:`~typing.TYPE_CHECKING` constant defined by the :py:mod:`typing` module
is ``False`` at runtime but ``True`` while type checking.

Python 3.5.1 doesn't have :py:data:`~typing.TYPE_CHECKING`. An alternative is
to define a constant named ``MYPY`` that has the value ``False``
at runtime. Mypy considers it to be ``True`` when type checking.
Here's the above example modified to use ``MYPY``:

.. code-block:: python

from typing import List

MYPY = False
if MYPY:
import bar

def listify(arg: 'bar.BarClass') -> 'List[bar.BarClass]':
return [arg]

.. _not-generic-runtime:

Using classes that are generic in stubs but not at runtime
----------------------------------------------------------

Some classes are declared as generic in stubs, but not at runtime. Examples
in the standard library include :py:class:`os.PathLike` and :py:class:`queue.Queue`.
Subscripting such a class will result in a runtime error:

.. code-block:: python

from queue import Queue

class Tasks(Queue[str]): # TypeError: 'type' object is not subscriptable
...

results: Queue[int] = Queue() # TypeError: 'type' object is not subscriptable

To avoid these errors while still having precise types you can either use
string literal types or :py:data:`~typing.TYPE_CHECKING`:

.. code-block:: python

from queue import Queue
from typing import TYPE_CHECKING

if TYPE_CHECKING:
BaseQueue = Queue[str] # this is only processed by mypy
else:
BaseQueue = Queue # this is not seen by mypy but will be executed at runtime.

class Tasks(BaseQueue): # OK
...

results: 'Queue[int]' = Queue() # OK

If you are running Python 3.7+ you can use ``from __future__ import annotations``
as a (nicer) alternative to string quotes, read more in :pep:`563`. For example:

.. code-block:: python

from __future__ import annotations
from queue import Queue

results: Queue[int] = Queue() # This works at runtime

.. _silencing-linters:

Silencing linters
Expand Down
1 change: 1 addition & 0 deletions docs/source/index.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,7 @@ Mypy is a static type checker for Python 3 and Python 2.7.
type_inference_and_annotations
kinds_of_types
class_basics
runtime_troubles
protocols
python2
dynamic_typing
Expand Down
161 changes: 8 additions & 153 deletions docs/source/kinds_of_types.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -243,93 +243,24 @@ more specific type:

.. _alternative_union_syntax:

Alternative union syntax
------------------------
X | Y syntax for Unions
-----------------------

`PEP 604 <https://www.python.org/dev/peps/pep-0604/>`_ introduced an alternative way
for writing union types. Starting with **Python 3.10** it is possible to write
``Union[int, str]`` as ``int | str``. Any of the following options is possible
:pep:`604` introduced an alternative way for spelling union types. In Python
3.10 and later, you can write ``Union[int, str]`` as ``int | str``. It is
possible to use this syntax in versions of Python where it isn't supported by
the runtime with some limitations, see :ref:`runtime_troubles`.

.. code-block:: python

from typing import List

# Use as Union
t1: int | str # equivalent to Union[int, str]

# Use as Optional
t2: int | None # equivalent to Optional[int]

# Use in generics
t3: List[int | str] # equivalent to List[Union[int, str]]

# Use in type aliases
T4 = int | None
x: T4

# Quoted variable annotations
t5: "int | str"

# Quoted function annotations
def f(t6: "int | str") -> None: ...

# Type comments
t6 = 42 # type: int | str

It is possible to use most of these even for earlier versions. However there are some
limitations to be aware of.

.. _alternative_union_syntax_stub_files:

Stub files
""""""""""

All options are supported, regardless of the Python version the project uses.

.. _alternative_union_syntax_37:

Python 3.7 - 3.9
""""""""""""""""

It is necessary to add ``from __future__ import annotations`` to delay the evaluation
of type annotations. Not using it would result in a ``TypeError``.
This does not apply for **type comments**, **quoted function** and **quoted variable** annotations,
as those also work for earlier versions, see :ref:`below <alternative_union_syntax_older_version>`.

.. warning::

Type aliases are **NOT** supported! Those result in a ``TypeError`` regardless
if the evaluation of type annotations is delayed.

Dynamic evaluation of annotations is **NOT** possible (e.g. ``typing.get_type_hints`` and ``eval``).
See `note PEP 604 <https://www.python.org/dev/peps/pep-0604/#change-only-pep-484-type-hints-to-accept-the-syntax-type1-type2>`_.
Use ``typing.Union`` or **Python 3.10** instead if you need those!

.. code-block:: python

from __future__ import annotations

t1: int | None

# Type aliases
T2 = int | None # TypeError!

.. _alternative_union_syntax_older_version:

Older versions
""""""""""""""

+------------------------------------------+-----------+-----------+-----------+
| Python Version | 3.6 | 3.0 - 3.5 | 2.7 |
+==========================================+===========+===========+===========+
| Type comments | yes | yes | yes |
+------------------------------------------+-----------+-----------+-----------+
| Quoted function annotations | yes | yes | |
+------------------------------------------+-----------+-----------+-----------+
| Quoted variable annotations | yes | | |
+------------------------------------------+-----------+-----------+-----------+
| Everything else | | | |
+------------------------------------------+-----------+-----------+-----------+
# Usable in type comments
t3 = 42 # type: int | str

.. _strict_optional:

Expand Down Expand Up @@ -565,82 +496,6 @@ valid for any type, but it's much more
useful for a programmer who is reading the code. This also makes
it easier to migrate to strict ``None`` checking in the future.

Class name forward references
*****************************

Python does not allow references to a class object before the class is
defined. Thus this code does not work as expected:

.. code-block:: python

def f(x: A) -> None: # Error: Name A not defined
...

class A:
...

In cases like these you can enter the type as a string literal — this
is a *forward reference*:

.. code-block:: python

def f(x: 'A') -> None: # OK
...

class A:
...

Starting from Python 3.7 (:pep:`563`), you can add the special import ``from __future__ import annotations``,
which makes the use of string literals in annotations unnecessary:

.. code-block:: python

from __future__ import annotations

def f(x: A) -> None: # OK
...

class A:
...

.. note::

Even with the ``__future__`` import, there are some scenarios that could still
require string literals, typically involving use of forward references or generics in:

* :ref:`type aliases <type-aliases>`;
* :ref:`casts <casts>`;
* type definitions (see :py:class:`~typing.TypeVar`, :py:func:`~typing.NewType`, :py:class:`~typing.NamedTuple`);
* base classes.

.. code-block:: python

# base class example
class A(Tuple['B', 'C']): ... # OK
class B: ...
class C: ...

Of course, instead of using a string literal type or special import, you could move the
function definition after the class definition. This is not always
desirable or even possible, though.

Any type can be entered as a string literal, and you can combine
string-literal types with non-string-literal types freely:

.. code-block:: python

def f(a: List['A']) -> None: ... # OK
def g(n: 'int') -> None: ... # OK, though not useful

class A: pass

String literal types are never needed in ``# type:`` comments and :ref:`stub files <stub-files>`.

String literal types must be defined (or imported) later *in the same
module*. They cannot be used to leave cross-module references
unresolved. (For dealing with import cycles, see
:ref:`import-cycles`.)

.. _type-aliases:

Type aliases
Expand Down
Loading