Skip to content

bpo-47062: Implement asyncio.Runner context manager #31799

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 32 commits into from
Mar 24, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
32 commits
Select commit Hold shift + click to select a range
61f10d0
Implement Runner class
asvetlov Mar 9, 2022
d471799
Add get_loop() / new_loop() methods
asvetlov Mar 10, 2022
c68ba85
Clarify
asvetlov Mar 10, 2022
d530481
Add tests
asvetlov Mar 10, 2022
6ee7c9a
Add a comment
asvetlov Mar 10, 2022
e15d70b
Adopt IsolatedAsyncioTestCase to use Runner
asvetlov Mar 10, 2022
5c7a669
Merge branch 'main' into asyncio-runner
asvetlov Mar 10, 2022
d41988d
Merge branch 'main' into asyncio-runner
asvetlov Mar 10, 2022
8014227
Add docs sketch
asvetlov Mar 12, 2022
98e39db
Merge branch 'main' into asyncio-runner
asvetlov Mar 14, 2022
98e7a22
Add context arg to Runner.run()
asvetlov Mar 14, 2022
f4cd673
Work on docs
asvetlov Mar 17, 2022
1764045
Merge branch 'main' into asyncio-runner
asvetlov Mar 18, 2022
a443b37
Work on
asvetlov Mar 18, 2022
e6be8f7
Improve docs
asvetlov Mar 18, 2022
b1dfe4f
Add NEWS
asvetlov Mar 18, 2022
759f72a
Drop not related file
asvetlov Mar 18, 2022
f47d66a
Fix doc
asvetlov Mar 18, 2022
546440b
Update Lib/asyncio/runners.py
asvetlov Mar 18, 2022
6935f7d
Update Doc/library/asyncio-runner.rst
asvetlov Mar 18, 2022
9b9a004
Update Lib/asyncio/runners.py
asvetlov Mar 18, 2022
b0c5b8c
Improve wording
asvetlov Mar 19, 2022
599c9db
Add a test for double 'with' usage
asvetlov Mar 19, 2022
b0da74b
Improve tests
asvetlov Mar 19, 2022
04cfff9
Work on
asvetlov Mar 22, 2022
8753465
Merge branch 'main' into asyncio-runner
asvetlov Mar 22, 2022
674ad4e
Lazy init version
asvetlov Mar 22, 2022
7cd5430
Tune
asvetlov Mar 22, 2022
dd28ef7
Drop explicit get_context() function, asyncio.Task has no it also
asvetlov Mar 23, 2022
5e13b2e
Add docs for .close() method
asvetlov Mar 23, 2022
c0b999d
Add better error message for recursive run() call
asvetlov Mar 24, 2022
4937cd0
Add a note
asvetlov Mar 24, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
121 changes: 121 additions & 0 deletions Doc/library/asyncio-runner.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,121 @@
.. currentmodule:: asyncio


=======
Runners
=======

**Source code:** :source:`Lib/asyncio/runners.py`


This section outlines high-level asyncio primitives to run asyncio code.

They are built on top of an :ref:`event loop <asyncio-event-loop>` with the aim
to simplify async code usage for common wide-spread scenarios.

.. contents::
:depth: 1
:local:



Running an asyncio Program
==========================

.. function:: run(coro, *, debug=None)

Execute the :term:`coroutine` *coro* and return the result.

This function runs the passed coroutine, taking care of
managing the asyncio event loop, *finalizing asynchronous
generators*, and closing the threadpool.

This function cannot be called when another asyncio event loop is
running in the same thread.

If *debug* is ``True``, the event loop will be run in debug mode. ``False`` disables
debug mode explicitly. ``None`` is used to respect the global
:ref:`asyncio-debug-mode` settings.

This function always creates a new event loop and closes it at
the end. It should be used as a main entry point for asyncio
programs, and should ideally only be called once.

Example::

async def main():
await asyncio.sleep(1)
print('hello')

asyncio.run(main())

.. versionadded:: 3.7

.. versionchanged:: 3.9
Updated to use :meth:`loop.shutdown_default_executor`.

.. versionchanged:: 3.10

*debug* is ``None`` by default to respect the global debug mode settings.


Runner context manager
======================

.. class:: Runner(*, debug=None, factory=None)

A context manager that simplifies *multiple* async function calls in the same
context.

Sometimes several top-level async functions should be called in the same :ref:`event
loop <asyncio-event-loop>` and :class:`contextvars.Context`.

If *debug* is ``True``, the event loop will be run in debug mode. ``False`` disables
debug mode explicitly. ``None`` is used to respect the global
:ref:`asyncio-debug-mode` settings.

*factory* could be used for overriding the loop creation.
:func:`asyncio.new_event_loop` is used if ``None``.

Basically, :func:`asyncio.run()` example can be rewritten with the runner usage::

async def main():
await asyncio.sleep(1)
print('hello')

with asyncio.Runner() as runner:
runner.run(main())

.. versionadded:: 3.11

.. method:: run(coro, *, context=None)

Run a :term:`coroutine <coroutine>` *coro* in the embedded loop.

Return the coroutine's result or raise its exception.

An optional keyword-only *context* argument allows specifying a
custom :class:`contextvars.Context` for the *coro* to run in.
The runner's default context is used if ``None``.

This function cannot be called when another asyncio event loop is
running in the same thread.

.. method:: close()

Close the runner.

Finalize asynchronous generators, shutdown default executor, close the event loop
and release embedded :class:`contextvars.Context`.

.. method:: get_loop()

Return the event loop associated with the runner instance.

.. note::

:class:`Runner` uses the lazy initialization strategy, its constructor doesn't
initialize underlying low-level structures.

Embedded *loop* and *context* are created at the :keyword:`with` body entering
or the first call of :meth:`run` or :meth:`get_loop`.
37 changes: 0 additions & 37 deletions Doc/library/asyncio-task.rst
Original file line number Diff line number Diff line change
Expand Up @@ -204,43 +204,6 @@ A good example of a low-level function that returns a Future object
is :meth:`loop.run_in_executor`.


Running an asyncio Program
==========================

.. function:: run(coro, *, debug=False)

Execute the :term:`coroutine` *coro* and return the result.

This function runs the passed coroutine, taking care of
managing the asyncio event loop, *finalizing asynchronous
generators*, and closing the threadpool.

This function cannot be called when another asyncio event loop is
running in the same thread.

If *debug* is ``True``, the event loop will be run in debug mode.

This function always creates a new event loop and closes it at
the end. It should be used as a main entry point for asyncio
programs, and should ideally only be called once.

Example::

async def main():
await asyncio.sleep(1)
print('hello')

asyncio.run(main())

.. versionadded:: 3.7

.. versionchanged:: 3.9
Updated to use :meth:`loop.shutdown_default_executor`.

.. note::
The source code for ``asyncio.run()`` can be found in
:source:`Lib/asyncio/runners.py`.

Creating Tasks
==============

Expand Down
1 change: 1 addition & 0 deletions Doc/library/asyncio.rst
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,7 @@ Additionally, there are **low-level** APIs for
:caption: High-level APIs
:maxdepth: 1

asyncio-runner.rst
asyncio-task.rst
asyncio-stream.rst
asyncio-sync.rst
Expand Down
124 changes: 106 additions & 18 deletions Lib/asyncio/runners.py
Original file line number Diff line number Diff line change
@@ -1,10 +1,112 @@
__all__ = 'run',
__all__ = ('Runner', 'run')

import contextvars
import enum
from . import coroutines
from . import events
from . import tasks


class _State(enum.Enum):
CREATED = "created"
INITIALIZED = "initialized"
CLOSED = "closed"


class Runner:
"""A context manager that controls event loop life cycle.

The context manager always creates a new event loop,
allows to run async functions inside it,
and properly finalizes the loop at the context manager exit.

If debug is True, the event loop will be run in debug mode.
If factory is passed, it is used for new event loop creation.

asyncio.run(main(), debug=True)

is a shortcut for

with asyncio.Runner(debug=True) as runner:
runner.run(main())

The run() method can be called multiple times within the runner's context.

This can be useful for interactive console (e.g. IPython),
unittest runners, console tools, -- everywhere when async code
is called from existing sync framework and where the preferred single
asyncio.run() call doesn't work.

"""

# Note: the class is final, it is not intended for inheritance.

def __init__(self, *, debug=None, factory=None):
self._state = _State.CREATED
self._debug = debug
self._factory = factory
self._loop = None
self._context = None

def __enter__(self):
self._lazy_init()
return self

def __exit__(self, exc_type, exc_val, exc_tb):
self.close()

def close(self):
"""Shutdown and close event loop."""
if self._state is not _State.INITIALIZED:
return
try:
loop = self._loop
_cancel_all_tasks(loop)
loop.run_until_complete(loop.shutdown_asyncgens())
loop.run_until_complete(loop.shutdown_default_executor())
finally:
loop.close()
self._loop = None
self._state = _State.CLOSED

def get_loop(self):
"""Return embedded event loop."""
self._lazy_init()
return self._loop

def run(self, coro, *, context=None):
"""Run a coroutine inside the embedded event loop."""
if not coroutines.iscoroutine(coro):
raise ValueError("a coroutine was expected, got {!r}".format(coro))

if events._get_running_loop() is not None:
# fail fast with short traceback
raise RuntimeError(
"Runner.run() cannot be called from a running event loop")

self._lazy_init()

if context is None:
context = self._context
task = self._loop.create_task(coro, context=context)
return self._loop.run_until_complete(task)

def _lazy_init(self):
if self._state is _State.CLOSED:
raise RuntimeError("Runner is closed")
if self._state is _State.INITIALIZED:
return
if self._factory is None:
self._loop = events.new_event_loop()
else:
self._loop = self._factory()
if self._debug is not None:
self._loop.set_debug(self._debug)
self._context = contextvars.copy_context()
self._state = _State.INITIALIZED



def run(main, *, debug=None):
"""Execute the coroutine and return the result.

Expand All @@ -30,26 +132,12 @@ async def main():
asyncio.run(main())
"""
if events._get_running_loop() is not None:
# fail fast with short traceback
raise RuntimeError(
"asyncio.run() cannot be called from a running event loop")

if not coroutines.iscoroutine(main):
raise ValueError("a coroutine was expected, got {!r}".format(main))

loop = events.new_event_loop()
try:
events.set_event_loop(loop)
if debug is not None:
loop.set_debug(debug)
return loop.run_until_complete(main)
finally:
try:
_cancel_all_tasks(loop)
loop.run_until_complete(loop.shutdown_asyncgens())
loop.run_until_complete(loop.shutdown_default_executor())
finally:
events.set_event_loop(None)
loop.close()
with Runner(debug=debug) as runner:
return runner.run(main)


def _cancel_all_tasks(loop):
Expand Down
Loading