Skip to content

Ensured explicit closing of async generators #3593

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 4 commits into
base: master
Choose a base branch
from

Conversation

agronholm
Copy link
Contributor

Summary

This change ensures that async generators are always explicitly closed, even if the task running the generator is cancelled or another exception is raised. This prevents unpredictable behavior and also avoids the asyncgen finalizer warnings on Trio.

Checklist

  • I understand that this PR may be closed in case there was no previous discussion. (This doesn't apply to typos!)
  • I've added a test for each change that was introduced, and I tried as much as possible to make a single atomic change.
  • I've updated the documentation accordingly.

@agronholm
Copy link
Contributor Author

Note that this depends on another PR: #3592. I will mark as ready to review once that one has been merged.

@agronholm agronholm changed the title Ensured explicit closing of async generators and dropped Python 3.8 Ensured explicit closing of async generators Jun 26, 2025
@agronholm agronholm marked this pull request as ready for review July 2, 2025 10:28
@agronholm
Copy link
Contributor Author

I checked locally with Python 3.10 that the supposedly missing coverage cannot be reproduced. I'm seeing 100% coverage, even without the httpcore changes.

@agronholm
Copy link
Contributor Author

I checked locally with Python 3.10 that the supposedly missing coverage cannot be reproduced. I'm seeing 100% coverage, even without the httpcore changes.

Never mind, I was on the wrong branch. Fixing.

@agronholm
Copy link
Contributor Author

@Kludex All good now.

@@ -99,9 +102,8 @@ class UnattachedStream(AsyncByteStream, SyncByteStream):
def __iter__(self) -> Iterator[bytes]:
raise StreamClosed()

async def __aiter__(self) -> AsyncIterator[bytes]:
def __aiter__(self) -> NoReturn:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is the async removed?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not really necessary here. __aiter__() can return any arbitrary object that supports the AsyncIterator protocol, and the implementation here just raises NotImplementedError.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@tomchristie this is not an important part of this PR so if you object, I can just revert it.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On another note, is either one of you at EuroPython?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On another note, is either one of you at EuroPython?

I'm not. 😞



@asynccontextmanager
async def safe_async_iterate(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Interesting. I'm curious about this vs. aclosing.

https://docs.python.org/3/library/contextlib.html#contextlib.aclosing

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I originally went with aclosing() but it doesn't work very well with arbitrary async iterables.

@@ -6,6 +6,10 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).

## [UNRELEASED]

### Fixed

* Explicitly close all async generators to ensure predictable behavior
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍🏼 Neat.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants