Skip to content

Normalize asyncio writer close races - #4217

Open
lh0156 wants to merge 2 commits into
redis:masterfrom
lh0156:agent/async-writer-race
Open

Normalize asyncio writer close races#4217
lh0156 wants to merge 2 commits into
redis:masterfrom
lh0156:agent/async-writer-race

Conversation

@lh0156

@lh0156 lh0156 commented Jul 26, 2026

Copy link
Copy Markdown

Fixes #3546

Problem

When an asyncio transport is closed concurrently with a write, Python can raise a low-level TypeError from StreamWriter.writelines() after its internal _write_ready callback has been cleared. redis-py currently lets that raw TypeError escape, so the existing retry policy cannot recognize the broken connection as retryable.

Solution

  • Keep a local writer reference for the complete write/drain operation.
  • Convert writer-close AttributeError/TypeError failures into redis-py's ConnectionError.
  • Reuse the existing disconnect path, so the connection is discarded and configured retries can handle the failure.
  • Route both timeout and no-timeout writes through the same helper to keep behavior consistent.

Tests

  • Add a deterministic async regression test for the closed-writer race.
  • Cover both socket_timeout=None and finite socket_timeout paths.
  • Full async connection suite: 65 passed, 2 skipped.
  • Async client suite: 7 passed.
  • invoke linters, compileall, and git diff --check passed.

The local integration skips are unchanged and are unrelated to this focused writer error path.


Note

Low Risk
Narrow change to asyncio write error normalization on the connection object, with targeted regression tests and no auth or data-path changes.

Overview
Fixes a race where concurrent transport close during an asyncio write could surface a raw AttributeError from StreamWriter.writelines() instead of a connection failure redis-py can treat as retryable.

_send_packed_command now holds a local writer reference, raises ConnectionError("Connection closed while writing") when the writer is already gone or writelines/drain fails with AttributeError, and send_packed_command routes both timed and untimed writes through that helper so behavior matches. Failures still go through the existing disconnect(nowait=True) path via the outer BaseException handler.

Adds async tests for the closed-writer case (with and without socket_timeout) and confirms unrelated TypeError values are not rewritten to ConnectionError.

Reviewed by Cursor Bugbot for commit d7dd61c. Bugbot is set up for automated code reviews on this repo. Configure here.

@lh0156
lh0156 marked this pull request as ready for review July 30, 2026 12:29

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 2132282bc4

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread redis/asyncio/connection.py Outdated
Comment on lines +1152 to +1153
except (AttributeError, TypeError) as error:
raise ConnectionError("Connection closed while writing") from error

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Preserve non-I/O TypeErrors from writes

When a caller passes a malformed packed iterable, such as a custom generator or list containing non-bytes, StreamWriter.writelines() raises TypeError before any writer-close race is involved; this broad catch now turns that caller/data error into ConnectionError("Connection closed while writing"), disconnects, and can let retry logic re-run a non-transient bad command instead of surfacing the original error. Please only translate the specific closed-writer failure and let unrelated TypeErrors propagate as before.

AGENTS.md reference: AGENTS.md:L120-L123

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in d7dd61c. The inner write wrapper now translates only AttributeError from a closed writer; caller/data TypeError values are allowed to propagate unchanged. Added a malformed packed-command regression test. The asyncio connection suite passes (67 passed, 1 skipped).

@vladvildanov

Copy link
Copy Markdown
Collaborator

Please make sure resolve Codex comments first

Comment thread redis/asyncio/connection.py Outdated
try:
writer.writelines(command)
await writer.drain()
except (AttributeError, TypeError) as error:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the AttributeError/TypeError catch is broader than the None-writer case: pack_command aside, send_packed_command takes a caller-supplied iterable, so a bad element (str in the list) raises TypeError from writelines and now surfaces as ConnectionError, which retry.call_with_retry treats as retryable. that turns a programming error into a silent reconnect loop. since you already null-check writer above, is the except clause still buying anything beyond the writer-set-to-None-mid-await race?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in d7dd61c. The broad TypeError catch was removed, so a caller-supplied invalid packed iterable is no longer reported as ConnectionError. The existing closed-writer coverage now models the AttributeError path explicitly.

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

Fix All in Cursor

Reviewed by Cursor Bugbot for commit d7dd61c. Configure here.

writer.writelines(command)
await writer.drain()
except AttributeError as error:
raise ConnectionError("Connection closed while writing") from error

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TypeError from transport close race not caught

High Severity

The PR's stated purpose is to convert writer-close TypeError failures into ConnectionError so the retry mechanism can handle them, but _send_packed_command only catches AttributeError, not TypeError. The transport-close-race TypeError (the core problem from issue #3546) passes through uncaught, hits the except BaseException handler in send_packed_command, and is re-raised as a raw TypeError. Since the retry mechanism's supported_errors only includes ConnectionError and TimeoutError, the TypeError is still not retryable — leaving the original bug unfixed.

Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit d7dd61c. Configure here.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d7dd61c5b1

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +1152 to +1153
except AttributeError as error:
raise ConnectionError("Connection closed while writing") from error

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Handle the actual closed-transport TypeError

For the reported Python 3.12 close race, the traceback in #3546 fails inside StreamWriter.writelines() as TypeError: 'NoneType' object is not callable, but this new handler only translates AttributeError; the actual failure still escapes as a raw TypeError and bypasses redis-py's ConnectionError retry path. Please classify that specific closed-transport TypeError without converting arbitrary caller/data TypeErrors.

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

self._write_ready() 'NoneType' object is not callable

3 participants