kafka: catch shutdown errors in list_offsets and fetch#28358
Merged
Conversation
Following [1], timequeries may throw. I managed to trigger the throw in a test by injecting some short sleeps; it results in the Kafka request timing out after the timequery throws. This commit updates the Kafka layer timequery call to explicitly handle shutdown errors with an error code indicating we're no longer leader, which seems reasonable if we're shutting down. I left other errors as they were before, since it isn't clear what a good error code would be. 1. redpanda-data#28328
Following [1], make_reader() may throw if the underlying log is closed while processing the request (rather than asserting). As is, the throw in the handler causes the client request to time out. Instead, this updates the shutdown case to return an error code indicating that we are no longer leader (which seems reasonable if the partition is shutting down). 1. redpanda-data#28328
Contributor
There was a problem hiding this comment.
Pull Request Overview
This PR adds exception handling for shutdown scenarios in Kafka request handlers. When the storage layer throws exceptions due to partition shutdown, the handlers now catch these exceptions and return not_leader_for_partition error codes instead of propagating uncaught exceptions.
Key changes:
- Wraps potentially-throwing storage operations in
ss::coroutine::as_future()to catch exceptions - Checks for shutdown exceptions using
ssx::is_shutdown_exception() - Returns appropriate Kafka error codes for shutdown scenarios
Reviewed Changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
| src/v/kafka/server/handlers/list_offsets.cc | Adds exception handling around timequery() call to catch shutdown exceptions and return not_leader_for_partition error |
| src/v/kafka/server/handlers/fetch.cc | Adds exception handling around read_from_partition() call to catch shutdown exceptions and return not_leader_for_partition error |
| std::rethrow_exception(ex); | ||
| } | ||
|
|
||
| // Note that units can be both increased and decreassed here. Increases |
There was a problem hiding this comment.
Corrected spelling of 'decreassed' to 'decreased'.
Suggested change
| // Note that units can be both increased and decreassed here. Increases | |
| // Note that units can be both increased and decreased here. Increases |
rockwotj
approved these changes
Nov 5, 2025
Collaborator
Retry command for Build#75614please wait until all jobs are finished before running the slash command |
Collaborator
CI test resultstest results on build#75614
|
Collaborator
|
/backport v25.3.x |
Collaborator
|
/backport v25.2.x |
Collaborator
|
/backport v25.1.x |
Collaborator
|
/backport v24.3.x |
Collaborator
|
Branch name "v25.3.x" not found. |
Collaborator
|
Failed to create a backport PR to v25.1.x branch. I tried: |
Collaborator
|
Failed to create a backport PR to v24.3.x branch. I tried: |
This was referenced Nov 5, 2025
Collaborator
|
Failed to create a backport PR to v25.2.x branch. I tried: |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Catches a couple throws that are possible in the Kafka layer when reaching into the storage layer, following #28328.
In both cases, we'll return an error code indicating we're no longer leader, which seems reasonable if we've shut the partition down.
I considered instead implementing a more invasive change that made the replicated_partition return a result type, but opted for this to avoid a bunch of refactoring.
Backports Required
Release Notes