[v25.2.x] partition/timequery: Fix tiered storage timequery bug#28870
Closed
vbotbuildovich wants to merge 3 commits into
Closed
[v25.2.x] partition/timequery: Fix tiered storage timequery bug#28870vbotbuildovich wants to merge 3 commits into
vbotbuildovich wants to merge 3 commits into
Conversation
This change refactors timequery logic to fix a rare bug where a timequery would return no result despite an offset matching the timestamp existing in cloud storage and local storage. The bug is observed in the tiered storage model test when querying timestamps corresponding to offsets in the final, active segment of the log of a partition with tiered storage enabled and with aggressive cleanup settings. The partition gets into a state where the local log is truncated up to the high watermark, but the active segment remains. Querying for a timestamp within the bounds of the active segment meant that the timestamp was after the start timestamp for the local log, which comes from the base timestamp of the only segment, but the start offset of the log was after the max offset set by the timequery, which is the HWM and comes from the Kafka handler. This caused the timequery to return no result, incorrectly, without checking local or cloud storage. The refactor fixes the bug and hopefully cleans up the logic a bit. Regression test included. (cherry picked from commit 1f4ee6c)
(cherry picked from commit c0b2445)
It can happen that a timequery hits an empty local log and its max offset is before the min offset, which is clamped to the start of the log. This will fail translation. This change adds extra early returns to local and cloud timequeries to handle this special case pre-translation. (cherry picked from commit d6b9adf)
Contributor
|
Needed some manual fixup and found it easier to open a new PR: #28898 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Backport of PR #28642