[CORE-15045] Cloud Topics: Object path random prefix#29066
Merged
Conversation
oleiman
force-pushed
the
ct/core-15045/l0-obj-key-pfx
branch
2 times, most recently
from
2542b57 to
8f2f7ca
Compare
Contributor
There was a problem hiding this comment.
Pull request overview
This PR adds a random 3-digit numeric prefix (ranging from 0-999) to cloud topic object paths to improve object distribution in cloud storage. The prefix is inserted between the data directory and the epoch ID in the path structure.
Key Changes:
- Adds a
prefixfield toobject_idstruct with random generation on object creation - Updates object path format from
level_zero/data/{epoch}/{uuid}tolevel_zero/data/{prefix}/{epoch}/{uuid} - Implements parsing function
level_zero_path_to_prefix()to extract prefix from object paths
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
src/v/cloud_topics/types.h |
Adds prefix field to object_id struct, updates serde version, and implements random prefix generation |
src/v/cloud_topics/object_utils.h |
Declares new level_zero_path_to_prefix() function for parsing prefix from paths |
src/v/cloud_topics/object_utils.cc |
Updates path formatting logic and implements prefix parsing with validation |
src/v/cloud_topics/tests/object_utils_test.cc |
Updates existing test cases for new path format and adds comprehensive tests for prefix parsing |
src/v/cloud_topics/BUILD |
Adds dependency on random generators library |
Collaborator
Retry command for Build#78177please wait until all jobs are finished before running the slash command |
Collaborator
oleiman
force-pushed
the
ct/core-15045/l0-obj-key-pfx
branch
from
8f2f7ca to
0f9df41
Compare
oleiman
force-pushed
the
ct/core-15045/l0-obj-key-pfx
branch
from
0f9df41 to
8342a18
Compare
But don't add it to the path yet.
oleiman
force-pushed
the
ct/core-15045/l0-obj-key-pfx
branch
from
8342a18 to
99031bf
Compare
Signed-off-by: Oren Leiman <oren.leiman@redpanda.com>
- should encourage good partitioning on the backend - useful for sharding outstanding collection work in GC Signed-off-by: Oren Leiman <oren.leiman@redpanda.com>
oleiman
force-pushed
the
ct/core-15045/l0-obj-key-pfx
branch
from
99031bf to
38a733c
Compare
7 tasks
dotnwat
approved these changes
Dec 22, 2025
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds a random int in [0, 999] to
cloud_topics::object_id, inserting it as a 3-digit prefix to the epoch ID in object path names.Backports Required
Release Notes