Skip to content

feat(kvbm-engine): integrate Mooncake Store as G4 object storage backend - #1

Merged
Dayuxiaoshui merged 1 commit into
yuechen-sys:mainfrom
Dayuxiaoshui:main
May 6, 2026
Merged

Dayuxiaoshui merged 1 commit into
yuechen-sys:mainfrom
Dayuxiaoshui:main

Conversation

@Dayuxiaoshui

Copy link
Copy Markdown
Collaborator

This commit adds Mooncake Store (distributed KV cache storage from Moonshot AI) as a G4 tier backend for Dynamo KVBM, enabling cross-node KV cache sharing.

Phase 1: TCP copy path

  • MooncakeObjectBlockClient implementing ObjectBlockOps trait
  • MooncakeObjectConfig with protocol selection (tcp/rdma)
  • MooncakeLockManager with optimistic locking (idempotent writes)

Phase 2: RDMA zero-copy + Batch API

  • with_zero_copy() builder and register_layout() for buffer registration
  • batch_put_from / batch_get_into for contiguous layouts (1 RPC batch)
  • Automatic fallback to copy path for non-contiguous layouts

Phase 3: KV Router G4 event linkage

  • OffloadEngineBuilder::with_event_sender()
  • ObjectTransferExecutor emits KvCacheEvent::Create on successful offload

Testing:

  • 3 integration tests pass (copy + zero-copy roundtrip)
  • cargo check with --features mooncake,testing clean

Overview:

Details:

Where should the reviewer start?

Related Issues: (use one of the action keywords Closes / Fixes / Resolves / Relates to)

  • closes GitHub issue: #xxx

This commit adds Mooncake Store (distributed KV cache storage from Moonshot AI)
as a G4 tier backend for Dynamo KVBM, enabling cross-node KV cache sharing.

Phase 1: TCP copy path
- MooncakeObjectBlockClient implementing ObjectBlockOps trait
- MooncakeObjectConfig with protocol selection (tcp/rdma)
- MooncakeLockManager with optimistic locking (idempotent writes)

Phase 2: RDMA zero-copy + Batch API
- with_zero_copy() builder and register_layout() for buffer registration
- batch_put_from / batch_get_into for contiguous layouts (1 RPC batch)
- Automatic fallback to copy path for non-contiguous layouts

Phase 3: KV Router G4 event linkage
- OffloadEngineBuilder::with_event_sender()
- ObjectTransferExecutor emits KvCacheEvent::Create on successful offload

Testing:
- 3 integration tests pass (copy + zero-copy roundtrip)
- cargo check with --features mooncake,testing clean
@Dayuxiaoshui
Dayuxiaoshui merged commit 7959654 into yuechen-sys:main May 6, 2026

@stmatengss stmatengss left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mooncake Store G4 Backend — Code Review

Solid integration overall. The phased approach (TCP copy → RDMA zero-copy → event linkage) is well-structured, and the trait-based pluggability via ObjectBlockOps is clean. Found several issues ranging from a data-loss bug to safety concerns in unsafe code.

Summary:

  • 1 bug (has_blocks drops non-existing keys from result)
  • 2 safety issues (null pointer in batch path, hostname portability)
  • 2 design concerns (out-of-tree path dep, error-swallowing lock manager)
  • Several minor suggestions

.into_iter()
.zip(formatted.into_iter())
.zip(exist_results.into_iter())
.filter_map(|((hash, key), exists)| if exists { Some((hash, key)) } else { None })

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🐛 Bug: has_blocks() drops non-existing keys from the result.

The filter_map here filters out non-existing keys, and keys was consumed by into_iter() above. The final collect() only returns entries for keys that exist in the store. Non-existing keys (which should appear as (hash, None)) are silently lost from the result.

This violates the trait contract — callers expect a result entry for every input key.

Fix: Preserve the non-existing keys in the result:

let mut results: Vec<(SequenceHash, Option<usize>)> = Vec::with_capacity(keys.len());
let mut size_queries = Vec::new();
for ((hash, key), exists) in keys.into_iter().zip(formatted).zip(exist_results) {
    if exists {
        size_queries.push((hash, key));
    } else {
        results.push((hash, None));
    }
}
// ... query sizes for size_queries, then extend results

.map(|(_, block_id)| {
layout.memory_region(*block_id, 0, 0)
.map(|r| r.addr as *mut std::ffi::c_void)
.unwrap_or(std::ptr::null_mut())

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Safety: null pointer passed to C library on memory_region failure.

.unwrap_or(std::ptr::null_mut())

If layout.memory_region() fails for any block, a null pointer is silently inserted into the buffers vec and passed to store.batch_put_from(). The Mooncake C library will likely segfault on this.

The same pattern appears in get_blocks_with_layout at the analogous batch path.

Suggestion: Collect into Result<Vec<_>> and fall through to the single-block path on error:

let buffers: Result<Vec<_>> = work_items.iter()
    .map(|(_, block_id)| {
        layout.memory_region(*block_id, 0, 0)
            .map(|r| r.addr as *mut std::ffi::c_void)
    })
    .collect();
let Ok(buffers) = buffers else {
    // fall through to single-block path
    ...
};

rayon = { version = "1", optional = true }
tokio-rayon = { version = "2", optional = true }
chrono = { version = "0.4", optional = true }
mooncake_store = { path = "../../../Mooncake/mooncake-store/rust", optional = true }

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Out-of-tree relative path dependency.

mooncake_store = { path = "../../../Mooncake/mooncake-store/rust", optional = true }

This assumes the Mooncake repo is checked out at a specific sibling directory relative to the workspace root (../../Mooncake/). This will break for any contributor who does not have that exact layout, and will fail in CI unless the CI script clones Mooncake to that exact path.

Consider:

  • Publishing mooncake_store to a private registry or crates.io
  • Using a git dependency: mooncake_store = { git = "https://github.com/...", optional = true }
  • At minimum, documenting the required checkout layout in the PR description or a README

}

fn default_hostname() -> String {
std::fs::read_to_string("/proc/sys/kernel/hostname")

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Non-portable default: /proc/sys/kernel/hostname does not exist on macOS/Windows.

The fallback to "localhost" will silently fire on non-Linux platforms, which may cause confusing RDMA addressing issues if someone develops or tests on macOS.

Suggestion: Use gethostname(2) via the libc crate (already a dependency) or the hostname crate for portable behavior. At minimum, add a comment noting this is Linux-only by design.

pub namespace: Option<String>,

/// Maximum concurrent requests.
#[serde(default = "default_max_concurrent")]

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Possible compilation error: missing default_max_concurrent function.

This field uses #[serde(default = "default_max_concurrent")] but I don't see a default_max_concurrent() function defined in this file. The other defaults (default_master_server_addr, default_hostname, default_mooncake_protocol, default_segment_size, default_buffer_size) are all defined below — but default_max_concurrent appears to be missing.

Is it defined in another module, or was it accidentally omitted from this PR?


/// Transport protocol: "tcp" or "rdma".
#[serde(default = "default_mooncake_protocol")]
pub protocol: String,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: protocol should be an enum, not a free-form String.

Currently accepts any string value, but only "tcp" and "rdma" are valid per the doc comment. A typo like "tcpp" would pass config validation and fail at runtime deep in the Mooncake C library.

#[derive(Debug, Clone, Serialize, Deserialize)]
#[serde(rename_all = "lowercase")]
pub enum MooncakeProtocol {
    Tcp,
    Rdma,
}

This gives compile-time + deserialization-time validation instead of a runtime C-library error.

Ok(exists) => Ok(exists),
Err(e) => {
tracing::warn!(key = %key, error = %e, "has_meta check failed");
Ok(false)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Design: has_meta swallows errors and returns Ok(false).

Returning Ok(false) on error means a transient network failure is indistinguishable from "meta doesn't exist." This could cause the system to re-upload blocks that already exist (wasted bandwidth) or re-acquire locks unnecessarily.

The optimistic-locking rationale (idempotent writes) makes duplicate uploads safe, so this isn't a correctness bug — but propagating the error would let callers make better decisions. At minimum, document this as an intentional design choice in the doc comment.

// SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
// SPDX-License-Identifier: Apache-2.0

#[cfg(feature = "mooncake")]

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: Redundant #[cfg(feature = "mooncake")] guards.

This module is already gated at the parent level in object/mod.rs:

#[cfg(feature = "mooncake")]
pub mod mooncake;

The inner #[cfg] guards here are redundant — the compiler will never reach this file without the feature enabled. Not harmful, but unnecessary noise. Consider removing them.

if is_contiguous && use_zero_copy && !work_items.is_empty() {
let keys_str: Vec<String> = work_items
.iter()
.map(|(hash, _)| match &namespace {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: Duplicated key formatting logic.

The format_key helper exists as a method on MooncakeObjectBlockClient (line ~72), but the closures inside put_blocks_with_layout, get_blocks_with_layout, and has_blocks all re-implement the namespace-prefixing logic inline:

match &namespace {
    Some(ns) => format!("{}/{}", ns, formatter.format_key(hash)),
    None => formatter.format_key(hash),
}

This pattern appears ~6 times. Consider extracting a standalone function that takes &namespace and &formatter to reduce repetition and risk of divergence.

// SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
// SPDX-License-Identifier: Apache-2.0

//! Integration test: verify Mooncake Store via raw API and MooncakeObjectBlockClient.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: This is an integration test disguised as an example.

This file lives in examples/ but is functionally a test (uses assert!, assert_eq!, prints pass/fail). It also requires a running Mooncake server to pass.

The #[ignore] tests in client.rs already cover the same scenarios (copy roundtrip, zero-copy roundtrip, has_blocks) more thoroughly with proper test framework support.

Consider either moving this to tests/mooncake_integration.rs or removing it in favor of the existing #[ignore] tests.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants