Skip to content

[Docs] Document multi-model KV cache isolation - #3082

Merged
ykwd merged 4 commits into
kvcache-ai:mainfrom
Lin-z-w:docs/model-aware-kv-isolation
Jul 24, 2026
Merged

ykwd merged 4 commits into
kvcache-ai:mainfrom
Lin-z-w:docs/model-aware-kv-isolation

Conversation

@Lin-z-w

@Lin-z-w Lin-z-w commented Jul 23, 2026

Copy link
Copy Markdown
Collaborator

Description

Add concise multi-model KV cache isolation guidance to the Mooncake Store Deployment & Tuning Guide.

Related upstream change: sgl-project/sglang#31920.

The new section explains:

  • how SGLang deployments should configure a consistent --served-model-name;
  • how vLLM derives the model identifier automatically and when to use cache_prefix; and
  • how optional multi-tenant mode adds a tenant_id-scoped object namespace and per-tenant quota admission.

This helps operators prevent unintended KV cache sharing when multiple models or deployments use the same Mooncake Store.

Module

  • Transfer Engine (mooncake-transfer-engine)
  • Mooncake Store (mooncake-store)
  • Mooncake EP (mooncake-ep)
  • Mooncake PG (mooncake-pg)
  • Integration (mooncake-integration)
  • P2P Store (mooncake-p2p-store)
  • Python Wheel (mooncake-wheel)
  • Common (mooncake-common)
  • Mooncake RL (mooncake-rl)
  • CI/CD
  • Docs
  • Other

Type of Change

  • Bug fix
  • New feature
  • Refactor
  • Breaking change
  • Documentation update
  • Performance improvement
  • Other

How Has This Been Tested?

Test commands:

SKIP=mooncake-code-format pre-commit run --files docs/source/deployment/mooncake-store-deployment-guide.md
git diff --check

The updated page was also rendered with the existing local Sphinx environment and the generated HTML was inspected for the new headings, command examples, and internal link. The build completed successfully; the remaining warnings were limited to external intersphinx inventories being unavailable in the restricted network environment.

Test results:

  • Unit tests pass (not applicable to this documentation-only change)
  • Integration tests pass (not applicable to this documentation-only change)
  • Manual testing done as described above

Checklist

  • I have performed a self-review of my own code
  • I have formatted my code using ./scripts/code_format.sh (not applicable)
  • I have run pre-commit run --all-files and all hooks pass
  • I have updated the documentation
  • I have added tests to prove my changes are effective (not applicable)
  • For changes >500 LOC: I have filed an RFC issue (not applicable)

AI Assistance Disclosure

  • No AI tools were used
  • AI tools were used (specified below)

Codex helped verify the relevant SGLang and vLLM configuration behavior, draft and refine the documentation, and run the checks listed above. I reviewed and edited the generated doc before submission.

@gemini-code-assist

Copy link
Copy Markdown
Contributor

Caution

The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased.

@github-actions github-actions Bot added documentation Improvements or additions to documentation run-ci labels Jul 24, 2026
@Lin-z-w
Lin-z-w force-pushed the docs/model-aware-kv-isolation branch from e29af1f to ffed4be Compare July 24, 2026 07:16
@ykwd
ykwd merged commit 3e2ccbc into kvcache-ai:main Jul 24, 2026
18 checks passed
@Lin-z-w
Lin-z-w deleted the docs/model-aware-kv-isolation branch July 24, 2026 09:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation run-ci

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants