Skip to content

[Feat] diffusion/disaggregation: Add a mock transfer engine and XPU device support - #44381

Open
ckvermaAI wants to merge 1 commit into
sgl-project:mainfrom
ckvermaAI:edd-xpu
Open

ckvermaAI wants to merge 1 commit into
sgl-project:mainfrom
ckvermaAI:edd-xpu

Conversation

@ckvermaAI

@ckvermaAI ckvermaAI commented Oct 15, 2026 •

Copy link
Copy Markdown
Contributor

Motivation

Disaggregated diffusion moves tensors between the encoder, denoiser and decoder roles through a transfer engine, and the only engine was Mooncake. A host without Mooncake (for example Intel XPU, where Mooncake is not available) could not run disaggregation at all.

--disagg-transfer-backend already accepts auto, mock and mooncake (#26119), but create_transfer_engine never read the flag and no mock engine existed. Passing mock was accepted and then ignored.

_check_mooncake() also could not detect a missing install. It imported sglang's Mooncake wrapper module, which imports mooncake.engine lazily inside MooncakeTransferEngine, so the check passed whenever sglang itself imported.

Modifications

  • runtime/disaggregation/transport/engine.py
    • Add MockTransferEngine, a transfer engine that copies over TCP between registered host-memory buffers, with no RDMA. Each transfer is sent as a (dst_addr, length) header plus payload. The receiver checks that the write fits inside its registered buffer, writes it there, and sends back a 1-byte ack. A batch reuses one connection. deregister_buffer stops the accept loop and joins in-flight connection threads before clearing the buffer, so no write lands after teardown.
    • create_transfer_engine now takes backend:
      • mooncake behaves as before and raises if Mooncake is not installed.
      • mock returns the mock engine.
      • auto (the default) uses Mooncake when it imports; otherwise it logs a warning and uses the mock engine.
    • _check_mooncake imports mooncake.engine directly.
  • runtime/disaggregation/scheduler_mixin.py: pass disagg_transfer_backend to the factory. The buffer-placement logic is unchanged. The mock engine reports supports_gpu_direct=False, so its buffer stays in pinned CPU memory, and Mooncake still gets a device buffer.
  • runtime/server_args: accept xpu for --disagg-role-device, next to cuda. Like cuda, the value is parsed but not read anywhere today (its reader was removed in [diffusion] Remove unused helpers and write-only state #43064); this only keeps the choices consistent across platforms.
  • test/single_test_file/test_disagg_server.py: _require_gpus counts XPU devices when CUDA is not available, so the end-to-end test can run on XPU.
  • Add test/unit/test_disagg_mock_transfer_engine.py.

Behavior change: with the default auto, a host without Mooncake now starts on the mock engine with a warning instead of failing at startup. Hosts with Mooncake installed are unchanged. To keep failing at startup, pass --disagg-transfer-backend mooncake.

Accuracy Tests

Not applicable: no model forward or kernel changes. The unit tests check that the received bytes match the sent bytes exactly.

Speed Tests and Profiling

The Mooncake path is unchanged. The mock engine copies through host memory over TCP; it exists for correctness and for platforms without RDMA, not for speed.

Testing

On an Intel XPU host (torch 2.14 + XPU):

  • pytest python/sglang/multimodal_gen/test/unit/test_disagg_mock_transfer_engine.py: 5 passed. Covers:
    • bytes delivered at the right offsets
    • out-of-bounds write rejected
    • batch delivered over one connection
    • auto falls back to mock without Mooncake
    • auto prefers Mooncake when it imports
  • pytest python/sglang/multimodal_gen/test/unit/test_server_args.py -k disagg: 7 passed.
  • Manual checks:
    • A zero-length transfer succeeds.
    • A header split across two sends is reassembled correctly.
    • A peer that closes without acking makes the call return -1 instead of hanging.
  • End-to-end: test_disagg_server.py

Checklist

Review and Merge Process

  1. Ping Merge Oncalls to start the process. See the PR Merge Process.
  2. Get approvals from CODEOWNERS and other reviewers.
  3. Trigger CI tests with comments or contact authorized users to do so.
    • Common commands include /tag-and-rerun-ci, /tag-run-ci-label, /rerun-failed-ci
  4. After green CI and required approvals, ask Merge Oncalls or people with Write permission to merge the PR.

CI States

Latest PR Test (Base): ❌ Run #39381754489
Latest PR Test (Extra): ❌ Run #39381754451
Latest PR Test (AMD ROCm 10): ❌ Run #39381754776

…regation

Disaggregated diffusion required mooncake, which is not available on
XPU. --disagg-transfer-backend already accepted "mock", but nothing
implemented it and the factory never read the flag.

- transport/engine.py: add MockTransferEngine, a non-RDMA engine that
  copies between registered host buffers over TCP. A batch reuses one
  connection, and deregister_buffer joins in-flight connection threads
  before clearing the buffer. create_transfer_engine now honors
  backend: "auto" uses mooncake when it imports and otherwise warns and
  falls back to mock. _check_mooncake imports mooncake.engine directly;
  the sglang wrapper imports mooncake lazily, so importing the wrapper
  always succeeded and auto could never fall back.
- scheduler_mixin.py: pass disagg_transfer_backend to the factory. The
  mock engine reports no GPUDirect, so its buffer stays in pinned CPU
  memory.
- server_args: add "xpu" to the --disagg-role-device choices, next to
  "cuda".
- test_disagg_server.py: count XPU devices in _require_gpus when CUDA
  is not available.
- Add unit tests for mock round-trip, out-of-bounds rejection, batch
  delivery, and auto-backend selection.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@ckvermaAI ckvermaAI changed the title [diffusion] Add mock transfer engine and XPU device choice for disaggregation [Feat] diffusion/disaggregation: Add a mock transfer engine and XPU device support Oct 20, 2026
@ckvermaAI
ckvermaAI marked this pull request as ready for review October 20, 2026 07:01

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

diffusion SGLang Diffusion

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant