Repository navigation
Conversation
…regation Disaggregated diffusion required mooncake, which is not available on XPU. --disagg-transfer-backend already accepted "mock", but nothing implemented it and the factory never read the flag. - transport/engine.py: add MockTransferEngine, a non-RDMA engine that copies between registered host buffers over TCP. A batch reuses one connection, and deregister_buffer joins in-flight connection threads before clearing the buffer. create_transfer_engine now honors backend: "auto" uses mooncake when it imports and otherwise warns and falls back to mock. _check_mooncake imports mooncake.engine directly; the sglang wrapper imports mooncake lazily, so importing the wrapper always succeeded and auto could never fall back. - scheduler_mixin.py: pass disagg_transfer_backend to the factory. The mock engine reports no GPUDirect, so its buffer stays in pinned CPU memory. - server_args: add "xpu" to the --disagg-role-device choices, next to "cuda". - test_disagg_server.py: count XPU devices in _require_gpus when CUDA is not available. - Add unit tests for mock round-trip, out-of-bounds rejection, batch delivery, and auto-backend selection. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
ckvermaAI
marked this pull request as ready for review
October 20, 2026 07:01
ckvermaAI
requested review from
AgainstEntropy,
BBuf,
HaiShaw,
kevin-mii,
mickqian,
niehen6174,
ping1jing2 and
yichiche
as code owners
October 20, 2026 07:01
This was referenced Nov 9, 2026
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
Disaggregated diffusion moves tensors between the encoder, denoiser and decoder roles through a transfer engine, and the only engine was Mooncake. A host without Mooncake (for example Intel XPU, where Mooncake is not available) could not run disaggregation at all.
--disagg-transfer-backendalready acceptsauto,mockandmooncake(#26119), butcreate_transfer_enginenever read the flag and no mock engine existed. Passingmockwas accepted and then ignored._check_mooncake()also could not detect a missing install. It imported sglang's Mooncake wrapper module, which importsmooncake.enginelazily insideMooncakeTransferEngine, so the check passed whenever sglang itself imported.Modifications
runtime/disaggregation/transport/engine.pyMockTransferEngine, a transfer engine that copies over TCP between registered host-memory buffers, with no RDMA. Each transfer is sent as a(dst_addr, length)header plus payload. The receiver checks that the write fits inside its registered buffer, writes it there, and sends back a 1-byte ack. A batch reuses one connection.deregister_bufferstops the accept loop and joins in-flight connection threads before clearing the buffer, so no write lands after teardown.create_transfer_enginenow takesbackend:mooncakebehaves as before and raises if Mooncake is not installed.mockreturns the mock engine.auto(the default) uses Mooncake when it imports; otherwise it logs a warning and uses the mock engine._check_mooncakeimportsmooncake.enginedirectly.runtime/disaggregation/scheduler_mixin.py: passdisagg_transfer_backendto the factory. The buffer-placement logic is unchanged. The mock engine reportssupports_gpu_direct=False, so its buffer stays in pinned CPU memory, and Mooncake still gets a device buffer.runtime/server_args: acceptxpufor--disagg-role-device, next tocuda. Likecuda, the value is parsed but not read anywhere today (its reader was removed in [diffusion] Remove unused helpers and write-only state #43064); this only keeps the choices consistent across platforms.test/single_test_file/test_disagg_server.py:_require_gpuscounts XPU devices when CUDA is not available, so the end-to-end test can run on XPU.test/unit/test_disagg_mock_transfer_engine.py.Behavior change: with the default
auto, a host without Mooncake now starts on the mock engine with a warning instead of failing at startup. Hosts with Mooncake installed are unchanged. To keep failing at startup, pass--disagg-transfer-backend mooncake.Accuracy Tests
Not applicable: no model forward or kernel changes. The unit tests check that the received bytes match the sent bytes exactly.
Speed Tests and Profiling
The Mooncake path is unchanged. The mock engine copies through host memory over TCP; it exists for correctness and for platforms without RDMA, not for speed.
Testing
On an Intel XPU host (torch 2.14 + XPU):
pytest python/sglang/multimodal_gen/test/unit/test_disagg_mock_transfer_engine.py: 5 passed. Covers:autofalls back to mock without Mooncakeautoprefers Mooncake when it importspytest python/sglang/multimodal_gen/test/unit/test_server_args.py -k disagg: 7 passed.test_disagg_server.pyChecklist
Review and Merge Process
/tag-and-rerun-ci,/tag-run-ci-label,/rerun-failed-ciCI States
Latest PR Test (Base): ❌ Run #39381754489
Latest PR Test (Extra): ❌ Run #39381754451
Latest PR Test (AMD ROCm 10): ❌ Run #39381754776