Repository navigation
Conversation
ckvermaAI
requested review from
AgainstEntropy,
BBuf,
HaiShaw,
kevin-mii,
mickqian,
niehen6174,
ping1jing2 and
yichiche
as code owners
November 4, 2026 04:35
This was referenced Nov 6, 2026
Route the USPAttention ring branch on XPU to deep_symm.ring_attn when SGLANG_DIFFUSION_XPU_DEEPSYMM_RING_PRECISION is bf16, fp8, fp8_mixed or mxfp4 (default off keeps the torch ring). The variable is read once at import, so the default "off" path makes no torch.compiler.disable call and a bad value fails at startup. - fp8_mixed / mxfp4 run only for batch 1 in deep_symm, so B > 1 takes the bf16 ring (a Wan request with n=2 would otherwise raise on the first layer). - deep_symm's shm control block is node-local, so a ring group that spans hosts falls back to the torch ring (checked once per group, gathering hostnames over the SP group's gloo group, not a oneCCL object gather). - deep_symm keys its IPC arenas on the K/V shape and never evicts them. The first ring call of a request that needs a shape deep_symm does not hold frees the arenas that request has not used, before deep_symm allocates, so a new resolution, frame count or batch never sits next to the previous request's arenas (all ring ranks see the same shapes in the same order, so they clear together; the shapes are the resolved ones, e.g. from an I2V condition image). Arenas are also freed after the full-shape warmup probe (next to IPC_A2A.drop_staging) and in destroy_model_parallel, which also drops the id(ring_group)-keyed host check (a recreated group could otherwise reuse a stale id on some ranks only and hang the others) and does not let a failed peer handshake block group teardown. - A failed ring pass (e.g. a peer timeout) leaves a poisoned relay in deep_symm's cache, so the next ring call clears it first instead of every later request with that shape failing. - Layers with dropout keep the torch ring (deep_symm has none); the package is imported at the first eligible layer, so a missing deep-symm fails before the first ring collective; batch > 1 on the MX precisions logs once. The variable is documented in environment_variables.mdx. Tests drive USPAttention on two ranks (bit-equal to the direct DeepSymm call, torch ring when off) and cover B=2, cross-host, arena release on a shape change (never two shape sets held, no rebuild on a repeat), recovery after a failed pass, the worker marking each request before its forward, and teardown through the real destroy_model_parallel(), also when the deep_symm clear fails. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ckvermaAI
force-pushed
the
dev/ckvermaAI/diffusion-xpu-deepsymm-ring
branch
from
November 13, 2026 13:41
6697ed1 to
cae5308
Compare
ckvermaAI
requested review from
JustinTong0323,
sogalin,
wisclmy0611 and
zijiexia
as code owners
November 13, 2026 13:41
Contributor
Preview approval requiredThis pull request comes from a fork, so Mintlify won't build a preview until someone with write access approves commit |
5 tasks done
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
On Intel XPU, diffusion ring attention (
--ring-degree N) runs torch's templated ring over BF16 flash attention, which is slow. DeepSymm (deep_symm.ring_attn) has an IPC ring that moves K/V over the copy engine, overlaps the transfer with compute, and runs attention in bf16, fp8, fp8_mixed or mxfp4.Modifications
backends/xpu_deepsymm_ring.py; theUSPAttentionring branch routes to it whenSGLANG_DIFFUSION_XPU_DEEPSYMM_RING_PRECISIONisbf16/fp8/fp8_mixed/mxfp4(defaultoffkeeps current behaviour). The variable is read once at import, so the off path makes notorch.compiler.disablecall and a bad value fails at startup. Documented inenvironment_variables.mdx.fp8_mixed/mxfp4run for batch 1 only (larger batches use the bf16 ring and log once).destroy_model_parallel, which also drops the per-group host check and logs instead of blocking if a peer is gone.Accuracy Tests
2-rank ring vs an fp32 full-sequence reference (rel-L2): bf16 0.0026, fp8 0.054, fp8_mixed 0.172, mxfp4 0.288.
Wan2.2-T2V-A14B 1280x720x81 with
mxfp4on every block keeps the scene and motion but looks grainy;bf16is clean. Keeping blocks 0, 33, 34, 38 at bf16 gives a clean video at near-mxfp4 speed; that block list is a follow-up PR on top of this one and #48059.Tested on 4x Intel XPU (Xe3P), torch 2.16, sgl-kernel-xpu main: the targeted unit tests pass, and the full
multimodal_gen/test/unitdirectory shows no new failures againstmain(same 42 failures / 3 errors, from tests that need CUDA ortrimesh).Speed Tests and Profiling
Wan2.2-T2V-A14B 1280x720x81, 40 steps, 2 Intel XPU (Xe3P) cards at 2500 MHz, cache off.
mxfp4bf16mxfp4Checklist
🤖 Generated with Claude Code
CI States
Latest PR Test (Base): ❌ Run #43435883159
Latest PR Test (Extra): ❌ Run #43435883025
Latest PR Test (AMD ROCm 10): ❌ Run #43435883299
Tip
Base CI only covers the core cases, and its workload may not exercise your change. Use
/rerun-testto run the tests your change touches, or copy this prompt for an agent: