Skip to content

[Feat] [Intel] diffusion: DeepSymm IPC ring attention on XPU - #48322

Open
ckvermaAI wants to merge 1 commit into
sgl-project:mainfrom
ckvermaAI:dev/ckvermaAI/diffusion-xpu-deepsymm-ring
Open

ckvermaAI wants to merge 1 commit into
sgl-project:mainfrom
ckvermaAI:dev/ckvermaAI/diffusion-xpu-deepsymm-ring

Conversation

@ckvermaAI

@ckvermaAI ckvermaAI commented Nov 4, 2026 •

Copy link
Copy Markdown
Contributor

Motivation

On Intel XPU, diffusion ring attention (--ring-degree N) runs torch's templated ring over BF16 flash attention, which is slow. DeepSymm (deep_symm.ring_attn) has an IPC ring that moves K/V over the copy engine, overlaps the transfer with compute, and runs attention in bf16, fp8, fp8_mixed or mxfp4.

Modifications

  • New backends/xpu_deepsymm_ring.py; the USPAttention ring branch routes to it when SGLANG_DIFFUSION_XPU_DEEPSYMM_RING_PRECISION is bf16 / fp8 / fp8_mixed / mxfp4 (default off keeps current behaviour). The variable is read once at import, so the off path makes no torch.compiler.disable call and a bad value fails at startup. Documented in environment_variables.mdx.
  • Eligibility: XPU, BF16, non-causal self-attention without dropout, head size 64/128, default softmax scale; MX/FP8 need MHA with head size 128; fp8_mixed / mxfp4 run for batch 1 only (larger batches use the bf16 ring and log once).
  • A ring group that spans hosts keeps the torch ring (deep_symm's control block is node-local); checked once per group over the SP group's gloo group.
  • IPC arenas: deep_symm keys them on the K/V shape and never evicts them. The first ring call of a request that needs a shape deep_symm does not hold frees the arenas that request has not used, before deep_symm allocates, so a new resolution, frame count or batch never sits next to the previous request's arenas (the shapes are the resolved ones, e.g. from an I2V condition image; all ring ranks decide identically). A failed ring pass clears before the next call, so a poisoned relay does not fail later requests. Arenas are also freed after the warmup probe and in destroy_model_parallel, which also drops the per-group host check and logs instead of blocking if a peer is gone.

Accuracy Tests

2-rank ring vs an fp32 full-sequence reference (rel-L2): bf16 0.0026, fp8 0.054, fp8_mixed 0.172, mxfp4 0.288.

Wan2.2-T2V-A14B 1280x720x81 with mxfp4 on every block keeps the scene and motion but looks grainy; bf16 is clean. Keeping blocks 0, 33, 34, 38 at bf16 gives a clean video at near-mxfp4 speed; that block list is a follow-up PR on top of this one and #48059.

Tested on 4x Intel XPU (Xe3P), torch 2.16, sgl-kernel-xpu main: the targeted unit tests pass, and the full multimodal_gen/test/unit directory shows no new failures against main (same 42 failures / 3 errors, from tests that need CUDA or trimesh).

Speed Tests and Profiling

Wan2.2-T2V-A14B 1280x720x81, 40 steps, 2 Intel XPU (Xe3P) cards at 2500 MHz, cache off.

Setup End to end
This PR, compiled, mxfp4 498.4 s
This PR, compiled, bf16 1755.8 s
With #48059 and #48060 (MXFP8 GEMMs), eager, torch ring 902.9 s
With #48059 and #48060 (MXFP8 GEMMs), eager, DeepSymm mxfp4 507.6 s

Checklist

🤖 Generated with Claude Code


CI States

Latest PR Test (Base): ❌ Run #43435883159
Latest PR Test (Extra): ❌ Run #43435883025
Latest PR Test (AMD ROCm 10): ❌ Run #43435883299

Tip

Base CI only covers the core cases, and its workload may not exercise your change. Use /rerun-test to run the tests your change touches, or copy this prompt for an agent:

Find the registered tests this PR affects and run them with /rerun-test, following https://github.com/sgl-project/sglang/blob/main/.agents/skills/rerun-test/SKILL.md

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

Route the USPAttention ring branch on XPU to deep_symm.ring_attn when
SGLANG_DIFFUSION_XPU_DEEPSYMM_RING_PRECISION is bf16, fp8, fp8_mixed or
mxfp4 (default off keeps the torch ring). The variable is read once at
import, so the default "off" path makes no torch.compiler.disable call and
a bad value fails at startup.

- fp8_mixed / mxfp4 run only for batch 1 in deep_symm, so B > 1 takes the
  bf16 ring (a Wan request with n=2 would otherwise raise on the first layer).
- deep_symm's shm control block is node-local, so a ring group that spans
  hosts falls back to the torch ring (checked once per group, gathering
  hostnames over the SP group's gloo group, not a oneCCL object gather).
- deep_symm keys its IPC arenas on the K/V shape and never evicts them. The
  first ring call of a request that needs a shape deep_symm does not hold
  frees the arenas that request has not used, before deep_symm allocates, so
  a new resolution, frame count or batch never sits next to the previous
  request's arenas (all ring ranks see the same shapes in the same order, so
  they clear together; the shapes are the resolved ones, e.g. from an I2V
  condition image). Arenas are also freed after the full-shape warmup probe
  (next to IPC_A2A.drop_staging) and in destroy_model_parallel, which also
  drops the id(ring_group)-keyed host check (a recreated group could
  otherwise reuse a stale id on some ranks only and hang the others) and
  does not let a failed peer handshake block group teardown.
- A failed ring pass (e.g. a peer timeout) leaves a poisoned relay in
  deep_symm's cache, so the next ring call clears it first instead of every
  later request with that shape failing.
- Layers with dropout keep the torch ring (deep_symm has none); the package
  is imported at the first eligible layer, so a missing deep-symm fails before
  the first ring collective; batch > 1 on the MX precisions logs once.

The variable is documented in environment_variables.mdx. Tests drive
USPAttention on two ranks (bit-equal to the direct DeepSymm call, torch ring
when off) and cover B=2, cross-host, arena release on a shape change (never
two shape sets held, no rebuild on a repeat), recovery after a failed pass, the worker marking each request before its forward,
and teardown through the real destroy_model_parallel(), also when the
deep_symm clear fails.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@ckvermaAI
ckvermaAI force-pushed the dev/ckvermaAI/diffusion-xpu-deepsymm-ring branch from 6697ed1 to cae5308 Compare November 13, 2026 13:41
@mintlify

mintlify Bot commented Nov 13, 2026

Copy link
Copy Markdown
Contributor

Preview approval required

This pull request comes from a fork, so Mintlify won't build a preview until someone with write access approves commit cae5308.

Approve preview

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

diffusion SGLang Diffusion documentation Improvements or additions to documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant