Skip to content

[Sglang-Diffusion-XPU] Add support for sage_attn - #45485

Open
ANSHUMAN87 wants to merge 3 commits into
sgl-project:mainfrom
ANSHUMAN87:jh-diffusion-sage-3
Open

ANSHUMAN87 wants to merge 3 commits into
sgl-project:mainfrom
ANSHUMAN87:jh-diffusion-sage-3

Conversation

@ANSHUMAN87

@ANSHUMAN87 ANSHUMAN87 commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

Current change add triton path for sage_attn. The kernels are borrowed from original repo .

JH...!


CI States

Latest PR Test (Base): ❌ Run #40933660935
Latest PR Test (Extra): ❌ Run #40933661090
Latest PR Test (AMD ROCm 10): ❌ Run #40933660978

@mintlify

mintlify Bot commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

Preview approval required

This pull request comes from a fork, so Mintlify won't build a preview until someone with write access approves commit 63fb179.

Approve preview

@github-actions github-actions Bot added documentation Improvements or additions to documentation diffusion SGLang Diffusion jit-kernel labels Oct 21, 2026
@ANSHUMAN87

Copy link
Copy Markdown
Contributor Author

Performance: --attention-backend sage_attn on Intel XPU

Setup: 1x Intel Arc Pro B60 (24 GB, Xe2), driver 1.15.39122, torch 2.13.0+xpu, triton-xpu 3.7.2, sglang-kernel-xpu 0.3.0, diffusers 0.38.0, bf16. Baseline is the existing XPU flash-attention backend (fa).

Kernel: one attention call vs fa

Median of 5 interleaved rounds, XPU event timing, batch 1, bf16.

Shape (tokens x heads x head_dim) fa sage_attn Speedup
Wan2.1 480p self-attention, 32760 x 12 x 128 94.26 ms 81.29 ms 1.16x
Long sequence, 16384 x 24 x 64 27.14 ms 22.82 ms 1.19x
Z-Image 1024², 4224 x 30 x 128 4.30 ms 4.12 ms 1.04x
4096 x 24 x 128 3.21 ms 3.26 ms 0.98x
4429 x 24 x 64 1.78 ms 1.95 ms 0.91x
Text cross-attention, 32760 q x 512 kv x 12 x 128 1.88 ms 2.23 ms 0.85x
  • Where it pays off: gains grow with sequence length.
  • Where it loses: on short sequences and text cross-attention, the INT8 quantization prepass (about 0.5 ms per call at image sizes) costs more than the INT8 Q·Kᵀ saves.
  • Accuracy vs fp32 attention (all test shapes, fp16 and bf16): cosine >= 0.9999, relative L2 error ~1.3e-2, LSE max abs error < 0.025.

End to end: sglang generate, fa / sage / sage / fa

Run with --warmup-mode request and SGLANG_DIFFUSION_SYNC_STAGE_PROFILING=1. "Steady step" is the median of denoise steps 1..N; step 0 includes the DiT upload from the CPU offload that is enabled automatically on 24 GB.

Model Backend Steady denoise step Request total Peak memory Quality vs fa
Wan2.1-T2V-1.3B, 832x480, 81 frames, 10 steps fa 8772 / 8781 ms 107.86 / 107.96 s 22.26 GB reference (two runs bit-identical)
sage_attn 8035 / 8033 ms (-8.5%) 100.50 / 100.50 s (-6.9%) 22.26 GB PSNR 32.40 dB mean / 30.24 dB worst frame; SSIM 0.931 / 0.923
torch_sdpa 10460 ms 124.76 s 22.26 GB PSNR 35.73 / 34.58 dB; SSIM 0.951 / 0.947
Z-Image-Turbo, 1024x1024, 9 steps fa 878.6 / 878.4 ms 17.31 / 17.30 s 12.38 GB reference (bit-identical)
sage_attn 875.4 / 874.3 ms (-0.4%) 17.27 / 17.27 s (-0.2%) 12.38 GB PSNR 20.74 dB, SSIM 0.823
torch_sdpa 903.9 ms 17.53 s 12.38 GB PSNR 23.58 dB, SSIM 0.880
  • Why the gains differ: with fa, attention is 66% of a Wan denoise step (5.77 s of 8.79 s) but only 15% of a Z-Image step (133 ms of 875 ms); GEMMs (58%) and element-wise ops dominate Z-Image. The profiler window covers 7 steps, normalized per step.
  • Wan quality passes the repo's video thresholds (SSIM 0.92 / PSNR 24 dB).
  • Z-Image quality: the 8-step distilled model amplifies any attention rounding change; even exact torch_sdpa differs from fa by a similar amount. The global flag also puts the Qwen3 text encoder on Sage. For image models, --component-attention-backends transformer=sage_attn keeps the text encoder on fa.
  • Recipes: Z-Image follows the cookbook's Intel BMG recipe (1 GPU, default flags). Wan2.1 is not a documented BMG recipe; it was run to measure a long, attention-heavy sequence, with 10 steps instead of the default 50 because per-step cost is unchanged.
Commands
export ZE_AFFINITY_MASK=2 SGLANG_DIFFUSION_SYNC_STAGE_PROFILING=1
# BACKEND = fa | sage_attn | torch_sdpa
sglang generate --model-path Wan-AI/Wan2.1-T2V-1.3B-Diffusers \
  --prompt "A curious raccoon wearing a tiny scarf walks through a snowy pine forest at dusk, cinematic lighting" \
  --width 832 --height 480 --num-frames 81 --num-inference-steps 10 --seed 42 \
  --warmup-mode request --attention-backend $BACKEND --save-output --perf-dump-path wan_$BACKEND.json

sglang generate --model-path Tongyi-MAI/Z-Image-Turbo \
  --prompt "A photorealistic portrait of an old fisherman mending nets on a wooden pier at sunrise, detailed skin texture" \
  --width 1024 --height 1024 --seed 42 \
  --warmup-mode request --attention-backend $BACKEND --save-output --perf-dump-path zimage_$BACKEND.json


JH...!

@ANSHUMAN87

ANSHUMAN87 commented Oct 29, 2026 •

Copy link
Copy Markdown
Contributor Author

Performance post Driver and Triton Version upgrade(As it caused regression, had to change tile configs to chose between perf and accuracy): --attention-backend sage_attn on Intel XPU

Setup: 1x Intel Arc Pro B60 (24 GB, Xe2), driver 1.15.39122, torch 2.14.0+xpu, triton-xpu 3.8.0, sglang-kernel-xpu 0.4.0, diffusers 0.38.0, bf16, base 41eaeb0445. Baseline is the existing XPU flash-attention backend (fa).

Kernel: one attention call vs fa

Median of 5 interleaved rounds, XPU event timing, batch 1, bf16.

Shape (tokens x heads x head_dim) fa sage_attn Speedup
Long sequence, 16384 x 24 x 64 26.92 ms 21.24 ms 1.27x
Wan2.1 480p self-attention, 32760 x 12 x 128 95.67 ms 87.12 ms 1.10x
Z-Image 1024², 4224 x 30 x 128 4.47 ms 4.18 ms 1.07x
4096 x 24 x 128 3.42 ms 3.37 ms 1.01x
4429 x 24 x 64 1.77 ms 1.82 ms 0.97x
Text cross-attention, 32760 q x 512 kv x 12 x 128 1.99 ms 2.08 ms 0.96x
1024 x 24 x 64 0.11 ms 0.14 ms 0.79x
  • Where it pays off: gains grow with sequence length.
  • Where it loses: on short sequences and text cross-attention, the INT8 quantization prepass and host launch cost exceed what INT8 Q·Kᵀ saves.
  • Quantization: Q is quantized per 128 rows (64 for head_dim 64) and K per 64 rows, and each attention tile uses exactly one Q scale and one K scale. A per-row scale vector inside the loop made triton-xpu 3.8 route it through shared memory on every K/V iteration (3x slower). Same speed on triton-xpu 3.7.2.
  • Accuracy vs fp32 attention (all test shapes, fp16 and bf16): cosine >= 0.9999, relative L2 error ~1.3e-2, LSE max abs error < 0.025.

End to end: sglang generate, fa / sage / sage / fa

Run with --warmup-mode request and SGLANG_DIFFUSION_SYNC_STAGE_PROFILING=1. "Steady step" is the median of denoise steps 1..N.

Model Backend Steady denoise step Request total Peak memory Quality vs fa
Wan2.1-T2V-1.3B, 832x480, 81 frames, 10 steps fa 8845 / 8848 ms 100.84 / 100.89 s 7.44 GB reference (two runs bit-identical)
sage_attn 8248 / 8256 ms (-6.7%) 94.87 / 94.88 s (-5.9%) 7.44 GB PSNR 33.26 dB mean / 30.70 dB worst frame; SSIM 0.940 / 0.936
torch_sdpa 10420 ms 116.51 s 7.44 GB PSNR 33.69 / 31.02 dB; SSIM 0.943 / 0.939
Z-Image-Turbo, 1024x1024, 9 steps fa 875.9 / 876.8 ms 9.37 / 9.38 s 12.38 GB reference (bit-identical)
sage_attn 872.8 / 873.3 ms (-0.4%) 9.34 / 9.36 s (-0.3%) 12.38 GB PSNR 20.66 dB, SSIM 0.824
torch_sdpa 897.8 ms 9.57 s 12.38 GB PSNR 23.15 dB, SSIM 0.873
  • Why the gains differ: attention is about two thirds of a Wan denoise step but only about 15% of a Z-Image step. The Z-Image change is below the 1.5% end-to-end bar, so this PR claims no speedup for 1024² images.
  • Wan quality passes the repo's video thresholds (SSIM 0.92 / PSNR 24 dB) and sits close to the exact torch_sdpa noise floor.
  • Z-Image quality: the 8-step distilled model amplifies any attention rounding change; even exact torch_sdpa differs from fa by a similar amount. For image models, --component-attention-backends transformer=sage_attn keeps the text encoder on fa.
  • Wan on 24 GB XPU needs --component-residency text_encoder=component-offload with any backend: the default warmup preload of its 21 GiB fp32 text encoder leaves too little memory for the Level Zero driver and the first denoise step fails with UR_RESULT_ERROR_OUT_OF_RESOURCES. This is unrelated to this PR; fa and torch_sdpa fail the same way.
  • Recipes: Z-Image follows the cookbook's Intel BMG recipe (1 GPU, default flags). Wan2.1 is not a documented BMG recipe; it was run to measure a long, attention-heavy sequence, with 10 steps instead of 50 because per-step cost is unchanged.
Commands
export ZE_AFFINITY_MASK=2 SGLANG_DIFFUSION_SYNC_STAGE_PROFILING=1
# BACKEND = fa | sage_attn | torch_sdpa
sglang generate --model-path Wan-AI/Wan2.1-T2V-1.3B-Diffusers \
  --prompt "A curious raccoon wearing a tiny scarf walks through a snowy pine forest at dusk, cinematic lighting" \
  --width 832 --height 480 --num-frames 81 --num-inference-steps 10 --seed 42 \
  --warmup-mode request --component-residency text_encoder=component-offload \
  --attention-backend $BACKEND --save-output --perf-dump-path wan_$BACKEND.json

sglang generate --model-path Tongyi-MAI/Z-Image-Turbo \
  --prompt "A photorealistic portrait of an old fisherman mending nets on a wooden pier at sunrise, detailed skin texture" \
  --width 1024 --height 1024 --seed 42 \
  --warmup-mode request --attention-backend $BACKEND --save-output --perf-dump-path zimage_$BACKEND.json

JH...!

@github-actions github-actions Bot added the dependencies Pull requests that update a dependency file label Oct 29, 2026
@ANSHUMAN87

Copy link
Copy Markdown
Contributor Author

Update: sage_attn on XPU now uses Intel ARK (auto-round-lib) when it's available

On XPU, sage_attn now runs Intel ARK's sagev1 kernel when auto-round-lib is installed. It's added to the XPU diffusion extra, pinned to 0.16.0. SGLang's Triton kernel still runs in these cases:

  • Calls that need the LSE (ring attention). ARK 0.16.0 returns one LSE per 16 query rows, so these calls log a one-time warning and use Triton. Upstream issue:
  • Head dims other than 64 and 128.
  • ARK missing or its XPU library failing to load. This logs a warning and uses Triton. It usually happens because the library needs <env>/lib on LD_LIBRARY_PATH.

Setup:

  • Hardware and software: Arc Pro B60 (one card), torch 2.14.0+xpu, triton-xpu 3.8.0, sglang-kernel-xpu 0.4.0, auto-round-lib 0.16.0.
  • Run settings: bf16, seed 42, --warmup-mode request.
  • Step time is the median of steps after the first.
  • For the "sage_attn (Triton)" rows, ARK was hidden so that importing it fails.

Wan2.1-T2V-1.3B, 832x480, 81 frames, 10 steps (--component-residency text_encoder=component-offload)

Backend Request Denoise step vs fa (step / request) PSNR / SSIM vs fa (mean, min frame)
fa 100.86 s 8847 ms n/a n/a
sage_attn (ARK) 85.13 s 7285 ms -17.7% / -15.6% 32.90 / 0.934 (min 31.32 / 0.928)
sage_attn (Triton) 94.87 s 8254 ms -6.7% / -5.9% 33.26 / 0.940 (min 30.70 / 0.936)

Z-Image-Turbo, 1024x1024

Backend Request Denoise step vs fa (step / request) PSNR / SSIM vs fa
fa 9.38 s 876.0 ms n/a n/a
sage_attn (ARK) 9.17 s 853.5 ms -2.6% / -2.2% 23.54 / 0.900
sage_attn (Triton) 9.34 s 872.7 ms -0.4% / -0.4% 20.66 / 0.824

For reference, torch_sdpa vs fa with the same settings gives 33.69 / 0.943 on Wan and 23.15 / 0.873 on Z-Image. Peak memory is the same for all backends: 7.44 GB for Wan and 12.38 GB for Z-Image.

Attention kernel alone (bf16; median of interleaved runs):

Shape (tokens x heads x head dim) fa Triton ARK
Wan self-attention 32760x12x128 95.7 ms 87.1 ms (1.10x) 67.9 ms (1.41x)
Z-Image 4224x30x128 4.47 ms 4.18 ms (1.07x) 3.49 ms (1.28x)
4096x24x128 3.41 ms 3.37 ms (1.01x) 2.74 ms (1.24x)
16384x24x64 26.9 ms 21.2 ms (1.27x) 21.6 ms (1.25x)
4429x24x64 1.77 ms 1.82 ms (0.97x) 1.94 ms (0.91x)
1024x24x64 0.11 ms 0.14 ms (0.79x) 0.23 ms (0.48x)
Cross-attention 32760 q x 512 kv x 12x128 1.94 ms 2.08 ms (0.96x) 1.93 ms (1.01x)

ARK and Triton were timed in separate processes, each against its own fa run, because importing ARK sets an IGC environment variable for the whole process. The fa timings in the two processes agreed within 0.1%, except cross-attention at about 2.5%. Speedups are against fa. ARK is slower than fa on short sequences, but neither model above hits that range.

Accuracy against an fp32 reference is the same for both kernels (relative error 1.30e-2 for ARK vs 1.32e-2 for Triton at the Z-Image shape).

Tests

  • test/registered/kernels/ops/diffusion/test_sage_attn_ark_parity.py (XPU, stage-b): ARK vs Triton through the backend, 13 cases covering bf16/fp16, D64/D128, GQA, causal, cross-attention, non-default scale, and strided or sliced inputs. The tolerance is 3e-2; the cases measure 2e-4 to 1.8e-2. The test also fails if both calls run the same kernel. It skips until the XPU CI image is rebuilt with the new diffusion extra.
  • python/sglang/multimodal_gen/test/unit/test_xpu_attention_backend.py: LSE calls never reach ARK, a library that fails to load falls back to Triton, and inputs are re-strided for ARK's stride check.

JH...!

@ANSHUMAN87
ANSHUMAN87 marked this pull request as ready for review November 2, 2026 17:30

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

@github-actions

github-actions Bot commented Nov 2, 2026

Copy link
Copy Markdown
Contributor

Required checks failed on 63fb179; this PR cannot be merged until they pass.

PR title (run)

PR title must use "[Type] scope: title" or "[Type] [Hardware] scope: title". Type must be one of: Feat, Fix, Refactor, Perf, Doc, Test, CI, Chore. Scope is required and may be any module or model name, with unrestricted letter case (letters, numbers, dots, underscores, slashes, or hyphens). Hardware is optional; use a single tag without spaces. Use uppercase hardware tags (AMD, NVIDIA, NPU, MI300X), except for exactly these four spellings: Intel, Apple, Huawei, Ascend. Reverts must use GitHub's default format: Revert "original title". Use one space between fields and after the colon, with a non-empty title and no leading or trailing whitespace. Examples: "[Fix] attention: Fix KV cache indexing", "[Feat] qwen3: Add model support", "[Perf] [AMD] flash-attn: Optimize decode performance".
Edit the title; the check reruns on its own.

@ANSHUMAN87

Copy link
Copy Markdown
Contributor Author

@mingfeima please help review, Thanks!

@kpjeeja kpjeeja left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The title and description need rework. The title only describes part of the PR, and the description leaves out ARK, a new dependency, a setup requirement, its limitations, and test evidence. It also doesn't back up the performance claim in the docs. correct the PR title and msg according to sglang required format.

)
return None
# ARK only prints when its XPU library fails to load, then every call raises
if ark.xpu_lib is None:

@kpjeeja kpjeeja Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

except ImportError:
...
ARK only prints when its XPU library fails to load, then every call raises
if ark.xpu_lib is None:

Only ImportError is caught, and ark.xpu_lib is read without a guard. An OSError from ARK's native library during import,an installed auto-round-lib version without xpu_lib or sagev1, raises inside SageAttentionImpl.init. Model load then crashes instead of falling back to Triton. Use getattr(ark, "xpu_lib", None) and getattr(ark, "sagev1", None), catch (ImportError, OSError, RuntimeError), log the error, and return None.

# auto-round-lib 0.16.0 returns one LSE per 16 query rows
logger.warning_once(
"Intel ARK SageAttention does not return a per-row LSE; calls that "
"need it (ring attention) run SGLang's Triton kernel instead"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if return_lse:
# auto-round-lib 0.16.0 returns one LSE per 16 query rows
logger.warning_once(
"Intel ARK SageAttention does not return a per-row LSE; calls that "
"need it (ring attention) run SGLang's Triton kernel instead"
)

Ring attention is a supported, expected setup (supports_ring_rotation returns True), so logging this at warning level looks like a fault in every ring deployment. Use info_once. Also reword the comment to say ARK's LSE layout can't be used for the ring merge, rather than the cryptic "one LSE per 16 query rows".

"Intel ARK SageAttention does not return a per-row LSE; calls that "
"need it (ring attention) run SGLang's Triton kernel instead"
)
elif head_dim in _ARK_HEAD_DIMS:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

      elif head_dim in _ARK_HEAD_DIMS:
          return ark_sagev1(
              _as_dense_nhd(query),

The ARK branch skips the input checks the Triton path runs in _validate (sage_attn_triton.py:353): fp16/bf16 only, matching Q/K/V shapes and dtypes, and causal requiring q_len == kv_len. A causal call where Q and KV lengths differ raises a clear ValueError on Triton, but could silently get a differently aligned mask on ARK. Run the same validation before dispatching to ARK, or send causal and q_len != kv_len to Triton. GQA does not need a guard, since the parity test already covers 8 query heads over 2 KV heads.

"need it (ring attention) run SGLang's Triton kernel instead"
)
elif head_dim in _ARK_HEAD_DIMS:
return ark_sagev1(

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

          return ark_sagev1(

If ARK loads but fails on a particular call (sequence length, tile or device limits), the whole denoise run fails even though Triton would work. Rather than wrapping every call in try/except, which hides real errors and adds hot-path cost, consider running a tiny probe call inside _load_ark_sagev1 so a broken ARK is caught once at load.

self.packed_trailing_padding = extra_impl_args.get(
"packed_trailing_padding", False
)
self._ark_sagev1 = _load_ark_sagev1() if current_platform.is_xpu() else None

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

self._ark_sagev1 = _load_ark_sagev1() if current_platform.is_xpu() else None

Installing the standard XPU diffusion extra now switches every non-ring sage_attn call to ARK, which produces slightly different numerics, and nothing lets users opt out. If ARK regresses on a model or driver, the only workarounds are uninstalling a pinned dependency or breaking LD_LIBRARY_PATH. Consider extending the existing SGLANG_USE_SAGE_ATTENTION_IMPL env var with triton (force the in-tree kernel) and ark (require ARK, and error if it can't load), checked before _load_ark_sagev1(). Keep auto-selection as the default.


)
if dtype not in (torch.float16, torch.bfloat16):
raise ValueError(f"SageAttention requires fp16/bf16 but got dtype={dtype}")
logger.info("Using Sage Attention (Triton) backend on Intel XPU.")

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

      logger.info("Using Sage Attention (Triton) backend on Intel XPU.")

This message is now wrong when ARK runs. Log "(ARK or Triton)", or rely on the kernel-specific message _load_ark_sagev1 already emits.

"auto_round_kernel": fake_ark,
"auto_round_kernel.version": fake_version,
}
with patch.dict(sys.modules, fake_modules):

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  def test_unloadable_ark_library_falls_back_to_triton(self):
      ...
      with patch.dict(sys.modules, fake_modules):
          self.assertIsNone(sage_attn._load_ark_sagev1.__wrapped__())

The test name promises a fallback to Triton, but the test only checks that the loader returns None. Several paths are untested:

  • ARK success: ark_sagev1 is called with is_causal, scale, tensor_layout="NHD" and dense strides, and Triton is not called.
  • Head dim 96 falls back to Triton.
  • The ImportError branch of the loader.
  • The current_platform.is_xpu() gate in init.

Add MagicMock tests that assert on call_args.kwargs and on whether Triton was called.

Comment thread python/pyproject_xpu.toml
checkpoint-engine = ["checkpoint-engine @ git+https://github.com/MoonshotAI/checkpoint-engine.git@f772d1858204fa82bdb0436fdfec137f5de2972c"]
diffusion = [
"addict==2.4.0",
"auto-round-lib==0.16.0",

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"auto-round-lib==0.16.0",

This puts a native-library wheel in the default XPU diffusion extra, but the code and docs treat ARK as optional. If the pinned wheel isn't published for a platform or Python version on the index the XPU build uses, every pip install .[diffusion] there fails. Check that the wheel resolves (pip install .[diffusion] --dry-run against the XPU index), and consider a platform marker or a separate optional extra.

pytestmark = pytest.mark.skipif(
ark_sagev1 is None, reason="Intel XPU with a loadable auto-round-lib required"
)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ark_sagev1 = sage_attn._load_ark_sagev1() if is_xpu() else None

pytestmark = pytest.mark.skipif(
ark_sagev1 is None, reason="Intel XPU with a loadable auto-round-lib required"
)

This test will likely skip on XPU CI and still show green:

  • pr-test-xpu.yml:175 installs . without the diffusion extra.
  • docker/xpu.Dockerfile never sets LD_LIBRARY_PATH to /opt/venv/lib, which the docs in this PR say ARK needs.

Together with the comment above, the new default kernel ships with no effective CI coverage. Install auto-round-lib and
set LD_LIBRARY_PATH for the XPU CI job. When running on XPU CI, fail instead of skipping if ARK can't load.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

dependencies Pull requests that update a dependency file diffusion SGLang Diffusion documentation Improvements or additions to documentation jit-kernel

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants