Skip to content

[Test] [XPU] kv_canary: Make kernel tests device-agnostic - #46775

Open
dayanandav wants to merge 12 commits into
sgl-project:mainfrom
dayanandav:xpu-kv-canary-device-agnostic
Open

dayanandav wants to merge 12 commits into
sgl-project:mainfrom
dayanandav:xpu-kv-canary-device-agnostic

Conversation

@dayanandav

@dayanandav dayanandav commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

Motivation

The kv_canary kernel tests hard-code torch.device("cuda") and torch.cuda.synchronize(), so they fail on XPU with Torch not compiled with CUDA enabled. _fixtures.py also resolved the device at import time, which broke CPU-only runners that import _canary_helpers.

Modifications

  • Kernel tests and _fixtures / _differential helpers get the device from get_device() and sync via get_device_module().
  • _fixtures.py resolves the device inside its helpers, not at import.
  • test_disabled_assert_inputs_passes_none_to_cuda skips off CUDA (CUDA JIT launch path only).
  • test/manual/kv_canary/test_xpu_kv_canary_pp.py: pass --pp-size 2 instead of --pp 2, which is ambiguous between --pp-size, --pp-max-micro-batch-size and --pp-async-batch-depth and made the server exit at argument parsing.
  • No CI registration changes.

On XPU the kernels run their byte-equal torch reference (kv_canary._dispatch). CUDA behavior is unchanged.

Accuracy Tests

N/A (test-only change).

Testing

  • Intel XPU: test/registered/kernels/ops/kv_canary/ 260 passed, 1 skipped
  • Intel XPU: test/registered/kv_canary/ + mock_model unit tests 193 passed
  • NVIDIA A100-80GB (CUDA 13.0, torch 2.14.1+cu130): test/registered/kernels/ops/kv_canary/ 261 passed, 0 skipped
  • NVIDIA A100-80GB: test/registered/kv_canary/ + mock_model unit tests 193 passed
  • Intel XPU (2 cards): test/manual/kv_canary/test_xpu_kv_canary_pp.py 1 passed
  • CPU-only (accelerators hidden): test_self_unit_canary_info.py 36 passed; fails on the previous import-time device lookup
  • Not run: AMD

Speed Tests and Profiling

N/A (test-only change).

🤖 Generated with Claude Code


CI States

Latest PR Test (Base): ❌ Run #43063072517
Latest PR Test (Extra): ❌ Run #43063071377
Latest PR Test (AMD ROCm 10): ❌ Run #43063072116

Tip

Base CI only covers the core cases, and its workload may not exercise your change. Use /rerun-test to run the tests your change touches, or copy this prompt for an agent:

Find the registered tests this PR affects and run them with /rerun-test, following https://github.com/sgl-project/sglang/blob/main/.agents/skills/rerun-test/SKILL.md

… on XPU

Use get_device()/get_device_module() instead of hard-coded CUDA in the
kv_canary kernel tests and helpers, register them on the XPU suite, and
skip the CUDA-JIT-only launch check on non-CUDA devices.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions

This comment has been minimized.

@dayanandav

Copy link
Copy Markdown
Contributor Author

@siju-samuel help to review the change, thanks.

@dayanandav dayanandav changed the title [Intel][XPU] Make kv_canary kernel tests device-agnostic and run them on XPU [Test] [XPU] kv_canary: Make kernel tests device-agnostic and run them on XPU Oct 27, 2026
…changes

The main_package filter skips python/sglang/kernels/ and
python/sglang/test/kernels/, so a PR touching only the torch reference
kernels XPU runs (verify_ref.py, plan_ref.py, ...) or the shared kv_canary
test helpers never ran the kv_canary kernel tests on XPU.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
airMeng pushed a commit to airMeng/sglang that referenced this pull request Nov 3, 2026
@airMeng

airMeng commented Nov 4, 2026

Copy link
Copy Markdown
Collaborator

@dayanandav — consolidation CI feedback from GitHub Actions: https://github.com/sgl-project/sglang/actions/runs/41680983915/job/126090441319

2026-11-03T17:41:10.4196770Z Traceback (most recent call last):
2026-11-03T17:41:10.4207171Z   File "/home/runner/work/sglang/sglang/python/sglang/srt/utils/common.py", line 1122, in get_device
2026-11-03T17:41:10.4207847Z     return current_platform.get_device(device_id)
2026-11-03T17:41:10.4208251Z            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2026-11-03T17:41:10.4208883Z   File "/home/runner/work/sglang/sglang/python/sglang/srt/platforms/device_mixin.py", line 189, in get_device
2026-11-03T17:41:10.4209527Z     raise NotImplementedError
2026-11-03T17:41:10.4209845Z NotImplementedError
2026-11-03T17:41:10.4210013Z 
2026-11-03T17:41:10.4210224Z During handling of the above exception, another exception occurred:
2026-11-03T17:41:10.4210541Z 
2026-11-03T17:41:10.4210667Z Traceback (most recent call last):
2026-11-03T17:41:10.4211405Z   File "/home/runner/work/sglang/sglang/python/sglang/test/ci/fork_test_worker.py", line 67, in _run_file
2026-11-03T17:41:10.4212006Z     runpy.run_path(filename, run_name="__main__")
2026-11-03T17:41:10.4212407Z   File "<frozen runpy>", line 287, in run_path
2026-11-03T17:41:10.4212804Z   File "<frozen runpy>", line 98, in _run_module_code
2026-11-03T17:41:10.4213186Z   File "<frozen runpy>", line 88, in _run_code
2026-11-03T17:41:10.4213825Z   File "/home/runner/work/sglang/sglang/test/registered/kv_canary/test_self_unit_canary_info.py", line 58, in <module>
2026-11-03T17:41:10.4214512Z     from sglang.test.kernels.kv_canary._canary_helpers import (
2026-11-03T17:41:10.4215225Z   File "/home/runner/work/sglang/sglang/python/sglang/test/kernels/kv_canary/_canary_helpers.py", line 20, in <module>
2026-11-03T17:41:10.4215910Z     from sglang.test.kernels.kv_canary._fixtures import (
2026-11-03T17:41:10.4216569Z   File "/home/runner/work/sglang/sglang/python/sglang/test/kernels/kv_canary/_fixtures.py", line 16, in <module>
2026-11-03T17:41:10.4217182Z     _DEVICE = torch.device(get_device())
2026-11-03T17:41:10.4217499Z                            ^^^^^^^^^^^^
2026-11-03T17:41:10.4218073Z   File "/home/runner/work/sglang/sglang/python/sglang/srt/utils/common.py", line 1124, in get_device
2026-11-03T17:41:10.4218629Z     raise RuntimeError(
2026-11-03T17:41:10.4219411Z RuntimeError: No accelerator (CUDA, XPU, HPU, NPU, MUSA, MPS) or platform plugin is available.

dayanandav and others added 7 commits November 11, 2026 00:14
- _fixtures.py resolved the device at import time, which raises on a
  CPU-only runner and broke test_self_unit_canary_info.py; resolve it
  inside the three helpers that need it.
- Import get_device / get_device_module from sglang.srt.utils, since
  common.py no longer defines them.
- Make the new test_canary_info_{hand,fuzz}.py device-agnostic and
  register them on the XPU suite.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
… lane

Keep them device-agnostic but leave XPU CI registration out to avoid
loading the XPU lane.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
The tests stay device-agnostic for manual runs; XPU CI registration is
left out of this PR.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@dayanandav dayanandav changed the title [Test] [XPU] kv_canary: Make kernel tests device-agnostic and run them on XPU [Test] [XPU] kv_canary: Make kernel tests device-agnostic Nov 11, 2026
@dayanandav

Copy link
Copy Markdown
Contributor Author

Fixed in 9683448. The device is now resolved inside the helpers and not at import, so the CPU-lane import works. Verified on CPU-only, XPU and A100.

--pp is ambiguous between --pp-size, --pp-max-micro-batch-size and
--pp-async-batch-depth, so the server exits with code 2 at argument parsing.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@dayanandav

Copy link
Copy Markdown
Contributor Author

@airMeng could you please run /tag-and-rerun-ci on this PR (or add the run-ci label)? The PR is now only the device-agnostic fix (no XPU CI changes), and the earlier CPU-lane import failure is fixed in the latest head. Thanks!

@airMeng

airMeng commented Nov 12, 2026

Copy link
Copy Markdown
Collaborator

/tag-and-rerun-ci

@github-actions github-actions Bot added the run-ci CI: run the baseline test suite on this PR label Nov 12, 2026
@airMeng

airMeng commented Nov 12, 2026

Copy link
Copy Markdown
Collaborator

For this week’s XPU consolidation (#49356, week of November 9), #46775 is excluded on scope grounds at source head 48d494ad405ca890a9392e79976666dd8d1fc279.

The remaining patch makes the shared KV-canary test helpers/device synchronization device-agnostic, retains the CUDA/AMD CI registrations, and corrects the existing manual XPU pipeline-parallel test argument. It does not add an XPU CI registration or change an XPU production operator. The consolidation job classified this residual as generic test-harness portability rather than XPU-specific production enablement or regression/registration coverage, so it remains for independent review in this source PR.

The earlier CPU-lane import failure is not the reason for this exclusion: the updated _fixtures.py resolves the device inside the helpers, and this week’s reassessment does not carry forward the old exact-head failure.

@dayanandav

Copy link
Copy Markdown
Contributor Author

@airMeng thanks for the scope note. Understood that this stays for independent review here. CI ran on head 48d494a: the kv_canary files all finished on AMD, and the remaining failures look unrelated (AMD job hit its 30-min limit in test_rope.py; CPU shard 6 fails only on test_prealloc_token_usage.py; the CUDA kernel jobs were cancelled by fail-fast and never ran). Could you comment /rerun-failed-ci so the CUDA jobs actually run? Thanks!

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

run-ci CI: run the baseline test suite on this PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants