Skip to content

[Fix] speculative: Validate QSA draft window before loading weights - #49869

Open
1eonyg wants to merge 1 commit into
sgl-project:mainfrom
1eonyg:fix/qsa-draft-window-validation
Open

1eonyg wants to merge 1 commit into
sgl-project:mainfrom
1eonyg:fix/qsa-draft-window-validation

Conversation

@1eonyg

@1eonyg 1eonyg commented Nov 11, 2026 •

Copy link
Copy Markdown

Motivation

QSA currently rejects speculative verification windows larger than the indexer compression ratio at runtime, after model weights have been loaded.

This PR moves the validation to server argument resolution, providing an early and actionable error for unsupported configurations.

Partially addresses #44099.

Modifications

  • Add _validate_qsa_speculative_window() to validate the maximum effective draft-token count against the QSA compression ratio.
  • Run validation after automatic parameter selection and draft-token normalization.
  • Account for adaptive speculative decoding candidate maxima using the existing max_speculative_num_draft_tokens() helper.
  • Preserve non-QSA behavior.
  • Add registered CPU regression tests for boundary values, normalized windows, adaptive maxima, and non-QSA configurations.

Accuracy Tests

CPU-only validation; no model forward or attention kernel changes.

  • New unit tests: 8 passed.
  • Related CPU regression: 376 passed, 265 subtests passed.
  • Real argument-resolution integration scenarios: 6 passed.
  • Changed-line coverage: 100% (9/9 executable lines).
  • Applicable pre-commit hooks: passed.
  • git diff --check: passed.

No end-to-end GPU inference accuracy tests were performed.

Speed Tests and Profiling

Not applicable. This PR adds startup argument validation only and does not modify the inference execution path or attention kernels.

Scope and Related Work

Checklist

  • Code formatting checked with pre-commit.
  • CPU unit tests added and registered.
  • Followed SGLang code style guidance.
  • Documentation updated (not applicable).
  • Model accuracy and speed benchmarks (not applicable; validation-only change).

CI States

Latest PR Test (Base): ❌ Run #43038726361
Latest PR Test (Extra): ❌ Run #43038726023
Latest PR Test (AMD ROCm 10): ❌ Run #43038726258

Tip

Base CI only covers the core cases, and its workload may not exercise your change. Use /rerun-test to run the tests your change touches, or copy this prompt for an agent:

Find the registered tests this PR affects and run them with /rerun-test, following https://github.com/sgl-project/sglang/blob/main/.agents/skills/rerun-test/SKILL.md

Reject QSA verification windows above the current backend limit during argument resolution, after normalization and including adaptive maxima.

Add registered CPU regression coverage for boundary values, normalized windows, adaptive maxima, and non-QSA configurations.

Partially addresses sgl-project#44099.

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant