Skip to content

[Scheduler] Replace extend_range with extend_end derived from prefix_len - #42822

Merged
hnyls2002 merged 1 commit into
mainfrom
lsyin/extend-end
Oct 7, 2026
Merged

hnyls2002 merged 1 commit into
mainfrom
lsyin/extend-end

Conversation

@hnyls2002

@hnyls2002 hnyls2002 commented Oct 6, 2026 •

Copy link
Copy Markdown
Collaborator
  • Store only Req.extend_end; extend_len is extend_end - prefix_len; remove Range
  • Logprob length snapshot and ngram/engram read batch extend_lens / prefix_lens, which already carry mixed-decode and encoder adjustments; drop the encoder start shift
  • PD decode prealloc and dLLM no longer pass a separate start

CI States

Latest PR Test (Base): 🚫 Run #37700074645
Latest PR Test (Extra): 🚫 Run #37700074368
Latest PR Test (AMD ROCm 10): 🚫 Run #37700074593

@github-actions github-actions Bot added documentation Improvements or additions to documentation apple-silicon unified-radix-cache labels Oct 6, 2026
@hnyls2002 hnyls2002 added run-ci CI: run the baseline test suite on this PR bypass-fail-fast CI: a failing job no longer aborts its siblings (lint still gates) parallel-stages CI: stages dispatch together instead of waiting on each other run-ci-extra CI: also run the extra suite (requires run-ci) labels Oct 7, 2026
@hnyls2002

Copy link
Copy Markdown
Collaborator Author

/rerun-test registered/unit/managers/test_hisparse_unit.py

@github-actions

github-actions Bot commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

🚀 1-gpu-5090 (1 test): ✅ View workflow run

cd test/ && python3 registered/unit/managers/test_hisparse_unit.py

@hnyls2002
hnyls2002 force-pushed the lsyin/prefix-writer branch from 6a87e03 to 3b27d73 Compare October 7, 2026 22:55
@hnyls2002
hnyls2002 force-pushed the lsyin/prefix-writer branch from 3b27d73 to 14e6c42 Compare October 7, 2026 22:59
Base automatically changed from lsyin/prefix-writer to main October 7, 2026 23:03
@hnyls2002
hnyls2002 merged commit 1066dec into main Oct 7, 2026
76 of 185 checks passed
@hnyls2002
hnyls2002 deleted the lsyin/extend-end branch October 7, 2026 23:07
vincentzed pushed a commit to vincentzed/sglang that referenced this pull request Oct 8, 2026
main dropped Req.prefix_indices and Req.extend_range (sgl-project#42822, sgl-project#42824, sgl-project#42825)
and moved the cache match into match_kv_cache. The replay bookkeeping now
lives in match_kv_cache and uses req.prefix_len. The request stubs of two
scheduler unit tests get the new replay fields.
detain added a commit to detain/sglang that referenced this pull request Oct 10, 2026
…chunk_alignment)

Ported sgl-project#42767 (74bee49 + style 25b56c3, squashed). The PR was written
against the pre-sgl-project#42822/sgl-project#42825 scheduler API; ported len(req.prefix_indices) ->
req.prefix_len and the two tests from extend_range to (prefix_len, extend_end).

Gating verdict: NOT hip-gated -- CUDA-visible by design. The alignment only
activates when the KV pool exposes prefill_chunk_alignment > 1 (default 1 =
exact upstream behavior); QSATokenToKVPool is the only pool that does
(page_size), i.e. the compression-group invariant owned by stack entries
118/121/125. For prod QSA (page 128) this now prevents budget-truncated
continuation chunks from ending mid-page/mid-group; full prefill-adder
battery stays green (51P+48sub).
nvpohanh added a commit to nvjullin/sglang that referenced this pull request Oct 12, 2026
- Rust bindings: pyo3 0.29 renamed Python::allow_threads to detach.
- Serialized load-back: sgl-project#43206 dropped anchor_index_parts and the
  FULL-anchored SWA device binding, so SWA rows are allocated
  independently and the per-node rebase helper (and its test) is obsolete.
- Tests: sgl-project#42822 replaced Req.set_extend_range/extend_range and
  prefix_indices with prefix_len/extend_end, and MatchResult now carries
  device_prefix_len instead of device_indices; init_load_back returns the
  loaded length.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
nvjullin pushed a commit to nvjullin/sglang that referenced this pull request Oct 21, 2026
- Rust bindings: pyo3 0.29 renamed Python::allow_threads to detach.
- Serialized load-back: sgl-project#43206 dropped anchor_index_parts and the
  FULL-anchored SWA device binding, so SWA rows are allocated
  independently and the per-node rebase helper (and its test) is obsolete.
- Tests: sgl-project#42822 replaced Req.set_extend_range/extend_range and
  prefix_indices with prefix_len/extend_end, and MatchResult now carries
  device_prefix_len instead of device_indices; init_load_back returns the
  loaded length.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
nvjullin pushed a commit to nvjullin/sglang that referenced this pull request Oct 21, 2026
- Rust bindings: pyo3 0.29 renamed Python::allow_threads to detach.
- Serialized load-back: sgl-project#43206 dropped anchor_index_parts and the
  FULL-anchored SWA device binding, so SWA rows are allocated
  independently and the per-node rebase helper (and its test) is obsolete.
- Tests: sgl-project#42822 replaced Req.set_extend_range/extend_range and
  prefix_indices with prefix_len/extend_end, and MatchResult now carries
  device_prefix_len instead of device_indices; init_load_back returns the
  loaded length.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
nvjullin pushed a commit to nvjullin/sglang that referenced this pull request Oct 21, 2026
- Rust bindings: pyo3 0.29 renamed Python::allow_threads to detach.
- Serialized load-back: sgl-project#43206 dropped anchor_index_parts and the
  FULL-anchored SWA device binding, so SWA rows are allocated
  independently and the per-node rebase helper (and its test) is obsolete.
- Tests: sgl-project#42822 replaced Req.set_extend_range/extend_range and
  prefix_indices with prefix_len/extend_end, and MatchResult now carries
  device_prefix_len instead of device_indices; init_load_back returns the
  loaded length.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
nvjullin pushed a commit to nvjullin/sglang that referenced this pull request Oct 27, 2026
- Rust bindings: pyo3 0.29 renamed Python::allow_threads to detach.
- Serialized load-back: sgl-project#43206 dropped anchor_index_parts and the
  FULL-anchored SWA device binding, so SWA rows are allocated
  independently and the per-node rebase helper (and its test) is obsolete.
- Tests: sgl-project#42822 replaced Req.set_extend_range/extend_range and
  prefix_indices with prefix_len/extend_end, and MatchResult now carries
  device_prefix_len instead of device_indices; init_load_back returns the
  loaded length.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
nvjullin pushed a commit to nvjullin/sglang that referenced this pull request Oct 27, 2026
- Rust bindings: pyo3 0.29 renamed Python::allow_threads to detach.
- Serialized load-back: sgl-project#43206 dropped anchor_index_parts and the
  FULL-anchored SWA device binding, so SWA rows are allocated
  independently and the per-node rebase helper (and its test) is obsolete.
- Tests: sgl-project#42822 replaced Req.set_extend_range/extend_range and
  prefix_indices with prefix_len/extend_end, and MatchResult now carries
  device_prefix_len instead of device_indices; init_load_back returns the
  loaded length.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
nvjullin pushed a commit to nvjullin/sglang that referenced this pull request Oct 29, 2026
- Rust bindings: pyo3 0.29 renamed Python::allow_threads to detach.
- Serialized load-back: sgl-project#43206 dropped anchor_index_parts and the
  FULL-anchored SWA device binding, so SWA rows are allocated
  independently and the per-node rebase helper (and its test) is obsolete.
- Tests: sgl-project#42822 replaced Req.set_extend_range/extend_range and
  prefix_indices with prefix_len/extend_end, and MatchResult now carries
  device_prefix_len instead of device_indices; init_load_back returns the
  loaded length.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
nvjullin pushed a commit to nvjullin/sglang that referenced this pull request Oct 29, 2026
- Rust bindings: pyo3 0.29 renamed Python::allow_threads to detach.
- Serialized load-back: sgl-project#43206 dropped anchor_index_parts and the
  FULL-anchored SWA device binding, so SWA rows are allocated
  independently and the per-node rebase helper (and its test) is obsolete.
- Tests: sgl-project#42822 replaced Req.set_extend_range/extend_range and
  prefix_indices with prefix_len/extend_end, and MatchResult now carries
  device_prefix_len instead of device_indices; init_load_back returns the
  loaded length.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
nvjullin pushed a commit to nvjullin/sglang that referenced this pull request Nov 4, 2026
- Rust bindings: pyo3 0.29 renamed Python::allow_threads to detach.
- Serialized load-back: sgl-project#43206 dropped anchor_index_parts and the
  FULL-anchored SWA device binding, so SWA rows are allocated
  independently and the per-node rebase helper (and its test) is obsolete.
- Tests: sgl-project#42822 replaced Req.set_extend_range/extend_range and
  prefix_indices with prefix_len/extend_end, and MatchResult now carries
  device_prefix_len instead of device_indices; init_load_back returns the
  loaded length.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
nvjullin pushed a commit to nvjullin/sglang that referenced this pull request Nov 4, 2026
- Rust bindings: pyo3 0.29 renamed Python::allow_threads to detach.
- Serialized load-back: sgl-project#43206 dropped anchor_index_parts and the
  FULL-anchored SWA device binding, so SWA rows are allocated
  independently and the per-node rebase helper (and its test) is obsolete.
- Tests: sgl-project#42822 replaced Req.set_extend_range/extend_range and
  prefix_indices with prefix_len/extend_end, and MatchResult now carries
  device_prefix_len instead of device_indices; init_load_back returns the
  loaded length.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

apple-silicon bypass-fail-fast CI: a failing job no longer aborts its siblings (lint still gates) documentation Improvements or additions to documentation parallel-stages CI: stages dispatch together instead of waiting on each other run-ci CI: run the baseline test suite on this PR run-ci-extra CI: also run the extra suite (requires run-ci) unified-radix-cache

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant