Skip to content

chore: bump v0.2.1 - #740

Merged
zhyncs merged 1 commit into
sgl-project:mainfrom
zhyncs:rel
Jul 26, 2024
Merged

zhyncs merged 1 commit into
sgl-project:mainfrom
zhyncs:rel

Conversation

@zhyncs

@zhyncs zhyncs commented Jul 26, 2024

Copy link
Copy Markdown
Contributor

Thank you for your contribution, we really appreciate it. The following instructions will help improve your pull request and make it easier to receive feedback. If there are any items you don't understand, don't worry. Just submit the pull request and ask the maintainers for help.

Motivation

Please explain the motivation behind this PR and the goal you aim to achieve with it.

Modification

Briefly describe the changes made in this PR.

Checklist

  1. Ensure pre-commit pre-commit run --all-files or other linting tools are used to fix potential lint issues.
  2. Confirm that modifications are covered by complete unit tests. If not, please add more unit tests for correctness.
  3. Modify documentation as needed, such as docstrings or example tutorials.

@zhyncs
zhyncs requested review from Ying1123 and merrymercy July 26, 2024 11:27
@zhyncs
zhyncs merged commit 1b77670 into sgl-project:main Jul 26, 2024
@zhyncs
zhyncs deleted the rel branch July 26, 2024 11:27
timethink pushed a commit to timethink/sglang that referenced this pull request Mar 9, 2025
amote-i pushed a commit to amote-i/sglang that referenced this pull request Dec 8, 2025
* Update pr-test-npu-debug.yml

* debug testcase

* debug testcase

* Update run_suite.py

* debug testcase
efschu added a commit to efschu/htsglang that referenced this pull request Aug 17, 2026
… exist, twice

Prior-art gate outcome: do NOT build steps (2) and (3). Periodic GDN
snapshots at chunk boundaries and resume-from-nearest-checkpoint are both
implemented, in two lineages, and verified at file:line.

Resume-from-nearest (step 3): _match_prefix_helper tracks the deepest node
on the matched path still carrying a state, gated on the checkpoint grid
(mamba_radix_cache.py:1513-1517); _match_post_processor truncates the KV
prefix to it (:1685), COWs the state into a fresh slot (:1611-1654), and
the scheduler re-prefills the tail (schedule_batch.py:2644-2660,
:2719-2733). Snapshot writer (step 2): retention at cache_unfinished_req
(:771, the chunked-prefill boundary) and cache_finished_req (:603).
Cadence flag: --mamba-checkpoint-interval (server_args.py:4106/:14048).
Host tier for mamba state: HiMambaRadixCache write_backup
(hi_mamba_radix_cache.py:373) / init_load_back (:553) into mamba_pool_host.

The real gap is an explicit refusal plus two switches that are off:
server_args.py:14066-14071 rejects --mamba-checkpoint-interval together
with --enable-hierarchical-cache, so today you get deterministic
grid-pinned checkpoints (VRAM-only) OR a host tier, never both. Live boot
has mamba_checkpoint_interval=None, enable_hierarchical_cache=False,
gdn_resident_state_slots=None, max_mamba_cache_size=12.

This also RETRACTS my own sgl-project#740 mechanism claim (NOTE_740 section 4a).
I said misses were "no state at that position". Resume-from-nearest
exists, so a full re-prefill needs no state ANYWHERE on the path. The
cause is EVICTION: 12 slots shared between running requests and every
retained checkpoint under LRU (mamba_radix_cache.py:1090). Same
conclusion as sgl-project#745 from the other end -- the states need somewhere to
live -- but the missing piece is tiering and survival, not a writer.

sgl-project#212 determination as asked: sgl-project#555's canonical blob does NOT obsolete it,
because the blocker was never the format. The encoded objection
(gdn_slot_ladder.py:26-30) is that a recurrent state cannot be TRUNCATED
to a shorter prefix. sgl-project#745 never truncates -- it anchors at exact
positions and replays the tail -- so it is the composition sgl-project#212 leaves
open, not a re-litigation of it.

Step (5) reconciled: FEATURE_CATALOG.md:514-521 (idle slots free an
INDEX, not memory) and sgl-project#364 (gdn_slot_ladder.py:10-15, pool boot-sized to
the cap, surplus bytes land in the KV pool) are both true in different
domains. Lowering the cap at BOOT returns real bytes via
handle_max_mamba_cache spending from total_rest_memory
(model_runner_kv_cache_mixin.py:1999, :2050-2068); vacating at runtime
returns an index. Framed throughout as raising the binding stage's token
ceiling, never as a KV share rising.

Idle-vacate x resume is already safe: vacate_plan takes resumed_ids and
makes restores unconditional before vacates (gdn_slot_ladder.py:169-215).
The obligation on the resume path is wiring, not machinery.

Recommended revised order is in section 7: measure an
--enable-hierarchical-cache arm FIRST, since it already tiers mamba state
to host; only lift the :14066 refusal if deterministic resume points are
also wanted.

Nothing boot-verified: source plus one read-only get_server_info probe.
Docs only.
efschu added a commit to efschu/htsglang that referenced this pull request Aug 18, 2026
…gated, resumable machine

scripts/merge_train_pass3.py executes LEDGER_merge_train_0818 (c)
mechanically, with the map's own disciplines enforced in code:

- GATE: refuses without the comp4-green marker
  (/spinning/evidence-665-f1/COMP4_ACCEPTED) -- pass 3 stays
  boot-proof-gated per the ledger's rule; the marker path is a CLI
  parameter so the refusal is testable.
- STANDING RULE first: re-derives the (a) lineage facts against the
  ACTUAL tip (contains review tip + composite, every step ref present,
  already-contained refs auto-skip) and aborts if the world moved.
- Mapped resolutions ONLY: 'ours' (the sgl-project#754 fixture hunk drop),
  'theirs' + load-bearing marker verification (the sgl-project#735 doc numbers
  32607/4577), 'manual' (resumable stop with instructions, the sgl-project#727
  gate hunks). An UNPREDICTED conflict aborts loudly WITHOUT resolving
  -- a stale map stops the train, it is never improvised around.
- Stop-on-first-failure with per-step targeted suites and a resumable
  JSON state file; an all-ours pick that becomes empty is skipped as
  the mapped outcome. Nothing in the script pushes (pinned by test).

SMOKED per the desk-written-never-executed rule, and the smoke EARNED
ITS KEEP: the --dry-run scratch-clone execution of the real plan
caught a ledger error -- d11b29d (sgl-project#740 s5a) is only a diff and
conflicts without the original note commit 0480f4b underneath; the
executor aborted on exactly that unpredicted conflict. Plan and ledger
corrected to the sgl-project#740 PAIR; the re-run completes all 10 steps clean in
the scratch. Bonus finding recorded in the ledger: the sgl-project#727 'manual'
gate-hunk predictions did NOT fire -- those picks apply cleanly on the
comp4 tip.

Unit suite (hermetic, synthetic mini-repos): 12 passed -- gate refusal
(helper + CLI end-to-end exit 2), unpredicted-conflict abort leaving
the conflict for the operator, ours/theirs resolutions, marker-loss
abort, manual resumable stop, plan-mirrors-ledger pins, state
roundtrip. ruff and codespell clean. The real pass was NOT run;
COMP4_ACCEPTED does not exist and no real branch moved.
efschu added a commit to efschu/htsglang that referenced this pull request Aug 18, 2026
…gl-project#754 retires into the 753 fold, 13-step plan re-smoked clean

The first map froze before today's second wave; new authority is
F4-r5's harvest composite 59ce2d8 (declared COMPLETE, review tip
still an ancestor). Pass 3 redefined: harvest tip = train base,
unabsorbed branches cherry-pick on top, feat/753 lands FIRST by its
owner (10 in-flight commits; carries sgl-project#749; folds sgl-project#754 at the same
seam -- distributed/utils.py:1709, its own pp_size=1 handling).

Sweep results, same git-cherry/merge-base rigor as the first map,
outputs quoted in the ledger REFRESH section:
- ABSORBED by ancestry: comp4 and its whole lineage, 915ce1b
  (F4-r5's own sgl-project#757), 57b04b2 (sgl-project#540 fix).
- ABSORBED as different commits (desk-sgl-project#752 hazard class, never merge):
  fix/748-armed-gate-scope, fix/759-arming-economy,
  feat/755-slot-reorder. The sgl-project#758 emitters need no branch -- the
  harvest TIP ITSELF is a sgl-project#758 commit.
- SUPERSEDED: my own fix/754 -- semantic-not-patch folded by 753
  (git cherry vs 082293f shows '+'); merging it after 753 lands
  guarantees a get_pp_layer_set conflict with zero gain. Retired from
  the plan without regret.
- REVIEW-never-merge: 9e56477 (independent sgl-project#757), per its
  reviewer's own in-composite note naming 915ce1b as the baseline.
- fix/706-remainder not yet visible; slot reserved.

Executor updated: HARVEST constant joins the lineage check,
DEFAULT_TIP moves to the harvest tip, the sgl-project#754 step is replaced by
the sgl-project#745 reachability pick, and the second-wave picks join (727
head-chain, sgl-project#738 verdict, sgl-project#535 tickets). Dry-run scratch smoke against
the REAL harvest tip: all 13 steps complete with ZERO conflicts
(exit 0) -- cleaner than the first wave; the sgl-project#740 pair ordering from
the previous smoke holds. Plan-mirror test updated (12 passed). Gate
unchanged: COMP4_ACCEPTED still required, nothing pushes, nothing
booted.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant