Repository navigation
chore: bump v0.2.1 - #740
Merged
Merged
Conversation
timethink
pushed a commit
to timethink/sglang
that referenced
this pull request
Mar 9, 2025
amote-i
pushed a commit
to amote-i/sglang
that referenced
this pull request
Dec 8, 2025
* Update pr-test-npu-debug.yml * debug testcase * debug testcase * Update run_suite.py * debug testcase
efschu
added a commit
to efschu/htsglang
that referenced
this pull request
Aug 17, 2026
… exist, twice Prior-art gate outcome: do NOT build steps (2) and (3). Periodic GDN snapshots at chunk boundaries and resume-from-nearest-checkpoint are both implemented, in two lineages, and verified at file:line. Resume-from-nearest (step 3): _match_prefix_helper tracks the deepest node on the matched path still carrying a state, gated on the checkpoint grid (mamba_radix_cache.py:1513-1517); _match_post_processor truncates the KV prefix to it (:1685), COWs the state into a fresh slot (:1611-1654), and the scheduler re-prefills the tail (schedule_batch.py:2644-2660, :2719-2733). Snapshot writer (step 2): retention at cache_unfinished_req (:771, the chunked-prefill boundary) and cache_finished_req (:603). Cadence flag: --mamba-checkpoint-interval (server_args.py:4106/:14048). Host tier for mamba state: HiMambaRadixCache write_backup (hi_mamba_radix_cache.py:373) / init_load_back (:553) into mamba_pool_host. The real gap is an explicit refusal plus two switches that are off: server_args.py:14066-14071 rejects --mamba-checkpoint-interval together with --enable-hierarchical-cache, so today you get deterministic grid-pinned checkpoints (VRAM-only) OR a host tier, never both. Live boot has mamba_checkpoint_interval=None, enable_hierarchical_cache=False, gdn_resident_state_slots=None, max_mamba_cache_size=12. This also RETRACTS my own sgl-project#740 mechanism claim (NOTE_740 section 4a). I said misses were "no state at that position". Resume-from-nearest exists, so a full re-prefill needs no state ANYWHERE on the path. The cause is EVICTION: 12 slots shared between running requests and every retained checkpoint under LRU (mamba_radix_cache.py:1090). Same conclusion as sgl-project#745 from the other end -- the states need somewhere to live -- but the missing piece is tiering and survival, not a writer. sgl-project#212 determination as asked: sgl-project#555's canonical blob does NOT obsolete it, because the blocker was never the format. The encoded objection (gdn_slot_ladder.py:26-30) is that a recurrent state cannot be TRUNCATED to a shorter prefix. sgl-project#745 never truncates -- it anchors at exact positions and replays the tail -- so it is the composition sgl-project#212 leaves open, not a re-litigation of it. Step (5) reconciled: FEATURE_CATALOG.md:514-521 (idle slots free an INDEX, not memory) and sgl-project#364 (gdn_slot_ladder.py:10-15, pool boot-sized to the cap, surplus bytes land in the KV pool) are both true in different domains. Lowering the cap at BOOT returns real bytes via handle_max_mamba_cache spending from total_rest_memory (model_runner_kv_cache_mixin.py:1999, :2050-2068); vacating at runtime returns an index. Framed throughout as raising the binding stage's token ceiling, never as a KV share rising. Idle-vacate x resume is already safe: vacate_plan takes resumed_ids and makes restores unconditional before vacates (gdn_slot_ladder.py:169-215). The obligation on the resume path is wiring, not machinery. Recommended revised order is in section 7: measure an --enable-hierarchical-cache arm FIRST, since it already tiers mamba state to host; only lift the :14066 refusal if deterministic resume points are also wanted. Nothing boot-verified: source plus one read-only get_server_info probe. Docs only.
efschu
added a commit
to efschu/htsglang
that referenced
this pull request
Aug 18, 2026
…gated, resumable machine scripts/merge_train_pass3.py executes LEDGER_merge_train_0818 (c) mechanically, with the map's own disciplines enforced in code: - GATE: refuses without the comp4-green marker (/spinning/evidence-665-f1/COMP4_ACCEPTED) -- pass 3 stays boot-proof-gated per the ledger's rule; the marker path is a CLI parameter so the refusal is testable. - STANDING RULE first: re-derives the (a) lineage facts against the ACTUAL tip (contains review tip + composite, every step ref present, already-contained refs auto-skip) and aborts if the world moved. - Mapped resolutions ONLY: 'ours' (the sgl-project#754 fixture hunk drop), 'theirs' + load-bearing marker verification (the sgl-project#735 doc numbers 32607/4577), 'manual' (resumable stop with instructions, the sgl-project#727 gate hunks). An UNPREDICTED conflict aborts loudly WITHOUT resolving -- a stale map stops the train, it is never improvised around. - Stop-on-first-failure with per-step targeted suites and a resumable JSON state file; an all-ours pick that becomes empty is skipped as the mapped outcome. Nothing in the script pushes (pinned by test). SMOKED per the desk-written-never-executed rule, and the smoke EARNED ITS KEEP: the --dry-run scratch-clone execution of the real plan caught a ledger error -- d11b29d (sgl-project#740 s5a) is only a diff and conflicts without the original note commit 0480f4b underneath; the executor aborted on exactly that unpredicted conflict. Plan and ledger corrected to the sgl-project#740 PAIR; the re-run completes all 10 steps clean in the scratch. Bonus finding recorded in the ledger: the sgl-project#727 'manual' gate-hunk predictions did NOT fire -- those picks apply cleanly on the comp4 tip. Unit suite (hermetic, synthetic mini-repos): 12 passed -- gate refusal (helper + CLI end-to-end exit 2), unpredicted-conflict abort leaving the conflict for the operator, ours/theirs resolutions, marker-loss abort, manual resumable stop, plan-mirrors-ledger pins, state roundtrip. ruff and codespell clean. The real pass was NOT run; COMP4_ACCEPTED does not exist and no real branch moved.
efschu
added a commit
to efschu/htsglang
that referenced
this pull request
Aug 18, 2026
…gl-project#754 retires into the 753 fold, 13-step plan re-smoked clean The first map froze before today's second wave; new authority is F4-r5's harvest composite 59ce2d8 (declared COMPLETE, review tip still an ancestor). Pass 3 redefined: harvest tip = train base, unabsorbed branches cherry-pick on top, feat/753 lands FIRST by its owner (10 in-flight commits; carries sgl-project#749; folds sgl-project#754 at the same seam -- distributed/utils.py:1709, its own pp_size=1 handling). Sweep results, same git-cherry/merge-base rigor as the first map, outputs quoted in the ledger REFRESH section: - ABSORBED by ancestry: comp4 and its whole lineage, 915ce1b (F4-r5's own sgl-project#757), 57b04b2 (sgl-project#540 fix). - ABSORBED as different commits (desk-sgl-project#752 hazard class, never merge): fix/748-armed-gate-scope, fix/759-arming-economy, feat/755-slot-reorder. The sgl-project#758 emitters need no branch -- the harvest TIP ITSELF is a sgl-project#758 commit. - SUPERSEDED: my own fix/754 -- semantic-not-patch folded by 753 (git cherry vs 082293f shows '+'); merging it after 753 lands guarantees a get_pp_layer_set conflict with zero gain. Retired from the plan without regret. - REVIEW-never-merge: 9e56477 (independent sgl-project#757), per its reviewer's own in-composite note naming 915ce1b as the baseline. - fix/706-remainder not yet visible; slot reserved. Executor updated: HARVEST constant joins the lineage check, DEFAULT_TIP moves to the harvest tip, the sgl-project#754 step is replaced by the sgl-project#745 reachability pick, and the second-wave picks join (727 head-chain, sgl-project#738 verdict, sgl-project#535 tickets). Dry-run scratch smoke against the REAL harvest tip: all 13 steps complete with ZERO conflicts (exit 0) -- cleaner than the first wave; the sgl-project#740 pair ordering from the previous smoke holds. Plan-mirror test updated (12 passed). Gate unchanged: COMP4_ACCEPTED still required, nothing pushes, nothing booted.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Thank you for your contribution, we really appreciate it. The following instructions will help improve your pull request and make it easier to receive feedback. If there are any items you don't understand, don't worry. Just submit the pull request and ask the maintainers for help.
Motivation
Please explain the motivation behind this PR and the goal you aim to achieve with it.
Modification
Briefly describe the changes made in this PR.
Checklist
pre-commit run --all-filesor other linting tools are used to fix potential lint issues.