Repository navigation
Move global_server_args_dict - #642
Merged
Merged
Conversation
timethink
pushed a commit
to timethink/sglang
that referenced
this pull request
Mar 9, 2025
efschu
pushed a commit
to efschu/htsglang
that referenced
this pull request
Aug 7, 2026
…ct#642) The draft KV pool rides the main transfer as extra layers addressed by the SAME index array as the target pool (prefill.py:186-195, decode.py:439-448, "The indices are always shared with a target model"), and mooncake/conn.py contains no occurrence of "draft" at all, so the transport could not treat them differently even in principle. On a DCP decode arm those indices are compact owner-rule rows (decode.py:1119-1125). Whether that is the right address depends entirely on --draft-kv-layout: 'dcp' gives the draft pool dcp_compact_pool_rows, the target's own coordinate system; 'replicated', the default, keeps "the full global context per rank" (model_runner_kv_cache_mixin.py:2819-2823). Under the default, token L's draft KV is written at row compact(L), which in a full-context pool is where a different token lives -- sgl-project#345's right-token/wrong-slot class, silent, error growing with slot id. Two guards. L11, a parse-time refusal naming the combination, placed with the sgl-project#636 gate because it also reads the resolved dcp_size. L10, draft_kv_layout added to DraftKvCanonicalLayout's compared fields, since the two arms must agree: disagreeing arms differ in both row count and per-row byte length while sharing one index array. L10 lives in the layout contract rather than in compute_model_identity_hash, and the reason is preserved in the code and pinned by a test: that hash is taken with include_parallel_vectors=False precisely so a PP prefill and a TP+DCP decode on identical weights still pair (#631a guard 1). draft_kv_layout is a parallelism decision, so putting it there would reintroduce exactly the false refusal that flag exists to prevent. Why the hazard is not reachable today, stated precisely because a latent guard that becomes dead code is its own defect: #631a REFUSES a PD arm that asks for speculation, and its SGLANG_PD_AUTO_DISABLE_SPEC escape hatch disables it instead. Neither branch lets a draft worker exist, so the registration never runs. The accurate reason is "no branch permits it", not "the refusal nulls it" -- the branches could diverge, and someone hardening only the escape hatch would not realise they were also holding a corruption path shut. This gate is what makes lifting #631a safe, and it is in the tree BEFORE that lift rather than bundled into it. Tests: 12 hermetic + 2 subtests, green. HazardTest is the point of the file and is guard-INDEPENDENT: it models the two code sites' own arithmetic and demonstrates the corruption executably -- rank 0 of a 3-way split over a 12-token context owns slots 0/3/6/9, and writing them at compact rows into a full-context pool puts token 3 in token 1's row, token 6 in token 2's, token 9 in token 3's, while rows 6 and 9 stay empty. 3 of 4 owned tokens land at a wrong address and nothing raises. A guard whose test only checks that the guard raises proves nothing about the hazard, so the hazard is shown first and stays green in both can-fail arms. Can-fail: removing the L11 refusal turns both GuardTest refusal cases red; dropping draft_kv_layout from the compared tuple turns the L10 disagreement test red. Base-vs-branch on unit/disaggregation + unit/server_args with the same command: 785 passed, 0 failed on branch, so no regression is possible. ruff and codespell clean.
efschu
pushed a commit
to efschu/htsglang
that referenced
this pull request
Aug 7, 2026
…-project#642) The pin added with sgl-project#642 asserted that "draft_kv_layout" does not appear in inspect.getsource(compute_model_identity_hash). That holds against the obvious regression -- someone adding the field to that function -- and fails against the likelier one. If the hash is later assembled from a tuple, constant or dataclass defined elsewhere, draft_kv_layout can enter the weights identity without ever appearing in the function's own text, and a text-only pin stays green while the invariant is gone. That is the "reads as coverage" failure this task has been removing, one level up, in the test meant to prevent it. The pin is now the property itself: two server_args differing ONLY in draft_kv_layout must hash the SAME, asserted both in the default call form and in the include_parallel_vectors=False form the PD handshake actually uses. That is what is really at stake -- a PP prefill and a TP+DCP decode on identical weights must pair regardless of draft layout -- and it survives any refactor of how the hash is assembled. The source assertion is kept as a fast tripwire, renamed and documented as such, because it names the field and fails legibly when someone edits that function directly. It is explicitly no longer the only thing standing there. Can-fail executed in the exact refactor shape the weakness describes: an _EXTRA_IDENTITY_FIELDS constant defined ABOVE the function, appended to identity_parts by a loop, so the field never appears in the function body. Under it the behaviour pin FAILS and the source tripwire PASSES -- which both proves the weakness was real and proves the new pin closes it. Restored after, with hicache_storage.py unmodified in the diff. Tests: 13 hermetic + 2 subtests green; unit/disaggregation 203 passed, 0 failed. Test-only change. ruff and codespell clean.
efschu
pushed a commit
to efschu/htsglang
that referenced
this pull request
Aug 7, 2026
…contract, path B verdict Seven commits from the sgl-project#625 Route A line, all desk-proven, none needing cards. #631a guard 1 (57e13eb): a PD pair whose arms hold different weights is refused, on model_identity_hash only with include_parallel_vectors=False so a PP prefill and a TP+DCP decode on the same weights still pair. Surfaced and fixed a latent TypeError: revision and quantization were the only identity parts not str()-coerced, harmless while every caller ran late with a resolved ServerArgs, reachable from the registration path. sgl-project#636 (4f486ab): the four preconditions of the DCP token-sharded handover become ONE boot-time contract instead of four runtime deaths on first transfer. Placed after _reject_unsupported_draft_kv_dcp because dcp_size is not resolved until _handle_uneven_tp (server_args.py:10319, called at :5957, after the PD hook at :5916) -- a gate at the hook reads the unresolved value and passes silently on the exact production configuration it guards. The ordering is pinned by a test, not by a comment. #631b contract (214433e) and spec (ec3a440): versioned canonical full-head draft-KV layout with a geometry gate whose bound is the rejected alternative's own cost rather than a constant in MB, plus the L1-L11 lift criterion for the #631a refusal. #631b composition verdict (66da8b3): --draft-kv-layout and disaggregation_mode were never considered together. Under the default 'replicated' layout the draft pool is sized to the full global context while the decode arm addresses every registered buffer with compact owner-rule rows -- two coordinate systems, silent wrong-slot drift. That makes 'dcp' the only layout on which the existing shared-index transfer is correct at all, not merely the cheaper one. sgl-project#642 (66ca5ee, a79fc3c): guards for that composition, with a hazard test that stays green when the guard is removed so the arms distinguish a broken guard from a broken hazard, and an identity-hash invariant pinned by behaviour after the source-text pin was shown to miss the likely refactor. The #631a refusal stays in force: the wiring slice needs cards and a half-wired draft path that boots and silently decodes without speculation is the failure this line exists to remove.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.