Skip to content

Move global_server_args_dict - #642

Merged
hnyls2002 merged 1 commit into
mainfrom
move-global-args
Jul 17, 2024
Merged

hnyls2002 merged 1 commit into
mainfrom
move-global-args

Conversation

@hnyls2002

Copy link
Copy Markdown
Collaborator

No description provided.

@hnyls2002
hnyls2002 merged commit abd5385 into main Jul 17, 2024
@hnyls2002
hnyls2002 deleted the move-global-args branch July 17, 2024 20:49
timethink pushed a commit to timethink/sglang that referenced this pull request Mar 9, 2025
efschu pushed a commit to efschu/htsglang that referenced this pull request Aug 7, 2026
…ct#642)

The draft KV pool rides the main transfer as extra layers addressed by the
SAME index array as the target pool (prefill.py:186-195, decode.py:439-448,
"The indices are always shared with a target model"), and mooncake/conn.py
contains no occurrence of "draft" at all, so the transport could not treat
them differently even in principle. On a DCP decode arm those indices are
compact owner-rule rows (decode.py:1119-1125). Whether that is the right
address depends entirely on --draft-kv-layout: 'dcp' gives the draft pool
dcp_compact_pool_rows, the target's own coordinate system; 'replicated', the
default, keeps "the full global context per rank"
(model_runner_kv_cache_mixin.py:2819-2823). Under the default, token L's draft
KV is written at row compact(L), which in a full-context pool is where a
different token lives -- sgl-project#345's right-token/wrong-slot class, silent, error
growing with slot id.

Two guards. L11, a parse-time refusal naming the combination, placed with the
sgl-project#636 gate because it also reads the resolved dcp_size. L10, draft_kv_layout
added to DraftKvCanonicalLayout's compared fields, since the two arms must
agree: disagreeing arms differ in both row count and per-row byte length while
sharing one index array.

L10 lives in the layout contract rather than in compute_model_identity_hash,
and the reason is preserved in the code and pinned by a test: that hash is
taken with include_parallel_vectors=False precisely so a PP prefill and a
TP+DCP decode on identical weights still pair (#631a guard 1).
draft_kv_layout is a parallelism decision, so putting it there would
reintroduce exactly the false refusal that flag exists to prevent.

Why the hazard is not reachable today, stated precisely because a latent guard
that becomes dead code is its own defect: #631a REFUSES a PD arm that asks for
speculation, and its SGLANG_PD_AUTO_DISABLE_SPEC escape hatch disables it
instead. Neither branch lets a draft worker exist, so the registration never
runs. The accurate reason is "no branch permits it", not "the refusal nulls
it" -- the branches could diverge, and someone hardening only the escape hatch
would not realise they were also holding a corruption path shut. This gate is
what makes lifting #631a safe, and it is in the tree BEFORE that lift rather
than bundled into it.

Tests: 12 hermetic + 2 subtests, green. HazardTest is the point of the file
and is guard-INDEPENDENT: it models the two code sites' own arithmetic and
demonstrates the corruption executably -- rank 0 of a 3-way split over a
12-token context owns slots 0/3/6/9, and writing them at compact rows into a
full-context pool puts token 3 in token 1's row, token 6 in token 2's, token 9
in token 3's, while rows 6 and 9 stay empty. 3 of 4 owned tokens land at a
wrong address and nothing raises. A guard whose test only checks that the
guard raises proves nothing about the hazard, so the hazard is shown first and
stays green in both can-fail arms.

Can-fail: removing the L11 refusal turns both GuardTest refusal cases red;
dropping draft_kv_layout from the compared tuple turns the L10 disagreement
test red. Base-vs-branch on unit/disaggregation + unit/server_args with the
same command: 785 passed, 0 failed on branch, so no regression is possible.
ruff and codespell clean.
efschu pushed a commit to efschu/htsglang that referenced this pull request Aug 7, 2026
…-project#642)

The pin added with sgl-project#642 asserted that "draft_kv_layout" does not appear in
inspect.getsource(compute_model_identity_hash). That holds against the obvious
regression -- someone adding the field to that function -- and fails against
the likelier one. If the hash is later assembled from a tuple, constant or
dataclass defined elsewhere, draft_kv_layout can enter the weights identity
without ever appearing in the function's own text, and a text-only pin stays
green while the invariant is gone. That is the "reads as coverage" failure
this task has been removing, one level up, in the test meant to prevent it.

The pin is now the property itself: two server_args differing ONLY in
draft_kv_layout must hash the SAME, asserted both in the default call form and
in the include_parallel_vectors=False form the PD handshake actually uses.
That is what is really at stake -- a PP prefill and a TP+DCP decode on
identical weights must pair regardless of draft layout -- and it survives any
refactor of how the hash is assembled.

The source assertion is kept as a fast tripwire, renamed and documented as
such, because it names the field and fails legibly when someone edits that
function directly. It is explicitly no longer the only thing standing there.

Can-fail executed in the exact refactor shape the weakness describes: an
_EXTRA_IDENTITY_FIELDS constant defined ABOVE the function, appended to
identity_parts by a loop, so the field never appears in the function body.
Under it the behaviour pin FAILS and the source tripwire PASSES -- which both
proves the weakness was real and proves the new pin closes it. Restored after,
with hicache_storage.py unmodified in the diff.

Tests: 13 hermetic + 2 subtests green; unit/disaggregation 203 passed, 0
failed. Test-only change. ruff and codespell clean.
efschu pushed a commit to efschu/htsglang that referenced this pull request Aug 7, 2026
…contract, path B verdict

Seven commits from the sgl-project#625 Route A line, all desk-proven, none needing cards.

#631a guard 1 (57e13eb): a PD pair whose arms hold different weights is
refused, on model_identity_hash only with include_parallel_vectors=False so a
PP prefill and a TP+DCP decode on the same weights still pair. Surfaced and
fixed a latent TypeError: revision and quantization were the only identity
parts not str()-coerced, harmless while every caller ran late with a resolved
ServerArgs, reachable from the registration path.

sgl-project#636 (4f486ab): the four preconditions of the DCP token-sharded handover
become ONE boot-time contract instead of four runtime deaths on first
transfer. Placed after _reject_unsupported_draft_kv_dcp because dcp_size is
not resolved until _handle_uneven_tp (server_args.py:10319, called at :5957,
after the PD hook at :5916) -- a gate at the hook reads the unresolved value
and passes silently on the exact production configuration it guards. The
ordering is pinned by a test, not by a comment.

#631b contract (214433e) and spec (ec3a440): versioned canonical
full-head draft-KV layout with a geometry gate whose bound is the rejected
alternative's own cost rather than a constant in MB, plus the L1-L11 lift
criterion for the #631a refusal.

#631b composition verdict (66da8b3): --draft-kv-layout and
disaggregation_mode were never considered together. Under the default
'replicated' layout the draft pool is sized to the full global context while
the decode arm addresses every registered buffer with compact owner-rule rows
-- two coordinate systems, silent wrong-slot drift. That makes 'dcp' the only
layout on which the existing shared-index transfer is correct at all, not
merely the cheaper one.

sgl-project#642 (66ca5ee, a79fc3c): guards for that composition, with a hazard
test that stays green when the guard is removed so the arms distinguish a
broken guard from a broken hazard, and an identity-hash invariant pinned by
behaviour after the source-text pin was shown to miss the likely refactor.

The #631a refusal stays in force: the wiring slice needs cards and a
half-wired draft path that boots and silently decodes without speculation is
the failure this line exists to remove.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant