Skip to content

Revert "Make the server random by default" - #492

Merged
Ying1123 merged 1 commit into
mainfrom
revert-488-fix-random
May 31, 2024
Merged

Ying1123 merged 1 commit into
mainfrom
revert-488-fix-random

Conversation

@Ying1123

@Ying1123 Ying1123 commented May 31, 2024 •

Copy link
Copy Markdown
Contributor

Reverts #488.
We need the same seed for tensor parallelism

@Ying1123
Ying1123 merged commit 83525a1 into main May 31, 2024
@Ying1123
Ying1123 deleted the revert-488-fix-random branch May 31, 2024 19:00
timethink pushed a commit to timethink/sglang that referenced this pull request Mar 9, 2025
vschandramourya pushed a commit to vschandramourya/sglang that referenced this pull request Feb 3, 2026
* new workflow-oss monitor

* update name

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
efschu added a commit to efschu/htsglang that referenced this pull request Aug 3, 2026
…-vector residual

The axis was not merely unpriced before sgl-project#485 -- NOTE_475 SS4 'Residual
honesty' already attributed 14 of the 41 ms/1k the sgl-project#424 INT8 pair moved to
--rank-kv-ratio redistributing DCP token ownership 'and therefore prefill
attention work', naming it a skew source the family model does not represent.
Recorded as the axis's outstanding measured anchor, explicitly NOT claimed as
a result: the CORE-PACED endpoint redistributes mass rather than growing it,
so it cannot reproduce a residual of that size, and asserting the match would
be the fitted constant the bracket exists to avoid.
efschu added a commit to efschu/htsglang that referenced this pull request Aug 3, 2026
efschu added a commit to efschu/htsglang that referenced this pull request Aug 3, 2026
…n annotations (UP037)

_prefill_sharded_time and prefill_time_model's core/base_core parameters
quoted AttnCorePlan, which is defined earlier in the module (line 3611) and
needs no forward-reference string -- ruff --select=UP037 flagged all three
as new. The pre-existing UP037 hit at per_family_prefill_compute_times'
return type (predates this merge) is left as documented pre-existing rot.
efschu added a commit to efschu/htsglang that referenced this pull request Aug 3, 2026
…inst FEATURE_CATALOG.md

Adds docs/dev/AUDIT_500_mechanism_reach.md (239 catalog condition claims
checked against the code predicate that gates them: 39 WIDER, 63 NARROWER,
126 EXACT, 9 NOT-FOUND, 20 consolidated bug candidates in §5).

Corrects FEATURE_CATALOG.md §1 in place with file:line-cited predicates,
resolving real conflicts against the two intervening merges that also
touched §1 (sgl-project#492 attention/KV replication axis, sgl-project#496 merge-duty backlog):
- Uneven-DCP / Draft-KV-DCP bullet: replaced with the audit's precise
  predicates (dcp_size > 1 and get_tp_partition_ratios() is not None,
  distributed/utils.py:346; three entry paths; REPLICATED-KV geometry at
  utils.py:1048; MLA no-combine at dcp/comm.py:81) and the audit's S1-34
  NOT-FOUND finding that demotes "replicated is DEGRADED above TP>kv_heads"
  from a two-sided code rule to CLI-help prose only, surfacing the sgl-project#500-B3/
  sgl-project#108 open defect (--rank-kv-ratio route refused by the draft-KV-DCP gate).
  The sgl-project#492 "TWO DISTRIBUTION AXES" paragraph is kept unchanged immediately
  after it (additive, not contradicted by the audit).
- Matrix Doctrine (§2) joint-prefill paragraph: replaced the sgl-project#492
  correction's enumerator claim with the audit's more precise one
  (uneven_perf.py:4133-4148, placement.py:813, scheduler.py:5860, :6408,
  :5218) and folded the sgl-project#492 measured bracket/loose-ctx findings back in as
  "still the right empirical read on this rig, refined by sgl-project#500-B1" rather
  than dropping them.

Reconciles NOTE_485_joint_phase_vectors.md's sgl-project#492 in-place REFUTED markers
with the audit's own correction of the same passages (header blockquote,
"attention grid" table row, conclusion item 2): both corrections are kept,
in chronological order, with the audit framed as refining sgl-project#492 rather than
replacing it — no duplicate REFUTED markers, full correction history intact.

Verified: branch's own diff (3b7569f..1d64fe3) touches only the three
doc files, no *.py; all 17 FEATURE_CATALOG.md §§ present; the five additive
paragraphs from intervening merges (two-axis §1, matrix doctrine §2, sgl-project#491
DSpark §4, merge-duty §12, NVENC §13) all present; codespell clean on the
three touched files.
efschu added a commit to efschu/htsglang that referenced this pull request Aug 3, 2026
Adds docs/dev/AUDIT_506_unswept_axes.md (1231 lines): a targeted sweep of
four axes this fork's existing audits (sgl-project#500, sgl-project#492) never covered --
index/dtype overflow (native byte-stride, not Python index tensors),
state-changing HTTP endpoints (auth/CORS), persistent-cache key completeness
(the sgl-project#241 class generalised), and test honesty (the sgl-project#380 "cannot fail"
class).

Top PRIO findings: /hibernate takes an unvalidated hibernate_dir override
from the request body and accepts GET (A2-F1); the api-key middleware is
skipped entirely unless a key is configured AND a route carries ADMIN_FORCE,
so the fork's newest state-changing routes are open by default, plus
wildcard CORS with credentials defeats the loopback-only bind (A2-F2/F3); the
only numeric gate on the native GGUF MXFP4 kernels is loose enough that an
all-zeros or sign-flipped output passes (A4-F1); a GGUF MoE MMQ byte-stride
computed in 32-bit crosses at 256 experts on DSV4-Flash Q4_K (A1-1).

Pure docs, no code diff (git diff --stat d653405..HEAD -- '*.py' empty).
Clean merge, single new file, no conflict. codespell clean. Scrubbed for
personal email/PAT/private-IP leakage (grep for gmail/googlemail,
github_pat/ghp_, 192.168., 10.10.10., the user's private address) -- no
hits; the only IPs present are loopback (127.0.0.1), 0.0.0.0 bind examples,
and the well-known 169.254.169.254 cloud-metadata address used as an
attack-payload example, none of which are a leak.
efschu added a commit to efschu/htsglang that referenced this pull request Aug 3, 2026
…tall predicate; refute sgl-project#500-B1

sgl-project#500-B1 claimed the phase-prefill enumerator's kv-head grid was an artefact
and that the runtime always ran on a replicated-KV / continuous-token-vector
geometry. sgl-project#503 executed the check and REFUTES that claim: two independent
predicates govern this family, and sgl-project#500-B1 conflated them.
`attn_kv_replicated(tp_size, total_num_kv_heads)` is strictly `kv < tp`
(`distributed/utils.py:1081`) and is what `linear.py:1423` /
`model_config.py:1332` read to build the q/k/v shards -- so at
`kv_heads >= tp` (4 kv heads over 3 ranks on the reference rig) the k/v
PROJECTIONS head-shard and the enumerator's kv-head grid is correct.
`uneven_dcp_kv_replicated` (`dcp_size > 1 and get_tp_partition_ratios() is
not None`, `distributed/utils.py:346`, now consolidated in
plan_uneven_dcp_kv_replicated and called from placement.py) replicates the KV
POOL, not the projections. Pricing attention weights on the pool predicate
models a layout the sgl-project#105 ragged-kernel guard refuses at the first forward.
Fix: the planner now reports the token axis only where its own install
predicate (`uneven_dcp_kv_replicated`) actually holds -- a bare
`--rank-kv-ratio coupled` boot without a non-uniform base plan never reaches
`dcp_size > 1`, head-shards the KV cache, and installs no token vector, so
the axis is refused by name there instead of being priced as a lever the
boot cannot actuate. `planner/placement.py:813` is flagged as an open
(narrower) defect: it still uses one flag for both mechanisms and reports
k/v projection heads as replicated under uneven DCP where the runtime shards
them.

Corrects FEATURE_CATALOG.md §1/§2, NOTE_485_joint_phase_vectors.md (three
correction sites) and TICKET_485_int8_joint_arm.md to carry this refutation
alongside the sgl-project#492/sgl-project#500 correction history already there (append-only,
strikethrough + REFUTED markers, nothing deleted). Clean auto-merge -- no
conflict: the touched lines in FEATURE_CATALOG.md §1/§2 were untouched by
every merge since d653405 (the sgl-project#500-audit merge these lines came from),
so the 3-way merge applied sgl-project#503's diff directly. All 17 catalog sections and
the four previously-preserved additive paragraphs (two-axis §1, matrix
doctrine §2, MERGE-DUTY owner-sites + SITREP §12, NVENC §13) verified
present.

Tests (CUDA_VISIBLE_DEVICES=99, PYTHONPATH pinned to this worktree):
- test/registered/unit/planner/ (HTSGLANG_TEST_MODEL_DIR set): 2205 passed,
  1 skipped, 334 subtests -- above the >=2202 target.
- test_attn_replication_axis_492.py alone: 21 passed (up from 18 at the
  sgl-project#492 merge -- sgl-project#503 added 3 new methods to
  TestWhatTheRuntimeCanActuallyDoToday).
- test/registered/unit/distributed/ (distributed/utils.py is touched): 24
  failed, 2131 passed, 8 skipped, 703 subtests. Verified NOT a regression:
  every one of the 24 failing tests lives in a file sgl-project#503's diff never
  touches (test_bar1_host_cleanup.py, test_dcp_token_vector_collective.py,
  test_s12_log_analyse.py, test_uneven_tp_nccl_env.py, test_vmm_utils.py,
  test_dcp_context_ceiling.py), and the failing code paths themselves
  (model_runner_kv_cache_mixin.py's uneven_kv_derived_mode() call, the
  _PoolRowStub fixture, the MPS-warning assertion, LOCAL_RANK env lookups)
  already existed byte-identical in the pre-merge base d653405 --
  confirmed by diff --stat, not just a count match. Root causes are a mix of
  missing test-fixture attributes, missing LOCAL_RANK env var under
  hermetic CPU-only invocation, and historical-log numeric drift, none
  related to the attention/KV geometry this merge touches.
- ruff check --select=F401,F821,UP037: 2 findings (placement.py:166
  `"PlacementFlags"`, uneven_perf.py:5028 `"Dict[str, List[float]]"`), both
  confirmed pre-existing at d653405 and outside sgl-project#503's diff hunks.
- codespell clean on all 8 touched/added files.
efschu added a commit to efschu/htsglang that referenced this pull request Aug 16, 2026
…tor co-solved

The sgl-project#602 capacity solve withdrew the cut for the POOL objective. The question
actually asked -- more prefill on the 5090 -- is a time objective and was never
solved for. This adds it, and then corrects its own first framing.

Timing model, calibrated from the measured 49.2/154.8/116.4 ms at [28,20,16]
(1.757/7.740/7.275 ms per layer). Two objectives are reported side by side
because they disagree: SERIAL sum(layers_r * ms_r) and PIPELINED max over
stages. Backtest: the model reproduces the calibration point exactly, and the
hand-arithmetic anchor holds -- [42,12,10] gives 239.43 ms against 320.4, i.e.
1.338x, matching the 1.34x expected by hand.

Result table under the rank0 cap (x_serial / x_pipelined vs incumbent):
  [28,20,16] 1.000 / 1.000  incumbent
  [42,12,10] 1.338 / 1.667  the anchor
  [42,11,11] 1.341 / 1.818  DOMINATES the anchor on BOTH axes
  [42, 1,21] 1.367 / 1.013  serial-optimal and useless once pipelined
The serial optimum is a degenerate cut that collapses pipelined throughput, so
reporting only one objective would have recommended it. [42,11,11] is the
candidate to arm.

Rank0 cap recomputed rather than quoted: at 724.3 MiB/layer against a 31,800
MiB budget, 42 layers leave 1,379 MiB (corridor OK), 43 leave 655 MiB
(CORRIDOR VIOLATED), 44 overflow by 69 MiB. So DESIGN_691's "about 42" is
confirmed, and the binding constraint at 42 is the CORRIDOR, not raw weight
overflow -- 43 fits the card and still must be refused.

Co-solve correction (user, restating sgl-project#320/sgl-project#492): revision 1 priced a "pool
cost" per candidate while holding the KV token vector PINNED. That is
single-family optimization and sgl-project#485 forbids it. Layers moved to rank0 free
exactly their weight bytes on the rank they left, and the uneven-DCP /
rank-kv-ratio machinery relocates the displaced KV share onto those bytes.
Rank0's cap therefore bounds rank0's SHARE of the token split, not the world
pool.

The conservation is exact, not approximate, and the tests prove it rather than
assert it: total VRAM is fixed and the same 64 layers of weights exist wherever
they sit, so total free bytes are invariant; under DCP token-sharding one token
costs kv_per_token_per_layer * total_layers wherever it lands. World pool is
therefore identical across every feasible cut, and the only residual is
second-order -- seam/staging in both directions plus TP-phase redistribution --
which is ITEMIZED and whose sum equals the delta exactly (no fudge term).

Scope note owed to the earlier verdict: "29,19,16 is 6.3 percent worse" held
only under a pinned vector. Co-solved, that comparison does not stand as a
capacity argument.

Calibration limit stated in the model docstring and pinned by a test: one
measured cut gives one point per rank and cannot separate per-layer slope from
fixed per-stage cost. fixed_ms defaults to zero, the OPTIMISTIC end, so every
speedup here is an upper bound. A second measured cut per rank would pin it.

Tests: 15, red first. Planner regression 41 passed across the 702 and 602
suites. ruff clean. No deploy, no arm.
efschu added a commit to efschu/htsglang that referenced this pull request Aug 16, 2026
…tor co-solved

The sgl-project#602 capacity solve withdrew the cut for the POOL objective. The question
actually asked -- more prefill on the 5090 -- is a time objective and was never
solved for. This adds it, and then corrects its own first framing.

Timing model, calibrated from the measured 49.2/154.8/116.4 ms at [28,20,16]
(1.757/7.740/7.275 ms per layer). Two objectives are reported side by side
because they disagree: SERIAL sum(layers_r * ms_r) and PIPELINED max over
stages. Backtest: the model reproduces the calibration point exactly, and the
hand-arithmetic anchor holds -- [42,12,10] gives 239.43 ms against 320.4, i.e.
1.338x, matching the 1.34x expected by hand.

Result table under the rank0 cap (x_serial / x_pipelined vs incumbent):
  [28,20,16] 1.000 / 1.000  incumbent
  [42,12,10] 1.338 / 1.667  the anchor
  [42,11,11] 1.341 / 1.818  DOMINATES the anchor on BOTH axes
  [42, 1,21] 1.367 / 1.013  serial-optimal and useless once pipelined
The serial optimum is a degenerate cut that collapses pipelined throughput, so
reporting only one objective would have recommended it. [42,11,11] is the
candidate to arm.

Rank0 cap recomputed rather than quoted: at 724.3 MiB/layer against a 31,800
MiB budget, 42 layers leave 1,379 MiB (corridor OK), 43 leave 655 MiB
(CORRIDOR VIOLATED), 44 overflow by 69 MiB. So DESIGN_691's "about 42" is
confirmed, and the binding constraint at 42 is the CORRIDOR, not raw weight
overflow -- 43 fits the card and still must be refused.

Co-solve correction (user, restating sgl-project#320/sgl-project#492): revision 1 priced a "pool
cost" per candidate while holding the KV token vector PINNED. That is
single-family optimization and sgl-project#485 forbids it. Layers moved to rank0 free
exactly their weight bytes on the rank they left, and the uneven-DCP /
rank-kv-ratio machinery relocates the displaced KV share onto those bytes.
Rank0's cap therefore bounds rank0's SHARE of the token split, not the world
pool.

The conservation is exact, not approximate, and the tests prove it rather than
assert it: total VRAM is fixed and the same 64 layers of weights exist wherever
they sit, so total free bytes are invariant; under DCP token-sharding one token
costs kv_per_token_per_layer * total_layers wherever it lands. World pool is
therefore identical across every feasible cut, and the only residual is
second-order -- seam/staging in both directions plus TP-phase redistribution --
which is ITEMIZED and whose sum equals the delta exactly (no fudge term).

Scope note owed to the earlier verdict: "29,19,16 is 6.3 percent worse" held
only under a pinned vector. Co-solved, that comparison does not stand as a
capacity argument.

Calibration limit stated in the model docstring and pinned by a test: one
measured cut gives one point per rank and cannot separate per-layer slope from
fixed per-stage cost. fixed_ms defaults to zero, the OPTIMISTIC end, so every
speedup here is an upper bound. A second measured cut per rank would pin it.

Tests: 15, red first. Planner regression 41 passed across the 702 and 602
suites. ruff clean. No deploy, no arm.
efschu pushed a commit to efschu/htsglang that referenced this pull request Aug 17, 2026
…le, and what the phase matrix owes

Docs only. No code changed, nothing built or wired.

## sgl-project#702 decision table (docs/DECISION_702_cut_table.md)

Re-derived on the sgl-project#707 closed form with the calibrated fixture. Incumbent pool
reads 436,275 here against the 436,766 in older notes; all percentages use
436,275 so they are internally consistent.

  cut          attn      pool               speed    noise(14.1%)  gate
  [28,20,16]   (7,5,4)   436,275            1.0000   --            is the gate
  [30,18,16]   (7,5,4)   436,275  (+0.0%)   1.1111   BELOW IT      PASSES
  [33,15,16]   (8,4,4)   415,859  (-4.7%)   1.3299   above         FAILS

Switch cost 1575 ms from the incumbent either way (distance-independent, per
#704a); payback 15.75 s and 6.35 s respectively.

THE TRADE: the two candidates fail in OPPOSITE places, which is the decision.
[30,18,16] passes the stage-1 "pool >= incumbent" gate and cannot be validated
by measurement -- its 11.1% gain sits BELOW this rig's measured 14.1% A-vs-A
noise floor, so no boot here can tell it from the incumbent. [33,15,16] clears
the noise floor comfortably but needs a 4.7% capacity waiver. No ranking is
offered; the pick is the user's.

[30,18,16]'s pool is identical STRUCTURALLY, not coincidentally: it keeps the
(7,5,4) attention split and leaves PP2 (layers 48-63) untouched, and PP2 is the
binder. Per-rank capacity is [594,615/462,920/436,275] at the incumbent and
[517,464/570,931/436,275] here -- PP2's column is unchanged, so moving two
layers between PP0 and PP1 cannot move a pool PP2 sets.

CORRECTION: I could not reproduce a "+20% pool" figure for [30,18,16]; its
coupled pool is +0.0%. The nearest +17.7% on the list is the DECOUPLED pool,
which is cut-INDEPENDENT (the #704b R6 sum-rule result: 513,875 for every cut
including the incumbent) and therefore not attributable to any cut.

SOLVER DEFECT FOUND, flagged not fixed: prefill_frontier.py:144-156 enumerates,
per lead depth, ONLY the tail split that minimises pipelined time. At n0=30
that is [30,16,18] (130.95 ms), so [30,18,16] (139.32 ms) is never generated --
yet [30,16,18] holds 343,951 tokens (-21.2%) against [30,18,16]'s 436,275
(+0.0%). The frontier is Pareto-optimal in SPEED but not complete in POOL: a
roomier, slightly slower cut at the same lead depth cannot appear on it.
[30,18,16] strictly dominates the incumbent (equal pool, more speed) and was
invisible for exactly that reason, which is why it never reached the sgl-project#702 block.
The fix would be to keep the Pareto set over (pool, speed) per lead depth
instead of the speed-argmin; other pool-preserving cuts may also be hidden.

Validation-boot evidence is specified in falsification order, and includes the
one that matters most: for [30,18,16] the prefill A/B is expected to be
INCONCLUSIVE by construction -- a boot reporting a clean 1.11x has measured its
own noise, and that result must be rejected rather than celebrated.

Appended to the morning plan file as a dated block per its amendment
convention; head-298 sha256 verified byte-identical before and after
(81560e94b7628e6c8...). That file is not version-controlled, hence this copy.

## sgl-project#485 completion verdict (docs/VERDICT_485_phase_matrix.md)

Directive quoted verbatim from /spinning/htsglang/CLAUDE.md:87-96 (user law
2026-08-03), which is NOT tracked in this worktree. The two facets under sgl-project#485
are the same ticket, not a number collision: the matrix doctrine plus its
prefill-column solver, and the seam/memory certification gating it.

VERDICT: PARTIALLY DELIVERED, do not close.

  DELIVERED  prefill column for the attention-vs-GDN/linear boundary
             (c08f613, b675630, b3c9802, e645aa7), with a second
             axis on the attention row (sgl-project#492 d82a778, corrected by sgl-project#503
             447249a). Its own feasibility facet (407b382) is RED: the
             threshold is not reachable at the certification pool.

  SUPERSEDED d937d5f refuses the decode-side family split on sound MEASURED
             reasoning -- against the honest proportional (uneven-TP) baseline
             the split is worth +0.090 ms/round, ~0.3% of a 30 ms bs=1 round.
             But this closes ONE CANDIDATE CELL (concentration), not the
             column. Named tension: the directive says single-family arms are
             DIAGNOSTIC only and must never be phrased as a phase-level
             verdict, and that commit is worded as one. The narrow reading is
             adopted.

  OPEN       O1 no (family x phase) structure exists anywhere -- pp_cut.py
             solves prefill only, family_split.py returns by_family with NO
             phase argument or dimension; the "matrix" is prose over two
             single-phase solvers. O2 the decode solver is DEAD CODE:
             solve_family_placement has exactly two references, its definition
             (family_split.py:141) and its test, zero call sites. O3 vocab,
             experts, nonlinear kernels and per-quant-lane linear layers were
             never cut at all. O4 the law covers diffusion, SR/video, TTS/ASR
             and training tenants -- entirely untouched, and the largest open
             surface. O5 the prefill column is not arm-able while Facet B is
             red.

  NOT sgl-project#485   sgl-project#704/#704a/#704b are a PP-cut layout LADDER and its arming
             machinery. They consume the prefill-column solve as an input and
             must not be counted toward the matrix; counting them would
             inflate its delivery with work on a different axis.

Status in one line: one of two columns, one of many families, one of several
workloads -- and the table is still prose.

Every load-bearing claim spot-verified against the tree rather than taken from
the survey: directive text read at source, solve_family_placement call sites
grepped, d937d5f's baselines read from the commit body, [30,18,16] priced
directly rather than inferred from frontier membership.

Sanity: 35 passed (test_prefill_frontier_702.py, test_fill_ladder_704a.py).
Desk only, no GPU, deploy tree untouched.
efschu pushed a commit to efschu/htsglang that referenced this pull request Sep 13, 2026
…not exclusive split (W1)

BOOT7 (Zwerg Qwen3.5-2B, num_key_value_heads=2, D=TP3): every rank's
canonical draft-KV window install raised CanonicalPageError "draft head
window mismatch (W1)" identically. _draft_head_window (cache_controller.py)
already gives every rank the full (0, total) window when
attn_kv_replicated(tp_size, total) is True (kv < tp) -- mirroring
ModelConfig._uneven_tp_num_kv_heads, which already sizes the draft device/
host pool the same way ("REPLICATED-KV geometry -- every rank holds ALL kv
heads. Rank-independent by construction."). local_head_window
(disaggregation/draft_kv_canonical.py), used as the independent reference
both P and D compute the window from, instead did a largest-remainder-first
EXCLUSIVE split with an empty window for ranks beyond num_kv_heads -- the
wrong convention below the threshold, disagreeing with attn_kv_replicated
everywhere else in the tree (distributed/utils.py, the sgl-project#492 falsifier,
test_kv_eq_tp_stays_in_normal_mode_by_measurement).

Fix: local_head_window returns (0, num_kv_heads) for every rank when
num_kv_heads < tp_size, before falling back to the existing largest-
remainder split (unchanged, still correct, for num_kv_heads >= tp_size).
Kept as a plain arithmetic check rather than importing attn_kv_replicated/
tp_plan_active: the module's whole point is that P (tp_size=1, no installed
ratio plan) and D compute the window independently from
(num_kv_heads, tp_size, tp_rank) alone, with no shared plan state.

Host-pool allocation needed no change -- already replicated via
_uneven_tp_num_kv_heads -- only a pinning test, closing the "head_num=0 or
replicated" question the ticket posed: replicated, and already so.

Tests, hermetic (CUDA_VISIBLE_DEVICES=""), reproduced red before the fix
(build_draft_window raising W1 at kv=2/tp=3 exactly as on BOOT7) and green
after:
- test_draft_kv_canonical_631b.py: rewrote the test that pinned the wrong
  convention (test_more_ranks_than_heads_yields_empty_windows ->
  test_kv_lt_tp_yields_full_replicated_windows_not_empty, with the reasoning
  in the docstring), scoped the exclusive-split exactness test to its real
  domain (num_heads >= tp_size).
- test_weg2_draftkv_canonical_1233.py: new TestDraftWindowKvLtTp -- full-
  window pin, kv==tp boundary stays exclusive-split, and a mutant test for
  the danger direction (a reintroduced exclusive split at kv<tp must still
  be refused by W1, not silently accepted as a smaller-but-plausible
  window).
- test_uneven_tp_memory.py: TestNumKvHeadsAutoRank.
  test_kv_lt_tp_replicates_full_head_count_on_every_rank pins the host/
  device pool allocation side.

Green: test_draft_kv_canonical_631b.py, test_weg2_draftkv_canonical_1233.py,
test_weg2_draftkv_keys_1233.py, test_canonical_page_store_706.py,
test_canonical_kv_page_kv_major_1233.py, test_canonical_kv_page_704b.py,
test_canonical_kv_page_args_706.py, test/registered/unit/disaggregation/
(250 passed), test_uneven_tp_memory.py, test_uneven_tp_args.py,
test_attn_replication_axis_492.py, test_weg2_draftkv_l3_write_1233.py,
test_embedding_cache_controller.py,
test_cache_controller_reset_joins_io_aux_1068.py. Import smoke clean.
ruff clean on all four changed files; codespell clean.

Two pre-existing failures unrelated to this change, verified not touched by
this diff: test_cache_controller_reset_terminates_inflight_1068.py
(AttributeError 'has_draft' in a background thread) and
test_hybrid_decode_pool_attrs_631.py (mamba pool attribute contract) --
both outside this ticket's scope.

Design doc: /spinning/gpu-arb/weg2/DESIGN_draftkv_kvlt_tp_0913.md
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant