Skip to content

SSM MTP-off C1 state restore, commit, and decode candidates - #11

Draft
YAMY1234 wants to merge 48 commits into
flashnext-verifyfuse-20260926from
flashnext-ssmoff-20260926
Draft

YAMY1234 wants to merge 48 commits into
flashnext-verifyfuse-20260926from
flashnext-ssmoff-20260926

Conversation

@YAMY1234

@YAMY1234 YAMY1234 commented Sep 26, 2026 •

Copy link
Copy Markdown
Owner

MTP-off state-only factorization at fork 56cdc14515daa4d4a586ec4d5d1be70af8a053cf passed the full numerical, static, length and 900-second confirmation gates at 2026-09-27 19:35Z. This implementation PR contains the opt-in kernels and state-path changes. Evidence is reviewed in report PR #192.

  1. Command: the source-bound ssmoff297_confirm872.py pair runner, submitted through subtasks/flashnext-ssmoff-20260926/analysis/hot-reference854-control/submit_confirm872_gpu.py, ran same-job, same-card stock → factor 900-second windows at C1×3/C8×2/C16×1. Directive Add more unit tests to CI sgl-project/sglang#880 added exactly two C16 pairs through submit_confirm880_gpu.py, retaining the original window and namespace. Commands, controls, admission hashes and all eight job receipts are in docs/117.
  2. Failure observed: factor state restore/commit and decode work initially regressed C1 timing. QKV's production hot-round KL initially exceeded a cold/mismatched self-reference; matching-shape production evidence in 898305 justified the authorized [Bug] AttributeError: 'NoneType' object has no attribute 'evict' sgl-project/sglang#854 rejudgment. The original final C16 window also had official P90 −0.230621%, despite decode-only P90 +4.277378%; this negative observation remains in the report.
  3. Call path: gdn_backend.forward_extend → gdn_prefill_qkv_prepare.prepare → factored_qk_ready combines mixed-QKV splitting with the existing Q/K L2 operations and skips the duplicate backend normalization. V is copied. This path is prefill-only. At the original C16 P90 boundary request, TTFT improved 96.503 ms while decoding grew 1,113.023 ms; the decomposition identifies mixed scheduling and completion-cohort effects, not additional normalization arithmetic or a proved single-kernel stall.
  4. Change: record the frozen QKV candidate's complete admission, all negative windows, paired-window intervals, seven-cell curve and costs. The inherited norm/output-gate fusion, PDL and cross-layer state work also contribute to the full candidate; there is no QKV-only B1 ablation. The four-batch slope −0.011016 ms comes from the measured B1/B8/B16/B32 differences.
Gate Result
Numerical sgl-project#854 13,059/13,059 deterministic positions exact; production KL/self-reference ratios cold/hot1/hot2 1.097272/2.370808/0.981693 ≤3; fixed and shape sets pass; GSM200 194→194
C1 static 898807/898808 Cold/hot TTFT −5.492481/−14.148432 ms; whole round −35.129573/−27.287013 ms; B1 −0.020587 ms/step; interruption −25.902272/−11.166720/−17.863616 ms
Four batches 899605 B1/B8/B16/B32 −0.056621/−0.042422/−0.165294/−0.371537 ms/step, slope −0.011016; predeclared first 20 stable steps. All-stable B8 +0.027795 ms remains report-only
Length (b) 898947 54/54, mean −12.541009 ms; strict (a) 49/54 retained as footnote
Low concurrency §10 899719 C1 official −0.511398%, 90% interval [−4.388437,+3.239450]%, nonblocking; C8 +16.906791%

All eight confirmation pairs are retained. Each entry below is the ratio of equal-weight window means and its paired-window bootstrap 90% interval. C8 has the two windows authorized by sgl-project#872; intervals are descriptive.

C / windows TPS/GPU Δ% [90% interval] Official P90 Δ% [90% interval] Decode-only P90 Δ% [90% interval]
1 / 3 +1.123615 [+0.371035,+1.887567] +0.753289 [−0.518268,+2.021936] +1.226381 [+0.532142,+1.923431]
8 / 2 0 [0,0] +1.962059 [−3.060682,+7.455445] +6.044248 [+5.452075,+6.635790]
16 / 3 +0.918202 [+0.823125,+1.013280] +1.963570 [+0.362717,+3.571840] +2.552776 [+1.485698,+3.627310]

Official P90 = 1/P90(E2EL/OSL); decode-only = 1/P90((E2EL−TTFT)/(OSL−1)). All pooled changes are nonnegative. Eight pairs cost 23.068889 allocated GPU-hours; the full line's closed total is 936.332222 GPU-hours. Supplemental jobs 901294/901295 ended 19:33:16/19:33:24Z, ahead of the 19:36–19:43Z ETA. Same-image admission tests, scalar cohort checks and attribution checks passed; exact receipts remain linked in the report.

Review docs/117, the final decision, and the seven-cell factor curve.

The report diff is restricted to docs/117, docs/figures, this line's subtasks and infra/aga scripts. No paper files, weights or raw jsonl are included. The 73 runtime/test paths excluded from this report are preserved at tooling commit 30b9b9a45889b5f92f3a4099244548bfdbeddd05 on flashnext-ssmoff-20260926-tools, with a reconstruction patch; measured fork and remote runtime are unchanged.

This admission covers the factor arm, which has no DUET component. Historical composite curves use v3-r4096-b. The k31-r4096-u batch has 9 independent numerical jobs and 14 both/latent performance cells prepared, pending K31U-READY and one same-image CPU preflight. Its admission is separate.

Fork CI is not green: the previous base run was blocked by draft status and lint reports local plus inherited formatting/registration issues. The audit and unapplied suggestions are recorded; the measured fork SHA remains frozen. Report PR sgl-project#192 is ready for Lead review. This implementation PR remains draft while its CI findings are outstanding; full experimental admission is complete.


CI States

Latest PR Test (Base): ❌ Run #36320000788
Latest PR Test (Extra): ❌ Run #36320000644

@YAMY1234 YAMY1234 changed the title Reduce factored SSM restore launches and decode metadata nodes SSM MTP-off C1 state restore, commit, and decode candidates Sep 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant