Skip to content

Commit cd31f01

Browse files
leon2k2k2kclaude
andcommitted
spec 009: align with implemented scope (2 modes, not 4)
Cleanup pass to resolve inconsistencies between the spec and what's actually in spinquant_hotstart.py + test_rotation_invariance.py: - Title + scope: 2-mode sweep (baseline, internal_only); full and port_1695 explicitly deferred to a follow-up spec. - Checkpoint path: pre_gptq.pt (what execution's spec-008 patch produced, after _unbank_state_dict), not final_model.pt. - Accept criteria: preflight via test_rotation_invariance.py (ALL TESTS PASS), then per-mode on pod. - Rotation structure: trimmed to just the implemented R_a class with exact banked-tensor indexing. R_0 / R_m / skip-stream / RMSNorm-fold sections moved to 'not implemented (deferred)'. - RMSNorm-fold section removed entirely: openai#1736's RMSNorm is gamma-free (F.rms_norm with no weight arg), so no fold needed. - Code-changes section: points at the files on disk instead of TODO pseudocode. - Execution protocol: 2 modes back-to-back on 8xH100, explicit preflight step. - Hardware ladder: 8xH100 required (phased TTT is 8-rank DDP). - Cost estimate: ~$15 total for 2 modes. - Open questions: reframed around unbanked-checkpoint load, bf16 drift, GPTQ interaction, phased-TTT compatibility. - What this spec does NOT do: clarified that residual rotation, R_m, resid_mix, and port_1695 are all deferred. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
1 parent 1572115 commit cd31f01

1 file changed

Lines changed: 81 additions & 203 deletions

File tree

0 commit comments

Comments
 (0)