Commit cd31f01
spec 009: align with implemented scope (2 modes, not 4)
Cleanup pass to resolve inconsistencies between the spec and what's
actually in spinquant_hotstart.py + test_rotation_invariance.py:
- Title + scope: 2-mode sweep (baseline, internal_only); full and
port_1695 explicitly deferred to a follow-up spec.
- Checkpoint path: pre_gptq.pt (what execution's spec-008 patch
produced, after _unbank_state_dict), not final_model.pt.
- Accept criteria: preflight via test_rotation_invariance.py
(ALL TESTS PASS), then per-mode on pod.
- Rotation structure: trimmed to just the implemented R_a class
with exact banked-tensor indexing. R_0 / R_m / skip-stream /
RMSNorm-fold sections moved to 'not implemented (deferred)'.
- RMSNorm-fold section removed entirely: openai#1736's RMSNorm is
gamma-free (F.rms_norm with no weight arg), so no fold needed.
- Code-changes section: points at the files on disk instead of
TODO pseudocode.
- Execution protocol: 2 modes back-to-back on 8xH100, explicit
preflight step.
- Hardware ladder: 8xH100 required (phased TTT is 8-rank DDP).
- Cost estimate: ~$15 total for 2 modes.
- Open questions: reframed around unbanked-checkpoint load,
bf16 drift, GPTQ interaction, phased-TTT compatibility.
- What this spec does NOT do: clarified that residual rotation,
R_m, resid_mix, and port_1695 are all deferred.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>1 parent 1572115 commit cd31f01
1 file changed
Lines changed: 81 additions & 203 deletions
0 commit comments