Skip to content

Commit 201f563

Browse files
committed
DEPRECATE specs 060G and 060H — refuted by PR openai#1898 measurement
PR openai#1898 (X-Abhishek-X) ran Partial SpinQuant + EMBED_BITS=6 reinvest on the same chain and reported val_bpb 1.06614 vs their base openai#1851's 1.06128 = +0.00486 REGRESSION. Their PR framed it as -0.01486 vs the 2-week-old merged SOTA openai#1493 (1.0810) instead of vs their actual parent. Implications: - 060G (Partial SpinQuant): empirically null/negative on this chain. - 060H (EMBED_BITS=6 alone or with LQER reinvest): even riskier without SpinQuant's rotation protection. Both specs marked as DEPRECATED at the top. Not deleted (kept as documentation for if conditions change later, e.g., deploy-time repair specifically targeting tok_emb precision).
1 parent 827f5ab commit 201f563

2 files changed

Lines changed: 31 additions & 0 deletions

File tree

research/specs/060G-partial-spinquant.md

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,20 @@
11
# Spec 060G — Partial SpinQuant from PR #1898 on 060A baseline
22

3+
**Status: DEPRECATED 2026-04-29 — empirically refuted by PR #1898 itself.**
4+
5+
PR #1898 ran this exact lever (Partial SpinQuant + EMBED_BITS=6 reinvest) on
6+
its own base (#1851 at 1.06128) and got **1.06614** — a **regression of
7+
+0.00486 BPB**. Their framing of "−0.01486 vs merged SOTA #1493 (1.0810)"
8+
is misleading; the like-for-like comparison vs their actual parent shows
9+
the lever doesn't help.
10+
11+
Reason to keep this spec on file: documentation of why we're NOT pursuing
12+
SpinQuant on the 060A line. If the lever ever becomes promising in the
13+
future (e.g., paired with deploy-time repair or different bit allocations),
14+
the spec is here as a starting point. Do not run.
15+
16+
---
17+
318
**Date:** 2026-04-29
419
**Branch:** `exp/060G-partial-spinquant` (forked from research)
520
**Parent:** 060A + #1898 SpinQuant code (port from `X-Abhishek-X/parameter-golf` PR #1898).

research/specs/060H-embed-bits-6.md

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,21 @@
11
# Spec 060H — EMBED_BITS=6 with LQER recovery on 060A baseline (eval-only)
22

3+
**Status: DEPRECATED 2026-04-29 — implied refutation by PR #1898.**
4+
5+
PR #1898 ran EMBED_BITS=6 *with* SpinQuant rotation as protection against
6+
INT6 noise and got a **+0.00486 BPB regression** vs their base. EMBED_BITS=6
7+
*without* SpinQuant (this spec's H1, H2, H3 arms) has *more* INT6 noise on
8+
`tok_emb`, so all arms here would likely regress further. Don't run on its
9+
own — the pessimistic scenario in our prediction table is the most likely
10+
outcome.
11+
12+
Document kept for reference. If we ever build deploy-time repair (060C)
13+
that specifically targets `tok_emb` precision recovery, this spec becomes
14+
worth re-examining as a stack candidate — but only after that's measured
15+
and shown to work.
16+
17+
---
18+
319
**Date:** 2026-04-29
420
**Branch:** `research` (config-only)
521
**Parent:** 060A `final_model.pt` + `RESUME_FROM_CKPT` infrastructure (commit `a7c0ed8`).

0 commit comments

Comments
 (0)