11L XSA4 + SmearGate + BigramHash + SWA + RoPE50K (mean val_bpb=1.1565, 3 seeds)#333

Open

mahsumaktas wants to merge 2 commits intoopenai:mainfrom

mahsumaktas:submission/v2-11L-xsa-swa-1.1538

Commits on Mar 20, 2026

Add Combined SOTA submission: INT6 + SmearGate + BigramHash + SWA (val_bpb=1.1754)

authored and
Mahsum
committed

Commits on Mar 21, 2026

Update submission: 1.1754 -> 1.1538 BPB (3-seed mean 1.1565)

authored and
Mahsum
committed