11L MLP3x int6: val_bpb=1.12109 (3-seed mean=1.12171, sliding window)#5

Open

devin-ai-integration[bot] wants to merge 12 commits intomainfrom

Update best: 11L MLP3x WD=0.02 prune=5% val_bpb=1.14198 (3-seed mean=1.14240, beats thwu1)

andrewgcodes
authored and
devin-ai-integration[bot]
committed
Update best: val_bpb=1.1284 sliding_window (EMA+XSA+RoPE16, LR=0.025, seq=2048)

andrewgcodes
authored and
devin-ai-integration[bot]
committed
Update best: val_bpb=1.12856 sliding_window (warmdown=4000, 15.59MB fits)

andrewgcodes
authored and
devin-ai-integration[bot]
committed
Update best: val_bpb=1.12319 sliding_window (LR=0.03, WD=0.035, warmdown=5000, 15.86MB fits)

andrewgcodes
authored and
devin-ai-integration[bot]
committed
Update best: warmdown=6000 3-seed mean=1.12359 (all fit under 16MB)

andrewgcodes
authored and
devin-ai-integration[bot]
committed

Provide feedback