[Notable Non-Record Submission] To JEPA or Not to JEPA: That Is Le Question (32.8M LeWorldModel Mamba2 Style Text Implementation - 1.2064 BPB )#903

Merged

valerio-oai merged 4 commits intoopenai:mainfrom

CiprianFlorin-Ifrim:lewm-jepa-ssm-mamba2

May 3, 2026

Commits on Mar 26, 2026

Record: 1.2064 BPB - 32.8M LeWM-style JEPA Mamba-2 SSM + JEPA + SIGReg

Ciprian-Florin Ifrim
authored and
Ciprian-Florin Ifrim
committed
Added diagrams to README.md.
CiprianFlorin-Ifrim
authored

Commits on Mar 28, 2026

Updated train_jepa_ssm.py to save SIGReg to the checkpoint, as on a checkpoint load before a new random SIGReg would be generated based on the seed.
CiprianFlorin-Ifrim
authored

Commits on Mar 29, 2026

Small fix to script.

Ciprian-Florin Ifrim
authored and
Ciprian-Florin Ifrim
committed