Non-record: 1x H100 SXM5 Explorations#1608

Open

User123331 wants to merge 70 commits intoopenai:mainfrom

User123331:main

Commits on Mar 20, 2026

Add Mixture of Softmax (MoS) with low-rank option for softmax bottleneck

Billy Endson
and
claude
committed
Add one-shot RunPod setup and pilot run script

Billy Endson
and
claude
committed
Simplify pilot script: MoS only, skip redundant baseline

Billy Endson
and
claude
committed
Fix setup script: remove clone step, assume already in repo

Billy Endson
and
claude
committed
Add HF_TOKEN to setup script for faster dataset downloads

Billy Endson
and
claude
committed
Record: MoS K=2 R=64 pilot — val_bpb=1.3932 (1xH100, 10min)

Billy Endson
and
claude
committed
Add 1-hour MoS validation script (targeting PR#111 baseline)

Billy Endson
and
claude
committed
Make 1h script survive terminal disconnects via nohup

User123331
and
claude
committed
Add vanilla baseline 10-min script for 1xH100 comparison

User123331
and
claude
committed
Add research artifacts: technique encyclopedia, combination matrix, speed optimizations, SOTA plan

User123331
and
claude
committed

Commits on Mar 21, 2026

Commits on Mar 22, 2026

Add RunPod SOTA launcher

Billy Endson
and
Copilot
committed
Add MoS + SOTA technique stack for competitive testing

Billy Endson
and
claude
committed
Run training with nohup to survive terminal disconnects

Billy Endson
and
claude
committed
Fix FA3 build: use --no-build-isolation so setup.py can find torch

Billy Endson
and
claude
committed
Add keep-alive heartbeat to prevent RunPod pod termination

Billy Endson
and
claude
committed
Fix FA3 build: clear stale build dir, fix variable scoping

Billy Endson
and
claude
committed
Add sentencepiece and numpy to deps check

Billy Endson
and
claude
committed
Fix FA3 install: mkdir flash_attn_3 before pip editable install

Billy Endson
and
claude
committed

Commits on Mar 23, 2026

Add hyperbolic.ai setup scripts for 8x H100

authored and
Billy Endson
committed
Update quickstart to use pre-compiled FA3 .so

authored and
Billy Endson
committed
Fix data paths for hyperbolic setup

authored and
Billy Endson
committed
Fix: use ~/golf instead of /workspace for hyperbolic

authored and
Billy Endson
committed
Fix: use $HOME instead of /workspace for FA3 build

authored and
Billy Endson
committed
Add --break-system-packages for externally-managed environments

authored and
Billy Endson
committed
Fix: use SCRIPT_DIR instead of hardcoded golf path

authored and
Billy Endson
committed
Fix: use user site-packages instead of system

authored and
Billy Endson
committed
Build FA3 from source (pre-compiled .so not in repo)

authored and
Billy Endson
committed
Add DISABLE_COMPILE option to fix torch.compile/inductor issues

authored and
Billy Endson
committed
Remove nohup wait - use tmux for persistence instead

authored and
Billy Endson
committed

Commits on Apr 14, 2026