Repository navigation
Add Beam BF16 LoRA support and a reference run - #252
Fred Robinson (frrad) wants to merge 3 commits into
Conversation
49a3a98 to
594edf2
Compare
Merge #252's current head. Its Miles v0.2.0 image already includes the Kimi K3 change, so the K3 recipe now uses trainer_image "miles-v0.2.0" and the K3-only trainer image and the fork_image() helper are gone. kimi_k3_image.py keeps only the sampler image. Kimi K3 trains with megatron_to_hf_mode "raw", which Spindle only passes to Miles, so miles-v0.2.0 no longer rejects it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
|
||
| def _gather_capture(self, written: Path, path: Path) -> None: | ||
| """Make every node's writes to a capture on the checkpoint volume visible here.""" | ||
| if not on_checkpoint_volume(path): |
There was a problem hiding this comment.
is this necessary to restore multi-node on 0.2.0 or why this change?
|
|
||
| image = image_at(MILES_COMMIT) | ||
| # Beam on the Megatron Bridge (modal-projects/miles#37). | ||
| beam_image = image_at("3280b7a5e3c4399c1ac18bed568859c9068852c3") |
There was a problem hiding this comment.
can we just call this image_ or something? ideally this is just a linear list of commits on our shared 0.2.0 fork branch. mb add a comment about this. is image_020_0001_sha mb better? ff to rename the existing one to be consistent
| revision="803bccbcb947d3041f5b57e11c986f8523a0cb56", | ||
| core_packages=CORE_SAMPLER_PACKAGES, | ||
| ).add_local_python_source("spindle", ignore=ignore_config_source) | ||
|
|
There was a problem hiding this comment.
same naming thing as above
| def on_checkpoint_volume(path: Path) -> bool: | ||
| """Whether ``path`` lives on the checkpoint volume, where nodes see only committed writes.""" | ||
| return ( | ||
| "SPINDLE_CHECKPOINT_VOLUME" in os.environ |
There was a problem hiding this comment.
how does this relate to the existing _checkpoint_volume_path function? can we do something cleaner?
| def run(args, batch_id, model, *batch_args, **kwargs): | ||
| rollout_data = batch_args[-1] | ||
| def run(args, batch_id, model, *slot_and_data, **kwargs): | ||
| # v0.2.0 passes adapter_slots before rollout_data; preserve both native arguments. |
There was a problem hiding this comment.
wait this is very scary - can you do something more explicit here? like actually check the version and branch on that potentially?
| self._bridge.save_slot(slot, path, include_optimizer=include_optimizer) | ||
| ) | ||
| # Outside _run: a bad capture fails this request, not the shared trainer. | ||
| self._gather_capture(written, Path(path)) |
There was a problem hiding this comment.
why are we adding this?
Reuse the stock Tinker workload with a pinned tokenizer and the existing TP2/CP8 H200 topology. Keep the response cap at 2048 tokens; this is not a 256K-context validation. Record no execution until a GPU run exists. Enable Miles v0.2.0 multi-node captures by committing each node's files before coordinator reload and sharing the checkpoint I/O lock for volume-backed captures, including their metadata writes. Local captures keep their independent persistence path. Adapt the generic backend changes from #252 commits 53f80f7, 594edf2 and 18054ed, with its owner's agreement. Leave Beam recipes and images out.
Reuse the stock Tinker workload with a pinned tokenizer and the existing TP2/CP8 H200 topology. Keep the response cap at 2048 tokens; this is not a 256K-context validation. Record no execution until a GPU run exists. Enable Miles v0.2.0 multi-node captures by committing each node's files before coordinator reload and sharing the checkpoint I/O lock for volume-backed captures, including their metadata writes. Local captures keep their independent persistence path. Adapt the generic backend changes from #252 commits 53f80f7, 594edf2 and 18054ed, with its owner's agreement. Leave Beam recipes and images out.
18054ed to
96ba5c4
Compare
c09c2f4 to
e36bfcc
Compare
6a3479d to
3996abc
Compare
d58899e to
f7ed75a
Compare
Miles v0.1.1 and v0.2.0 put rollout_data at different positions in run_forward_backward, because v0.2.0 adds adapter_slots before it. The replay wrapper now binds each call to the wrapped function's own signature and reads rollout_data by name, so positional and keyword calls work on both versions. Installing the hooks fails at trainer startup when that function has no rollout_data parameter. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
beam-501b-a23b-lora-5k trains reflection/Beam-1.0-501B-A23B with BF16 LoRA on two eight-H200 trainer nodes, with separate attention and MLP gate targets and routing replay. It trains on miles-v0.2.0-8c5de26 and samples on sglang-v0.5.23-e003cf9, which serves Beam LoRA. reference_runs/configs/beam_blog_repro.toml selects that recipe and scripts/beam_blog_repro.py, a stock-Tinker client that trains on DAPO-Math, evaluates on AIME25 and can resume from checkpoints. The index holds an earlier 42-update execution, converted to the catalog output format, and the digitized curve from the published Beam LoRA run, which records source instead of Spindle provenance. catalog.py now defines the output format every reference script writes (Record, Evaluation, Update and read_output), and tests check every indexed output and each script's output against it. A Spindle index entry needs its trainer image, Miles commit, Tinker SDK and hardware; an external entry records only its source. The Beam script and its tests are fully typed, without ANN ignores. AGENTS.md and the ignore list say that list only shrinks. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The test checked that /proc/PID/stat existed and then read it. When the killed descendant disappeared between the two calls, the read raised ProcessLookupError and the test failed, although the supervisor had done its job. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
f7ed75a to
b5140f9
Compare
Summary
Adds BF16 LoRA support for
reflection/Beam-1.0-501B-A23B(#235). It builds on two fork changes:3280b7a5: the Beam model and Bridge provider on Miles v0.2.0, ported from upstream feat(beam): add Reflection Beam 1.0 RL support radixark/miles#4149 without its shared Bridge LoRA changes.568ab742: Beam LoRA in the sampler, a backport of upstream [Feat] beam: support LoRA for BF16 and FP8 sgl-project/sglang#48264.What changes in Spindle:
miles-v0.2.0-3280b7afor training andsglang-v0.5.23-568ab74for sampling. Following the frozen-image rule in Pin frozen, commit-named images in every bundled config #313, no existing image changes, so other configs are unaffected.miles-v0.2.0-*trainers. Each node commits its adapter checkpoint shards to the checkpoint volume, and the head reloads the volume before checking the capture. This only happens for captures on the volume, so single-node trainers behave as before.dangerously_allow_checkpoint_revision_mismatch.reference_runs/beam_config.py, with a 2×8 H200 trainer and an 8×B300 sampler. The client isreference_runs/beam.py, on the stock Tinker SDK.Reference run
The reference client trained against this branch and was compared with the published Beam LoRA run and an earlier run on Miles v0.1.2. The goal is confidence that training on this stack is healthy, not a parity claim. The blog curve is a manual digitization, and its step axis isn't confirmed to count optimizer updates. Accuracy above the blog most likely comes from config differences, such as the loss, async lag and hardware, not a real improvement.
AIME25 accuracy, 30 problems × 8 samples:
Accuracy settles at 64–70% from step 40 onward. Mean response length falls from 3,164 tokens to about 2,200 by step 55. The trainer call takes 332–499 s per update, against 478–574 s on v0.1.2.
The chart also shows an earlier run of this branch on Miles
f45c63c(before review) and SGLang991fb375. It tracks this run closely.Testing
exp_avg, 624exp_avg_sq).a049c55with Milesd1915e68and SGLang568ab742, through update 70 so far.d1915e68has the same tree as the merged3280b7a5.miles-v0.2.0after this change;sglang-v0.5.23-568ab74.