Repository navigation
docs: add the Beam 1.0 RL doc - #4138
Conversation
Add docs/models/reflection/beam.md for Reflection's Beam 1.0 in the six-section recipe layout, register it after Muse Spark in docs.json and add a Reflection row to the models table. The scripts and the image come from the beam-image branch, which the page links. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- The softcap, not the LM head, runs in FP32; the FP32 tensors are the router weights and expert-score biases. - 9ac1c65b also carries Beam LoRA serving; later commits are on `beam`. - LoRA optimizer state is ~4.6 GB per GPU; the adapter sync is ~8.7 GiB. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Describe SSMax and the depth-scaled residuals in words, as the other model pages do, give SSMax its own highlight, and state the BF16 / FP8 / NVFP4 release and the 1M-token context from the model card. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Follow the Inkling page: one row per model with active / total parameters, layers and HF ID. Note there, instead of in 3.1, that the bridge loads only the BF16 checkpoint (the FP8 and NVFP4 releases carry a quantization_config, which the Beam bridge rejects). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Follow the Step-5-Preview page: 3.2 points at the published multi-arch radixark/miles:beam image (radixark/miles:v0.1.2 + SGLang beam-miles 9ac1c65b + the beam-image branch, built by docker/Dockerfile.beam) instead of a local docker build, and 4 opens with the Ray cluster the launchers submit to, with one subsection per topology. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The Beam launchers set MILES_SCRIPT_EXTERNAL_RAY=1 and never start Ray, so add a 3.3 like DeepSeek-V4.1's: run the image on every node with the checkpoint, datasets and --output-dir on shared storage, start the head and the workers, and check the GPU count before launching. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Follow the Inkling page: sections 4-5 describe the full-parameter recipe (the LoRA launch command stays in 4.4), and a new section 6 LoRA RL gathers the LoRA layout, targets, expert-TP 1, optimizer and LR schedule, and adapter-only weight sync. Pairs Well With becomes section 7. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Drop the separate LoRA RL section: the LoRA row returns to the 5.1 table, and a short 5.5 LoRA lists the flags that differ from the full-parameter recipe. 5.4 Rollout now shows the SGLang and weight-update flags as a block (values match the launcher snapshots). Pairs Well With is section 6 again. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
There was a problem hiding this comment.
Claude Code Review
This repository is configured for manual code reviews. Comment @claude review for a one-time review, or @claude review always to subscribe this PR to a review on every future push.
Tip: disable this comment in your organization's Code Review settings.
| `prepare` downloads the checkpoint into `--model-dir` and DAPO-Math-17K and AIME25 into `--data-dir`; every node must see both at the same path. `run_beam_lora.py prepare` is identical, so one run serves both recipes: | ||
|
|
||
| ```bash | ||
| python scripts/run_beam.py prepare --model-dir /root/models/Beam --data-dir /root/datasets |
There was a problem hiding this comment.
Currently we have a code path that can one-click to convert ckpt and start training. Could you use that path?
| description: Launch recipes for Reflection's Beam 1.0 (501B MoE, 23B active) — full-parameter BF16 GRPO on 96 or 64 H200 GPUs, and LoRA on 32. | ||
| --- | ||
| The complete Beam RL implementation is open on the Miles | ||
| [`beam-image`](https://github.com/radixark/miles/tree/beam-image) branch. The scripts and image |
There was a problem hiding this comment.
Could you add miles branch name for beam model and open a draft PR? It will be easier to find SoT
|
|
||
| `ray status` on the head must list 96, 64 or 32 GPUs before launching, for 4.2, 4.3 and 4.4 respectively. | ||
|
|
||
| ### 4.2 Full parameter, 12 nodes |
There was a problem hiding this comment.
Is this async trianing? If so, we can mentioned it
| --sglang-dtype bfloat16 | ||
| --sglang-attention-backend triton | ||
| --sglang-moe-runner-backend triton | ||
| --sglang-context-length 5120 |
There was a problem hiding this comment.
Why add this constrians?
|
|
||
| ```bash | ||
| --sglang-dtype bfloat16 | ||
| --sglang-attention-backend triton |
There was a problem hiding this comment.
Currently we only have triton? Better check sgl cookbook for the beam model to find best perf solution
…h from rollout flags Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
zhihengy
left a comment
There was a problem hiding this comment.
Todo: verify alternative attention backend;
Other comments are fixed; this pr is merged for time reason
Follows the launcher change: execute becomes train, execute-colocate becomes train --colocate, and --join-ray-workers replaces bringing the Ray cluster up by hand.
Section 3.1 is now optional: the launch commands download what they need.
full-train fetches the checkpoint and datasets itself, so 3.1 no longer walks through prepare; it keeps the facts (shared path, no Megatron conversion, the answer-format eval file) and mentions prepare only for staging the download on its own.
Adds the RL recipe page for Reflection Beam 1.0 (501B MoE, 23B active).
docs/models/reflection/beam.md: hybrid sliding-window / full-attention (no RoPE, SSMax) model, both launchers (full parameter on 96 H200 or 64 colocated, LoRA on 32 H200), the per-recipe parallelism, algorithm, optimizer (NVMe-streamed for full parameter) and rollout settings, and resume.docs.jsonand the models table.The scripts and
docker/Dockerfile.beamlive on thebeam-imagebranch, which the page links; SGLang support is thebeam-milesbranch (9ac1c65b).