Skip to content

Fix Beam 256k padding for variable-length batches - #765

Merged
zhouhelena1 merged 1 commit into
mainfrom
codex/beam-cp-padding
Oct 28, 2026
Merged

zhouhelena1 merged 1 commit into
mainfrom
codex/beam-cp-padding

Conversation

@zhouhelena1

@zhouhelena1 zhouhelena1 commented Oct 28, 2026 •

Copy link
Copy Markdown
Contributor

The default batch-256 validation of #762 generated all 256 responses but failed before training in Miles fill_replay_data: assert seqlen % tp_size == 0. With TP8/CP2, padding to eight global tokens can leave a CP-local sequence that is not divisible by eight.

Set the 256k preset's padding multiplier to two so global lengths align to 16 tokens. The observed maximum sample length of 33,013 now pads to 33,024 instead of 33,016; the extra tokens are masked. Keeps upstream's 32 prompts × 8 completions (global batch 256).

Follow-up #766 adds effective-config alignment validation and native Beam LoRA resume with complete-checkpoint checks. Its GPU save/restart proof passed; it is a separate follow-up PR. These guards are separate from this merged padding fix.

Validation:

  • Fresh batch-256 generation: 32 prompts, 256 responses, 2,286,539 output tokens; 240 natural completions and 16 truncated at 32k. Training exposed the alignment assertion.
  • Pinned Miles replay-layout check: reproduces the old assertion and verifies the corrected token-to-route layout across all 16 TP/CP ranks with two variable lengths (32 checks).
  • Full-batch trainer replay: completed the full forward/backward pass and one valid optimizer update on all 256 saved responses, using 16 H200s; no fresh generation. Logprob absolute difference 0.00963, trainer/rollout KL 0.000233, grad norm 0.00564. App stopped.

Autoconfig: Beam 256k DAPO-math

@github-actions

Copy link
Copy Markdown

@zhouhelena1
zhouhelena1 marked this pull request as ready for review October 28, 2026 01:58
@zhouhelena1
zhouhelena1 added this pull request to the merge queue Oct 28, 2026
Merged via the queue into main with commit 0e8a5d7 Oct 28, 2026
9 of 10 checks passed
@zhouhelena1 zhouhelena1 changed the title Fix Beam 256k padding for variable-length batches Fix Beam 256k padding and disable unsafe LoRA retries Oct 28, 2026
@zhouhelena1 zhouhelena1 changed the title Fix Beam 256k padding and disable unsafe LoRA retries Fix Beam 256k padding for variable-length batches Oct 28, 2026

This branch was successfully deployed

1 active deployment
modal-preview — 27c1a677 Deployed Oct 28, 2026 by zhouhelena1 via preview #7321
modal — 27c1a677 Deployed Oct 28, 2026 by zhouhelena1 via validate (Beam-1.0-501B-A23B-LoRA) #3213
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants