Skip to content

fix(minimax-m3): score bf16/FP8 index prefill into a fixed workspace - #3427

Open
Jasen2201 wants to merge 1 commit into
mainfrom
Jasen/m3-index-prefill-workspace
Open

Jasen2201 wants to merge 1 commit into
mainfrom
Jasen/m3-index-prefill-workspace

Conversation

@Jasen2201

Copy link
Copy Markdown
Collaborator

Problem

With a bf16 or FP8 index cache, MiniMax-M3's prefill index scorer (_triton_prefill_score) allocated its fp32 block scores, [index heads, rows, context blocks], on every forward. In a chunked long prefill the width grows with every chunk (64 blocks per 8K chunk), so the caching allocator never reuses the previous, smaller block and reserves a new segment each chunk. The memory profile never sees them: its warmup prefill is short.

On a PP4 prefill (TP1 per stage, 4 index heads, 8K chunks, 1M context, --gpu-memory-utilization 0.9), one 702K-token prompt raises reserved memory by 21-27 GiB per stage while allocated memory stays flat: 36-54 free cached segments a stage, 8 MiB apart, all from that line. Under AgentX traffic the device then runs out, an RCCL p2p kernel cannot get its scratch memory, and the stage aborts with HSA_STATUS_ERROR_OUT_OF_RESOURCES, taking the server down. --index-cache-dtype fp4 is not affected: since #2962 its scorer writes into the builder's LogitsWorkspace.

Fix

  • allocate_prefill_scores gives the bf16 / FP8 index caches the same kind of workspace: one LogitsWorkspace of prefill_score_capacity fp32 scores, index heads x max(max_num_batched_tokens, max_num_seqs x query width) rows x the max_model_len blocks. That is the bound the MXFP4 record already uses for its prefill rows, and allocate_index_scorers now takes it from the same function. The builder allocates it in its constructor, so the memory profile counts it.
  • make_sparse_prefill_metadata(score_workspace=) carries it to the sparse layers, and minimax_m3_index_topk(score_workspace=) hands it to _triton_prefill_score, which writes into it instead of allocating. The kernels and the selection are unchanged. A step wider than the workspace fails in LogitsWorkspace.logits, as on MXFP4, instead of allocating.

Resident cost

It scales with max_model_len: 1 GiB per GPU at TP1 with --max-num-batched-tokens 8192 --max-model-len 1048576 (4 index heads), also 1 GiB at TP4 with 32768 (1 head a rank), and 128 MiB at TP4 with a 131072 model length. KV blocks: 86,299 -> 85,827 (-0.55%) on the PP4 configuration above, 61,327 -> 61,283 (-0.07%) on the TP4 GSM8K servers below.

Validation

MI355X (gfx950), image rocm/atom-dev:nightly_202611151511 with each tree mounted: base = main 246df9d, this PR on top of it. MiniMax-M3.1-Flash-Quark-MXFP4, --kv_cache_dtype fp8 --index-cache-dtype fp8, ptpc FP8, --block-size 128. PP4 = -tp 1 -pp 4 --enforce-eager --max-num-batched-tokens 8192 --max-model-len 1048576 --gpu-memory-utilization 0.9, prefix caching on, Mooncake producer, the AgentX prefill-only setup the crash was found in. Base and PR ran at the same time on GPUs 0-3 and 4-7, from fresh servers.

PP4, one 702K-token prompt:

base this PR
Reserved memory, per stage +21.0 / +27.5 / +27.4 / +26.7 GiB +1.7 / +2.6 / +2.6 / +1.7 GiB
Free cached segments from the scorer (allocator snapshot) 36-54 per stage, 18.9-24.5 GiB none
TTFT 11.78 s 11.74 s
KV blocks 86,299 85,827

PP4, AgentX trace replay (AIPerf inferencex-agentx-mvp, one output token, 300 s), fresh servers for each concurrency:

C16 base C16 this PR C32 base C32 this PR
Server ran ran aborted after about 3 min of traffic: HSA_STATUS_ERROR_OUT_OF_RESOURCES in ncclDevKernel_Generic ran
Reserved memory, peak per stage (of 288 GiB) 276.4-277.0 GiB 236.6-244.5 GiB, its level at start 264.6-276.9 GiB 236.8-244.4 GiB, its level at start
Allocator out-of-memory retries 1 per stage 0 1 0
Requests, errors (an error here = one empty response) 4,253, 1 4,455, 0 - 4,832, 1
req/s (one run each) 12.89 13.50 - 14.64
TTFT p50 / p90 / p99 0.80 / 2.11 / 4.8 s 0.79 / 1.79 / 5.1 s - 1.84 / 3.46 / 5.7 s
Longest context 839K 838K - 874K

The C16 base run survived because the first allocation to fail was PyTorch's, which frees its cache and retries. At C32 an RCCL kernel's scratch allocation failed first.

Long-context retrieval through chat, thinking off (a document of Room r has access code c. lines, the question for 10 rooms from 1% to 99% depth), on the same PP4 servers:

Context base this PR
82K tokens 9/10 9/10, the same answers
296K tokens (aiter's selector) 8/10, 8/10, 8/10 8/10, 9/10, 8/10

At 296K both miss the same two edge rooms; their wrong answers change from run to run on either server.

GSM8K chat (5-shot multi-turn, thinking on, temperature 0, num_concurrent 64), TP4 aggregated servers (--max-num-batched-tokens 32768 --max-model-len 131072, CUDA graphs, no prefix caching), 8 runs per arm, the two arms swapping GPU sets after the third:

base this PR
flexible-extract, mean (range) 0.9570 (0.9530-0.9621) 0.9562 (0.9530-0.9598)
strict-match, mean (range) 0.9160 (0.9067-0.9219) 0.9137 (0.9060-0.9174)

The differences are within one standard error of the means. Responses change from run to run on one server about as much as between the arms (302 of 1,319 identical between two base runs, 294-307 between base and PR runs).

Unit and GPU tests (M3 suites plus the MHA builder and transfer tests, 16 files): base 325 passed, this PR 333 passed (the 8 new tests), 32 skipped on both, no failures.

  • test_chunks_scored_into_one_stale_workspace_select_as_fresh_buffers (GPU): the chunks of one prefill share a workspace filled with +inf before each call. Both selectors (Triton, and aiter's from 2048 blocks), with bf16 and FP8 caches, return exactly what a fresh buffer gives. With the scorer redirected so the selection reads the stale workspace, all four cases fail.
  • tests/test_minimax_m3_index_buffers.py (CPU, runs in CI): the workspace holds a full chunk at the model length and refuses one row more; a verify step's rows widen it; MXFP4 gets none.

Not in this PR

  • The vLLM and SGLang bridges call the scorer without a workspace and still allocate per prefill.
  • Decode's Triton scorer keeps its per-step buffer: at most max_num_seqs x query width rows (16 MiB at TP1, 128 sequences, 1M context).

🤖 Generated with Claude Code

The Triton prefill scorer allocated its fp32 block scores
[index heads, rows, blocks] on every forward. In a chunked long prefill
the width grows by 64 blocks a chunk, so the caching allocator kept a
new, slightly larger segment each chunk. The memory profile never saw
them (its prefill is short). On a PP4 stage (TP1, 4 index heads, 8K
chunks) one 702K-token prompt left 21-27 GiB of cached segments, and
under AgentX traffic at --gpu-memory-utilization 0.9 an RCCL kernel then
failed to get scratch memory and the server aborted
(HSA_STATUS_ERROR_OUT_OF_RESOURCES).

The builder now allocates one LogitsWorkspace for the bf16 / FP8 index
caches, sized like the MXFP4 record's (index heads x a step's rows x the
model length's blocks), before the memory profile, and the prefill
metadata hands it to the Triton scorer. The MXFP4 path is unchanged and
shares the sizing (prefill_score_capacity). The vLLM and SGLang bridges
pass no workspace and keep allocating per step.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Copilot AI balanced review requested due to automatic review settings November 16, 2026 12:11
@github-actions

Copy link
Copy Markdown
Contributor

🏷️ CI Guide

Runs automatically on every eligible PR before approval:

  • ✅ Pre Checkin: Black, Ruff, catalog schema validation, non-GPU unit tests

Heavy model tests:

  • ✅ Gate checks are published on every PR so required checks do not stay missing
  • ✅ Expensive jobs run only after the PR is approved, or when a matching ci:* label is present
  • ✅ Pre Checkin must pass before approved heavy jobs continue
  • ✅ Without a matching label, expensive jobs also require a relevant path
  • ✅ A matching ci:* label is an explicit request and can open the gate even when paths would otherwise skip it

Path scope:

  • Native ATOM, vLLM, and SGLang use ignore lists: docs/markdown and unrelated plugin or Mesh paths are skipped
  • ATOMesh uses an include list: Mesh code, ATOMesh scripts/workflows/catalogs, OpenAI entrypoints, and Slurm helpers
  • Relevant path alone does not start expensive jobs; approval still opens the automatic path-scoped gate
Label Tests
ci:full Run all heavy PR model tests: native ATOM, vLLM, SGLang, and ATOMesh
ci:atom Run native ATOM model accuracy tests
ci:vllm Run ATOM vLLM OOT model accuracy tests
ci:sglang Run ATOM SGLang model accuracy tests
ci:mesh Run ATOMesh accuracy/benchmark gates for Mesh-related changes

Without a matching ci:* label, heavy jobs are skipped when the path is not relevant or the PR is not approved.
Add labels via the sidebar or gh pr edit 3427 --add-label <label>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues were identified for the prefill score workspace changes.

No open issues 🎉

What changed in this PR

This PR moves MiniMax-M3’s native bf16/FP8 prefill index scores into a fixed workspace, avoiding growing per-chunk allocations during long prefills. It extends the workspace approach already used by MXFP4 without changing scoring kernels or selection logic.

Changes:

  • Allocate a configuration-sized workspace during metadata-builder initialization so memory profiling accounts for it.
  • Pass the workspace through sparse-prefill metadata to the Triton scorer, preserving allocation fallback for bridges.
  • Add capacity checks and GPU regression tests for workspace reuse with both selectors.
File Description
tests/​test_minimax_m3_mxfp4_decode_batch.py Updates the prefill fixture with the optional workspace field.
tests/​test_minimax_m3_index_topk.py Tests reused, poisoned workspaces against fresh score buffers.
tests/​test_minimax_m3_index_buffers.py Tests capacity bounds, verify-row sizing, and MXFP4 exclusion.
atom/​model_ops/​minimax_m3/​sparse_attn.py Carries the workspace in sparse-prefill metadata.
atom/​model_ops/​minimax_m3/​index_topk.py Lets the Triton prefill scorer write into the supplied workspace.
atom/​model_ops/​minimax_m3/​index_buffers.py Adds workspace allocation and shares prefill-capacity calculation.
atom/​model_ops/​attentions/​aiter_attention.py Allocates the workspace at initialization and attaches it to metadata.
atom/​model_ops/​attention_mha.py Passes the prefill workspace to index scoring.

Review effort: Balanced

@valar-cop valar-cop left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

a few small things..

ci green.

block_table: torch.Tensor
max_query_len: int
max_seq_len: int
score_workspace: LogitsWorkspace | None = None

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

bridge paths still allocate per chunk and stay exposed to the OOM abort, please track the follow-up or pass the workspace there too

@valarLip valarLip left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Action required before merge (5)

  1. Key the bf16/FP8 workspace off the scorer record, not off == "fp4". index_buffers.py:65 (allocate_prefill_scores): #3310 makes allocate_index_scorers return a record for nvfp4 too, so after it lands NVFP4 would also get this up-to-1 GiB workspace and never use it. Allocate only when "" not in self._index_scorers (the switch the builder already uses, aiter_attention.py:1118). Then the "workspace xor index_scorer" rule is set in one place.
  2. Coordinate with #2493 before merging. #2493 makes aiter's FlyDSL minimax_m3_index_score_prefill_flydsl the default bf16/FP8 prefill scorer, and that call takes no out=. Once it lands, Triton runs only in tests: this workspace stays resident but unused, and the per-chunk growth comes back. Whichever PR merges second must pass the workspace into the FlyDSL call, which needs aiter to accept out=. Please agree on the order with @ganyi1996ppo and say it in the description.
  3. Address valar-cop's bridge comment (sparse_attn.py:75). Open an issue for the vLLM (plugin/vllm/attention/minimax_m3_attnetion.py:691) and SGLang (plugin/sglang/minimax_m3_bridge.py:648) paths, which still allocate on every call and can still hit the OOM abort, and link it from the PR. Or pass the workspace on both paths.
  4. Fix the docstrings. Verify steps take the decode sparse path (aiter_attention.py:1111), so "or a verify step" (index_buffers.py:55) and test_verify_rows_past_the_chunk_widen_it describe a case this prefill-only workspace never sees. Either drop the max_bs * max_qlen term from the bf16/FP8 sizing or say it is only there to match MXFP4. "the bridges, which allocate per step" (sparse_attn.py:77) should say per layer call.
  5. Small cleanups. Type score_workspace / out as LogitsWorkspace | None (index_topk.py:1159, :1279). Optionally add a GPU check that a second chunk with the workspace allocates nothing (torch.cuda.memory_allocated before and after), because no test fails on main for the growth itself.

Root cause is in ATOM: the Triton scorer allocated per forward, and LogitsWorkspace (sparse_indexer_chunk.py:205) exists for exactly this. The before/after memory, TTFT and accuracy numbers are convincing. Pre-checkin is green. Heavy CI has not run yet; it needs ci:atom before approval.


Reviewed at b6cd839. Full findings list kept with the reviewer.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants