Skip to content

perf(minimax-m3): tune long-context decode topk tiles - #2979

Draft
XiaobingSuper wants to merge 1 commit into
ROCm:mainfrom
XiaobingSuper:xiaobingsuper/m3-gfx1250-decode-topk-opt
Draft

XiaobingSuper wants to merge 1 commit into
ROCm:mainfrom
XiaobingSuper:xiaobingsuper/m3-gfx1250-decode-topk-opt

Conversation

@XiaobingSuper

@XiaobingSuper XiaobingSuper commented Nov 4, 2026 •

Copy link
Copy Markdown
Contributor

Part of https://github.com/wuhuikx/LLM-Booster/issues/341. Independent of #2873: this changes only the decode selector tile, while #2873 changes native score split policy for small grids.

Summary

Select the packed decode top-k tile from the gfx1250 capture-time context bound. The selected tile is fixed for all graph replays. Other architectures retain tile 512; score computation, ranking, sparse-table emission, and graph-stable bounds are unchanged.

Current status

Draft: fixed-bound retesting does not show a stable production benefit, so this is not ready to merge.

MI455X e15-06, fixed max_model_len with live context varied from 32K to the capture bound, exact ordered top-k plus sparse-table emission:

  • 128K bound (max_block=1024): tile1024 vs tile512 ranged from 0.939x to 1.005x.
  • 193K bound (max_block=1544): tile1024 vs tile512 ranged from 0.958x to 1.007x.
  • All outputs were exact.
  • At 256K (max_block>=2048) the production emit path normally selects aiter first, so tile2048 is only a no-aiter/declined-shape fallback.

These fixed-bound results supersede the earlier per-context table. The branch is retained as a draft while the selector path is reassessed; it should not be reviewed as a merge candidate yet.

Validation

  • Added gfx1250 512/513 and 2047/2048 boundary coverage plus the non-gfx1250 default.
  • Focused tests: 5 passed.
  • Black and Ruff: clean.

@github-actions

github-actions Bot commented Nov 4, 2026

Copy link
Copy Markdown
Contributor

🏷️ CI Guide

Runs automatically on every eligible PR before approval:

  • ✅ Pre Checkin: Black, Ruff, catalog schema validation, non-GPU unit tests

Heavy model tests:

  • ✅ Run after the PR is approved and Pre Checkin passes
  • ✅ Run immediately when an approval review is submitted
  • ✅ Can be requested before approval with labels
Label Tests
ci:full Run all heavy PR model tests: native ATOM, vLLM, and SGLang
ci:atom Run native ATOM model accuracy tests
ci:vllm Run ATOM vLLM OOT model accuracy tests
ci:sglang Run ATOM SGLang model accuracy tests

Heavy jobs are skipped when the PR is not approved and no matching ci:* label is present.
Add labels via the sidebar or gh pr edit 2979 --add-label <label>

@XiaobingSuper
XiaobingSuper force-pushed the xiaobingsuper/m3-gfx1250-decode-topk-opt branch from d34282c to 22fe592 Compare November 10, 2026 08:12
@XiaobingSuper
XiaobingSuper marked this pull request as ready for review November 10, 2026 08:12
Copilot AI balanced review requested due to automatic review settings November 10, 2026 08:12

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.

@XiaobingSuper
XiaobingSuper marked this pull request as draft November 10, 2026 08:13
@XiaobingSuper
XiaobingSuper force-pushed the xiaobingsuper/m3-gfx1250-decode-topk-opt branch from 22fe592 to e67548d Compare November 10, 2026 08:22
@XiaobingSuper
XiaobingSuper marked this pull request as ready for review November 10, 2026 08:22
Copilot AI balanced review requested due to automatic review settings November 10, 2026 08:22

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.

@XiaobingSuper
XiaobingSuper force-pushed the xiaobingsuper/m3-gfx1250-decode-topk-opt branch from e67548d to 686aa8e Compare November 10, 2026 08:28
Copilot AI balanced review requested due to automatic review settings November 10, 2026 08:28

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.

@XiaobingSuper
XiaobingSuper marked this pull request as draft November 10, 2026 08:39
@XiaobingSuper
XiaobingSuper force-pushed the xiaobingsuper/m3-gfx1250-decode-topk-opt branch from 686aa8e to 5228dd4 Compare November 10, 2026 08:40
@XiaobingSuper
XiaobingSuper marked this pull request as ready for review November 10, 2026 08:40
Copilot AI balanced review requested due to automatic review settings November 10, 2026 08:40

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.

@valar-cop valar-cop left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

a few small things..

decode latency at one fixed max_model_len across varying live context?

ci green.

if arch != "gfx1250" or max_block <= 512:
return DECODE_TOPK_BLOCK_SIZE_K
if max_block >= 2048:
return 2048

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

at 2048 with emit the shipped path already takes aiter, so this tile is only the no-aiter fallback? leave a comment so we know.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added the fallback-path comment in 40af5ad. The fixed-bound retest also confirms that at 2048+ the production emit path normally selects aiter, so this tile is not a production optimization there.

DECODE_TOPK_NUM_WARPS = 8


def _decode_topk_block_size_k(arch: str, max_block: int) -> int:

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nothing pins the new policy.. add a parametrized test for the 512/513 and 2047/2048 boundaries and the non-gfx1250 default?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added parameterized coverage for gfx1250 512/513 and 2047/2048, plus the non-gfx1250 default, in 40af5ad. Focused tests, Black, and Ruff pass.

Co-authored-by: Cursor <cursoragent@cursor.com>
@XiaobingSuper
XiaobingSuper force-pushed the xiaobingsuper/m3-gfx1250-decode-topk-opt branch from 5228dd4 to 40af5ad Compare November 11, 2026 04:18
Copilot AI balanced review requested due to automatic review settings November 11, 2026 04:18

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.

@XiaobingSuper
XiaobingSuper marked this pull request as draft November 11, 2026 04:18
@zufayu
zufayu requested review from JiaoliangYu and removed request for valarLip November 11, 2026 05:48
@XiaobingSuper
XiaobingSuper marked this pull request as ready for review November 11, 2026 10:37
@XiaobingSuper
XiaobingSuper marked this pull request as draft November 12, 2026 09:16
@zufayu
zufayu removed the request for review from JiaoliangYu November 13, 2026 00:03

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants