Repository navigation
perf(minimax-m3): tune long-context decode topk tiles - #2979
XiaobingSuper wants to merge 1 commit into
Conversation
🏷️ CI GuideRuns automatically on every eligible PR before approval:
Heavy model tests:
|
d34282c to
22fe592
Compare
22fe592 to
e67548d
Compare
e67548d to
686aa8e
Compare
686aa8e to
5228dd4
Compare
valar-cop
left a comment
There was a problem hiding this comment.
a few small things..
decode latency at one fixed max_model_len across varying live context?
ci green.
| if arch != "gfx1250" or max_block <= 512: | ||
| return DECODE_TOPK_BLOCK_SIZE_K | ||
| if max_block >= 2048: | ||
| return 2048 |
There was a problem hiding this comment.
at 2048 with emit the shipped path already takes aiter, so this tile is only the no-aiter fallback? leave a comment so we know.
There was a problem hiding this comment.
Added the fallback-path comment in 40af5ad. The fixed-bound retest also confirms that at 2048+ the production emit path normally selects aiter, so this tile is not a production optimization there.
| DECODE_TOPK_NUM_WARPS = 8 | ||
|
|
||
|
|
||
| def _decode_topk_block_size_k(arch: str, max_block: int) -> int: |
There was a problem hiding this comment.
nothing pins the new policy.. add a parametrized test for the 512/513 and 2047/2048 boundaries and the non-gfx1250 default?
There was a problem hiding this comment.
Added parameterized coverage for gfx1250 512/513 and 2047/2048, plus the non-gfx1250 default, in 40af5ad. Focused tests, Black, and Ruff pass.
Co-authored-by: Cursor <cursoragent@cursor.com>
5228dd4 to
40af5ad
Compare
Part of https://github.com/wuhuikx/LLM-Booster/issues/341. Independent of #2873: this changes only the decode selector tile, while #2873 changes native score split policy for small grids.
Summary
Select the packed decode top-k tile from the gfx1250 capture-time context bound. The selected tile is fixed for all graph replays. Other architectures retain tile 512; score computation, ranking, sparse-table emission, and graph-stable bounds are unchanged.
Current status
Draft: fixed-bound retesting does not show a stable production benefit, so this is not ready to merge.
MI455X e15-06, fixed
max_model_lenwith live context varied from 32K to the capture bound, exact ordered top-k plus sparse-table emission:max_block=1024): tile1024 vs tile512 ranged from 0.939x to 1.005x.max_block=1544): tile1024 vs tile512 ranged from 0.958x to 1.007x.max_block>=2048) the production emit path normally selects aiter first, so tile2048 is only a no-aiter/declined-shape fallback.These fixed-bound results supersede the earlier per-context table. The branch is retained as a draft while the selector path is reassessed; it should not be reviewed as a merge candidate yet.
Validation