Repository navigation
fix(EPv2): clamp the untuned flydsl combine block count to the xdb flag slots - #876
Merged
kawhil-amd merged 3 commits intoOct 31, 2026
Merged
Conversation
…ag slots On a device with no tuned table, lookup() falls back to _cu_scaled_default(), which set combine_block_num to the CU count. The combine kernel keeps one xdb flag per block in an array of xdb_flag_slots (256), so a part with more than 256 CUs (MI300X, 304) failed at op construction with "combine block_num 304 exceeds xdb_flag_slots 256". Clamp the fallback's combine_block_num to 256. The flag array is not resized to the block count: block 0 fills its unused tail so that calls with different block_num stay in step, which needs a fixed size. dispatch_block_num has no such limit and is unchanged. Host-only tests cover the clamp and check that the constant in tuning_configs equals intranode_kernels.xdb_flag_slots. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Contributor
Author
|
cc @jhchouuu @jeffdaily plz help to review this fix for EPv2 flydsl backend. |
jhchouuu
approved these changes
Oct 30, 2026
jhchouuu
left a comment
Collaborator
There was a problem hiding this comment.
LGTM. Tested at 7f4ad5cb, merged onto current main:
- Host. The new test fails on
main(3 failed / 2 passed) and passes with this change (5 passed). - MI355X, forced onto the untuned path as a 304-CU part. I set
_cu_countto 304 and_device_keyto None, then ranbench_ep.pywith the flydsl backend and the gather combine.- On
mainthe op dies at construction with the #794 assert ("combine block_num 304 exceeds xdb_flag_slots 256"). - With this change it builds and validates: eager and graph, 128 and 4096 tokens.
- On
- MI308X (gfx942, 80 CUs). Validates on its tuned table, and also when forced through the untuned fallback. Nothing changes there, since 80 < 256.
Two small things:
- The license header in
test_tuning_configs.pyappears twice. The second copy is cut off after "ARISING FROM," and runs straight into the docstring. - Two comments now say the opposite of the new code.
intranode_kernels.py:558says "256 == the CU count (the max combine block_num), so block_num never exceeds it", andflydsl_backend.py:177has "(== CU count, the max combine block_num)". Neither holds on a 304-CU part, which is exactly what this PR handles. Pointing them at_MAX_COMBINE_BLOCKSand the clamp would keep the next reader from undoing it.
…ents Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Contributor
Author
Thanks for review comments, all fixed now. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
Fixes #794. On a device with no tuned table, the flydsl EP v2
lookup()falls back to_cu_scaled_default(), which setscombine_block_numto the CU count. The combine kernel keeps one xdb flag per block in an array ofxdb_flag_slots(256), so a part with more than 256 CUs (MI300X has 304) fails at op construction:#795 adds a tuned MI300X table, which fixes MI300X. It leaves the fallback unchanged, so any other untuned part with more than 256 CUs still hits the assert (#795 says the issue stays open for the clamp). This PR is that clamp. It does not depend on #795 and does not touch the same lines.
Technical Details
tuning_configs.py:_cu_scaled_default()now returnscombine_block_num=min(cu, _MAX_COMBINE_BLOCKS), with_MAX_COMBINE_BLOCKS = 256.dispatch_block_numis unchanged: only combine has the slot limit.- 256 blocks is already enough to saturate P2P bandwidth, so the cap does not limit the peak the fallback can reach on a part with more CUs.
block_num, which [EPv2] flydsl backend asserts at op construction on MI300X: CU-scaled combine_block_num 304 exceeds xdb_flag_slots 256 #794 lists as the other option. Block 0 fills the unused tail[block_num, xdb_flag_slots)so that calls with differentblock_num(the per-token schedule) stay in step, and that needs a fixed size.tuning_configs.pymust not importintranode_kernels(it pulls in flydsl), so the limit is a second constant. A test checks that it equalsintranode_kernels.xdb_flag_slots.combine_block_numabove 256 still fails the assert. That is correct: the user asked for something the kernel cannot do.A clamped fallback is only a safe default, not a fast one. #795's table shows the CU fallback with combine capped at 256 is about a third slower than a tuned schedule at large token counts on MI300X.
Test Plan
New host-only test
tests/python/ops/dispatch_combine_v2/test_tuning_configs.py(no GPU needed):Nonetakes the same clamped path._MAX_COMBINE_BLOCKS == intranode_kernels.xdb_flag_slots.Without the change, the 304-CU case and the other two tests that use the constant fail.
pre-commitpasses on the changed files.Not covered: I have no MI300X here (the run was on MI355X), so I did not construct a real
EpDispatchCombineOpon a 304-CU part. The test checks the geometry thatlookup()returns, not the kernel assert itself.🤖 Generated with Claude Code