Repository navigation
Conversation
463a0ef to
76ea15b
Compare
Under expert parallelism, run only the top-k stage of the aiter routing and sort the gates with aiter's ep_sort_routing, as ROCm/ATOM does. Non-local gates are binned out of the histogram, so the GEMMs schedule tiles for this rank's experts only and index the local weight shard directly; gate_valid still masks the dead gates in the combine. TP routing is unchanged. Signed-off-by: Rohan Potdar <rohan.potdar@amd.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
W4A4 already accepted naive EP but routed globally and indexed the local weight shard with global expert ids. Route over local experts with aiter's ep_sort_routing, as the W4A16 expert does, and mask dead gates with gate_valid. Signed-off-by: Rohan Potdar <rohan.potdar@amd.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The DSV4 hash-routing and AITER W4A16 EP selection tests exercise the MXFP4 MoE oracle, not the softplus-sqrt top-k kernels; keep them next to the other MXFP4 oracle tests. Signed-off-by: Rohan Potdar <rohan.potdar@amd.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
76ea15b to
d76a6c4
Compare
|
/ci run |
|
/amd-ci run |
|
✅ Triggered Buildkite CI #98013 for commit |
|
✅ Triggered Buildkite AMD CI #15635 for commit |
|
Hi @Rohan138, the pre-commit checks have failed. Please run: uv pip install pre-commit>=4.5.1
pre-commit install
pre-commit run --all-filesThen, commit the changes and push to your branch. For future commits, |
| gammas=None if apply_router_weight_on_input else gammas, | ||
| swizzle_mx_scale=swizzle_mx_scale, | ||
| apply_swiglu=False, | ||
| gate_valid=gate_valid, |
There was a problem hiding this comment.
cc @jpvillam-amd: #65716 moves these W4A4 GEMMs into a new _aiter_w4a4_gemms helper that takes gate_valid, and its monolithic forward calls that helper with None.
|
Related PRs: |
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Rohan Potdar <rohan.potdar@amd.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Rohan Potdar <rohan.potdar@amd.com>
|
This pull request has merge conflicts that must be resolved before it can be |
Signed-off-by: Rohan Potdar <rohan.potdar@amd.com> # Conflicts: # tests/kernels/moe/test_ocp_mx_moe.py
|
/ci run |
|
/amd-ci run |
|
✅ Triggered Buildkite CI #102257 for commit |
|
✅ Triggered Buildkite AMD CI #15819 for commit |
|
/ci retry |
|
✅ Triggered Buildkite CI #103578 for commit |
Overview
Under EP, route the AITER Triton W4A16 and W4A4 MoE over local experts only, as ROCm/ATOM does: the aiter top-k feeds
ep_sort_routing. Follow-up to #59255.Claims
triton_unfusedat every concurrency.Validation
gfx950:
triton_unfusedtriton(triton_kernels)gfx942:
Note: W4A4 EP is partially tested on gfx1250, cc @jpvillam-amd
I used vLLM's
/pr-checklistskill. (Mandatory for agents, optional for humans).AI assistance was used during the creation of this PR.
Design Fit: Minimizes impact on core components, reuses existing functionality, and justifies added complexity.
Testing and Validation: Validates the change and ensures any added tests are meaningful and reliable, with CI coverage or documented CI resource constraints and validation performed outside CI.
Code Quality and Style: Keeps code and comments clear and concise, and updates relevant documentation and examples.
Pull Request Contents: Includes a brief summary and relevant links, supports claims with evidence, explains root causes and implementation trade-offs, and follows the contributing guide.
🤖 Generated with Claude Code