Repository navigation
fix(rocm): DeepSeek-V4 asymmetric SwiGLU clamp in the HIP Triton MoE path - #41358
Open
xiaohong42 wants to merge 1 commit into
Open
xiaohong42 wants to merge 1 commit into
xiaohong42 wants to merge 1 commit into
Conversation
DeepSeek-V4 clamps gate and up asymmetrically before SiLU: gate at `max` only, up at both ends. The fused `silu_and_mul_clamp` kernel that implements this is CUDA/XPU-only, and the unfiltered-expert path guards it with `assert _is_cuda or _is_xpu`, so on HIP any DSV4 MoE that reaches that path (expert parallelism off, e.g. --ep-size 1) failed on the assertion. Apply the same clamp explicitly on HIP before `silu_and_mul`. The CUDA/XPU assert is kept: the new branch catches HIP ahead of it, so it still holds, and it keeps documenting which targets own the fused path. Only the unfiltered path changes; `filter_expert` already passed the limit through to Triton. Co-authored-by: Cursor <cursoragent@cursor.com> Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Co-authored-by: Cursor <cursoragent@cursor.com>
xiaohong42
requested review from
BBuf,
Edwardf0t1,
Fridge003,
HaiShaw,
Ying1123,
ch-wan,
ispobock and
merrymercy
as code owners
September 26, 2026 16:11
This was referenced Sep 29, 2026
This was referenced Nov 9, 2026
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
DeepSeek-V4 clamps gate and up asymmetrically before SiLU (gate at
maxonly, up at both ends). In the Triton MoE runner the fusedsilu_and_mul_clampthat implements this is CUDA/XPU-only, and the unfiltered-expert path guards it withassert _is_cuda or _is_xpu, so on HIP any DSV4 MoE that reaches that path (expert parallelism off, e.g.--ep-size 1) fails on the assertion.Modifications
Apply the same clamp explicitly on HIP before
silu_and_mul. The CUDA/XPU branch and its assert are unchanged;filter_expert=True(EP > 1) already passed the limit through and is untouched.Accuracy Tests
test/registered/amd/test_dsv4_triton_swiglu.pyrunsfused_experts_impl(..., filter_expert=False, swiglu_limit=10)on HIP and compares againstsilu(gate.clamp(max=10)) * up.clamp(-10, 10). The earlier revision of the test patchedfused_moe.get_tp_group, which does not exist onmain, so it raised before reaching the code under test. It now patches whichever TP-group accessor the module has (get_parallelonmain,get_tp_grouponsglang-miles). Passes on MI308X (gfx942) against thesglang-miles-based integration tree.Speed Tests and Profiling
Only the HIP unfiltered-expert path changes, which previously failed on an assertion, so there is no prior speed to compare against. The clamp is two elementwise ops before
silu_and_mul.Checklist
Scope
Optional for the miles 2-node recipe, which runs with
--ep-size 4and never takes this path. It matters for single-GPU or EP=1 DeepSeek-V4 serving on ROCm.Supersedes #40579.