Repository navigation
Conversation
This was referenced Sep 14, 2026
1 of 7 tasks
amd-weisun
added a commit
to amd-weisun/ATOM
that referenced
this pull request
Sep 18, 2026
A CI-parity re-validation sweep found that mega (and therefore hybrid) produced near-zero GSM8K accuracy (~0.01 vs. an expected ~0.97) specifically under --tensor-parallel-size 8 --enable-expert-parallel (no --enable-dp-attention), i.e. EP built by flattening the TP group rather than the DP group. standard scored normally (0.97) on the identical topology; all three backends scored normally on TP1/DP8/EP8 (this feature's originally-validated topology). All requests completed without error -- this was silent output corruption, not a crash. Root cause: MoE.combine_outputs (atom/models/deepseek_v4.py) unconditionally all-reduces whenever the real, physical TP group size (self.tp_size) is >1, regardless of which backend produced the routed output or whether that output is already complete. Standard's fallback under dp_size<=1 (use_all2all_kernels=False) computes a genuine partial per-rank sum via the masked-local fused_moe() kernel, which this all-reduce correctly completes. Mega's dispatch/combine, by contrast, always produces an already-complete, EP-group-wide result -- so under TP8/DP1/EP8 (dp_size=1, real tp_size=8), mega's identical-across-ranks complete output got summed 8x by the same all-reduce. Fix (MegaMxfp4MoEMethod): 1. init_prepare_finalize now gates mega's backend construction on self.moe.moe_parallel_config.use_all2all_kernels, mirroring standard's own fallback: when False, self.fused_experts falls back to the standard/hybrid-resolved value (possibly None, which routes apply() to the masked-local fused_moe() kernel) instead of unconditionally installing MegaFusedExperts. 2. _process_weight_layout_after_loading previously only preserved the standard-shuffled w13_weight/w2_weight when self._hybrid_enabled was True. Once fix ROCm#1 could route non-hybrid mega to the standard fallback kernel too, that kernel needed those weights, which had been freed. Fixed by changing the gate to self._hybrid_enabled or not self.moe.moe_parallel_config.use_all2all_kernels. GPU-validated (TP8/DP1/EP8, GSM8K LIMIT=300/5-shot): tp8dp1ep8_mega: 0.0100 -> 0.9700 tp8dp1ep8_hybrid: 0.0133 -> 0.9733 Both now match tp8dp1ep8_standard (0.9700) and the TP1/DP8/EP8 controls. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This was referenced Oct 8, 2026
1 task
6 tasks done
This was referenced Nov 12, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
Technical Details
Test Plan
Test Result
Submission Checklist