Skip to content

move from private repo to ROCm - #1

Merged
valarLip merged 131 commits into
mainfrom
tmp
Nov 21, 2025
Merged

valarLip merged 131 commits into
mainfrom
tmp

Conversation

@valarLip

Copy link
Copy Markdown
Collaborator

Motivation

Technical Details

Test Plan

Test Result

Submission Checklist

amd-weisun added a commit to amd-weisun/ATOM that referenced this pull request Sep 18, 2026
A CI-parity re-validation sweep found that mega (and therefore hybrid)
produced near-zero GSM8K accuracy (~0.01 vs. an expected ~0.97)
specifically under --tensor-parallel-size 8 --enable-expert-parallel
(no --enable-dp-attention), i.e. EP built by flattening the TP group
rather than the DP group. standard scored normally (0.97) on the
identical topology; all three backends scored normally on TP1/DP8/EP8
(this feature's originally-validated topology). All requests completed
without error -- this was silent output corruption, not a crash.

Root cause: MoE.combine_outputs (atom/models/deepseek_v4.py)
unconditionally all-reduces whenever the real, physical TP group size
(self.tp_size) is >1, regardless of which backend produced the routed
output or whether that output is already complete. Standard's fallback
under dp_size<=1 (use_all2all_kernels=False) computes a genuine partial
per-rank sum via the masked-local fused_moe() kernel, which this
all-reduce correctly completes. Mega's dispatch/combine, by contrast,
always produces an already-complete, EP-group-wide result -- so under
TP8/DP1/EP8 (dp_size=1, real tp_size=8), mega's identical-across-ranks
complete output got summed 8x by the same all-reduce.

Fix (MegaMxfp4MoEMethod):
1. init_prepare_finalize now gates mega's backend construction on
   self.moe.moe_parallel_config.use_all2all_kernels, mirroring
   standard's own fallback: when False, self.fused_experts falls back
   to the standard/hybrid-resolved value (possibly None, which routes
   apply() to the masked-local fused_moe() kernel) instead of
   unconditionally installing MegaFusedExperts.
2. _process_weight_layout_after_loading previously only preserved the
   standard-shuffled w13_weight/w2_weight when self._hybrid_enabled was
   True. Once fix ROCm#1 could route non-hybrid mega to the standard
   fallback kernel too, that kernel needed those weights, which had
   been freed. Fixed by changing the gate to self._hybrid_enabled or
   not self.moe.moe_parallel_config.use_all2all_kernels.

GPU-validated (TP8/DP1/EP8, GSM8K LIMIT=300/5-shot):
  tp8dp1ep8_mega:   0.0100 -> 0.9700
  tp8dp1ep8_hybrid: 0.0133 -> 0.9733
Both now match tp8dp1ep8_standard (0.9700) and the TP1/DP8/EP8 controls.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@minmengdie minmengdie mentioned this pull request Nov 12, 2026
6 tasks done
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants