Repository navigation
Conversation
This was referenced Sep 15, 2026
Arist12
marked this pull request as ready for review
September 15, 2026 17:27
Arist12
requested review from
Shi-Dong,
Zhichenzzz,
fzyzcjy,
maocheng23,
yueming-yuan and
yushengsu-thu
as code owners
September 15, 2026 17:27
Arist12
force-pushed
the
amd/bridge-router-gemm-fidelity
branch
from
September 15, 2026 17:42
015c1e2 to
72e533c
Compare
Both bridge whitelists drop moe_router_use_torch_mm, so the provider keeps the TransformerConfig default False even though set_default_megatron_args forces it True for every run, and the router GEMM takes the TE bf16 branch while the args table reports torch mm. Mirror it on both paths, plus moe_router_dtype on the non-LoRA path, guarded because bridge MoE providers default it to fp32.
Arist12
force-pushed
the
amd/bridge-router-gemm-fidelity
branch
from
September 16, 2026 18:42
72e533c to
719c53e
Compare
Contributor
|
LGTM. This correctly passes the router settings to the Bridge provider. The LoRA changes overlap with the recently #3584 |
# Conflicts: # miles/backends/megatron_utils/lora/bridge.py
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Both bridge paths copy a whitelist of Megatron args onto the Megatron-Bridge provider.
moe_router_use_torch_mmis in neither list, so the provider keeps theTransformerConfigdefault
False— even though miles forces the arg on for every run:The provider is the model config, so
TopKRouterreads the unset value andRouterGatingLinearFunctiontakes the TE branch:te_general_gemmis available on ROCm, so onscripts/amd/run_gpt_oss_20b_lora.pythe routerGEMM runs on bf16 operands while the resolved args table reports
fp32+ torch mm.moe_router_dtypeis dropped by the non-LoRA whitelist in the same way. Building the providerexactly as miles does:
Megatron reports the same thing on every rank — the warning fires exactly on
num_moe_experts >= 32 and not self.moe_router_dtype, and gpt-oss-20b has 32 experts:Change
Four assignments, at the existing MoE block of each whitelist.
moe_router_use_torch_mmis copied unconditionally: nothing inmegatron.bridgeever assignsit, so no provider has an opinion to preserve.
moe_router_dtypeisNone-guarded because ~13megatron.bridgeproviders deliberately default it tofp32, and an unconditional copy woulddowngrade them.
Validation
8×MI355X gpt-oss-20b LoRA run, before/after: the warning goes 2 → 0 and the args table's values
are now the effective ones.
tests/fast/backends/megatron_utils/— 326 passed, 2 skipped. The tworuntime_argsfixturelines are required, not cosmetic: the new bare reads make the existing
test_bridge_mtp_detachmentcases fail with
AttributeErrorwithout them. Bare access matches the file's convention —Megatron auto-registers every
TransformerConfigfield as a CLI arg, so both always exist on areal parsed
args; the neighbouringgetattrguards cover miles-only or derived fields.One misconfiguration becomes loud:
--moe-router-dtype fp64together with--moe-combine-in-fp32now trips the assert at
transformer_config.py:3402, where the bridge'sfp32default previouslywon silently. No in-repo recipe is affected — every
--moe-router-dtypeunderscripts/andexamples/passesfp32.Supersedes #3257 (now closed), which covered only the LoRA-path
moe_router_dtypeline.Rebase onto the multi-LoRA v2 move
[multi-lora v2 0/7]renamedbridge_lora_helpers.pytolora/bridge.py, which is whatturned this PR red. Rebased onto
f6d0257; the conflict was purely positional, so the fourassignments are unchanged and land at the same MoE block in the new file. Re-ran
tests/fast/backends/megatron_utils/on the rebase: 314 passed, 2 skipped (the count movedwith
main, not with this change).