Skip to content

fix: support explicit Qwen3 MoE router precision (after cutoff) - #40997

Open
yueming-yuan wants to merge 37 commits into
sglang-milesfrom
yueming/qwen3-router-dtype
Open

yueming-yuan wants to merge 37 commits into
sglang-milesfrom
yueming/qwen3-router-dtype

Conversation

@yueming-yuan

@yueming-yuan yueming-yuan commented Sep 24, 2026 •

Copy link
Copy Markdown
Collaborator

Support explicit FP32 Qwen3-MoE router logits. Deferred until after the v0.1.1 cutoff, paired with Miles #3697.

Gate weights and inputs use FP32 only when configured. Covers normal, DeepEP, MegaMoE and staged execution; defaults remain unchanged. Reject unsupported dtypes and quantized-gate overrides. One file, +15/-4.

The expert-index bounds and complete fused shared-expert pool/loading fixes have moved to cutoff PR #41074. Rebased onto the current sglang-miles base; only the router changes remain.

Validation: real-weight router replay covered 6 layers × 2 dtypes × 3 execution paths. Earlier Miles #3584 Qwen full-model CI exercised the router configuration and exposed the paired P2P staging fix now in #3697. Full local pre-commit passed after rebasing. A CPU probe of all four production gate-call expressions passes in default and FP32 modes; before the added MegaMoE cast it fails with a BF16/FP32 matmul mismatch. This is call-site coverage, not MegaMoE GPU execution. Fresh rebased-head GPU CI and throughput validation remain pending.

Model tracking and release split.


CI States

Latest PR Test (Base): ❌ Run #37040379385
Latest PR Test (Extra): ❌ Run #37040378777
Latest PR Test (AMD ROCm 10): ❌ Run #37040379094

@yueming-yuan yueming-yuan changed the title fix: support explicit Qwen3 MoE router precision fix: correct multi-LoRA routing and fused shared expert loading Sep 24, 2026
@yueming-yuan yueming-yuan changed the title fix: correct multi-LoRA routing and fused shared expert loading fix: support explicit Qwen3 MoE router precision (after cutoff) Sep 25, 2026
Kangyan-Zhou and others added 12 commits September 29, 2026 17:30
…nd pin SentencePiece for InternVL (#41777) (#41788)

Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
…ty check to a warning (#41681) (#41802)

Co-authored-by: Mohammad Miadh Angkad <176301910+mmangkad@users.noreply.github.com>
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
…allel math (#18639, #30742)

Co-authored-by: Xinyu Jiang <xinyuj2@andrew.cmu.edu>
…8642, #36404)

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.