Skip to content

[Fix] [Intel] moe: Route dropped slots at weight 0 - #47664

Closed
dayanandav wants to merge 2 commits into
sgl-project:mainfrom
dayanandav:xpu-ep-drop-sentinel
Closed

dayanandav wants to merge 2 commits into
sgl-project:mainfrom
dayanandav:xpu-ep-drop-sentinel

Conversation

@dayanandav

@dayanandav dayanandav commented Oct 31, 2026 •

Copy link
Copy Markdown
Contributor

Motivation

moe: Route EP-dropped expert slots to a local expert at weight 0, EP and padded rows put -1 in topk_ids; sgl-kernel-xpu fused_experts has no drop sentinel, so the device is lost on the first prefill with --ep-size > 1.

Modifications

  • moe/utils: route_dropped_experts_to_local and fold_router_weight_into_input.
  • Use them at the four XPU fused_experts call sites (fused_moe, unquant, fp8, mxfp4). Other backends are unchanged.
  • Tests: unit/layers/moe/test_moe_utils.py (CPU), xpu/test_moe_ep_drop_sentinel.py.

Accuracy Tests

gpt-oss-20b on Arc Pro B60, TP2/4 + EP2/4: greedy first tokens match TP 16/16.

Speed Tests and Profiling

EP prefill 0.70-0.77x TP on 2 cards, 0.56-0.61x on 4 (previously crashed).

Checklist


CI States

Latest PR Test (Base): ❌ Run #41569384253
Latest PR Test (Extra): ❌ Run #41569384196
Latest PR Test (AMD ROCm 10): ❌ Run #41569384214

Tip

Base CI only covers the core cases, and its workload may not exercise your change. Use /rerun-test to run the tests your change touches, or copy this prompt for an agent:

Find the registered tests this PR affects and run them with /rerun-test, following https://github.com/sgl-project/sglang/blob/main/.agents/skills/rerun-test/SKILL.md

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

@github-actions

This comment has been minimized.

… weight 0

Motivation:
EP and padded rows put -1 in topk_ids; sgl-kernel-xpu fused_experts has no
drop sentinel, so the device is lost on the first prefill with --ep-size > 1.

Modifications:
- moe/utils: route_dropped_experts_to_local maps -1 to expert 0 at weight 0.
- Use it at the four XPU fused_experts call sites (fused_moe, unquant, fp8,
  mxfp4). Other backends are unchanged.
- Tests: unit/layers/moe/test_moe_utils.py (CPU), xpu/test_moe_ep_drop_sentinel.py.

Accuracy Tests:
gpt-oss-20b on Arc Pro B60, TP2/4 + EP2/4: greedy first tokens match TP 16/16.

Speed Tests and Profiling:
EP prefill 0.70-0.77x TP on 2 cards, 0.56-0.61x on 4 (previously crashed).

Checklist:
- [x] pre-commit
- [x] unit tests
- [x] accuracy and speed results

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
@dayanandav
dayanandav force-pushed the xpu-ep-drop-sentinel branch from 6421695 to d90a81d Compare October 31, 2026 15:06
@github-actions

This comment has been minimized.

@dayanandav dayanandav changed the title [Fix][Intel] Route dropped slots at weight 0 [Fix] [Intel] moe: Route dropped slots at weight 0 Nov 3, 2026
airMeng pushed a commit to airMeng/sglang that referenced this pull request Nov 4, 2026
@dayanandav

Copy link
Copy Markdown
Contributor Author

@siju-samuel @ANSHUMAN87 help to review.

@dayanandav

Copy link
Copy Markdown
Contributor Author

Merged as part of #47956 consolidation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

quant LLM Quantization

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant