Repository navigation
[MORI] Add MORI EP V2 support #61361
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from 32 commits
Commits
Show all changes
50 commits
Select commit
Hold shift + click to select a range
285d899
add direct MORI EPv2 backend
inkcherry 20d8842
skip attention preshuffle on gfx1250
inkcherry 68c3359
clear unused MORI EPv2 receive rows
inkcherry f7499b1
clear unwritten Triton MoE output slots
inkcherry b6e3fa7
consolidate Triton MoE workspace shape checks
inkcherry ad66023
isolate MoE activation scales from unused scratch
inkcherry bb28fdb
trim redundant MoE test prose
inkcherry b0322eb
consolidate MoE workspace shape checks
inkcherry 3afb93a
fix MORI EPv2 configuration for non-gated experts
inkcherry e8ebcaf
allow MORI EPv2 across one LSA fabric
inkcherry 308fcfa
Bound MORI EPv2 receive rows and harden its start-up
ChengYM5685 59771d1
Size MORI EPv2 CCO windows from op arenas
ChengYM5685 2af0662
Let experts write MORI EPv2 combine staging and trim receive-row work
ChengYM5685 887c4fb
Reject experts MORI EPv2 cannot feed, and tidy the backend
ChengYM5685 bc75e08
Add a mori all2all backend that picks MORI EPv2 or EPv1
ChengYM5685 84d7e69
Merge branch 'main' into feat/mori-epv2
ChengYM5685 c3d2286
Resolve the mori all2all backend in the ROCm platform hook
ChengYM5685 3c2b208
Broadcast the CCO unique ID from the EP group's first rank
ChengYM5685 8c00ce7
Merge branch 'main' into feat/mori-epv2
ChengYM5685 a1c33c0
Keep MORI EPv2 under MORI_ENABLE_SDMA on gfx1250
ChengYM5685 7dbe5a3
Drop the Triton MoE unrouted-slot changes
ChengYM5685 9f07df7
Document how to choose among the MoRI backends
ChengYM5685 88efbfc
Point users from the legacy MoRI backends to mori
ChengYM5685 7283714
Merge branch 'main' into feat/mori-epv2
ChengYM5685 ca13162
Name the MoRI EPv1 kernel flag after EPv1
ChengYM5685 d238711
Point MORI EPv2 expert rejections at VLLM_MORI_EP_V2=0
ChengYM5685 9fef971
Correct the MoRI EPv1 fallback for quantized-input experts
ChengYM5685 d7b3208
Merge branch 'main' into feat/mori-epv2
ChengYM5685 921b876
Merge branch 'main' into feat/mori-epv2
ChengYM5685 94a08bd
Merge branch 'main' into feat/mori-epv2
vllm-agent e648839
[Test] Set all2all_backend in the ROCm mm prefix-LM config test
ChengYM5685 c233884
Merge branch 'main' into feat/mori-epv2
vllm-agent bb62ddb
Merge remote-tracking branch 'upstream/main' into feat/mori-epv2
ChengYM5685 a793d2c
Drop gfx1250 bring-up changes unrelated to MORI EPv2
ChengYM5685 2d65093
Merge branch 'main' into feat/mori-epv2
shen-shanshan 62b99b0
Deprecate mori_high_throughput and mori_low_latency
ChengYM5685 2eab8b4
Merge remote-tracking branch 'upstream/main' into feat/mori-epv2
ChengYM5685 c383fe8
Fix DeepSeek-V3.2 on MoRI EPv2 and the host-span check
ChengYM5685 1ccc073
Merge branch 'main' into feat/mori-epv2
shen-shanshan 4b73f2e
Fall back to MoRI EPv1 where EPv2 cannot run
ChengYM5685 2e26247
Merge remote-tracking branch 'chengym/feat/mori-epv2' into feat/mori-…
ChengYM5685 d35f5b9
Docs: list every MoRI EPv2 fallback and its FP8 support
ChengYM5685 fbdf2ae
Test: run the pinned-EPv2 mori backend without per-tensor FP8
ChengYM5685 a9ce493
Merge branch 'main' into feat/mori-epv2
vllm-agent 921086e
Detect compressed-tensors and Quark per-tensor FP8; find HIP via Cuda…
ChengYM5685 715635b
Keep MoRI EPv2 ops alive in the manager; match the arena estimate to …
ChengYM5685 0503177
Merge branch 'main' into feat/mori-epv2
vllm-agent ed7689a
Merge upstream main to fix depyf fullgraph CI
ChengYM5685 5782d84
Merge branch 'main' into feat/mori-epv2
vllm-agent 94603b3
Merge branch 'main' into feat/mori-epv2
vllm-agent File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.