Skip to content

[SGLang] Add support between mcore0.11 and sglang #1055

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 24 commits into from
May 7, 2025

Conversation

BearBiscuit05
Copy link
Collaborator

Based on the ongoing alignment between mcore and vllm #851 , I believe we can simultaneously advance the alignment between mcore and sglang, as their interfaces are similar. In the end, we will only need to obtain a generator parameter. link

@BearBiscuit05 BearBiscuit05 changed the title [sglang] add support between mcore and sglang [WIP] add support between mcore and sglang Apr 13, 2025
@BearBiscuit05 BearBiscuit05 changed the title [WIP] add support between mcore and sglang [WIP] Add support between mcore and sglang Apr 13, 2025
@BearBiscuit05 BearBiscuit05 changed the title [WIP] Add support between mcore and sglang [WIP] Add support between mcore0.11 and sglang Apr 13, 2025
@coderkaka
Copy link

Based on the ongoing alignment between mcore and vllm #851 , I believe we can simultaneously advance the alignment between mcore and sglang, as their interfaces are similar. In the end, we will only need to obtain a generator parameter. link

Hi, BearBiscuit05, Could you confirm if this code commit has successfully cleared the preliminary QA validation and is currently functioning as intended in the test environment?

@BearBiscuit05
Copy link
Collaborator Author

Hello, this code currently cannot run. The mcore runtime environment requires PyTorch 2.6, but SGLang has issues in this environment.

@coderkaka
Copy link

Hello, this code currently cannot run. The mcore runtime environment requires PyTorch 2.6, but SGLang has issues in this environment.

Thank you for your kind reply. I read the code, but found that it does not take expert parallelism into account. Will this be supported in the future?

@BearBiscuit05
Copy link
Collaborator Author

Hello, this code currently cannot run. The mcore runtime environment requires PyTorch 2.6, but SGLang has issues in this environment.

Thank you for your kind reply. I read the code, but found that it does not take expert parallelism into account. Will this be supported in the future?

Yes, mcore in verl is no support for ep yet, and future plans will depend on the progress of mcore.

@BearBiscuit05 BearBiscuit05 marked this pull request as ready for review April 22, 2025 12:46
@BearBiscuit05 BearBiscuit05 changed the title [WIP] Add support between mcore0.11 and sglang [SGLang] Add support between mcore0.11 and sglang May 7, 2025
Copy link
Collaborator

@zhaochenyang20 zhaochenyang20 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@FrankLeeeee
Copy link

@zhaochenyang20 LGTM.

@vermouth1992
Copy link
Collaborator

Shall we merge this @ocss884 @zhaochenyang20?

@zhaochenyang20 zhaochenyang20 merged commit 312a8cb into volcengine:main May 7, 2025
26 checks passed
@zhaochenyang20
Copy link
Collaborator

@BearBiscuit05 @vermouth1992 thanks so much for help!

@ocss884
Copy link
Collaborator

ocss884 commented May 7, 2025

Shall we merge this @ocss884 @zhaochenyang20?

Yes, LGTM

@BearBiscuit05 BearBiscuit05 deleted the xya/sglang/resharding branch May 14, 2025 01:44
GitMonkey0 pushed a commit to GitMonkey0/verl that referenced this pull request Jun 14, 2025
Based on the ongoing alignment between mcore and vllm volcengine#851 , I believe
we can simultaneously advance the alignment between mcore and sglang, as
their interfaces are similar. In the end, we will only need to obtain a
generator parameter.
[link](sgl-project/sglang#5345)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants