-
Notifications
You must be signed in to change notification settings - Fork 1.9k
[SGLang] Add support between mcore0.11 and sglang #1055
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[SGLang] Add support between mcore0.11 and sglang #1055
Conversation
Hi, BearBiscuit05, Could you confirm if this code commit has successfully cleared the preliminary QA validation and is currently functioning as intended in the test environment? |
Hello, this code currently cannot run. The mcore runtime environment requires PyTorch 2.6, but SGLang has issues in this environment. |
Thank you for your kind reply. I read the code, but found that it does not take expert parallelism into account. Will this be supported in the future? |
Yes, mcore in verl is no support for ep yet, and future plans will depend on the progress of mcore. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
@zhaochenyang20 LGTM. |
Shall we merge this @ocss884 @zhaochenyang20? |
@BearBiscuit05 @vermouth1992 thanks so much for help! |
Yes, LGTM |
Based on the ongoing alignment between mcore and vllm volcengine#851 , I believe we can simultaneously advance the alignment between mcore and sglang, as their interfaces are similar. In the end, we will only need to obtain a generator parameter. [link](sgl-project/sglang#5345)
Based on the ongoing alignment between mcore and vllm #851 , I believe we can simultaneously advance the alignment between mcore and sglang, as their interfaces are similar. In the end, we will only need to obtain a generator parameter. link