Repository navigation
[Feat] http_server: Make the multi-worker ASGI app path configurable - #44583
Merged
Merged
Conversation
With more than one tokenizer worker, Uvicorn and Granian spawn worker processes that re-import the ASGI app from a fixed import string, "sglang.srt.entrypoints.http_server:app". A package that embeds SGLang and registers extra routes on that app never sees them in the workers, because each worker loads the bare module-level app. Add SGLANG_HTTP_SERVER_APP_PATH to the Envs registry, read it at both multi-worker launch sites, and document it with the other HTTP-server variables, so an embedding package can point the workers at a module that extends the SGLang app. The default is unchanged, and the single-worker path still passes the in-process app object. Co-authored-by: joshuadeng <joshuadeng@users.noreply.github.com>
metamergebot
requested review from
CatherineSue,
JustinTong0323,
ispobock,
merrymercy,
slin1237,
sogalin,
wisclmy0611 and
zijiexia
as code owners
October 15, 2026 21:38
Contributor
|
Preview deployment for your docs. Learn more about Mintlify Previews.
💡 Tip: Enable Automations to automatically generate PRs for you. |
Collaborator
Author
|
This port clears our bar on gate✓ lint✓ build✓ A✓ B✓ C✓ other✓ · accel ✗amd,npu @fei-xx @merrymercy @joshuadeng — could one of you take a look and merge it if you are happy? Nothing else is waiting on our side. |
cctry
approved these changes
Oct 16, 2026
cctry
left a comment
Collaborator
There was a problem hiding this comment.
LGTM, OSS sync of an internal change.
This was referenced Oct 17, 2026
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
With more than one tokenizer worker (
--tokenizer-worker-num > 1), both the Uvicorn and the Granian launch paths hand the server a fixed import string,sglang.srt.entrypoints.http_server:app, and every worker process re-imports that module-level app. A package that embeds SGLang and registers extra routes on the app in the parent process never sees those routes in the workers, because each worker loads the bare SGLang app. There is currently no way to point the workers at an extended app without forking the launch code.Modifications
SGLANG_HTTP_SERVER_APP_PATHto theEnvsregistry (HTTP server section), defaulting tosglang.srt.entrypoints.http_server:app.http_server.py: the Graniantargetand theuvicorn.run(...)import string.docs/docs/references/environment_variables.mdx.appobject; with the variable unset, behaviour is byte-for-byte the same as before.Example: a package
my_pkg.serverthat doesfrom sglang.srt.entrypoints.http_server import appand adds its own routes can now runSGLANG_HTTP_SERVER_APP_PATH=my_pkg.server:appwith multiple tokenizer workers, and every worker serves the extended app.Accuracy Tests
Not applicable; no model or kernel code is touched.
Speed Tests and Profiling
Not applicable; the change only selects which ASGI app string the multi-worker server loads.
Checklist
ruff format/ruff check/isort/codespellat the pinned versions pass on both files.)envs...override()all reachenvs.SGLANG_HTTP_SERVER_APP_PATH.get(); no new test, since the only testable contract is the pass-through of a string intouvicorn.run/Granian.)environment_variables.mdx.)CI States
Latest PR Test (Base): ✅ Run #38811398923
Latest PR Test (Extra): ❌ Run #38811398768
Latest PR Test (AMD ROCm 10): ❌ Run #38811398850