Repository navigation
[sglang-miles] Add allowlisted external weight-update receivers - #42271
Open
ajcasagrande wants to merge 35 commits into
Open
ajcasagrande wants to merge 35 commits into
ajcasagrande wants to merge 35 commits into
Conversation
…nd pin SentencePiece for InternVL (sgl-project#41777) (sgl-project#41788) Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
…ty check to a warning (sgl-project#41681) (sgl-project#41802) Co-authored-by: Mohammad Miadh Angkad <176301910+mmangkad@users.noreply.github.com> Co-authored-by: hnyls2002 <lsyincs@gmail.com>
…allel math (sgl-project#18639, sgl-project#30742) Co-authored-by: Xinyu Jiang <xinyuj2@andrew.cmu.edu>
…l-project#18642, sgl-project#36404) Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
…ject#22623) Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…gl-project#25141, sgl-project#29874, sgl-project#31251, sgl-project#32376) Co-authored-by: Yusheng Su <yushengsu.thu@gmail.com>
…lication (sgl-project#27268, sgl-project#31759, sgl-project#30913, sgl-project#30912, sgl-project#33299, sgl-project#36384, sgl-project#38165) Register adapter identities, stream their tensors within weight-update sessions, and publish checksum-verified fresh versions alongside generation. Preserve per-rank failure cleanup and request-prefix cancellation for adapter lifecycle management. Co-authored-by: Mathew Han <mathewjhan@gmail.com> Co-authored-by: Ethan (Yusheng) Su <yushengsu.thu@gmail.com> Co-authored-by: Yueming Yuan <yym022502@gmail.com>
…ism (sgl-project#21278, sgl-project#22663, sgl-project#33478) Expose rank topology for P2P weight mapping and create replicas using scoped runtime-context overrides. Restore the previous configuration on exit, including failed context entry. Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> Co-authored-by: yueming-yuan <yym022502@gmail.com>
…capture + meta_info threading
…RL weight checks (sgl-project#29339) Co-authored-by: yueming-yuan <yym022502@gmail.com>
…30499, sgl-project#37466, sgl-project#37887) Co-authored-by: fzyzcjy <ch271828n@outlook.com> Co-authored-by: yueming-yuan <yym022502@gmail.com>
…ed weights into a host-local checkpoint (sgl-project#30366, sgl-project#28524)
…gl-project#27110, sgl-project#38718) Co-authored-by: Shi Dong <hello@dongshi.me>
…gl-project#27723) Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Co-authored-by: yueming-yuan <yym022502@gmail.com>
…educe-fusion auto-enable
…CL all-gather path (sgl-project#36110)
…tils (sgl-project#35127) Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: Shi Dong <shi.dong@radixark.ai>
…weight checks (sgl-project#37649) Co-authored-by: Zhichenzzz <zczeng@uw.edu>
…ace in the weight checker (sgl-project#40028)
guapisolo
requested review from
1am9trash,
AgainstEntropy,
Alisehen,
AniZpZ,
ByronHsu,
DarkSharpness,
Duyi-Wang,
Edwardf0t1,
FlamingoPg,
HydraQYH,
OrangeRedeng,
ShangmingCai,
YAMY1234,
celve,
hebiao064,
hubertlu-tw,
kevin-mii,
kkHuang-amd,
niehen6174,
ping1jing2,
rainj-me and
yichiche
as code owners
October 17, 2026 00:28
guapisolo
force-pushed
the
sglang-miles
branch
from
November 1, 2026 02:54
29b34bc to
b0ed761
Compare
This was referenced Nov 7, 2026
3 tasks done
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Add a generic, allowlisted hook that lets an external weight-update receiver own the transfer for a distributed weight-update group, in place of SGLang's torch process group. RL frameworks can then bring their own transport, such as an NCCL M2N reshard managed by a control plane, without SGLang plugins and without encoding control data in group names. With no receivers configured, the default path is unchanged.
--weight-update-receiverslists the dottedmodule.functionfactory paths an engine may load. A path is matched exactly before anything is imported. With no flag, every receiver is rejected.WeightUpdateReceiverContext(model, device, tp_rank, tp_size, group_name, master_address, master_port, world_size, rank_offset, init_payload)and returns an object withreceive(payload)anddestroy(). A malformed receiver is rejected with aTypeError; if itsdestroy()also fails, that failure is chained as the cause.receive()is called, with nothing recorded, when:load_format;begin_weight_update/end_weight_updatesession.The full contract, including the lifecycle and session side effects, is documented in the RL guide (
docs/docs/advanced_features/sglang_for_rl.mdx, "External Weight-Update Receivers"), and the new request fields are added to that page's request tables. The first consumer is ModelExpress's NCCL M2N receiver for Miles.Also in this PR
engine.pyruff format: removes one blank line. The pinned ruff-format hook (v0.15.1) rewrites the file whenever it is touched.test/registered/unit/managers/test_scheduler_weight_version_tracking.pyalready failed onsglang-miles@6c94e57f9f(6 failed / 14 passed).pending_versioninside a session and records it atend_weight_update, but the old tests asserted immediate recording.receive_weights_from_distributedunstubbed.Test and Validation
io_struct, the derived-cache guard suites, and the msgpack round trip: 168 passed, 2 skipped. Both skips already exist intest_io_struct.py.CI States
Latest PR Test (Base): ❌ Run #37078371728
Latest PR Test (Extra): ❌ Run #37078371380
Latest PR Test (AMD ROCm 10): ❌ Run #37078371700