Repository navigation
[RL] Stream LoRA adapter weights through the weight-update session - #41044
yueming-yuan wants to merge 7 commits into
Conversation
Port the sglang-miles LoRA weight-update line to main as its final state:
- Streamed adapter updates: register_lora_adapter creates or refreshes an
adapter's identity and config (weights zeroed); the bytes then arrive as
"{lora_name}:{hf_key}"-prefixed tensors in the ordinary
update_weights_from_{tensor,distributed} stream, are stashed for the
session, and are applied whole (optionally checksum-verified) at
end_weight_update. begin_weight_update(sync_base=False) opens an
adapter-only session that skips the base unpack/finalize and rejects base
tensors.
- Staged publication: a defer_publish registration keeps the name unservable
until its session commits; such sessions run under the reader lock,
concurrent with generation, and an abort discards the stash and the
pending identities. Requests may carry lora_backfill_paths so a fresh
engine can serve a published adapter from disk.
- Multi-LoRA lease redesign: every terminal path releases a request's LoRA
lease exactly once through _finalize_lora_lease; the registry tracks
reloadable vs wire-only adapters and the over-cap eviction is one helper.
- load_lora_adapter_from_distributed / upsert (the intermediate API on
sglang-miles) is superseded by the stream and not added; main's
load_lora_adapter_from_tensors stays.
Squash of sglang-miles #27268, #31759, #30913, #33299, #36384, #38165,
rebased onto the _WeightUpdateSession design of #40777.
Co-authored-by: Yusheng Su <yushengsu.thu@gmail.com>
Co-authored-by: gongyisheng <yishenggong9437@gmail.com>
Co-authored-by: Mathew Han <mathewjhan@gmail.com>
|
/tag-run-ci-label |
|
/rerun-test test/registered/unit/lora/test_lora_stream_update.py test/registered/unit/lora/test_lora_staged_publication.py test/registered/unit/lora/test_lora_lease.py test/registered/unit/lora/test_lora_upsert.py test/registered/unit/lora/test_eviction_policy.py test/registered/unit/lora/test_lora_manager_tied_lm_head.py test/registered/unit/lora/test_lora_moe_inplace_unit.py test/registered/unit/lora/test_uno_inactive_lora_batch.py test/registered/unit/managers/test_msgpack_ipc_roundtrip.py test/registered/unit/managers/test_scheduler_weight_version_tracking.py test/registered/unit/managers/test_tokenizer_manager_rid_cleanup.py test/registered/unit/managers/test_tokenizer_config_updates.py test/registered/unit/rl/test_distributed_weight_update_spec_worker.py test/registered/rl/test_lora_load_from_tensor.py test/registered/rl/test_update_weights_from_tensor.py test/registered/rl/test_update_weights_from_distributed.py test/registered/rl/test_weight_checker_e2e.py test/registered/lora/test_lora_update.py test/registered/lora/test_lora_eviction.py test/registered/lora/test_lora_overlap_loading.py test/registered/lora/test_lora_drainer.py test/registered/lora/test_lora_openai_api.py test/registered/lora/test_lora_qwen3.py test/registered/lora/test_lora_tp.py test/registered/lora/test_multi_lora_backend.py test/registered/lora/test_lora_spec_decoding.py test/registered/lora/test_lora_hf_sgl_logprob_diff.py |
|
Results for 🚀 🚀 🚀 🚀 🚀 ⛔ ⛔ |
… reads unload_lora_adapter checks --enable-lora and dp_size through get_lora() / get_parallel(), not server_args, and rejects names awaiting publication, so the bare TokenizerManager needs the context override and an empty _pending_lora_publications.
|
/rerun-test test/registered/unit/lora/test_lora_lease.py test/registered/unit/lora/test_lora_stream_update.py test/registered/unit/lora/test_lora_staged_publication.py test/registered/unit/lora/test_lora_upsert.py test/registered/unit/managers/test_scheduler_weight_version_tracking.py test/registered/unit/rl/test_distributed_weight_update_spec_worker.py |
|
Results for 🚀 ⛔ |
sglang-miles turned an explicit unload into a DELETE of the lora_ref_cache entry. main's dynamic-update contract (test_lora_update) is that a request for an unloaded disk-backed adapter reloads it implicitly, so the catalog entry stays; wire-only adapters are already refused by the reloadable check. Also hand the version-tracking fixture an iterable weight list, since the distributed path now splits the received tensors before loading.
|
/rerun-test test/registered/unit/lora/test_lora_lease.py test/registered/unit/lora/test_lora_stream_update.py test/registered/unit/lora/test_lora_staged_publication.py test/registered/unit/lora/test_lora_upsert.py test/registered/unit/managers/test_scheduler_weight_version_tracking.py test/registered/unit/rl/test_distributed_weight_update_spec_worker.py test/registered/lora/test_lora_update.py test/registered/lora/test_lora_drainer.py test/registered/lora/test_lora_openai_api.py test/registered/lora/test_lora_qwen3.py test/registered/lora/test_multi_lora_backend.py test/registered/lora/test_lora_spec_decoding.py test/registered/lora/test_lora_hf_sgl_logprob_diff.py |
|
Results for 🚀 🚀 🚀 ⛔ ⛔ |
|
/rerun-test test/registered/unit/managers/test_scheduler_weight_version_tracking.py test/registered/unit/rl/test_distributed_weight_update_spec_worker.py |
|
Results for 🚀 ⛔ |
update_weights_from_* inside a session hold the version they carry until end_weight_update, so a version never names a half-applied refit; the tracking test asserted the pre-session immediate record.
|
/rerun-test test/registered/unit/managers/test_scheduler_weight_version_tracking.py test/registered/unit/rl/test_distributed_weight_update_spec_worker.py test/registered/unit/lora/test_lora_stream_update.py |
|
Results for 🚀 ⛔ |
The fixture opened a session for every path; inside one the version is held until commit, which is not the checkpoint-engine IPC flow.
|
/rerun-test test/registered/unit/managers/test_scheduler_weight_version_tracking.py test/registered/unit/rl/test_distributed_weight_update_spec_worker.py |
|
Results for 🚀 ⛔ |
|
/rerun-test test/registered/unit/managers/test_scheduler_weight_version_tracking.py test/registered/unit/rl/test_distributed_weight_update_spec_worker.py |
|
Results for 🚀 ⛔ |
|
/rerun-test test/registered/unit/managers/test_scheduler_weight_version_tracking.py test/registered/unit/rl/test_distributed_weight_update_spec_worker.py |
|
Results for 🚀 ⛔ |
Motivation
RL training that fine-tunes LoRA adapters needs to push adapter weights into the rollout engine on every step, alongside or instead of base weights, without a second transport, without a full reload, and without serving a half-written adapter. On main the only route is
load_lora_adapter_from_tensors, which is a separate whole-adapter IPC call outside the weight-update session.Modifications
POST /register_lora_adapter(alsoEngine.register_lora_adapter) creates or refreshes an adapter's identity and config with zeroed weights. The weights then travel as{lora_name}:{hf_key}-prefixed tensors in the ordinaryupdate_weights_from_{tensor,distributed}stream: the scheduler stashes them for the session and applies each adapter whole atend_weight_update, verifying an optionalexpected_lora_checksumsmanifest (name set + sha256).begin_weight_update(sync_base=False)opens an adapter-only session that skips the base unpack/finalize and rejects base tensors.register_lora_adapter(defer_publish=True)keeps a fresh name unservable until its session commits; such sessions take the reader lock and run alongside generation,end_weight_update(abort=True)discards the stash and the pending identities, and requests may carrylora_backfill_pathsso a fresh engine serves a published adapter from disk._finalize_lora_lease), the registry distinguishes reloadable (disk-backed) from wire-only adapters, and over-cap eviction is one helper (_evict_lru_over_cap).weight_updater.py: the session struct gainssync_baseandpending_version; the version reported by buckets is recorded only when the session commits.test_lora_stream_update.py,test_lora_staged_publication.py,test_lora_lease.py,test_lora_upsert.py(CPU units) andtest_lora_load_from_tensor.py(e2e, now through the stream).Squash of sglang-miles #27268, #31759, #30913, #33299, #36384, #38165, rebased onto the
_WeightUpdateSessiondesign of #40777 (authors credited as co-authors). Not carried over:load_lora_adapter_from_distributed/upsert(the intermediate API on sglang-miles, superseded by the stream). main'sload_lora_adapter_from_tensorsis unchanged.Deliberately different from sglang-miles: an explicit
unload_lora_adapterkeeps the adapter's entry in the reload catalog (lora_ref_cache). sglang-miles made it a delete; main's dynamic-update contract (test_lora_update) is that a later request for an unloaded disk-backed adapter reloads it implicitly, and wire-only adapters are already refused by thereloadablecheck.Testing
pre-commit clean; file runs via
/rerun-teston the LoRA unit tests, the LoRA e2e set, and the weight-update e2e tests.CI States
Latest PR Test (Base): ❌ Run #35969117738
Latest PR Test (Extra): ❌ Run #35969117453
Latest PR Test (AMD ROCm 10): ❌ Run #35969117682