Repository navigation
feat: Add model version tracking with API endpoints and response metadata - #8795
Conversation
|
Warning You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again! |
|
Hi @yitianlian, thanks for ur contribution! Could you provide a few cases of this feat, so that people with no RL background can easily understand this? |
In current RL practice, we may need to 1. interrupt the inference engine -> 2. fetch all ongoing requests -> 3. update weight -> 4. re-construct batch and do generation. Therefore each training batch can contain data itself from multiple prior policy versions and that could affect the final performance of model. We'd like to add-on one more property to monitor the data. Recommended reading: https://arxiv.org/pdf/2505.24298 |
|
Great work Chengxing! |
|
check notebook CI: |
Thanks, this makes sense. But we might want to emphasize that this WDYT? |
Yeah we may need a better naming for it. The scenario is that you need to update the instance's weight online. Just come up some: instance/model/weights_version/revision. And change the default value to string "default" or int 0. @yitianlian Any opinions on it |
Sure! I'm also concerned that users will be confused by the name. I thought As for the failed notebook CI, I’ll add an extra key in the test to ensure it passes. |
|
Hi, is there anything I need to do to help get this PR merged? |
Sounds good, I have approved already |
|
Hi, is there anything I need to do to help get this PR merged? QAQ |
|
|
||
| # Update weight version if provided and weights update was successful | ||
| if success and obj.weight_version is not None: | ||
| _update_weight_version_if_provided(obj.weight_version) |
There was a problem hiding this comment.
qq: why do we put logic here
Motivation
This PR introduces a model_version tracking capability for SGLang server deployments. With this feature, each deployed model is associated with a specific version that can be queried or updated.
It is essential to identify which version of a model is currently deployed, particularly in reinforcement learning (RL) systems, where knowing the model version for each rollout is crucial for data selection and the design of off-policy RL algorithms.
Modifications
Core Infrastructure Changes
Server Arguments & Configuration:
model_version: str = "version_0"toServerArgsdataclass inserver_args.py--model-versionCLI argument for deployment-time version specificationtokenizer_manager.server_args.model_versionas single source of the model versionAPI Response Integration:
/generateresponses to includemodel_versioninmeta_infofieldmetadata: Optional[Dict[str, Any]]field to OpenAIChatCompletionResponseandCompletionResponsemetadata.model_versionin both chat and completion OpenAI endpointsNew API Endpoints
Version Management:
GET /get_model_version: Retrieve current model versionPOST /update_model_version: Update model version with optional request abortionEnhanced Weight Update APIs:
UpdateWeightFromDiskReqInput,UpdateWeightsFromTensorReqInput, andUpdateWeightsFromDistributedReqInputwith optionalmodel_versionparameter/update_weights_from_disk,/update_weights_from_tensor, and/update_weights_from_distributedendpoints to accept and apply version updatesAccuracy Test
None
Benchmark & Profiling
None
Checklist