Repository navigation
[Scheduler] Track prefill progress as prefix_len - #42824
Conversation
|
Follow-up note (not in this PR): for requests that already hold their row (later chunks, borrowed session records), |
|
/tag-and-rerun-ci |
|
/rerun-test registered/chunked_prefill/test_scripted_core_1gpu.py registered/chunked_prefill/test_scripted_swa_1gpu.py registered/scheduler/test_mixed_chunked_prefill.py registered/scheduler/test_retract_decode.py registered/radix_cache/test_radix_cache_hit.py registered/radix_cache/unified_radix_tree/test_unified_radix_cache_kl_full.py registered/radix_cache/unified_radix_tree/test_unified_radix_cache_kl_swa.py registered/radix_cache/unified_radix_tree/test_unified_radix_cache_kl_mamba.py registered/hicache/test_hicache_variants.py registered/hicache/test_hicache_storage_file_backend.py registered/sessions/test_streaming_session.py registered/dllm/test_dllm_batching_fdfo.py registered/disaggregation/test_disaggregation_basic.py registered/disaggregation/test_disaggregation_chunked_prefill_abort.py |
|
🚀 🚀 🚀 🚀 |
|
/rerun-test registered/dllm/test_dllm_batching_fdfo.py registered/unit/mem_cache/test_hisparse_allocator.py |
|
🚀 🚀 |
8ae6f07 to
c134d31
Compare
341737b to
7d9ff5e
Compare
main dropped Req.prefix_indices and Req.extend_range (sgl-project#42822, sgl-project#42824, sgl-project#42825) and moved the cache match into match_kv_cache. The replay bookkeeping now lives in match_kv_cache and uses req.prefix_len. The request stubs of two scheduler unit tests get the new replay fields.
Req.prefix_lenas the prefill progress cursor; replacelen(req.prefix_indices)readerscheckpointimplementations no longer write the request's prefix;checkpoint_kv_cacheadvancesprefix_lento the chunk end, published or notalloc_for_extendreads the prefix from the row for requests that already hold one, and consumesprefix_indicesfor new onesCI States
Latest PR Test (Base): 🚫 Run #37698422874
Latest PR Test (Extra): ❌ Run #37698422660
Latest PR Test (AMD ROCm 10): ⏳ Run #37698422788