Repository navigation
Conversation
…arallelism Models' set_eagle3_layers_to_capture / set_dflash_layers_to_capture return early on non-last PP stages, so capture layers on earlier stages are never collected and the draft silently receives incomplete aux hidden states. Fail at setup with NotImplementedError instead. Single-layer EAGLE/MTP drafts are unaffected. Tracking: sgl-project#39634
|
/rerun-test test/registered/unit/model_executor/model_runner_components/test_attention_backend_setup.py test/registered/unit/model_executor/model_runner_components/test_spec_aux_hidden_state.py test/registered/spec/eagle/test_spec_eagle.py |
|
Results for 🚀 🚀 |
…ain guard pp_size is required like the other keyword arguments so a future caller cannot silently disable the guard by omitting it. The new test cases were appended after the __main__ block and never ran under `python3 test_file.py`.
|
Closing: superseded by #40045, which made PP prefill + DSpark aux hidden-state capture a supported path that this guard would have rejected. The remaining silent-skip risk for EAGLE3/DFLASH capture on non-last PP stages will be addressed by relaying the captured states across stages in a follow-up PR instead of rejecting the configuration. |
Motivation
Part of #39634 (P2 hygiene).
set_eagle3_layers_to_capture/set_dflash_layers_to_capturein the target models return early when the stage is not the last PP rank. Under pipeline parallelism the default capture layers (2,n/2,n-3) live on earlier stages, so nothing is captured there and the draft silently gets incomplete aux hidden states. Today this is mostly masked by the PP+spec validation gate, but nothing in the runtime protects it once that gate is loosened (#39378, #39602, #38698).Modifications
configure_aux_hidden_state_capturetakespp_sizeand raisesNotImplementedErrorwhen aux hidden-state capture (EAGLE3 / DFLASH / DSPARK) is requested withpp_size > 1. Single-layer EAGLE/MTP drafts are unaffected.ModelRunnerpassespp_size=self.ps.pp_size(pure pass-through).test_attention_backend_setup.py: the rejection and the negative branch.Checklist
CI States
Latest PR Test (Base): ❌ Run #35034305511
Latest PR Test (Extra): ❌ Run #35034305267
Latest PR Test (AMD ROCm 10): ❌ Run #35034305373