Skip to content

[Spec][PP] Reject EAGLE3/DFLASH aux hidden-state capture under pipeline parallelism - #39644

Closed
YAMY1234 wants to merge 2 commits into
sgl-project:mainfrom
YAMY1234:pp-spec-reject-aux-capture
Closed

YAMY1234 wants to merge 2 commits into
sgl-project:mainfrom
YAMY1234:pp-spec-reject-aux-capture

Conversation

@YAMY1234

@YAMY1234 YAMY1234 commented Sep 15, 2026 •

Copy link
Copy Markdown
Collaborator

Motivation

Part of #39634 (P2 hygiene). set_eagle3_layers_to_capture / set_dflash_layers_to_capture in the target models return early when the stage is not the last PP rank. Under pipeline parallelism the default capture layers (2, n/2, n-3) live on earlier stages, so nothing is captured there and the draft silently gets incomplete aux hidden states. Today this is mostly masked by the PP+spec validation gate, but nothing in the runtime protects it once that gate is loosened (#39378, #39602, #38698).

Modifications

  • configure_aux_hidden_state_capture takes pp_size and raises NotImplementedError when aux hidden-state capture (EAGLE3 / DFLASH / DSPARK) is requested with pp_size > 1. Single-layer EAGLE/MTP drafts are unaffected.
  • ModelRunner passes pp_size=self.ps.pp_size (pure pass-through).
  • Two cases added to test_attention_backend_setup.py: the rejection and the negative branch.

Checklist


CI States

Latest PR Test (Base): ❌ Run #35034305511
Latest PR Test (Extra): ❌ Run #35034305267
Latest PR Test (AMD ROCm 10): ❌ Run #35034305373

…arallelism

Models' set_eagle3_layers_to_capture / set_dflash_layers_to_capture return early
on non-last PP stages, so capture layers on earlier stages are never collected
and the draft silently receives incomplete aux hidden states. Fail at setup with
NotImplementedError instead. Single-layer EAGLE/MTP drafts are unaffected.

Tracking: sgl-project#39634
@YAMY1234

Copy link
Copy Markdown
Collaborator Author

/rerun-test test/registered/unit/model_executor/model_runner_components/test_attention_backend_setup.py test/registered/unit/model_executor/model_runner_components/test_spec_aux_hidden_state.py test/registered/spec/eagle/test_spec_eagle.py

@github-actions

github-actions Bot commented Sep 15, 2026 •

Copy link
Copy Markdown
Contributor

Results for /rerun-test test/registered/unit/model_executor/model_runner_components/test_attention_backend_setup.py test/registered/unit/model_executor/model_runner_components/test_spec_aux_hidden_state.py test/registered/spec/eagle/test_spec_eagle.py:

🚀 ubuntu-latest (2 tests): ✅ View workflow run

cd test/ && python3 registered/unit/model_executor/model_runner_components/test_attention_backend_setup.py
cd test/ && python3 registered/unit/model_executor/model_runner_components/test_spec_aux_hidden_state.py

🚀 1-gpu-5090 (1 test): ✅ View workflow run

cd test/ && python3 registered/spec/eagle/test_spec_eagle.py

@YAMY1234 YAMY1234 changed the title [Spec] Reject EAGLE3/DFLASH aux hidden-state capture under pipeline parallelism [Spec][PP] Reject EAGLE3/DFLASH aux hidden-state capture under pipeline parallelism Sep 15, 2026
…ain guard

pp_size is required like the other keyword arguments so a future caller cannot
silently disable the guard by omitting it. The new test cases were appended
after the __main__ block and never ran under `python3 test_file.py`.
@YAMY1234

Copy link
Copy Markdown
Collaborator Author

Closing: superseded by #40045, which made PP prefill + DSpark aux hidden-state capture a supported path that this guard would have rejected. The remaining silent-skip risk for EAGLE3/DFLASH capture on non-last PP stages will be addressed by relaying the captured states across stages in a follow-up PR instead of rejecting the configuration.

@YAMY1234 YAMY1234 closed this Sep 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant