Repository navigation
[feat][DCP] Switch DCP decode to persistent PA + make DCP compatible with MTP - #1746
Merged
Merged
Conversation
Contributor
🏷️ CI GuideRuns automatically on every eligible PR before approval:
Heavy model tests:
|
1 task
valarLip
approved these changes
Aug 3, 2026
whx-sjtu
added a commit
that referenced
this pull request
Aug 4, 2026
Resolves against #1746 (DCP persistent PA): - layer_mla.py: take main's removal of the DCP persistent-mode guard; our side only reworded its comment, and the guard is gone because DCP now runs persistent on gfx950. - metadata.py: size the work descriptors with main's persistent_num_heads inside our per-query-length worst-case loop, so the DCP head scaling and the spec-decode capacity fix both hold. - metadata.py: run main's DCP-on-gfx942 persistent opt-out before our capacity assertion, so the assertion cannot fire on a path that just disabled persistent metadata. Co-authored-by: Cursor <cursoragent@cursor.com>
whx-sjtu
added a commit
that referenced
this pull request
Aug 4, 2026
The rationale claimed no aiter decode kernel returns a usable LSE, and pinned that on the persistent kernel leaving final_lse at FLT_MAX. That is backwards. Offline comparison against a torch reference showed the persistent kernel's LSE is correct across batch sizes, and #1746 has since switched DCP decode onto it precisely because gfx950 emits a per-token LSE for the cross-rank merge. The kernel that cannot be used is the split-KV one, whose stage2 reduce leaves final_lse untouched for a request whose valid split count collapses. So causal drafting is a scope decision -- one drafting semantics shared with ATOM's native DSpark, no merge to build -- not a kernel limitation, and non-causal is a buildable follow-up. Say that instead. Co-authored-by: Cursor <cursoragent@cursor.com>
4 of 5 tasks
whx-sjtu
added a commit
that referenced
this pull request
Aug 4, 2026
The rationale claimed no aiter decode kernel returns a usable LSE, and pinned that on the persistent kernel leaving final_lse at FLT_MAX. That is backwards. Offline comparison against a torch reference showed the persistent kernel's LSE is correct across batch sizes, and #1746 has since switched DCP decode onto it precisely because gfx950 emits a per-token LSE for the cross-rank merge. The kernel that cannot be used is the split-KV one, whose stage2 reduce leaves final_lse untouched for a request whose valid split count collapses. So causal drafting is a scope decision -- one drafting semantics shared with ATOM's native DSpark, no merge to build -- not a kernel limitation, and non-causal is a buildable follow-up. Say that instead. Co-authored-by: Cursor <cursoragent@cursor.com>
whx-sjtu
added a commit
that referenced
this pull request
Aug 5, 2026
Numbers now come from a run against the rebased tree rather than the one taken before #1746 (DCP persistent PA) was integrated: 0.9545 flexible / 0.9538 strict, acceptance 3.777. All inside the error bar of the earlier 0.9522, as expected for a change that only re-sizes work descriptors. Also note the TimeoutError trap. A run whose requests hit the local-completions client timeout read 0.9424 while its acceptance figures stayed normal -- the retries re-generate under a different batch composition, and greedy chain of thought is sensitive enough to flip on that. Worth checking before anyone reports a regression from these numbers. Co-authored-by: Cursor <cursoragent@cursor.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
1. Motivation
2. Usage
Persistent mode is automatic on gfx950 — no new flag. MTP+DCP: combine existing flags.
fp8 + DCP + MTP raises
NotImplementedErrorinstead of silently miscomputing.3. Design
Flow — MTP draft loop under DCP
4. Test
MTP+DCP: native + plugin,
-tp8 -dcp8, bf16,--method mtp --num-speculative-tokens {1,2,3}; sustained concurrent load to forcebs>1cudagraph decode, then full gsm8k (no--limit), confirm 0 crashes.MTP+DCP, native, full gsm8k (1319 questions, bf16, cudagraph):
num_speculative_tokensMTP+DCP, vLLM plugin (
limit=100, not yet full):num_speculative_tokensSubmission Checklist