Repository navigation
[DSV4.1][PD][5/N] Decode node support DP attention - #40177
Conversation
6ef0ebb to
19a0277
Compare
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
/rerun-test test/registered/unit/server_args/test_platform_prefill_cp_deprecation.py test/registered/disaggregation/test_disaggregation_dsv4.py test/registered/disaggregation/test_disaggregation_dp_attention.py test/registered/unit/disaggregation/test_dsv4_unified_fp8_pd.py test/registered/e2e/models/test_deepseek_v4_flash_fp4_megamoe_b200.py test/registered/e2e/models/test_deepseek_v4_flash_fp4_b200.py test/registered/e2e/models/test_deepseek_v4_flash_fp8_h200.py test/registered/cp/test_deepseek_v4_flash_fp4_b200_cp.py |
|
🚀 🚀 🚀 🚀 |
|
@Oasis-Git could you review this? |
|
/rerun-test test/registered/unit/server_args/test_platform_prefill_cp_deprecation.py test/registered/disaggregation/test_disaggregation_dsv4.py test/registered/disaggregation/test_disaggregation_dp_attention.py test/registered/unit/disaggregation/test_dsv4_unified_fp8_pd.py test/registered/e2e/models/test_deepseek_v4_flash_fp4_megamoe_b200.py test/registered/e2e/models/test_deepseek_v4_flash_fp4_b200.py test/registered/e2e/models/test_deepseek_v4_flash_fp8_h200.py test/registered/cp/test_deepseek_v4_flash_fp4_b200_cp.py |
|
🚀 🚀 🚀 🚀 |
|
UT CI pass |
Motivation
Issue #39700. Currently, decode cannot use dp attention in PD disaggregation. This PR supports dp attn in PD.
Modifications
Remove the restricts and reuse the dp attn and kv transfer path of dsv4.
Accuracy Tests
Speed Tests and Profiling
We test the performance gains on 1P1D on a single node B200. Prefill is CP4 and Decode is DP4. On both E2E and output throughput, this PR achieves 10% performance gains.
Baseline

With this PR

Launch command
Checklist
Review and Merge Process
/tag-and-rerun-ci,/tag-run-ci-label,/rerun-failed-ciCI States
Latest PR Test (Base): ⏳ Run #37844962128
Latest PR Test (Extra): ❌ Run #37844961572
Latest PR Test (AMD ROCm 10): ⏳ Run #37844962075