Skip to content

[Feat] [NVIDIA] DeepSeek-V4.1: Support indexer & compressor to be captured into BCG (3/N) - #40548

Open
SYChen123 wants to merge 1 commit into
sgl-project:mainfrom
SYChen123:v4.1/cp-dp-baseline
Open

SYChen123 wants to merge 1 commit into
sgl-project:mainfrom
SYChen123:v4.1/cp-dp-baseline

Conversation

@SYChen123

@SYChen123 SYChen123 commented Sep 21, 2026 •

Copy link
Copy Markdown
Contributor

Motivation

Issue: #39700. The PR #40333 provides an initial support on BCG and CP. However, the indexer and compressor still remains eager. In this PR, we enables the indexer and compressor to be captured into BCG as well.

This PR is fundamental to support cp and multistream forward prepare (#40179), because without BCG the cpu launch will erase the performance gains brought by multistream prepare.

Modifications

Developed based on #40333

  1. Enable DeepSeek V4.1 CP prefill BCG to capture the ratio 1/2 compressor and indexer, with capture-stable metadata and a bounded context width.
  2. Add graph-compatible two-level candidate selection and remove the 16K seqlen restriction for BCG.
  3. Provide an alternative dense indexer path for BCG (encoder and candidate source), since paged_mqa_logits has worse perf than mqa_logits on long seqs.
  4. Switch the captured indexer to topk_v2 with replay-stable plans for either chunked rows or dense rows.
  5. Optimize the candidate consumer to eliminate the performance gaps between bcg and eager, including
  • Only scan the real positions when doing topk rather than the full padded seq len.
  • In the dense indexer path, use paged_sparse_mqa_logits for candidate consumer to avoid either costly full width indexer computation or dense k gather.

Accuracy Tests

Speed Tests and Profiling

Profile

From profile, we can see that one prefill forward has around 13ms performance gains.
image

E2E benchmark

The "baseline" in the table is the performance of PR #40333, which captures BCG with compressor and indexer being eager.
image

Checklist

Review and Merge Process

  1. Ping Merge Oncalls to start the process. See the PR Merge Process.
  2. Get approvals from CODEOWNERS and other reviewers.
  3. Trigger CI tests with comments or contact authorized users to do so.
    • Common commands include /tag-and-rerun-ci, /tag-run-ci-label, /rerun-failed-ci
  4. After green CI and required approvals, ask Merge Oncalls or people with Write permission to merge the PR.

CI States

Latest PR Test (Base): ❌ Run #43826323637
Latest PR Test (Extra): ❌ Run #43826323596
Latest PR Test (AMD ROCm 10): ❌ Run #43826323549

Tip

Base CI only covers the core cases, and its workload may not exercise your change. Use /rerun-test to run the tests your change touches, or copy this prompt for an agent:

Find the registered tests this PR affects and run them with /rerun-test, following https://github.com/sgl-project/sglang/blob/main/.agents/skills/rerun-test/SKILL.md

@SYChen123
SYChen123 requested a review from yuan-luo as a code owner September 23, 2026 10:59
@Oasis-Git Oasis-Git mentioned this pull request Oct 2, 2026
17 of 56 tasks
@SYChen123
SYChen123 force-pushed the v4.1/cp-dp-baseline branch from d0d3d84 to 865c333 Compare October 8, 2026 05:12
@SYChen123
SYChen123 force-pushed the v4.1/cp-dp-baseline branch from 3f55b4d to f86e061 Compare October 13, 2026 06:06
@SYChen123 SYChen123 changed the title [DSV4.1][PD] Support indexer & compressor to be captured into BCG [Feat][NVIDIA] DeepSeek-V4.1: Support indexer & compressor to be captured into BCG (3/N) Oct 20, 2026
@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@SYChen123 SYChen123 changed the title [Feat][NVIDIA] DeepSeek-V4.1: Support indexer & compressor to be captured into BCG (3/N) [Feat] [NVIDIA] DeepSeek-V4.1: Support indexer & compressor to be captured into BCG (3/N) Nov 18, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

blackwell SM100/SM120 deepseek jit-kernel Multi-modal multi-modal language model

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant