[ZJY0516]

Purpose

add dcp support for GLM 5.2

Test Plan

VLLM_FLASHINFER_WORKSPACE_BUFFER_SIZE=1073741824
vllm serve zai-org/GLM-5.2-FP8 \
    --host 0.0.0.0 \
    --port 8000 \
    --kv-cache-dtype fp8_e4m3 \
    --tensor-parallel-size 8 \
    --decode-context-parallel-size 4 \
    --tool-call-parser glm47 \
    --enable-auto-tool-choice \
    --reasoning-parser glm45

logs

DCP 4, TP 8
(EngineCore pid=88967) INFO 06-18 17:02:00 [kv_cache_utils.py:2078] GPU KV cache size: 10,853,376 tokens
(EngineCore pid=88967) INFO 06-18 17:02:00 [kv_cache_utils.py:2079] Maximum concurrency for 1,048,576 tokens per request: 10.35x

TP 8
(EngineCore pid=93451) INFO 06-18 17:27:37 [kv_cache_utils.py:2078] GPU KV cache size: 2,765,568 tokens
(EngineCore pid=93451) INFO 06-18 17:27:37 [kv_cache_utils.py:2079] Maximum concurrency for 1,048,576 tokens per request: 2.64x

lm_eval --model local-completions --model_args "model=zai-org/GLM-5.2-FP8,base_url=http://0.0.0.0:8000/v1/completions,tokenized_requests=False,tokenizer_backend=None,num_concurrent=128" --tasks gsm8k --num_fewshot 5

Test Result

Tasks	Version	Filter	n-shot	Metric		Value		Stderr
gsm8k	3	flexible-extract	5	exact_match	↑	0.9340	±	0.0068
		strict-match	5	exact_match	↑	0.9333	±	0.0069

Perf test

GB200

vllm serve lukealonso/GLM-5.2-NVFP4 --kv-cache-dtype fp8_e4m3 -tp 4 [-dcp 4]

concurrency 256 decode performance

dcp 4
Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 4071.3 tokens/s, Running: 241 reqs, Waiting: 15 reqs, GPU KV cache usage: 99.7%, Prefix cache hit rate: 0.6% 
tp 4
Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 2508.7 tokens/s, Running: 63 reqs, Waiting: 193 reqs, GPU KV cache usage: 99.2%, Prefix cache hit rate: 0.0% 

But dcp prefill speed is only 1/2 of tp, so the e2e throughput is not ideal

---

Essential Elements of an Effective PR Description Checklist

• The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
• The test plan, such as providing test command.
• The test results, such as pasting the results comparison before and after, or e2e results
• (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.

[mergify]

Documentation preview: https://vllm--46076.org.readthedocs.build/en/46076/

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 981b6e075b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @ZJY0516.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

[LucasWilkinson]

do we need a whole new kernel? can we just add support to triton_convert_req_index_to_global_index?

[GirasoleY]

Done in 30f6fec — folded into _convert_req_index_to_global_index_kernel via DCP_SIZE/DCP_RANK constexprs.

[drakosha]

Hi @ZJY0516 @GirasoleY @LucasWilkinson — following up on the suggestion to consolidate sparse MLA DCP support here.

We've been working on #45426 (DCP for FLASHMLA_SPARSE). Rather than keep a duplicate exact-merge path, we'd like to build on this PR's shared indexer merge (_merge_dcp_topk_global + the radix stable_top_k_from_candidates kernel) and add the FlashMLA-specific layer on top.

Coverage this PR doesn't reach yet. On Hopper / SM90 (e.g. H200), the sparse MLA backend the selector picks today is FLASHMLA_SPARSE — the generic FLASHINFER_MLA_SPARSE path is currently in the SM10 priority block in vllm/platforms/cuda.py. The production path we need is also fp8_ds_mla KV cache via FlashMLA, while this PR validates the FlashInfer sparse path with fp8_e4m3. So #46076 gives the shared DCP/indexer infrastructure, but it doesn't yet cover the H200 + fp8_ds_mla sparse path — where any Hopper deployment of these models (e.g. GLM-5.2 at TP4/DCP4) sits today.

What we'd add on top of your shared contract:

• FLASHMLA_SPARSE DCP localization: consume the indexer's global ids → this rank's physical fp8_ds_mla slots.
• FlashMLA decode plumbing: base-e LSE (FlashMLA already returns base-e, matching vLLM's DCP merge convention) + the fp8-kernel out/LSE handling.
• MTP + DCP + full cudagraphs (supports_dcp_with_varlen).
• GLM-5.2 validation on 4×H200.

(Plus fail-closed guards for combinations we won't support under DCP, e.g. compress_ratio > 1. We don't plan to upstream the union/approximate path from #45426 — only the exact merge, which preserves the original top-k set.)

The intended end state, if this sounds right: one shared exact top-k merge + radix kernel for sparse MLA, with FlashInfer and FlashMLA differing only in the backend-specific global-id → physical-slot localization and LSE/kernel plumbing.

Our preference is to open this as a stacked PR on top of #46076 — so this PR keeps moving while the FlashMLA-specific validation stays isolated, landing after #46076 merges. If you'd rather fold it in here after the rebase, we're happy to do that.

One contract question before we move code: can we treat "the indexer emits global ids, and each backend localizes them" as the intended shared contract? #45426 currently returns local shard indices from the indexer; we'll switch to the global-id contract so both backends share one merge.

We'll also leave two small inline notes separately: the fp64 composite fallback isn't strictly equivalent to the radix key for arbitrary fp32 scores, and the localizer asserts cp_kv_cache_interleave_size == 1 while the merge helper already handles general interleave.

[LucasWilkinson]

are these needed? cant we just make cu_seqlen_ks and cu_seqlen_ke local when DCP is enabled?

[GirasoleY]

Done in 3b1b408 — kernel writes local row bounds; the extra local_cu_seqlen fields are gone.