[ilmarkov]

Overview

Coalesce eligible, physically adjacent MLA KV-cache layer views into packed NIXL memory regions within each transfer group. This reduces repeated block-descriptor registration while preserving the transfer-group mappings.
The new partial-span coalescing applies to NIXL pull workers only. Push keeps its existing packed-row registration; this change does not add partial-span coalescing or mixed-layout fallback for push. Push can be supported with this optimization in follow-up PRs.

Claims

The optimization requires a block-outermost KV-cache layout; the GLM-5.3 deployment used BLHNC. Coalescing also requires compatible region geometry and adjacency. Other layouts and ineligible regions use the standard registration path.

Validated with RedHatAI/GLM-5.3-MXFP4 on a GB200 NVL72, using one EP8 prefill replica and one EP8 decode replica. The deployment used the block-outermost BLHNC layout. The workload was Weka agentic multi-turn replay, with roughly 110–160K input tokens and 900–1,000 output tokens per request.

At concurrency 32, the NIXL numbers are:

• NIXL post p95: about 3.85 s before optimization, with spikes to 5 s; about 52 ms after.
• NIXL transfer p95: about 4.52 s before; about 81 ms after.
• Pool-level registered descriptors: 3,234,222 before versus 121,580 after, about 96% fewer, assuming comparable KV-pool sizes and configuration.

Latency figures are medians across the dashboards’ repeated p95 estimates during each run.

Validation

Added unit coverage for NIXL descriptor geometry and push-connector layout metadata.

gsm8k validation
RedHatAI/GLM-5.3-MXFP4, P/D deployment prefill DEP8, decode DEP8, VLLM_KV_CACHE_LAYOUT: "BLHNC", dspark(RedHatAI/GLM-5.3-speculator.dspark)

|Tasks|Version|     Filter     |n-shot|  Metric   |   |Value |   |Stderr|
|-----|------:|----------------|-----:|-----------|---|-----:|---|-----:|
|gsm8k|      3|flexible-extract|     5|exact_match|↑  |0.9598|±  |0.0054|
|     |       |strict-match    |     5|exact_match|↑  |0.9598|±  |0.0054|

Details

Root problem

With GLM-5.3’s block-outermost KV layout, adjacent MLA layer views can occupy contiguous memory. NIXL’s registration loop still registers those views separately, repeating block descriptors across layers. This increases registered descriptor volume and contributes to NIXL post overhead.

Design

Coalesce adjacent, geometry-compatible MLA views into a wider NIXL region within the same transfer group. Preserve the layer-to-region mapping and exclude packed-span regions from later address-based deduplication.

The optimization is guarded by backing-storage, device, cache-spec, stride, block-count, and adjacency checks. Ineligible regions continue through the normal registration path.

---

Pull Request Checklist

• I used vLLM's /pr-checklist skill. (Mandatory for agents, optional for humans).

• AI assistance was used during the creation of this PR.

• Design Fit: Minimizes impact on core components, reuses existing functionality, and justifies added complexity.

• Testing and Validation: Validates the change and ensures any added tests are meaningful and reliable, with CI coverage or documented CI resource constraints and validation performed outside CI.

• Code Quality and Style: Keeps code and comments clear and concise, and updates relevant documentation and examples.

• Pull Request Contents: Includes a brief summary and relevant links, supports claims with evidence, explains root causes and implementation trade-offs, and follows the contributing guide.

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @ilmarkov.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @ilmarkov.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

[claude]

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

[ilmarkov]

Related to #60564

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @ilmarkov.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork