[MatthewBonanni]

Overview

Make GLM-5.3-Flash's kpool indexer list each top-k row's valid entries first, with no -1 gaps. This is a prerequisite for Flash + HiSparse (#61243): HiSparse's top-k conversion keeps entries in place, so the gaps reach FLASH_ATTN_MLA_SPARSE, which crashes with an illegal memory access on Hopper.

Claims

• With [HiSparse] GLM-5.3-Flash support #61243, Flash + HiSparse on FLASH_ATTN_MLA_SPARSE (H200, CUDA graphs) now serves instead of failing every request with CUDA error: an illegal memory access.
• No change without HiSparse: the generic conversion already moves valid entries to the front, and outputs are bit-identical before and after this PR.
• No performance change: the kernel keeps the same grid, loads and stores, plus one min per row.

Validation

H200, zai-org/GLM-5.3-Flash@eb9eb208eb0d988989d07a6a12d0fdeb5f52574a, TP4, FULL_AND_PIECEWISE, on top of #61243 head (dac3a3b). Log shows Using FLASH_ATTN_MLA_SPARSE attention backend. GSM8K 5-shot completions, greedy, one server per row.

With HiSparse:

vllm serve zai-org/GLM-5.3-Flash --revision eb9eb208eb0d988989d07a6a12d0fdeb5f52574a --trust-remote-code --language-model-only -tp 4 --kv-cache-dtype bfloat16 --max-model-len 8192 --max-num-batched-tokens 4096 --max-num-seqs 32 --attention-config '{"hisparse_config":{}}' --kv-transfer-config '{"kv_connector":"HiSparseConnector","kv_role":"kv_both","kv_connector_extra_config":{"host_pool_gib":32}}'

	1319 @ c32	1319 @ c32	200 @ c1
#61243 head	64/64 requests fail (IMA)
+ this PR	0.909	0.914	0.915

Without HiSparse (same command, --attention-config '{"backend":"FLASH_ATTN_MLA_SPARSE"}', no connector), this PR leaves the output unchanged:

• 100 GSM8K greedy decodes at c=1 are bit-identical before and after: all texts and all 8529 token logprobs.
• GSM8K 1319 @ c32 is 0.914 with graphs; eager is 0.918.

Unit (H200): pytest tests/v1/attention/test_sparse_indexer_decode_seq_lens.py tests/models/glm5next/ → 41 passed, 8 skipped. The new test_pooled_rows_keep_valid_prefix fails without the kernel change. CI runs it in the V1 Attention jobs (H100/B200 plus the AMD mirrors). The AMD kernel copy got the same one-line change; only those AMD CI jobs exercise it, not MI3xx hardware.

Details

The kpool indexer's pooled path anchored the incomplete-pool tail at column index_topk. When a row has fewer than index_topk / index_kpool complete pools, this leaves -1 gaps between the selected pools and the tail. The exact-causal shortcut for max_seq_len <= index_topk avoids these gaps, but it is a host check. FULL CUDA graphs capture with max_seq_len = max_model_len, so decode replay always takes the pooled path.

Without HiSparse the gaps are harmless. triton_convert_req_index_to_global_index moves valid entries to the front whenever it returns valid_counts (#46076). HiSparse's hisparse_resolve_residency keeps entries in place and assumes the indexer's rows have no gaps. With HiSparse, instrumented graph replay showed 288 decode rows per layer with -1 inside [0, valid_count). FA3 uses those entries as its page table, so it reads row -1.

The gappy layout dates to #53906. It only reaches an attention kernel through HiSparse, which #61243 enables for Flash.

The fix anchors the tail at min(pool_len * kpool, topk) in the NVIDIA and AMD fused kernels and in the pure-torch reference. Short rows now match the exact-causal fill, and full rows are unchanged. fit_kpool_indices_to_aiter stays correct with either layout.

Not a duplicate: no open PR changes the kpool tail layout.

---

Pull Request Checklist

• I used vLLM's /pr-checklist skill. (Mandatory for agents, optional for humans).

• AI assistance was used during the creation of this PR.

• Design Fit: Minimizes impact on core components, reuses existing functionality, and justifies added complexity.

• Testing and Validation: Validates the change and ensures any added tests are meaningful and reliable, with CI coverage or documented CI resource constraints and validation performed outside CI.

• Code Quality and Style: Keeps code and comments clear and concise, and updates relevant documentation and examples.

• Pull Request Contents: Includes a brief summary and relevant links, supports claims with evidence, explains root causes and implementation trade-offs, and follows the contributing guide.

🤖 Generated with Claude Code

[claude]

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

[MatthewBonanni]

Folded into #61243 (commit 7754987): the gaps only reach an attention kernel through HiSparse's decode conversion, which #61243 enables for GLM-5.3-Flash.