[wunnii9]

Overview

Enable Qwen3.5 GDN standard-decode ReplaySSM on Model Runner V2 by forwarding each request's decode origin to the existing GDN metadata builder.

Stacked on #62004 at 3c8678368a6358cea60db7841a5cca7c3263ce69; draft until that dependency lands. Only the final commit is new. All preceding ReplaySSM commits retain their original authors and SHAs. This adapter adds no ReplaySSM kernel or Skinny GEMM implementation.

Claims

• Permit Triton ReplaySSM with V2 for qwen3_5_text, preserving the guard for other models.
• Snapshot V2's batch-ordered CPU prefill_len_np, including resumed-request endpoints, as replayssm_decode_base_cpu; zero-fill graph padding.
• Reuse the existing GDN ring/flush calculations and persistent CUDA-graph buffers, without GPU-to-CPU reads or new synchronization.
• Add configuration, metadata, and compiled V2 engine regression coverage, plus usage documentation. Existing compilation mode and Breakable graph settings remain user choices.

Validation

Single GB10 (SM121), Qwen3.5-4B BF16, FP32 state, TP=1, L16, no speculative decoding or prefix caching.

The proposed branch is tested on the real #62004 head. Offline fixtures use the available local checkpoint configuration; submitted tests contain no local paths. Test commands:

.venv/bin/python -m pytest \
  tests/v1/worker/test_mamba_hybrid_model_state.py \
  tests/v1/attention/test_gdn_metadata_builder.py \
  tests/v1/worker/test_gpu_batch_ordering.py -q
.venv/bin/python -m pytest tests/test_config.py \
  -k 'triton_replayssm_v2_model_scope or kda_recoverssm_derivation' -q
.venv/bin/python -m pytest \
  tests/kernels/test_replayssm_standard_decode_gdn.py \
  tests/kernels/test_replayssm_teacher_decode_equivalence_gdn.py \
  -k '(qwen4b and s32_a16) or (per_row and s32_a16) or flush_keeps' -q
VLLM_GPU_SYNC_CHECK=error .venv/bin/python -m pytest \
  tests/v1/e2e/test_replayssm_decode.py \
  -k test_replayssm_gdn_v2_decode_matches_baseline -q
pre-commit run

On the final dependency-head branch: 128 passed, 551 deselected; full pre-commit run and commit hooks pass. The compiled V2 engine comparison ran with VLLM_GPU_SYNC_CHECK=error. Tests completed without failures. The earlier standalone checkout had 131 passing cases; the count changes because the real dependency head contains a different upstream test revision.

The V2 engine case explicitly uses compilation mode 3, Breakable off, graph capture size 4, and two prompts with up to 32 output tokens, exercising L16 flushes. It uses the existing parity helper: identical greedy tokens, or mutual top-five membership at the first divergence. This is a limited output regression, not full-vocabulary equivalence or GSM8K/MMLU evaluation.

Earlier standalone validation also passed 24/24 arithmetic checks, C1/C8 generation, and 32/32 mixed requests under GPU sync checking. Historical experiment on main a9fe30e9f6 plus #62004, both runners compiled with Breakable off, 512 input / 128 output tokens:

Runner + Replay L16	C1 TPOT ms	C64 TPOT ms	C64 output tok/s
V1	48.287	77.576	649.696
V2	48.092	75.189	666.496

Three rounds per condition, one startup per runner; C1 four requests per round and C64 one burst of 64. TPOT ranges: C1 V1 48.184–48.368 / V2 48.056–48.130; C64 V1 77.369–77.761 / V2 75.083–75.260 ms. These historical measurements support the compiled V2 path on this workload; they are not a fresh benchmark of this exact dependency-head branch. That benchmark build contained a disabled Skinny hook, absent here.

Details

Relaxing the V2 configuration guard alone leaves the GDN builder without its required CPU decode base. V2 already gathers the prefill endpoint in batch order and refreshes it after preemption. Forwarding a per-step CPU snapshot supplies the builder's existing cursor calculation while keeping request-state ownership in V2. Tests cover a flush boundary, resume origin, final single-token prompt chunk, null graph padding, and persistent device-buffer reuse.

This extends #62004, whose stated scope excludes V2. Related RecoverSSM speculative decoding and Mamba2 FlashInfer lifecycle PRs address different paths. Duplicate searches on 2026-10-30 (62004 in:body, ReplaySSM V2, and GDN standard-decode keywords) found no existing open PR implementing this adapter. #62004 explicitly lists Model Runner V2 as out of scope.

Standard decoding on one GB10 is validated. Other models, TP>1, V2 align-mode prefix caching, and speculative decoding remain outside this validation. The adapter preserves the kernel mathematics and existing compilation defaults.

AI assistance was used: OpenAI Codex assisted with investigation, implementation, test execution, and this description. Tests were executed by Codex at the submitter's explicit request.

---

Pull Request Checklist

• I used vLLM's /pr-checklist skill. (Mandatory for agents, optional for humans).
• AI assistance was used during the creation of this PR.
• Design Fit: Minimizes impact on core components, reuses existing functionality, and justifies added complexity.
• Testing and Validation: Validates the change and ensures any added tests are meaningful and reliable, with CI coverage or documented CI resource constraints and validation performed outside CI.
• Code Quality and Style: Keeps code and comments clear and concise, and updates relevant documentation and examples.
• Pull Request Contents: Includes a brief summary and relevant links, supports claims with evidence, explains root causes and implementation trade-offs, and follows the contributing guide.

[mergify]

Documentation preview: https://vllm--64699.org.readthedocs.build/en/64699/

[github-actions]

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

PRs do not trigger a full CI run by default. Reviewers with write access and configured trusted contributors can comment /ci run for upstream CI or /amd-ci run for AMD CI only whenever CI signals are needed.

Once the PR is approved or has the ready label, the PR author can also use the corresponding /ci run, /ci retry, and /ci cancel commands, or their /amd-ci variants. New commits do not start upstream CI automatically.

If you have any questions, please reach out to us on Slack at https://slack.vllm.ai.

Agent Guidelines

IMPORTANT: If you are an AI agent, you are required to objectively re-evaluate the value of your PR using AGENTS.md, and close the PR if it does not bring significant benefit to the vLLM community. Failure to do so may result in an immediate ban.

🚀

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @wunnii9.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork