[askliar]

Overview

Adds separate CuTeDSL ReplaySSM checkpoints on top of the shared FP8/INT8 extension.
PR #55688 merged into main at 750cad1973f037fe00116cbf58cd3b3bd8701cd4.
Dependency: this branch remains stacked directly on shared FP8/INT8 PR #52962 at 911ffe1d78cbb31b618a87fd2911fa6b57fd4597.
The shared branch uses main at 4527515cd705db206ae927cf5acef506abc64bdd.
GitHub targets main, so this PR's aggregate diff includes the shared layer.
Review the separate-only comparison.
This PR remains draft and WIP.

FP16 recurrence checkpoint with SR7
  → FP32 flush work
  → FP8/NVFP4 read checkpoint
  → BF16 readout

Claims

• --replayssm-checkpoint-mode separate --replayssm-state-dtype fp8|nvfp4 selects the separate layout.
• FP16 recurrence uses SR7. FP8/NVFP4 read checkpoints use nearest encoding.
• Preserves the SM103 kernels and GB10/SM121 accepted-prefix path, including the saved paired-output readout optimization.
• Separate mode supports up to seven draft tokens on SM121, requires history 16, and rejects prefix caching.
• Shared FlashInfer behavior remains in the parent PR.

Validation

Current rebase checks at c996ff8609ce23e0c67cc7df4487308c38294feb:

• Focused CPU checks: 50 passed, 22 skipped, 605 deselected. The separate arithmetic suite recorded 14 CUDA skips.
• Changed-file AST, Ruff, formatting, diff checks, and applicable pre-commit hooks passed, including mypy.
• The rebase applied without conflicts. git range-diff confirms the feature patch is unchanged.
• No GPU, model-quality, or throughput run was repeated for this update.

Prior validation at ec4c604395924bda25f5955f75aa02d9379916c2 (child-only diff: 44 files, +7412/−154):

• Focused CPU checks: 50 passed, 22 skipped, 602 deselected. The separate arithmetic file has 14 CUDA skips.
• Changed Python files passed AST parsing, Ruff, formatting, and diff checks. All applicable pre-commit hooks passed, including mypy.
• The native pending-kernel diff matches the saved paired-readout patch. Other native source files retain the published implementation.
• CUDA arithmetic tests skipped locally. The updated branch at c996ff8609ce23e0c67cc7df4487308c38294feb has no fresh GPU, model-quality, or TPS result.
• Existing CI selects the affected ordering tests. The native SM103/SM121 arithmetic path lacks a matching hosted GPU lane.

Historical evidence only: the October 28 GB10 Bolt/LiLiCorr matrix recorded 76.3966 shared BF16, 78.6049 separate FP8, and 78.6739 separate NVFP4 decode TPS/user.
It used older combined source with the paired-readout optimization, simulated subset-72 routing, synthetic AL4.3, B1/T8, 32K input, and 4K output.
Five measured requests followed two warmups per mode. These results do not establish a repeatable gain for the updated branch.
Earlier small natural-acceptance GSM8K checks do not establish quality parity.

The published GB10 optimization previously passed 17 GPU tests and matched 104 differential output/cache snapshots.
Those checks precede this branch update. The paired-readout component measurements also precede the original split.
No GPU or model evaluation was repeated for this update.

Details

The native kernels derive from ReplaySSM low_precision.
The vendored sources retain their license and attribution.
GB10 executes prepare, products, update/readout, and stage kernels.
Only the accepted pending prefix can modify canonical checkpoints. Rejected candidates remain outside canonical state.
The request classifier keeps intermediate prompt chunks on prefill so every chunk reaches the recurrence checkpoint.
The merged ReplaySSM checkpoint-exporter, ring helpers, and committed-memory warmup contracts remain intact.

This is the separate portion extracted from #52962. It adds no benchmark-only routing simulation.
AI assistance was used for implementation, review, validation, and this description.

---

Pull Request Checklist

• I used vLLM's /pr-checklist skill.
• AI assistance was used during the creation of this PR.
• Design Fit: Separate state layouts and native kernels are isolated in this child layer.
• Testing and Validation: Fresh GPU and model validation of the updated branch remains outstanding.
• Code Quality and Style: Independent review, static checks, and signed-off commits completed. AI assistance is disclosed.
• Pull Request Contents: Scope, stack dependency, validation revisions, and historical evidence are explicit.

[mergify]

Documentation preview: https://vllm--65660.org.readthedocs.build/en/65660/

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @askliar.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @askliar.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork