[zyongye]

Purpose

Bump the vllm-project/DeepGEMM fork pin from ad1f172 to a6bbb800 (current head of the fork's dev branch), picking up three fixes:

• DeepGEMM#9 — corrects the ported SM120 GEMM and MQA execution paths, and moves the SM120 regression tests into dedicated tests/test_sm120_*.py files so the fork's copies of upstream-owned test files stay byte-identical to dev (removes a recurring sync-conflict surface on upstream's per-release test rewrites).
• DeepGEMM#11 — drops the Mega-Gate finite-score device trap. The selected-expert-ID clamp is retained; non-finite scores now propagate instead of aborting on device.
• DeepGEMM#13 — raises kNumTimeoutCycles from 60s to 300s. That is the device-side spin-wait budget shared by every wait in comm/barrier.cuh: the NVLink cross-rank barrier, the intra-device grid sync, and the mega-MoE combine peer wait. At 60s it could fire on long-running multi-rank workloads that were still making progress, turning a slow peer into a device assert.

Both pin sites are updated: cmake/external_projects/deepgemm.cmake and tools/install_deepgemm.sh, which each carry a comment requiring them to be kept in sync.

Not a duplicate

Checked open PRs against vllm-project/vllm:

gh pr list --repo vllm-project/vllm --state open --search "deepgemm"
gh pr list --repo vllm-project/vllm --state open --search "bump DeepGEMM"
gh search prs --repo vllm-project/vllm --state open "install_deepgemm"

No open PR bumps the DeepGEMM pin. #56266 ([DSv4.1] Integrate Mega-Gate from DeepGEMM, draft) is the only open PR touching deepgemm.cmake, and it refactors the FetchContent_Populate block — it does not change _DEEPGEMM_UPSTREAM_TAG. No conflict beyond a trivial textual one if both land.

Test Plan

The pin's risk is twofold: that the ref is fetchable as declared, and that DeepGEMM's _C still compiles against it. The changed csrc/apis/{attention,einsum,gemm,sm120_dispatch}.hpp are compiled into _C on every arch, so a _C build exercises them even on non-SM120 hardware.

1. Confirm the new ref is the dev head and fetches with the two submodules deepgemm.cmake declares (third-party/cutlass, third-party/deep_jit).
2. Build _C at the new pin using tools/build_deepgemm_C.py — the same script deepgemm.cmake drives.
3. pre-commit run on the changed files.

Test Result

$ git ls-remote https://github.com/vllm-project/DeepGEMM.git dev
a6bbb8000161c0dc3a85a0300a905f76898a7913        refs/heads/dev

$ git submodule status
 f3fde58372d33e9a5650ba7b80fc48b3b49d40c8 third-party/cutlass (f3fde58)
 e5bdee2bc4ca519eba00cfc5f0c6e950e6a96a16 third-party/deep_jit (e5bdee2)

$ CUDA_HOME=/usr/local/cuda-13.1 .venv/bin/python tools/build_deepgemm_C.py <src> <out> .venv/bin/python
# succeeded; only pre-existing -Wattributes visibility warnings from deep_jit/utils/gil.hpp
$ ls <out>
_C.cpython-312-aarch64-linux-gnu.so

pre-commit run on the two changed files: all applicable hooks pass (shellcheck, typos, dockerfile-graph, spaces-in-filenames, config-validation); the rest are skipped as no-files-to-check.

Environment: GB300 (SM103), CUDA 13.1, aarch64, Python 3.12.

Evaluation

No model evaluation is included. This PR changes only a dependency pin and no vLLM code path. Of the three picked-up commits, none alters numerics on the hardware available here:

• Implement LLaMA #9 is SM120-only (this host is SM103), and its non-SM120 portion is a test-file reorganization inside DeepGEMM.
• Add cache watermark to avoid frequent cache eviction #11 removes a device-side abort on non-finite Mega-Gate scores; it changes failure behavior, not values on well-formed inputs.
• Refactor the test code for attention kernels #13 only lengthens a hang-detection timeout.

Evaluating the SM120 correctness fixes in #9 would need SM120 hardware, which I do not have. If a reviewer wants an eval on SM120 or a Mega-MoE accuracy run before merge, please say so and I will arrange it.

Note on AI assistance

AI assistance (Claude Code) was used for this change. I have reviewed every changed line, ran the commands above myself, and can defend the change end-to-end.

🤖 Generated with Claude Code

[claude]

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

[zyongye]

/ci run

[github-actions]

✅ Triggered Buildkite CI #89396 for commit af87467866e3.

[Harry-Chen]

@cleonard530 I think you can port your libtorch stable ABI changes on top of this deepgemm commit. Meanwhile could you please audit the wheels again to see if the CUDA variant has any remaining call to non-stable APIs?

[cleonard530]

@cleonard530 I think you can port your libtorch stable ABI changes on top of this deepgemm commit. Meanwhile could you please audit the wheels again to see if the CUDA variant has any remaining call to non-stable APIs?

Thanks @Harry-Chen!

I can port the changes I made in deepseek-ai/DeepGEMM#393 on top of this branch, but this is only to get DeepGEMM of of pybind11 and onto TORCH_LIBRARY. A followup PR is going to be sent to migrate it from the unstable to the stable ABI afterwards (which I have a draft of locally, but probably needs major updates). I can use the tool to show that it abi3-ok after the first PR (i.e. using the stable CPython abi). I will also open a draft of the next PR and show the torch Stable ABI compliance, but that will come a little later.

[ivanusto]

Verified this pin bump on 2x DGX Spark (GB10, sm_121). Same nightly image on both sides (0.29.1rc1.dev9+g2671fedfc), only DeepGEMM differs: A = ad1f1726 (the previous pin), B = a6bbb80 (this PR). Weight-free, kernel level.

ue8m0 is unchanged, bit for bit.

M x N x K	A ad1f1726	B a6bbb80
1 x 7168 x 4096	rel 0.0	rel 0.0
83 x 4096 x 7168	rel 1.310e-7	rel 1.310e-7
2048 x 7168 x 4096	rel 1.250e-7	rel 1.250e-7
4096 x 13824 x 16384	rel 6.57e-7	rel 6.57e-7

tf32_hc_prenorm_gemm (T=64, H=4096, hc=4) rel L2 2.1290e-4 on both. get_paged_mqa_logits_metadata still accepts 32 and 64 only. tests/kernels/test_mhc_kernels.py 138 passed / 23 skipped / 4 errors, test_mhc_jit_warmup.py 20 passed, test_fused_indexer_q_rope_quant.py 55 passed / 61 skipped, attention/test_deepgemm_attention.py 4 passed / 1 failed: all identical on both sides. The single failure is test_deepgemm_fp8_mqa_logits[True], the pre-existing clean_logits=True assert; the serving path passes clean_logits=False.

Float32 block scales changed behaviour. With float32 scale factors, ad1f1726 silently returned NaN on sm_121 (0.78% to 38.7% of outputs depending on shape). a6bbb80 instead raises before launch, from DeepGEMM#9:

RuntimeError: Assertion error (csrc/apis/sm120_dispatch.hpp:127):
sfa.scalar_type() == torch::kInt and sfb.scalar_type() == torch::kInt

That is a clear improvement: it is a host-side assert, so there is no NaN, no Xid, and the CUDA context stays usable. It does mean the SM12x pure-fp8 1d1d path is now a hard error rather than a silent one, which matters for a configuration I have filed separately (vLLM still selects the DeepGEMM FP8 linear kernel on SM12x when VLLM_USE_DEEP_GEMM_E8M0=0).

One process note. None of the SM120 changes in DeepGEMM#9 are exercised by CI anywhere:

• cmake/external_projects/deepgemm.cmake appears in exactly one job's source_file_dependencies, (H100) DeepGEMM Kernels in .buildkite/test_areas/kernels.yaml, which runs on sm_90.
• The two DGX Spark jobs, b12x-linear-kernels-dgx-spark-nightly and gpqa-eval-gpt-oss-spark, are both optional: true and neither lists any DeepGEMM file, so a pin change never triggers them. The b12x one also filters with -k b12x.
• The fork's own workflows build TORCH_CUDA_ARCH_LIST="7.0 7.2 7.5 8.0 8.6 8.7 9.0+PTX", so the new tests/test_sm120_*.py files added in DeepGEMM#9 are not compiled or run there either.

Since there is DGX Spark hardware in CI already, adding cmake/external_projects/deepgemm.cmake, tools/install_deepgemm.sh and vllm/utils/deep_gemm.py to the Spark job's source_file_dependencies would catch SM12x regressions on pin bumps at no extra hardware cost. Happy to keep running these checks on 2x GB10 in the meantime.