[xijiaat]

Summary

Add compile_tf32_hc_prenorm_gemm(n, k, num_splits) for metadata-only startup warmup. This is a prerequisite for addressing vllm-project/vllm#60536 without dummy GPU launches.

The SM90/SM100/SM120 paths share their existing JIT source and cache keys with normal execution, but return before tensor-map construction and launch in compile-only mode. Runtime split selection and kernel math are unchanged.

Validation

Rebased onto dev at 15c02c8d96d2476dac6b96065a117b99c6e8eb83 on 2026-10-10. The PR now contains only the two compile-only warmup commits (7 files, +110/-55); the inherited binding-migration history has been removed. Every added and deleted line in each commit was checked against the original patch. Python syntax parsing and git diff --check passed. The H20 GPU and serving results below were collected on the pre-rebase version (7deaadb215277b71400e534cd520a76c77292bb4); GPU tests have not been rerun on the rebased head.

• Added a GPU regression to tests/test_hyperconnection.py: compile without tensor inputs or CUDA kernel events and check the allocator high-water mark. Reuse the existing projection and squared-sum numerical tests.
• The human submitter confirms reviewing every changed line and personally running relevant tests successfully.
• Independent H20 validation: built with DG_FORCE_BUILD=1 DG_USE_LOCAL_VERSION=0; .venv/bin/python tests/test_hyperconnection.py completed, including all 24 existing numerical/performance cases and the compile-only GPU regression. The build environment needs libdw-dev and the CUDA development headers.
• A 258-shape vLLM integration sweep with separate empty caches compiled 13 specializations during runtime without explicit warmup and 0 with this entry point. Six numerical reference checks passed in each run.
• H20 end-to-end serving with [Bugfix] Warm GLM DeepGEMM mHC pre-norm specializations vllm#60567: GLM-5.3-Flash FP8, TP8, v0.31.0 plus the PR's Python diff. All 41 requests passed (32 split-bucket requests, one arithmetic chat, eight streaming requests at concurrency four). The 16 mHC specializations existed at readiness, with zero additional mHC cubins during requests.

An initial expanded numerical matrix also exercised M=137, N=24, K=16384, num_splits=3; it showed a projection discrepancy in both the original wheel and this build. That existing runtime issue is outside this compile-only change. The new regression is scoped to the compile-only contract, rather than duplicating the existing numerical suite.

Checked the open DeepGEMM PRs; none adds this mHC compile-only entry point. AI assistance was used for implementation and test preparation.

[xijiaat]

Hi @cleonard530 @Harry-Chen, could you take a look at this PR, particularly the tensor-free TORCH_LIBRARY registration and compile-only API design? Since you worked on the binding migration in #15, your feedback would be especially helpful.

This adds metadata-only mHC pre-norm warmup for vllm-project/vllm#60567, addressing the runtime JIT stalls in vllm-project/vllm#60536. It reuses the existing JIT source and cache keys without launching kernels or changing runtime split selection.

I have rebased the two commits onto the latest dev; the diff is now limited to 7 files and has no merge conflicts. The added/deleted patch lines are unchanged, and Python syntax and diff checks passed. The documented pre-rebase H20 validation reduced runtime compilations from 13 to 0 in the 258-shape sweep, with all 41 end-to-end serving requests passing.

Thanks!

[Harry-Chen]

Hi @cleonard530 @Harry-Chen, could you take a look at this PR, particularly the tensor-free TORCH_LIBRARY registration and compile-only API design? Since you worked on the binding migration in #15, your feedback would be especially helpful.

@zyongye PTAL. We (I and Chris) do not make functionality changes to this library at the moment.

[xijiaat]

Hi @zyongye, when you have a chance, could you please take a look at this PR? It adds a compile-only mHC pre-norm warmup API for vllm-project/vllm#60567, reusing the existing JIT source and cache keys without dummy kernel launches or changes to runtime split selection. I would appreciate your feedback on the API design and any changes needed. Thanks for your time!