[lucifer1004]

Summary

Correct SM120 execution paths in dev (audited baseline ad1f1726), without importing missing nv_dev features or changing README/build configuration.

• Fix K-grouped references to the current scheduler's K prefix. Wait for prior TMA descriptor reads before overwriting the per-CTA descriptor on group changes, following the existing SM90 pattern.
• Preserve C accumulation in split-K reduction and respect supported output strides in GEMM, BF16 einsum and HC.
• Use the existing flat loop for mixed FP8/FP4 configurations whose SF-major tail omitted MMA work; handle scalar odd-N tails and legal single-N-tile BF16 specializations.
• Bound masked output by each group's physical capacity. Only a tile crossing that boundary uses scalar output; complete tiles retain TMA stores.
• Preserve independent scale recipes during small-M AB swap and reject incompatible transformed SF dtypes before SM120 GEMM execution.
• Preserve HeadSplits addressing in scalar and split-K output, including the untouched middle region; split-K remains enabled.
• Match FP8 dense/paged MQA shared-memory swizzle to head dimension and synchronize metadata prefix publication.
• Add focused regressions and release seven no-longer-used per-case tensors in the paged test's existing cleanup block. No shapes, reference calculations or tolerances are reduced.

Reproductions

The unmodified baseline exhibited K-grouped JIT failures, split-K dropping C, mixed gran128/128 K-tail omissions, stride/padding corruption, cross-group masked stores, skip-head output mapping failures, and incorrect FP8 MQA D32/D64 logits.

After fixing the scheduler member mismatch, full-SM randomized K-grouped tests exposed an additional descriptor-reuse race. On identical inputs and the same SM120 device, adding only the six descriptor lifecycle lines removed numerical and repeatability failures for both BF16 and FP8. Small-SM controls remained correct. Sanitizer instrumentation alone masked this failure, so the conclusion relies on ordinary A/B runs, not merely clean sanitizer output.

Validation

Standard standalone builds and external artifact imports passed. Source, dependency, native and packaged JIT-header identities were checked throughout; native hashes alone are insufficient for JIT-header fixes.

• Final combined artifact: 34 focused checks passed in ordinary execution, memcheck and racecheck, with zero sanitizer errors/hazards/warnings and no suppression or kernel filters. The default-SM K-grouped probes also passed memcheck/racecheck/synccheck. These are execution selections, not additive counts of unique test cases.
• Dense correctness-only coverage: all 84 BF16 and 143 FP8/FP4/mixed cases from the existing SM120 enumeration passed, retaining the original aliases, references, tolerances, 20-repeat determinism and FP4-to-FP8 equivalence checks.
• Supported K-grouped accumulation scope: BF16 63 variants and FP8 TN/NT 189 variants passed, including unequal/empty groups, alignments 128/256/384 and packed-SF checks. FP8 retained 3,780 bitwise repeat checks.
• Existing M-grouped and batched-einsum standard nodes passed on the combined GEMM component. The final descriptor increment is compile-time confined to K-grouped execution.
• Combined MQA artifact: all 192 dense and 660 paged internal standard cases passed. Paged execution required invocation-only PYTORCH_ALLOC_CONF=expandable_segments:True and the included one-line test-lifetime cleanup. The final K-grouped-only increment leaves the 23 checked MQA source/test files byte-identical; all focused MQA checks were rerun on the final artifact.
• Actual generated kernels confirm HeadSplits split-K=2; the fix does not obtain correctness by disabling split-K.

Qualification limits

This is not a claim that an unfiltered repository-wide suite is green:

• The original K-grouped standard nodes reach C=None direct-output combinations that this fork already rejects on SM120. Those full-node failures are retained. Supported-scope validation used an external enumerator filter; it did not change the committed generator or add the missing feature.
• The original dense nodes encounter a cuBLAS benchmark NOT_SUPPORTED or empty-score reporting division by zero. Separate correctness-only execution omits benchmark/reporting, not numerical checks, and does not overwrite those failures.
• SM121 runtime and other architectures were not hardware-qualified by this campaign. Shared API changes are limited to SM120 branches.

Performance scope

Six-round paired measurements used a full 512 MiB read/write L2 eviction before each measured invocation (128 MiB L2). Measurements cover GPU activity spans of public calls, including applicable SF conversion, copies, metadata and reductions; they exclude Python/CPU allocator time. PDL was disabled, and input/output qualification was performed before and after measurement.

• Sixteen numerically valid GEMM control cases showed no major stable regression. Mixed gran128/128 cases had approximately 0.4–3.3% higher paired latency; the gran128/32 control and most other paths were approximately flat.
• Eight FP8-D128/metadata controls showed no major stable regression. Incorrect baseline D32/D64 paths were not used to claim speedups.
• Six correct K-grouped descriptor controls plus two non-K-grouped controls showed no major stable regression. Incorrect full-SM baseline cases have no before/after speedup claims.

Percentages use the median of within-round latency ratios. Small profiler timestamp overlaps were retained without correction; there is no claimed absolute timestamp error bound, dynamic spill measurement or serving-throughput guarantee. Static LOCAL0 does not imply absence of local-memory instructions.