[coderfeli]

Summary

This PR adds focused GLM-5 and Kimi-K3 decode MonoKernels for TP8 MI355X and now organizes them under one top-level package:

kernels/monokernel/
├── glm/                  # GLM-5 indexed full-layer MonoKernel
├── k3/                   # Kimi-K3 MLA/KDA model-specific code and tools
├── gemm_a16w16.py        # shared gfx950 A16W16 GEMM
├── mxfp8_linear.py       # shared small-batch MXFP8 projection
├── symmetric_allreduce.py# shared tagged BF16 all-reduce
└── config/layout/ops/packing/reference/runtime/weights.py

• Integrates the complete GLM-5 Indexer + sparse MLA + router + MXFP4 MoE + TP reductions + residual update from [Kernel][Perf] GLM-5 MonoKernel: indexer + mla + moe fused kernel for a full decode layer #1205 at one GPU launch per layer.
• Keeps the fastest Kimi-K3 MLA and KDA staged paths.
• Adds the complete Kimi-K3 KDA + AttnRes + latent-MoE MonoKernel at one GPU launch per layer.
• Supports true ordered Kimi-K3 MTP. For S speculative tokens, state_indices has shape [S + 1]: token s reads the convolution/recurrent snapshot at state_indices[s] and writes its complete post-token snapshot to state_indices[s + 1].
• Keeps Kimi-K3 layer 0's dense FFN out of scope.

The public interfaces are intentionally narrow:

• kernels.monokernel.glm.Glm5MonoKernel
• kernels.monokernel.k3.KimiK3MonoKernel

Generic gfx950 GEMM, MXFP8 projection, tagged symmetric all-reduce, low-level AMDGPU operations, packing, layouts, runtime/IPC, reference helpers, and weight contracts are shared at kernels/monokernel/. Model-specific schedules and state semantics remain under glm/ and k3/. Unused experimental storage-resolution helpers, shared-expert Torch code, and packing aliases were removed; no source-inspection tests were added.

MTP schedule

The retained implementation preserves the real snapshot chain rather than treating speculative tokens as independent requests.

• Four 32-row recurrent-state splits per head.
• At S=8, one recurrence CTA advances two consecutive tokens and keeps its FP32 state fragment resident across the pair: 4 token pairs x 12 heads x 4 splits = 192 CTAs.
• Gate projection is issued before the causal-convolution dependency wait so all gate work can overlap.
• Recurrence scalar loads are staged through LDS.
• Recurrence writes FP32 state snapshots; grouped norm CTAs produce a separate packed-BF16 output-projection buffer.
• S<=4 keeps dedicated stage-local LDS views because that layout benchmarks faster; S=8 uses a safe phase-overlaid LDS arena while keeping activation and MFMA reduction storage disjoint.

TileRT design lineage

The resident-grid schedule, tagged-pair mailbox protocol, and phase-overlaid LDS arena follow the TileRT/GLM MonoKernel design lineage. The relevant GLM and Kimi source comments and kernels/monokernel/README.md reference:

https://github.com/SemiAnalysisAI/InferenceX/tree/8ac98344b038a3f2da20a565fe9b974772a67ef9

The ordered KDA snapshot pipeline and state-resident MTP recurrence are Kimi-K3-specific FlyDSL extensions.

Launch count

Path	S=4	S=8
GLM-5 full Indexer MonoKernel	1	1
Kimi MLA staged full layer	7	7
Kimi KDA staged full layer	7	9
Kimi KDA true-MTP staged full layer	7	9
Kimi KDA true-MTP MonoKernel	1	1

Performance

Hardware: 8 x AMD Instinct MI355X, TP8, position 3000. Kimi results use HIP-graph replay with 32 layer invocations and 40 measured repeats; values are critical-rank medians.

Path	Samples	FlyDSL Mono kernel latency	ATOM latest	Relative
GLM-5 full Indexer MonoKernel	4	78.4-78.6 us	205.8-213.7 us	2.62-2.73x faster
GLM-5 full Indexer MonoKernel	8	118.2-118.3 us	223.4 us	1.89x faster
Kimi MLA staged, layer 3	4	115.54 us	222.56 us	1.93x faster
Kimi MLA staged, layer 3	8	127.76 us	253.93 us	1.99x faster
Kimi KDA staged, independent decode, layer 1	4	113.66 us	197.30 us	1.74x faster
Kimi KDA staged, independent decode, layer 1	8	128.42 us	227.38 us	1.77x faster
Kimi KDA staged, true MTP	4	120.83 us	207.24 us	1.72x faster
Kimi KDA staged, true MTP	8	155.75 us	241.80 us	1.55x faster

True-MTP attention alone is 39.29 us at S=4 and 63.57 us at S=8. Relative to the previous retained MTP baseline (39.76/71.73 us), S=8 attention improves by 11.4%. The complete one-launch MTP layer improves from 164.94/260.99 us to 153.11/243.15 us at S=4/S=8.

Excluding the out-of-scope layer-0 dense FFN, weighting 24 MLA layers and 68 KDA layers gives a 92-layer independent-decode core estimate of 1.786x at S=4 and 1.827x at S=8 versus ATOM. This is not a full-model end-to-end result.

Validation

• 16 passed: complete GLM-5 and Kimi-K3 GPU integration suites.
• True MTP TP8 correctness at S=4/S=8: rank agreement, finite output, ordered convolution/recurrent snapshots, routing selection, and full-layer output checks all pass.
• Independent KDA decode and Kimi MLA S=4/S=8 show no regression.
• GLM-5 TP8 Indexer + native MXFP4 correctness passes at S=4/S=8.
• Black, Ruff, Python compilation, and git diff --check pass.
• The branch includes [Kernel][Perf] GLM-5 MonoKernel: indexer + mla + moe fused kernel for a full decode layer #1205 and is merged with current main (e4245c23).
• The two local handoff files remain untracked and are not part of the PR.

Cold-link validation used a local external-LLVM compatibility bridge because the bundled parser cannot consume newer ROC-DL kernel metadata for some GEMM binaries. That local bridge is not included in this PR.

No new third-party dependencies.