[jcaraban]

Summary

mha_v4 evaluates its K-smoothing gate (#5798) on every quantized call as ~10 torch ops; at small shapes that costs more than the attention kernel. This runs the same gate as two deterministic HIP kernels behind one call (aiter::mha_v4_k_smooth_mean).

Performance

MI355X, eager mha_v4 per call (us): main / this PR / gate disabled.

Workload	main	PR	gate off
MXFP4 hd256, b1 h16 s4096	210.6	135.2	119.5
MXFP6 hd256, b1 h32 s2048	229.4	125.7	101.8
FP8 hd128, b1 h16 s4096	270.5	219.2	189.2
FP8 hd128, b1 h5 s65536	3882	3857	3849

PR Self-Review

Reviewed revision

e227af3

Behavior and intent

Same sample, threshold and output (up to fp32 summation order); only the cost changes. The gate drops from ~10 launches to one host call, removing most of the overhead reported against #5798 while keeping its accuracy protection.

Validation and unverified scope

MI355X: python3 op_tests/test_mha_v4.py 332 passed, python3 op_tests/test_mha_v4_sparse.py 290 passed (5 skipped). The new test fails when the op's sample size differs from the reference. Unverified: gfx942 (MI300X) not run, although the FP8 recipe reaches this op there.

PR-specific question

Every quantized mha_v4 recipe on gfx942/gfx950 calls it through _k_mean; BF16 recipes and non-CUDA K skip it. The host checks BSHD fp16/bf16 K with a contiguous last dim and head dim 64/128/256; misaligned views take a scalar-load path.

Changed op tests

op_tests/test_mha_v4.py: adds test_mha_v4_k_smooth_mean_matches_reference, comparing the op with the previous torch formula on both gate outcomes, head-sliced and misaligned K, hd128/hd256. Existing tests only check the gate's end effect on accuracy, not its value.

[github-actions]

🏷️ CI Guide

Runs automatically on every PR:

• ✅ Pre-checks (submodule verification, code formatting)
• ✅ Aiter standard tests on MI35X
• ✅ Triton tests on MI35X (only when aiter/ops/triton/** or related paths are changed)

Extended tests (opt-in via labels):

Label	Tests
ci:mi300x	Run Aiter standard tests on MI300X in PRs; main branch always runs them
ci:gfx1250-ffm-triton	Run the five-shard gfx1250 FFM Triton test suite
ci:triton-300x	Run an additional Triton test job on MI300X in PRs (added automatically when gfx942 configs change); main branch always runs both MI35X and MI300X
ci:triton-355	Run the full Triton test suite on MI35X, not only the tests the change affects
ci:windows	Windows JIT op smoke test on the Radeon R9700 ROCm runner
ci:multi-gpu	Aiter multi-GPU tests on the 8-GPU runner
ci:sglang	SGLang integration tests: DeepSeek-R1-MXFP4 accuracy, Qwen 3.5 accuracy
ci:atom	ATOM benchmark: DeepSeek-R1-0528, GPT-OSS-120B
ci:atom_full	ATOM accuracy suite for PR and main models from ATOM models_accuracy.json
ci:vllm	vLLM benchmark: DeepSeek-V4-Pro, MiniMax-M3-MXFP4
ci:all	All standard extended tests (excludes ci:atom_full)

Only add ci:atom_full for FlyDSL or Triton upgrades.
Add labels via the sidebar or gh pr edit 7754 --add-label <label>

One backend per PR:
A PR changes one kernel backend: [Triton/Gluon] (Triton and Gluon count as one), [HIP], [ASM], [CK], [OPUS] or [FlyDSL]. If the title ends up with two backend tags, split the PR -- as stacked pull requests when one part cannot merge without the other.

PR title tags & labels:
Component tags ([Triton/Gluon], [HIP], [CK], [ASM], ...) are added to the PR title and as PR labels automatically from the changed files and re-synced on every push — change-type tags like [fix]/[Perf], op tags like [MLA], and human labels (ci:*) are left untouched. Add the no-auto-title label to stop the title rewrites; labels stay in sync either way.

Reviews:
A reviewer is usually assigned within one working day after the PR is ready for review (see Code Review Process). If nobody has responded for about a week, mention the area contact listed in CONTACTS.md. Want to help review? See REVIEWERS.md.

[aiter-bot]

WAITING ON AUTHOR

PR Self-Review

Revision dce26cb1ecdc · mode shadow

Waiting for the PR author. The readiness report will appear after the self-review answers are complete for this revision.

Questions to answer in the PR description

1. What behavior does this PR change, and why is the existing behavior insufficient?
2. What did you validate, and what remains unverified? Include commands, hardware/configuration and results.
3. Which architecture, dtype and shape boundaries select the changed kernel path, and what invariant prevents unsupported inputs from reaching it?
4. For every changed op_tests/ file, what behavior does it cover and why is it not redundant? Write None if no op test changed.

Edit the PR Self-Review section in the PR description. Include the current revision SHA; this comment refreshes automatically.

Missing or invalid answers

• FAIL — Reviewed revision must contain at least the first 7 characters of the current head SHA

_{Shadow mode reports what would block but keeps the status successful. Set AITER_PR_SELF_REVIEW_MODE=enforce only after reviewing real false positives.}