[chushic]

Summary

This PR speeds up the compact-dispatch MegaMoEV2 Stage1 kernel (EP8 dispatch producers + persistent FP8 x MXFP4 grouped GEMM1 + SwiGLU + MXFP8 quant) on gfx950. Output is bit-identical to main.

• Kernel (aiter main, FlyDSL 0.4.0, DeepSeek-V4-Pro a8w4, EP8, MTPR 8192, cold L2): the prepare + Stage1 region is 1.07-1.24x faster per bucket, and the Stage1 kernel alone 1.08-1.32x. Weighted by the calls of a live DeepSeek-V4-Pro server: 1752 → 1490 µs per call (1.176x, worst process 1.169x). No measured shape is slower.
• E2E, CI flags (V4-Pro TP8/DP8/EP8, agentic c128, n=3 vs 3, same node): output tok/s +3.44 % after warm-up and +3.11 % over the full 35 min, and the runs do not overlap. TTFT p50 −6.6 %, interactivity p50 +5.8 %.
• E2E, current cookbook recipe: +2.61 % / +1.37 %, with a small overlap between arms. Decode runs on the fixed-slot instance there, which this PR does not change.
• Accept length is unchanged, there are 0 request errors, and no MegaMoE kernel JIT-compiles at runtime with the AOT cache built by the repo tool.

Where it applies: the producer change applies to every compact-dispatch Stage1 variant. The new GEMM1 tile bodies apply only to compact, route-major, FP8-activation/FP8-output Stage1. Fixed-slot dispatch, A4W4, the indexed payload (MTPR ≥ 32768) and standalone compile_gemm1 keep main's GEMM code.

Motivation

On DeepSeek-V4-Pro with the InferenceX CI recipe, MegaMoE Stage1 runs in every decode and prefill step, and the profile put it at about 15 % of GPU time. On main it had three costs:

• At decode, the dispatch producer was bound by latency, not bandwidth. It visited every task serially, most tasks are empty, and each publish paid several system-scope release fences.
• At prefill, the sort_block_m=128 GEMM1 tile lost time to LDS bank conflicts, to vmcnt(0) waits that serialized the A copy with the MFMAs, and to VGPR spills.
• At decode, the smallest buckets had one HBM round trip per K step.

Changes

All changes are in aiter/ops/flydsl/kernels/mega_moe/{dispatch,gemm1,gemm_util,mega_moe_stage1}.py. The API, the workspace layout, the configs and the dispatch wire protocol are unchanged.

Dispatch producer (emit_dispatch_payload, every compact variant)

• Lane-parallel prefilter. One lane per task classifies a window of 64 tasks and loads their plan entries. A wave scan then skips the empty tasks, instead of visiting them one by one.
• Wave-per-task mode. When every task in the window has ≤ 16 rows, each wave copies and publishes its own task without a CTA barrier, and it writes the route headers lane-parallel. Larger tasks keep the CTA-wide copy, including main's two-rows-in-flight fast copy.
• Fewer fences and atomics. I removed the per-task PAYLOAD_CHUNK_DONE update. Nothing reads that counter, and each update cost one system release, one atomic and two barriers. Each task now issues one system release before it publishes all its tile ranges, instead of one per range. The extra release fence before the ready-queue epoch store is gone, since that store is already a system-scope release.
• The fan-out pair is loaded once per producer instead of once per task.
• Wire protocol unchanged. Every task still publishes the same rows to the same tiles, and the ready queue is pushed by every sender variant. Ranks pick their Stage1 variant from their own token count, so senders and receivers in different variants must interoperate.

Consumer. The per-tile system acquire is now done once by the thread that waited for the tile, before the CTA barrier. On main, every thread did it after the barrier, and an all-thread acquire is a CU-wide cache invalidate per wave. This relies on the acquire invalidating the CU and L2 caches, and on the barrier ordering the other waves' loads after it. Please look at this one closely.

GEMM1 tile bodies (build_fused_gemm1(..., tuned=True))

• sort_block_m=128 prefill tile (512 columns, 8 waves, LDS-DMA A, A-major):
  • The E8M0 A scales are staged once per tile, pre-packed as the i32 words the MFMA consumes, so each K step reads them with one ds_read_b128 instead of 16 ds_read_u8 plus shifts.
  • The fp32 C tile goes through an XOR-swizzled LDS shuffle. This removes the 4-way bank conflicts of the row-major write (35.9 % → 0.8 % conflict cycles, profiled on an earlier aiter build). The quant pass still reads 8 columns per lane, as main's 8-column epilogue does.
  • Alias scopes tell LLVM that the A and A-scale ds_reads never alias the in-flight LDS-DMA, which always writes the idle ping-pong half. SIInsertWaitcnts therefore stops inserting a vmcnt(0) before the first A read of each K step.
  • All next-step B packs are issued at the top of the K step. The K loop is unrolled by two, so the B double buffer alternates register sets with no loop-carried copies. The ping-pong offsets pass through an opaque SGPR, so LLVM no longer hoists their addresses out of the persistent loop and spills them at the 256-VGPR cap (scratch 200 → 48 B/lane on that build).
• Other LDS-DMA variants: the A tile base is wave-uniform (readfirstlane), so the LDS-DMA buffer descriptor stays in SGPRs, with no waterfall loop around each buffer_load ... lds.
• Register-staged sbm32 variant (the 1-4 token buckets): A moves to LDS-DMA with prefetch distance 2, through a 3-stage LDS ring. B and the B scales also run two K steps ahead. The Stage1 LDS pool grows by one A stage for this variant only.
• Order of operations unchanged. Every accumulator sums the same products in the same order, and the epilogue math is unchanged, so outputs are bit-identical.

Relation to #6127 (merged). #6127 added the 8-column GEMM1 quant epilogue, the compact fast copy, the fixed-slot paired-K loop and dedicated LDS broadcast slots. This PR keeps all of them. The prefill-tile epilogue is #6127's 8-column pass on a swizzled C tile, and the fast copy still runs for CTA-mode tasks. The rest is new: the prefilter, wave mode, removed counter and fence changes, consumer acquire placement, and all three GEMM bodies. The microbench below is measured on top of #6127.

Guard. TUNED_GEMM1 = compact and not indexed_payload and a_dtype == out_dtype == "fp8" sits in compile_mega_moe_stage1. build_fused_gemm1 picks a new tile body only when the variant matches it exactly, and every other variant (and tuned=False) runs main's do_tile code path. There is no new environment variable. Rollback is a revert.

An earlier, more aggressive revision was dropped. It added a static first work unit, decode split-K tails, work stealing and fast SwiGLU, and it was about 1.38x vs 1.23x on the same isolated test. Its decode variants skipped the ready-queue publish on the sender side, so the receive protocol depended on the sender's variant. With uneven DP batches (idle ranks at 1 token, skewed routing), a receiver that consumes in ready order then waits forever, and a live server deadlocked on its first real batches. The 8-rank unit test never mixes variants, so it did not catch this. In e2e, that revision gained no more than this one. This PR is the conservative version. It ran in live servers: on the earlier build in 4 × 60-min runs, and on main in a smoke run plus 6 scored 35-min e2e arms. The mixed-variant tests below cover the failure mode.

Performance

Setup. 8x MI355X (gfx950), ROCm 7.2.0. The Docker image is built from aiter main a29547a and sglang main d0a7565436 (plus a one-line import fix needed for sglang main to serve DeepSeek-V4 on gfx950, unrelated to MoE), FlyDSL 0.4.0. The stock and PR arms use the same image and differ only in these 4 files and their regenerated AOT cache. As of aiter 26f4f1f and sglang bf1a252953 (2026-10-24), nothing under mega_moe/, the AOT tool, the FlyDSL pin or sglang's mega_moe_flydsl.py has changed, and this commit applies to that main unchanged.

Microbench: prepare + Stage1 (µs), DeepSeek-V4-Pro a8w4, EP8

Method. I used my own 8-rank harness, which is not in the repo. It builds a real MegaMoEV2 from test_mega_moe_v2.py's inputs, and each rank picks its config from its own token count, as sglang does. It times _run_fused_stage1 (main's prepare + Stage1) with CUDA events on the slowest rank, with a 1 GiB L2 flush before every call. Decode runs under CUDA graph and prefill eager, as served. "Stage1 kernel" is the profiler time of the Stage1 kernel alone (max over ranks of the per-rank median). Stock and PR runs are interleaved in fresh processes, 5 per arm at MTPR 8192 and 3 at MTPR 16384. Values are medians over processes, with the range in brackets; "worst" = slowest PR process vs fastest stock process.

tokens/rank	stock	+ PR	speedup median / worst	Stage1 kernel
1 (graph)	177.0 (172.5-179.3)	144.5 (143.8-146.0)	1.23x / 1.18x	1.32x
7 (graph)	267.9 (259.0-269.3)	233.6 (233.3-240.9)	1.15x / 1.07x	1.17x
42 (graph)	362.0 (360.7-364.5)	335.0 (334.2-341.2)	1.08x / 1.06x	1.09x
224 (graph)	411.1 (405.2-413.2)	377.2 (374.6-378.8)	1.09x / 1.07x	1.10x
452 (eager)	446.9 (441.7-448.6)	405.7 (404.6-407.9)	1.10x / 1.08x	1.10x
868	554.0 (548.7-564.2)	490.0 (484.0-493.0)	1.13x / 1.11x	1.15x
2394	1005.0 (1000.3-1012.9)	871.3 (869.7-879.2)	1.15x / 1.14x	1.18x
4108 (sbm128 tile)	1419.5 (1415.0-1432.1)	1145.5 (1135.6-1151.1)	1.24x / 1.23x	1.27x
8192 (sbm128 tile)	2438.4 (2433.9-2456.2)	2062.7 (2050.4-2067.4)	1.18x / 1.18x	1.16x
mixed per-rank 1,6,42,224,452,1548,4108,8192	1569.2	1416.2	1.11x / 1.10x	1.11x
MTPR 16384: 1 / 452 / 8192 / 16384	173.2 / 442.5 / 2422.2 / 4596.8	145.6 / 409.1 / 2034.8 / 3852.9	1.19x / 1.08x / 1.19x / 1.19x	1.07-1.27x

• Live-weighted. Over the 51 Stage1 shapes recorded from a live V4-Pro server with the CI flag set (nearest measured point): 1752 → 1490 µs per call, 1.176x (worst 1.169x). Decode shapes 1.094x, prefill 1.182x; 8192 tokens carries 66.6 % of the weight.
• More points. 23 uniform buckets from 1 to 8192 tokens: every median is ≥ 1.07x. The only worst-case value below 1.0 is 0.99 at 14 tokens, where the median is 1.09x. 8 mixed per-rank points: 1.06-1.19x. Zipf-skewed routing: 1.11-1.19x.
• Guarded paths. Fixed-slot (MTPR 512 with AITER_MEGA_FIXED_SLOT_MAX_MTPR=1023, the cookbook decode instance): 1.00x and bit-exact. Paths that keep main's GEMM but get the producer change, 1 run each: a4w4 1.07-1.11x, MTPR 32768 1.03-1.10x.
• Other networks (1 run each): kimi_k3_route 1.19-1.28x, R1 1.08-1.32x, GLM-5.2 1.07-1.21x, all bit-exact.
• Repo test timing. test_mega_moe_v2.py's own Stage1 timing (warm L2, graph, mean over ranks) shows the same picture: 1 token 168 → 138 µs, 8192 tokens 2461 → 2008 µs.

End to end: DeepSeek-V4-Pro, TP8 / DP-attention 8 / EP8, MegaMoE a8w4, MTP, c128

Setup. Agentic trace replay with aiperf (InferenceX dsv4/sglang/mi355x-fp4-mtp/agentic.yaml, variant override_tp8_c128_dp_ep8_megamoe_hicache, seed 42, simulated accept length 3.77). Each arm gets a fresh container and fresh caches. The PR arm ships only the 4 files plus the cache written by python -m aiter.aot.flydsl.mega_moe --quant a8w4, and prepare is main's. Warm-up requests are dropped, then a 35-min profile runs. "[10,35]" scores the requests that start after minute 10 and end by minute 35; "full" is the whole profile. Interactivity = 1/ITL. Runs alternate PR, stock, PR, stock on one node per flag set. Flag sets:

• CI: the flags of the InferenceX CI job (SGLANG_AMD_FLYDSL_MEGA_MOE_MTPR=8192). Every decode and prefill step runs compact Stage1.
• cookbook: the current recipe: MTPR 16384, SGLANG_AMD_FLYDSL_MEGA_MOE_DECODE_MTPR=512, AITER_MEGA_FIXED_SLOT_MAX_MTPR=1023. Pure-decode steps run the fixed-slot instance, which this PR does not speed up.

flags	window	stock tok/s (runs)	+ PR tok/s (runs)	Δ	stock spread	min(PR) vs max(stock)	TTFT p50	interactivity p50 / p90
CI	[10,35]	3186.5 (3192.8, 3182.3, 3184.5)	3296.2 (3274.7, 3306.9, 3306.9)	+3.44 %	0.33 %	+2.57 %	−6.6 %	+5.8 % / +3.7 %
CI	full	3163.6	3262.1	+3.11 %	0.96 %	+2.36 %	−5.6 %	+5.4 % / +4.1 %
cookbook	[10,35]	3306.6 (3271.2, 3280.2, 3368.3)	3392.9 (3341.3, 3420.8, 3416.8)	+2.61 %	2.97 %	−0.80 %	−2.8 %	+1.5 % / +1.8 %
cookbook	full	3293.3	3338.4	+1.37 %	1.55 %	−0.65 %	−2.3 %	+1.0 % / +1.2 %

• CI flags. The two arms separate cleanly: every PR run is above every stock run, and the time-adjacent pairs are +2.6 / +3.9 / +3.8 %. Per-rank decode iteration time at equal batch drops 3.0-6.6 % (1-23 running requests), and the prefill step drops 2.6-5.7 %. The throughput gain is above the ~2.3 % that Stage1's average GPU-time share times the kernel gain would predict. The per-rank step times agree with it, but I have not broken the difference down (with DP attention every step waits for the slowest rank's MoE).
• cookbook flags. The gain is smaller and the arms overlap slightly. All three time-adjacent pairs are positive (+2.1 / +4.3 / +1.4 % on [10,35]), and every step bucket moves the right way (decode −1.1 to −3.2 %, prefill −1.2 to −2.1 %). The third stock run is the highest stock cookbook run I saw on that node, so I read this flag set as a small gain, not a firm number.
• Clean runs. In every scored arm: 0 request errors, accept length 3.764-3.779 in both arms, and no EPLB warm-up hang. One PR run with CI flags overlapped another job on the node; I excluded it and re-ran it.

Accuracy

• Bit-exact vs stock. The per-rank sha256 of the bf16 MoE output is identical between stock and PR at all 78 harness points (v4_pro a8w4/a4w4, MTPR 512/8192/16384/32768, R1, GLM-5.2, kimi_k3_route, Zipf, mixed and zero-token ranks).
• Repo test. test_mega_moe_v2.py passes in all 15 configurations below, with per-bs rel-L2 identical to stock (a8w4 0.0589-0.0593, a4w4 0.212). CUDA graph replay is exact.
• Mixed Stage1 variants. These are the 7 tests under "How to reproduce" with per-rank variants, skew and bursts of up to 2000 forwards. All pass in stock and PR. A negative control that skips the sender-side ready-queue publish, as the dropped revision did, hangs in the 4 skewed cases, so the tests catch that failure mode.

FlyDSL AOT cache

Every Stage1 kernel key changes (the source of build_fused_gemm1 and emit_dispatch_payload is part of the key), so the shipped cache has to be regenerated: python -m aiter.aot.flydsl.mega_moe --quant a8w4 (and --quant a4w4). On main this prints 192 ok, 0 failed, and adds 96 new Stage1 entries per quant.

• With only that cache, the repo test at MTPR 8192 and 16384, with mixed and zero-token ranks, had 0 runtime cache misses and 0 new kernels, and it also passes with FLYDSL_RUNTIME_RUN_ONLY=1.
• No mutable module global reaches the Stage1 launch key. All 96 variants share one call key, and an independent AOT run on a second node produced byte-identical files.
• In the CI-flag e2e arms, 0 MegaMoE kernels compiled at runtime. The MTPR-512 fixed-slot instance of the cookbook recipe is not in the AOT list, on main as well as here, so it JIT-compiles at server startup in both arms.

How to reproduce

run() { MORI_SHMEM_HEAP_SIZE=40G torchrun --standalone --nproc_per_node=8 "$@"; }
T=op_tests/multigpu_tests/test_mega_moe_v2.py
python -m aiter.aot.flydsl.mega_moe --quant a8w4 && python -m aiter.aot.flydsl.mega_moe --quant a4w4
run $T --network v4_pro --quant a8w4 --max-tok-per-rank 8192 --accuracy-max-bs 8192 \
  --bs-list 1,7,14,42,84,112,168,224,452,868,1548,2394,3190,4108,4917,7281,8192
run $T --network v4_pro --quant a8w4 --max-tok-per-rank 16384 --accuracy-max-bs 16384 --bs-list 1,42,224,452,868,2394,4917,8192,16384
U="--max-tok-per-rank 8192 --accuracy-max-bs 8192 --bs-list 8192 --config-tokens 8192 --unify-fields stage1.payload_chunk_rows"
run $T $U --rank-tokens 0,1,42,224,2048,2394,3190,4917
run $T --max-tok-per-rank 8192 --accuracy-max-bs 8192 --bs-list 7,224,2394 --pad-tail-rows 3
run $T --max-tok-per-rank 8192 --accuracy-max-bs 8192 --bs-list 42,452,8192 --zipf 1.2
run $T --max-tok-per-rank 8192 --bs-list 1,7,42,224,8192 --concentrate-experts 1 --burst-depth 2000
run $T --network kimi_k3_route --max-tok-per-rank 8192 --accuracy-max-bs 8192 --bs-list 1,16,128,1024,8192
run $T --network kimi_k3_route --force-fanout-boundary --bs-list 1,64 --burst-depth 2000
run $T --network kimi_k3_route --inject-invalid-route --bs-list 1
for N in r1_v3 glm52; do run $T --network $N --max-tok-per-rank 8192 --accuracy-max-bs 8192 --bs-list 1,42,452,8192; done
run $T --quant a4w4 --max-tok-per-rank 8192 --accuracy-max-bs 8192 --bs-list 1,42,452,2394,8192
run $T --max-tok-per-rank 32768 --accuracy-max-bs 8192 --bs-list 42,2394,8192
AITER_MEGA_FIXED_SLOT_MAX_MTPR=1023 run $T --max-tok-per-rank 512 --accuracy-max-bs 512 --bs-list 1,7,42,224,512
run $T --bs-list 1,42,128,512,1024,4096 --accuracy-max-bs 4096
# mixed per-rank Stage1 variants (each rank keeps its own config; the unified field is already identical)
for R in "1,1,1,1,1,1,42,1" "1,1,1,1,1,1,24,1 --zipf 1.5" "4917,112,112,112,112,112,112,112 --zipf 1.5" \
         "1,6,42,224,452,1548,4108,4917 --zipf 1.5" "1,6,42,224,452,1548,4108,8192 --burst-depth 2000" \
         "1,1,1,1,1,1,42,1 --burst-depth 2000" "4108,42,1,1,4108,42,1,1 --zipf 1.5 --burst-depth 500"; do
  run $T $U --rank-tokens $R; done

E2E. sglang d0a7565436, with server and router flags exactly as in the InferenceX recipe above. The CI flag set is the 2026-10-07 CI job of that recipe; cookbook is the recipe after InferenceX #4315. aiperf agentic replay at concurrency 128, 35-min profile after warm-up. Both arms use the same image; the PR arm bind-mounts the 4 files and the AOT cache from the command above.

Interaction with other PRs

• [FlyDSL] [Perf] MegaMoE: shorten EP8 prepare chain, coalesce MXFP8 quant (gfx950) #6900 (MegaMoE prepare, mine). The two PRs are independent and can merge in either order. They touch disjoint files, and git merge-tree is clean. [FlyDSL] [Perf] MegaMoE: shorten EP8 prepare chain, coalesce MXFP8 quant (gfx950) #6900 imports nine dispatch.py helpers (plan/group emitters, wave scans, fan-out pair loader), and this PR changes none of them; it changes only _publish_tile_range and emit_dispatch_payload. The prepare → Stage1 hand-off (PLAN_READY and the plan arrays) is unchanged on Stage1's side, and [FlyDSL] [Perf] MegaMoE: shorten EP8 prepare chain, coalesce MXFP8 quant (gfx950) #6900 keeps that contract. All numbers here use main's prepare kernel; I have not measured the two together. Whichever lands second, I will rerun the AOT tool, the repo tests and the mixed-variant tests on the merged tree.
• [FlyDSL] [CI] MegaMoEV2: fuse Dispatch, GEMM1, GEMM2 and Combine into one kernel #6881 (fused small-M / large-M MegaMoE, opt-in env). Merges cleanly with this PR, and it uses only dispatch.py and Stage1 helpers that this PR leaves unchanged. Calls it takes over do not run Stage1; all other calls get this PR.
• [FlyDSL] Add full-expert DeepSeek V4.1 Flash MegaMoE tuning for gfx950 #6154 (V4.1-Flash EP2/EP4 tuning, draft). Touches the same files but already conflicts with main. Its pair_k applies to main's do_tile, which this PR keeps for non-matching variants.

Known gaps

• gfx950 only, and DeepSeek-V4-Pro is the only model with e2e data. R1, GLM-5.2 and kimi_k3_route have one microbench run each, plus the repo test.
• Cookbook recipe: smaller gain with slight overlap between arms (see above), because pure-decode steps run the unchanged fixed-slot instance.
• Consumer acquire placement (see Changes) is a memory-model argument. It ran clean in ~10k mixed-variant bursts and in all e2e arms, but I would like a second pair of eyes on it.
• Repo test harness issue, not caused by this PR (stock too). With --unify-fields and a rank in the 8192 bucket, only that rank runs the extra aligned-pair forward in _run_size, which deadlocks. The mixed tests above cap that rank at 4917 unless they run in burst mode. I can send a separate fix.
• My microbench harness is not in the repo. I can add it as a test-only change if that helps.

Testing

test_mega_moe_v2.py and bench_mega_moe_v2.py are in skip_tests of .github/scripts/aiter_test.sh, so aiter CI runs no test of this kernel. All runs above were manual, on 8x MI355X with main a29547a, which is identical to current main for every file involved.

• Unit tests run: op_tests/multigpu_tests/test_mega_moe_v2.py (8 GPU, 15 configurations + 7 mixed-variant tests, see above)
• Performance benchmarks run (kernel microbench and e2e, see Performance)
• Tested on MI250X: not applicable (gfx95x-only kernel)
• Tested on MI300X: not applicable (gfx95x-only kernel)
• Tested on MI355X (gfx950)

Documentation

• Docstrings updated
• User guide updated: not needed
• Performance guide updated: not needed

Dependencies

• No new third-party dependencies added

Breaking Changes

None. build_fused_gemm1 gains an optional tuned=False and gemm1.py gains gemm1_a_lds_stages. The PAYLOAD_CHUNK_DONE workspace slot stays allocated but is no longer written.

[github-actions]

🏷️ CI Guide

Runs automatically on every PR:

• ✅ Pre-checks (submodule verification, code formatting)
• ✅ Aiter op tests (gfx942 + gfx950)
• ✅ Triton tests on MI35X (only when aiter/ops/triton/** or related paths are changed)

Extended tests (opt-in via labels):

Label	Tests
ci:gfx1250-ffm-triton	Run the five-shard gfx1250 FFM Triton test suite
ci:triton-300x	Run an additional Triton test job on MI300X in PRs (added automatically when gfx942 configs change); main branch always runs both MI35X and MI300X
ci:triton-355	Run the full Triton test suite on MI35X, not only the tests the change affects
ci:windows	Windows JIT op smoke test on the gfx1201 ROCm runner
ci:multi-gpu	Aiter multi-GPU tests on the 8-GPU runner
ci:sglang	SGLang integration tests: DeepSeek-R1-MXFP4 accuracy, Qwen 3.5 accuracy
ci:atom	ATOM benchmark: DeepSeek-R1-0528, GPT-OSS-120B
ci:atom_full	ATOM accuracy suite for PR and main models from ATOM models_accuracy.json
ci:vllm	vLLM benchmark: DeepSeek-V4-Pro, MiniMax-M3-MXFP4
ci:all	All standard extended tests (excludes ci:atom_full)

Only add ci:atom_full for FlyDSL or Triton upgrades.
Add labels via the sidebar or gh pr edit 7001 --add-label <label>

One backend per PR:
A PR changes one kernel backend: [Triton/Gluon] (Triton and Gluon count as one), [HIP], [ASM], [CK], [OPUS] or [FlyDSL]. If the title ends up with two backend tags, split the PR -- as stacked pull requests when one part cannot merge without the other.

PR title tags & labels:
Component tags ([Triton/Gluon], [HIP], [CK], [ASM], ...) are added to the PR title and as PR labels automatically from the changed files and re-synced on every push — change-type tags like [fix]/[Perf], op tags like [MLA], and human labels (ci:*) are left untouched. Add the no-auto-title label to stop the title rewrites; labels stay in sync either way.

[github-actions]

No activity for 15 days, so this is now labelled stale. A push or a comment clears it; keep-open exempts it. Nothing is closed automatically.