[jatseng-ai]

What this does

On gfx942, a BF16 QuantType.No fused MoE with SiLU or SwiGLU and a set swiglu_limit lands on CK two-stage. CK has no swiglu_limit parameter, so the bound is silently dropped (AITERKER-247). This PR routes that case to the existing FlyDSL a16wmix port and makes the port apply the limit.

There are two separable changes, in separate commits:

1. Open the bf16×bf16 dtype gate (3ad2b21a4). The a16wmix port already handled bf16 A × {mxfp4, int4} W. This adds the a16w16 (bf16 W) registry, compile, and dispatch entries, plus the host heuristic _can_reroute_bf16_to_a16wmix. The port's SwiGLU epilogue already clamped, so SwiGLU is covered by this commit alone.
2. Clamp SiLU in the a16wmix epilogue (f84139457). SiLU now runs clamp_gate_up, then the unchanged, unclamped silu_mul_batch. The shared helper is deliberately not modified, because mixed_moe already clamps before calling it.

The remaining commits are supporting:

3. a4ecba2c4: stop logging using 2stage default when the a16wmix reroute is about to return. That line named the CK choice being discarded. The reroute's own untuned fallback (kn1=..., kn2=...) warning names the kernel that actually runs. All other paths log as before.
4. cb2ed1316: make the path tunable. See Tuner.
5. db1a740c5: tests at the qualified shape, and a stricter sentinel test.
6. 7af3c2341: dict(...) → literals in the test helpers (ruff 0.16 C408, which CI's Check Code Style with Ruff enforces).
7. 230d6b1c8: the tuner stops enumerating a16w16 stage-2 *_reduce kernels. bf16 stage 2 has no reduce epilogue, so those names atomic-scattered into an unzeroed buffer and failed accuracy (#6724).
8. 02cb2b718: AITER_FLYDSL_FORCE_REDUCE=1 no longer treats bf16×bf16 as honoring the override (same as bf16×fp4/int4). Sorting still zero-inits; stage2 stays on the atomic epilogue.
9. 42e98fa14: gate the reroute to gfx942/gfx950, reject a mismatched caller block_size_M before stage1, register bf16 a16w16 names only, and give AOT a QuantType.No bf16×bf16 recipe. No AITER_* debug knob.
10. a2e3318a9: fold the clamp/dispatch tests into op_tests/test_moe_a16w16_default_dispatch.py and drop the extra file.

Clamp semantics, both activations: gate min(g, L) (upper bound only), up clamp(u, -L, L), then silu(g)*u or SwiGLU-OAI g*sigmoid(1.702g)*(u+1).

Blast radius

The route is taken only when swiglu_limit is positive and finite. None, 0.0 and +inf keep the existing CK two-stage path (ck_moe_stage1, run_1stage=False) and its numerics. 0.0 matters because vLLM's ROCm adapter sends it whenever the model config has no limit. Every existing caller, none of which sets a limit, is unaffected.

The reroute also requires all of the following; otherwise behavior is unchanged:

• live GPU is gfx942 or gfx950 (get_gfx_runtime()); gfx90a and RDNA stay on CK
• bf16 activations and weights, QuantType.No, SiLU or SwiGLU
• use_g1u1, preshuffled weights, no stage-1 bias, no doweight_stage1
• no padding, and model_dim % 128 == 0, inter_dim % 128 == 0

A tuned row whose kernelName1 starts with flydsl_ is respected. A CK or asm row is overridden only when the limit is active, because neither can clamp.

There is no AITER_* override. To time the same a16wmix kernel without a binding clamp, pass a huge finite limit (1e9); sentinels stay on CK. A caller block_size_M that disagrees with the selected tile is rejected at the host. Registry/AOT/tuner emit bf16 a16w16 names only.

Evidence

Kernel accuracy and routing were measured on gfx942, 304 CU, FlyDSL 0.4.1, torch 2.14.1+rocm7.2. The earlier server-side clamp-bind numbers used image vllm/vllm-openai-rocm:v0.30.0.

Kernel selected inside a vLLM server (Command A geometry, TP8, AITER MoE on, swiglu_limit=7, no tuned rows):

padded tokens	run_1stage	stage 1	kernelName1
1, 2, 8, 16, 32, 64, 128, 512	False	_flydsl_stage1_wrapper	flydsl_moe1_abf16_wbf16_bf16_t16x128x128
4096	False	_flydsl_stage1_wrapper	flydsl_moe1_abf16_wbf16_bf16_t32x128x128

This covers both decode and prefill. With the limit unset (adapter sends 0.0), every one of those padded counts took ck_moe_stage1.

The clamp binds in the server. Weights were scaled so the pre-activation gate is in the hundreds against a bound of 7. Layer-0 routed-MoE output absmax was 12–17 with the clamp, versus 1728–2352 on the unset (CK) server.

Kernel level, at the server's per-rank shape (H=4096, I=512, E=128, top-8, limit 7). The table gives max-abs error against each reference:

tokens	vs clamped reference	vs unclamped reference
1 – 256	9.4e-4 – 3.7e-3	1.6e9 – 7.4e9

Asymmetry. Gate fixed at −25 with limit 1.0: the error is 0.021 against the asymmetric reference and 20.7 against a symmetric clamp. The negative side is not floored.

This PR's tests (op_tests/test_moe_a16w16_default_dispatch.py a16w16 clamp cases, 17 passed). The server-shape tests (32 tokens, t16 tile) report:

• limit 7: error 0.64 against the clamped reference, whose own absmax is 59.5. The error against the unclamped reference is 3014.
• the bound is reachable: routed gate max is 213, routed |up| max is 212.
• limit 1e9 gives a different output from limit 7, and matches the unclamped reference to 26.9 (that reference's absmax is 3037).
• gate −25 at limit 1.0: 0.0024 against the asymmetric reference, 1.07 against symmetric.

Kernel latency, same box, head 42e98fa14 (kernel unchanged in a2e3318a9). gfx942 304 CU, torch 2.14.1+rocm7.2, HIP 7.2.53211, AITER_BYPASS_TUNE_CONFIG=1, CUDA-event median of 50 iters after 10 warmup. Command A per-rank shape (H=4096, I=512, E=128, top-8, SiLU). Arms: CK (swiglu_limit=0.0, the pre-patch path) vs a16wmix with the clamp (7.0) vs the same a16wmix kernel with a no-op limit (1e9).

tokens	CK µs	a16wmix L=7 µs	a16wmix L=1e9 µs	L=7 / CK	L=7 / L=1e9	a16wmix kn1
1	194.4	248.0	246.8	1.28×	1.00×	t16x128x128
4	239.3	274.1	270.7	1.15×	1.01×	t16x128x128
32	513.2	511.9	510.8	1.00×	1.00×	t16x128x128
256	662.8	712.3	708.3	1.07×	1.01×	t16x128x128
4096	1551.3	2514.0	2517.9	1.62×	1.00×	t32x128x128

The clamp itself is free (L=7 matches L=1e9 to 1%). Decode (t=1/4) is 15–28% slower than CK on the untuned t16 fallback. t=32 matches CK. Prefill (t=4096) is 1.62× CK because CK uses block_m=128 while the fallback is still t32; that is a tile/heuristic gap, not clamp cost. No tuned a16w16 CSV row is in this PR.

Tuner

Before this PR, TUNE_ONLY=flydsl returned nothing for BF16 QuantType.No, because gen_flydsl_2stages_task admitted only per_1x32 fp4/fp8 weights. No tuned row could name this kernel. flydslv2 is the fused-quant layout-v2 family (per_1x32 fp4/fp8 only), so it does not cover bf16 either. cb2ed1316 does four things:

• enumerates get_flydsl_stage{1,2}_kernels_a16w16 through the existing a16w-mix branch. That is 84 candidates (76 stage 1, 8 stage 2) at the Command A shape, against 0 before. It was 92 until 230d6b1c8 dropped the 8 stage-2 *_reduce names.
• times stage 1 with waves_per_eu=None, which is what the runtime passes for these entries. The tuner's default of 3 would time a different build.
• builds the sorted bf16 stage-2 input for a16w16, as it already did for a16w4.
• isolates a16w16 from CK when pairing stages. CK also enumerates these keys. An a16w16 stage 1 writes a sorted intermediate that CK stage 2 cannot read, and the tuner times stages separately. Without isolation, dedup plus the stage-1/stage-2 join could emit an invalid mixed row. In the check run below, the two stage-2 candidates differed by 0.0065 µs, so timing noise alone would have decided between a valid pair and an invalid one. An a16w16 marker now sits in the dedup and join keys, like v2, and is dropped before the CSV.

To check enumeration, a regex-limited tuner run on one Command A row produced a complete row naming flydsl_moe1_abf16_wbf16_bf16_t16x128x128 + ..._t16x128x128_atomic (errors 0.0% / 0.6%). With CK candidates included at block_m=32, the row it picked was a homogeneous pair. This was not a sweep, and no tuned CSV row is committed.

Tuning notes:

• The tuner does not see swiglu_limit. The tuning key has no limit column, so one tuned row serves clamped and unclamped callers of the same shape. If a CK row wins, the runtime still overrides it with the untuned a16wmix fallback when a limit is active. To get a tuned a16wmix row for a clamped model, tune with TUNE_ONLY=flydsl.
• --run_config can check the clamp end to end. Set AITER_MXFP4_TUNE_SWIGLU_LIMIT (despite the name it applies to all quant types). The check only means something if its inputs exceed the bound.

Known limitations

• Untuned. No 304-CU tuned rows exist for any model; every shipped tuned_fmoe.csv row is cu_num=80. The server takes the fallback tile shown above. Prefill 1.62× vs CK is that fallback, not the clamp.
• End-to-end TPOT vs Triton is still unmeasured. The table above is same-box fused_moe CUDA-event time, CK vs this path. A TPOT comparison against TritonExperts is not in this PR.
• Sentinel outputs are not bit-identical to main. Max abs difference is 3.6e-7–9.5e-7, about one bf16 ulp. A second launch of the same #6552 binary against the first differs by the same amount (2.4e-7–9.5e-7). This is CK stage-1 run-to-run nondeterminism, not the clamp leaking into the sentinel path. Routing is unchanged: sentinels select ck_moe_stage1 with run_1stage=False, now asserted by the tests.
• Qualified at one shape: H=4096, per-rank I=512, E=128, top-8, SiLU. Other shapes that meet the conditions above are structurally eligible but not measured. SwiGLU is covered by the small-shape bind tests only.
• Not covered: the FP8 whole-graph path (fused_moe_gfx942.py), the asm 1-stage path for non-g1u1 bf16, and other CK paths. #6335 is the fail-loud guard for those.

Base and relation to #6335

Ten commits on upstream/main at f1965cd26. Rebase was clean (no conflict hunks). None of #6335's commits are in this branch, so it can merge on its own. gfx942 a16wmix stage 2 is unchanged from main: fx.atomic_add on a typed bf16 pointer. The llvm.AtomicRMWOp commit was dropped after review. It scaled the byte offset in Int32, and it existed only for FlyDSL 0.3.2, which this repo does not pin. #6335's guard already treats _flydsl_stage1_wrapper as clamp-applying, so it does not raise on this route.

If both land, the second must drop its copy of _swiglu_limit_is_active. Both PRs define the function identically and the texts merge without conflict, but ruff F811 (redefinition) would fail.

Test plan

• op_tests/test_moe_a16w16_default_dispatch.py a16w16 clamp/dispatch cases on gfx942 / 304 CU: 17 passed (folded out of a dedicated file). Routing, sentinels, gfx90a stays on CK, block_size_M mismatch, SiLU + SwiGLU bind at 4 tokens, Command A shape bind and asymmetry.
• Tuner enumeration at BF16 QuantType.No (0 → 84 a16w16 candidates; asm and flydslv2 0), plus a regex-limited end-to-end tuner run
• black 26.10 and ruff 0.16.0 (the versions CI pins) clean on changed files
• Same-box kernel timing CK vs a16wmix at Command A shape (table in Evidence)
• TPOT against TritonExperts (not in this PR)
• Do not squash: commit 1 (dtype gate, SwiGLU) and commit 2 (SiLU epilogue) are intentionally separate

PR Self-Review

Reviewed revision

a2e3318

Behavior and intent

CK two-stage silently drops swiglu_limit for BF16 QuantType.No SiLU/SwiGLU. This PR opens the a16wmix a16w16 (bf16×bf16) dtype gate and clamps SiLU in that port so a finite limit binds. Sentinels (None / 0.0 / +inf) stay on CK, so existing unclamped callers keep the previous path.

Validation and unverified scope

• python -m pytest op_tests/test_moe_a16w16_default_dispatch.py -k 'not ck_default and not qwen_mtp and not gfx950' on gfx942 304 CU, head a2e3318a9: 17 passed.
• Same-box CUDA-event fused_moe timing at H=4096 I=512 E=128 top-8 SiLU, AITER_BYPASS_TUNE_CONFIG=1: clamp vs no-op a16wmix is 1.00×; vs CK the untuned fallback is 1.00× at 32 tokens and 1.62× at 4096 tokens (table in Evidence).
• Unverified: end-to-end TPOT against TritonExperts, SwiGLU at the Command A shape, gfx950 runtime (compile path only), and any shape outside H=4096 I=512 E=128 top-8.

PR-specific question

The changed kernel path is selected only when all of these hold: live get_gfx_runtime() is gfx942 or gfx950, bf16×bf16 QuantType.No, SiLU or SwiGLU, g1u1, preshuffled weights, no stage-1 bias, no doweight_stage1, no pad, model_dim and inter_dim divisible by 128, and swiglu_limit positive and finite. _can_reroute_bf16_to_a16wmix returns false otherwise, so gfx90a/RDNA, sentinels, and every existing unclamped caller stay on CK. A caller block_size_M that disagrees with the a16w16 tile is rejected before launch. Registry, tuner, and AOT emit bf16 names only.

Changed op tests

op_tests/test_moe_a16w16_default_dispatch.py already covers CK a16w16 default dispatch. The clamp/reroute cases live there now (no extra file). test_moe_2stage.py is a CLI harness and _effective_swiglu_limit returns None for QuantType.No, so it cannot catch a silent CK drop of the bound, a gfx90a compile-path leak, an f16 name alias, or a block_size_M vs tile mismatch. Dropped: silu_mul_batch source grep, cartesian sentinel×activation, tokens=256 bind, and the small-shape asymmetric duplicate of the Command A case.

[github-actions]

🏷️ CI Guide

Runs automatically on every PR:

• ✅ Pre-checks (submodule verification, code formatting)
• ✅ Aiter op tests (gfx942 + gfx950)
• ✅ Triton tests on MI35X (only when aiter/ops/triton/** or related paths are changed)

Extended tests (opt-in via labels):

Label	Tests
ci:gfx1250-ffm-triton	Run the five-shard gfx1250 FFM Triton test suite
ci:triton-300x	Run an additional Triton test job on MI300X in PRs (added automatically when gfx942 configs change); main branch always runs both MI35X and MI300X
ci:triton-355	Run the full Triton test suite on MI35X, not only the tests the change affects
ci:windows	Windows JIT op smoke test on the gfx1201 ROCm runner
ci:multi-gpu	Aiter multi-GPU tests on the 8-GPU runner
ci:sglang	SGLang integration tests: DeepSeek-R1-MXFP4 accuracy, Qwen 3.5 accuracy
ci:atom	ATOM benchmark: DeepSeek-R1-0528, GPT-OSS-120B
ci:atom_full	ATOM accuracy suite for PR and main models from ATOM models_accuracy.json
ci:vllm	vLLM benchmark: DeepSeek-V4-Pro, MiniMax-M3-MXFP4
ci:all	All standard extended tests (excludes ci:atom_full)

Only add ci:atom_full for FlyDSL or Triton upgrades.
Add labels via the sidebar or gh pr edit 6552 --add-label <label>

One backend per PR:
A PR changes one kernel backend: [Triton/Gluon] (Triton and Gluon count as one), [HIP], [ASM], [CK], [OPUS] or [FlyDSL]. If the title ends up with two backend tags, split the PR -- as stacked pull requests when one part cannot merge without the other.

PR title tags & labels:
Component tags ([Triton/Gluon], [HIP], [CK], [ASM], ...) are added to the PR title and as PR labels automatically from the changed files and re-synced on every push — change-type tags like [fix]/[Perf], op tags like [MLA], and human labels (ci:*) are left untouched. Add the no-auto-title label to stop the title rewrites; labels stay in sync either way.

[jatseng-ai]

P1 vs Triton (same exclusive node/image as the PR body). Still draft.

TritonExperts does apply the clamp (same vs-clamped error as a16wmix to printed precision). fused_experts() without activation_config does not — that is not Cohere's production path.

UNTUNED a16wmix median µs vs TritonExperts (clamped) vs CK unclamped SOL (invalid): 279/201/222 (t=1), 280/217/228 (t=4), 284/216/232 (t=32), 317/219/235 (t=256). gfx942 has no K=32 bf16 MFMA; no tuned rows. Losing while untuned is not a lane verdict.

No accuracy run (no checkpoint). FP8 Family A not in this PR.

[jatseng-ai]

Rebased onto current main (395babca1). No conflicts. The bf16 a16wmix reroute is still on the gfx942 dispatch path. ruff check (0.16.0) and black are clean on the changed files. Still draft, so the full CI suite remains skipped.

[jatseng-ai]

Rebased onto current main (1c464e51b), which clears the merge conflict.

• Two conflicts with [FlyDSL] Simplify kernel APIs and remove redundant wrappers #6380 (FlyDSL kernel cleanup), both in the a16wmix port. Each was resolved by adopting main's new style, with no semantic change. gemm1.py: situ_params is still built unconditionally so SiLU sees the limit, now without fx.Float32(...) wrappers. gemm2.py: the gfx942 stage-2 atomic is still the unconditional llvm.AtomicRMWOp. Commits 4–8 rebased unchanged.
• op_tests/test_a16wmix_bf16_clamp.py on gfx942 (MI300X), head 5c98a2e8d, FlyDSL 0.4.0, torch 2.14.1+rocm7.2: 23 passed. [FlyDSL] Simplify kernel APIs and remove redundant wrappers #6380 also moved sigmoid_batch to fx.exp2(fastmath="afn"), and the clamp tolerances still hold.
• ruff check 0.16.0 and black 26.10.1 are clean on the changed files.
• The PR body now has the current commit SHAs and the 8th commit (5c98a2e8d, [Bug][gfx942] a16w16 stage2 *_reduce kernels are wrong (~99.7% of elements): bf16 mode=reduce still atomic-scatters into an unzeroed buffer #6724). The tuner count is now 84 a16w16 candidates (76 + 8), down from 92 after the *_reduce stage-2 names were dropped.

[jatseng-ai]

Rebased onto current main (2fa9f7e7c). No conflicts. The patch is unchanged from 5c98a2e8d.

• op_tests/test_a16wmix_bf16_clamp.py on gfx942 (MI300X), head ed8b6561b: 23 passed.
• ruff check 0.16.0 and black 26.10.1 are clean on the changed files.
• CI on the previous head was green (37 passed, including Standard Tests and Aiter Test Gate). This push re-runs it.

vLLM has said they will review vllm#61508 after this PR and #6335 are merged and in an aiter release.

[Bernard-Liu]

gfx942 stage-2 atomic: the Int32 byte offset wraps at 2^30 elements; suggest dropping c470ea11b

aiter/ops/flydsl/kernels/moe_2stage_a16wmix/gemm2.py#L141-L148

This commit does more than swap the atomic API: it also changes the address arithmetic. Before, the gfx942 branch offset a typed bf16 pointer by elements (out_elems + out_off), so the scaling to bytes happened inside the 64-bit GEP. Now the offset is scaled in Int32 first (out_off * fx.Int32(2)) and the result is used as a byte offset. Once out_off reaches 2^30, the product wraps to a negative value and the atomic add lands before the start of the output buffer.

Example: M=131073, model_dim=8192 gives an output of 2,147,500,032 bytes. That is just over 2 GiB but still below the 4 GiB requires_flydsl_stage2_reduce threshold, so the call stays on this atomic path, and the first element of the last row sits at out_off = 2^30. The old code was correct for any offset up to 2^31 - 1 elements. The failure mode is a silent out-of-bounds write, not an exception.

The kernel is shared, so this affects every gfx942 atomic stage 2 of the a16w-mix port, not only the new a16w16 path:

• a16w4 (bf16 x mxfp4) and a16wi4 (bf16 x int4) already run in production on gfx942: 64 rows in kimik3_a16w4_tuned_fmoe.csv and 26 rows in kimik3_i4_tuned_fmoe.csv use atomic stage 2.
• The PR description says only a16w16 was launched, and CI does not cover these paths on gfx942: op_tests/test_moe_2stage.py returns early for per_1x32 on non-gfx950, and op_tests/test_flydsl_moe_a16wfp4.py is pytest-only while aiter_test.sh runs it with plain python3, so none of its cases execute.

I'd also suggest not landing this commit as part of this PR:

• As the description notes, it is unrelated to the clamp; its only purpose is FlyDSL 0.3.2 compatibility. aiter pins flydsl==0.4.0 (requirements.txt, pyproject.toml, setup.py), and .github/instructions/aiter-ops-flydsl.instructions.md asks that kernels be judged against the pinned version and that fx.* be preferred over raw dialects wherever a wrapper exists. fx.atomic_add is that wrapper.
• The new comment calls this "the same pattern as mxfp4_gemm2", but mxfp4_gemm2.py uses fx.atomic_add. After this change, this would be the only llvm.AtomicRMWOp under aiter/ops/flydsl/.

Suggested fix: drop c470ea11b and keep the pre-PR code:

out_elems = global_typed_ptr(arg_out, T.bf16, align=4)
...
fx.atomic_add(
    fx.recast_iter(out_elems.type, out_elems + out_off),
    pk,
    syncscope=fx.rocdl.SyncScope.Agent,
)

If 0.3.2 support is really needed, please move it to a separate PR, widen the offset to Int64 before scaling, and validate the a16w4 / a16wi4 gfx942 paths there, ideally with a case whose output crosses 2 GiB (for example the M=131073, model_dim=8192 shape above).

Side note (pre-existing, not introduced here): mxfp4_gemm2.py also builds its byte offset with an Int32 multiply ((row_base_addr + s * 64) * 2), so it likely has the same 2 GiB limit.

[jatseng-ai]

@Bernard-Liu Dropped. gfx942 a16wmix stage 2 is main's fx.atomic_add again, on a typed bf16 pointer (out_elems + out_off), so the byte scale stays in the 64-bit GEP. The Int32 out_off * 2 offset is gone.

That commit was only there so the kernel would also compile on FlyDSL 0.3.2. aiter pins flydsl==0.4.0, so this PR does not carry a 0.3.2 fallback. gemm2.py is unchanged from main.

Rebased onto current main (7c362a7a0). No remaining conflict. op_tests/test_a16wmix_bf16_clamp.py on gfx942 (MI300X), head 6bf3c5302: 23 passed.

[jatseng-ai]

@Bernard-Liu The llvm.AtomicRMWOp commit is dropped and gemm2.py matches main again, in the reply above. Could you take another look and approve if that resolves it? CI on this head is still building the wheels, so the suite is not green yet. The 23 op tests passed on gfx942 at 6bf3c5302.

[jatseng-ai]

@coderfeli @xytpai You are on the requested-reviewer list for this fused_moe dispatch change (_can_reroute_bf16_to_a16wmix in get_2stage_cfgs). Bernard has been iterating on the FlyDSL/AOT/tuner side; the remaining ask is fused_moe owner sign-off if that is still needed to unblock merge.

Head 42e98fa14, rebased on main f1965cd26. Clamp tests 25 passed on gfx942. Kernel timing is in the comment above / PR body.

[Bernard-Liu]

This PR adds a new test file. Are all of these tests needed? If some of them are not necessary, could you drop them? Also, could the remaining tests go into an existing test script instead of a new one?

[jatseng-ai]

@Bernard-Liu Dropped the extra file. The remaining cases are in op_tests/test_moe_a16w16_default_dispatch.py (already the a16w16 dispatch test).

Kept: FlyDSL name/f16 gate, FORCE_REDUCE stays atomic, reroute sentinels + arch, get_2stage_cfgs selects a16wmix vs CK, AOT QuantType.No recipe, untuned t16/t32, host block_size_M mismatch, one small-shape bind per activation, Command A shape bind + asymmetric gate.

Dropped: silu_mul_batch source grep, sentinel×activation cartesian product, tokens=256 bind, and the small-shape asymmetric duplicate of the Command A case. test_moe_2stage.py is a CLI harness and _effective_swiglu_limit returns None for QuantType.No, so it would not catch this path.

python -m pytest op_tests/test_moe_a16w16_default_dispatch.py -k 'not ck_default and not qwen_mtp and not gfx950' on gfx942: 17 passed. Head f08806977.

[jatseng-ai]

@Bernard-Liu @coderfeli Aiter Test Gate is red because Standard Test shards 0 and 1 died at docker run with Docker Hub toomanyrequests (unauthenticated pull; fork PRs skip docker login). Tests never ran on those two shards.

Shards 2–7 passed, including op_tests/test_moe_a16w16_default_dispatch.py on shard 3. I cannot click "Re-run failed jobs" on ROCm/aiter (403, needs admin). Could someone with write rerun the failed jobs on https://github.com/ROCm/aiter/actions/runs/41413362449 so we keep the py3.12 wheel?

[jatseng-ai]

@xytpai Follow-up: still blocked on a Re-run failed jobs of https://github.com/ROCm/aiter/actions/runs/41413362449 (shards 0/1 Docker Hub toomanyrequests; tests never ran). Shards 2–7 including test_moe_a16w16_default_dispatch.py already passed. I get 403 from this fork account. If you have Actions write, that rerun would clear the gate without rebuilding the py3.12 wheel.

[jatseng-ai]

@coderfeli Afternoon ping: still need Re-run failed jobs on https://github.com/ROCm/aiter/actions/runs/41413362449 (shards 0/1 Docker Hub toomanyrequests; tests never ran). Fork PRs cannot rerun (403). Shards 2–7, including the clamp tests, already passed.

[jatseng-ai]

Rebased onto current main (now CONFLICTING after #6970 energy ranking in the tuner). Kept the 10-commit split.

Conflict was gemm_moe_tune.py dedup: took main's policy.sort_frame and kept our a16w16 key so bf16×bf16 kernels are not collapsed with CK QuantType.No.

[jatseng-ai]

Rebased head a2e3318a9 is fully green: Black/Ruff/deps, both wheels, OPUS, all eight Standard Test shards (including 0/1 that previously died on Docker Hub), and Aiter Test Gate. Still needs an approving review to merge.

@Bernard-Liu @coderfeli

[jatseng-ai]

@Bernard-Liu thanks for the approval on a2e3318a9.

Merge is still blocked on remaining requested reviews: @coderfeli @xytpai / TEAM_AITER. Required CI is green (Black/Ruff/deps + Aiter Test Gate, all eight Standard Test shards). This is a merge-commit PR (please do not squash — the dtype-gate vs SiLU-epilogue split is intentional).

[jatseng-ai]

Morning ping: @coderfeli @xytpai / TEAM_AITER — Bernard approved overnight on a2e3318a9, required CI is still green. Merge remains blocked on remaining requested reviews. Please do not squash (dtype-gate vs SiLU-epilogue split is intentional).