[afanfa]

Motivation

Add initial support for MI300A

Technical Details

This PR has three parts: 1) improve the device detection for MI300A; 2) add encoding for fp8 e4m3fnuz ; 3) Add MI300A (gfx942, cu_num=228) configs for GEMM and fused MoE

Test Plan

New tests for the newly added features have been added. The test suite has been run

Test Result

No regressions or failures on the newly added tests.

PR Self-Review

Reviewed revision

518476e

Behavior and intent

Adds MI300A (gfx942, 228 CU) support. Before this PR, MI300A was reported as "MI300" (same as MI300X), so SKU-specific dispatch could not tell them apart. Its 228-CU tuned rows did not exist, so every GEMM/MoE shape fell back to heuristic kernels. Tuning shape lists that name torch.float8_e4m3fn also aborted with "Unsupported dtype" (or were silently skipped by the bpreshuffle tuner) on gfx942, which only implements e4m3fnuz.

• chip_info: map PCI chip id 0x74A0 to "MI300A" in get_device_name().
• dtypes.normalize_fp8_dtype() + TunerCommon._normalize_fp8_columns(): resolve either e4m3 flavor in untuned shape lists to the arch-native encoding on ingest (name lookup, no eval), in the shared loader so every tuner family gets it.
• Tuned configs: MI300A rows for bf16, a8w8, a8w8 blockscale GEMM and fused MoE.
• Build targets: after merging main, this PR uses upstream's GFX_CU_NUMS_MAP (gfx942 → 304/228/80) unchanged. The earlier live-GPU CU refinement was removed, so wheels do not depend on the build host.

Validation and unverified scope

Hardware: AMD Instinct MI300A (gfx942, 228 CU), ROCm 7.14, torch 2.14.0+rocm7.14, Python 3.11.

• python -m pytest op_tests/tuning_tests/test_arch_resolution.py op_tests/test_build_targets_architecture.py tests/test_chip_info_xcds.py → 84 passed (this exercises the fn→fnuz direction, since the host is gfx942).
• python -m pytest op_tests/test_opus_a16w16_policy_parity.py → 19 passed.
• Level 0+1 suite from tuning-tests.yaml → all pass except 4 tests (test_tuner_infra GroupedFmoe x2, test_fast_tuning a16w16 policy, test_mp_tuner_logic timing) that fail identically on a clean upstream/main checkout in this environment (Triton version), so they are unrelated.

Unverified: the new tuned rows were not re-benchmarked at this revision; the gfx950 (fnuz→fn) direction is covered only by the CI tuning job on MI35X. The Aiter Test wheel build/GPU tests have not completed (Docker Hub 429 on the fork PR's anonymous image pull).

PR-specific question

The only CI change adds op_tests.tuning_tests.test_arch_resolution to the existing Level 0+1 unittest command in .github/workflows/tuning-tests.yaml. It adds no new triggers, permissions, secrets or steps. Fork PR code already runs in that same unprivileged container job, and no PR text is interpolated into a command.

Changed op tests

• op_tests/tuning_tests/test_arch_resolution.py (new): pins MI300A SKU naming by PCI id, the version-gated PciChipId ordinal, legacy cu_num→gfx backfill, that GPU_ARCHS builds include (gfx942, 228) alongside 304/80, fp8 e4m3 resolution, that every shipped shape list (incl. bpreshuffle) is tunable on the running arch, and that the cu_num=228 tuned rows resolve to gfx942. No existing test covers MI300A identity or fp8 flavor resolution of shape lists.
• op_tests/test_opus_a16w16_policy_parity.py: the pre-PR codegen reference map now keys on cu_num. Without it, the MI300A (228) and MI308X (80) rows collide with MI300X (304) rows for the same shape and the parity assertion compares the wrong kernel ids. The test is otherwise unchanged.
• op_tests/tuning_tests/README.md: documentation only; lists the new module and adds it to the suggested Level 1 command.

[github-actions]

🏷️ CI Guide

Runs automatically on every PR:

• ✅ Pre-checks (submodule verification, code formatting)
• ✅ Aiter op tests (gfx942 + gfx950)
• ✅ Triton tests on MI35X (only when aiter/ops/triton/** or related paths are changed)

Extended tests (opt-in via labels):

Label	Tests
ci:gfx1250-ffm-triton	Run the five-shard gfx1250 FFM Triton test suite
ci:triton-300x	Run an additional Triton test job on MI300X in PRs; main branch always runs both MI35X and MI300X
multigpu	Aiter multi-GPU tests on the 8-GPU runner
ci:sglang	SGLang integration tests: DeepSeek-R1-MXFP4 accuracy, Qwen 3.5 accuracy
ci:atom	ATOM benchmark: DeepSeek-R1-0528, GPT-OSS-120B
ci:atom_full	ATOM accuracy suite for PR and main models from ATOM models_accuracy.json
ci:vllm	vLLM benchmark: GPT-OSS-120B, DeepSeek-R1-0528, Kimi-K2.5
ci:all	All standard extended tests (excludes ci:atom_full)

Only add ci:atom_full for FlyDSL or Triton upgrades.
Add labels via the sidebar or gh pr edit 5186 --add-label <label>

PR title tags & labels:
Component tags ([Triton/Gluon], [HIP], [CK], [ASM], ...) are added to the PR title and as PR labels automatically from the changed files and re-synced on every push — change-type tags like [fix]/[Perf], op tags like [MLA], and human labels (ci:*) are left untouched. Add the no-auto-title label to opt this PR out.

[Copilot]

Pull request overview

Adds initial MI300A (gfx942, 228 CU) enablement by improving SKU/CU resolution, normalizing FP8 e4m3 flavor handling (fn vs fnuz), and introducing MI300A-tuned GEMM/MoE configuration rows, with accompanying unit tests wired into CI.

Changes:

• Refine device/SKU detection and build-target CU selection for gfx942 variants (MI300X/MI300A/MI308X), including a safer PCI chip-id probe path.
• Add FP8 e4m3 flavor normalization and apply it during tuning/shape-list ingestion so shipped shape lists remain tunable on gfx942/gfx950.
• Add MI300A tuned configs (GEMM + fused MoE) and add Level-1 unit tests + CI hooks validating arch/dtype resolution invariants.

Reviewed changes

Copilot reviewed 13 out of 13 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
op_tests/tuning_tests/test_arch_resolution.py	New unit tests for gfx942 SKU resolution, PCI chip-id probing behavior, CU-based arch backfill, build-target CU refinement, and FP8 e4m3 normalization.
op_tests/tuning_tests/README.md	Documents the new Level-1 test module and adds it to the suggested unittest invocation list.
docs/autotuning_pipeline.md	Adds the new test module to the documented Level-1 pipeline list.
csrc/ck_gemm_moe_2stages_codegen/gemm_moe_tune.py	Normalizes fp8 e4m3 dtype strings on untuned-shape ingestion; normalizes q dtypes when reading/tuning configs.
csrc/ck_gemm_a8w8/gemm_a8w8_tune.py	Normalizes fp8 e4m3 dtype strings for q_dtype_w on untuned-shape ingestion.
aiter/utility/dtypes.py	Introduces normalize_fp8_dtype() to map e4m3 flavors to the arch-native fp8 encoding.
aiter/jit/utils/chip_info.py	Adds MI300A chip-id mapping, updates legacy CU→gfx backfill, refines get_build_targets() CU selection under GPU_ARCHS, and makes PCI chip-id probing degrade to None.
aiter/jit/utils/build_targets.py	Comment updates for gfx942 default CU count context.
aiter/configs/model_configs/mi300a_tuned_fmoe.csv	Adds MI300A (gfx942, 228 CU) tuned fused-MoE rows.
aiter/configs/bf16_tuned_gemm.csv	Adds MI300A (gfx942, 228 CU) tuned bf16 GEMM rows.
aiter/configs/a8w8_tuned_gemm.csv	Adds MI300A (gfx942, 228 CU) tuned a8w8 GEMM rows (incl. fp8 fnuz).
aiter/configs/a8w8_blockscale_tuned_gemm.csv	Adds MI300A (gfx942, 228 CU) tuned a8w8 blockscale GEMM rows.
.github/workflows/tuning-tests.yaml	Runs the new Level-1 arch-resolution test in CI.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Copilot]

Pull request overview

Copilot reviewed 13 out of 13 changed files in this pull request and generated 1 comment.

Suppressed comments (1)

aiter/jit/utils/chip_info.py:463

• _get_pci_chip_id() only catches OSError/ValueError/UnicodeDecodeError. If libamdhip64.so loads but does not export hipDeviceGetPCIBusId (or the symbol name changes), ctypes will raise AttributeError and this will currently propagate instead of degrading to None as intended.

    try:
        libhip = ctypes.CDLL("libamdhip64.so")
        buf = ctypes.create_string_buffer(64)
        if libhip.hipDeviceGetPCIBusId(buf, len(buf), device_id) != 0:
            return None
        bdf = buf.value.decode()
        with open(f"/sys/bus/pci/devices/{bdf}/device") as f:
            return int(f.read().strip(), 16)
    except (OSError, ValueError, UnicodeDecodeError):
        return None

[Copilot]

Pull request overview

Copilot reviewed 13 out of 13 changed files in this pull request and generated no new comments.

Suppressed comments (3)

csrc/ck_gemm_moe_2stages_codegen/gemm_moe_tune.py:474

• This normalization uses eval(s) on CSV cell contents. If the untuned CSV path is user-supplied (e.g., via env/arg), this is code execution; it also misses rows with leading/trailing whitespace because the startswith check doesn’t strip. Prefer parsing the known dtype strings and mapping via getattr(torch, ...) on the torch. suffix (no eval) and strip before checking.

            untunedf[col] = untunedf[col].map(
                lambda s: (
                    str(dtypes.normalize_fp8_dtype(eval(s)))
                    if isinstance(s, str) and s.startswith("torch.float8_e4m3")
                    else s

csrc/ck_gemm_a8w8/gemm_a8w8_tune.py:137

• This normalization uses eval(s) on CSV content to obtain a torch dtype. Even if typical shape lists are trusted, untuned_gemm_file can be overridden and eval turns the tuner into a code-execution sink. Parse the known torch.float8_e4m3* strings without eval (e.g., getattr(torch, name[len('torch.'):])) and then normalize.

            untunedf["q_dtype_w"] = untunedf["q_dtype_w"].map(
                lambda s: (
                    str(dtypes.normalize_fp8_dtype(eval(s)))
                    if isinstance(s, str) and s.strip().startswith("torch.float8_e4m3")
                    else s

aiter/jit/utils/chip_info.py:462

• _get_pci_chip_id() intends to return None when HIP/sysfs probing is unavailable, but a missing hipDeviceGetPCIBusId symbol (or other attribute lookup failure) would currently raise AttributeError and break get_device_name() dispatch on gfx942. Treat AttributeError the same as other probe failures and return None.

    except (OSError, ValueError, UnicodeDecodeError):

[Copilot]

🟡 Changes recommended

The new unit test uses eval() on CSV-derived strings and fully materializes large CSVs into memory, creating avoidable security and performance risks in CI.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details

Suppressed comments (1)

Previously missed (1) — in code that hasn't changed since the last review.

op_tests/tuning_tests/test_arch_resolution.py:346

• This test loads every tuned CSV fully into memory (rows = list(...)) just to check for cu_num=228 rows, which can make Level-1 tests slow and memory-heavy as config CSVs grow. You can stream the DictReader and use reader.fieldnames to detect the schema without materializing the whole file.

• Files reviewed: 13/13 changed files
• Comments generated: 1
• Review effort level: Lite

[Copilot]

🔵 Needs a closer look

Hardware-SKU-specific dispatch/build-target logic plus large tuned-config expansions warrant final human verification on representative MI300A environments.

Review details

Suppressed comments (2)

Previously missed (2) — in code that hasn't changed since the last review.

aiter/jit/utils/build_targets.py:41

• The comment for the gfx942 entry implies MI300A/MI308X builds always require an explicit CU_NUM override, but chip_info.get_build_targets() can now refine the CU count automatically when a live GPU is present. Clarifying that CU_NUM is required specifically for env-only/no-GPU target resolution would avoid confusion for users reading this table.
  op_tests/tuning_tests/test_arch_resolution.py:345
• In TestTunedConfigArchConsistency, rows = list(csv.DictReader(f)) loads each tuned CSV fully into memory even though the test only needs to scan for cu_num == 228. This can be unnecessarily slow/heavy given large tuned CSVs; streaming the DictReader avoids the allocation and keeps the same behavior.

• Files reviewed: 13/13 changed files
• Comments generated: 0 new
• Review effort level: Lite

[afanfa]

ping

[github-actions]

No activity for 15 days, so this is now labelled stale. A push or a comment clears it; keep-open exempts it. Nothing is closed automatically.

[yzhou103]

Hi please resolve the conflicts

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota.

[yzhou103]

@valar-cop review

[valar-cop]

a few small things..

need a run on mi300a (gfx942, 228 cu) to cover the new tuned rows and the fnuz branch?

ci green.

[zoe-cop]

Follow-up review of head 44ef14392998a314433adcc93a6532a4dda5467b: two non-blocking fixes on the new content, plus a separate pre-existing CSV update issue worth tracking.

1. Reconcile the numeric IDs in six new CK-Tile rows.

In a8w8_blockscale_tuned_gemm.csv, these gfx942/228 rows save a kernelId that differs from the current ID of their kernelName:

M (N=7168, K=256)	Saved ID	Current ID of the saved name
1024, 1536, 4096	2	20
8192, 20480	3	21
2048	4	22

Please regenerate/reconcile these IDs against the current registry. ID-based manual reproduction would otherwise select a different candidate. The saved names exist, and codegen/runtime dispatch resolves them by name; this is stale metadata, not a demonstrated wrong-kernel runtime failure.

2. Make the FP8 regression tests assert both native encodings independently of the test host.

TestNormalizeFp8Dtype and the shipped-list check use the actual host's dtypes.fp8. The tuning-tests job uses gfx950/MI35x, so its conversion check exercises fnuz -> fn; it does not explicitly guard the fn -> fnuz direction needed by MI300A.

Please add subcases that patch the native alias to each of torch.float8_e4m3fn and torch.float8_e4m3fnuz, and assert both input flavors resolve correctly in each subcase. Keep the real-host integration check as well.

3. Separate follow-up: decimal-formatted numeric CSV keys can silently prevent an approved update (already present in base).

_merge_compare_filtered_results converts identity columns to strings on both sides. Numerically equal values such as 228/228.0 and 16/16.0 then fail to match.

Reproduction at the CSV-method level:

• Existing tuned CSV stores integral identity fields as decimals; the untuned shape/comparison plan has integer-valued keys.
• The candidate writer saves a new kernel. Synthetic pre/post results of 10/5 us make _build_compare_update_plan set update=True.
• The final merge fails to select that candidate by its stringified key and retains the old kernel.

This was reproduced in head and base with native FP8 spelling, so it is not caused by the new FP8 resolver. The reproduction executes the source CSV methods with mocked device/dtype attributes and synthetic benchmark results; those times are not GPU measurements.

Please normalize integral key fields consistently across old/candidate/comparison data, validating integrality rather than truncating, and add a round-trip regression that checks the final saved kernel after write/reload. This can be a separate fix; _resolve_fp8_name does not address numeric CSV formatting.

Scoped GPU validation: on MI308X (gfx942, 80 CU), the new architecture/FP8 test file passed 25 tests with no skips. All seven CK-Tile shapes above/in the new CSV, each with two random seeds, plus M=127/129/1023/1025 boundary cases passed on both head and base (18 cases per side). Outputs were initialized to NaN and compared against a PyTorch FP32 dequantization/matmul reference converted to BF16; all elements were within atol=rtol=0.01. Head's actual shared CSV loader also converted fn -> fnuz, followed by successful GPU quantization and GEMM.

These kernel checks used a diagnostic subset build of the four saved names from unmodified kernel sources and direct native calls. They do not establish full default-registry build coverage, MI300A 228-CU automatic dispatch, performance, or a complete autotuner --update_improved run.

[aiter-bot]

READY FOR HUMAN REVIEW

PR Self-Review

Revision 518476e854a6 · mode shadow

Author self-review accepted. Readiness report:

Author answers

• PASS — current-SHA answers are present and structurally specific

Show the author's self-review answers

Behavior and intent

Adds MI300A (gfx942, 228 CU) support. Before this PR, MI300A was reported as "MI300" (same as MI300X), so SKU-specific dispatch could not tell them apart. Its 228-CU tuned rows did not exist, so every GEMM/MoE shape fell back to heuristic kernels. Tuning shape lists that name `torch.float8_e4m3fn` also aborted with "Unsupported dtype" (or were silently skipped by the bpreshuffle tuner) on gfx942, which only implements e4m3fnuz.
- `chip_info`: map PCI chip id 0x74A0 to "MI300A" in `get_device_name()`.
- `dtypes.normalize_fp8_dtype()` + `TunerCommon._normalize_fp8_columns()`: resolve either e4m3 flavor in untuned shape lists to the arch-native encoding on ingest (name lookup, no `eval`), in the shared loader so every tuner family gets it.
- Tuned configs: MI300A rows for bf16, a8w8, a8w8 blockscale GEMM and fused MoE.
- Build targets: after merging main, this PR uses upstream's `GFX_CU_NUMS_MAP` (gfx942 → 304/228/80) unchanged. The earlier live-GPU CU refinement was removed, so wheels do not depend on the build host.

Validation and unverified scope

Hardware: AMD Instinct MI300A (gfx942, 228 CU), ROCm 7.14, torch 2.14.0+rocm7.14, Python 3.11.
- `python -m pytest op_tests/tuning_tests/test_arch_resolution.py op_tests/test_build_targets_architecture.py tests/test_chip_info_xcds.py` → 84 passed (this exercises the fn→fnuz direction, since the host is gfx942).
- `python -m pytest op_tests/test_opus_a16w16_policy_parity.py` → 19 passed.
- Level 0+1 suite from `tuning-tests.yaml` → all pass except 4 tests (`test_tuner_infra` GroupedFmoe x2, `test_fast_tuning` a16w16 policy, `test_mp_tuner_logic` timing) that fail identically on a clean `upstream/main` checkout in this environment (Triton version), so they are unrelated.

Unverified: the new tuned rows were not re-benchmarked at this revision; the gfx950 (fnuz→fn) direction is covered only by the CI tuning job on MI35X. The Aiter Test wheel build/GPU tests have not completed (Docker Hub 429 on the fork PR's anonymous image pull).

PR-specific question

The only CI change adds `op_tests.tuning_tests.test_arch_resolution` to the existing Level 0+1 `unittest` command in `.github/workflows/tuning-tests.yaml`. It adds no new triggers, permissions, secrets or steps. Fork PR code already runs in that same unprivileged container job, and no PR text is interpolated into a command.

Changed op tests

- `op_tests/tuning_tests/test_arch_resolution.py` (new): pins MI300A SKU naming by PCI id, the version-gated PciChipId ordinal, legacy cu_num→gfx backfill, that `GPU_ARCHS` builds include (gfx942, 228) alongside 304/80, fp8 e4m3 resolution, that every shipped shape list (incl. bpreshuffle) is tunable on the running arch, and that the cu_num=228 tuned rows resolve to gfx942. No existing test covers MI300A identity or fp8 flavor resolution of shape lists.
- `op_tests/test_opus_a16w16_policy_parity.py`: the pre-PR codegen reference map now keys on cu_num. Without it, the MI300A (228) and MI308X (80) rows collide with MI300X (304) rows for the same shape and the parity assertion compares the wrong kernel ids. The test is otherwise unchanged.
- `op_tests/tuning_tests/README.md`: documentation only; lists the new module and adds it to the suggested Level 1 command.

PR state

• PASS — pull request is marked ready for review

Copilot findings

• PASS — no active Copilot finding lacks a substantive author reply
• ADVISORY — Copilot did not review the current head SHA

Code and op-test hygiene

• PASS — no committed artifact or conflict marker detected

Required CI

• PASS — Black: passed
• PASS — Ruff: passed
• PASS — Repository dependency: passed
• PASS — Aiter Test Gate: passed

_{Shadow mode reports what would block but keeps the status successful. Set AITER_PR_SELF_REVIEW_MODE=enforce only after reviewing real false positives.}

[afanfa]

@zoe-cop Thanks. Items 1 and 2 are fixed in 67aa535. The six blockscale rows now carry the registry IDs for their kernel names (2→20, 3→21, 4→22). New subcases patch the native fp8 alias to each e4m3 flavor and assert both inputs resolve, alongside the real-host checks. Item 3 is in base, so I'll leave it to a separate fix.

[zoe-cop]

The six rows now store 20/20/22/20/21/21, matching the BlockPerCu 2/3/4 expansion of kernel 0 in gemm_a8w8_blockscale_cktile_instance.py, so each saved ID and kernelName agree.

The new subcases patch dtypes.fp8 to each e4m3 flavor, and normalize_fp8_dtype reads that alias at call time, so fn->fnuz is now covered on any host. Item 3 is pre-existing; deferring it is fine.

— zoe-cop · automated reply.

[afanfa]

CI Everything is green except MI35X shard 2: op_tests/test_flydsl_hgemm.py::test_hgemm_memory, case M=19 N=120 K=1088, split_k=9. Its zero-tolerance "changed-input graph" check sees 1 of 2,280 elements off by one bf16 ulp (−3.8438 vs −3.8594). The test, the check, and the case were all added in #7546 and came in with the latest merge from main. This PR doesn't touch FlyDSL or that test, and its kernel-facing changes are gfx942 tuned rows the test doesn't read. It failed on the original run and the re-run, both on linux-aiter-mi35x-1-mtck5 runners, while the same test passes on main (1eab5ae, fvzc6 runner). I can't reproduce it locally (MI300A only), so I can't tell whether it's split-K non-determinism on those runners or a real numerical difference. Could you re-run or override the gate for that job?

[yzhou103]

LGTM