Skip to content

[Triton/Gluon] [Conv2D] Add Conv2D configuration files for gfx1101 (RDNA3) and gfx1150 (RDNA3.5) - #5246

Merged
brunomazzottiamd merged 4 commits into
ROCm:mainfrom
Ragua1:users/Ragua1/conv-configs-gfx1101-gfx1150
Sep 8, 2026
Merged

brunomazzottiamd merged 4 commits into
ROCm:mainfrom
Ragua1:users/Ragua1/conv-configs-gfx1101-gfx1150

Conversation

@Ragua1

@Ragua1 Ragua1 commented Sep 3, 2026 •

Copy link
Copy Markdown
Contributor

Motivation

Conv2D raises FileNotFoundError on gfx1101: it ships no tables and eight families are mandatory. This PR tunes and adds them, following the invitation in #4869 (comment).

gfx1150 is included as well because that part is also available to me for measurement, and it is in the same position — no tables on main. main has them for gfx1100, gfx1151, gfx1200, gfx1201, gfx1250, gfx942 and gfx950.

Data plus one test-helper line; no kernel, routing or loader code changed.

Eight families per architecture, not nine. CONV-3X3-NCHW is the optional ninth and is deliberately not here — see below.

Technical Details

gfx1101 (RDNA3, RX 7800 XT) gfx1150 (RDNA3.5, Radeon 890M)
entries 182 181
tuned on the card 16 97
inherited from the donor 166 84
donor gfx1100 gfx1151
torch 2.15.0a0+rocm10.1.0a20260822 2.11.0+rocm7.15.0a20260728
HIP runtime 7.16.26332 7.15.26290
measurements 2313 1201

Both version rows are given because the two move independently: the torch wheel tag went 7.15 → 10.1 while the HIP runtime went 7.15 → 7.16, so quoting either alone is ambiguous.

entries counts every section — each shapes[key], shapes_<variant>[key], M_LEQ_<n> and any is one. gfx1101 is 149 + 25 + 8, gfx1150 is 152 + 7 + 14 + 8.

Every file's structure is identical to its donor's — same sections, same key set per section, same route_exact_only. That makes the next two claims checkable rather than rhetorical:

  • Every inherited entry is byte-identical to the donor's value at the same key (166 and 84), frozen at the revision named in the campaign record, not referenced.
  • Routing is unchanged, and here that is a property of the file set rather than of any file: none of the eight families is read by a routing predicate. The only one that is, CONV-3X3-NCHW, is not in this PR. These files change which config the chosen kernel runs with, never which kernel is chosen.

No architecture name appears inside the sixteen files.

Per file, tuned / inherited:

family gfx1101 gfx1150
conv_1x1 2 / 22 11 / 22
conv_3x3_cblocked 2 / 32 20 / 9
conv_3x3_nhwc 0 / 33 16 / 11
conv_general 0 / 2 0 / 2
conv_prepack 2 / 17 12 / 14
conv_wino_f4x3_input 5 / 17 14 / 6
conv_wino_f4x3_output 3 / 19 12 / 8
conv_wino_f4x3_gemm 2 / 24 12 / 12

Shape keys are the donor's choice in all eight families: the tuner iterates the own-arch table, and that table was seeded from the donor.

$ git diff --shortstat main
 18 files changed, 2656 insertions(+), 2 deletions(-)

$ git diff --numstat main | grep -v DEFAULT.json
1       1       aiter/ops/triton/configs/CLAUDE.md
9       1       op_tests/triton_tests/conv/_helpers.py

Four commits: the config files, the SUPPORTED_ARCHS line, the layout table in configs/CLAUDE.md, and the removal described next. Without the SUPPORTED_ARCHS line the tables are never exercised by the conv tests.

What is excluded, and why

CONV-3X3-NCHW is not in this PR on either architecture, although both campaigns tuned it — 26 shapes each.

It is the optional ninth family: has_conv_config() gates it, and gfx1200, gfx1250, gfx942 and gfx950 ship none. Adding it is a routing change on these two cards, which the other eight families are not. _nchw_direct_is_profitable (conv2d.py:82-84) returns False when no table is present, so today every 3×3 shape on gfx1101 and gfx1150 takes CBLOCKED_NCHW; shipping the table with route_exact_only moves its keyed shapes onto DIRECT_NCHW_3X3.

Neither campaign measured which of those two is faster, having timed each family through its own public entry point rather than through the router. The one direct-against-cblocked probe I have run — on gfx1150, in one process, with the NCHWc pack inside the cblocked arm where the router pays it — says the answer is not obvious. Ratio is cblocked_with_pack / direct:

shape run A run B
C=512, H=W=256, K=512 1.105 1.074 cblocked ahead, reproduced
C=512, H=W=128, K=512 0.982 1.016 flips sign
C=256, H=W=512, K=256 0.985 1.003 within noise

A follow-up PR will carry the table with a per-key measurement of both routes, and will present itself as the routing change it is.

If you would rather have it here, say so and I will run that probe against this PR instead of a later one — I have both cards. Splitting it out is my judgement about what is reviewable in one piece, not a constraint.

any is tuned, and it is the weakest claim here

any is tier 4 — the fallback for every shape the walk does not match, which is most real shapes. It is tuned in 5 of 8 families on gfx1101 and 7 of 8 on gfx1150, taking the majority value among that family's confirmed entries.

On gfx1150 those votes run over 10–19 entries. On gfx1101 four of the five rest on a single entry, because that card's acceptance rule confirms few (below). If you would rather see any reverted to the donor below some vote threshold, that is a one-line change per file and I will make it.

Test Plan

Correctness is checked on every measurement before it is timed: rel_l2 < 1e-2 against torch.nn.functional.conv2d, the bar op_tests/triton_tests/conv uses, and a reshape/permute reference for the pack. A candidate producing wrong data cannot become an entry. Each family is driven through its own public entry point, so a second family's cost is not folded into every number.

Two stages per family: a coarse sweep of the candidate space on five M-spread shapes, then the four finalists plus the donor value on every shape in the family's list.

The shipped files are verified against the donor rather than trusted — section set, per-section key set, route_exact_only, and a byte comparison of every inherited entry at its own key. That check needs no GPU and is what caught the stray CONV-3X3-NCHW keys above.

⚠️ op_tests/triton_tests/conv could not be run on either host. AITER_TRITON_ONLY is unconditional on win32 and that branch of aiter/__init__.py does not bind aiter.dtypes, which the test helpers import. Independent of this change; I preferred not to put a workaround into the verification I am reporting. The repository's CI has no RDNA machine, so extending SUPPORTED_ARCHS does not by itself get these tables tested anywhere but on the two hosts above.

Protocol detail, and two findings that reduced what is claimed

The two sets are not comparable with each other, and it is protocol rather than hardware. gfx1150 timed one call per iteration, median of 20 after 5 warmup, so its numbers include per-call launch and sync overhead. gfx1101 times reps back-to-back calls in one body and divides. On torch.nn.functional.conv2d, C=64, H=W=56, K=64, 3×3, five repeats each:

reps ms
1 0.115
10 0.077
50 0.0655

A 1.76× spread on one unchanged workload. Neither set's absolute figures should be read against the other's; within each set, every comparison is against the donor measured in the same run.

Why the protocol changed: a measurement floor. 51 of the first 1339 gfx1101 records reported ≤10 µs and 7 of 73 shape winners were physically impossible — conv_3x3_nchw at C=64, H=W=56, K=64 reported 0.000295 ms, which is 784 TFLOP/s, over ten times this part's fp16 peak, on a shape torch does in 0.100–0.103 ms. The cheapest possible GPU op through the same timer returned −0.000945 to +0.004385 ms across five repeats: it cannot resolve that band or its sign. Amortising over reps removes it at source; afterwards the smallest record is 0.0544 ms. That campaign was discarded and re-run.

Why few gfx1101 entries are tuned: a noise floor. Most entries ship the donor's own value, so validating a shipped entry against the donor measures an identical configuration — the spread is the card's own noise. 405 such controls over three repeats:

all > 0.5 ms 0.1–0.5 ms < 0.1 ms
p50 1.016×
p90 1.095× 1.049× 1.235× 1.118×
p95 1.146× outliers past 6×

A 1.05× bar therefore sits near p70 of the noise, and re-measuring the 40 initially accepted entries once returned 9 of them below 1.0.

gfx1150's load figure is bimodal — 16 readings of one fixed workload on an idle machine gave 13 near 27.9 ms and 3 near 57 ms — so its values were re-checked for rank preservation on the merged tree rather than compared as absolute times across revisions. Within-run ratios carry; cross-run totals do not.

Machine state. Every record carries a quiet-machine check and the run refuses at 1.5× off it. The gfx1101 campaign's records span 0.90×–1.05× of its reference, so machine state is not a candidate explanation for any of its numbers.

Test Result

gfx1101's acceptance rule is deliberately stricter than tuning normally is here. An entry keeps a tuned value only if all three repeats on the tuning stack exceed 1.0, the median is ≥ 1.05×, and it does not regress on either of two older stacks (torch 2.11 with HIP 7.16.26332, and torch 2.11 with HIP 7.15.26290). Failures revert to the donor and count as inherited. That third condition is the main reason the count is 16 rather than 40.

It earns its place on this card: conv here regressed 2.4× (0.401 → 0.981 ms) on the torch 2.10 → 2.11 move alone, and across the 40 candidates the gain moved between stacks by p50 1.038×, p90 1.482×, max 2.875×. Of the 17 rejections, 15 cite that condition and 2 fail on it alone — those two passed every single-stack test.

gfx1150 was tuned on its own stack in the usual way, which is also how every table already in the repository was produced.

gfx1101, the 11 shape entries that ship:

median gain over the donor 1.218×
range 1.052× – 1.339×

Cross-stack agreement on the largest of them, conv_1x1 at C=128, H=W=256, K=256:

stack gain
torch 2.15 / HIP 7.16 (tuning) 1.289, 1.339, 1.499
torch 2.11 / HIP 7.16 1.299
torch 2.11 / HIP 7.15 1.306

On gfx1101 the donor table is already unbeaten on 163 of its 174 shape-and-bucket entries. What this PR gives that card is the tables it does not have at all today, plus 11 improvements that survived the rule above — 4 in conv_wino_f4x3_input, 3 in conv_wino_f4x3_output, and one each in conv_1x1, conv_3x3_cblocked, conv_prepack and conv_wino_f4x3_gemm.

What is not claimed

  • conv_general is not tuned on either arch. One shape, zero tuned, any inherited. On gfx1101 it improved 4.4 %, below the bar — "measured and insufficient", not "unmeasured".
  • conv_3x3_nhwc on gfx1101 is entirely inherited, all 33 entries: three were accepted and all three reverted by the confirmation pass.
  • The three CONV-WINO-F4X3-* families are unreachable through the router on both parts (Wave32; conv2d.py:110-135), and only 3 of their 19 shapes even pass the C >= 512 and K_out >= 512 and T >= 98 precondition. They must exist regardless. They were tuned rather than grafted because on gfx1150 the inherited transforms measured 2–3× off, which was concealing how fast the disabled path is; on gfx1101 the equivalent is at most 1.40×. No claim that enabling the path would help.
  • The donor choice is not claimed better than any alternative. gfx1100 was chosen for gfx1101 because it is the only Wave32 RDNA3 arch with a shipped conv table, so its shape list comes from a part of that class. On this card gfx1100's shipped table measured indistinguishable from an RDNA4 table at 0.998×, so numbering proximity is not the lever it looks like. gfx1151 or gfx1201 as donors was not measured.
  • No end-to-end or model-level claim. Everything above is kernel-level, one call per shape.

Both cards remain available to me and I expect to send further tuning work for them, so nothing here needs to be the last word on either.

Separate defect, filed separately

conv2d_nchw_cblocked fails under torch.inference_mode() on main: _pack_cache_key() (conv/_prepack.py:71-83) reads w._version, which raises RuntimeError: Inference tensors do not track version counter. Every 3×3 route that packs a filter is unavailable to a framework loading weights inside inference_mode(). Independent of any config file — it reproduces with gfx1100's own shipped tables — so it is filed on its own.

Nine tables per architecture in the nested layout, tuned on hardware. Both
architectures currently have none, so every Conv2D family is a hard
FileNotFoundError from load_config_json(required=True) rather than a fallback.

gfx1101 (RDNA3, RX 7800 XT): 190 entries, 20 tuned on the card, 170 inherited
byte-identical from gfx1100.
gfx1150 (RDNA3.5, Radeon 890M): 186 entries, 100 tuned, 86 inherited from
gfx1151.

Entry counts cover every section - shapes, variant pins, M_LEQ_* buckets and
any - not just shapes.

The structure of every file matches its donor exactly: same sections, same key
set per section, same route_exact_only. Routing behaviour is therefore unchanged
and only the values differ. In particular CONV-3X3-NCHW ships only the shape
keys the donor already has, because under route_exact_only the key set is what
_nchw_direct_is_profitable uses to choose between DIRECT_NCHW_3X3 and
CBLOCKED_NCHW, and that comparison was not part of these measurements.

Every candidate was checked for correctness before being timed - rel_l2 < 1e-2
against torch.nn.functional.conv2d, the bar op_tests/triton_tests/conv uses -
and each family was driven through its own public entry point rather than
through the router.

Follows the invitation in
ROCm#4869 (comment)

Signed-off-by: Martin Domanský <ragua@email.cz>
…ARCHS

Without this the two new tables are never exercised by op_tests/triton_tests/conv.

Note that the repository's CI has no RDNA machine, so this does not by itself
cause the tables to be tested anywhere - it only makes the tests runnable on
hardware that has them.

Signed-off-by: Martin Domanský <ragua@email.cz>
configs/CLAUDE.md documents which architectures have a config tree. ROCm#5085
rewrote that document for the nested layout, so it is maintained and a new arch
belongs in it.

Separate commit from the data on purpose: it is your document, and a doc change
should be reviewable on its own rather than folded into eighteen JSON files.

Signed-off-by: Martin Domanský <ragua@email.cz>
@Ragua1
Ragua1 requested a review from a team September 3, 2026 10:46
@github-actions github-actions Bot changed the title Users/ragua1/conv configs gfx1101 gfx1150 [Triton/Gluon] Users/ragua1/conv configs gfx1101 gfx1150 Sep 3, 2026
@github-actions

github-actions Bot commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

🏷️ CI Guide

Runs automatically on every PR:

  • ✅ Pre-checks (submodule verification, code formatting)
  • ✅ Aiter op tests (gfx942 + gfx950)
  • ✅ Triton tests on MI35X (only when aiter/ops/triton/** or related paths are changed)

Extended tests (opt-in via labels):

Label Tests
ci:gfx1250-ffm-triton Run the five-shard gfx1250 FFM Triton test suite
ci:triton-300x Run an additional Triton test job on MI300X in PRs; main branch always runs both MI35X and MI300X
multigpu Aiter multi-GPU tests on the 8-GPU runner
ci:sglang SGLang integration tests: DeepSeek-R1-MXFP4 accuracy, Qwen 3.5 accuracy
ci:atom ATOM benchmark: DeepSeek-R1-0528, GPT-OSS-120B
ci:atom_full ATOM accuracy suite for PR and main models from ATOM models_accuracy.json
ci:vllm vLLM benchmark: GPT-OSS-120B, DeepSeek-R1-0528, Kimi-K2.5
ci:all All standard extended tests (excludes ci:atom_full)

Only add ci:atom_full for FlyDSL or Triton upgrades.
Add labels via the sidebar or gh pr edit 5246 --add-label <label>

PR title tags & labels:
Component tags ([Triton/Gluon], [HIP], [CK], [ASM], ...) are added to the PR title and as PR labels automatically from the changed files and re-synced on every push — change-type tags like [fix]/[Perf], op tags like [MLA], and human labels (ci:*) are left untouched. Add the no-auto-title label to opt this PR out.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Direct-NCHW routing lacks required comparative benchmarks, and CI route coverage for both architectures is missing.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

Adds Triton Conv2D configuration coverage for gfx1101 and gfx1150, preventing missing-table failures.

Changes:

  • Adds nine configuration families for each architecture.
  • Enables both architectures in Conv2D test helpers.
  • Documents the expanded configuration layout.
File summaries
File Review
op_tests/triton_tests/conv/_helpers.py Moderate (2 votes): Existing CI does not exercise gfx1101/gfx1150 exact routing. Add pinned and unpinned route tests for both architectures.
aiter/ops/triton/configs/gfx1150/triton/conv/conv_wino_f4x3_output/DEFAULT.json Adds Winograd output tuning.
aiter/ops/triton/configs/gfx1150/triton/conv/conv_wino_f4x3_input/DEFAULT.json Adds Winograd input tuning.
aiter/ops/triton/configs/gfx1150/triton/conv/conv_wino_f4x3_gemm/DEFAULT.json Adds Winograd GEMM tuning.
aiter/ops/triton/configs/gfx1150/triton/conv/conv_prepack/DEFAULT.json Adds prepack tuning.
aiter/ops/triton/configs/gfx1150/triton/conv/conv_general/DEFAULT.json Adds general-convolution fallback.
aiter/ops/triton/configs/gfx1150/triton/conv/conv_3x3_nhwc/DEFAULT.json Adds NHWC tuning.
aiter/ops/triton/configs/gfx1150/triton/conv/conv_3x3_nchw/DEFAULT.json Moderate (1 vote): The table changes default dispatch from cblocked to direct without target-card route comparisons. Benchmark all four pins including packing cost, or remove the unproven pins/table.
aiter/ops/triton/configs/gfx1150/triton/conv/conv_3x3_cblocked/DEFAULT.json Adds cblocked tuning.
aiter/ops/triton/configs/gfx1150/triton/conv/conv_1x1/DEFAULT.json Adds 1×1 tuning.
aiter/ops/triton/configs/gfx1101/triton/conv/conv_wino_f4x3_output/DEFAULT.json Adds Winograd output tuning.
aiter/ops/triton/configs/gfx1101/triton/conv/conv_wino_f4x3_input/DEFAULT.json Adds Winograd input tuning.
aiter/ops/triton/configs/gfx1101/triton/conv/conv_wino_f4x3_gemm/DEFAULT.json Adds Winograd GEMM tuning.
aiter/ops/triton/configs/gfx1101/triton/conv/conv_prepack/DEFAULT.json Adds prepack tuning.
aiter/ops/triton/configs/gfx1101/triton/conv/conv_general/DEFAULT.json Adds general-convolution fallback.
aiter/ops/triton/configs/gfx1101/triton/conv/conv_3x3_nhwc/DEFAULT.json Adds NHWC configuration coverage.
aiter/ops/triton/configs/gfx1101/triton/conv/conv_3x3_nchw/DEFAULT.json Moderate (1 vote): The table changes default dispatch from cblocked to direct without target-card route comparisons. Benchmark every pin including packing cost, or omit the optional table.
aiter/ops/triton/configs/gfx1101/triton/conv/conv_3x3_cblocked/DEFAULT.json Adds cblocked tuning.
aiter/ops/triton/configs/gfx1101/triton/conv/conv_1x1/DEFAULT.json Adds 1×1 tuning.
aiter/ops/triton/configs/CLAUDE.md Documents the new architecture directories.
Review details
  • Files reviewed: 20/20 changed files
  • Comments generated: 3
  • Review effort level: Balanced

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread op_tests/triton_tests/conv/_helpers.py Resolved
@Ragua1 Ragua1 changed the title [Triton/Gluon] Users/ragua1/conv configs gfx1101 gfx1150 [Triton] [Conv2D] Add Conv2D configuration files for gfx1101 (RDNA3) and gfx1150 (RDNA3.5) Sep 3, 2026
@Ragua1

Ragua1 commented Sep 3, 2026

Copy link
Copy Markdown
Contributor Author

@saeid-rostami — adding you as you asked on #4869. This is the gfx1101 tuning you invited, with gfx1150 alongside it since I have that part too.

I cannot use the Reviewers field from a fork, so this mention is the request.

That table is the optional ninth family and adding it is a routing change on
these two architectures, which the rest of this PR is not.

_nchw_direct_is_profitable returns False when has_conv_config("CONV-3X3-NCHW")
is False, so today every 3x3 shape on gfx1101 and gfx1150 takes CBLOCKED_NCHW.
Shipping the table with route_exact_only moves 7 shapes on gfx1101 and 4 on
gfx1150 onto DIRECT_NCHW_3X3, and none of those 11 has a direct-against-cblocked
measurement on either card.

The tables had already been trimmed to the donor's key set so they would carry no
routing change relative to gfx1100 and gfx1151. That was the wrong baseline: the
comparison that matters is against what these cards do now, which is no table at
all.

gfx1200, gfx1250, gfx942 and gfx950 ship no CONV-3X3-NCHW either, so the
remaining eight families are the mandatory set and this PR now carries values
only.

Counts after this commit:

  gfx1101   182 entries, 16 tuned on the card, 166 inherited from gfx1100
  gfx1150   181 entries, 97 tuned on the card,  84 inherited from gfx1151

The table returns in a follow-up that measures both routes per key.

Signed-off-by: Martin Domanský <ragua@email.cz>
@github-actions github-actions Bot changed the title [Triton] [Conv2D] Add Conv2D configuration files for gfx1101 (RDNA3) and gfx1150 (RDNA3.5) [Triton/Gluon] [Conv2D] Add Conv2D configuration files for gfx1101 (RDNA3) and gfx1150 (RDNA3.5) Sep 3, 2026
@Ragua1

Ragua1 commented Sep 3, 2026

Copy link
Copy Markdown
Contributor Author

If you would rather have that family in this PR after all, ask and I will do it here — I have both cards, and it means running the two-route probe per key and reporting both arms, including the keys that fail. I split it out because a routing change and a values change read better separately, but that is my judgement about reviewability, not a constraint.

On the CI route tests: the repository has no RDNA runner, so a route test would not execute for either architecture. With conv_3x3_nchw gone there is also no route behaviour left in this PR to cover

@saeid-rostami

saeid-rostami commented Sep 3, 2026 •

Copy link
Copy Markdown
Contributor

If you would rather have that family in this PR after all, ask and I will do it here — I have both cards, and it means running the two-route probe per key and reporting both arms, including the keys that fail. I split it out because a routing change and a values change read better separately, but that is my judgement about reviewability, not a constraint.

On the CI route tests: the repository has no RDNA runner, so a route test would not execute for either architecture. With conv_3x3_nchw gone there is also no route behaviour left in this PR to cover

@Ragua1
I think it's ok to keep it separate. Can you please tell me which Triton version did you use for tuning?

@saeid-rostami saeid-rostami left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@brunomazzottiamd
Overall, this PR LGTM, I just do not know which Triton version was used for tuning.

@Ragua1

Ragua1 commented Sep 3, 2026

Copy link
Copy Markdown
Contributor Author

@saeid-rostami it was Triton 3.7.1 on windows (triton-windows 3.7.1.post27)

@brunomazzottiamd

Copy link
Copy Markdown
Contributor

@saeid-rostami it was Triton 3.7.1 on windows (triton-windows 3.7.1.post27)

Please keep in mind that our team won't be conducting any performance tuning or maintenance work on Windows. We currently don't have resources to do it (engineers and CI nodes).

@brunomazzottiamd brunomazzottiamd left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! It's basically a new set of config files, it won't hurt CDNA at all. I'm going to merge this PR on behalf of @saeid-rostami approval, he's the author of Conv2D Triton kernels.

@brunomazzottiamd
brunomazzottiamd merged commit 40553a1 into ROCm:main Sep 8, 2026
71 checks passed
@Ragua1
Ragua1 deleted the users/Ragua1/conv-configs-gfx1101-gfx1150 branch September 10, 2026 05:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants