Repository navigation
[Triton/Gluon] [Conv2D] Add Conv2D configuration files for gfx1101 (RDNA3) and gfx1150 (RDNA3.5) - #5246
Conversation
Nine tables per architecture in the nested layout, tuned on hardware. Both architectures currently have none, so every Conv2D family is a hard FileNotFoundError from load_config_json(required=True) rather than a fallback. gfx1101 (RDNA3, RX 7800 XT): 190 entries, 20 tuned on the card, 170 inherited byte-identical from gfx1100. gfx1150 (RDNA3.5, Radeon 890M): 186 entries, 100 tuned, 86 inherited from gfx1151. Entry counts cover every section - shapes, variant pins, M_LEQ_* buckets and any - not just shapes. The structure of every file matches its donor exactly: same sections, same key set per section, same route_exact_only. Routing behaviour is therefore unchanged and only the values differ. In particular CONV-3X3-NCHW ships only the shape keys the donor already has, because under route_exact_only the key set is what _nchw_direct_is_profitable uses to choose between DIRECT_NCHW_3X3 and CBLOCKED_NCHW, and that comparison was not part of these measurements. Every candidate was checked for correctness before being timed - rel_l2 < 1e-2 against torch.nn.functional.conv2d, the bar op_tests/triton_tests/conv uses - and each family was driven through its own public entry point rather than through the router. Follows the invitation in ROCm#4869 (comment) Signed-off-by: Martin Domanský <ragua@email.cz>
…ARCHS Without this the two new tables are never exercised by op_tests/triton_tests/conv. Note that the repository's CI has no RDNA machine, so this does not by itself cause the tables to be tested anywhere - it only makes the tests runnable on hardware that has them. Signed-off-by: Martin Domanský <ragua@email.cz>
configs/CLAUDE.md documents which architectures have a config tree. ROCm#5085 rewrote that document for the nested layout, so it is maintained and a new arch belongs in it. Separate commit from the data on purpose: it is your document, and a doc change should be reviewable on its own rather than folded into eighteen JSON files. Signed-off-by: Martin Domanský <ragua@email.cz>
🏷️ CI GuideRuns automatically on every PR:
Extended tests (opt-in via labels):
PR title tags & labels: |
There was a problem hiding this comment.
🟡 Changes recommended
Direct-NCHW routing lacks required comparative benchmarks, and CI route coverage for both architectures is missing.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Pull request overview
Adds Triton Conv2D configuration coverage for gfx1101 and gfx1150, preventing missing-table failures.
Changes:
- Adds nine configuration families for each architecture.
- Enables both architectures in Conv2D test helpers.
- Documents the expanded configuration layout.
File summaries
| File | Review |
|---|---|
op_tests/triton_tests/conv/_helpers.py |
Moderate (2 votes): Existing CI does not exercise gfx1101/gfx1150 exact routing. Add pinned and unpinned route tests for both architectures. |
aiter/ops/triton/configs/gfx1150/triton/conv/conv_wino_f4x3_output/DEFAULT.json |
Adds Winograd output tuning. |
aiter/ops/triton/configs/gfx1150/triton/conv/conv_wino_f4x3_input/DEFAULT.json |
Adds Winograd input tuning. |
aiter/ops/triton/configs/gfx1150/triton/conv/conv_wino_f4x3_gemm/DEFAULT.json |
Adds Winograd GEMM tuning. |
aiter/ops/triton/configs/gfx1150/triton/conv/conv_prepack/DEFAULT.json |
Adds prepack tuning. |
aiter/ops/triton/configs/gfx1150/triton/conv/conv_general/DEFAULT.json |
Adds general-convolution fallback. |
aiter/ops/triton/configs/gfx1150/triton/conv/conv_3x3_nhwc/DEFAULT.json |
Adds NHWC tuning. |
aiter/ops/triton/configs/gfx1150/triton/conv/conv_3x3_nchw/DEFAULT.json |
Moderate (1 vote): The table changes default dispatch from cblocked to direct without target-card route comparisons. Benchmark all four pins including packing cost, or remove the unproven pins/table. |
aiter/ops/triton/configs/gfx1150/triton/conv/conv_3x3_cblocked/DEFAULT.json |
Adds cblocked tuning. |
aiter/ops/triton/configs/gfx1150/triton/conv/conv_1x1/DEFAULT.json |
Adds 1×1 tuning. |
aiter/ops/triton/configs/gfx1101/triton/conv/conv_wino_f4x3_output/DEFAULT.json |
Adds Winograd output tuning. |
aiter/ops/triton/configs/gfx1101/triton/conv/conv_wino_f4x3_input/DEFAULT.json |
Adds Winograd input tuning. |
aiter/ops/triton/configs/gfx1101/triton/conv/conv_wino_f4x3_gemm/DEFAULT.json |
Adds Winograd GEMM tuning. |
aiter/ops/triton/configs/gfx1101/triton/conv/conv_prepack/DEFAULT.json |
Adds prepack tuning. |
aiter/ops/triton/configs/gfx1101/triton/conv/conv_general/DEFAULT.json |
Adds general-convolution fallback. |
aiter/ops/triton/configs/gfx1101/triton/conv/conv_3x3_nhwc/DEFAULT.json |
Adds NHWC configuration coverage. |
aiter/ops/triton/configs/gfx1101/triton/conv/conv_3x3_nchw/DEFAULT.json |
Moderate (1 vote): The table changes default dispatch from cblocked to direct without target-card route comparisons. Benchmark every pin including packing cost, or omit the optional table. |
aiter/ops/triton/configs/gfx1101/triton/conv/conv_3x3_cblocked/DEFAULT.json |
Adds cblocked tuning. |
aiter/ops/triton/configs/gfx1101/triton/conv/conv_1x1/DEFAULT.json |
Adds 1×1 tuning. |
aiter/ops/triton/configs/CLAUDE.md |
Documents the new architecture directories. |
Review details
- Files reviewed: 20/20 changed files
- Comments generated: 3
- Review effort level: Balanced
💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
|
@saeid-rostami — adding you as you asked on #4869. This is the gfx1101 tuning you invited, with gfx1150 alongside it since I have that part too. I cannot use the Reviewers field from a fork, so this mention is the request. |
That table is the optional ninth family and adding it is a routing change on
these two architectures, which the rest of this PR is not.
_nchw_direct_is_profitable returns False when has_conv_config("CONV-3X3-NCHW")
is False, so today every 3x3 shape on gfx1101 and gfx1150 takes CBLOCKED_NCHW.
Shipping the table with route_exact_only moves 7 shapes on gfx1101 and 4 on
gfx1150 onto DIRECT_NCHW_3X3, and none of those 11 has a direct-against-cblocked
measurement on either card.
The tables had already been trimmed to the donor's key set so they would carry no
routing change relative to gfx1100 and gfx1151. That was the wrong baseline: the
comparison that matters is against what these cards do now, which is no table at
all.
gfx1200, gfx1250, gfx942 and gfx950 ship no CONV-3X3-NCHW either, so the
remaining eight families are the mandatory set and this PR now carries values
only.
Counts after this commit:
gfx1101 182 entries, 16 tuned on the card, 166 inherited from gfx1100
gfx1150 181 entries, 97 tuned on the card, 84 inherited from gfx1151
The table returns in a follow-up that measures both routes per key.
Signed-off-by: Martin Domanský <ragua@email.cz>
|
If you would rather have that family in this PR after all, ask and I will do it here — I have both cards, and it means running the two-route probe per key and reporting both arms, including the keys that fail. I split it out because a routing change and a values change read better separately, but that is my judgement about reviewability, not a constraint. On the CI route tests: the repository has no RDNA runner, so a route test would not execute for either architecture. With conv_3x3_nchw gone there is also no route behaviour left in this PR to cover |
@Ragua1 |
saeid-rostami
left a comment
There was a problem hiding this comment.
@brunomazzottiamd
Overall, this PR LGTM, I just do not know which Triton version was used for tuning.
|
@saeid-rostami it was Triton 3.7.1 on windows (triton-windows 3.7.1.post27) |
Please keep in mind that our team won't be conducting any performance tuning or maintenance work on Windows. We currently don't have resources to do it (engineers and CI nodes). |
brunomazzottiamd
left a comment
There was a problem hiding this comment.
LGTM! It's basically a new set of config files, it won't hurt CDNA at all. I'm going to merge this PR on behalf of @saeid-rostami approval, he's the author of Conv2D Triton kernels.
Motivation
Conv2D raises
FileNotFoundErroron gfx1101: it ships no tables and eight families are mandatory. This PR tunes and adds them, following the invitation in #4869 (comment).gfx1150 is included as well because that part is also available to me for measurement, and it is in the same position — no tables on
main.mainhas them for gfx1100, gfx1151, gfx1200, gfx1201, gfx1250, gfx942 and gfx950.Data plus one test-helper line; no kernel, routing or loader code changed.
Eight families per architecture, not nine.
CONV-3X3-NCHWis the optional ninth and is deliberately not here — see below.Technical Details
2.15.0a0+rocm10.1.0a202608222.11.0+rocm7.15.0a20260728Both version rows are given because the two move independently: the torch wheel tag went 7.15 → 10.1 while the HIP runtime went 7.15 → 7.16, so quoting either alone is ambiguous.
entriescounts every section — eachshapes[key],shapes_<variant>[key],M_LEQ_<n>andanyis one. gfx1101 is 149 + 25 + 8, gfx1150 is 152 + 7 + 14 + 8.Every file's structure is identical to its donor's — same sections, same key set per section, same
route_exact_only. That makes the next two claims checkable rather than rhetorical:CONV-3X3-NCHW, is not in this PR. These files change which config the chosen kernel runs with, never which kernel is chosen.No architecture name appears inside the sixteen files.
Per file, tuned / inherited:
conv_1x1conv_3x3_cblockedconv_3x3_nhwcconv_generalconv_prepackconv_wino_f4x3_inputconv_wino_f4x3_outputconv_wino_f4x3_gemmShape keys are the donor's choice in all eight families: the tuner iterates the own-arch table, and that table was seeded from the donor.
Four commits: the config files, the
SUPPORTED_ARCHSline, the layout table inconfigs/CLAUDE.md, and the removal described next. Without theSUPPORTED_ARCHSline the tables are never exercised by the conv tests.What is excluded, and why
CONV-3X3-NCHWis not in this PR on either architecture, although both campaigns tuned it — 26 shapes each.It is the optional ninth family:
has_conv_config()gates it, and gfx1200, gfx1250, gfx942 and gfx950 ship none. Adding it is a routing change on these two cards, which the other eight families are not._nchw_direct_is_profitable(conv2d.py:82-84) returnsFalsewhen no table is present, so today every 3×3 shape on gfx1101 and gfx1150 takesCBLOCKED_NCHW; shipping the table withroute_exact_onlymoves its keyed shapes ontoDIRECT_NCHW_3X3.Neither campaign measured which of those two is faster, having timed each family through its own public entry point rather than through the router. The one direct-against-cblocked probe I have run — on gfx1150, in one process, with the NCHWc pack inside the cblocked arm where the router pays it — says the answer is not obvious. Ratio is
cblocked_with_pack / direct:A follow-up PR will carry the table with a per-key measurement of both routes, and will present itself as the routing change it is.
If you would rather have it here, say so and I will run that probe against this PR instead of a later one — I have both cards. Splitting it out is my judgement about what is reviewable in one piece, not a constraint.
anyis tuned, and it is the weakest claim hereanyis tier 4 — the fallback for every shape the walk does not match, which is most real shapes. It is tuned in 5 of 8 families on gfx1101 and 7 of 8 on gfx1150, taking the majority value among that family's confirmed entries.On gfx1150 those votes run over 10–19 entries. On gfx1101 four of the five rest on a single entry, because that card's acceptance rule confirms few (below). If you would rather see
anyreverted to the donor below some vote threshold, that is a one-line change per file and I will make it.Test Plan
Correctness is checked on every measurement before it is timed:
rel_l2 < 1e-2againsttorch.nn.functional.conv2d, the barop_tests/triton_tests/convuses, and areshape/permutereference for the pack. A candidate producing wrong data cannot become an entry. Each family is driven through its own public entry point, so a second family's cost is not folded into every number.Two stages per family: a coarse sweep of the candidate space on five M-spread shapes, then the four finalists plus the donor value on every shape in the family's list.
The shipped files are verified against the donor rather than trusted — section set, per-section key set,
route_exact_only, and a byte comparison of every inherited entry at its own key. That check needs no GPU and is what caught the strayCONV-3X3-NCHWkeys above.op_tests/triton_tests/convcould not be run on either host.AITER_TRITON_ONLYis unconditional on win32 and that branch ofaiter/__init__.pydoes not bindaiter.dtypes, which the test helpers import. Independent of this change; I preferred not to put a workaround into the verification I am reporting. The repository's CI has no RDNA machine, so extendingSUPPORTED_ARCHSdoes not by itself get these tables tested anywhere but on the two hosts above.Protocol detail, and two findings that reduced what is claimed
The two sets are not comparable with each other, and it is protocol rather than hardware. gfx1150 timed one call per iteration, median of 20 after 5 warmup, so its numbers include per-call launch and sync overhead. gfx1101 times
repsback-to-back calls in one body and divides. Ontorch.nn.functional.conv2d, C=64, H=W=56, K=64, 3×3, five repeats each:repsA 1.76× spread on one unchanged workload. Neither set's absolute figures should be read against the other's; within each set, every comparison is against the donor measured in the same run.
Why the protocol changed: a measurement floor. 51 of the first 1339 gfx1101 records reported ≤10 µs and 7 of 73 shape winners were physically impossible —
conv_3x3_nchwat C=64, H=W=56, K=64 reported 0.000295 ms, which is 784 TFLOP/s, over ten times this part's fp16 peak, on a shape torch does in 0.100–0.103 ms. The cheapest possible GPU op through the same timer returned −0.000945 to +0.004385 ms across five repeats: it cannot resolve that band or its sign. Amortising overrepsremoves it at source; afterwards the smallest record is 0.0544 ms. That campaign was discarded and re-run.Why few gfx1101 entries are tuned: a noise floor. Most entries ship the donor's own value, so validating a shipped entry against the donor measures an identical configuration — the spread is the card's own noise. 405 such controls over three repeats:
A 1.05× bar therefore sits near p70 of the noise, and re-measuring the 40 initially accepted entries once returned 9 of them below 1.0.
gfx1150's load figure is bimodal — 16 readings of one fixed workload on an idle machine gave 13 near 27.9 ms and 3 near 57 ms — so its values were re-checked for rank preservation on the merged tree rather than compared as absolute times across revisions. Within-run ratios carry; cross-run totals do not.
Machine state. Every record carries a quiet-machine check and the run refuses at 1.5× off it. The gfx1101 campaign's records span 0.90×–1.05× of its reference, so machine state is not a candidate explanation for any of its numbers.
Test Result
gfx1101's acceptance rule is deliberately stricter than tuning normally is here. An entry keeps a tuned value only if all three repeats on the tuning stack exceed 1.0, the median is ≥ 1.05×, and it does not regress on either of two older stacks (torch 2.11 with HIP 7.16.26332, and torch 2.11 with HIP 7.15.26290). Failures revert to the donor and count as inherited. That third condition is the main reason the count is 16 rather than 40.
It earns its place on this card: conv here regressed 2.4× (0.401 → 0.981 ms) on the torch 2.10 → 2.11 move alone, and across the 40 candidates the gain moved between stacks by p50 1.038×, p90 1.482×, max 2.875×. Of the 17 rejections, 15 cite that condition and 2 fail on it alone — those two passed every single-stack test.
gfx1150 was tuned on its own stack in the usual way, which is also how every table already in the repository was produced.
gfx1101, the 11 shape entries that ship:
Cross-stack agreement on the largest of them,
conv_1x1at C=128, H=W=256, K=256:On gfx1101 the donor table is already unbeaten on 163 of its 174 shape-and-bucket entries. What this PR gives that card is the tables it does not have at all today, plus 11 improvements that survived the rule above — 4 in
conv_wino_f4x3_input, 3 inconv_wino_f4x3_output, and one each inconv_1x1,conv_3x3_cblocked,conv_prepackandconv_wino_f4x3_gemm.What is not claimed
conv_generalis not tuned on either arch. One shape, zero tuned,anyinherited. On gfx1101 it improved 4.4 %, below the bar — "measured and insufficient", not "unmeasured".conv_3x3_nhwcon gfx1101 is entirely inherited, all 33 entries: three were accepted and all three reverted by the confirmation pass.CONV-WINO-F4X3-*families are unreachable through the router on both parts (Wave32;conv2d.py:110-135), and only 3 of their 19 shapes even pass theC >= 512 and K_out >= 512 and T >= 98precondition. They must exist regardless. They were tuned rather than grafted because on gfx1150 the inherited transforms measured 2–3× off, which was concealing how fast the disabled path is; on gfx1101 the equivalent is at most 1.40×. No claim that enabling the path would help.Both cards remain available to me and I expect to send further tuning work for them, so nothing here needs to be the last word on either.
Separate defect, filed separately
conv2d_nchw_cblockedfails undertorch.inference_mode()onmain:_pack_cache_key()(conv/_prepack.py:71-83) readsw._version, which raisesRuntimeError: Inference tensors do not track version counter.Every 3×3 route that packs a filter is unavailable to a framework loading weights insideinference_mode(). Independent of any config file — it reproduces with gfx1100's own shipped tables — so it is filed on its own.