Skip to content

[FlyDSL] Precompile only the GPU_ARCHS jobs in the FlyDSL AOT build - #6847

Closed
RElbers wants to merge 5 commits into
mainfrom
relbers/flydsl-gemm-aot-arch-filter
Closed

RElbers wants to merge 5 commits into
mainfrom
relbers/flydsl-gemm-aot-arch-filter

Conversation

@RElbers

@RElbers RElbers commented Oct 22, 2026 •

Copy link
Copy Markdown
Contributor

Motivation

When setup.py precompiles FlyDSL kernels for a wheel (run_aot), it collects jobs from every kernel family. MoE and MegaMoE keep only the archs GPU_ARCHS names (#6362). The other families do not:

  • GEMM filters by arch only in main(), the python -m aiter.aot.flydsl.gemm entry point. run_aot never goes through main(), so a gfx950 wheel also builds the gfx1250 kernels, and a gfx942 wheel builds 7885 kernels it can never run.
  • chunk_gdn_h and fmha_fp8 do not filter at all, so a gfx950 wheel builds the gfx942-tuned chunk kernels and a gfx942 wheel builds the gfx950-only FP8 FMHA kernels.
  • Conv filters inside parse_csv, but reads ARCH before GPU_ARCHS. Build images export ARCH as a machine type such as x86_64, so on those hosts the filter keeps nothing and the wheel ships no precompiled conv kernels.

An AOT miss is not fatal (the kernel is JIT-compiled at runtime), but the first two cases waste build time and wheel size, and the third turns every conv call into a runtime compile.

Technical details

  • Every family run_aot pools now reads GPU_ARCHS through aot_target_archs() in common.py, the helper [OPUS] [FlyDSL] [JIT] [Feature] Build the MoE and MegaMoE AOT caches by replaying the runtime on CPU #6362 added for MoE, so all kinds agree on what a target list is. It does not read ARCH.
  • GEMM, conv and chunk_gdn_h drop the jobs of unlisted archs in parse_csv. A job's arch is resolved the way its compile step resolves it: the CSV's gfx column with a cu_num fallback for GEMM and conv, cu_num for chunk_gdn_h, which compiles from it. chunk_gdn_h gets a job_arch() that both its filter and its compile step use.
  • fmha_fp8 compiles for gfx950 only, so jobs_for_shape returns nothing unless GPU_ARCHS is unset or lists gfx950 (keeps_arch() in common.py).
  • On the python -m entry points, collect_cli_jobs() / cli_arch_env() apply ARCH as GPU_ARCHS while collecting, so ARCH still takes precedence there, as on main, and parse_csv is the only filter. GEMM's second filter in main() is removed.

Behavior changes

FlyDSL jobs collected by run_aot from the in-tree tuned CSVs:

Environment Build GEMM conv chunk_gdn_h fmha_fp8
GPU_ARCHS unset main 7885 139 288 92
this PR 7885 139 288 92
ARCH=x86_64 main 7885 0 288 92
this PR 7885 139 288 92
GPU_ARCHS=gfx950 main 7885 139 288 92
this PR 7151 139 0 92
GPU_ARCHS=gfx942 main 7885 0 288 92
this PR 0 0 288 0
GPU_ARCHS=gfx1250 main 7885 0 288 92
this PR 734 0 0 0

The 734 GEMM jobs a gfx950 build drops are the gfx1250 rows. Every chunk_gdn_h tuned row is gfx942. Unset and GPU_ARCHS=native keep every job, as before.

  • GPU_ARCHS lists are ;-separated, as aot_target_archs() and the CK build read them. Conv also split on ,; it no longer does.
  • On the python -m entry points, ARCH still overrides GPU_ARCHS, as on main.

Test plan

  • New op_tests/test_flydsl_aot_arch_filter.py:
    • collect_cli_jobs with GPU_ARCHS=gfx942 ARCH=gfx950 collects for gfx950 and restores GPU_ARCHS afterwards (no FlyDSL needed).
    • GEMM and conv parse_csv on rows of two archs under unset and a single listed arch; conv also under ARCH=x86_64, which it now ignores (skips without FlyDSL).
    • Every kind run_aot pools collects nothing for an unlisted arch and something when GPU_ARCHS is unset, so a new kind cannot skip the filter.
    • 4 tests / 10 subtests pass.
  • The job counts above come from _collect_aot_jobs_for, the collection step run_aot uses.
  • ruff and black are clean on the touched files.

@github-actions

Copy link
Copy Markdown
Contributor

🏷️ CI Guide

Runs automatically on every PR:

  • ✅ Pre-checks (submodule verification, code formatting)
  • ✅ Aiter op tests (gfx942 + gfx950)
  • ✅ Triton tests on MI35X (only when aiter/ops/triton/** or related paths are changed)

Extended tests (opt-in via labels):

Label Tests
ci:gfx1250-ffm-triton Run the five-shard gfx1250 FFM Triton test suite
ci:triton-300x Run an additional Triton test job on MI300X in PRs (added automatically when gfx942 configs change); main branch always runs both MI35X and MI300X
ci:triton-355 Run the full Triton test suite on MI35X, not only the tests the change affects
ci:windows Windows JIT op smoke test on the gfx1201 ROCm runner
ci:multi-gpu Aiter multi-GPU tests on the 8-GPU runner
ci:sglang SGLang integration tests: DeepSeek-R1-MXFP4 accuracy, Qwen 3.5 accuracy
ci:atom ATOM benchmark: DeepSeek-R1-0528, GPT-OSS-120B
ci:atom_full ATOM accuracy suite for PR and main models from ATOM models_accuracy.json
ci:vllm vLLM benchmark: DeepSeek-V4-Pro, MiniMax-M3-MXFP4
ci:all All standard extended tests (excludes ci:atom_full)

Only add ci:atom_full for FlyDSL or Triton upgrades.
Add labels via the sidebar or gh pr edit 6847 --add-label <label>

One backend per PR:
A PR changes one kernel backend: [Triton/Gluon] (Triton and Gluon count as one), [HIP], [ASM], [CK], [OPUS] or [FlyDSL]. If the title ends up with two backend tags, split the PR -- as stacked pull requests when one part cannot merge without the other.

PR title tags & labels:
Component tags ([Triton/Gluon], [HIP], [CK], [ASM], ...) are added to the PR title and as PR labels automatically from the changed files and re-synced on every push — change-type tags like [fix]/[Perf], op tags like [MLA], and human labels (ci:*) are left untouched. Add the no-auto-title label to stop the title rewrites; labels stay in sync either way.

@RElbers
RElbers force-pushed the relbers/flydsl-gemm-aot-arch-filter branch 2 times, most recently from c6560f9 to 0906386 Compare October 23, 2026 12:43
@RElbers
RElbers marked this pull request as ready for review October 23, 2026 12:46
@RElbers
RElbers requested review from a team and a balanced review from Copilot October 23, 2026 12:46
@github-actions github-actions Bot changed the title Precompile only the FlyDSL GEMM and conv jobs for GPU_ARCHS [FlyDSL] Precompile only the FlyDSL GEMM and conv jobs for GPU_ARCHS Oct 23, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The scoped changes consistently filter build jobs while preserving CLI precedence and compile-target resolution, with no blocking issues identified.

No open issues 🎉

What changed in this PR

This PR makes FlyDSL GEMM and convolution AOT job collection respect wheel build targets in GPU_ARCHS, avoiding unrelated GEMM builds and convolution jobs being dropped because of a machine-type ARCH value.

Changes:

  • Centralizes build-target parsing and applies filtering inside both CSV parsers.
  • Preserves ARCH precedence for CLI entry points while restoring the environment after collection.
  • Adds focused regression tests and documents the build-versus-CLI behavior.
File Description
op_tests/​test_flydsl_aot_arch_filter.py Tests target parsing, CLI overrides, and GEMM filtering.
aiter/​aot/​flydsl/​README.md Documents architecture selection for builds and CLI usage.
aiter/​aot/​flydsl/​gemm.py Moves architecture filtering into CSV job collection.
aiter/​aot/​flydsl/​conv.py Uses shared filtering without build-time ARCH interference.
aiter/​aot/​flydsl/​common.py Adds shared target parsing and CLI collection helpers.

Review effort: Balanced

- Filters FlyDSL GEMM jobs by GPU_ARCHS inside parse_csv, so setup.py's
  run_aot skips archs the wheel is not built for instead of precompiling
  every arch in the tuned CSVs.
- Reads only GPU_ARCHS in the conv parse_csv, since build images export ARCH
  as a machine type such as x86_64, which dropped every conv job from a wheel
  build.
- Adds get_gfx_list_env to common, which parses an explicit GPU_ARCHS split on
  ';' or ',' and treats unset or native as no filter.
- Adds collect_cli_jobs to common, which applies ARCH as GPU_ARCHS while the
  python -m entry points collect jobs, so parse_csv stays the only arch filter
  and ARCH still takes precedence there.
- Covers GPU_ARCHS parsing, ARCH over GPU_ARCHS, and the GEMM parse_csv filter
  in a new arch filter test.
@RElbers
RElbers force-pushed the relbers/flydsl-gemm-aot-arch-filter branch from 0906386 to 487e35e Compare October 26, 2026 08:56
@RElbers RElbers changed the title [FlyDSL] Precompile only the FlyDSL GEMM and conv jobs for GPU_ARCHS [FlyDSL] Precompile only the GPU_ARCHS jobs in the FlyDSL AOT build Oct 27, 2026
- Reads GPU_ARCHS through aot_target_archs for GEMM and conv, replacing
  get_gfx_list_env, so every kind agrees on what a target list is.
- Drops the chunk_gdn_h jobs of unlisted archs in parse_csv, resolving a job's
  arch through a job_arch shared with its compile step.
- Builds no fmha_fp8 jobs unless GPU_ARCHS is unset or lists gfx950, the only
  arch those kernels compile for.
- Lets ARCH replace GPU_ARCHS for the fmha_fp8 entry point, as for the others.
- Checks that every kind run_aot pools collects nothing for an unlisted arch.
@github-actions github-actions Bot changed the title [FlyDSL] Precompile only the GPU_ARCHS jobs in the FlyDSL AOT build [FlyDSL] Precompile only the GPU_ARCHS jobs in the FlyDSL AOT build Oct 27, 2026
@RElbers

RElbers commented Nov 3, 2026

Copy link
Copy Markdown
Contributor Author

Closing now that #6956 is merged

@RElbers RElbers closed this Nov 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants