Skip to content

[FlyDSL] [Misc] Simplify PA decode reducer bound and reuse shared helpers - #7077

Merged
coderfeli merged 1 commit into
mainfrom
pa-decode-cleanup
Oct 28, 2026
Merged

coderfeli merged 1 commit into
mainfrom
pa-decode-cleanup

Conversation

@fsx950223

Copy link
Copy Markdown
Contributor

Motivation

Cleanups in the FlyDSL PA decode wrapper and kernels. The reducer launcher took two partition limits where one is enough, and several spots duplicated helpers that aiter or FlyDSL already provide.

Changes

  • One reducer partition bound: launch_pa_decode_ps_reduce takes a single max_partitions, which is the tight per-call bound. pa_decode computes it from the plan cap, plan capacity, context tiles and the sliding-window cap, and the separate partition_bound argument is removed.
    • Small bounds still compile exactly, so short contexts keep the register-only reducer.
    • Above a wave, the power-of-two round-up is now capped by the device CU count instead of the plan's max_partitions. With a non-default plan cap that isn't a power of two, the compiled bound can round past it (for example 100 → 128). This is safe because the reducer's per-sequence buffer bound zero-fills partitions beyond the live count.
  • Shared helpers instead of local copies:
    • pa_decode uses dtypes.fp8 instead of its own arch → FP8 dtype table. The supported-arch check stays.
    • The reducer uses tensor_shim._FX_ELEM instead of a local _DTYPE_MAP, and still accepts only f32, f16 and bf16.
    • The gfx1250 WMMA and FP8-conversion calls use fx.as_ir_value instead of tensor_shim._to_raw.
  • gfx1250 thread count: Gfx1250Traits.BLOCK_THREADS (NWARP * WAVE = 128) replaces the hardcoded 128 in known_block_size, the launch block, the direct-output copy loop and stage_vector_dma. It's a class constant, not a dataclass field, so traits cache keys don't change.
  • sliding_window: pa_decode and plan_pa_decode treat any nonpositive value as disabled instead of raising for values below -1. The docstrings say so.

Test

  • op_tests/test_flydsl_pa_decode.py on gfx1250 with FlyDSL 0.4.0: 119 passed, 5 skipped.
  • plan_pa_decode(..., sliding_window=-5) builds the same plan as -1.
  • gfx942 and gfx950 were not run on hardware. The reducer-bound, dtypes.fp8 and _FX_ELEM changes also apply there.
  • black and ruff pass.

🤖 Generated with Claude Code

- Fold the reducer's partition_bound into max_partitions: pa_decode
  passes one tight per-call bound; above a wave the power-of-two
  round-up is capped by the CU count.
- Use dtypes.fp8 for the KV FP8 dtype, tensor_shim._FX_ELEM for the
  reducer's dtype lookup, and fx.as_ir_value instead of _to_raw.
- Name the gfx1250 thread count Gfx1250Traits.BLOCK_THREADS instead of
  hardcoding 128.
- Treat any nonpositive sliding_window as disabled in pa_decode and
  plan_pa_decode instead of rejecting values below -1.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@fsx950223
fsx950223 requested review from a team and a balanced review from Copilot October 28, 2026 07:08

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota.

@github-actions

Copy link
Copy Markdown
Contributor

🏷️ CI Guide

Runs automatically on every PR:

  • ✅ Pre-checks (submodule verification, code formatting)
  • ✅ Aiter op tests (gfx942 + gfx950)
  • ✅ Triton tests on MI35X (only when aiter/ops/triton/** or related paths are changed)

Extended tests (opt-in via labels):

Label Tests
ci:gfx1250-ffm-triton Run the five-shard gfx1250 FFM Triton test suite
ci:triton-300x Run an additional Triton test job on MI300X in PRs (added automatically when gfx942 configs change); main branch always runs both MI35X and MI300X
ci:triton-355 Run the full Triton test suite on MI35X, not only the tests the change affects
ci:windows Windows JIT op smoke test on the gfx1201 ROCm runner
ci:multi-gpu Aiter multi-GPU tests on the 8-GPU runner
ci:sglang SGLang integration tests: DeepSeek-R1-MXFP4 accuracy, Qwen 3.5 accuracy
ci:atom ATOM benchmark: DeepSeek-R1-0528, GPT-OSS-120B
ci:atom_full ATOM accuracy suite for PR and main models from ATOM models_accuracy.json
ci:vllm vLLM benchmark: DeepSeek-V4-Pro, MiniMax-M3-MXFP4
ci:all All standard extended tests (excludes ci:atom_full)

Only add ci:atom_full for FlyDSL or Triton upgrades.
Add labels via the sidebar or gh pr edit 7077 --add-label <label>

One backend per PR:
A PR changes one kernel backend: [Triton/Gluon] (Triton and Gluon count as one), [HIP], [ASM], [CK], [OPUS] or [FlyDSL]. If the title ends up with two backend tags, split the PR -- as stacked pull requests when one part cannot merge without the other.

PR title tags & labels:
Component tags ([Triton/Gluon], [HIP], [CK], [ASM], ...) are added to the PR title and as PR labels automatically from the changed files and re-synced on every push — change-type tags like [fix]/[Perf], op tags like [MLA], and human labels (ci:*) are left untouched. Add the no-auto-title label to stop the title rewrites; labels stay in sync either way.

@coderfeli
coderfeli merged commit b0043b6 into main Oct 28, 2026
47 checks passed
@coderfeli
coderfeli deleted the pa-decode-cleanup branch October 28, 2026 11:54
samremes added a commit that referenced this pull request Oct 29, 2026
#7077 made pa_decode require dtypes.fp8 for FP8 KV instead of a per-arch
dtype table. The capability and default-budget helpers still used that
table, so they could accept a dtype pa_decode rejects. Compare against
dtypes.fp8 there too.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
solinzby1 pushed a commit that referenced this pull request Nov 2, 2026
…7077)

- Fold the reducer's partition_bound into max_partitions: pa_decode
  passes one tight per-call bound; above a wave the power-of-two
  round-up is capped by the CU count.
- Use dtypes.fp8 for the KV FP8 dtype, tensor_shim._FX_ELEM for the
  reducer's dtype lookup, and fx.as_ir_value instead of _to_raw.
- Name the gfx1250 thread count Gfx1250Traits.BLOCK_THREADS instead of
  hardcoding 128.
- Treat any nonpositive sliding_window as disabled in pa_decode and
  plan_pa_decode instead of rejecting values below -1.

Co-authored-by: who who who <17592563+fsx950223@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants