Skip to content

feat(v4): reuse sparse prefill ASM for decode - #2271

Merged
valarLip merged 1 commit into
mainfrom
feat/v4-prefill-asm-decode
Sep 17, 2026
Merged

valarLip merged 1 commit into
mainfrom
feat/v4-prefill-asm-decode

Conversation

@amd-ruitang3

Copy link
Copy Markdown
Contributor

Summary

  • add an opt-in path that reuses the gfx1250 H=128 sparse-prefill ASM kernel for DeepSeek-V4 FP8 decode
  • represent decode as a prefix-only stream using shared empty extend CSR metadata, while preserving the dedicated decode ASM fallback
  • cover environment parsing, dispatch eligibility, and prefix/extend argument wiring

Tests

  • python -m pytest -q tests/test_envs.py tests/test_v4_prefill_asm_decode_dispatch.py (64 passed)
  • python -m ruff check atom/model_ops/attentions/deepseek_v4_attn.py atom/model_ops/v4_kernels/paged_decode.py atom/models/deepseek_v4.py atom/utils/envs.py tests/test_envs.py tests/test_v4_prefill_asm_decode_dispatch.py

Co-authored-by: Cursor <cursoragent@cursor.com>
@github-actions

Copy link
Copy Markdown
Contributor

🏷️ CI Guide

Runs automatically on every eligible PR before approval:

  • ✅ Pre Checkin: Black, Ruff, catalog schema validation, non-GPU unit tests

Heavy model tests:

  • ✅ Run after the PR is approved and Pre Checkin passes
  • ✅ Run immediately when an approval review is submitted
  • ✅ Can be requested before approval with labels
Label Tests
ci:full Run all heavy PR model tests: native ATOM, vLLM, and SGLang
ci:atom Run native ATOM model accuracy tests
ci:vllm Run ATOM vLLM OOT model accuracy tests
ci:sglang Run ATOM SGLang model accuracy tests

Heavy jobs are skipped when the PR is not approved and no matching ci:* label is present.
Add labels via the sidebar or gh pr edit 2271 --add-label <label>

@valarLip
valarLip merged commit 46b3d0a into main Sep 17, 2026
23 of 25 checks passed
@valarLip
valarLip deleted the feat/v4-prefill-asm-decode branch September 17, 2026 09:21
sajandhy pushed a commit to sajandhy/ATOM that referenced this pull request Sep 17, 2026
Co-authored-by: ShaoChunLee <Shao-Chun.Lee@amd.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
edwingao28 pushed a commit to SemiAnalysisAI/InferenceX that referenced this pull request Oct 1, 2026
…V4 ATOM AgentX 镜像更新至 nightly_202609291501 (#3605)

* perf(amd): bump DSV4 ATOM AgentX image to nightly_202609291501

Image-only update of dsv4-fp4-mi355x-atom-agentic-mtp on its native
srt-slurm recipe; points, DSpark K6, golden AL and env are unchanged.

将 dsv4-fp4-mi355x-atom-agentic-mtp 的原生 srt-slurm 配方镜像更新为
nightly_202609291501;测试点、DSpark K6、golden AL 和环境变量均不变。

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* chore: set perf-changelog pr-link to #3605

将 perf-changelog 的 pr-link 设为 #3605。

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* perf(amd): bump DSV4 ATOM AgentX image to nightly_202609291501

Update the ATOM image from rocm/atom-dev:nightly_202609161445 (ATOM 0.1.6rc1.dev477+g709c25be0) to rocm/atom-dev:nightly_202609291501 (digest sha256:e0d7134efff68c251acdc2612a9b6c238bc4da5d613832b2bd1251f34bd93355; ATOM 0.1.7.dev46+g74fd942b0, ROCm 7.2.4). Image-only change on the existing native srt-slurm recipe: all ten points (TP8/EP1 at concurrency 1, 2, 4, 8, 16; TP8/DPA8/EP8 native RCCL at 48, 64, 96, 128, 256), DSpark K6 with the thinking_on golden AL 3.77 for throughput and real acceptance for eval, KV and index-cache dtypes, graph capture and all environment settings are unchanged.

Relevant ATOM changes in the range: V4 decode reuses the sparse prefill ASM (ROCm/ATOM#2271), greedy sampler picks via aiter.topk_select (ROCm/ATOM#2244), and FP8 block scales declared scale_fmt ue8m0 are now stored as E8M0 on gfx950 by default (ROCm/ATOM#2419; previously FP32 unless ATOM_FP8_BLOCKSCALE_USE_E8M0_SCALE=1). The upstream DeepSeek-V4 recipes are unchanged across the range, and every recipe flag and choice (all2all-backend rccl, dp-load-balance least_tokens, moe-backend standard) remains valid.

No data-type or precision change to the DeepSeek-V4-Pro-0813 DSpark draft: no online quantization is configured, so it keeps its checkpoint precision; the E8M0 scale storage represents the checkpoint's power-of-two block scales exactly. kv-cache-dtype and index-cache-dtype touch cache storage only.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Honglie Yi <hyi@crsuse2-m2m-v2-020.us-east2-a.compute.internal>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: seungrokj <144636725+seungrokj@users.noreply.github.com>
Co-authored-by: seungrokj <seungrok.jung@amd.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants