Skip to content

[AMD][Qwen3.5] Run MI355X FP8 AgentX on the 20261117 image with the monokernel - #5126

Open
yichiche wants to merge 4 commits into
mainfrom
amd/qwen35-fp8-mi355x-daily-monokernel
Open

yichiche wants to merge 4 commits into
mainfrom
amd/qwen35-fp8-mi355x-daily-monokernel

Conversation

@yichiche

Copy link
Copy Markdown
Collaborator

Qwen3.5-397B-A17B FP8 on MI355X still ran AgentX from lmsysorg/sglang-rocm:v0.5.22-rocm720-mi35x-20261020 with SGLang's own decode. This moves qwen3.5-fp8-mi355x-sglang-agentic-mtp to lmsysorg/sglang-rocm:v0.5.23-rocm720-mi35x-20261117 and adds --enable-amd-monokernel.

sgl_amd_mono.qwen35_a17b serves TP4 decode widths 1–8 and this recipe's EAGLE chain (topk 1, 3 steps, 4 draft tokens). Wider steps stay on SGLang's forward. The concurrency grid, HiCache sizes, and draft token counts are unchanged.

The ROCm image installs the sgl-amd-mono commit pinned in SGLang. sgl-project/sglang#50813 pins a04ceab, the merge of sgl-project/sgl-amd-mono#58, so qwen35_a17b is in the 20261117 tag when that pin is in the build.

AI model disclosure

Prepared by Grok 4.7 in Cursor. The model edited the image pin, the server flag, and the performance changelog, and opened this pull request.

…onokernel

The daily arm was still on the October 20 image. Point it at today's
ROCm 7.2 image and enable the AMD monokernel for TP4 widths 1-8 and the
EAGLE verify chain this recipe already uses.
The daily tag moves from 20261116 to 20261117 so the arm uses the image built after the monokernel pin.
@github-actions

Copy link
Copy Markdown
Contributor

Thanks for the contribution!

  • Review: If this PR changes files owned by someone other than a repository admin or @SemiAnalysisAI/core, ask one eligible CODEOWNER to complete the latest PR_REVIEW_CHECKLIST.md before contacting a core maintainer on Slack. Follow the template exactly, including As a PR reviewer and CODEOWNER, I have reviewed this and have, so sign-off verification triggers.
  • PR verification: Sweeps run only when the PR appends an inferencex-e2e/perf-changelog.yaml entry and carries exactly one primary label: full-sweep-fail-fast (strongly recommended; canary plus per-matrix fail-fast), full-sweep-enabled (canary; matrix jobs continue after a failure), or non-canary-full-sweep-enabled (no canary or fail-fast). The modifiers all-evals, evals-only, and agentx-fast require a primary label. On fork PRs, a maintainer applies the label. See sweep labels and reuse.
  • After merging: PR authors must ensure all GitHub Actions jobs pass. Transient failures often pass on rerun; see how to rerun failed jobs.
中文

感谢你的贡献!

  • **审阅:**如果 PR 修改的文件归属于仓库管理员及 @SemiAnalysisAI/core 之外的 CODEOWNER,请先联系一位有资格的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md,再通过 Slack 联系核心维护者。必须严格遵循模板,并保留 As a PR reviewer and CODEOWNER, I have reviewed this and have,才能触发签核验证。
  • **PR 验证:**只有当 PR 在 inferencex-e2e/perf-changelog.yaml 末尾追加了条目,并且恰好带有一个主标签时,才会运行扫描:full-sweep-fail-fast(强烈推荐;canary 加逐矩阵 fail-fast)、full-sweep-enabled(有 canary;矩阵任务在失败后继续运行)或 non-canary-full-sweep-enabled(无 canary,也无 fail-fast)。修饰标签 all-evals、evals-only 和 agentx-fast 必须与主标签一起使用。fork PR 的标签由维护者添加。参见扫描标签与复用。
  • **合并后:**PR 作者必须确保所有 GitHub Actions 任务通过。临时性失败通常可以通过重新运行恢复;参见重新运行失败任务的说明。

@yichiche yichiche added the full-sweep-fail-fast Full sweep with canary gate; first failure cancels the rest of that matrix (recommended) label Nov 18, 2026
- "Concurrency 1 and 4 (latency_specialized decode backend) pick up the seungrokj/mum#28 levers through their defaults (all 1): VLLM_K31_BEAT_MARKOV, VLLM_K31_BEAT_DRAFTMETA, VLLM_K31_BEAT_K2SYNC, VLLM_K31_BEAT_EAGERPREP, VLLM_K31_BEAT_KDAK1 and VLLM_K3_INDEXER_SAMPLED_TOPK, plus VLLM_KIMI_SD_DSA_PROLOGUE_INDEXER at concurrency 1, where VLLM_KIMI_SD_DSA_PROLOGUE is on. Concurrency 16-48 use the stock decode backend, which ignores these flags."
- "No change to layouts, draft token counts, capture sizes or the DSpark draft's data types."
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/5089
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/5089

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 Merging this rewrites a byte in an existing changelog entry, which the repo forbids. The changelog conflict tooling treats this file as append-only. The hunk strips two trailing spaces from the previous entry's pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/5089 line at perf-changelog.yaml:12920. AGENTS.md:88 says to preserve every existing byte and append only at the tail. Fix: restore the two trailing spaces on line 12920 exactly as on the base, and change only the lines after it to add the new block.

Why this was flagged

The diff removes from the end of line 12920 (pr-link: .../pull/5089), which belongs to the entry before this PR's new block. AGENTS.md:88 says perf-changelog.yaml is append-only and byte-sensitive, and conflicts must be resolved only with the changelog conflict helper. On the base branch that line ends with two spaces, so this PR edits a historical entry rather than only appending. Another open PR that also appends after the #5089 entry will then conflict on that line, and the append-only helper and checks that assume an unchanged prefix see a modified existing byte. No safeguard in the diff keeps the old bytes, because the editor's trailing-whitespace stripping touched the context line.

Verification: Trigger: any run of the repo's changelog tooling on this PR's diff. The hunk replaces pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/5089 $ with ...pull/5089$. infx/matrix/plan.py:62-69 raises ValueError("Deletions are not allowed in ...") for the removed non-blank line, so the PR sweep fails at validation and the post-merge ingest fails too.

@yichiche yichiche added full-sweep-fail-fast Full sweep with canary gate; first failure cancels the rest of that matrix (recommended) and removed full-sweep-fail-fast Full sweep with canary gate; first failure cancels the rest of that matrix (recommended) labels Nov 18, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

full-sweep-fail-fast Full sweep with canary gate; first failure cancels the rest of that matrix (recommended)

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

1 participant