Skip to content
Open
Show file tree
Hide file tree
Changes from 4 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,7 @@ base:
speculative-num-steps: 3
speculative-eagle-topk: 1
speculative-num-draft-tokens: 4
enable-amd-monokernel: true
enable-metrics: true
enable-cache-report: true
env:
Expand Down
2 changes: 1 addition & 1 deletion inferencex-e2e/configs/amd-master.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -181,7 +181,7 @@ dsr1-fp8-mi355x-sglang-mtp:
# checkpoint leaves too little KV headroom there for the 256k-capped AgentX
# corpus.
qwen3.5-fp8-mi355x-sglang-agentic-mtp:
image: lmsysorg/sglang-rocm:v0.5.22-rocm720-mi35x-20261020
image: lmsysorg/sglang-rocm:v0.5.23-rocm720-mi35x-20261117
model: Qwen/Qwen3.5-397B-A17B-FP8
model-prefix: qwen3.5
runner: cluster:mi355x-amds
Expand Down
12 changes: 11 additions & 1 deletion inferencex-e2e/perf-changelog.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -12917,4 +12917,14 @@
- "Every point sets VLLM_K3_SP_PREFILL=1 (seungrokj/mum#26): prefill batches of at least 4096 tokens keep the attn-res residual stream token-sharded across TP, replacing each layer's three all-reduces with reduce-scatters and all-gathers over symmetric memory that sum the TP partials in FP32 in a fixed order."
- "Concurrency 1 and 4 (latency_specialized decode backend) pick up the seungrokj/mum#28 levers through their defaults (all 1): VLLM_K31_BEAT_MARKOV, VLLM_K31_BEAT_DRAFTMETA, VLLM_K31_BEAT_K2SYNC, VLLM_K31_BEAT_EAGERPREP, VLLM_K31_BEAT_KDAK1 and VLLM_K3_INDEXER_SAMPLED_TOPK, plus VLLM_KIMI_SD_DSA_PROLOGUE_INDEXER at concurrency 1, where VLLM_KIMI_SD_DSA_PROLOGUE is on. Concurrency 16-48 use the stock decode backend, which ignores these flags."
- "No change to layouts, draft token counts, capture sizes or the DSpark draft's data types."
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/5089
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/5089

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 Merging this rewrites a byte in an existing changelog entry, which the repo forbids. The changelog conflict tooling treats this file as append-only. The hunk strips two trailing spaces from the previous entry's pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/5089 line at perf-changelog.yaml:12920. AGENTS.md:88 says to preserve every existing byte and append only at the tail. Fix: restore the two trailing spaces on line 12920 exactly as on the base, and change only the lines after it to add the new block.

Why this was flagged

The diff removes from the end of line 12920 (pr-link: .../pull/5089), which belongs to the entry before this PR's new block. AGENTS.md:88 says perf-changelog.yaml is append-only and byte-sensitive, and conflicts must be resolved only with the changelog conflict helper. On the base branch that line ends with two spaces, so this PR edits a historical entry rather than only appending. Another open PR that also appends after the #5089 entry will then conflict on that line, and the append-only helper and checks that assume an unchanged prefix see a modified existing byte. No safeguard in the diff keeps the old bytes, because the editor's trailing-whitespace stripping touched the context line.

Verification: Trigger: any run of the repo's changelog tooling on this PR's diff. The hunk replaces pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/5089 $ with ...pull/5089$. infx/matrix/plan.py:62-69 raises ValueError("Deletions are not allowed in ...") for the removed non-blank line, so the PR sweep fails at validation and the post-merge ingest fails too.


- config-keys:
- qwen3.5-fp8-mi355x-sglang-agentic-mtp
scenario-type:
- agentic-coding
description:
- "Update the Qwen3.5-397B-A17B FP8 MI355X SGLang AgentX image from lmsysorg/sglang-rocm:v0.5.22-rocm720-mi35x-20261020 to lmsysorg/sglang-rocm:v0.5.23-rocm720-mi35x-20261117."
- "Add --enable-amd-monokernel. sgl_amd_mono.qwen35_a17b serves TP4 decode widths 1-8 and this recipe's EAGLE chain (eagle-topk 1, num-steps 3, num-draft-tokens 4). Wider steps keep SGLang's forward. The grid, HiCache sizes, and draft token counts are unchanged."
- "The ROCm image installs the sgl-amd-mono commit pinned in SGLang. sgl-project/sglang#50813 pins a04ceab, the merge of sgl-amd-mono#58, so qwen35_a17b is in the 20261117 tag when that pin is in the build."
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/5099
Loading