Repository navigation
perf(nvidia): GLM-5.3 NVFP4 B300 SGLang AgentX: FlashMLA sparse prefill + TRT-LLM sparse decode / perf(nvidia):GLM-5.3 NVFP4 B300 SGLang AgentX:FlashMLA 稀疏预填充 + TRT-LLM 稀疏解码 - #5115
Conversation
… + trtllm decode NSA backends
|
Thanks for the contribution!
中文感谢你的贡献!
|
| description: | ||
| - "Pin the DSA sparse-attention backends instead of auto selection: nsa-prefill-backend flashmla_sparse and nsa-decode-backend trtllm." | ||
| - "MoE stays on flashinfer_trtllm; image, search space and every other flag are unchanged." | ||
| pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/XXXX |
There was a problem hiding this comment.
🔴 The new changelog entry links to a placeholder PR, so dashboard readers following it reach no real pull request. Line 12975 has pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/XXXX. Because perf-changelog.yaml is append-only, a bad link merged now cannot be corrected later. Fix: replace XXXX with this PR's real number before merge.
Why this was flagged
The block appended at inferencex-e2e/perf-changelog.yaml:12966-12975 sets pr-link to the literal https://github.com/SemiAnalysisAI/InferenceX/pull/XXXX. AGENTS.md and the conventions treat perf-changelog.yaml as append-only and byte-sensitive, so existing entries cannot be edited afterwards. Every other entry in the file points at a real PR, for example #5029 at line 12966. If this merges unchanged, the glm5.3-fp4-b300-sglang-agentic-mtp result has no working link back to the change that produced it.
Verification: Trigger: every PR run of the check-changelog job. The block appended at inferencex-e2e/perf-changelog.yaml:12975 has pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/XXXX, with four X's. So the four-X link fails changelog validation, which blocks the PR's sweep and /use flow.
| # Pin the DSA sparse-attention kernels instead of auto selection: | ||
| # FlashMLA sparse for prefill, TRT-LLM sparse MLA for decode. |
There was a problem hiding this comment.
🟡 nit (optional): The new comment breaks AGENTS.md comment rules, so maintainers get history and a restatement of the code instead of a reason. "instead of auto selection" says what the line replaced, and the second line just repeats the two backend values. Fix: replace both lines with one short reason for the pin, such as the measured gain or the auto-selection problem it avoids. Otherwise delete the comment and leave the history to perf-changelog.yaml.
Why this was flagged
The diff adds agentic.yaml:29-30: "# Pin the DSA sparse-attention kernels instead of auto selection:" and "# FlashMLA sparse for prefill, TRT-LLM sparse MLA for decode." AGENTS.md:57 bans history comments that say what code replaced. AGENTS.md:58 says to give only why a value was chosen and never to restate the code. AGENTS.md:50 also warns against replacing a documented automatic selection with a constant unless it is justified, yet the comment gives no reason for the pin.
Verification: The diff adds these lines at agentic.yaml:29-30: "# Pin the DSA sparse-attention kernels instead of auto selection:" and "# FlashMLA sparse for prefill, TRT-LLM sparse MLA for decode." AGENTS.md:57 bans history comments, including "what it replaced". AGENTS.md:58 allows only "why a value was chosen, or context the code cannot show" and says "never restate the code". Nothing breaks at runtime.
|
View unofficial run (performance): https://inferencex.semianalysis.com/inference?unofficialRun=44069770534 View unofficial run (accuracy): https://inferencex.semianalysis.com/evaluation?unofficialRun=44069770534 |
Pin the DSA sparse-attention backends on
glm5.3-fp4-b300-sglang-agentic-mtpinstead of leaving attention on auto:nsa-prefill-backend: flashmla_sparsensa-decode-backend: trtllmMoE stays on
flashinfer_trtllm; image, search space and all other flags unchanged.固定 GLM-5.3 B300 的 DSA 稀疏注意力后端:预填充使用 flashmla_sparse,解码使用 trtllm;其余配置不变。
Suggested by Tejas Bhakta (@bhaktatejas922).