Skip to content

perf(nvidia): GLM-5.3 NVFP4 B300 SGLang AgentX: FlashMLA sparse prefill + TRT-LLM sparse decode / perf(nvidia):GLM-5.3 NVFP4 B300 SGLang AgentX:FlashMLA 稀疏预填充 + TRT-LLM 稀疏解码 - #5115

Open
Oseltamivir wants to merge 4 commits into
mainfrom
tejas/glm53-b300-nsa-flashmla-trtllm
Open

Oseltamivir wants to merge 4 commits into
mainfrom
tejas/glm53-b300-nsa-flashmla-trtllm

Conversation

@Oseltamivir

Copy link
Copy Markdown
Collaborator

Pin the DSA sparse-attention backends on glm5.3-fp4-b300-sglang-agentic-mtp instead of leaving attention on auto:

  • nsa-prefill-backend: flashmla_sparse
  • nsa-decode-backend: trtllm

MoE stays on flashinfer_trtllm; image, search space and all other flags unchanged.

固定 GLM-5.3 B300 的 DSA 稀疏注意力后端:预填充使用 flashmla_sparse,解码使用 trtllm;其余配置不变。

Suggested by Tejas Bhakta (@bhaktatejas922).

@github-actions

Copy link
Copy Markdown
Contributor

Thanks for the contribution!

  • Review: If this PR changes files owned by someone other than a repository admin or @SemiAnalysisAI/core, ask one eligible CODEOWNER to complete the latest PR_REVIEW_CHECKLIST.md before contacting a core maintainer on Slack. Follow the template exactly, including As a PR reviewer and CODEOWNER, I have reviewed this and have, so sign-off verification triggers.
  • PR verification: Sweeps run only when the PR appends an inferencex-e2e/perf-changelog.yaml entry and carries exactly one primary label: full-sweep-fail-fast (strongly recommended; canary plus per-matrix fail-fast), full-sweep-enabled (canary; matrix jobs continue after a failure), or non-canary-full-sweep-enabled (no canary or fail-fast). The modifiers all-evals, evals-only, and agentx-fast require a primary label. On fork PRs, a maintainer applies the label. See sweep labels and reuse.
  • After merging: PR authors must ensure all GitHub Actions jobs pass. Transient failures often pass on rerun; see how to rerun failed jobs.
中文

感谢你的贡献!

  • **审阅:**如果 PR 修改的文件归属于仓库管理员及 @SemiAnalysisAI/core 之外的 CODEOWNER,请先联系一位有资格的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md,再通过 Slack 联系核心维护者。必须严格遵循模板,并保留 As a PR reviewer and CODEOWNER, I have reviewed this and have,才能触发签核验证。
  • **PR 验证:**只有当 PR 在 inferencex-e2e/perf-changelog.yaml 末尾追加了条目,并且恰好带有一个主标签时,才会运行扫描:full-sweep-fail-fast(强烈推荐;canary 加逐矩阵 fail-fast)、full-sweep-enabled(有 canary;矩阵任务在失败后继续运行)或 non-canary-full-sweep-enabled(无 canary,也无 fail-fast)。修饰标签 all-evals、evals-only 和 agentx-fast 必须与主标签一起使用。fork PR 的标签由维护者添加。参见扫描标签与复用。
  • **合并后:**PR 作者必须确保所有 GitHub Actions 任务通过。临时性失败通常可以通过重新运行恢复;参见重新运行失败任务的说明。

@Oseltamivir Oseltamivir added the full-sweep-fail-fast Full sweep with canary gate; first failure cancels the rest of that matrix (recommended) label Nov 17, 2026

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Findings marked 🟡 are optional suggestions and need no follow-up push.

Comment thread inferencex-e2e/perf-changelog.yaml Outdated
description:
- "Pin the DSA sparse-attention backends instead of auto selection: nsa-prefill-backend flashmla_sparse and nsa-decode-backend trtllm."
- "MoE stays on flashinfer_trtllm; image, search space and every other flag are unchanged."
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/XXXX

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 The new changelog entry links to a placeholder PR, so dashboard readers following it reach no real pull request. Line 12975 has pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/XXXX. Because perf-changelog.yaml is append-only, a bad link merged now cannot be corrected later. Fix: replace XXXX with this PR's real number before merge.

Why this was flagged

The block appended at inferencex-e2e/perf-changelog.yaml:12966-12975 sets pr-link to the literal https://github.com/SemiAnalysisAI/InferenceX/pull/XXXX. AGENTS.md and the conventions treat perf-changelog.yaml as append-only and byte-sensitive, so existing entries cannot be edited afterwards. Every other entry in the file points at a real PR, for example #5029 at line 12966. If this merges unchanged, the glm5.3-fp4-b300-sglang-agentic-mtp result has no working link back to the change that produced it.

Verification: Trigger: every PR run of the check-changelog job. The block appended at inferencex-e2e/perf-changelog.yaml:12975 has pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/XXXX, with four X's. So the four-X link fails changelog validation, which blocks the PR's sweep and /use flow.

Comment on lines +29 to +30
# Pin the DSA sparse-attention kernels instead of auto selection:
# FlashMLA sparse for prefill, TRT-LLM sparse MLA for decode.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 nit (optional): The new comment breaks AGENTS.md comment rules, so maintainers get history and a restatement of the code instead of a reason. "instead of auto selection" says what the line replaced, and the second line just repeats the two backend values. Fix: replace both lines with one short reason for the pin, such as the measured gain or the auto-selection problem it avoids. Otherwise delete the comment and leave the history to perf-changelog.yaml.

Why this was flagged

The diff adds agentic.yaml:29-30: "# Pin the DSA sparse-attention kernels instead of auto selection:" and "# FlashMLA sparse for prefill, TRT-LLM sparse MLA for decode." AGENTS.md:57 bans history comments that say what code replaced. AGENTS.md:58 says to give only why a value was chosen and never to restate the code. AGENTS.md:50 also warns against replacing a documented automatic selection with a constant unless it is justified, yet the comment gives no reason for the pin.

Verification: The diff adds these lines at agentic.yaml:29-30: "# Pin the DSA sparse-attention kernels instead of auto selection:" and "# FlashMLA sparse for prefill, TRT-LLM sparse MLA for decode." AGENTS.md:57 bans history comments, including "what it replaced". AGENTS.md:58 allows only "why a value was chosen, or context the code cannot show" and says "never restate the code". Nothing breaks at runtime.

@Oseltamivir Oseltamivir added non-canary-full-sweep-enabled Run the full sweep without the canary gate (full search space, no trim) and removed full-sweep-fail-fast Full sweep with canary gate; first failure cancels the rest of that matrix (recommended) labels Nov 17, 2026
@github-actions

github-actions Bot commented Nov 17, 2026 •

Copy link
Copy Markdown
Contributor

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

non-canary-full-sweep-enabled Run the full sweep without the canary gate (full search space, no trim)

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

2 participants