Repository navigation
[NV] Qwen3.8-Flash-Next FP4 GB300 SGLang AgentX update / [NV] Qwen3.8-Flash-Next FP4 GB300 SGLang AgentX 更新 - #4787
Conversation
Signed-off-by: hshrivastava-droid <hshrivastava@nvidia.com>
|
Thanks for the contribution!
中文感谢你的贡献!
|
Signed-off-by: hshrivastava-droid <hshrivastava@nvidia.com>
| description: | ||
| - "Update to lmsysorg/sglang:nightly-dev-cu13-20261103-797af717 and enable SGLANG_ENABLE_NV_MEGA_RUNTIME_OPTIMIZATION and breakable prefill CUDA graphs." | ||
| - "Replace the search space with TP4/EP4 at concurrency 3, TP1/EP1 at concurrency 8 and 20, and TP1/EP1 with HiCache at concurrency 36, 56 and 80." | ||
| pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/PRNUM |
There was a problem hiding this comment.
🔴 The changelog check will reject this PR because its new entry's pr-link ends in PRNUM, which is not an accepted placeholder. The validator accepts only the exact PR URL or an XXX placeholder (PR_LINK_PLACEHOLDERS in validate_perf_changelog.py:21). The merge-prep step will also refuse to rewrite the link. Fix: set pr-link to https://github.com/SemiAnalysisAI/InferenceX/pull/XXX or to this PR's real URL.
Why this was flagged
The appended entry at perf-changelog.yaml:12023 has pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/PRNUM. On PR runs, validate_added_pr_link (infx/workflows/validate_perf_changelog.py:105-116) raises ChangelogValidationError("new PR entry must use ... or an XXX placeholder") because PR_LINK_PLACEHOLDERS (line 21) holds only "XXX" and ".../pull/XXX". prepare_perf_changelog_merge.py:108-111 raises "appended entry N has unexpected pr-link" for the same reason, so the link is never rewritten to the real PR URL at merge. Without this entry, the changelog validation passes on the base branch.
Verification: The new entry in inferencex-e2e/perf-changelog.yaml:12023 reads pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/PRNUM. On every PR run, validate_added_pr_link raises because PRNUM is neither the PR URL nor in PR_LINK_PLACEHOLDERS (validate_perf_changelog.py:21-24). prepare_perf_changelog_merge.py also raises "appended entry N has unexpected pr-link".
| override_c36: | ||
| roles: | ||
| agg: |
There was a problem hiding this comment.
🟡 nit (optional): The new recipe comment includes past benchmark results, which the AGENTS.md comments rule does not allow in code. The phrase "they gave no gain at 36 and 56" records past results, and that history belongs in git and perf-changelog.yaml. The out-of-memory failure at concurrency 80 counts as a measured failure, so that part can stay. Fix: shorten the comment to the technical reason only, for example "Post-verify CUDA graphs captured lazily while serving exhaust GPU memory at high concurrency; compact verify sampling stays on."
Why this was flagged
The comment on the &hicache_env anchor (agentic.yaml:185-187) explains why the post-verify CUDA graph flags are turned off for override_c36, override_c56 and override_c80. Its clause "they gave no gain at 36 and 56" reports earlier benchmark outcomes. AGENTS.md (Comments, mandatory, every file) bans comments about past results and points to git and perf-changelog.yaml as the record. It allows only technical explanations, such as a measured failure. Nothing breaks at runtime. The cost is that the recipe goes against the repo's mandatory comment convention, and the comment will go stale once the image or settings change.
Verification: The diff adds this text at agentic.yaml:190-191, inside the &hicache_env anchor under override_c36. AGENTS.md:54-58 says: "No history: how code evolved, what it replaced, where it came from, or past results." The OOM at concurrency 80 is a measured failure, so that part is allowed. "they gave no gain at 36 and 56" reports an earlier benchmark outcome. Nothing breaks at runtime.
|
View unofficial run (performance): https://inferencex.semianalysis.com/inference?unofficialRun=42124076518 View unofficial run (accuracy): https://inferencex.semianalysis.com/evaluation?unofficialRun=42124076518 |
|
[by Claude Code] The SGLang cookbook integration of this PR's AgentX config ( This PR has no authorized |
|
[by Claude Code] sgl-project/sglang#48634 has been merged. This PR's AgentX config ( This PR has no authorized |
Signed-off-by: hshrivastava-droid <hshrivastava@nvidia.com>
…ng-agentx-public # Conflicts: # inferencex-e2e/perf-changelog.yaml
…ng-agentx-public Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
|
[by Claude Code] The SGLang cookbook update of this PR's AgentX config ( This PR has no authorized |
|
[by Claude Code] sgl-project/sglang#48857 has been merged. This PR's AgentX config ( This PR has no authorized |
|
/use 42124076518 |
|
@hshrivastava-droid staged run 42124076518. This run remains available across future |
|
As a PR reviewer and CODEOWNER, I have reviewed this and have: Checklist items
Additional detail section:
中文
Prepared with assistance from Codex. Signed: |
✅✅✅ Verdict: PASS ✅✅✅Passed and not applicable checks✅ Check 0 (CODEOWNER): PASS — ✅ Check 1 (In-PR sweep + evals): PASS — Pinned head ✅ Check 2 (Eval accuracy): PASS — The gsm8k evals ran on ✅ Check 3 (Upstream recipe): PASS — sgl-project/sglang#48857 is MERGED (2026-11-06T08:53Z). Its ✅ Check 4 (Reuse command): PASS — COLLABORATOR ✅ Check 5 (Latest checklist): PASS — All 19 items of the current template, plus the upstream-PR sub-item, are present and checked. ✅ Check 6 (Upstream images + ordering + provenance): PASS — The only image is ✅ Check 7 (Deprecated models): PASS — ✅ Check 8 (Benchmark hacks): PASS — There are no ✅ Check 9 (Chat templates): PASS — The AgentX client replays chat conversation traces through ✅ Check 10 (Engine patches): PASS — No patch, setup script, wheel install or source overlay is reachable from the pinned recipe or its launch path. ✅ Check 11 (Golden AL): PASS — The benchmark logs show ➖ Check 12 (Append-only): N/A — The new changelog entry does not use ✅ Check 13 (Draft precision): PASS — The embedded MTP head is from checkpoint ✅ Check 14 (Pareto coverage): PASS — Curve qwen3.8next / agentic-coding / gb300 SGLang / fp4 / p90 E2EL, run 42124076518 attempt 3, artifact ✅ Check 15 (PR description): PASS — The config key and image ✅ Check 16 (NVIDIA overview): PASS — In the NVIDIA overview Framework Recipes table, Qwen3.8-Flash-Next has an SGLang entry linking the cookbook page; this covers GB300. ➖ Check 17 (Metrics publication): N/A — This is a direct SGLang submission; no Dynamo TRT-LLM serving path is affected. ✅ Check 18 (Recipe/image/sweep consistency): PASS — (a) No head generation errors. (b) Master TP4/EP4 c3, TP1 c8/c20 (no KV offload) and TP1 HiCache c36/c56/c80 match the recipe overrides' gpus, TP/EP and Assessed commit: |
…ng-agentx-public Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
|
Ingest report — ✅ Results ingested into the Ingest workflow · Merge Ingest run · Benchmark source run
benchmarkFiles 7, rawBenchmarkRows 12, coveredByPerJobFiles 6, purgedPoints 0, failedRunRows 0, runStatsNew 0, runStatsDuplicate 0, evalSamplesNew 6595, gpuMetricSamplesNew 45507, powerPublicationPoints 6, changelogEntriesWritten 1 Counts cover only this run. Rows: benchmark_results, eval_results, agentic rows linked to a trace, gpu_metrics series. This comment updates if the ingest is rerun. 中文Ingest 报告 — ✅ 结果已写入 上表仅统计本次运行写入的数据,各行单位依次为 benchmark_results 行、eval_results 行、已关联 trace 的 agentic 行和 gpu_metrics 序列。重新运行 ingest 时本评论会原地更新。 |
Update the Qwen3.8-Flash-Next FP4 GB300 SGLang AgentX config (
qwen3.8next-fp4-gb300-sglang-agentic-mtp).lmsysorg/sglang:nightly-dev-cu13-20261103-797af717AI model disclosure
PR checklist and change type
Type of Change
Checklist
python -m infx.matrix.changed --base origin/mainlists affected config keys, I have appended one new entry covering them to the physical end ofinferencex-e2e/perf-changelog.yaml; if it lists none, I have added none. I have not edited historical entriesinferencex-e2e/perf-changelog.yamlentry, it carries exactly one primary sweep label:full-sweep-enabled/use <run_id>on this PR中文
更新 Qwen3.8-Flash-Next FP4 GB300 SGLang AgentX 配置(
qwen3.8next-fp4-gb300-sglang-agentic-mtp)。镜像同上。🤖 Generated with Claude Code