Skip to content

[NV] Qwen3.8-Flash-Next FP4 GB300 SGLang AgentX update / [NV] Qwen3.8-Flash-Next FP4 GB300 SGLang AgentX 更新 - #4787

Merged
hshrivastava-droid merged 8 commits into
mainfrom
qwen38next-gb300-sglang-agentx-public
Nov 6, 2026
Merged

hshrivastava-droid merged 8 commits into
mainfrom
qwen38next-gb300-sglang-agentx-public

Conversation

@hshrivastava-droid

Copy link
Copy Markdown
Collaborator

Update the Qwen3.8-Flash-Next FP4 GB300 SGLang AgentX config (qwen3.8next-fp4-gb300-sglang-agentic-mtp).

  • Image: lmsysorg/sglang:nightly-dev-cu13-20261103-797af717
AI model disclosure
  • Model/version: claude-sonnet-5-5
  • Role: ported the config to this repository
PR checklist and change type

Type of Change

  • Bug fix
  • New feature
  • Configuration change
  • Documentation update
  • Other (please describe)

Checklist

  • I have completed the AI model disclosure and kept it current
  • I have tested my changes locally
  • I have updated documentation if necessary
  • If python -m infx.matrix.changed --base origin/main lists affected config keys, I have appended one new entry covering them to the physical end of inferencex-e2e/perf-changelog.yaml; if it lists none, I have added none. I have not edited historical entries
  • If this PR appends an inferencex-e2e/perf-changelog.yaml entry, it carries exactly one primary sweep label: full-sweep-enabled
  • Before merging via reuse, an authorized maintainer has commented /use <run_id> on this PR
中文

更新 Qwen3.8-Flash-Next FP4 GB300 SGLang AgentX 配置(qwen3.8next-fp4-gb300-sglang-agentic-mtp)。镜像同上。

🤖 Generated with Claude Code

Signed-off-by: hshrivastava-droid <hshrivastava@nvidia.com>
@github-actions

github-actions Bot commented Nov 4, 2026

Copy link
Copy Markdown
Contributor

Thanks for the contribution!

  • Review: If this PR changes files owned by someone other than a repository admin or @SemiAnalysisAI/core, ask one eligible CODEOWNER to complete the latest PR_REVIEW_CHECKLIST.md before contacting a core maintainer on Slack. Follow the template exactly, including As a PR reviewer and CODEOWNER, I have reviewed this and have, so sign-off verification triggers.
  • PR verification: Sweeps run only when the PR appends an inferencex-e2e/perf-changelog.yaml entry and carries exactly one primary label: full-sweep-fail-fast (strongly recommended; canary plus per-matrix fail-fast), full-sweep-enabled (canary; matrix jobs continue after a failure), or non-canary-full-sweep-enabled (no canary or fail-fast). The modifiers all-evals, evals-only, and agentx-fast require a primary label. On fork PRs, a maintainer applies the label. See sweep labels and reuse.
  • After merging: PR authors must ensure all GitHub Actions jobs pass. Transient failures often pass on rerun; see how to rerun failed jobs.
中文

感谢你的贡献!

  • **审阅:**如果 PR 修改的文件归属于仓库管理员及 @SemiAnalysisAI/core 之外的 CODEOWNER,请先联系一位有资格的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md,再通过 Slack 联系核心维护者。必须严格遵循模板,并保留 As a PR reviewer and CODEOWNER, I have reviewed this and have,才能触发签核验证。
  • **PR 验证:**只有当 PR 在 inferencex-e2e/perf-changelog.yaml 末尾追加了条目,并且恰好带有一个主标签时,才会运行扫描:full-sweep-fail-fast(强烈推荐;canary 加逐矩阵 fail-fast)、full-sweep-enabled(有 canary;矩阵任务在失败后继续运行)或 non-canary-full-sweep-enabled(无 canary,也无 fail-fast)。修饰标签 all-evals、evals-only 和 agentx-fast 必须与主标签一起使用。fork PR 的标签由维护者添加。参见扫描标签与复用。
  • **合并后:**PR 作者必须确保所有 GitHub Actions 任务通过。临时性失败通常可以通过重新运行恢复;参见重新运行失败任务的说明。

Signed-off-by: hshrivastava-droid <hshrivastava@nvidia.com>
@hshrivastava-droid hshrivastava-droid added the full-sweep-enabled Full sweep with canary gate; matrix jobs run to completion despite failures label Nov 4, 2026

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Findings marked 🟡 are optional suggestions and need no follow-up push.

Comment thread inferencex-e2e/perf-changelog.yaml Outdated
description:
- "Update to lmsysorg/sglang:nightly-dev-cu13-20261103-797af717 and enable SGLANG_ENABLE_NV_MEGA_RUNTIME_OPTIMIZATION and breakable prefill CUDA graphs."
- "Replace the search space with TP4/EP4 at concurrency 3, TP1/EP1 at concurrency 8 and 20, and TP1/EP1 with HiCache at concurrency 36, 56 and 80."
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/PRNUM

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 The changelog check will reject this PR because its new entry's pr-link ends in PRNUM, which is not an accepted placeholder. The validator accepts only the exact PR URL or an XXX placeholder (PR_LINK_PLACEHOLDERS in validate_perf_changelog.py:21). The merge-prep step will also refuse to rewrite the link. Fix: set pr-link to https://github.com/SemiAnalysisAI/InferenceX/pull/XXX or to this PR's real URL.

Why this was flagged

The appended entry at perf-changelog.yaml:12023 has pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/PRNUM. On PR runs, validate_added_pr_link (infx/workflows/validate_perf_changelog.py:105-116) raises ChangelogValidationError("new PR entry must use ... or an XXX placeholder") because PR_LINK_PLACEHOLDERS (line 21) holds only "XXX" and ".../pull/XXX". prepare_perf_changelog_merge.py:108-111 raises "appended entry N has unexpected pr-link" for the same reason, so the link is never rewritten to the real PR URL at merge. Without this entry, the changelog validation passes on the base branch.

Verification: The new entry in inferencex-e2e/perf-changelog.yaml:12023 reads pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/PRNUM. On every PR run, validate_added_pr_link raises because PRNUM is neither the PR URL nor in PR_LINK_PLACEHOLDERS (validate_perf_changelog.py:21-24). prepare_perf_changelog_merge.py also raises "appended entry N has unexpected pr-link".

Comment on lines +185 to +187
override_c36:
roles:
agg:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 nit (optional): The new recipe comment includes past benchmark results, which the AGENTS.md comments rule does not allow in code. The phrase "they gave no gain at 36 and 56" records past results, and that history belongs in git and perf-changelog.yaml. The out-of-memory failure at concurrency 80 counts as a measured failure, so that part can stay. Fix: shorten the comment to the technical reason only, for example "Post-verify CUDA graphs captured lazily while serving exhaust GPU memory at high concurrency; compact verify sampling stays on."

Why this was flagged

The comment on the &hicache_env anchor (agentic.yaml:185-187) explains why the post-verify CUDA graph flags are turned off for override_c36, override_c56 and override_c80. Its clause "they gave no gain at 36 and 56" reports earlier benchmark outcomes. AGENTS.md (Comments, mandatory, every file) bans comments about past results and points to git and perf-changelog.yaml as the record. It allows only technical explanations, such as a measured failure. Nothing breaks at runtime. The cost is that the recipe goes against the repo's mandatory comment convention, and the comment will go stale once the image or settings change.

Verification: The diff adds this text at agentic.yaml:190-191, inside the &hicache_env anchor under override_c36. AGENTS.md:54-58 says: "No history: how code evolved, what it replaced, where it came from, or past results." The OOM at concurrency 80 is a measured failure, so that part is allowed. "they gave no gain at 36 and 56" reports an earlier benchmark outcome. Nothing breaks at runtime.

@github-actions

github-actions Bot commented Nov 4, 2026 •

Copy link
Copy Markdown
Contributor

@nvpohanh

nvpohanh commented Nov 5, 2026

Copy link
Copy Markdown
Collaborator

[by Claude Code] The SGLang cookbook integration of this PR's AgentX config (qwen3.8next-fp4-gb300-sglang-agentic-mtp) will happen in sgl-project/sglang#48634, in the Agentic Long-Context Deployment section of the Qwen3.8-Flash-Next cookbook page.

This PR has no authorized /use <run_id> comment yet, so the cookbook shows the recipe at this PR's head (5178cd363) as "in progress". A later cookbook PR switches it to the measured run once a run is pinned or this PR merges.

@nvpohanh

nvpohanh commented Nov 5, 2026

Copy link
Copy Markdown
Collaborator

[by Claude Code] sgl-project/sglang#48634 has been merged. This PR's AgentX config (qwen3.8next-fp4-gb300-sglang-agentic-mtp) is now in the Agentic Long-Context Deployment section of the SGLang Qwen3.8-Flash-Next cookbook: https://docs.sglang.io/cookbook/autoregressive/Qwen/Qwen3.8-Flash-Next#agentic-long-context-deployment

This PR has no authorized /use <run_id> comment yet, so the cookbook shows the recipe at this PR's head (5178cd363) as "in progress". A later cookbook PR switches it to the measured run once a run is pinned or this PR merges.

Signed-off-by: hshrivastava-droid <hshrivastava@nvidia.com>
…ng-agentx-public

# Conflicts:
#	inferencex-e2e/perf-changelog.yaml
@Ankur-singh Ankur-singh added full-sweep-enabled Full sweep with canary gate; matrix jobs run to completion despite failures and removed full-sweep-enabled Full sweep with canary gate; matrix jobs run to completion despite failures labels Nov 5, 2026
…ng-agentx-public

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@SemiAnalysisAI SemiAnalysisAI deleted a comment from github-actions Bot Nov 5, 2026
@hshrivastava-droid hshrivastava-droid added full-sweep-enabled Full sweep with canary gate; matrix jobs run to completion despite failures and removed full-sweep-enabled Full sweep with canary gate; matrix jobs run to completion despite failures labels Nov 5, 2026
@nvpohanh

nvpohanh commented Nov 6, 2026

Copy link
Copy Markdown
Collaborator

[by Claude Code] The SGLang cookbook update of this PR's AgentX config (qwen3.8next-fp4-gb300-sglang-agentic-mtp) will happen in sgl-project/sglang#48857, in the Agentic Long-Context Deployment section of the Qwen3.8-Flash-Next cookbook page. The config has been live in that section since sgl-project/sglang#48634; sgl-project/sglang#48857 refreshes it with the pin below.

This PR has no authorized /use <run_id> comment yet, so the cookbook shows the recipe at this PR's head (1aa39c5bd) as "in progress". A later cookbook PR switches it to the measured run once a run is pinned or this PR merges.

@nvpohanh

nvpohanh commented Nov 6, 2026

Copy link
Copy Markdown
Collaborator

[by Claude Code] sgl-project/sglang#48857 has been merged. This PR's AgentX config (qwen3.8next-fp4-gb300-sglang-agentic-mtp) is now in the Agentic Long-Context Deployment section of the SGLang Qwen3.8-Flash-Next cookbook: https://docs.sglang.io/cookbook/autoregressive/Qwen/Qwen3.8-Flash-Next#agentic-long-context-deployment

This PR has no authorized /use <run_id> comment yet, so the cookbook shows the recipe at this PR's head (1aa39c5bd) as "in progress". A later cookbook PR switches it to the measured run once a run is pinned or this PR merges.

@SemiAnalysisAI SemiAnalysisAI deleted a comment from github-actions Bot Nov 6, 2026
@SemiAnalysisAI SemiAnalysisAI deleted a comment from github-actions Bot Nov 6, 2026
@SemiAnalysisAI SemiAnalysisAI deleted a comment from github-actions Bot Nov 6, 2026
@hshrivastava-droid

Copy link
Copy Markdown
Collaborator Author

/use 42124076518

@github-actions

github-actions Bot commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

@hshrivastava-droid staged run 42124076518.

This run remains available across future /stage-results requests. Staging the same run ID again updates its staged data. Staging workflow

@SemiAnalysisAI SemiAnalysisAI deleted a comment from github-actions Bot Nov 6, 2026
@RohitNagraj

Copy link
Copy Markdown
Collaborator

As a PR reviewer and CODEOWNER, I have reviewed this and have:

Checklist items
  • Verified that as of the moment of typing this, this is the latest version of PR_REVIEW_CHECKLIST.md
  • Verified that the general code quality meets the InferenceX standard and does not make the code quality any worse.
  • Verified that this PR has passed PR validation. Please link to GitHub Action workflow that shows this.
  • Verified that this PR passes evals. Please link to GitHub Action workflow that shows this.
  • Verified that speculative decoding PRs uses chat templates to align the AL distribution to real world
  • Verified that every draft model and draft head is served as it ships: the draft that ships with the served checkpoint, at its stored precision, through the pinned upstream image's default handling, with the shipped and effective draft precision recorded in the additional detail section. No submission-side quantization, dtype override, checkpoint substitution, or patch may lower draft precision below that default, regardless of eval results or AL. Explicitly verified that SGLANG_NVFP4_CKPT_FP8_NEXTN_MOE is not enabled in the effective recipe, including inherited settings; enabling it is prohibited going forward, and historical runs do not grant an exception. See Draft-model precision for what counts as the default and the MLPerf comparison.
  • For agentic workloads: verified that speculative-decoding configs (EAGLE / MTP / draft models) run with simulated synthetic acceptance, with the acceptance-length value taken from the committed golden AL curve in infx/golden_al_distribution/ for that model, thinking mode, and draft length. A submission may choose any supported draft length, but it may not substitute a different acceptance target.
  • Verified against the current MODELS.md that this PR does not submit a deprecated model, scenario, or model-scenario combination.
  • Verified that the model architecture isn't changed with benchmark hacks like using --hf-overrides to skipping indexer for every x layers on models that don't natively support this. As a general rule, we won't accept optimizations that reduces the number of model architecture FLOPs. Anything that makes that same computation run faster is fair game; target/verifier FLOPs at lower precisions is fine, given that the config passes private evals, but this does not permit lowering draft-model or draft-head precision below what ships. As an general north star princple, we should only use optimizations which is used in production by customers that care about accuracy
  • Verified that new models disable integer all-reduce, with SGLang ROCm explicitly setting ROCM_QUICK_REDUCE_QUANTIZATION=NONE. Existing models are grandfathered per MODELS.md.
  • For affected Dynamo TRT-LLM benchmarks, verified that iteration/KV-cache metrics are collected and published under the pinned Dynamo implementation and effective engine settings. Resolve the installed Dynamo version separately from the container tag; use supported metrics controls without requiring KV events. Record the version and configuration evidence in the additional detail section. See verifier Check 17. N/A for unaffected serving paths or frameworks without these Dynamo settings.
  • If an company claims that they support vLLM/SGLang as first class LLM inference engines on their hardware, I have verified that the respective vLLM submission made using upstream https://hub.docker.com/u/vllm docker repo, upstream SGLang https://hub.docker.com/u/lmsysorg docker repo. The only exceptions are for new hardware, such as MI455X UALoE72, Vera Rubin NVL72, Rubin NVL8, etc., and for new model architectures where there is an actual reason why vLLM/SGLang does not fundamentally support them yet as supported by vLLM/SGLang community maintainers
  • For official InferenceX (non-preview), verified that established models use upstream engine images built from master (or the equivalent default development branch), with new models transitioning within a reasonable time after release. See MODELS.md.
  • If an company claims that they support vLLM/SGLang as first class upstream in-tree LLM inference engines on their hardware, I have have verified that the respective vLLM/SGLang submission has been made before additional frameworks (TRT-LLM, ATOM, etc.). The only exceptions are for new hardware, such as MI455X UALoE72, Vera Rubin NVL72, Rubin NVL8, etc., and for new model architectures where there is an actual reason why vLLM/SGLang does not fundamentally support them yet.
  • Verified that every affected vLLM/SGLang production serving configuration in this PR, including Dynamo deployments, is documented in the official vLLM recipes and/or the SGLang cookbook, for both single-node and multi-node aggregated and disaggregated serving. The documentation covers production-useful flags, environment variables, and launch commands (including prefill, decode, and router/frontend commands for disaggregation); InferenceX-only benchmark/harness settings are excluded with reasoning in the additional detail section:
    • I linked the corresponding upstream PR in the vLLM recipe repo or SGLang cookbook source repo and verified that it is MERGED before this InferenceX PR merges. An opened, draft, or closed-without-merge upstream PR does not satisfy this requirement. If the matching recipe was already published, I linked the published recipe/cookbook page in the additional detail section below.
  • Verified that this PR does not patch the inference engine or serving stack — the pinned image must run as shipped. This covers .patch files / git apply / patch, inline patches embedded in benchmark scripts (e.g. a python3/sed heredoc that rewrites installed engine sources before serving), in-place edits of site-packages, monkey-patching, overwriting container files, and installing forked/rebuilt engine wheels on top of the pinned image. The only exception is a patch covered by a filled-out waiver at docs/waiver/<PR_NUMBER>.md — named after the PR that introduces the patch and filed in that same PR, stating what is patched, why the unmodified upstream image cannot run this benchmark, the upstream PR/issue link, and the removal plan — which I have linked below in the additional detail section.
  • If this PR uses append-only: true, verified that it only adds generated points or recipe variants inside a selected existing config/scenario and existing same-image visual curve: every previously generated point remains present with the same recipe, no prior point is removed or rerun, and every benchmark-affecting change in the complete diff can affect only the corresponding newly appended points (never an existing point), regardless of which file contains it.
  • If any of the above criteria cannot reasonably be satisfied, I have provided additional reasoning below.
  • Reported measured throughput/E2EL Pareto counts and evidence per affected curve (≥5 points strongly recommended). Below 5 or unverifiable: tag a core maintainer for review; recorded admin bypass required before merge. N/A if no curves are affected. Details.

Additional detail section:

  • Validation and evals: CI Tests and Lint passed on head 1aa39c5bd55c4c13de0a5ab6190a80f4781e63bd. The public sweep and AgentX evals passed for this head. All benchmark points and four eval representatives succeeded in attempt 3; the remaining eval representative succeeded in attempt 5. Eval worker logs show FP8 QSA cache initialization, graph captures and CUDA graph execution during eval requests. Direct eval-time dispatch of the changed conditional routing and QSA kernels remains unverified.
  • Model and architecture: Qwen3.8-Flash-Next Agentic is active in MODELS.md. The configuration does not change the model architecture or skip layers. Integer all-reduce: this existing model is grandfathered under the policy.
  • Draft precision: The pinned checkpoint stores MTP routed-expert weights in FP8 E4M3 with BF16 scales and other MTP tensors in BF16. The effective recipe sets SGLANG_NVFP4_CKPT_FP8_NEXTN_MOE=0. The eval worker reports an FP8 QSA indexer cache and MTP index sharing; the pinned indexer code casts index queries to the cache dtype, while the auto default is BF16. The submitted FP8 setting applies to the QSA indexer cache and its query computation; it does not change the stored draft weights. The policy treatment of the draft indexer query cast relative to the BF16 default remains a separate review point.
  • Chat templates and acceptance: The AgentX benchmark uses chat completions. The committed golden curve records chat-template collection with thinking enabled. The benchmark configurations use the matching synthetic acceptance for their effective draft lengths; evals use real verification.
  • Engine and patches: This submission uses Qwen3.8-Flash-Next's primary engine, SGLang, with an upstream lmsysorg/sglang image whose source commit is an ancestor of the upstream development branch. The effective recipe, shared AgentX block and traced launch path do not patch the engine or replace its wheel.
  • Production recipe: The SGLang cookbook covers the GB300 configurations in its Agentic Long-Context Deployment section. The corresponding cookbook refresh, which references this head, is merged. The same model and hardware's Deployment panel documents --enable-metrics, --enable-cache-report and --decode-log-interval. That panel references an earlier image; the Agentic section covers the image used here. Result collection and synthetic benchmark controls are harness-only settings.
  • Pareto coverage: The Qwen3.8-Flash-Next FP4 GB300 SGLang AgentX curve has six measured points on the throughput/p90-E2EL Pareto frontier in the public sweep, on head 1aa39c5bd55c4c13de0a5ab6190a80f4781e63bd, attempt 3. Run visualization.
  • Reuse and applicability: No authorized /use comment is currently present; an authorized maintainer must select the run before a reuse merge. Dynamo TRT-LLM metrics do not apply to this direct SGLang submission. This PR does not use append-only: true. The PR currently has a merge conflict.
中文
  • **验证与评测:**当前 head 的 CI 测试与代码检查已通过;该版本的公开扫描与 AgentX 评测已通过。所有基准点及四个评测代表在第 3 次尝试中成功,其余评测代表在第 5 次尝试中成功。评测日志显示 FP8 QSA 缓存初始化、图捕获以及评测请求期间 CUDA 图的执行;变更后的条件路由与 QSA 内核是否实际调用仍未直接验证。
  • **模型与架构:**Qwen3.8-Flash-Next Agentic 在 MODELS.md 中处于启用状态。该配置未更改模型架构或跳过模型层;整数全归约政策对这一已有模型适用既有豁免。
  • 草稿精度:固定版本的检查点以 FP8 E4M3 存储 MTP 路由专家权重、以 BF16 存储缩放参数及其他 MTP 张量。生效配置设置了 SGLANG_NVFP4_CKPT_FP8_NEXTN_MOE=0。评测 worker 报告了 FP8 QSA 索引缓存及 MTP 索引共享;固定版本的索引器代码将索引查询转换为缓存精度,而自动默认精度为 BF16。提交的 FP8 设置作用于 QSA 索引缓存及其查询计算,不改变存储的草稿权重。与 BF16 默认值相比,草稿索引器查询转换的政策适用性仍需单独审查。
  • **聊天模板与接受长度:**AgentX 基准使用 chat completions。提交的黄金曲线记录了启用 thinking 的聊天模板采集方式。基准配置按实际草稿长度使用对应的模拟接受长度;评测使用真实验证。
  • **引擎与补丁:**本次提交使用该模型的主引擎 SGLang,以及源代码提交位于上游开发分支历史中的 lmsysorg/sglang 镜像。生效配置、共享 AgentX 配置和已追踪的启动路径均未修补引擎或替换其 wheel。
  • 生产配方:SGLang cookbook 的 Agentic Long-Context Deployment 章节覆盖 GB300 配置;引用当前 head 的 cookbook 更新已合并。同一模型和硬件的 Deployment 面板记录了 --enable-metrics、--enable-cache-report 和 --decode-log-interval。该面板引用较早的镜像,Agentic 章节覆盖本次使用的镜像。结果收集和模拟基准控制属于测试框架设置。
  • **Pareto 覆盖:**当前 head 的公开扫描中,Qwen3.8-Flash-Next FP4 GB300 SGLang AgentX 曲线有六个实测点位于吞吐量/p90-E2EL Pareto 前沿;基准数据来自第 3 次尝试。运行可视化。
  • **复用与适用性:**目前没有授权的 /use 评论;通过复用方式合并之前,需要由授权维护者选择运行。此提交直接使用 SGLang,因此 Dynamo TRT-LLM 指标要求不适用;此 PR 未使用 append-only: true,且当前存在合并冲突。

Prepared with assistance from Codex.

Signed: RohitNagraj

@github-actions

github-actions Bot commented Nov 6, 2026

Copy link
Copy Markdown
Contributor

✅✅✅ Verdict: PASS ✅✅✅

Passed and not applicable checks

✅ Check 0 (CODEOWNER): PASS — @RohitNagraj is a named owner of inferencex-e2e/configs/nvidia-master.yaml and of the srt-slurm-recipes/**/gb300-*/ recipe; perf-changelog.yaml falls under the catch-all, which any recognized CODEOWNER satisfies.

✅ Check 1 (In-PR sweep + evals): PASS — Pinned head 1aa39c5b (no later commits) has a successful canary c3 job, five agentic / jobs (c8, c20, c36, c56, c80) and five agentic eval / jobs in run 42124076518 (headSha 1aa39c5b; benchmarks from attempt 3, c8 eval from attempt 5 after a pyxis container-start failure in attempt 3).

✅ Check 2 (Eval accuracy): PASS — The gsm8k evals ran on lmsysorg/sglang:nightly-dev-cu13-20261103-797af717@sha256:c7c348bc…: em_strict 0.9742 for TP4 c3, 0.9757 for c8, 0.9765 for c20, 0.9773 for c36 and 0.9765 for c80, all with n=1319. None is materially weaker than its siblings or than the exact ingested cohort (0.9742; largest gap 0.003, threshold about 0.012). The c20 eval server was launched with --qsa-indexer-dtype fp8_e4m3, --kv-cache-dtype fp8_e4m3, breakable prefill graphs and the NV mega/routing env. The pinned engine runs the QSA indexer on every QSA layer and raises an error when FP8 is unsupported, so there is no silent fallback.

✅ Check 3 (Upstream recipe): PASS — sgl-project/sglang#48857 is MERGED (2026-11-06T08:53Z). Its agentx/Qwen/qwen3.8-flash-next.jsx has three GB300 cells, with provenance 1aa39c5b and the same image. Together they cover TP4/EP4 c3, TP1 c8/c20/c56 (MTP 3-1-4) and TP1 c36/c80 (MTP 2-1-3). They match the pinned recipe's args and env per point: FP8 KV/draft KV, qsa-indexer-dtype, HiCache sizes 224/241, prefill-delayer, min-free-slots-delay, strip-thinking-cache, mamba-max-states-per-path, the post-verify env disables, SGLANG_ENABLE_NV_MEGA_RUNTIME_OPTIMIZATION and SGLANG_ENABLE_METADATA_GLUE_GRAPH. The sign-off links the Deployment panel for the metrics/logging flags.

✅ Check 4 (Reuse command): PASS — COLLABORATOR hshrivastava-droid posted /use 42124076518 on its own line at 2026-11-06T20:17Z; this is the Check 1 run. The sign-off's statement that no /use comment is present is outdated.

✅ Check 5 (Latest checklist): PASS — All 19 items of the current template, plus the upstream-PR sub-item, are present and checked.

✅ Check 6 (Upstream images + ordering + provenance): PASS — The only image is lmsysorg/sglang (upstream) on GB300, and the framework is SGLang only. Its source commit 797af7170f34 is an ancestor of sgl-project/sglang main (main is 283 commits ahead and 0 behind).

✅ Check 7 (Deprecated models): PASS — qwen3.8next agentic coding is active in MODELS.md as of 2026-11-06.

✅ Check 8 (Benchmark hacks): PASS — There are no --hf-overrides or architecture edits. The FP8 KV and FP8 QSA-indexer accuracy levers are covered by the passing c20 and c80 evals; c56 uses the same levers. Integer all-reduce: qwen3.8next is in the policy-baseline matrix (grandfathered), and this is a CUDA path with no quick-reduce controls.

✅ Check 9 (Chat templates): PASS — The AgentX client replays chat conversation traces through infx.bench agentic. Acceptance is simulated from the golden curve, which was collected with the chat template and thinking enabled; the AL gate renders with apply_chat_template.

✅ Check 10 (Engine patches): PASS — No patch, setup script, wheel install or source overlay is reachable from the pinned recipe or its launch path.

✅ Check 11 (Golden AL): PASS — The benchmark logs show SGLANG_SIMULATE_ACC_LEN=2.32 for the 3-step points and 2.06 for the 2-step points (c36, c80), with match-expected/real-draft-token. The golden-al helper confirms both against qwen3.8-flash-next-fp8__mtp__native__default-default.yaml (thinking_on, standard verify).

➖ Check 12 (Append-only): N/A — The new changelog entry does not use append-only: true.

✅ Check 13 (Draft precision): PASS — The embedded MTP head is from checkpoint nvidia/Qwen3.8-Flash-Next-NVFP4@fc694b54: FP8 routed experts and BF16 for the rest, with no draft-quantization flags. SGLANG_NVFP4_CKPT_FP8_NEXTN_MOE='0' is set explicitly. The FP8 settings are allowed: in pinned SGLang 797af717, qsa-indexer-dtype is a global QSA KV-pool cache dtype, set without distinguishing target and draft (kv_cache_configurator.py, resolve_qsa_indexer_dtype). It is therefore applied uniformly to target and draft, and the draft KV is FP8 together with the target KV.

✅ Check 14 (Pareto coverage): PASS — Curve qwen3.8next / agentic-coding / gb300 SGLang / fp4 / p90 E2EL, run 42124076518 attempt 3, artifact results_bmk: 6/5 frontier points (c3, c8, c20, c36, c56, c80), confirmed by pareto_coverage.

✅ Check 15 (PR description): PASS — The config key and image lmsysorg/sglang:nightly-dev-cu13-20261103-797af717 in the PR body match the pinned master config.

✅ Check 16 (NVIDIA overview): PASS — In the NVIDIA overview Framework Recipes table, Qwen3.8-Flash-Next has an SGLang entry linking the cookbook page; this covers GB300.

➖ Check 17 (Metrics publication): N/A — This is a direct SGLang submission; no Dynamo TRT-LLM serving path is affected.

✅ Check 18 (Recipe/image/sweep consistency): PASS — (a) No head generation errors. (b) Master TP4/EP4 c3, TP1 c8/c20 (no KV offload) and TP1 HiCache c36/c56/c80 match the recipe overrides' gpus, TP/EP and KV_OFFLOADING/241 GB. (c) The image digest resolves. (d) and (e) do not apply. (f) The changelog accurately lists the image, the env var, breakable prefill graphs, FP8 KV/draft KV, FP8 QSA-indexer and the step changes. (g) All 8 removed v0.5.22 points are dominated by head points or segments (undominated_removed is empty). (h) There are no re-tuned deltas; at the same concurrency, the new c8 measures 39.4k tok/s/GPU against 39.7k on the old image.

Assessed commit: 1aa39c5bd55c4c13de0a5ab6190a80f4781e63bd.

…ng-agentx-public

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@hshrivastava-droid
hshrivastava-droid merged commit 610e59e into main Nov 6, 2026
5 checks passed
@hshrivastava-droid
hshrivastava-droid deleted the qwen38next-gb300-sglang-agentx-public branch November 6, 2026 21:07
@github-actions

github-actions Bot commented Nov 6, 2026

Copy link
Copy Markdown
Contributor

Ingest report — ✅ Results ingested into the production database.

Ingest workflow · Merge Ingest run · Benchmark source run

Category new duplicate skipped unmapped failed
benchmark 6 0 6 0 0
eval 5 5 6 0 0
trace 6 0 0 0 0
power 6 0 0 0 0

benchmarkFiles 7, rawBenchmarkRows 12, coveredByPerJobFiles 6, purgedPoints 0, failedRunRows 0, runStatsNew 0, runStatsDuplicate 0, evalSamplesNew 6595, gpuMetricSamplesNew 45507, powerPublicationPoints 6, changelogEntriesWritten 1

Counts cover only this run. Rows: benchmark_results, eval_results, agentic rows linked to a trace, gpu_metrics series. This comment updates if the ingest is rerun.

中文

Ingest 报告 — ✅ 结果已写入 production 数据库。

上表仅统计本次运行写入的数据,各行单位依次为 benchmark_results 行、eval_results 行、已关联 trace 的 agentic 行和 gpu_metrics 序列。重新运行 ingest 时本评论会原地更新。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

full-sweep-enabled Full sweep with canary gate; matrix jobs run to completion despite failures

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

5 participants