Skip to content

[NV] Muse-Spark-Open NVFP4 GB300 SGLang AgentX configs / [NV] Muse-Spark-Open NVFP4 GB300 SGLang AgentX 配置 - #4454

Closed
hshrivastava-droid wants to merge 1 commit into
mainfrom
musespark-gb300-sglang-agentx-nightly1019
Closed

hshrivastava-droid wants to merge 1 commit into
mainfrom
musespark-gb300-sglang-agentx-nightly1019

Conversation

@hshrivastava-droid

Copy link
Copy Markdown
Collaborator

Summary

Add two Muse-Spark-Open NVFP4 GB300 SGLang AgentX configs on lmsysorg/sglang:nightly-dev-20261019-3c560089 (pinned by digest):

  • musespark-fp4-gb300-sglang-agentic-dflash-multinode: aggregated TP8/EP8 over two nodes, concurrency 5, 10, 18 and 32.
  • musespark-fp4-gb300-sglang-agentic-dflash-nightly1019: aggregated TP4/EP4 on one node, concurrency 28 and 52.
AI model disclosure
  • Model/version: Claude Sonnet 5.5 (claude-sonnet-5-5)
  • Role: Prepared the config bundle, changelog entry and PR text.
PR checklist and change type

Type of Change

  • Bug fix
  • New feature
  • Configuration change
  • Documentation update
  • Other (please describe)

Checklist

  • I have completed the AI model disclosure and kept it current
  • I have tested my changes locally
  • I have updated documentation if necessary
  • If python -m infx.matrix.changed --base origin/main lists affected config keys, I have appended one new entry covering them to the physical end of inferencex-e2e/perf-changelog.yaml; if it lists none, I have added none. I have not edited historical entries
  • If this PR appends an inferencex-e2e/perf-changelog.yaml entry, it carries exactly one primary sweep label (a maintainer applies it on fork PRs): full-sweep-fail-fast (recommended), full-sweep-enabled, or non-canary-full-sweep-enabled. Optional modifiers all-evals, evals-only, and agentx-fast require a primary label; the last two block reuse while applied.
  • Before merging via reuse, an authorized maintainer (OWNER/MEMBER/COLLABORATOR) has commented /use <run_id> (or the legacy /reuse-sweep-run) on this PR. Do this only once there is a final full sweep that is all green with evals passing, since after this comment the primary sweep label will no longer automatically kick off new sweeps. Remove and re-add the primary sweep label to force a new sweep.
中文

新增两个 Muse-Spark-Open NVFP4 GB300 SGLang AgentX 配置,镜像为 lmsysorg/sglang:nightly-dev-20261019-3c560089(按摘要固定):多节点 TP8/EP8(并发 5、10、18、32)与单节点 TP4/EP4(并发 28、52)。

AI 模型使用说明:Claude Sonnet 5.5(claude-sonnet-5-5),负责准备配置、变更日志和 PR 文本。变更类型:配置变更。

🤖 Generated with Claude Code

…0261019)

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Signed-off-by: hshrivastava-droid <hshrivastava@nvidia.com>
@github-actions

Copy link
Copy Markdown
Contributor

Thanks for the contribution!

  • Review: If this PR changes files owned by someone other than a repository admin or @SemiAnalysisAI/core, ask one eligible CODEOWNER to complete the latest PR_REVIEW_CHECKLIST.md before contacting a core maintainer on Slack. Follow the template exactly, including As a PR reviewer and CODEOWNER, I have reviewed this and have, so sign-off verification triggers.
  • PR verification: Sweeps run only when the PR appends an inferencex-e2e/perf-changelog.yaml entry and carries exactly one primary label: full-sweep-fail-fast (strongly recommended; canary plus per-matrix fail-fast), full-sweep-enabled (canary; matrix jobs continue after a failure), or non-canary-full-sweep-enabled (no canary or fail-fast). The modifiers all-evals, evals-only, and agentx-fast require a primary label. On fork PRs, a maintainer applies the label. See sweep labels and reuse.
  • After merging: PR authors must ensure all GitHub Actions jobs pass. Transient failures often pass on rerun; see how to rerun failed jobs.
中文

感谢你的贡献!

  • **审阅:**如果 PR 修改的文件归属于仓库管理员及 @SemiAnalysisAI/core 之外的 CODEOWNER,请先联系一位有资格的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md,再通过 Slack 联系核心维护者。必须严格遵循模板,并保留 As a PR reviewer and CODEOWNER, I have reviewed this and have,才能触发签核验证。
  • **PR 验证:**只有当 PR 在 inferencex-e2e/perf-changelog.yaml 末尾追加了条目,并且恰好带有一个主标签时,才会运行扫描:full-sweep-fail-fast(强烈推荐;canary 加逐矩阵 fail-fast)、full-sweep-enabled(有 canary;矩阵任务在失败后继续运行)或 non-canary-full-sweep-enabled(无 canary,也无 fail-fast)。修饰标签 all-evals、evals-only 和 agentx-fast 必须与主标签一起使用。fork PR 的标签由维护者添加。参见扫描标签与复用。
  • **合并后:**PR 作者必须确保所有 GitHub Actions 任务通过。临时性失败通常可以通过重新运行恢复;参见重新运行失败任务的说明。

@hshrivastava-droid hshrivastava-droid added the full-sweep-enabled Full sweep with canary gate; matrix jobs run to completion despite failures label Oct 23, 2026
@hshrivastava-droid hshrivastava-droid changed the title [NV] Muse-Spark-Open NVFP4 GB300 SGLang AgentX configs (nightly-dev-20261019) / [NV] Muse-Spark-Open NVFP4 GB300 SGLang AgentX 配置(nightly-dev-20261019) [NV] Muse-Spark-Open NVFP4 GB300 SGLang AgentX configs / [NV] Muse-Spark-Open NVFP4 GB300 SGLang AgentX 配置 Oct 23, 2026
Comment on lines +79 to +80
# The client container has no /model mount; it loads the tokenizer by Hub id.
HF_HUB_OFFLINE: '0'

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 All four new multi-node runs may fail at benchmark start because the AIPerf client must fetch the tokenizer from a gated Hub repo. The client env sets HF_HUB_OFFLINE: '0' and no TOKENIZER, so the tokenizer is pulled by the served model id. The sibling Muse-Spark recipes say that repo is unreadable with the job token, and they use TOKENIZER: /model. Fix: make the client tokenizer source match what works for this model on the same cluster, for example TOKENIZER: /model as in agg-variants.yaml:92. If /model really is missing in this mode, point it at a readable tokenizer path instead.

Why this was flagged

The trigger is the job token lacking read access to the gated repo meta-models/Muse-Spark-Open-816B-A42B-NVFP4. Two comments in the repo say it does: agentic.yaml:65 says "The gated Hub repo is unreadable with the job token". In the new recipe, base.benchmark.env (agg-tp8-direct-variants.yaml:79-80) sets HF_HUB_OFFLINE: '0' and no TOKENIZER, so AIPerf loads the tokenizer by Hub id. The existing multi-node recipe in the same directory and on the same cluster:gb300-nv runner (agg-variants.yaml:92) uses TOKENIZER: /model, and this is the only recipe in the repo that sets HF_HUB_OFFLINE: '0'. If the Hub download is denied, every row of musespark-fp4-gb300-sglang-agentic-dflash-multinode (conc 5/10/18/32) fails before any benchmark data is collected, so the full sweep goes red. The new comment says the client has no /model mount, but the sibling recipe gets the same srt multi-node client and uses /model, so that comment does not rule this failure out.

Verification: Triggering condition: the job's HF token cannot read the gated repo meta-models/Muse-Spark-Open-816B-A42B-NVFP4, as gb300-fp4-mtp/agentic.yaml:61 says. agg-tp8-direct-variants.yaml:78-82 sets HF_HUB_OFFLINE: '0' and no TOKENIZER. infx/bench/agentic/replay.py:77 then falls back to values["MODEL"], the gated Hub id. agg-variants.yaml:92 sets TOKENIZER: /model.

@github-actions

Copy link
Copy Markdown
Contributor

@nvpohanh

Copy link
Copy Markdown
Collaborator

[by Claude Code] The SGLang cookbook integration of this PR's AgentX configs (musespark-fp4-gb300-sglang-agentic-dflash-multinode, musespark-fp4-gb300-sglang-agentic-dflash-nightly1019) will happen in sgl-project/sglang#46807, in the Agentic Long-Context Deployment section of the Muse Spark Open cookbook page.

This PR has no authorized /use <run_id> comment yet, so the cookbook shows the recipe at this PR's head (a73b274a6) as "in progress". A later cookbook PR switches it to the measured run once a run is pinned or this PR merges.

@nvpohanh

Copy link
Copy Markdown
Collaborator

[by Claude Code] sgl-project/sglang#46807 has been merged. This PR's AgentX configs (musespark-fp4-gb300-sglang-agentic-dflash-multinode, musespark-fp4-gb300-sglang-agentic-dflash-nightly1019) are now in the Agentic Long-Context Deployment section of the SGLang Muse Spark Open cookbook: https://docs.sglang.io/cookbook/autoregressive/Meta/MuseSparkOpen#agentic-long-context-deployment

This PR has no authorized /use <run_id> comment yet, so the cookbook shows the recipe at this PR's head (a73b274a6) as "in progress". A later cookbook PR switches it to the measured run once a run is pinned or this PR merges.

@nvpohanh

Copy link
Copy Markdown
Collaborator

[by Claude Code] The SGLang cookbook update of this PR's AgentX configs (musespark-fp4-gb300-sglang-agentic-dflash-multinode, musespark-fp4-gb300-sglang-agentic-dflash-nightly1019) will happen in sgl-project/sglang#46948, in the Agentic Long-Context Deployment section of the Muse Spark Open cookbook page. The configs have been live in that section since sgl-project/sglang#46807; sgl-project/sglang#46948 refreshes them with the pin below.

This PR has no authorized /use <run_id> comment yet, so the cookbook shows the recipe at this PR's head (a73b274a6) as "in progress". A later cookbook PR switches it to the measured run once a run is pinned or this PR merges.

@nvpohanh

Copy link
Copy Markdown
Collaborator

[by Claude Code] sgl-project/sglang#46948 has been merged. This PR's AgentX configs (musespark-fp4-gb300-sglang-agentic-dflash-multinode, musespark-fp4-gb300-sglang-agentic-dflash-nightly1019) are now in the Agentic Long-Context Deployment section of the SGLang Muse Spark Open cookbook: https://docs.sglang.io/cookbook/autoregressive/Meta/MuseSparkOpen#agentic-long-context-deployment

This PR has no authorized /use <run_id> comment yet, so the cookbook shows the recipe at this PR's head (a73b274a6) as "in progress". A later cookbook PR switches it to the measured run once a run is pinned or this PR merges.

@nvpohanh

Copy link
Copy Markdown
Collaborator

[by Claude Code] The SGLang cookbook update of this PR's AgentX configs (musespark-fp4-gb300-sglang-agentic-dflash-multinode, musespark-fp4-gb300-sglang-agentic-dflash-nightly1019) will happen in sgl-project/sglang#47027, in the Agentic Long-Context Deployment section of the Muse Spark Open cookbook page. The configs have been live in that section since sgl-project/sglang#46807; sgl-project/sglang#47027 refreshes them with the pin below.

This PR has no authorized /use <run_id> comment yet, so the cookbook shows the recipe at this PR's head (a73b274a6) as "in progress". A later cookbook PR switches it to the measured run once a run is pinned or this PR merges.

@nvpohanh

Copy link
Copy Markdown
Collaborator

[by Claude Code] sgl-project/sglang#47027 has been merged. This PR's AgentX configs (musespark-fp4-gb300-sglang-agentic-dflash-multinode, musespark-fp4-gb300-sglang-agentic-dflash-nightly1019) are now in the Agentic Long-Context Deployment section of the SGLang Muse Spark Open cookbook: https://docs.sglang.io/cookbook/autoregressive/Meta/MuseSparkOpen#agentic-long-context-deployment

This PR has no authorized /use <run_id> comment yet, so the cookbook shows the recipe at this PR's head (a73b274a6) as "in progress". A later cookbook PR switches it to the measured run once a run is pinned or this PR merges.

@nvpohanh

Copy link
Copy Markdown
Collaborator

[by Claude Code] The SGLang cookbook integration of this PR's AgentX configs (musespark-fp4-gb300-sglang-agentic-dflash-multinode, musespark-fp4-gb300-sglang-agentic-dflash-nightly1019) will happen in sgl-project/sglang#47464, in the Agentic Long-Context Deployment section of the Muse Spark Open cookbook page. They were in that section from sgl-project/sglang#46807 until a later cookbook refresh took them out; sgl-project/sglang#47464 adds them back with the pin below.

This PR has no authorized /use <run_id> comment yet, so the cookbook shows the recipe at this PR's head (a73b274a6) as "in progress". A later cookbook PR switches it to the measured run once a run is pinned or this PR merges.

@nvpohanh

Copy link
Copy Markdown
Collaborator

[by Claude Code] sgl-project/sglang#47464 has been merged. This PR's AgentX configs (musespark-fp4-gb300-sglang-agentic-dflash-multinode, musespark-fp4-gb300-sglang-agentic-dflash-nightly1019) are now in the Agentic Long-Context Deployment section of the SGLang Muse Spark Open cookbook: https://docs.sglang.io/cookbook/autoregressive/Meta/MuseSparkOpen#agentic-long-context-deployment

This PR has no authorized /use <run_id> comment yet, so the cookbook shows the recipe at this PR's head (a73b274a6) as "in progress". A later cookbook PR switches it to the measured run once a run is pinned or this PR merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

full-sweep-enabled Full sweep with canary gate; matrix jobs run to completion despite failures

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

2 participants