Skip to content

[AgentX][B300] Dsv4 update with dspark - #3606

Closed
Oasis-Git wants to merge 3 commits into
mainfrom
config/dsv4-b300-dspark-coordination
Closed

Oasis-Git wants to merge 3 commits into
mainfrom
config/dsv4-b300-dspark-coordination

Conversation

@Oasis-Git

Copy link
Copy Markdown
Collaborator

更新 B300 DSV4 DSpark 配置,启用 DP 推测解码与预填充协调,保留 TP4/TP8 低延迟测试点。

更新 B300 DSV4 DSpark 配置,启用 DP 推测解码与预填充协调,保留 TP4/TP8 低延迟测试点。
@github-actions

Copy link
Copy Markdown
Contributor

Thanks for the contribution!

  • Review: If this PR changes files owned by someone other than a repository admin or @SemiAnalysisAI/core, ask one eligible CODEOWNER to complete the latest PR_REVIEW_CHECKLIST.md before contacting a core maintainer on Slack. Follow the template exactly, including As a PR reviewer and CODEOWNER, I have reviewed this and have, so sign-off verification triggers.
  • PR verification: Sweeps only run on labeled PRs. Add full-sweep-fail-fast (strongly recommended); use full-sweep-enabled only when matrix jobs should continue after a failure.
  • After merging: PR authors must ensure all GitHub Actions jobs pass. Transient failures often pass on rerun; see how to rerun failed jobs.
中文

感谢你的贡献!

  • **审阅:**如果 PR 修改的文件归属于仓库管理员及 @SemiAnalysisAI/core 之外的 CODEOWNER,请先联系一位有资格的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md,再通过 Slack 联系核心维护者。必须严格遵循模板,并保留 As a PR reviewer and CODEOWNER, I have reviewed this and have,才能触发签核验证。
  • **PR 验证:**扫描仅在带有标签的 PR 上运行。强烈建议添加 full-sweep-fail-fast;仅当需要矩阵任务在失败后继续运行时才使用 full-sweep-enabled。
  • **合并后:**PR 作者必须确保所有 GitHub Actions 任务通过。临时性失败通常可以通过重新运行恢复;参见重新运行失败任务的说明。

@Oasis-Git Oasis-Git added the full-sweep-fail-fast Full sweep with canary gate; first failure cancels the rest of that matrix (recommended) label Sep 30, 2026
补充 DSpark 提交的变更日志 PR 链接,修复矩阵规划校验失败。

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Beyond the inline finding, I checked the disable-circuit-breaker addition to the shared DEP8 router config and the max-running-requests cut to 768 for c512/c576 — both read as deliberate tuning tradeoffs rather than a missing safety guard or admission regression. I also confirmed the container image digest matches exactly between agentic.yaml and nvidia-master.yaml, and that none of the new env vars hardcode a synthetic acceptance length.

Extended reasoning...

The diff bumps the SGLang container image and reworks scheduling/memory/env knobs for the DSV4 B300 DSpark benchmark across the recipe, master config, and changelog; it touches no auth/crypto/permission surface. A confirmed inline finding already flags a missing required pr-link in the new changelog entry, which should block merge until fixed. I additionally verified image-digest parity between the recipe and master config (an AGENTS.md invariant) and ruled out two candidate concerns (circuit-breaker removal on the shared DEP8 router, reduced max-running-requests for c512/c576) as intentional tuning rather than bugs, and found no reintroduction of hardcoded synthetic acceptance lengths.

Comment thread inferencex-e2e/perf-changelog.yaml Outdated
Comment on lines +9131 to +9137
- config-keys:
- dsv4-fp4-b300-sglang-agentic-hicache-mtp
scenario-type:
- agentic-coding
description:
- "Update B300 DSV4 DSpark performance with DP spec/prefill coordination, mixed 64K chunks and interval 32; retain TP4/TP8 low-latency coverage."
- "更新 B300 DSV4 DSpark 性能,启用 DP 推测解码与预填充协调、64K 混合分块和间隔 32;保留 TP4/TP8 低延迟测试点。"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 The new changelog entry for dsv4-fp4-b300-sglang-agentic-hicache-mtp is missing the required pr-link: field, which every other entry in the file has. AGENTS.md requires this field, and infx/workflows/validate_perf_changelog.py's compare_entries() calls validate_added_pr_link(str(entry.get("pr-link") or ""), pr_number) on every newly added entry, raising ChangelogValidationError for an empty/missing link. Fix: append pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/<N> (or the XXX placeholder for PR runs) to this entry, matching every other block in the file.

Why this was flagged

The entry appended at inferencex-e2e/perf-changelog.yaml:9131-9137 (config-keys: dsv4-fp4-b300-sglang-agentic-hicache-mtp) has description lines but no pr-link: key. validate_perf_changelog.py's compare_entries() (called from the CI changelog check) iterates additions and calls validate_added_pr_link(str(entry.get("pr-link") or ""), pr_number); with the key absent this evaluates to an empty string, which fails CANONICAL_PR_LINK.fullmatch(link) (main-branch run) or the pr_number equality check (PR run), raising ChangelogValidationError. A repo-wide count confirms every other of the file's 1057 config-keys blocks has a matching pr-link line (1056 pr-link lines total), so this is the sole entry missing it, and the merge/CI gate for this PR will fail until it is added.

Verification: normal. The appended entry at inferencex-e2e/perf-changelog.yaml:9131-9137 (config-keys: dsv4-fp4-b300-sglang-agentic-hicache-mtp) contains only config-keys/scenario-type/description and no pr-link: key, whereas every other entry has one (lines 9110, 9122, 9129). validate_perf_changelog.py wires this into CI (.github/workflows/run-sweep.yml). compare_entries() collects additions = head_entries[len(base_entries):] and for each calls validate_added_pr_link(str(entry.get("pr-link") or ""), pr_number) (line 193).

@github-actions

github-actions Bot commented Sep 30, 2026 •

Copy link
Copy Markdown
Contributor

更新 B300 DSpark 镜像至 2026-10-08 CUDA 13 开发版 nightly,合并最新 main 并解决变更日志冲突。
@nvpohanh

nvpohanh commented Oct 9, 2026

Copy link
Copy Markdown
Collaborator

[by Claude Code] The SGLang cookbook integration of this PR's AgentX config (dsv4-fp4-b300-sglang-agentic-hicache-mtp) will happen in sgl-project/sglang#43083, in the Agentic Long-Context Deployment section of the DeepSeek-V4 cookbook page. This PR has no authorized /use <run_id> comment yet, so the cookbook shows the PR-head recipe as "in progress" until a measured run is pinned.

@nvpohanh

nvpohanh commented Oct 9, 2026

Copy link
Copy Markdown
Collaborator

[by Claude Code] sgl-project/sglang#43083 has been merged. This PR's AgentX config (dsv4-fp4-b300-sglang-agentic-hicache-mtp) is now in the Agentic Long-Context Deployment section of the SGLang DeepSeek-V4 cookbook: https://docs.sglang.io/cookbook/autoregressive/DeepSeek/DeepSeek-V4#agentic-long-context-deployment

This PR has no authorized /use <run_id> comment yet, so the cookbook shows the recipe at this PR's head (5b394d395) as "in progress". It switches to the measured run at the next cookbook sync after a run is pinned or this PR merges.

@nvpohanh

nvpohanh commented Oct 9, 2026

Copy link
Copy Markdown
Collaborator

[by Claude Code] @Oasis-Git The full sweep for this PR is blocked by the new TP4 DSpark lane: the server hangs at startup every time (3 of 3 attempts). Could you debug it?

Symptom. The fail-fast canary is override_tp4_c1 (TP4, DSpark block 6, flashinfer_mxfp4, DeepSeek-V4-Pro-0813). All checkpoint shards finish loading, then the server prints nothing until the 7200 s health timeout kills it. There is no traceback, OOM or NCCL error. The canary fails, so fail-fast skips every other point.

Run Image Node Result
36687449966 dev-cu13-nightly-0929 b300 gpu-00 Hung after 66/66 shards
37841237652 attempt 1 nightly-dev-cu13-20261008-7760d526 b300 gpu-01 Hung after 66/66 shards
37841237652 attempt 2 nightly-dev-cu13-20261008-7760d526 b300 gpu-00 Hung after 66/66 shards (rerun)

Where it stops. In the merged EAGLE TP4 recipe (b300-fp4-eagle/agentic.yaml, base DeepSeek-V4-Pro, almost the same args), DeepSeek V4 MHC prenorm prewarm: ... n_splits buckets is logged about 1 s after the shards finish loading. In these DSpark/0813 TP4 runs, that line never appears. In SGLang 7760d526, this places the hang in DeepseekV4ForCausalLM.load_weights, after the shard iterator ends and before _prewarm_mhc_kernels logs. That covers draining the async weight-loader futures and post_load_weights. TP4 is new in this PR (main only has TP8 for this config), and no merged SGLang config runs DeepSeek-V4-Pro-0813 + DSpark at TP4.

Suggestions.

  • Reproduce TP4 c1 directly and take a py-spy dump of all four TP ranks once loading stalls.
  • If TP4 can't be fixed quickly, drop the TP4 lane (the search-space entry plus the override_tp4_* blocks) and update the changelog text, so the TP8 and DEP8 points can be measured.

The server logs are in each run's server_logs_dsv4_tp4_conc1_* artifact.

@nvpohanh

Copy link
Copy Markdown
Collaborator

[by Claude Code] The SGLang cookbook update of this PR's AgentX config (dsv4-fp4-b300-sglang-agentic-hicache-mtp) will happen in sgl-project/sglang#43983, in the Agentic Long-Context Deployment section of the DeepSeek-V4 cookbook page. The config has been live in that section since sgl-project/sglang#43083; sgl-project/sglang#43983 refreshes it with the pin below.

This PR has no authorized /use <run_id> comment yet, so the cookbook shows the recipe at this PR's head (5b394d395) as "in progress". It switches to the measured run at the next cookbook sync after a run is pinned or this PR merges.

@nvpohanh

Copy link
Copy Markdown
Collaborator

[by Claude Code] sgl-project/sglang#43983 has been merged. This PR's AgentX config (dsv4-fp4-b300-sglang-agentic-hicache-mtp) is now in the Agentic Long-Context Deployment section of the SGLang DeepSeek-V4 cookbook: https://docs.sglang.io/cookbook/autoregressive/DeepSeek/DeepSeek-V4#agentic-long-context-deployment

This PR has no authorized /use <run_id> comment yet, so the cookbook shows the recipe at this PR's head (5b394d395) as "in progress". It switches to the measured run at the next cookbook sync after a run is pinned or this PR merges.

@nvpohanh

Copy link
Copy Markdown
Collaborator

[by Claude Code] The SGLang cookbook update of this PR's AgentX config (dsv4-fp4-b300-sglang-agentic-hicache-mtp) will happen in sgl-project/sglang#44065, in the Agentic Long-Context Deployment section of the DeepSeek-V4 cookbook page. The config has been live in that section since sgl-project/sglang#43983; sgl-project/sglang#44065 refreshes it with the pin below.

This PR has no authorized /use <run_id> comment yet, so the cookbook shows the recipe at this PR's head (5b394d395) as "in progress". It switches to the measured run at the next cookbook sync after a run is pinned or this PR merges.

@nvpohanh

Copy link
Copy Markdown
Collaborator

[by Claude Code] Correction to my earlier comment: this PR is no longer overlaid in sgl-project/sglang#44065, so the SGLang cookbook update of its AgentX config (dsv4-fp4-b300-sglang-agentic-hicache-mtp) will not happen there. It will be integrated in a future SGLang cookbook PR instead.

When sgl-project/sglang#44065 merges, the Agentic Long-Context Deployment section of the DeepSeek-V4 cookbook page shows the measured InferenceX main version of this config (run 34595563156) instead of this PR's in-progress revision, until that future PR.

@functionstackx

Copy link
Copy Markdown
Collaborator

NVIDIA Ankur confirmed no more DSv4 Pro & M3 (not M3.1) submissions from NVIDIA

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

full-sweep-fail-fast Full sweep with canary gate; first failure cancels the rest of that matrix (recommended)

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

3 participants