Skip to content

[Doc] cookbook: Refresh SemiAnalysis AgentX launches to InferenceX main a0f8080c - #49787

Merged
nvpohanh merged 1 commit into
sgl-project:mainfrom
nvpohanh:pohanh/cookbook-agentx-refresh-1110
Nov 11, 2026
Merged

nvpohanh merged 1 commit into
sgl-project:mainfrom
nvpohanh:pohanh/cookbook-agentx-refresh-1110

Conversation

@nvpohanh

@nvpohanh nvpohanh commented Nov 11, 2026 •

Copy link
Copy Markdown
Collaborator

[by Claude Code]

Motivation

The cookbook's Agentic Long-Context Deployment sections (#42726, #43083, #43799, #43983, #44065, #44228, #44654, #44712, #45039, #45111, #45311, #45513, #45749, #46005, #46506, #46547, #46807, #46948, #47027, #47214, #47464, #47929, #48392, #48551, #48634, #48857, #49260) mirror SemiAnalysis InferenceX AgentX submissions as raw launch commands, each linked to the run that measured it. The configurations are mirrored from SemiAnalysis submissions. This PR:

  1. Syncs every page to InferenceX main (a0f8080c). New or updated merged lanes come from their dashboard runs:
  2. Overlays three open InferenceX PRs:

Source

InferenceX main (a0f8080c), with these InferenceX PRs overlaid:

PR Page Lanes Pinned to
#5006 Qwen3.5 B300 NVFP4 SGLang AgentX Qwen3.5 qwen3.5-fp4-b300-sglang-agentic-mtp-pareto (new) no /use yet: in progress at head c01d1923
#4964 Kimi-K3.1-Pro B200 Dynamo+SGLang: EP4/DCP8 KV-router points at concurrency 1-28 Kimi-K3.1-Pro kimik3.1-fp4-b200-dynamo-sglang-agentic-dflash no /use yet: in progress at head e986c470
#4853 Qwen3.5 GB300 Dynamo+SGLang on the official SGLang nightly with NVIDIA megakernels Qwen3.5 qwen3.5-fp4-gb300-dynamo-sglang-agentic-agg, -disagg no /use yet: in progress at head c914e43c

"In progress" lanes are rendered at the PR head and labeled "In progress in SA InferenceX AgentX". A later sync pins them once the PR gets a /use run or merges.

  • Kimi-K3.1-Pro B200 is in progress. InferenceX #4964 replaces the 11 TP16/DCP16 points that main measured in InferenceX #4801's run with eight TP16 EP4 DCP8 points (c1, 2, 4, 12, 16, 20, 24, 28) on the same image. The submitted frontend is now Dynamo with the KV router (it was the SGLang router).
  • Qwen3.5 B300 SGLang is new and in progress. InferenceX #5006 adds 14 points on nightly-dev-20261013-7224fce1: TP4 at c4-16, TP4·EP4 at c22 / 26, TP2·EP2 at c16-48, and TP2·EP2 with HiCache at c64.
  • Qwen3.5 GB300 Dynamo+SGLang is still in progress. InferenceX #4853's new head keeps the same deployments. Several disaggregated prefill workers now use --hicache-ratio 1.0 (was 1.5), and the c392 decode workers use --mem-fraction-static 0.72 (was 0.76). As in [Doc] cookbook: Refresh SemiAnalysis AgentX launches to InferenceX main 0d7f94e7 #49260, qwen3.5-fp4-gb300-dynamo-sglang-agentic-agg-pareto and -pp-pareto stay off the page, because the PR deletes them.
  • DeepSeek-V4 page unchanged, as in [Doc] cookbook: Refresh SemiAnalysis AgentX launches to InferenceX main ea8b3439 #47464: InferenceX #4664 retired DeepSeek-V4-Pro.
  • Not mirrored (unmeasured, unchanged): musespark-fp4-h100-sglang-agentic-dflash and qwen3.5-fp8-h100-sglang-agentic. The dashboard has no rows for them.
  • InferenceX main was frozen for this PR at a0f8080c. Later merges are left for the next refresh.

What this PR does

Page Lanes Deployments Points Hardware
Qwen3.5 17 (was 16) 61 (was 58) 244 (was 230) GB300, GB200, B300, B200, H200, H100
Kimi-K3.1-Pro 6 19 39 (was 42) GB300, GB200, B300, B200 (+ MI355X, hand-maintained)
Muse Spark Open 7 15 61 GB300, GB200, B300, B200, H200
DeepSeek-V4.1 10 22 67 GB300, GB200, B300, B200, H200, H100
MiniMax-M3.1 6 36 109 GB300, GB200, B300, B200
GLM-5.2 14 34 75 GB300, GB200, B300, B200, H200
GLM-5.3 1 1 10 B300
Qwen3.8-Flash-Next 4 10 26 GB300, B300, B200, H200
DeepSeek-V4 unchanged
  • New mirrored settings, kept as submitted:
    • Kimi-K3.1-Pro B200 (InferenceX #4964): --enable-mscclpp, --load-format runai_streamer, and MSCCLPP_HCA_DEVICES. The last one names the node's RDMA NICs, so it is shown as the $IB_DEVICES placeholder.
    • Qwen3.5 B300 (InferenceX #5006): --soft-watchdog-timeout 60 on the TP2·EP2 HiCache c64 point. It only dumps debugging information when a forward batch runs longer than that.
  • MiniMax-M3.1, GLM-5.2, GLM-5.3 and Qwen3.8-Flash-Next change only in the shared source block (the InferenceX main SHA and the overlay list). MiniMax-M3.1's per-point provenance is regrouped, with the same values.

Validation

  • Browser checks: every page this PR changes was rendered in a headless browser (mint dev, Mintlify as in CI).

    • Every hardware / checkpoint / deployment / concurrency / KV tier / router combination renders exactly the decoded commands: 3,492 command sets, 0 mismatches, 0 unseen, no browser errors. (On Kimi-K3.1-Pro the 18 MI355X combinations come from the hand-maintained file, which the sync does not generate, so they have no decoded reference and were not compared.)
    • Every tab was clicked, and all 201 "Copy link" URLs were reopened.
    • The section links from the Deployment panels and the copy buttons were also checked.
  • Decode and KV-flag checks:

    • The component's own decoder reproduces every generated plan (3,492 router × KV-tier variants).
    • Every KV flag exists in its image's SGLang commit (213 image × flag pairs).
  • Arg-parse: all 179 commands not validated in an earlier round parse. Each image was parsed with itself:

    Image Commands Build commit
    nightly-dev-20261013-7224fce1 78 7224fce1
    nightly-dev-cu13-20261107-e45e1d0c 61 e45e1d0c
    dev-kimi-k31-pro 16 586355a8
    nightly-dev-cu13-20261030-302b78cc 10 amd64 302b78cc, arm64 52604e1d
    nightly-dev-cu13-20261106-858073e4 10 858073e4
    nightly-dev-cu13-20261103-797af717 4 797af717

    The 10 302b78cc commands are Muse Spark GB300 (arm64), so they were also parsed with the 52604e1d source and passed.

  • node scripts/check_cookbook_configs.mjs, mint broken-links --check-anchors --check-redirects, mint validate and pre-commit run --all-files (in lmsysorg/sglang:dev-cu13) pass.

🤖 Generated with Claude Code


CI States

Latest PR Test (Base): ✅ Run #42989333625
Latest PR Test (Extra): ❌ Run #42989333335
Latest PR Test (AMD ROCm 10): ➖ No AMD PR run found for this commit.

Tip

Base CI only covers the core cases, and its workload may not exercise your change. Use /rerun-test to run the tests your change touches, or copy this prompt for an agent:

Find the registered tests this PR affects and run them with /rerun-test, following https://github.com/sgl-project/sglang/blob/main/.agents/skills/rerun-test/SKILL.md

…in a0f8080c

Sync every Agentic Long-Context Deployment section to InferenceX main
a0f8080c with InferenceX sgl-project#5006 (Qwen3.5 B300 SGLang), sgl-project#4964 (Kimi-K3.1-Pro
B200 Dynamo+SGLang) and sgl-project#4853 (Qwen3.5 GB300 Dynamo+SGLang) overlaid.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

@mintlify

mintlify Bot commented Nov 11, 2026 •

Copy link
Copy Markdown
Contributor

✅ Preview approved

@nvpohanh approved a Mintlify preview for commit 2bc691c.

@mintlify

mintlify Bot commented Nov 11, 2026 •

Copy link
Copy Markdown
Contributor

Preview deployment for your docs. Learn more about Mintlify Previews.

Project Status Preview Updated
sglang-doc 🟢 Ready View Preview Nov 11, 2026, 8:53 AM

💡 Tip: Enable Automations to automatically generate PRs for you.

@nvpohanh

Copy link
Copy Markdown
Collaborator Author

preview looks good. merging

This branch was successfully deployed

1 active deployment
staging - docs — 2bc691cf Deployed Nov 11, 2026 by mintlify[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

deepseek documentation Improvements or additions to documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant