Repository navigation
chore: bump srt-slurm to v2.65.2 / chore:将 srt-slurm 升级到 v2.65.2 - #4601
Conversation
Move the srt-slurm submodule from v2.59.1 (ae3d6877) to v2.65.2 (59920ea1). v2.65.2 adds NVIDIA/srt-slurm#629: Dynamo frontends since ai-dynamo/dynamo#16454 answer /health status-only, so the readiness wait never counted any workers on Dynamo 1.7 images. DynamoFrontend.probe_ready now asks /health?instances=true only when /health has no instances key. 573-participating-gpus.patch still applies, and src/srtctl/core/power is unchanged between the pins, so it stays. Validation: infx generate for every config key writes 2023 recipes that the new srtctl validates, byte-identical to the v2.59.1 output; all patches apply to v2.65.2; infx.matrix.changed lists no config keys.
|
Thanks for the contribution!
中文感谢你的贡献!
|
| @@ -1 +1 @@ | |||
| Subproject commit ae3d6877c46f3147e3a5dafa46ee4cd89322e52f | |||
| Subproject commit 59920ea16a13017a990e449d034a495c0b05bc57 | |||
There was a problem hiding this comment.
🔴 Every multi-node srt-slurm benchmark switches to a new srtctl (v2.65.2), yet no sweep has to run it before merge. The cause is the missing perf-changelog entry. The bump changes what each launch runs: GPU allocation (#312), KV-event ports (#617), trtllm-serve srun options (#623) and readiness probing (#629). AGENTS.md:87 says infx.matrix.changed cannot see launcher changes, so an empty list proves nothing here. Fix: add one entry at the end of perf-changelog.yaml covering the affected multi-node config keys ("*" as in the v2.39.1→v2.43.4 bump), then get an accepted /use <run_id> before merging.
Why this was flagged
The srt driver clones this pinned submodule for every NVIDIA srt-slurm multi-node launch and runs its srtctl (infx/launch/drivers/srt/checkout.py:82-97). Moving from ae3d6877 to 59920ea1 changes how jobs are placed and started. Per the PR's own list, that covers multi-node worker GPU allocation (#312), KV-event port choice (#617), trtllm-serve disagg srun options (#623) and frontend readiness probing (#629). AGENTS.md:87 says a change to what a benchmark runs needs a perf-changelog.yaml entry. It also says python -m infx.matrix.changed 'does not see script or launcher code, so judge script and launcher changes by what they run'. The PR leaves out the entry only because that tool listed no keys. AGENTS.md:89 says main never runs sweeps, and the entry is what triggers result collection and the /use gate. So the new orchestrator merges without any hardware sweep. A regression would first show up in some unrelated later PR's results, and nothing would tie it to this bump. An earlier pin bump (perf-changelog.yaml:9344-9347, v2.39.1 to v2.43.4) did add a "*" entry.
Verification: Trigger: any NVIDIA srt-slurm multi-node config launched after merge. It runs the new srtctl, and no sweep has run that srtctl in this repo first. inferencex-e2e/infx/launch/drivers/srt/checkout.py:82-97 clones repository_root() / SUBMODULE, so the submodule pointer decides which srtctl runs. AGENTS.md:87 requires a perf-changelog.yaml entry. The PR adds no entry.
…2.65.2 Main bumped inferencex-e2e/utils/srt-slurm to v2.65.2 (#4601), which includes NVIDIA/srt-slurm#629, so runners/srt-slurm/patches/629-dynamo-health-instances.patch and its README row are removed and the changelog entry no longer mentions it.
- perf-changelog: resolve the conflict with main's new entries (#4524, #4599, #4647, #4627, #4596, #4526, #4664) using prepare_perf_changelog_merge resolve-conflict; main's bytes are kept and this PR's entry is re-appended unchanged. - amd-master.yaml: #4664 deleted dsv4-fp4-mi355x-sglang-agentic-mtp right after this PR's glm5.2-fp4-mi355x-atom-disagg-agentic-mtp block. Keep the GLM-5.2 block byte-for-byte and take main's deletion. - main's srt-slurm pin moves to v2.65.2 (#4601); 573 and 604-atom-tp-from-pp.patch still apply in order on 59920ea1. All six files this PR changes are byte-identical to the previous head 1a2707f. - perf-changelog:用 prepare_perf_changelog_merge resolve-conflict 解决与 main 新条目(#4524、#4599、#4647、#4627、#4596、#4526、#4664)的冲突; 保留 main 的原始字节,本 PR 的条目原样追加在末尾。 - amd-master.yaml:#4664 删除了紧跟在本 PR 的 glm5.2-fp4-mi355x-atom-disagg-agentic-mtp 之后的 dsv4-fp4-mi355x-sglang-agentic-mtp。GLM-5.2 条目逐字节保留,同时采纳 main 的删除。 - main 的 srt-slurm 固定版本升到 v2.65.2(#4601);573 和 604-atom-tp-from-pp.patch 仍能在 59920ea1 上按顺序打上。 本 PR 修改的六个文件与上一个 head 1a2707f 逐字节一致。 [skip-sweep] Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…s 59920ea1 This reverts commit a898651. main moved the srt-slurm submodule to v2.65.2 (59920ea1, SemiAnalysisAI#4601), the base NVIDIA/srt-slurm#630 is written against, where start_srun_process takes reuse_container and container_prepare_timeout_seconds. 630-infera-frontend.patch is again SemiAnalysisAI#630's source change; it applies to 59920ea1 after 573-participating-gpus.patch. Signed-off-by: liyingli <liyingli@amd.com>
…-fixed nightlies with prefill-HiCache disagg points and TP4 c1 megakernel / [NVIDIA][AgentX] GB300 DSV4.1-Flash Dynamo + SGLang 升级到含 bonus 修复的 nightly,分离式改用 prefill HiCache,TP4 c1 启用 megakernel (#4595) * Add GB300 DSV4.1-Flash SGLang DEP16 disagg c7480 points with prefill HiCache Root cause: at c7480 the Mooncake external linker reads L3 hits layer by layer on the forward path; after the AgentX turn burst, prefill stalls on those reads (about 40 % of wall time), device prefixes age out, and the deployment does not recover. Fix: a new config key on nightly-dev-cu13-20261027-5884e4fe (Dynamo 1.7.0.dev20261026) with 9 and 10 DEP2 prefill workers and one DEP16 decode. Prefill keeps evicted prefixes in a local host-DRAM HiCache tier (ratio 1.8, write-back, direct I/O); decode keeps SGLang's default polling interval of 1. Dynamo 1.7 backend admission limits are raised so turns queue in the scheduler, and NVIDIA/srt-slurm#629 is backported so the readiness wait asks /health?instances=true when /health is status-only. The agg and dspark-disagg keys are listed so they rerun in the same sweep. Validation: on GB300 with the same recipe and AgentX client, 10 x DEP2 batch 128 gave 924,652 tok/s/GPU at p90 117.2 tok/s/user over 3600 s. * Set perf-changelog pr-link to #4595 * Restore historical perf-changelog entries; set only this entry's pr-link * Move the GB300 DSV4.1-Flash Dynamo + SGLang agg row to the 20261028 nightly and drop TP2/EP2 c16 nightly-dev-cu13-20261028-586355a8 includes sgl-project/sglang#46404. Raise the Dynamo 1.7 backend admission limits as on the disagg rows. TP2/EP2 c16 sits far below the line from TP4 c16 to the disaggregated points, so it is removed with its override. * Move the GB300 DSV4.1-Flash Dynamo + SGLang HiCache disagg key to the 20261028 nightly and retire the Mooncake-linker disagg key The hicache key now runs on nightly-dev-cu13-20261028-586355a8, which includes sgl-project/sglang#46404. On that image the KV pool sizes at 890 B/token, so hicache-ratio drops to 1.2 (1.8 exceeds host memory) and decode mem-fraction-static to 0.75 (0.85 runs out of memory at graph capture). Points: 5P c2884 (bs40), 8P c4096 (bs64) and 10P c7480 (bs160). The 9P and 10P bs128 c7480 points are dropped; bs160 is higher at the same interactivity. The dspark-disagg key on 65633129 and its recipe are retired. * Use 9 prefill workers for the c7480 HiCache point At decode batch 160, 9 x DEP2 reaches the same p90 interactivity as 10 x DEP2 with two fewer GPUs, so its throughput per GPU is higher. * Name the KV layout, precision and Engram changes against the retired recipe in the changelog * Move the GB300 DSV4.1-Flash Dynamo + SGLang agg row to the 20261029 nightly and use the megakernel runtime-optimization switch at TP4 c1 nightly-dev-cu13-20261029-c10e0e17 bundles sgl-nvidia-mega 54b428921749 (megakernel API 2, sgl-project/sglang#47145), so TP4 c1 starts with --enable-nvidia-megakernel again. SGLANG_ENABLE_NV_MEGA_RUNTIME_OPTIMIZATION (sgl-project/sglang#47105) replaces the 16 runtime-optimization variables the recipe set one by one. * Drop the srt-slurm #629 backport patch now that main pins srt-slurm v2.65.2 Main bumped inferencex-e2e/utils/srt-slurm to v2.65.2 (#4601), which includes NVIDIA/srt-slurm#629, so runners/srt-slurm/patches/629-dynamo-health-instances.patch and its README row are removed and the changelog entry no longer mentions it. * chore: refresh PR #4595 for sweep reuse [skip-sweep] --------- Co-authored-by: weireweire <20922698+weireweire@users.noreply.github.com> Co-authored-by: cquil11 <60715037+cquil11@users.noreply.github.com>
…6 and 3P1D c1228 points (append-only, follows #4595) / [NVIDIA][AgentX] 追加 GB300 DSV4.1-Flash Dynamo + SGLang TP2/EP2 c16 和 3P1D c1228 点(仅追加,接续 #4595) (#4679) * Add GB300 DSV4.1-Flash SGLang DEP16 disagg c7480 points with prefill HiCache Root cause: at c7480 the Mooncake external linker reads L3 hits layer by layer on the forward path; after the AgentX turn burst, prefill stalls on those reads (about 40 % of wall time), device prefixes age out, and the deployment does not recover. Fix: a new config key on nightly-dev-cu13-20261027-5884e4fe (Dynamo 1.7.0.dev20261026) with 9 and 10 DEP2 prefill workers and one DEP16 decode. Prefill keeps evicted prefixes in a local host-DRAM HiCache tier (ratio 1.8, write-back, direct I/O); decode keeps SGLang's default polling interval of 1. Dynamo 1.7 backend admission limits are raised so turns queue in the scheduler, and NVIDIA/srt-slurm#629 is backported so the readiness wait asks /health?instances=true when /health is status-only. The agg and dspark-disagg keys are listed so they rerun in the same sweep. Validation: on GB300 with the same recipe and AgentX client, 10 x DEP2 batch 128 gave 924,652 tok/s/GPU at p90 117.2 tok/s/user over 3600 s. * Set perf-changelog pr-link to #4595 * Restore historical perf-changelog entries; set only this entry's pr-link * Move the GB300 DSV4.1-Flash Dynamo + SGLang agg row to the 20261028 nightly and drop TP2/EP2 c16 nightly-dev-cu13-20261028-586355a8 includes sgl-project/sglang#46404. Raise the Dynamo 1.7 backend admission limits as on the disagg rows. TP2/EP2 c16 sits far below the line from TP4 c16 to the disaggregated points, so it is removed with its override. * Move the GB300 DSV4.1-Flash Dynamo + SGLang HiCache disagg key to the 20261028 nightly and retire the Mooncake-linker disagg key The hicache key now runs on nightly-dev-cu13-20261028-586355a8, which includes sgl-project/sglang#46404. On that image the KV pool sizes at 890 B/token, so hicache-ratio drops to 1.2 (1.8 exceeds host memory) and decode mem-fraction-static to 0.75 (0.85 runs out of memory at graph capture). Points: 5P c2884 (bs40), 8P c4096 (bs64) and 10P c7480 (bs160). The 9P and 10P bs128 c7480 points are dropped; bs160 is higher at the same interactivity. The dspark-disagg key on 65633129 and its recipe are retired. * Use 9 prefill workers for the c7480 HiCache point At decode batch 160, 9 x DEP2 reaches the same p90 interactivity as 10 x DEP2 with two fewer GPUs, so its throughput per GPU is higher. * Name the KV layout, precision and Engram changes against the retired recipe in the changelog * Move the GB300 DSV4.1-Flash Dynamo + SGLang agg row to the 20261029 nightly and use the megakernel runtime-optimization switch at TP4 c1 nightly-dev-cu13-20261029-c10e0e17 bundles sgl-nvidia-mega 54b428921749 (megakernel API 2, sgl-project/sglang#47145), so TP4 c1 starts with --enable-nvidia-megakernel again. SGLANG_ENABLE_NV_MEGA_RUNTIME_OPTIMIZATION (sgl-project/sglang#47105) replaces the 16 runtime-optimization variables the recipe set one by one. * Drop the srt-slurm #629 backport patch now that main pins srt-slurm v2.65.2 Main bumped inferencex-e2e/utils/srt-slurm to v2.65.2 (#4601), which includes NVIDIA/srt-slurm#629, so runners/srt-slurm/patches/629-dynamo-health-instances.patch and its README row are removed and the changelog entry no longer mentions it. * Add back GB300 DSV4.1-Flash Dynamo+SGLang TP2/EP2 c16 and 3P1D c1228 points (append-only) Fill the interactivity/E2EL gap between agg TP4 c16 and disagg 5P1D c2884 on each config's existing image: agg TP2/EP2 c16 with its previous per-point settings, and a 3 x DEP2 / 1 x DEP16 c1228 point on the prefill HiCache disagg recipe with the previous c1228 decode sizing. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Set the append-only changelog pr-link to #4679 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * chore: drop history clauses from DSV4.1-Flash hicache decode comments --------- Co-authored-by: weireweire <20922698+weireweire@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: cquil11 <60715037+cquil11@users.noreply.github.com>
Update
inferencex-e2e/utils/srt-slurmfrom v2.59.1 (ae3d6877) to v2.65.2 (59920ea1), and synchronize the documented pin inRECIPES.mdandCONFIGS.md.The main reason is NVIDIA/srt-slurm#629. Dynamo frontends since ai-dynamo/dynamo#16454 (Dynamo 1.7 dev builds) answer
/healthwith status only ({"status":"healthy","stage":"ready"}) and list registered instances only for/health?instances=true. srtctl's readiness wait counts workers from that list, so on images that ship Dynamo 1.7 it never sees a worker and polls until the health-check timeout. With #629,DynamoFrontend.probe_readyreads/healthas before and asks/health?instances=trueonly when the answer has noinstanceskey. Older frontends still get the same single request. With this pin, configs on Dynamo 1.7 images need no backport patch.Three other upstream changes reach existing recipes at launch. They were found by rendering the full srun plan of all 2023 bound recipes under both pins with srt-slurm's mock launcher:
DYN_VLLM_KV_EVENT_PORTand the SGLang and vLLM--kv-events-configendpoints. The highest allocated port is about 3060 plus the DP block, below NATS (4222) and OTEL (4317). Nothing in InferenceX hardcodes 5200.srun_options(chore: add conc to multinode benchmarks job names for viz #624): the frontend step now receives the recipe'ssrun_options, as the workers already do.--container-workdir=/infmax-workspace.container-remap-rootand/ormem: 0.--cpu-bind=none. These aredsv4-fp4-gb200-dynamo-vllm-agentic-dspark-{agg,disagg}anddsv4-fp4-gb300-dynamo-vllm-agentic-mtp-{agg,disagg}, and frontend CPU placement can move their results.dynamo.sidecarrecipes (dsv41flash-fp4-gb300-dynamo-vllm-agentic-dspark-{agg,disagg}),vllm-rsbinds HTTP and gRPC on0.0.0.0instead of127.0.0.1, and Tachometer adds engine scrape targets.AIPERF_SERVER_METRICS_URLSis unchanged.No node assignment or GPU mask changes in any render. #312 makes an uneven multi-node GPU split a hard error, and no current recipe has one. The other changes in the range are opt-in or unused by InferenceX: #614, #615, #616, #618, #623, #624
reuse_container, sa-bench, dsight and mcp.573-participating-gpus.patchstays: it still applies to v2.65.2, andsrc/srtctl/core/poweris unchanged between the two pins, so the reason in its README row still holds. No other patch is affected. No benchmark config changes, so there is no changelog entry.Validation details
infx generate --config-key '*'writes 2023 bound recipes, and the pinned srtctl validates every one. The output is byte-identical between v2.59.1 and v2.65.2.runners/srt-slurm/patches/applies to a fresh v2.65.2 checkout withgit apply --check, as the launcher applies them.python -m infx.matrix.changed --base origin/mainlists no changed config keys.中文
将
inferencex-e2e/utils/srt-slurm从 v2.59.1(ae3d6877)升级到 v2.65.2(59920ea1),并同步RECIPES.md和CONFIGS.md中的版本说明。主要原因是 NVIDIA/srt-slurm#629。自 ai-dynamo/dynamo#16454(Dynamo 1.7 dev 版本)起,Dynamo frontend 的
/health只返回状态,只有/health?instances=true才列出已注册的实例。srtctl 的 readiness 等待按这个列表数 worker,所以在带 Dynamo 1.7 的镜像上永远等不到 worker,直到健康检查超时。#629 之后,DynamoFrontend.probe_ready照旧先读/health,只有返回里没有instances时才请求/health?instances=true,旧版 frontend 收到的请求不变。升级后,使用 Dynamo 1.7 镜像的配置不再需要回移补丁。另外三项上游改动会影响现有 recipe 的启动。这是用 srt-slurm 的 mock launcher 在新旧两个版本下渲染全部 2023 个 recipe 的 srun 计划后发现的:
srun_options(chore: add conc to multinode benchmarks job names for viz #624): frontend 现在也会收到 recipe 的srun_options。所有 513 个 Dynamo recipe 的 frontend 多了--container-workdir=/infmax-workspace。其中 20 个还会多--cpu-bind=none,即dsv4-fp4-gb200-dynamo-vllm-agentic-dspark-{agg,disagg}和dsv4-fp4-gb300-dynamo-vllm-agentic-mtp-{agg,disagg},frontend 的 CPU 绑核可能影响它们的结果。dynamo.sidecarrecipe(dsv41flash-fp4-gb300-dynamo-vllm-agentic-dspark-{agg,disagg})中,vllm-rs改为在0.0.0.0上监听,Tachometer 增加 engine 抓取目标。AIPERF_SERVER_METRICS_URLS不变。所有渲染结果中节点分配和 GPU mask 都没有变化。其余改动对 InferenceX 是可选的或未使用。
保留
573-participating-gpus.patch:它仍能应用于 v2.65.2,且两个版本间src/srtctl/core/power没有变化,README 中保留它的理由仍然成立。其他补丁不受影响。没有 benchmark 配置改动,因此不加 changelog 条目。验证:
infx generate --config-key '*'生成 2023 个绑定后的 recipe,全部通过固定版本 srtctl 的校验,且新旧版本的输出逐字节相同;runners/srt-slurm/patches/中的所有补丁都能用git apply --check应用到全新的 v2.65.2;python -m infx.matrix.changed --base origin/main未列出任何改动的 config key。AI model disclosure
aws/anthropic/eccn-claude-opus-5-5) prepared the submodule bump, checked the patches and the generated recipes against the new pin, and wrote this description. A delegated agent on the same model reviewed the upstream changes in the range and rendered the srun plans under both pins to find their runtime impact on InferenceX launches.PR checklist and change type
Type of Change
Checklist
python -m infx.matrix.changed --base origin/mainlists affected config keys, I have appended one new entry covering them to the physical end ofinferencex-e2e/perf-changelog.yaml; if it lists none, I have added none. I have not edited historical entries