Repository navigation
feat: support Qwen3.8-2.4T-A95B - #13105
Conversation
Signed-off-by: Yogendra Pandey <ypandey@nvidia.com>
2P1D = 2 prefill replicas × 16 GPUs + 1 decode replica × 16 GPUs = 48 GPUs, not 32. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> Signed-off-by: chw001 <52851226+chw001@users.noreply.github.com>
WalkthroughAdds the Qwen3.8-2.4T-A95B recipe to Dynamo documentation, catalog metadata, shared model-cache resources, and GB200/GB300 vLLM and SGLang deployment configurations. ChangesQwen3.8-2.4T-A95B recipe
Estimated code review effort: 5 (Critical) | ~120 minutes 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
Comment |
There was a problem hiding this comment.
Actionable comments posted: 14
Note
Due to the large number of review comments, Critical, Major severity comments were prioritized as inline comments.
🟡 Minor comments (8)
docs/fern/pages/recipes/model-recipes/qwen-3-8-2-4t-a95b.mdx-2538-2542 (1)
2538-2542: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick winThe Benchmark section is empty when GB300 is selected.
The only benchmark block is scoped to
data-sku="gb200"at line 2542. GB300 is the default GPU selection at line 2272 and is the recommended target in the catalog. A GB300 reader sees the## Benchmarkheading, the one-line workload description at line 2540, and no command.Add a
data-sku="gb300"benchmark block, or remove thedata-skuscope if the sameaiperf profileinvocation applies to both SKUs.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@docs/fern/pages/recipes/model-recipes/qwen-3-8-2-4t-a95b.mdx` around lines 2538 - 2542, Update the Benchmark section near the existing data-sku="gb200" block so GB300 also receives the benchmark command: either add an equivalent data-sku="gb300" block or remove the SKU restriction when the same aiperf profile invocation applies to both GPUs, while preserving the existing workload description.docs/fern/pages/recipes/model-recipes/qwen-3-8-2-4t-a95b.mdx-2568-2568 (1)
2568-2568: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick winThe picker allows a
Disaggregated+vLLMselection that renders no deploy command.This sentence states that disaggregated targets are SGLang only. The Topology and Engine axes are independent, so a reader can select
Disaggregated+vLLM. Nodata-variant="disagg" data-engine="vllm"block exists, so the Deploy section, the summary strip, and the port-forward step all render empty for that combination.Add a fallback block that names the constraint, so the reader is not left with an empty page.
🔧 Proposed fix
Add near the Deploy blocks:
<div data-variant="disagg" data-engine="vllm"> <Note> Disaggregated serving for this model is SGLang only. Select **SGLang** in the Engine row, or switch the Topology row to **Aggregated**. </Note> </div>🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@docs/fern/pages/recipes/model-recipes/qwen-3-8-2-4t-a95b.mdx` at line 2568, Add a `data-variant="disagg" data-engine="vllm"` fallback block near the existing Deploy blocks in the model recipe. Include a Note stating that disaggregated serving is SGLang-only and instructing readers to select SGLang or switch to Aggregated topology, so this combination renders an explicit constraint instead of empty sections.docs/fern/pages/recipes/model-recipes/qwen-3-8-2-4t-a95b.mdx-2335-2341 (1)
2335-2341: 📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick winThe GB300 prerequisites omit the
model-cachePVC.The GB200 list states the
model-cachePVC requirement at line 2348. The GB300 Deploy step at line 2378 appliesmodel-cache/model-cache.yamlas well, so GB300 has the same prerequisite. Add the matching bullet so both paths list the same requirements.🔧 Proposed fix
- The NVIDIA DRA driver with ComputeDomain support installed (required for multi-node NVLink). - A Hugging Face token with access to `Qwen/Qwen3.8-2.4T-A95B`. +- The `model-cache` PVC, populated by the model-download Job (see the Deploy section below).🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@docs/fern/pages/recipes/model-recipes/qwen-3-8-2-4t-a95b.mdx` around lines 2335 - 2341, Add a bullet to the GB300 prerequisites in the data-sku="gb300" section documenting the required model-cache PVC, matching the existing GB200 prerequisite wording and referencing the model-cache/model-cache.yaml deployment resource.docs/fern/pages/recipes/model-recipes/qwen-3-8-2-4t-a95b.mdx-2337-2337 (1)
2337-2337: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick winUpdate both Kubernetes Deployment Guide links to
../../kubernetes/getting-started/quickstart.mdx. The current path resolves to a nonexistent file.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@docs/fern/pages/recipes/model-recipes/qwen-3-8-2-4t-a95b.mdx` at line 2337, Update both Kubernetes Deployment Guide links in the affected recipe documentation to use ../../kubernetes/getting-started/quickstart.mdx, replacing the existing ../kubernetes/quickstart.mdx targets while preserving the link text.docs/fern/pages/recipes/_catalog/recipes/qwen-3-8-2-4t-a95b.yaml-7-7 (1)
7-7: 🗄️ Data Integrity & Integration | 🟡 Minor | ⚡ Quick winSet
provider: qwenfor this recipe. The checkpoint isQwen/Qwen3.8-2.4T-A95B, and the catalog's sibling Qwen entries useqwen. The provider filter uses this value to include the recipe under the correct chip.🔧 Proposed fix
-provider: nvidia +provider: qwen🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@docs/fern/pages/recipes/_catalog/recipes/qwen-3-8-2-4t-a95b.yaml` at line 7, Update the provider field in the Qwen recipe metadata from nvidia to qwen so the catalog provider filter includes this recipe with the sibling Qwen entries.recipes/qwen3.8-2.4t-a95b/README.md-206-217 (1)
206-217: 📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick winCorrect the statement about the frontend and the PVC.
Line 209 states that the frontend does not mount the PVC. Every deployment in this recipe mounts
model-cacheinto theFrontendcontainer, for examplerecipes/qwen3.8-2.4t-a95b/vllm/agg-gb300-chat/deploy.yamllines 44-45 andrecipes/qwen3.8-2.4t-a95b/sglang/agg-gb200-chat/deploy.yamllines 67-68.Line 211 also states that "Both workers publish KV events". The aggregated profiles run a single worker. Restrict that sentence to the disaggregated profiles, or reword it to "each worker".
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@recipes/qwen3.8-2.4t-a95b/README.md` around lines 206 - 217, The README’s “Model resolution” bullet incorrectly says the frontend does not mount the model-cache PVC; update it to reflect that Frontend deployments mount model-cache. In the “Event-driven KV routing” bullet, replace the “Both workers” wording with “each worker” or otherwise scope it to disaggregated profiles, while preserving the existing routing requirements.recipes/qwen3.8-2.4t-a95b/README.md-90-90 (1)
90-90: 📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick winFix the quickstart link path.
Use
../../docs/fern/pages/kubernetes/getting-started/quickstart.mdx.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@recipes/qwen3.8-2.4t-a95b/README.md` at line 90, Update the Kubernetes Deployment Guide link in the README’s Dynamo Platform prerequisite to use ../../docs/fern/pages/kubernetes/getting-started/quickstart.mdx instead of the current path.recipes/qwen3.8-2.4t-a95b/sglang/agg-gb300-chat/deploy.yaml-94-97 (1)
94-97: 🗄️ Data Integrity & Integration | 🟡 Minor | ⚡ Quick winUpdate the SGLang config checksum annotation.
The
agg.yamlpayload hashes to22f23b24b1c008e7773f95b99830b73c4f3bd69312fb6a5b6f8e7b76df8863fe, not the current annotation value.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@recipes/qwen3.8-2.4t-a95b/sglang/agg-gb300-chat/deploy.yaml` around lines 94 - 97, Update the dynamo.nvidia.com/sglang-config-sha256 annotation under podTemplate.metadata.annotations to 22f23b24b1c008e7773f95b99830b73c4f3bd69312fb6a5b6f8e7b76df8863fe, matching the hash of the agg.yaml payload.
🧹 Nitpick comments (3)
recipes/qwen3.8-2.4t-a95b/README.md (1)
95-95: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valueAlign the ComputeDomain name with the manifests.
The text names
qwen38max-compute-domain. Only the GB300 vLLM manifests use that name. The GB200 vLLM manifests useqwen38max-agg-cdandqwen38max-agg-agentic-cd, and the SGLang manifests useqwen38max-sgl-agg-cd,qwen38max-sgl-disagg-cd, andsglang-qwen38max-compute-domain. State that each manifest creates its ownComputeDomain, without a single fixed name.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@recipes/qwen3.8-2.4t-a95b/README.md` at line 95, Update the ComputeDomain description in the README to state that each manifest creates and uses its own ComputeDomain, rather than naming a single fixed domain. Remove the specific qwen38max-compute-domain reference and avoid implying one shared name across GB300 vLLM, GB200 vLLM, and SGLang manifests.recipes/qwen3.8-2.4t-a95b/model-cache/model-download.yaml (1)
36-41: 🗄️ Data Integrity & Integration | 🔵 Trivial | ⚡ Quick winPin the checkpoint revision.
The Job downloads the branch head. A later upstream commit changes the cached weights while the deployment manifests stay the same, so two clusters can serve different weights from the same recipe.
recipes/qwen3-235b-a22b-fp8/model-cache/model-download.yamlpinsMODEL_REVISIONand passes--revision.Also move the comment on line 36 so it describes the wait, not the
argskey, and reword it: the download can run for hours, not the checkpoint.♻️ Proposed change
env: - name: HF_HOME value: /model-cache - name: HF_XET_HIGH_PERFORMANCE value: "1" - # The checkpoint can run for hours on a cold PVC. + - name: MODEL_REVISION + value: <pinned-commit-sha> + # The download can run for hours on a cold PVC. args: - | set -eux pip install --no-cache-dir huggingface_hub==1.16.4 - hf download Qwen/Qwen3.8-2.4T-A95B + hf download Qwen/Qwen3.8-2.4T-A95B --revision $MODEL_REVISION🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@recipes/qwen3.8-2.4t-a95b/model-cache/model-download.yaml` around lines 36 - 41, Update the model-download Job to define and use a fixed MODEL_REVISION value, passing it to hf download via --revision so cached weights remain reproducible. Move the cold-PVC comment above the download command and reword it to state that the download can run for hours.recipes/qwen3.8-2.4t-a95b/vllm/agg-gb200-agentic/deploy.yaml (1)
92-114: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winThe agentic profile carries the same engine flags as the chat profile.
Line 5 documents a different target workload (64k/400, 90% prefix reuse). Every engine flag here matches
recipes/qwen3.8-2.4t-a95b/vllm/agg-gb200-chat/deploy.yamllines 92-113 exactly, including--max-num-seqs 512,--max-num-batched-tokens 12288, and--gpu-memory-utilization 0.90. Two manifests that differ only in resource names add maintenance cost without a behavior difference. Either tune this profile for the documented workload, or drop the separate agentic manifests and document one profile.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@recipes/qwen3.8-2.4t-a95b/vllm/agg-gb200-agentic/deploy.yaml` around lines 92 - 114, The agentic deployment command in the manifest duplicates the chat profile’s engine configuration despite documenting a different workload. Update the flags in the deployment’s vllm command to reflect the agentic workload’s documented 64k context, 400-sequence limit, and 90% prefix reuse, or remove the separate agentic manifests and document the shared profile instead; keep only one maintained configuration.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@docs/fern/pages/recipes/_catalog/recipes/qwen-3-8-2-4t-a95b.yaml`:
- Around line 138-142: Update the hardware count for the
sglang-disagg-gb200-chat catalog entry to 48 so it matches the 2P1D recipe page
and its comparison table; leave the GPU type and other target metadata
unchanged.
In `@docs/fern/pages/recipes/model-recipes/qwen-3-8-2-4t-a95b.mdx`:
- Line 8: Remove the inlined _DEV_CSS stylesheet and its usage from the recipe
page, then import and render the shared RecipeStyles component using the
standard pattern used by other recipe pages. Ensure the page relies exclusively
on RecipeStyles for recipe styling.
- Around line 2399-2407: The Workload picker and related visibility rules use
the wrong identifiers and scope. Rename the radio group and all four workload
data attributes to recipe-usecase and data-usecase, move the picker outside the
data-sku="gb200" wrapper, and tag SGLang chat-only blocks with
data-usecase="chat" (or block the unsupported Agentic + SGLang combination) so
untagged commands do not bypass filtering.
In `@recipes/qwen3.8-2.4t-a95b/model-cache/model-cache.yaml`:
- Around line 11-15: Increase the PVC storage request under
resources.requests.storage from 3000Gi to a value exceeding the 4.45 TiB
checkpoint size and providing additional staging headroom; leave
storageClassName unchanged.
In `@recipes/qwen3.8-2.4t-a95b/model-cache/model-download.yaml`:
- Around line 34-41: Add container resource requests and limits near the
HF_XET_HIGH_PERFORMANCE environment setting in the model-download container,
matching the established qwen3-235b-a22b-fp8 configuration: CPU "2" and memory
"64Gi" for both requests and limits.
In `@recipes/qwen3.8-2.4t-a95b/sglang/agg-gb200-chat/deploy.yaml`:
- Around line 43-59: Add the --dyn-tool-call-parser qwen3_coder arguments to the
worker command configuration, matching the parser used by the GB300 manifests
and documented support. Update the relevant worker command section while
preserving the existing frontend arguments and ordering.
- Around line 94-123: Update the worker command in the agg-gb200-chat deployment
to configure KV event publishing via the same --kv-events-config pattern used by
the GB300 deployment, matching the frontend’s --router-mode kv and
--router-kv-events settings. Apply the equivalent publisher configuration to the
worker command in disagg-gb200-chat as well.
In `@recipes/qwen3.8-2.4t-a95b/sglang/agg-gb300-chat/deploy.yaml`:
- Around line 69-75: The frontend and worker deployment specs use incomplete
scheduling configuration: the frontend’s empty tolerations cannot tolerate the
GB300 taint, and the worker’s empty nodeSelector/tolerations leave placement
unconfigured. Update the frontend and worker scheduling blocks to use the actual
GB300 selector and dedicated=GoldenPrairie:NoSchedule toleration values used by
the vLLM deployment, or remove the empty keys and document the required cluster
labels.
In `@recipes/qwen3.8-2.4t-a95b/sglang/disagg-gb300-chat/deploy.yaml`:
- Around line 94-106: Update the ResourceClaimTemplate’s RoCE deviceClassName to
use a configurable value rather than hard-coding the AWS class, and ensure both
worker claims reference that configuration. Add the required RoCE/device-class
prerequisite to the recipe README, then align the documented KV-transfer row
with the manifest’s actual UCX/RoCE transport settings instead of NIXL over
MNNVL.
In `@recipes/qwen3.8-2.4t-a95b/vllm/agg-gb200-chat/deploy.yaml`:
- Around line 84-91: Make the post-patch startup commands in
recipes/qwen3.8-2.4t-a95b/vllm/agg-gb200-chat/deploy.yaml lines 84-91,
agg-gb200-agentic/deploy.yaml lines 85-91, agg-gb300-chat/deploy.yaml lines
108-116, and agg-gb300-agentic/deploy.yaml lines 108-116 validate qwen3_5.py
after both sed edits. Check that hf_to_vllm_mapper and
mapper=self.hf_to_vllm_mapper are present, and exit non-zero with a clear
failure message if either is missing; retain the existing exec python3 -m
dynamo.vllm flow after successful validation and note that the patch should be
removed once the image has native support.
- Around line 77-83: Add a startupProbe to the GB200 worker’s main container
definition, matching the probe configuration used by the GB300 manifests,
including failureThreshold 720 and periodSeconds 10. Place it alongside the
existing container settings and preserve the documented 120-minute startup
budget.
In `@recipes/qwen3.8-2.4t-a95b/vllm/agg-gb300-agentic/deploy.yaml`:
- Around line 84-92: Update the environment variables in the deployment’s worker
containers around the existing HF_HOME entries, including the additional
affected block, to set both HF_HUB_OFFLINE and TRANSFORMERS_OFFLINE to enabled
values. Preserve the read-only model-cache mount and existing HF_HOME
configuration, matching the offline-variable setup used by the GB200 deployment
manifests.
- Around line 6-18: Rename the colliding resources and references so each
profile can coexist: in
recipes/qwen3.8-2.4t-a95b/vllm/agg-gb300-agentic/deploy.yaml lines 6-18, use
profile-specific names for the ComputeDomain and DynamoGraphDeployment and
update resourceClaimTemplateName; apply equivalent GB300-chat names in
recipes/qwen3.8-2.4t-a95b/vllm/agg-gb300-chat/deploy.yaml lines 6-18, rename the
conflicting DynamoGraphDeployment in
recipes/qwen3.8-2.4t-a95b/vllm/agg-gb200-chat/deploy.yaml lines 8-20, and give
distinct aggregated and disaggregated ComputeDomain names in
recipes/qwen3.8-2.4t-a95b/sglang/agg-gb300-chat/deploy.yaml lines 49-56 and
recipes/qwen3.8-2.4t-a95b/sglang/disagg-gb300-chat/deploy.yaml lines 85-92; then
update the kubectl wait selector and port-forward service name in
recipes/qwen3.8-2.4t-a95b/README.md lines 145-156 to match the renamed
resources.
- Around line 117-138: Update the command block invoking dynamo.vllm in the
GB300 agentic deployment to include the documented flashinfer_trtllm MoE backend
flag, matching the GB200 manifests and README configuration. Keep the existing
FP8 and other model options unchanged.
---
Minor comments:
In `@docs/fern/pages/recipes/_catalog/recipes/qwen-3-8-2-4t-a95b.yaml`:
- Line 7: Update the provider field in the Qwen recipe metadata from nvidia to
qwen so the catalog provider filter includes this recipe with the sibling Qwen
entries.
In `@docs/fern/pages/recipes/model-recipes/qwen-3-8-2-4t-a95b.mdx`:
- Around line 2538-2542: Update the Benchmark section near the existing
data-sku="gb200" block so GB300 also receives the benchmark command: either add
an equivalent data-sku="gb300" block or remove the SKU restriction when the same
aiperf profile invocation applies to both GPUs, while preserving the existing
workload description.
- Line 2568: Add a `data-variant="disagg" data-engine="vllm"` fallback block
near the existing Deploy blocks in the model recipe. Include a Note stating that
disaggregated serving is SGLang-only and instructing readers to select SGLang or
switch to Aggregated topology, so this combination renders an explicit
constraint instead of empty sections.
- Around line 2335-2341: Add a bullet to the GB300 prerequisites in the
data-sku="gb300" section documenting the required model-cache PVC, matching the
existing GB200 prerequisite wording and referencing the
model-cache/model-cache.yaml deployment resource.
- Line 2337: Update both Kubernetes Deployment Guide links in the affected
recipe documentation to use ../../kubernetes/getting-started/quickstart.mdx,
replacing the existing ../kubernetes/quickstart.mdx targets while preserving the
link text.
In `@recipes/qwen3.8-2.4t-a95b/README.md`:
- Around line 206-217: The README’s “Model resolution” bullet incorrectly says
the frontend does not mount the model-cache PVC; update it to reflect that
Frontend deployments mount model-cache. In the “Event-driven KV routing” bullet,
replace the “Both workers” wording with “each worker” or otherwise scope it to
disaggregated profiles, while preserving the existing routing requirements.
- Line 90: Update the Kubernetes Deployment Guide link in the README’s Dynamo
Platform prerequisite to use
../../docs/fern/pages/kubernetes/getting-started/quickstart.mdx instead of the
current path.
In `@recipes/qwen3.8-2.4t-a95b/sglang/agg-gb300-chat/deploy.yaml`:
- Around line 94-97: Update the dynamo.nvidia.com/sglang-config-sha256
annotation under podTemplate.metadata.annotations to
22f23b24b1c008e7773f95b99830b73c4f3bd69312fb6a5b6f8e7b76df8863fe, matching the
hash of the agg.yaml payload.
---
Nitpick comments:
In `@recipes/qwen3.8-2.4t-a95b/model-cache/model-download.yaml`:
- Around line 36-41: Update the model-download Job to define and use a fixed
MODEL_REVISION value, passing it to hf download via --revision so cached weights
remain reproducible. Move the cold-PVC comment above the download command and
reword it to state that the download can run for hours.
In `@recipes/qwen3.8-2.4t-a95b/README.md`:
- Line 95: Update the ComputeDomain description in the README to state that each
manifest creates and uses its own ComputeDomain, rather than naming a single
fixed domain. Remove the specific qwen38max-compute-domain reference and avoid
implying one shared name across GB300 vLLM, GB200 vLLM, and SGLang manifests.
In `@recipes/qwen3.8-2.4t-a95b/vllm/agg-gb200-agentic/deploy.yaml`:
- Around line 92-114: The agentic deployment command in the manifest duplicates
the chat profile’s engine configuration despite documenting a different
workload. Update the flags in the deployment’s vllm command to reflect the
agentic workload’s documented 64k context, 400-sequence limit, and 90% prefix
reuse, or remove the separate agentic manifests and document the shared profile
instead; keep only one maintained configuration.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: 1f6d0e89-9f2a-4b33-87b3-3a17e9f8ac1c
📒 Files selected for processing (17)
docs/fern/components/RecipeStyles.tsxdocs/fern/index.ymldocs/fern/pages/recipes/_catalog/index.yamldocs/fern/pages/recipes/_catalog/recipes/qwen-3-8-2-4t-a95b.yamldocs/fern/pages/recipes/model-recipes/qwen-3-8-2-4t-a95b.mdxrecipes/README.mdrecipes/qwen3.8-2.4t-a95b/README.mdrecipes/qwen3.8-2.4t-a95b/model-cache/model-cache.yamlrecipes/qwen3.8-2.4t-a95b/model-cache/model-download.yamlrecipes/qwen3.8-2.4t-a95b/sglang/agg-gb200-chat/deploy.yamlrecipes/qwen3.8-2.4t-a95b/sglang/agg-gb300-chat/deploy.yamlrecipes/qwen3.8-2.4t-a95b/sglang/disagg-gb200-chat/deploy.yamlrecipes/qwen3.8-2.4t-a95b/sglang/disagg-gb300-chat/deploy.yamlrecipes/qwen3.8-2.4t-a95b/vllm/agg-gb200-agentic/deploy.yamlrecipes/qwen3.8-2.4t-a95b/vllm/agg-gb200-chat/deploy.yamlrecipes/qwen3.8-2.4t-a95b/vllm/agg-gb300-agentic/deploy.yamlrecipes/qwen3.8-2.4t-a95b/vllm/agg-gb300-chat/deploy.yaml
|
/ok to test 721ad51 |
@ynpandey-nv, there was an error processing your request: See the following link for more information: https://docs.gha-runners.nvidia.com/cpr/e/2/ |
- provider: nvidia → qwen so the recipe is grouped under Qwen on the browse page instead of NVIDIA - README quickstart link: ../../docs/fern/kubernetes/quickstart.mdx → ../../docs/fern/pages/kubernetes/getting-started/quickstart.mdx Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> Signed-off-by: chw001 <52851226+chw001@users.noreply.github.com>
|
/ok to test 54f7c50 |
Frontend requested --router-kv-events but workers never published events — requests were routed without real cache information, silently losing prefix-reuse benefit. Add --kv-events-config (zmq, tcp://*:5571) to: - sglang/agg-gb200-chat: Decode worker - sglang/disagg-gb200-chat: PrefillWorker and Decode worker Matches the kv-events-config already present in the GB300 ConfigMaps. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> Signed-off-by: chw001 <52851226+chw001@users.noreply.github.com>
- GB200 vLLM (chat + agentic): remove cloud.google.com/gke-nodepool - GB200 SGLang (agg + disagg): remove cloud.google.com/gke-nodepool - GB300 vLLM (chat + agentic): remove dedicated: GoldenPrairie, nvidia.com/gpu.clique UUID, and matching dedicated toleration Only nvidia.com/gpu.product remains, matching the pattern used by all other recipes in the repo. Cluster-specific values must be added by the operator at deploy time. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> Signed-off-by: chw001 <52851226+chw001@users.noreply.github.com>
- docs: rename recipe-workload → recipe-usecase and data-workload → data-usecase to match CSS rules; move picker outside data-sku="gb200" so it renders for all SKUs; add data-usecase="chat" to all SGLang blocks so Agentic selection hides them - model-cache.yaml: increase PVC from 3000Gi to 6000Gi to exceed the 4.45 TiB checkpoint and provide staging headroom - model-download.yaml: add resource requests/limits (cpu: 2, memory: 64Gi) matching the qwen3-235b-a22b-fp8 reference configuration Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> Signed-off-by: chw001 <52851226+chw001@users.noreply.github.com>
The 1.4.0-qwen-3.8-2.4t-dev.1 image has native support for the weight prefix remap. Keep --hf-overrides architectures override which is still required. GB300 recipes unchanged. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> Signed-off-by: chw001 <52851226+chw001@users.noreply.github.com>
|
/ok to test a55da71 |
Signed-off-by: Shreyas Misra <shreyasm@nvidia.com>
- Replace inlined _DEV_CSS block (2255 lines) with shared RecipeStyles component import — dev preview aid must not be committed - Fix SPDX copyright year: 2026 → 2025-2026 in both mdx and catalog yaml - Remove data-sku="gb200" scope from Benchmark section so GB300 readers see the aiperf command (identical invocation for both SKUs) Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> Signed-off-by: chw001 <52851226+chw001@users.noreply.github.com>
Both GB200 SGLang frontends were missing the qwen3_coder tool-call parser. Added to the frontend (alongside --reasoning-parser qwen3) to match GB200's frontend-side parsing pattern, matching GB300 worker behavior. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> Signed-off-by: chw001 <52851226+chw001@users.noreply.github.com>
a55da71 to
788bafb
Compare
|
/ok to test 788bafb |
Signed-off-by: Yogendra Pandey <ypandey@nvidia.com> Signed-off-by: chw001 <52851226+chw001@users.noreply.github.com> Signed-off-by: Shreyas Misra <shreyasm@nvidia.com> Co-authored-by: chw001 <52851226+chw001@users.noreply.github.com> Co-authored-by: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: Shreyas Misra <shreyasm@nvidia.com> Signed-off-by: Yogendra Pandey <ypandey@nvidia.com>
Summary
Adds support for Qwen3.8-2.4T-A95B.
Summary by CodeRabbit
New Features
Documentation