Skip to content

feat: support Qwen3.8-2.4T-A95B - #13105

Merged
ynpandey-nv merged 11 commits into
mainfrom
ypandey/cherrypick-qwen3-8-2-4t-a95b
Aug 12, 2026
Merged

ynpandey-nv merged 11 commits into
mainfrom
ypandey/cherrypick-qwen3-8-2-4t-a95b

Conversation

@ynpandey-nv

@ynpandey-nv ynpandey-nv commented Aug 12, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Adds support for Qwen3.8-2.4T-A95B.


Open in Devin Review

Summary by CodeRabbit

  • New Features

    • Added the Qwen3.8-2.4T-A95B model recipe to the recipe catalog.
    • Added deployment options for vLLM and SGLang on GB200 and GB300 hardware, including chat, agentic, aggregated, and disaggregated configurations.
    • Added model-cache setup and automated checkpoint download instructions.
    • Added interactive filtering for inference engine and deployment targets.
  • Documentation

    • Added prerequisites, deployment steps, smoke tests, benchmarking guidance, supported features, operational notes, and known limitations.

Signed-off-by: Yogendra Pandey <ypandey@nvidia.com>
@ynpandey-nv
ynpandey-nv requested review from a team as code owners August 12, 2026 15:57
@copy-pr-bot

copy-pr-bot Bot commented Aug 12, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@github-actions github-actions Bot added documentation Improvements or additions to documentation feat labels Aug 12, 2026

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 14 potential issues.

View 1 additional finding in Devin Review.

Open in Devin Review

2P1D = 2 prefill replicas × 16 GPUs + 1 decode replica × 16 GPUs = 48 GPUs, not 32.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Signed-off-by: chw001 <52851226+chw001@users.noreply.github.com>
@coderabbitai

coderabbitai Bot commented Aug 12, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Walkthrough

Adds the Qwen3.8-2.4T-A95B recipe to Dynamo documentation, catalog metadata, shared model-cache resources, and GB200/GB300 vLLM and SGLang deployment configurations.

Changes

Qwen3.8-2.4T-A95B recipe

Layer / File(s) Summary
Catalog and interactive recipe page
docs/fern/components/RecipeStyles.tsx, docs/fern/index.yml, docs/fern/pages/recipes/_catalog/*, docs/fern/pages/recipes/model-recipes/qwen-3-8-2-4t-a95b.mdx
Adds catalog entries, navigation, engine filtering, deployment selectors, target summaries, responsive styles, and recipe page content.
Recipe documentation and model cache
recipes/README.md, recipes/qwen3.8-2.4t-a95b/README.md, recipes/qwen3.8-2.4t-a95b/model-cache/*
Documents supported profiles, prerequisites, deployment commands, smoke tests, benchmarking, operational settings, and known limitations. Adds a 3000Gi PVC and a Hugging Face checkpoint-download Job.
vLLM aggregated deployments
recipes/qwen3.8-2.4t-a95b/vllm/*/deploy.yaml
Adds GB200 and GB300 Dynamo vLLM deployments for aggregated chat and agentic workloads, including frontend routing, multinode workers, model-cache mounts, resource claims, runtime patches, and vLLM settings.
SGLang aggregated and disaggregated deployments
recipes/qwen3.8-2.4t-a95b/sglang/*/deploy.yaml
Adds GB200 and GB300 SGLang deployments with aggregated workers and disaggregated prefill/decode workers, including ComputeDomain resources, GPU and RoCE claims, model caches, health checks, and NIXL networking.

Estimated code review effort: 5 (Critical) | ~120 minutes

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Description check ⚠️ Warning The description only states the feature and omits the required overview, details, reviewer-start, and related-issues sections. Add the required sections, describe the implementation and review entry points, and either link a related issue or confirm that no issue exists.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly and concisely summarizes the pull request's main change: adding support for Qwen3.8-2.4T-A95B.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 14

Note

Due to the large number of review comments, Critical, Major severity comments were prioritized as inline comments.

🟡 Minor comments (8)
docs/fern/pages/recipes/model-recipes/qwen-3-8-2-4t-a95b.mdx-2538-2542 (1)

2538-2542: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

The Benchmark section is empty when GB300 is selected.

The only benchmark block is scoped to data-sku="gb200" at line 2542. GB300 is the default GPU selection at line 2272 and is the recommended target in the catalog. A GB300 reader sees the ## Benchmark heading, the one-line workload description at line 2540, and no command.

Add a data-sku="gb300" benchmark block, or remove the data-sku scope if the same aiperf profile invocation applies to both SKUs.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/fern/pages/recipes/model-recipes/qwen-3-8-2-4t-a95b.mdx` around lines
2538 - 2542, Update the Benchmark section near the existing data-sku="gb200"
block so GB300 also receives the benchmark command: either add an equivalent
data-sku="gb300" block or remove the SKU restriction when the same aiperf
profile invocation applies to both GPUs, while preserving the existing workload
description.
docs/fern/pages/recipes/model-recipes/qwen-3-8-2-4t-a95b.mdx-2568-2568 (1)

2568-2568: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

The picker allows a Disaggregated + vLLM selection that renders no deploy command.

This sentence states that disaggregated targets are SGLang only. The Topology and Engine axes are independent, so a reader can select Disaggregated + vLLM. No data-variant="disagg" data-engine="vllm" block exists, so the Deploy section, the summary strip, and the port-forward step all render empty for that combination.

Add a fallback block that names the constraint, so the reader is not left with an empty page.

🔧 Proposed fix

Add near the Deploy blocks:

<div data-variant="disagg" data-engine="vllm">

<Note>
Disaggregated serving for this model is SGLang only. Select **SGLang** in the Engine row, or switch the Topology row to **Aggregated**.
</Note>

</div>
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/fern/pages/recipes/model-recipes/qwen-3-8-2-4t-a95b.mdx` at line 2568,
Add a `data-variant="disagg" data-engine="vllm"` fallback block near the
existing Deploy blocks in the model recipe. Include a Note stating that
disaggregated serving is SGLang-only and instructing readers to select SGLang or
switch to Aggregated topology, so this combination renders an explicit
constraint instead of empty sections.
docs/fern/pages/recipes/model-recipes/qwen-3-8-2-4t-a95b.mdx-2335-2341 (1)

2335-2341: 📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win

The GB300 prerequisites omit the model-cache PVC.

The GB200 list states the model-cache PVC requirement at line 2348. The GB300 Deploy step at line 2378 applies model-cache/model-cache.yaml as well, so GB300 has the same prerequisite. Add the matching bullet so both paths list the same requirements.

🔧 Proposed fix
 - The NVIDIA DRA driver with ComputeDomain support installed (required for multi-node NVLink).
 - A Hugging Face token with access to `Qwen/Qwen3.8-2.4T-A95B`.
+- The `model-cache` PVC, populated by the model-download Job (see the Deploy section below).
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/fern/pages/recipes/model-recipes/qwen-3-8-2-4t-a95b.mdx` around lines
2335 - 2341, Add a bullet to the GB300 prerequisites in the data-sku="gb300"
section documenting the required model-cache PVC, matching the existing GB200
prerequisite wording and referencing the model-cache/model-cache.yaml deployment
resource.
docs/fern/pages/recipes/model-recipes/qwen-3-8-2-4t-a95b.mdx-2337-2337 (1)

2337-2337: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Update both Kubernetes Deployment Guide links to ../../kubernetes/getting-started/quickstart.mdx. The current path resolves to a nonexistent file.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/fern/pages/recipes/model-recipes/qwen-3-8-2-4t-a95b.mdx` at line 2337,
Update both Kubernetes Deployment Guide links in the affected recipe
documentation to use ../../kubernetes/getting-started/quickstart.mdx, replacing
the existing ../kubernetes/quickstart.mdx targets while preserving the link
text.
docs/fern/pages/recipes/_catalog/recipes/qwen-3-8-2-4t-a95b.yaml-7-7 (1)

7-7: 🗄️ Data Integrity & Integration | 🟡 Minor | ⚡ Quick win

Set provider: qwen for this recipe. The checkpoint is Qwen/Qwen3.8-2.4T-A95B, and the catalog's sibling Qwen entries use qwen. The provider filter uses this value to include the recipe under the correct chip.

🔧 Proposed fix
-provider: nvidia
+provider: qwen
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/fern/pages/recipes/_catalog/recipes/qwen-3-8-2-4t-a95b.yaml` at line 7,
Update the provider field in the Qwen recipe metadata from nvidia to qwen so the
catalog provider filter includes this recipe with the sibling Qwen entries.
recipes/qwen3.8-2.4t-a95b/README.md-206-217 (1)

206-217: 📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win

Correct the statement about the frontend and the PVC.

Line 209 states that the frontend does not mount the PVC. Every deployment in this recipe mounts model-cache into the Frontend container, for example recipes/qwen3.8-2.4t-a95b/vllm/agg-gb300-chat/deploy.yaml lines 44-45 and recipes/qwen3.8-2.4t-a95b/sglang/agg-gb200-chat/deploy.yaml lines 67-68.

Line 211 also states that "Both workers publish KV events". The aggregated profiles run a single worker. Restrict that sentence to the disaggregated profiles, or reword it to "each worker".

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@recipes/qwen3.8-2.4t-a95b/README.md` around lines 206 - 217, The README’s
“Model resolution” bullet incorrectly says the frontend does not mount the
model-cache PVC; update it to reflect that Frontend deployments mount
model-cache. In the “Event-driven KV routing” bullet, replace the “Both workers”
wording with “each worker” or otherwise scope it to disaggregated profiles,
while preserving the existing routing requirements.
recipes/qwen3.8-2.4t-a95b/README.md-90-90 (1)

90-90: 📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win

Fix the quickstart link path.

Use ../../docs/fern/pages/kubernetes/getting-started/quickstart.mdx.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@recipes/qwen3.8-2.4t-a95b/README.md` at line 90, Update the Kubernetes
Deployment Guide link in the README’s Dynamo Platform prerequisite to use
../../docs/fern/pages/kubernetes/getting-started/quickstart.mdx instead of the
current path.
recipes/qwen3.8-2.4t-a95b/sglang/agg-gb300-chat/deploy.yaml-94-97 (1)

94-97: 🗄️ Data Integrity & Integration | 🟡 Minor | ⚡ Quick win

Update the SGLang config checksum annotation.

The agg.yaml payload hashes to 22f23b24b1c008e7773f95b99830b73c4f3bd69312fb6a5b6f8e7b76df8863fe, not the current annotation value.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@recipes/qwen3.8-2.4t-a95b/sglang/agg-gb300-chat/deploy.yaml` around lines 94
- 97, Update the dynamo.nvidia.com/sglang-config-sha256 annotation under
podTemplate.metadata.annotations to
22f23b24b1c008e7773f95b99830b73c4f3bd69312fb6a5b6f8e7b76df8863fe, matching the
hash of the agg.yaml payload.
🧹 Nitpick comments (3)
recipes/qwen3.8-2.4t-a95b/README.md (1)

95-95: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Align the ComputeDomain name with the manifests.

The text names qwen38max-compute-domain. Only the GB300 vLLM manifests use that name. The GB200 vLLM manifests use qwen38max-agg-cd and qwen38max-agg-agentic-cd, and the SGLang manifests use qwen38max-sgl-agg-cd, qwen38max-sgl-disagg-cd, and sglang-qwen38max-compute-domain. State that each manifest creates its own ComputeDomain, without a single fixed name.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@recipes/qwen3.8-2.4t-a95b/README.md` at line 95, Update the ComputeDomain
description in the README to state that each manifest creates and uses its own
ComputeDomain, rather than naming a single fixed domain. Remove the specific
qwen38max-compute-domain reference and avoid implying one shared name across
GB300 vLLM, GB200 vLLM, and SGLang manifests.
recipes/qwen3.8-2.4t-a95b/model-cache/model-download.yaml (1)

36-41: 🗄️ Data Integrity & Integration | 🔵 Trivial | ⚡ Quick win

Pin the checkpoint revision.

The Job downloads the branch head. A later upstream commit changes the cached weights while the deployment manifests stay the same, so two clusters can serve different weights from the same recipe. recipes/qwen3-235b-a22b-fp8/model-cache/model-download.yaml pins MODEL_REVISION and passes --revision.

Also move the comment on line 36 so it describes the wait, not the args key, and reword it: the download can run for hours, not the checkpoint.

♻️ Proposed change
           env:
             - name: HF_HOME
               value: /model-cache
             - name: HF_XET_HIGH_PERFORMANCE
               value: "1"
-          # The checkpoint can run for hours on a cold PVC.
+            - name: MODEL_REVISION
+              value: <pinned-commit-sha>
+          # The download can run for hours on a cold PVC.
           args:
             - |
               set -eux
               pip install --no-cache-dir huggingface_hub==1.16.4
-              hf download Qwen/Qwen3.8-2.4T-A95B
+              hf download Qwen/Qwen3.8-2.4T-A95B --revision $MODEL_REVISION
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@recipes/qwen3.8-2.4t-a95b/model-cache/model-download.yaml` around lines 36 -
41, Update the model-download Job to define and use a fixed MODEL_REVISION
value, passing it to hf download via --revision so cached weights remain
reproducible. Move the cold-PVC comment above the download command and reword it
to state that the download can run for hours.
recipes/qwen3.8-2.4t-a95b/vllm/agg-gb200-agentic/deploy.yaml (1)

92-114: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

The agentic profile carries the same engine flags as the chat profile.

Line 5 documents a different target workload (64k/400, 90% prefix reuse). Every engine flag here matches recipes/qwen3.8-2.4t-a95b/vllm/agg-gb200-chat/deploy.yaml lines 92-113 exactly, including --max-num-seqs 512, --max-num-batched-tokens 12288, and --gpu-memory-utilization 0.90. Two manifests that differ only in resource names add maintenance cost without a behavior difference. Either tune this profile for the documented workload, or drop the separate agentic manifests and document one profile.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@recipes/qwen3.8-2.4t-a95b/vllm/agg-gb200-agentic/deploy.yaml` around lines 92
- 114, The agentic deployment command in the manifest duplicates the chat
profile’s engine configuration despite documenting a different workload. Update
the flags in the deployment’s vllm command to reflect the agentic workload’s
documented 64k context, 400-sequence limit, and 90% prefix reuse, or remove the
separate agentic manifests and document the shared profile instead; keep only
one maintained configuration.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@docs/fern/pages/recipes/_catalog/recipes/qwen-3-8-2-4t-a95b.yaml`:
- Around line 138-142: Update the hardware count for the
sglang-disagg-gb200-chat catalog entry to 48 so it matches the 2P1D recipe page
and its comparison table; leave the GPU type and other target metadata
unchanged.

In `@docs/fern/pages/recipes/model-recipes/qwen-3-8-2-4t-a95b.mdx`:
- Line 8: Remove the inlined _DEV_CSS stylesheet and its usage from the recipe
page, then import and render the shared RecipeStyles component using the
standard pattern used by other recipe pages. Ensure the page relies exclusively
on RecipeStyles for recipe styling.
- Around line 2399-2407: The Workload picker and related visibility rules use
the wrong identifiers and scope. Rename the radio group and all four workload
data attributes to recipe-usecase and data-usecase, move the picker outside the
data-sku="gb200" wrapper, and tag SGLang chat-only blocks with
data-usecase="chat" (or block the unsupported Agentic + SGLang combination) so
untagged commands do not bypass filtering.

In `@recipes/qwen3.8-2.4t-a95b/model-cache/model-cache.yaml`:
- Around line 11-15: Increase the PVC storage request under
resources.requests.storage from 3000Gi to a value exceeding the 4.45 TiB
checkpoint size and providing additional staging headroom; leave
storageClassName unchanged.

In `@recipes/qwen3.8-2.4t-a95b/model-cache/model-download.yaml`:
- Around line 34-41: Add container resource requests and limits near the
HF_XET_HIGH_PERFORMANCE environment setting in the model-download container,
matching the established qwen3-235b-a22b-fp8 configuration: CPU "2" and memory
"64Gi" for both requests and limits.

In `@recipes/qwen3.8-2.4t-a95b/sglang/agg-gb200-chat/deploy.yaml`:
- Around line 43-59: Add the --dyn-tool-call-parser qwen3_coder arguments to the
worker command configuration, matching the parser used by the GB300 manifests
and documented support. Update the relevant worker command section while
preserving the existing frontend arguments and ordering.
- Around line 94-123: Update the worker command in the agg-gb200-chat deployment
to configure KV event publishing via the same --kv-events-config pattern used by
the GB300 deployment, matching the frontend’s --router-mode kv and
--router-kv-events settings. Apply the equivalent publisher configuration to the
worker command in disagg-gb200-chat as well.

In `@recipes/qwen3.8-2.4t-a95b/sglang/agg-gb300-chat/deploy.yaml`:
- Around line 69-75: The frontend and worker deployment specs use incomplete
scheduling configuration: the frontend’s empty tolerations cannot tolerate the
GB300 taint, and the worker’s empty nodeSelector/tolerations leave placement
unconfigured. Update the frontend and worker scheduling blocks to use the actual
GB300 selector and dedicated=GoldenPrairie:NoSchedule toleration values used by
the vLLM deployment, or remove the empty keys and document the required cluster
labels.

In `@recipes/qwen3.8-2.4t-a95b/sglang/disagg-gb300-chat/deploy.yaml`:
- Around line 94-106: Update the ResourceClaimTemplate’s RoCE deviceClassName to
use a configurable value rather than hard-coding the AWS class, and ensure both
worker claims reference that configuration. Add the required RoCE/device-class
prerequisite to the recipe README, then align the documented KV-transfer row
with the manifest’s actual UCX/RoCE transport settings instead of NIXL over
MNNVL.

In `@recipes/qwen3.8-2.4t-a95b/vllm/agg-gb200-chat/deploy.yaml`:
- Around line 84-91: Make the post-patch startup commands in
recipes/qwen3.8-2.4t-a95b/vllm/agg-gb200-chat/deploy.yaml lines 84-91,
agg-gb200-agentic/deploy.yaml lines 85-91, agg-gb300-chat/deploy.yaml lines
108-116, and agg-gb300-agentic/deploy.yaml lines 108-116 validate qwen3_5.py
after both sed edits. Check that hf_to_vllm_mapper and
mapper=self.hf_to_vllm_mapper are present, and exit non-zero with a clear
failure message if either is missing; retain the existing exec python3 -m
dynamo.vllm flow after successful validation and note that the patch should be
removed once the image has native support.
- Around line 77-83: Add a startupProbe to the GB200 worker’s main container
definition, matching the probe configuration used by the GB300 manifests,
including failureThreshold 720 and periodSeconds 10. Place it alongside the
existing container settings and preserve the documented 120-minute startup
budget.

In `@recipes/qwen3.8-2.4t-a95b/vllm/agg-gb300-agentic/deploy.yaml`:
- Around line 84-92: Update the environment variables in the deployment’s worker
containers around the existing HF_HOME entries, including the additional
affected block, to set both HF_HUB_OFFLINE and TRANSFORMERS_OFFLINE to enabled
values. Preserve the read-only model-cache mount and existing HF_HOME
configuration, matching the offline-variable setup used by the GB200 deployment
manifests.
- Around line 6-18: Rename the colliding resources and references so each
profile can coexist: in
recipes/qwen3.8-2.4t-a95b/vllm/agg-gb300-agentic/deploy.yaml lines 6-18, use
profile-specific names for the ComputeDomain and DynamoGraphDeployment and
update resourceClaimTemplateName; apply equivalent GB300-chat names in
recipes/qwen3.8-2.4t-a95b/vllm/agg-gb300-chat/deploy.yaml lines 6-18, rename the
conflicting DynamoGraphDeployment in
recipes/qwen3.8-2.4t-a95b/vllm/agg-gb200-chat/deploy.yaml lines 8-20, and give
distinct aggregated and disaggregated ComputeDomain names in
recipes/qwen3.8-2.4t-a95b/sglang/agg-gb300-chat/deploy.yaml lines 49-56 and
recipes/qwen3.8-2.4t-a95b/sglang/disagg-gb300-chat/deploy.yaml lines 85-92; then
update the kubectl wait selector and port-forward service name in
recipes/qwen3.8-2.4t-a95b/README.md lines 145-156 to match the renamed
resources.
- Around line 117-138: Update the command block invoking dynamo.vllm in the
GB300 agentic deployment to include the documented flashinfer_trtllm MoE backend
flag, matching the GB200 manifests and README configuration. Keep the existing
FP8 and other model options unchanged.

---

Minor comments:
In `@docs/fern/pages/recipes/_catalog/recipes/qwen-3-8-2-4t-a95b.yaml`:
- Line 7: Update the provider field in the Qwen recipe metadata from nvidia to
qwen so the catalog provider filter includes this recipe with the sibling Qwen
entries.

In `@docs/fern/pages/recipes/model-recipes/qwen-3-8-2-4t-a95b.mdx`:
- Around line 2538-2542: Update the Benchmark section near the existing
data-sku="gb200" block so GB300 also receives the benchmark command: either add
an equivalent data-sku="gb300" block or remove the SKU restriction when the same
aiperf profile invocation applies to both GPUs, while preserving the existing
workload description.
- Line 2568: Add a `data-variant="disagg" data-engine="vllm"` fallback block
near the existing Deploy blocks in the model recipe. Include a Note stating that
disaggregated serving is SGLang-only and instructing readers to select SGLang or
switch to Aggregated topology, so this combination renders an explicit
constraint instead of empty sections.
- Around line 2335-2341: Add a bullet to the GB300 prerequisites in the
data-sku="gb300" section documenting the required model-cache PVC, matching the
existing GB200 prerequisite wording and referencing the
model-cache/model-cache.yaml deployment resource.
- Line 2337: Update both Kubernetes Deployment Guide links in the affected
recipe documentation to use ../../kubernetes/getting-started/quickstart.mdx,
replacing the existing ../kubernetes/quickstart.mdx targets while preserving the
link text.

In `@recipes/qwen3.8-2.4t-a95b/README.md`:
- Around line 206-217: The README’s “Model resolution” bullet incorrectly says
the frontend does not mount the model-cache PVC; update it to reflect that
Frontend deployments mount model-cache. In the “Event-driven KV routing” bullet,
replace the “Both workers” wording with “each worker” or otherwise scope it to
disaggregated profiles, while preserving the existing routing requirements.
- Line 90: Update the Kubernetes Deployment Guide link in the README’s Dynamo
Platform prerequisite to use
../../docs/fern/pages/kubernetes/getting-started/quickstart.mdx instead of the
current path.

In `@recipes/qwen3.8-2.4t-a95b/sglang/agg-gb300-chat/deploy.yaml`:
- Around line 94-97: Update the dynamo.nvidia.com/sglang-config-sha256
annotation under podTemplate.metadata.annotations to
22f23b24b1c008e7773f95b99830b73c4f3bd69312fb6a5b6f8e7b76df8863fe, matching the
hash of the agg.yaml payload.

---

Nitpick comments:
In `@recipes/qwen3.8-2.4t-a95b/model-cache/model-download.yaml`:
- Around line 36-41: Update the model-download Job to define and use a fixed
MODEL_REVISION value, passing it to hf download via --revision so cached weights
remain reproducible. Move the cold-PVC comment above the download command and
reword it to state that the download can run for hours.

In `@recipes/qwen3.8-2.4t-a95b/README.md`:
- Line 95: Update the ComputeDomain description in the README to state that each
manifest creates and uses its own ComputeDomain, rather than naming a single
fixed domain. Remove the specific qwen38max-compute-domain reference and avoid
implying one shared name across GB300 vLLM, GB200 vLLM, and SGLang manifests.

In `@recipes/qwen3.8-2.4t-a95b/vllm/agg-gb200-agentic/deploy.yaml`:
- Around line 92-114: The agentic deployment command in the manifest duplicates
the chat profile’s engine configuration despite documenting a different
workload. Update the flags in the deployment’s vllm command to reflect the
agentic workload’s documented 64k context, 400-sequence limit, and 90% prefix
reuse, or remove the separate agentic manifests and document the shared profile
instead; keep only one maintained configuration.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 1f6d0e89-9f2a-4b33-87b3-3a17e9f8ac1c

📥 Commits

Reviewing files that changed from the base of the PR and between f4625cd and 721ad51.

📒 Files selected for processing (17)
  • docs/fern/components/RecipeStyles.tsx
  • docs/fern/index.yml
  • docs/fern/pages/recipes/_catalog/index.yaml
  • docs/fern/pages/recipes/_catalog/recipes/qwen-3-8-2-4t-a95b.yaml
  • docs/fern/pages/recipes/model-recipes/qwen-3-8-2-4t-a95b.mdx
  • recipes/README.md
  • recipes/qwen3.8-2.4t-a95b/README.md
  • recipes/qwen3.8-2.4t-a95b/model-cache/model-cache.yaml
  • recipes/qwen3.8-2.4t-a95b/model-cache/model-download.yaml
  • recipes/qwen3.8-2.4t-a95b/sglang/agg-gb200-chat/deploy.yaml
  • recipes/qwen3.8-2.4t-a95b/sglang/agg-gb300-chat/deploy.yaml
  • recipes/qwen3.8-2.4t-a95b/sglang/disagg-gb200-chat/deploy.yaml
  • recipes/qwen3.8-2.4t-a95b/sglang/disagg-gb300-chat/deploy.yaml
  • recipes/qwen3.8-2.4t-a95b/vllm/agg-gb200-agentic/deploy.yaml
  • recipes/qwen3.8-2.4t-a95b/vllm/agg-gb200-chat/deploy.yaml
  • recipes/qwen3.8-2.4t-a95b/vllm/agg-gb300-agentic/deploy.yaml
  • recipes/qwen3.8-2.4t-a95b/vllm/agg-gb300-chat/deploy.yaml

@ynpandey-nv

Copy link
Copy Markdown
Contributor Author

/ok to test 721ad51

@copy-pr-bot

copy-pr-bot Bot commented Aug 12, 2026

Copy link
Copy Markdown

/ok to test 721ad51

@ynpandey-nv, there was an error processing your request: E2

See the following link for more information: https://docs.gha-runners.nvidia.com/cpr/e/2/

- provider: nvidia → qwen so the recipe is grouped under Qwen on the
  browse page instead of NVIDIA
- README quickstart link: ../../docs/fern/kubernetes/quickstart.mdx →
  ../../docs/fern/pages/kubernetes/getting-started/quickstart.mdx

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Signed-off-by: chw001 <52851226+chw001@users.noreply.github.com>
@ynpandey-nv

Copy link
Copy Markdown
Contributor Author

/ok to test 54f7c50

chw001 and others added 2 commits August 12, 2026 09:18
Frontend requested --router-kv-events but workers never published
events — requests were routed without real cache information, silently
losing prefix-reuse benefit.

Add --kv-events-config (zmq, tcp://*:5571) to:
- sglang/agg-gb200-chat: Decode worker
- sglang/disagg-gb200-chat: PrefillWorker and Decode worker

Matches the kv-events-config already present in the GB300 ConfigMaps.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Signed-off-by: chw001 <52851226+chw001@users.noreply.github.com>
- GB200 vLLM (chat + agentic): remove cloud.google.com/gke-nodepool
- GB200 SGLang (agg + disagg): remove cloud.google.com/gke-nodepool
- GB300 vLLM (chat + agentic): remove dedicated: GoldenPrairie,
  nvidia.com/gpu.clique UUID, and matching dedicated toleration

Only nvidia.com/gpu.product remains, matching the pattern used by
all other recipes in the repo. Cluster-specific values must be added
by the operator at deploy time.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Signed-off-by: chw001 <52851226+chw001@users.noreply.github.com>
@github-actions

github-actions Bot commented Aug 12, 2026 •

Copy link
Copy Markdown
Contributor

chw001 and others added 2 commits August 12, 2026 09:24
- docs: rename recipe-workload → recipe-usecase and data-workload →
  data-usecase to match CSS rules; move picker outside data-sku="gb200"
  so it renders for all SKUs; add data-usecase="chat" to all SGLang
  blocks so Agentic selection hides them
- model-cache.yaml: increase PVC from 3000Gi to 6000Gi to exceed the
  4.45 TiB checkpoint and provide staging headroom
- model-download.yaml: add resource requests/limits (cpu: 2, memory: 64Gi)
  matching the qwen3-235b-a22b-fp8 reference configuration

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Signed-off-by: chw001 <52851226+chw001@users.noreply.github.com>
The 1.4.0-qwen-3.8-2.4t-dev.1 image has native support for the weight
prefix remap. Keep --hf-overrides architectures override which is still
required. GB300 recipes unchanged.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Signed-off-by: chw001 <52851226+chw001@users.noreply.github.com>
@ynpandey-nv

Copy link
Copy Markdown
Contributor Author

/ok to test a55da71

NVShreyas and others added 4 commits August 12, 2026 10:32
Signed-off-by: Shreyas Misra <shreyasm@nvidia.com>
- Replace inlined _DEV_CSS block (2255 lines) with shared RecipeStyles
  component import — dev preview aid must not be committed
- Fix SPDX copyright year: 2026 → 2025-2026 in both mdx and catalog yaml
- Remove data-sku="gb200" scope from Benchmark section so GB300 readers
  see the aiperf command (identical invocation for both SKUs)

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Signed-off-by: chw001 <52851226+chw001@users.noreply.github.com>
Both GB200 SGLang frontends were missing the qwen3_coder tool-call
parser. Added to the frontend (alongside --reasoning-parser qwen3)
to match GB200's frontend-side parsing pattern, matching GB300 worker
behavior.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Signed-off-by: chw001 <52851226+chw001@users.noreply.github.com>
@NVShreyas
NVShreyas force-pushed the ypandey/cherrypick-qwen3-8-2-4t-a95b branch from a55da71 to 788bafb Compare August 12, 2026 17:33
@NVShreyas

Copy link
Copy Markdown
Contributor

/ok to test 788bafb

@ynpandey-nv
ynpandey-nv merged commit c8a33bf into main Aug 12, 2026
101 of 102 checks passed
@ynpandey-nv
ynpandey-nv deleted the ypandey/cherrypick-qwen3-8-2-4t-a95b branch August 12, 2026 18:07
ynpandey-nv added a commit that referenced this pull request Aug 25, 2026
Signed-off-by: Yogendra Pandey <ypandey@nvidia.com>
Signed-off-by: chw001 <52851226+chw001@users.noreply.github.com>
Signed-off-by: Shreyas Misra <shreyasm@nvidia.com>
Co-authored-by: chw001 <52851226+chw001@users.noreply.github.com>
Co-authored-by: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Shreyas Misra <shreyasm@nvidia.com>
Signed-off-by: Yogendra Pandey <ypandey@nvidia.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation feat size/XXL

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants