Skip to content

feat(examples/sglang): add Intel XPU deployment configs - #10023

Merged
dmitry-tokarev-nv merged 29 commits into
ai-dynamo:mainfrom
yao531441:feat/sglang-xpu-deploy-configs
Oct 29, 2026
Merged

dmitry-tokarev-nv merged 29 commits into
ai-dynamo:mainfrom
yao531441:feat/sglang-xpu-deploy-configs

Conversation

@yao531441

@yao531441 yao531441 commented May 27, 2026 •

Copy link
Copy Markdown
Contributor

Details:

Adds Intel XPU deployment templates for SGLang on Kubernetes under
examples/backends/sglang/deploy/xpu/, using Dynamic Resource Allocation (DRA)
plus one device-plugin variant:

  • agg_xpu_dra.yaml: aggregated, one worker
  • agg_router_xpu_dra.yaml: aggregated, two workers behind the KV router
  • disagg_xpu_dra.yaml: disaggregated prefill and decode (NIXL)
  • disagg_planner_xpu_dra.yaml: disaggregated with the Planner
  • disagg_xpu.yaml: disaggregated with the Intel device plugin (gpu.intel.com/xe)
  • xpu/README.md, and a link from sglang/deploy/README.md

Notes on the XPU launch:

  • The workers unset DYN_FORWARDPASS_METRIC_PORT before starting. The operator
    injects it, and the XPU SGLang build crashes on the first request when forward
    pass metrics are on (they use torch.cuda.Event).
  • The workers pass --attention-backend intel_xpu. The default Triton backend
    returned garbage tokens (!!!!) in testing.
  • Images are placeholders (my-registry/...:my-tag). The operator reads the
    Dynamo version from the image tag, so the manifests set no runtimeVersionOverride.
  • The Planner template ships no profile data (pre_deployment_sweeping_mode: none).

Validation

Tested on an Intel XPU node (single node, Kubernetes with DRA) with a
dynamo_xpu runtime image:

  • agg_xpu_dra.yaml: pods ready; /v1/completions and /v1/chat/completions
    return correct text.
  • disagg_xpu_dra.yaml: frontend, prefill and decode ready; completions correct.
  • All five manifests pass kubectl apply --dry-run=server, and pre-commit passes.

Related Issues

#16321

@yao531441
yao531441 requested a review from a team as a code owner May 27, 2026 06:17
@copy-pr-bot

copy-pr-bot Bot commented May 27, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@github-actions

Copy link
Copy Markdown
Contributor

👋 Hi yao531441! Thank you for contributing to ai-dynamo/dynamo.

Just a reminder: The NVIDIA Test Github Validation CI runs an essential subset of the testing framework to quickly catch errors.Your PR reviewers may elect to test the changes comprehensively before approving your changes.

🚀

@github-actions github-actions Bot added external-contribution Pull request is from an external contributor feat documentation Improvements or additions to documentation backend::sglang Relates to the sglang backend xpu labels May 27, 2026
@coderabbitai

coderabbitai Bot commented May 27, 2026 •

Copy link
Copy Markdown
Contributor

Walkthrough

This pull request adds comprehensive Intel XPU deployment support to the sglang backend, including documentation and four Kubernetes manifest patterns for different serving architectures: single-service aggregated, disaggregated with device plugin, disaggregated with DRA, and disaggregated with planner-orchestrated serving.

Changes

Intel XPU Deployment Templates and Documentation

Layer / File(s) Summary
Documentation and deployment guide
examples/backends/sglang/deploy/README.md, examples/backends/sglang/deploy/xpu/README.md
Main README updated with XPU deployment section and template references. New XPU-specific README covers prerequisites (Kubernetes/DRA, Intel GPU driver, custom runtime image), configuration differences from NVIDIA templates (device class, grammar backend, CUDA graph, KV transfer), DRA warnings, deployment/verification steps, and testing commands.
Aggregated XPU deployment with DRA
examples/backends/sglang/deploy/xpu/agg_xpu_dra.yaml
Single Kubernetes manifest combining Frontend and decode services using dynamo-sglang-xpu image, GPU ResourceClaimTemplate requesting gpu.intel.com device, HuggingFace token secret injection, working directory setup, and sglang serving command with XPU device selection and disabled CUDA graphs.
Disaggregated XPU deployment with device plugin
examples/backends/sglang/deploy/xpu/disagg_xpu.yaml
Kubernetes manifest for disaggregated serving using traditional device plugin. Defines Frontend service and two worker subcomponents (decode and prefill) with hostIPC configuration, XPU device limits (gpu.intel.com/xe), UCX/ZE_AFFINITY_MASK environment variables, and disaggregation-mode-specific launch parameters including transfer backend and port assignments.
Disaggregated XPU deployment with DRA
examples/backends/sglang/deploy/xpu/disagg_xpu_dra.yaml
Kubernetes manifest for disaggregated serving via DRA. Includes ResourceClaimTemplate requesting gpu.intel.com, DynamoGraphDeployment with Frontend and worker subcomponents (decode with 2 replicas, prefill with 1 replica), ephemeral storage requests (2Gi), hostIPC activation, UCX_TLS configuration, and sglang invocation with disaggregation modes and transfer backend setup (nixl backend, port 12345).
Disaggregated XPU deployment with planner and DRA
examples/backends/sglang/deploy/xpu/disagg_planner_xpu_dra.yaml
Kubernetes manifest for planner-orchestrated disaggregated serving. Defines ConfigMap with prefill/decode profiling arrays and max KV tokens, ResourceClaimTemplate for GPU requests, and DynamoGraphDeployment with Frontend, Planner (single replica running dynamo.planner with inline JSON config), decode workers (2 replicas), and prefill worker (1 replica). All components share GPU resource claims, host networking settings (hostIPC/hostPID), UCX variables, and disaggregation parameters with fixed bootstrap port (12345).

🎯 2 (Simple) | ⏱️ ~12 minutes

Pre-merge checks: Passed 4
Check name Status Explanation
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Title check Passed The title "feat(examples/sglang): add Intel XPU deployment configs" matches the main change. The PR adds Kubernetes deployment manifests and a README for SGLang on Intel XPU GPUs under examples/backen…
Description check Passed The description follows the template well. It has a Details section that lists each new manifest, the README, and the README link. It explains the XPU-specific launch notes (DYN_FORWARDPASS_METRIC_POR…
Full details: Title check

Explanation

The title "feat(examples/sglang): add Intel XPU deployment configs" matches the main change. The PR adds Kubernetes deployment manifests and a README for SGLang on Intel XPU GPUs under examples/backends/sglang/deploy/xpu/. The title is short, specific, and uses a conventional commit style. It does not list every file, which is acceptable.

Full details: Description check

Explanation

The description follows the template well. It has a Details section that lists each new manifest, the README, and the README link. It explains the XPU-specific launch notes (DYN_FORWARDPASS_METRIC_PORT, intel_xpu attention backend, placeholder images, and the Planner profile data). It includes a Validation section with test results. It has a Related Issues section that links issue #16321. Gaps: the template's 'Overview' section and 'Where should the reviewer start?' section are missing. The 'Closes' keyword is absent, so the link is plain and does not auto-close the issue. The PR objectives mention reviewer start points, but the description does not state them. The raw summary also lists agg_router_xpu_dra.yaml, which the objectives do not mention, so the file list may be inconsistent with the other sources. These gaps are non-critical, so the check passes.

  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@examples/backends/sglang/deploy/xpu/disagg_planner_xpu_dra.yaml`:
- Around line 5-7: Update the header warning comment to match the manifest's
replica config: change the note from "Requires 4+ GPU devices (2 prefill + 2
decode)" to "Requires 3 GPU devices (1 prefill + 2 decode)" and remove or adjust
the subsequent line that suggests a 3-GPU fallback; ensure the top comment
reflects the actual spec of 1 prefill + 2 decode replicas.
- Around line 109-110: The UCX_TLS environment entry in
disagg_planner_xpu_dra.yaml is missing the "ze_ipc" transport; update the
UCX_TLS value string for the existing env var named UCX_TLS so it matches the
other disaggregated XPU templates by changing the value to
"ze_ipc,ze_copy,tcp,cma,sysv,posix,self" (ensure the UCX_TLS env block that
currently has "ze_copy,tcp,cma,sysv,posix,self" is replaced with the new
string).

In `@examples/backends/sglang/deploy/xpu/README.md`:
- Line 24: The secret creation command currently uses the key `token` which
mismatches runtime expectations; update the kubectl command that creates
`hf-token-secret` to use the key `HF_TOKEN` (e.g., create secret generic
hf-token-secret --from-literal=HF_TOKEN=<your-token>) so the runtime
templates/env variables that read `HF_TOKEN` find the token; also scan
references to `hf-token-secret`/`HF_TOKEN` in deployment manifests and env
templates to ensure they consistently use `HF_TOKEN`.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: fdbbdd7e-6225-4dd4-b215-da100d882730

📥 Commits

Reviewing files that changed from the base of the PR and between d97c889 and 87f5bee.

📒 Files selected for processing (6)
  • examples/backends/sglang/deploy/README.md
  • examples/backends/sglang/deploy/xpu/README.md
  • examples/backends/sglang/deploy/xpu/agg_xpu_dra.yaml
  • examples/backends/sglang/deploy/xpu/disagg_planner_xpu_dra.yaml
  • examples/backends/sglang/deploy/xpu/disagg_xpu.yaml
  • examples/backends/sglang/deploy/xpu/disagg_xpu_dra.yaml

@dagil-nvidia

Copy link
Copy Markdown
Collaborator

/ok to test 93d6e69

@yao531441
yao531441 requested a review from a team as a code owner June 11, 2026 01:22
@yao531441
yao531441 force-pushed the feat/sglang-xpu-deploy-configs branch from 788220d to a8a442b Compare June 11, 2026 02:10
@yao531441
yao531441 force-pushed the feat/sglang-xpu-deploy-configs branch from fb6d9e5 to a6dac87 Compare July 7, 2026 01:27
@yao531441
yao531441 temporarily deployed to external_collaborator July 7, 2026 01:27 — with GitHub Actions Inactive
@yao531441
yao531441 requested a review from a team as a code owner July 22, 2026 00:52
@yao531441
yao531441 temporarily deployed to external_collaborator July 22, 2026 00:52 — with GitHub Actions Inactive
@yao531441
yao531441 temporarily deployed to external_collaborator July 22, 2026 01:10 — with GitHub Actions Inactive
yao531441 and others added 7 commits August 13, 2026 16:55
Add Kubernetes deployment templates for SGLang on Intel XPU GPUs:
- agg_xpu_dra.yaml: Aggregated deployment with DRA
- disagg_xpu_dra.yaml: Disaggregated deployment with DRA
- disagg_planner_xpu_dra.yaml: Disaggregated with Planner
- disagg_xpu.yaml: Traditional device plugin (non-DRA)

Key configurations for XPU disaggregated KV transfer:
- hostIPC: true for ZE_IPC shared memory
- UCX_TLS=ze_ipc,ze_copy,... for UCX transport
- ResourceClaimTemplate for GPU allocation via DRA

Aligns with vLLM XPU configs structure from PR ai-dynamo#9253.

Signed-off-by: Yao, Qing <qing.yao@intel.com>
Add agg_router_xpu_dra.yaml for KV-aware aggregated serving on Intel XPU:
- 2 workers behind KV router, each on separate GPU via DRA
- KV events via ZMQ for routing optimization
- XPU-specific flags: --device xpu, --disable-cuda-graph, --grammar-backend none

Aligns with existing XPU deployment configs structure.

Tested on XPU-817225 kind cluster (8x Intel Arc Pro B60).

Signed-off-by: Yao, Qing <qing.yao@intel.com>
Signed-off-by: Yao, Qing <qing.yao@intel.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Yao, Qing <qing.yao@intel.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Yao, Qing <qing.yao@intel.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Yao, Qing <qing.yao@intel.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Yao, Qing <qing.yao@intel.com>
pallavijaini0525 and others added 2 commits October 27, 2026 20:19
- remove disagg_planner_xpu_dra.yaml: forward pass metrics are off on XPU, so the
  Planner has no performance model and never scales; re-add once the XPU SGLang
  pin includes sgl-project/sglang#33739
- set --page-size 64; the intel_xpu attention backend accepts only 64 or 128

AICE-1084

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Signed-off-by: pallavi jaini <pallavi.jaini@intel.com>
@pallavijaini0525
pallavijaini0525 temporarily deployed to external_collaborator October 28, 2026 04:01 — with GitHub Actions Inactive
@pallavijaini0525
pallavijaini0525 temporarily deployed to external_collaborator October 28, 2026 16:05 — with GitHub Actions Inactive
@pallavijaini0525
pallavijaini0525 temporarily deployed to external_collaborator October 29, 2026 16:08 — with GitHub Actions Inactive
@dmitry-tokarev-nv

Copy link
Copy Markdown
Contributor

/ok to test cdb9601

@dmitry-tokarev-nv dmitry-tokarev-nv left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No P0, P1 or P2 found. This review read head cdb960121ca8a9f48f46d30b6e3739d3d97b4a89. It found three P3 items.

  1. [P3] The test command forwards to a Deployment, which does not exist on a cluster that runs Grove. examples/backends/sglang/deploy/xpu/README.md:85. See the inline comment, which has a suggestion.
  2. [P3] The table of differences does not name the page size (16 to 64). examples/backends/sglang/deploy/xpu/README.md:56. See the inline comment, which has a suggestion.
  3. [P3] The PR description is still out of date, as the review on 2026-10-26 said. It now has a Validation section. But it still names disagg_planner_xpu_dra.yaml, says "All five manifests", and says "The Planner template ships no profile data". This head has four manifests and no Planner template. Please update the description.

The image tag rule of the review on 2026-10-26 is fixed. Its thread has the measurement. The --skip-tokenizer-init question is answered: no worker sets the flag now.

Open items: 14 threads of other reviewers and bots are still unresolved. main requires resolved threads before merge.

What this review measured, and where it stopped.
  • Change since 24c592cd38: five content commits and five merges. Each merge equals a clean redo of its parents with git merge-tree. This head differs from main ebad592 in exactly the 6 files of this pull request (+488/-0).
  • Shell wrapper: this review rendered each component of the four manifests with the operator code of main, on the DCD path and on the Grove path. For all six workers, the command is ["/bin/bash", "-c"], and the script reaches the pod unchanged. The operator sets DYN_FORWARDPASS_METRIC_PORT=20380 on every worker (component_worker.go:114). If that variable is not empty, dynamo.sglang turns on forward pass metrics (components/src/dynamo/sglang/args.py:223). Bash ran each script with a stub python3: the variable was unset, and each argument list matched the manifest. A copy without the unset line kept 20380.
  • For one node, the SGLang backend of the operator adds no flag (backend_sglang.go:54). It changes nothing for routing or disaggregation. With two nodes, it puts --dist-init-addr, --nnodes and --node-rank right after python3 -m dynamo.sglang inside the script.
  • SGLang flags, read at v0.5.19: intel_xpu is a valid --attention-backend, and --page-size 64 is valid with it for Qwen3. All six workers set the same page size, attention backend, device and wrapper. Prefill and decode differ only in --disaggregation-mode.
  • CRD and webhook: the four DGD objects pass the v1beta1 schema, the CEL rules and strict decoding at main. The three ResourceClaimTemplate objects decode strictly as resource.k8s.io/v1. Six planted errors fail as expected, one each: a misspelled resourceClaims, a string replicas, an unknown component type, a misspelled deviceClassName, a string hostIPC, and a duplicate component name.
  • Storage: each worker requests 8Gi. Qwen/Qwen3-0.6B is 1,519,209,243 bytes (1.41 GiB) on Hugging Face.
  • README: container/render.py --framework=sglang --device=xpu --target=runtime writes container/sglang-runtime-xpu-amd64-rendered.Dockerfile, the name in the README. That image copies examples to /workspace/examples and puts the sglang environment first in PATH. The secret name and key match the six envFrom references. The model name in curl matches --served-model-name. docs_lint.py and detect_broken_links.py find 0 errors in the two READMEs, and each one catches a planted broken link. The two new links use github.com, and both return 200. At this head, only the vLLM XPU README names disagg_planner_xpu_dra.yaml, and it names its own file.
  • A merge of this head with main 9f88a47 has no conflict and differs from it only in the 6 files of this pull request. The one new commit on main changes only Rust crates.
  • CI on this head at 17:02Z: Pre Merge passed, including pre-commit and Recipe Check. It skipped Docs Lint (that job scans docs/ only), the Fern, operator and Rust jobs. Docs link check, Copyright Checks, codeowners and DCO passed. On pull-request/10023, Fern Docs passed, and PR-XPU passed with no test lane: its only lane, vLLM, was skipped. PR was still running, with 25 jobs passed, 44 skipped and 2 running. No CI job deploys these manifests, because the deploy tests read only examples/backends/<framework>/deploy/*.yaml.
  • Not tested: no cluster, no Intel GPU, no image build and no SGLang run. The torch.cuda.Event cause in the README note comes from reading SGLang v0.5.19 (python/sglang/srt/utils/device_timer.py:93), not from a run.

Co-authored-by: Dmitry Tokarev <dtokarev@nvidia.com>
Signed-off-by: Pallavi Jaini <pallavi.jaini@intel.com>
@pallavijaini0525
pallavijaini0525 temporarily deployed to external_collaborator October 29, 2026 17:06 — with GitHub Actions Inactive
Co-authored-by: Dmitry Tokarev <dtokarev@nvidia.com>
Signed-off-by: Pallavi Jaini <pallavi.jaini@intel.com>
@pallavijaini0525
pallavijaini0525 temporarily deployed to external_collaborator October 29, 2026 17:07 — with GitHub Actions Inactive

@dmitry-tokarev-nv dmitry-tokarev-nv left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No P0, P1 or P2 found at head 6622c853ec6de4dc904566a54c44930567db051b. The two new commits apply the two suggestions of my previous review as written: the test command now forwards to svc/sglang-agg-xpu-dra-frontend, and the table of differences names the page size. One P3 item is open: the PR description still names the removed Planner manifest. The image tag rule and the --skip-tokenizer-init question of the earlier reviews are fixed. The operator passes the bash -c wrapper to all six workers unchanged, and SGLang v0.5.19 accepts --page-size 64 with intel_xpu. No CI job deploys these manifests, and no Intel GPU ran them for this review. The unresolved threads of other reviewers must be resolved before merge.

@dmitry-tokarev-nv

Copy link
Copy Markdown
Contributor

/ok to test 6622c85

@dmitry-tokarev-nv
dmitry-tokarev-nv merged commit 389a509 into ai-dynamo:main Oct 29, 2026
111 checks passed
Jont828 added a commit to Jont828/dynamo that referenced this pull request Nov 4, 2026
Rebasing onto main brought in upstream's merged diffusion templates
(ai-dynamo#14660) and new example sources. Reflect them in the catalog:

- Describe the merged image, language-model, and Qwen3-TTS templates as
  validated on Dynamo 1.5.1, drop the obsolete SGLang
  shareProcessNamespace note, and explain the vLLM-Omni canary health
  check workaround. The main Qwen3-TTS template is no longer
  experimental; the experimental/ manifests keep their pinned builds.
- Add the SGLang Intel XPU DGDs (ai-dynamo#10023) to the aggregated, KV-routing,
  and disaggregated pages, the SGLang LoRA router launcher (ai-dynamo#15746) to
  LoRA Serving, and the vLLM transcription launcher (ai-dynamo#16208) to Audio and
  Realtime Serving.
- Note that the KVCR memory-service DGD now mounts a content-addressed
  startup ConfigMap that deploy.sh applies first.

Signed-off-by: Jonathan Tong <jt572@cornell.edu>
Jont828 added a commit to Jont828/dynamo that referenced this pull request Nov 4, 2026
Rebasing onto main brought in upstream's merged diffusion templates
(ai-dynamo#14660) and new example sources. Reflect them in the catalog:

- Describe the merged image, language-model, and Qwen3-TTS templates as
  validated on Dynamo 1.5.1, drop the obsolete SGLang
  shareProcessNamespace note, and explain the vLLM-Omni canary health
  check workaround. The main Qwen3-TTS template is no longer
  experimental; the experimental/ manifests keep their pinned builds.
- Add the SGLang Intel XPU DGDs (ai-dynamo#10023) to the aggregated, KV-routing,
  and disaggregated pages, the SGLang LoRA router launcher (ai-dynamo#15746) to
  LoRA Serving, and the vLLM transcription launcher (ai-dynamo#16208) to Audio and
  Realtime Serving.
- Note that the KVCR memory-service DGD now mounts a content-addressed
  startup ConfigMap that deploy.sh applies first.

Signed-off-by: Jonathan Tong <jt572@cornell.edu>
Jont828 added a commit to Jont828/dynamo that referenced this pull request Nov 6, 2026
Rebasing onto main brought in upstream's merged diffusion templates
(ai-dynamo#14660) and new example sources. Reflect them in the catalog:

- Describe the merged image, language-model, and Qwen3-TTS templates as
  validated on Dynamo 1.5.1, drop the obsolete SGLang
  shareProcessNamespace note, and explain the vLLM-Omni canary health
  check workaround. The main Qwen3-TTS template is no longer
  experimental; the experimental/ manifests keep their pinned builds.
- Add the SGLang Intel XPU DGDs (ai-dynamo#10023) to the aggregated, KV-routing,
  and disaggregated pages, the SGLang LoRA router launcher (ai-dynamo#15746) to
  LoRA Serving, and the vLLM transcription launcher (ai-dynamo#16208) to Audio and
  Realtime Serving.
- Note that the KVCR memory-service DGD now mounts a content-addressed
  startup ConfigMap that deploy.sh applies first.

Signed-off-by: Jonathan Tong <jt572@cornell.edu>
Jont828 added a commit to Jont828/dynamo that referenced this pull request Nov 6, 2026
Rebasing onto main brought in upstream's merged diffusion templates
(ai-dynamo#14660) and new example sources. Reflect them in the catalog:

- Describe the merged image, language-model, and Qwen3-TTS templates as
  validated on Dynamo 1.5.1, drop the obsolete SGLang
  shareProcessNamespace note, and explain the vLLM-Omni canary health
  check workaround. The main Qwen3-TTS template is no longer
  experimental; the experimental/ manifests keep their pinned builds.
- Add the SGLang Intel XPU DGDs (ai-dynamo#10023) to the aggregated, KV-routing,
  and disaggregated pages, the SGLang LoRA router launcher (ai-dynamo#15746) to
  LoRA Serving, and the vLLM transcription launcher (ai-dynamo#16208) to Audio and
  Realtime Serving.
- Note that the KVCR memory-service DGD now mounts a content-addressed
  startup ConfigMap that deploy.sh applies first.

Signed-off-by: Jonathan Tong <jt572@cornell.edu>
Jont828 added a commit to Jont828/dynamo that referenced this pull request Nov 6, 2026
Rebasing onto main brought in upstream's merged diffusion templates
(ai-dynamo#14660) and new example sources. Reflect them in the catalog:

- Describe the merged image, language-model, and Qwen3-TTS templates as
  validated on Dynamo 1.5.1, drop the obsolete SGLang
  shareProcessNamespace note, and explain the vLLM-Omni canary health
  check workaround. The main Qwen3-TTS template is no longer
  experimental; the experimental/ manifests keep their pinned builds.
- Add the SGLang Intel XPU DGDs (ai-dynamo#10023) to the aggregated, KV-routing,
  and disaggregated pages, the SGLang LoRA router launcher (ai-dynamo#15746) to
  LoRA Serving, and the vLLM transcription launcher (ai-dynamo#16208) to Audio and
  Realtime Serving.
- Note that the KVCR memory-service DGD now mounts a content-addressed
  startup ConfigMap that deploy.sh applies first.

Signed-off-by: Jonathan Tong <jt572@cornell.edu>
Jont828 added a commit to Jont828/dynamo that referenced this pull request Nov 9, 2026
Rebasing onto main brought in upstream's merged diffusion templates
(ai-dynamo#14660) and new example sources. Reflect them in the catalog:

- Describe the merged image, language-model, and Qwen3-TTS templates as
  validated on Dynamo 1.5.1, drop the obsolete SGLang
  shareProcessNamespace note, and explain the vLLM-Omni canary health
  check workaround. The main Qwen3-TTS template is no longer
  experimental; the experimental/ manifests keep their pinned builds.
- Add the SGLang Intel XPU DGDs (ai-dynamo#10023) to the aggregated, KV-routing,
  and disaggregated pages, the SGLang LoRA router launcher (ai-dynamo#15746) to
  LoRA Serving, and the vLLM transcription launcher (ai-dynamo#16208) to Audio and
  Realtime Serving.
- Note that the KVCR memory-service DGD now mounts a content-addressed
  startup ConfigMap that deploy.sh applies first.

Signed-off-by: Jonathan Tong <jt572@cornell.edu>
Jont828 added a commit to Jont828/dynamo that referenced this pull request Nov 10, 2026
Rebasing onto main brought in upstream's merged diffusion templates
(ai-dynamo#14660) and new example sources. Reflect them in the catalog:

- Describe the merged image, language-model, and Qwen3-TTS templates as
  validated on Dynamo 1.5.1, drop the obsolete SGLang
  shareProcessNamespace note, and explain the vLLM-Omni canary health
  check workaround. The main Qwen3-TTS template is no longer
  experimental; the experimental/ manifests keep their pinned builds.
- Add the SGLang Intel XPU DGDs (ai-dynamo#10023) to the aggregated, KV-routing,
  and disaggregated pages, the SGLang LoRA router launcher (ai-dynamo#15746) to
  LoRA Serving, and the vLLM transcription launcher (ai-dynamo#16208) to Audio and
  Realtime Serving.
- Note that the KVCR memory-service DGD now mounts a content-addressed
  startup ConfigMap that deploy.sh applies first.

Signed-off-by: Jonathan Tong <jt572@cornell.edu>
Jont828 added a commit to Jont828/dynamo that referenced this pull request Nov 11, 2026
Rebasing onto main brought in upstream's merged diffusion templates
(ai-dynamo#14660) and new example sources. Reflect them in the catalog:

- Describe the merged image, language-model, and Qwen3-TTS templates as
  validated on Dynamo 1.5.1, drop the obsolete SGLang
  shareProcessNamespace note, and explain the vLLM-Omni canary health
  check workaround. The main Qwen3-TTS template is no longer
  experimental; the experimental/ manifests keep their pinned builds.
- Add the SGLang Intel XPU DGDs (ai-dynamo#10023) to the aggregated, KV-routing,
  and disaggregated pages, the SGLang LoRA router launcher (ai-dynamo#15746) to
  LoRA Serving, and the vLLM transcription launcher (ai-dynamo#16208) to Audio and
  Realtime Serving.
- Note that the KVCR memory-service DGD now mounts a content-addressed
  startup ConfigMap that deploy.sh applies first.

Signed-off-by: Jonathan Tong <jt572@cornell.edu>
Jont828 added a commit to Jont828/dynamo that referenced this pull request Nov 15, 2026
Rebasing onto main brought in upstream's merged diffusion templates
(ai-dynamo#14660) and new example sources. Reflect them in the catalog:

- Describe the merged image, language-model, and Qwen3-TTS templates as
  validated on Dynamo 1.5.1, drop the obsolete SGLang
  shareProcessNamespace note, and explain the vLLM-Omni canary health
  check workaround. The main Qwen3-TTS template is no longer
  experimental; the experimental/ manifests keep their pinned builds.
- Add the SGLang Intel XPU DGDs (ai-dynamo#10023) to the aggregated, KV-routing,
  and disaggregated pages, the SGLang LoRA router launcher (ai-dynamo#15746) to
  LoRA Serving, and the vLLM transcription launcher (ai-dynamo#16208) to Audio and
  Realtime Serving.
- Note that the KVCR memory-service DGD now mounts a content-addressed
  startup ConfigMap that deploy.sh applies first.

Signed-off-by: Jonathan Tong <jt572@cornell.edu>
Jont828 added a commit to Jont828/dynamo that referenced this pull request Nov 17, 2026
Rebasing onto main brought in upstream's merged diffusion templates
(ai-dynamo#14660) and new example sources. Reflect them in the catalog:

- Describe the merged image, language-model, and Qwen3-TTS templates as
  validated on Dynamo 1.5.1, drop the obsolete SGLang
  shareProcessNamespace note, and explain the vLLM-Omni canary health
  check workaround. The main Qwen3-TTS template is no longer
  experimental; the experimental/ manifests keep their pinned builds.
- Add the SGLang Intel XPU DGDs (ai-dynamo#10023) to the aggregated, KV-routing,
  and disaggregated pages, the SGLang LoRA router launcher (ai-dynamo#15746) to
  LoRA Serving, and the vLLM transcription launcher (ai-dynamo#16208) to Audio and
  Realtime Serving.
- Note that the KVCR memory-service DGD now mounts a content-addressed
  startup ConfigMap that deploy.sh applies first.

Signed-off-by: Jonathan Tong <jt572@cornell.edu>

This branch was previously deployed

2 inactive (1 outdated) deployments
external_collaborator — 6622c853 Deployed Oct 29, 2026 by pallavijaini0525 via ok-to-test #41491
GITLAB — 93d6e696 Deployed Jun 9, 2026 by copy-pr-bot[bot] via Trigger CI Pipeline #30659
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backend::sglang Relates to the sglang backend backend::vllm Relates to the vllm backend documentation Improvements or additions to documentation external-contribution Pull request is from an external contributor feat size/L xpu

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants