Skip to content
Merged
Show file tree
Hide file tree
Changes from 5 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 2 additions & 6 deletions .lycheeignore
Original file line number Diff line number Diff line change
Expand Up @@ -60,13 +60,9 @@
# (afterwards lychee validates the URL again, catching any later rename/move).
^https://github\.com/ai-dynamo/dynamo/blob/main/deploy/inference-gateway/ext-proc/examples/onramp/agg\.yaml

# The v1.5.0 release entry lands with the current-release promotion; the tag
# URL 404s from CI until v1.5.0 is cut. Remove after the tag exists.
^https://github\.com/ai-dynamo/dynamo/releases/tag/v1\.5\.0$

# The classic dynamo-platform 1.5.0 chart publishes to helm.ngc.nvidia.com in a
# The classic dynamo-platform 1.6.0 chart publishes to helm.ngc.nvidia.com in a
# follow-up NGC publishing MR; the .tgz URL 404s until then. Remove once live.
^https://helm\.ngc\.nvidia\.com/nvidia/ai-dynamo/charts/dynamo-platform-1\.5\.0\.tgz$
^https://helm\.ngc\.nvidia\.com/nvidia/ai-dynamo/charts/dynamo-platform-1\.6\.0\.tgz$

# The Dynamo Enterprise NGC collection is not published on the public catalog
# yet, so the canonical collection URL 404s from CI. The org and team segments
Expand Down
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -144,7 +144,7 @@ Choose either vLLM or SGLang:

```bash
# Pull a prebuilt container
docker run --gpus all --network host --rm -it nvcr.io/nvidia/ai-dynamo/vllm-runtime:1.5.1
docker run --gpus all --network host --rm -it nvcr.io/nvidia/ai-dynamo/vllm-runtime:1.6.0

# Inside the container — start frontend and worker
python3 -m dynamo.frontend --http-port 8000 --discovery-backend file > /dev/null 2>&1 &
Expand All @@ -162,7 +162,7 @@ curl -s localhost:8000/v1/chat/completions -H "Content-Type: application/json" -

```bash
# Pull a prebuilt container
docker run --gpus all --network host --rm -it nvcr.io/nvidia/ai-dynamo/sglang-runtime:1.5.1
docker run --gpus all --network host --rm -it nvcr.io/nvidia/ai-dynamo/sglang-runtime:1.6.0

# Inside the container — start frontend and worker
python3 -m dynamo.frontend --http-port 8000 --discovery-backend file > /dev/null 2>&1 &
Expand All @@ -176,7 +176,7 @@ curl -s localhost:8000/v1/chat/completions -H "Content-Type: application/json" -
}' | jq
```

Also available: [`tensorrtllm-runtime:1.5.1`](https://docs.nvidia.com/dynamo/resources/release-artifacts).
Also available: [`tensorrtllm-runtime:1.6.0`](https://docs.nvidia.com/dynamo/resources/release-artifacts).

### Option B: Install from PyPI

Expand Down
4 changes: 2 additions & 2 deletions README.zh-CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -113,7 +113,7 @@ limitations under the License.

```bash
# 拉取预构建容器(SGLang 示例)
docker run --gpus all --network host --rm -it nvcr.io/nvidia/ai-dynamo/sglang-runtime:1.5.0
docker run --gpus all --network host --rm -it nvcr.io/nvidia/ai-dynamo/sglang-runtime:1.6.0

# 在容器内启动 frontend 和 worker
python3 -m dynamo.frontend --http-port 8000 --discovery-backend file > /dev/null 2>&1 &
Expand All @@ -127,7 +127,7 @@ curl -s localhost:8000/v1/chat/completions -H "Content-Type: application/json" -
}' | jq
```

另有 [`tensorrtllm-runtime:1.5.0`](https://docs.nvidia.com/dynamo/resources/release-artifacts) 和 [`vllm-runtime:1.5.0`](https://docs.nvidia.com/dynamo/resources/release-artifacts) 可用。
另有 [`tensorrtllm-runtime:1.6.0`](https://docs.nvidia.com/dynamo/resources/release-artifacts) 和 [`vllm-runtime:1.6.0`](https://docs.nvidia.com/dynamo/resources/release-artifacts) 可用。

### 选项 B:从 PyPI 安装

Expand Down
146 changes: 87 additions & 59 deletions docs/fern/components/releases.data.ts

Large diffs are not rendered by default.

3 changes: 3 additions & 0 deletions docs/fern/index.yml
Original file line number Diff line number Diff line change
Expand Up @@ -1082,6 +1082,9 @@ navigation:
path: pages/reference/general/releases/release-history.mdx
collapsed: true
contents:
- page: Dynamo v1.6.0
path: pages/reference/general/releases/dynamo-v1-6-0.mdx
slug: v1-6-0
- page: Dynamo v1.5.0
path: pages/reference/general/releases/dynamo-v1-5-0.mdx
slug: v1-5-0
Expand Down
4 changes: 2 additions & 2 deletions docs/fern/pages/cli/installation/install-dynamo.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -189,10 +189,10 @@ Ensure you passed `--gpus all` to `docker run`. Without this flag, the container

```bash
# Correct
docker run --gpus all --network host --rm -it nvcr.io/nvidia/ai-dynamo/sglang-runtime:1.5.1
docker run --gpus all --network host --rm -it nvcr.io/nvidia/ai-dynamo/sglang-runtime:1.6.0

# Wrong -- no GPU access
docker run --network host --rm -it nvcr.io/nvidia/ai-dynamo/sglang-runtime:1.5.1
docker run --network host --rm -it nvcr.io/nvidia/ai-dynamo/sglang-runtime:1.6.0
```

## Next Steps
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -315,7 +315,7 @@ spec:
value: /opt/models
extraPodSpec:
mainContainer:
image: nvcr.io/nvidia/ai-dynamo/vllm-runtime:1.5.0
image: nvcr.io/nvidia/ai-dynamo/vllm-runtime:1.6.0
imagePullPolicy: IfNotPresent

worker:
Expand All @@ -335,7 +335,7 @@ spec:
value: /opt/models
extraPodSpec:
mainContainer:
image: nvcr.io/nvidia/ai-dynamo/vllm-runtime:1.5.0
image: nvcr.io/nvidia/ai-dynamo/vllm-runtime:1.6.0
workingDir: /workspace
imagePullPolicy: IfNotPresent
command:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,7 @@ huggingface-cli download openai/gpt-oss-120b --exclude "original/*" --exclude "m

Set the container image:
```bash
export DYNAMO_CONTAINER_IMAGE=nvcr.io/nvidia/ai-dynamo/tensorrtllm-runtime:1.5.0
export DYNAMO_CONTAINER_IMAGE=nvcr.io/nvidia/ai-dynamo/tensorrtllm-runtime:1.6.0
```

Launch the Dynamo TensorRT-LLM container with the necessary configurations:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -74,9 +74,9 @@ ENTRYPOINT ["my-backend"]
When you deploy a custom image with a `DynamoGraphDeployment` (DGD) or standalone
`DynamoComponentDeployment` (DCD), admission determines the Dynamo runtime version
from the component's main-container image tag. The tag itself must be a semantic version. Tags such
as `1.5.0`, `v1.5.0`, and `1.5.0-cuda13` can provide the version. Set
as `1.6.0`, `v1.6.0`, and `1.6.0-cuda13` can provide the version. Set
`runtimeVersionOverride` when the image is tagless, digest-only, uses a tag such as `latest`,
`main`, `sha-abc`, or `cuda13-1.5.0`, or when a semantic-version tag does not identify the Dynamo
`main`, `sha-abc`, or `cuda13-1.6.0`, or when a semantic-version tag does not identify the Dynamo
runtime version packaged in the image.

Set the override to the canonical `MAJOR.MINOR.PATCH` Dynamo runtime version without a `v` prefix,
Expand All @@ -90,7 +90,7 @@ spec:
backendFramework: vllm
components:
- name: worker
runtimeVersionOverride: "1.5.0"
runtimeVersionOverride: "1.6.0"
podTemplate:
spec:
containers:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -81,7 +81,7 @@ docker run --gpus all -it --rm \
--ulimit nofile=65536:65536 \
--cap-add CAP_SYS_PTRACE --ipc host \
-v $HOME/.cache/huggingface:/home/dynamo/.cache/huggingface \
nvcr.io/nvidia/ai-dynamo/sglang-runtime:1.5.0
nvcr.io/nvidia/ai-dynamo/sglang-runtime:1.6.0
```

Mount the host Hugging Face cache (`-v $HOME/.cache/huggingface:/home/dynamo/.cache/huggingface`) so each container restart doesn't re-download model weights. The container runs as user `dynamo` (UID 1000), which is why the in-container path is `/home/dynamo/.cache/huggingface`.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -73,9 +73,9 @@ The matrix below shows which TensorRT-LLM feature pairs are validated to work to

| Container tag | Backend version | CUDA | Min NVIDIA driver |
|---|---|---|---|
| `tensorrtllm-runtime:1.5.0` | TRT-LLM `v1.3.0rc25` | `v13.1` | `580+` |
| `vllm-runtime:1.5.0` | vLLM `v0.28.0` | `v13.0` | `580+` |
| `sglang-runtime:1.5.0` | SGLang `v0.5.18` | `v13.0` | `580+` |
| `tensorrtllm-runtime:1.6.0` | TRT-LLM `v1.3.0rc29` | `v13.1` | `580+` |
| `vllm-runtime:1.6.0` | vLLM `v0.31.0` | `v13.0` | `580+` |
| `sglang-runtime:1.6.0` | SGLang `v0.5.21` | `v13.0` | `580+` |

Source of truth: [`docs/fern/pages/reference/general/compatibility.mdx`](../../../../../reference/general/compatibility.mdx#release-support-matrix) and [`docs/fern/pages/reference/general/release-artifacts.mdx`](../../../../../reference/general/release-artifacts.mdx). If those differ from the values above, the source-of-truth files win.

Expand All @@ -90,7 +90,7 @@ docker compose -f dev/docker-compose.yml up -d
**Step 2 (host terminal):** Pull and run the prebuilt container:

```bash
DYNAMO_VERSION=1.5.0
DYNAMO_VERSION=1.6.0
docker pull nvcr.io/nvidia/ai-dynamo/tensorrtllm-runtime:$DYNAMO_VERSION
docker run --gpus all -it --network host --ipc host \
nvcr.io/nvidia/ai-dynamo/tensorrtllm-runtime:$DYNAMO_VERSION
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -183,7 +183,7 @@ metadata:
spec:
model: meta-llama/Llama-3.3-70B-Instruct
backend: vllm
image: nvcr.io/nvidia/ai-dynamo/dynamo-planner:1.5.0 # dynamo-frontend for Dynamo < 1.1.0
image: nvcr.io/nvidia/ai-dynamo/dynamo-planner:1.6.0 # dynamo-frontend for Dynamo < 1.1.0
workload:
isl: 2048
osl: 256
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ metadata:
spec:
model: "Qwen/Qwen3-0.6B"
backend: vllm
image: "nvcr.io/nvidia/ai-dynamo/dynamo-planner:1.5.0" # dynamo-frontend for Dynamo < 1.1.0
image: "nvcr.io/nvidia/ai-dynamo/dynamo-planner:1.6.0" # dynamo-frontend for Dynamo < 1.1.0

workload:
isl: 3000 # Average input sequence length
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ metadata:
name: qwen-0-6b
spec:
model: "Qwen/Qwen3-0.6B"
image: "nvcr.io/nvidia/ai-dynamo/dynamo-planner:1.5.0" # dynamo-frontend for Dynamo < 1.1.0
image: "nvcr.io/nvidia/ai-dynamo/dynamo-planner:1.6.0" # dynamo-frontend for Dynamo < 1.1.0
```

### Dense Model: Thorough
Expand All @@ -35,7 +35,7 @@ metadata:
spec:
model: "Qwen/Qwen3-0.6B"
backend: vllm
image: "nvcr.io/nvidia/ai-dynamo/dynamo-planner:1.5.0" # dynamo-frontend for Dynamo < 1.1.0
image: "nvcr.io/nvidia/ai-dynamo/dynamo-planner:1.6.0" # dynamo-frontend for Dynamo < 1.1.0
searchStrategy: thorough
```

Expand All @@ -56,7 +56,7 @@ metadata:
spec:
model: "deepseek-ai/DeepSeek-R1"
backend: sglang
image: "nvcr.io/nvidia/ai-dynamo/dynamo-planner:1.5.0" # dynamo-frontend for Dynamo < 1.1.0
image: "nvcr.io/nvidia/ai-dynamo/dynamo-planner:1.6.0" # dynamo-frontend for Dynamo < 1.1.0

hardware:
numGpusPerNode: 8
Expand Down Expand Up @@ -86,7 +86,7 @@ metadata:
name: llama-private
spec:
model: "meta-llama/Llama-3.1-8B-Instruct"
image: "nvcr.io/nvidia/ai-dynamo/dynamo-planner:1.5.0" # dynamo-frontend for Dynamo < 1.1.0
image: "nvcr.io/nvidia/ai-dynamo/dynamo-planner:1.6.0" # dynamo-frontend for Dynamo < 1.1.0
```

### Custom SLA Targets
Expand All @@ -103,7 +103,7 @@ metadata:
name: low-latency-dense
spec:
model: "Qwen/Qwen3-0.6B"
image: "nvcr.io/nvidia/ai-dynamo/dynamo-planner:1.5.0" # dynamo-frontend for Dynamo < 1.1.0
image: "nvcr.io/nvidia/ai-dynamo/dynamo-planner:1.6.0" # dynamo-frontend for Dynamo < 1.1.0

sla:
ttft: 500 # Time To First Token target in milliseconds
Expand Down Expand Up @@ -137,7 +137,7 @@ metadata:
name: dense-with-tolerations
spec:
model: "Qwen/Qwen3-0.6B"
image: "nvcr.io/nvidia/ai-dynamo/dynamo-planner:1.5.0" # dynamo-frontend for Dynamo < 1.1.0
image: "nvcr.io/nvidia/ai-dynamo/dynamo-planner:1.6.0" # dynamo-frontend for Dynamo < 1.1.0

overrides:
profilingJob:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -209,7 +209,7 @@ the operator defaults it to

```yaml
spec:
image: "nvcr.io/nvidia/ai-dynamo/dynamo-planner:1.5.0" # dynamo-frontend for Dynamo < 1.1.0
image: "nvcr.io/nvidia/ai-dynamo/dynamo-planner:1.6.0" # dynamo-frontend for Dynamo < 1.1.0
```

> [!NOTE]
Expand Down Expand Up @@ -241,7 +241,7 @@ metadata:
spec:
model: "Qwen/Qwen3-0.6B"
backend: vllm
image: "nvcr.io/nvidia/ai-dynamo/dynamo-planner:1.5.0" # dynamo-frontend for Dynamo < 1.1.0
image: "nvcr.io/nvidia/ai-dynamo/dynamo-planner:1.6.0" # dynamo-frontend for Dynamo < 1.1.0
```

**Step 2: Apply the DGDR**
Expand Down Expand Up @@ -427,7 +427,7 @@ metadata:
spec:
model: "Qwen/Qwen3-0.6B"
backend: vllm
image: "nvcr.io/nvidia/ai-dynamo/dynamo-planner:1.5.0" # dynamo-frontend for Dynamo < 1.1.0
image: "nvcr.io/nvidia/ai-dynamo/dynamo-planner:1.6.0" # dynamo-frontend for Dynamo < 1.1.0

searchStrategy: rapid # or thorough
autoApply: true
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -271,7 +271,7 @@ deploy the reviewed snapshot, enable `autoApply`:

```bash
kubectl patch dgdr my-model -n <namespace> --type=merge \
-p '{"spec":{"runtimeVersionOverride":"1.5.0"}}'
-p '{"spec":{"runtimeVersionOverride":"1.6.0"}}'
kubectl patch dgdr my-model -n <namespace> --type=merge \
-p '{"spec":{"autoApply":true}}'
```
Expand Down
6 changes: 3 additions & 3 deletions docs/fern/pages/kubernetes/disaggregated-serving/overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -119,7 +119,7 @@ spec:
replicas: 1
extraPodSpec:
mainContainer:
image: nvcr.io/nvidia/ai-dynamo/vllm-runtime:1.5.0
image: nvcr.io/nvidia/ai-dynamo/vllm-runtime:1.6.0
imagePullPolicy: IfNotPresent
```

Expand Down Expand Up @@ -147,7 +147,7 @@ rather than a plain worker:
size: 16Gi
extraPodSpec:
mainContainer:
image: nvcr.io/nvidia/ai-dynamo/vllm-runtime:1.5.0
image: nvcr.io/nvidia/ai-dynamo/vllm-runtime:1.6.0
imagePullPolicy: IfNotPresent
command: ["python3", "-m", "dynamo.vllm"]
args:
Expand Down Expand Up @@ -184,7 +184,7 @@ rather than prompt load.
size: 16Gi
extraPodSpec:
mainContainer:
image: nvcr.io/nvidia/ai-dynamo/vllm-runtime:1.5.0
image: nvcr.io/nvidia/ai-dynamo/vllm-runtime:1.6.0
imagePullPolicy: IfNotPresent
command: ["python3", "-m", "dynamo.vllm"]
args:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ creates the Gateway in `NAMESPACE`. Keep the Dynamo version aligned with the pla
```bash
export NAMESPACE=my-model
export DYNAMO_SYSTEM_NAMESPACE=dynamo-system
export DYNAMO_VERSION=1.5.0
export DYNAMO_VERSION=1.6.0
export AGW_NAMESPACE=agentgateway-system
```

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -234,7 +234,7 @@ Then copy the new recipe's manifest and apply again.

This example deploys a disaggregated prefill/decode Dynamo Inference Graph that uses NIXL with the LIBFABRIC backend using [Elastic Fabric Adapter (EFA)](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/efa.html) for high-throughput KV-cache transfer between workers.

It targets `g7e.12xlarge` instances, which support GPUDirect RDMA, and uses the Dynamo EFA-enabled vLLM container `nvcr.io/nvidia/ai-dynamo/vllm-runtime:1.5.0-efa` that ships with the [EFA Installer](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/efa-changelog.html) pre-installed.
It targets `g7e.12xlarge` instances, which support GPUDirect RDMA, and uses the Dynamo EFA-enabled vLLM container `nvcr.io/nvidia/ai-dynamo/vllm-runtime:1.6.0-efa` that ships with the [EFA Installer](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/efa-changelog.html) pre-installed.

*Note: For a full list of EFA-supported instance types, see [the AWS EC2 Docs](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/efa.html#efa-instance-types).*

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@ git clone https://github.com/ai-dynamo/dynamo.git
cd dynamo

# Checkout the release branch matching your Dynamo platform version
git checkout release/1.5.0
git checkout release/1.6.0
```

</Step>
Expand Down Expand Up @@ -116,7 +116,7 @@ export PATH=$PATH:/usr/local/nvidia/bin:/usr/local/nvidia/lib64
/sbin/ldconfig
```

For example, refer to the [`release/1.5.0` disaggregated manifest](https://github.com/ai-dynamo/dynamo/blob/release/1.5.0/examples/deployments/GKE/vllm/disagg.yaml):
For example, refer to the [`release/1.6.0` disaggregated manifest](https://github.com/ai-dynamo/dynamo/blob/release/1.6.0/examples/deployments/GKE/vllm/disagg.yaml):

```yaml
metadata:
Expand All @@ -133,7 +133,7 @@ spec:
export PATH=$PATH:/usr/local/nvidia/bin:/usr/local/nvidia/lib64
/sbin/ldconfig
python3 -m dynamo.vllm --model Qwen/Qwen3-0.6B --disaggregation-mode decode
image: nvcr.io/nvidia/ai-dynamo/vllm-runtime:1.5.0
image: nvcr.io/nvidia/ai-dynamo/vllm-runtime:1.6.0
name: main
resources:
limits:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -221,21 +221,21 @@ The worker in the previous step runs **vLLM**. Dynamo also supports **SGLang** a
<Tab title="vLLM" language="vllm">

```bash
export RUNTIME_IMAGE=nvcr.io/nvidia/ai-dynamo/vllm-runtime:1.5.0
export RUNTIME_IMAGE=nvcr.io/nvidia/ai-dynamo/vllm-runtime:1.6.0
```

</Tab>
<Tab title="SGLang" language="sglang">

```bash
export RUNTIME_IMAGE=nvcr.io/nvidia/ai-dynamo/sglang-runtime:1.5.0
export RUNTIME_IMAGE=nvcr.io/nvidia/ai-dynamo/sglang-runtime:1.6.0
```

</Tab>
<Tab title="TensorRT-LLM" language="trtllm">

```bash
export RUNTIME_IMAGE=nvcr.io/nvidia/ai-dynamo/tensorrtllm-runtime:1.5.0
export RUNTIME_IMAGE=nvcr.io/nvidia/ai-dynamo/tensorrtllm-runtime:1.6.0
```

</Tab>
Expand Down
Loading
Loading