Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 7 additions & 3 deletions .lycheeignore
Original file line number Diff line number Diff line change
Expand Up @@ -56,9 +56,13 @@
# (afterwards lychee validates the URL again, catching any later rename/move).
^https://github\.com/ai-dynamo/dynamo/blob/main/deploy/inference-gateway/ext-proc/examples/onramp/agg\.yaml

# The v1.4.0 release entry lands with the current-release promotion; the tag
# URL 404s from CI until v1.4.0 is cut. Remove after the tag exists.
^https://github\.com/ai-dynamo/dynamo/releases/tag/v1\.4\.0$
# The v1.5.0 release entry lands with the current-release promotion; the tag
# URL 404s from CI until v1.5.0 is cut. Remove after the tag exists.
^https://github\.com/ai-dynamo/dynamo/releases/tag/v1\.5\.0$

# The classic dynamo-platform 1.5.0 chart publishes to helm.ngc.nvidia.com in a
Comment thread
dagil-nvidia marked this conversation as resolved.
Resolved
# follow-up NGC publishing MR; the .tgz URL 404s until then. Remove once live.
^https://helm\.ngc\.nvidia\.com/nvidia/ai-dynamo/charts/dynamo-platform-1\.5\.0\.tgz$
Comment thread
dagil-nvidia marked this conversation as resolved.
Resolved

# The Dynamo Enterprise NGC collection is not published on the public catalog
# yet, so the canonical collection URL 404s from CI. The org and team segments
Expand Down
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -140,7 +140,7 @@ the Gateway API setup, supported features, and configuration.

```bash
# Pull a prebuilt container (SGLang example)
docker run --gpus all --network host --rm -it nvcr.io/nvidia/ai-dynamo/sglang-runtime:1.4.2
docker run --gpus all --network host --rm -it nvcr.io/nvidia/ai-dynamo/sglang-runtime:1.5.0

# Inside the container — start frontend and worker
python3 -m dynamo.frontend --http-port 8000 --discovery-backend file > /dev/null 2>&1 &
Expand All @@ -154,7 +154,7 @@ curl -s localhost:8000/v1/chat/completions -H "Content-Type: application/json" -
}' | jq
```

Also available: [`tensorrtllm-runtime:1.4.2`](https://docs.nvidia.com/dynamo/resources/release-artifacts) and [`vllm-runtime:1.4.2`](https://docs.nvidia.com/dynamo/resources/release-artifacts).
Also available: [`tensorrtllm-runtime:1.5.0`](https://docs.nvidia.com/dynamo/resources/release-artifacts) and [`vllm-runtime:1.5.0`](https://docs.nvidia.com/dynamo/resources/release-artifacts).

### Option B: Install from PyPI

Expand Down
4 changes: 2 additions & 2 deletions README.zh-CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -113,7 +113,7 @@ limitations under the License.

```bash
# 拉取预构建容器(SGLang 示例)
docker run --gpus all --network host --rm -it nvcr.io/nvidia/ai-dynamo/sglang-runtime:1.4.0
docker run --gpus all --network host --rm -it nvcr.io/nvidia/ai-dynamo/sglang-runtime:1.5.0

# 在容器内启动 frontend 和 worker
python3 -m dynamo.frontend --http-port 8000 --discovery-backend file > /dev/null 2>&1 &
Expand All @@ -127,7 +127,7 @@ curl -s localhost:8000/v1/chat/completions -H "Content-Type: application/json" -
}' | jq
```

另有 [`tensorrtllm-runtime:1.4.0`](https://docs.nvidia.com/dynamo/resources/release-artifacts) 和 [`vllm-runtime:1.4.0`](https://docs.nvidia.com/dynamo/resources/release-artifacts) 可用。
另有 [`tensorrtllm-runtime:1.5.0`](https://docs.nvidia.com/dynamo/resources/release-artifacts) 和 [`vllm-runtime:1.5.0`](https://docs.nvidia.com/dynamo/resources/release-artifacts) 可用。

### 选项 B:从 PyPI 安装

Expand Down
158 changes: 77 additions & 81 deletions docs/fern/components/releases.data.ts

Large diffs are not rendered by default.

3 changes: 3 additions & 0 deletions docs/fern/index.yml
Original file line number Diff line number Diff line change
Expand Up @@ -975,6 +975,9 @@ navigation:
path: pages/reference/general/releases/release-history.mdx
collapsed: true
contents:
- page: Dynamo v1.5.0
path: pages/reference/general/releases/dynamo-v1-5-0.mdx
slug: v1-5-0
- page: Dynamo v1.4.0
path: pages/reference/general/releases/dynamo-v1-4-0.mdx
slug: v1-4-0
Expand Down
4 changes: 2 additions & 2 deletions docs/fern/pages/cli/installation/install-dynamo.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -189,10 +189,10 @@ Ensure you passed `--gpus all` to `docker run`. Without this flag, the container

```bash
# Correct
docker run --gpus all --network host --rm -it nvcr.io/nvidia/ai-dynamo/sglang-runtime:1.4.0
docker run --gpus all --network host --rm -it nvcr.io/nvidia/ai-dynamo/sglang-runtime:1.5.0

# Wrong -- no GPU access
docker run --network host --rm -it nvcr.io/nvidia/ai-dynamo/sglang-runtime:1.4.0
docker run --network host --rm -it nvcr.io/nvidia/ai-dynamo/sglang-runtime:1.5.0
```

## Next Steps
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -313,7 +313,7 @@ spec:
value: /opt/models
extraPodSpec:
mainContainer:
image: nvcr.io/nvidia/ai-dynamo/vllm-runtime:1.4.0
image: nvcr.io/nvidia/ai-dynamo/vllm-runtime:1.5.0
imagePullPolicy: IfNotPresent

worker:
Expand All @@ -333,7 +333,7 @@ spec:
value: /opt/models
extraPodSpec:
mainContainer:
image: nvcr.io/nvidia/ai-dynamo/vllm-runtime:1.4.0
image: nvcr.io/nvidia/ai-dynamo/vllm-runtime:1.5.0
workingDir: /workspace
imagePullPolicy: IfNotPresent
command:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,7 @@ huggingface-cli download openai/gpt-oss-120b --exclude "original/*" --exclude "m

Set the container image:
```bash
export DYNAMO_CONTAINER_IMAGE=nvcr.io/nvidia/ai-dynamo/tensorrtllm-runtime:1.4.0
export DYNAMO_CONTAINER_IMAGE=nvcr.io/nvidia/ai-dynamo/tensorrtllm-runtime:1.5.0
```

Launch the Dynamo TensorRT-LLM container with the necessary configurations:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -74,9 +74,9 @@ ENTRYPOINT ["my-backend"]
When you deploy a custom image with a `DynamoGraphDeployment` (DGD) or standalone
`DynamoComponentDeployment` (DCD), admission determines the Dynamo runtime compatibility version
from the component's main-container image tag. The tag itself must be a semantic version. Tags such
as `1.4.0`, `v1.4.0`, and `1.4.0-cuda13` can provide the version. Set
as `1.5.0`, `v1.5.0`, and `1.5.0-cuda13` can provide the version. Set
`runtimeVersionOverride` when the image is tagless, digest-only, uses a tag such as `latest`,
`main`, `sha-abc`, or `cuda13-1.4.0`, or when a semantic-version tag does not identify the Dynamo
`main`, `sha-abc`, or `cuda13-1.5.0`, or when a semantic-version tag does not identify the Dynamo
runtime version packaged in the image.

Set the override to the canonical `MAJOR.MINOR.PATCH` Dynamo runtime version without a `v` prefix,
Expand All @@ -90,7 +90,7 @@ spec:
backendFramework: vllm
components:
- name: worker
runtimeVersionOverride: "1.4.0"
runtimeVersionOverride: "1.5.0"
podTemplate:
spec:
containers:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -81,7 +81,7 @@ docker run --gpus all -it --rm \
--ulimit nofile=65536:65536 \
--cap-add CAP_SYS_PTRACE --ipc host \
-v $HOME/.cache/huggingface:/home/dynamo/.cache/huggingface \
nvcr.io/nvidia/ai-dynamo/sglang-runtime:1.4.0
nvcr.io/nvidia/ai-dynamo/sglang-runtime:1.5.0
```

Mount the host Hugging Face cache (`-v $HOME/.cache/huggingface:/home/dynamo/.cache/huggingface`) so each container restart doesn't re-download model weights. The container runs as user `dynamo` (UID 1000), which is why the in-container path is `/home/dynamo/.cache/huggingface`.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -73,9 +73,9 @@ The matrix below shows which TensorRT-LLM feature pairs are validated to work to

| Container tag | Backend version | CUDA | Min NVIDIA driver |
|---|---|---|---|
| `tensorrtllm-runtime:1.4.0` | TRT-LLM `v1.3.0rc22` | `v13.1` | `580+` |
| `vllm-runtime:1.4.0` | vLLM `v0.26.0` | `v13.0` | `580+` |
| `sglang-runtime:1.4.0` | SGLang `v0.5.16` | `v13.0` | `580+` |
| `tensorrtllm-runtime:1.5.0` | TRT-LLM `v1.3.0rc25` | `v13.1` | `580+` |
| `vllm-runtime:1.5.0` | vLLM `v0.28.0` | `v13.0` | `580+` |
| `sglang-runtime:1.5.0` | SGLang `v0.5.18` | `v13.0` | `580+` |

Source of truth: [`docs/fern/pages/reference/general/compatibility.mdx`](../../../../../reference/general/compatibility.mdx#release-support-matrix) and [`docs/fern/pages/reference/general/release-artifacts.mdx`](../../../../../reference/general/release-artifacts.mdx). If those differ from the values above, the source-of-truth files win.

Expand All @@ -90,7 +90,7 @@ docker compose -f dev/docker-compose.yml up -d
**Step 2 (host terminal):** Pull and run the prebuilt container:

```bash
DYNAMO_VERSION=1.4.0
DYNAMO_VERSION=1.5.0
docker pull nvcr.io/nvidia/ai-dynamo/tensorrtllm-runtime:$DYNAMO_VERSION
docker run --gpus all -it --network host --ipc host \
nvcr.io/nvidia/ai-dynamo/tensorrtllm-runtime:$DYNAMO_VERSION
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -183,7 +183,7 @@ metadata:
spec:
model: meta-llama/Llama-3.3-70B-Instruct
backend: vllm
image: nvcr.io/nvidia/ai-dynamo/dynamo-planner:1.4.0 # dynamo-frontend for Dynamo < 1.1.0
image: nvcr.io/nvidia/ai-dynamo/dynamo-planner:1.5.0 # dynamo-frontend for Dynamo < 1.1.0
workload:
isl: 2048
osl: 256
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ metadata:
spec:
model: "Qwen/Qwen3-0.6B"
backend: vllm
image: "nvcr.io/nvidia/ai-dynamo/dynamo-planner:1.4.0" # dynamo-frontend for Dynamo < 1.1.0
image: "nvcr.io/nvidia/ai-dynamo/dynamo-planner:1.5.0" # dynamo-frontend for Dynamo < 1.1.0

workload:
isl: 3000 # Average input sequence length
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ metadata:
name: qwen-0-6b
spec:
model: "Qwen/Qwen3-0.6B"
image: "nvcr.io/nvidia/ai-dynamo/dynamo-planner:1.4.0" # dynamo-frontend for Dynamo < 1.1.0
image: "nvcr.io/nvidia/ai-dynamo/dynamo-planner:1.5.0" # dynamo-frontend for Dynamo < 1.1.0
```

### Dense Model: Thorough
Expand All @@ -35,7 +35,7 @@ metadata:
spec:
model: "Qwen/Qwen3-0.6B"
backend: vllm
image: "nvcr.io/nvidia/ai-dynamo/dynamo-planner:1.4.0" # dynamo-frontend for Dynamo < 1.1.0
image: "nvcr.io/nvidia/ai-dynamo/dynamo-planner:1.5.0" # dynamo-frontend for Dynamo < 1.1.0
searchStrategy: thorough
```

Expand All @@ -56,7 +56,7 @@ metadata:
spec:
model: "deepseek-ai/DeepSeek-R1"
backend: sglang
image: "nvcr.io/nvidia/ai-dynamo/dynamo-planner:1.4.0" # dynamo-frontend for Dynamo < 1.1.0
image: "nvcr.io/nvidia/ai-dynamo/dynamo-planner:1.5.0" # dynamo-frontend for Dynamo < 1.1.0

hardware:
numGpusPerNode: 8
Expand Down Expand Up @@ -86,7 +86,7 @@ metadata:
name: llama-private
spec:
model: "meta-llama/Llama-3.1-8B-Instruct"
image: "nvcr.io/nvidia/ai-dynamo/dynamo-planner:1.4.0" # dynamo-frontend for Dynamo < 1.1.0
image: "nvcr.io/nvidia/ai-dynamo/dynamo-planner:1.5.0" # dynamo-frontend for Dynamo < 1.1.0
```

### Custom SLA Targets
Expand All @@ -103,7 +103,7 @@ metadata:
name: low-latency-dense
spec:
model: "Qwen/Qwen3-0.6B"
image: "nvcr.io/nvidia/ai-dynamo/dynamo-planner:1.4.0" # dynamo-frontend for Dynamo < 1.1.0
image: "nvcr.io/nvidia/ai-dynamo/dynamo-planner:1.5.0" # dynamo-frontend for Dynamo < 1.1.0

sla:
ttft: 500 # Time To First Token target in milliseconds
Expand Down Expand Up @@ -137,7 +137,7 @@ metadata:
name: dense-with-tolerations
spec:
model: "Qwen/Qwen3-0.6B"
image: "nvcr.io/nvidia/ai-dynamo/dynamo-planner:1.4.0" # dynamo-frontend for Dynamo < 1.1.0
image: "nvcr.io/nvidia/ai-dynamo/dynamo-planner:1.5.0" # dynamo-frontend for Dynamo < 1.1.0

overrides:
profilingJob:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -209,7 +209,7 @@ the operator defaults it to

```yaml
spec:
image: "nvcr.io/nvidia/ai-dynamo/dynamo-planner:1.4.0" # dynamo-frontend for Dynamo < 1.1.0
image: "nvcr.io/nvidia/ai-dynamo/dynamo-planner:1.5.0" # dynamo-frontend for Dynamo < 1.1.0
```

> [!NOTE]
Expand Down Expand Up @@ -241,7 +241,7 @@ metadata:
spec:
model: "Qwen/Qwen3-0.6B"
backend: vllm
image: "nvcr.io/nvidia/ai-dynamo/dynamo-planner:1.4.0" # dynamo-frontend for Dynamo < 1.1.0
image: "nvcr.io/nvidia/ai-dynamo/dynamo-planner:1.5.0" # dynamo-frontend for Dynamo < 1.1.0
```

**Step 2: Apply the DGDR**
Expand Down Expand Up @@ -427,7 +427,7 @@ metadata:
spec:
model: "Qwen/Qwen3-0.6B"
backend: vllm
image: "nvcr.io/nvidia/ai-dynamo/dynamo-planner:1.4.0" # dynamo-frontend for Dynamo < 1.1.0
image: "nvcr.io/nvidia/ai-dynamo/dynamo-planner:1.5.0" # dynamo-frontend for Dynamo < 1.1.0

searchStrategy: rapid # or thorough
autoApply: true
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -271,7 +271,7 @@ deploy the reviewed snapshot, enable `autoApply`:

```bash
kubectl patch dgdr my-model -n <namespace> --type=merge \
-p '{"spec":{"runtimeVersionOverride":"1.4.0"}}'
-p '{"spec":{"runtimeVersionOverride":"1.5.0"}}'
kubectl patch dgdr my-model -n <namespace> --type=merge \
-p '{"spec":{"autoApply":true}}'
```
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -119,7 +119,7 @@ spec:
replicas: 1
extraPodSpec:
mainContainer:
image: nvcr.io/nvidia/ai-dynamo/vllm-runtime:1.4.0
image: nvcr.io/nvidia/ai-dynamo/vllm-runtime:1.5.0
imagePullPolicy: IfNotPresent
```

Expand Down Expand Up @@ -147,7 +147,7 @@ rather than a plain worker:
size: 16Gi
extraPodSpec:
mainContainer:
image: nvcr.io/nvidia/ai-dynamo/vllm-runtime:1.4.0
image: nvcr.io/nvidia/ai-dynamo/vllm-runtime:1.5.0
imagePullPolicy: IfNotPresent
command: ["python3", "-m", "dynamo.vllm"]
args:
Expand Down Expand Up @@ -184,7 +184,7 @@ rather than prompt load.
size: 16Gi
extraPodSpec:
mainContainer:
image: nvcr.io/nvidia/ai-dynamo/vllm-runtime:1.4.0
image: nvcr.io/nvidia/ai-dynamo/vllm-runtime:1.5.0
imagePullPolicy: IfNotPresent
command: ["python3", "-m", "dynamo.vllm"]
args:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ creates the Gateway in `NAMESPACE`. Keep the Dynamo version aligned with the pla
```bash
export NAMESPACE=my-model
export DYNAMO_SYSTEM_NAMESPACE=dynamo-system
export DYNAMO_VERSION=1.4.0
export DYNAMO_VERSION=1.5.0
export AGW_NAMESPACE=agentgateway-system
```

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -234,7 +234,7 @@ Then copy the new recipe's manifest and apply again.

This example deploys a disaggregated prefill/decode Dynamo Inference Graph that uses NIXL with the LIBFABRIC backend using [Elastic Fabric Adapter (EFA)](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/efa.html) for high-throughput KV-cache transfer between workers.

It targets `g7e.12xlarge` instances, which support GPUDirect RDMA, and uses the Dynamo EFA-enabled vLLM container `nvcr.io/nvidia/ai-dynamo/vllm-runtime:1.4.0-efa` that ships with the [EFA Installer](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/efa-changelog.html) pre-installed.
It targets `g7e.12xlarge` instances, which support GPUDirect RDMA, and uses the Dynamo EFA-enabled vLLM container `nvcr.io/nvidia/ai-dynamo/vllm-runtime:1.5.0-efa` that ships with the [EFA Installer](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/efa-changelog.html) pre-installed.

*Note: For a full list of EFA-supported instance types, see [the AWS EC2 Docs](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/efa.html#efa-instance-types).*

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@ git clone https://github.com/ai-dynamo/dynamo.git
cd dynamo

# Checkout the release branch matching your Dynamo platform version
git checkout release/1.4.0
git checkout release/1.5.0
```

</Step>
Expand Down Expand Up @@ -116,7 +116,7 @@ export PATH=$PATH:/usr/local/nvidia/bin:/usr/local/nvidia/lib64
/sbin/ldconfig
```

For example, refer to the [`release/1.4.0` disaggregated manifest](https://github.com/ai-dynamo/dynamo/blob/release/1.4.0/examples/deployments/GKE/vllm/v1beta1/disagg.yaml):
For example, refer to the [`release/1.5.0` disaggregated manifest](https://github.com/ai-dynamo/dynamo/blob/release/1.5.0/examples/deployments/GKE/vllm/disagg.yaml):
Comment thread
dagil-nvidia marked this conversation as resolved.
Resolved

```yaml
metadata:
Expand All @@ -133,7 +133,7 @@ spec:
export PATH=$PATH:/usr/local/nvidia/bin:/usr/local/nvidia/lib64
/sbin/ldconfig
python3 -m dynamo.vllm --model Qwen/Qwen3-0.6B --disaggregation-mode decode
image: nvcr.io/nvidia/ai-dynamo/vllm-runtime:1.4.0
image: nvcr.io/nvidia/ai-dynamo/vllm-runtime:1.5.0
name: main
resources:
limits:
Expand All @@ -143,7 +143,7 @@ spec:
</Step>
<Step title="Deploy the model" id="deploy-the-model">
```bash
cd examples/deployments/GKE/vllm/v1beta1
cd examples/deployments/GKE/vllm

kubectl apply -f disagg.yaml -n ${NAMESPACE}
```
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -221,21 +221,21 @@ The worker in the previous step runs **vLLM**. Dynamo also supports **SGLang** a
<Tab title="vLLM" language="vllm">

```bash
export RUNTIME_IMAGE=nvcr.io/nvidia/ai-dynamo/vllm-runtime:1.4.0
export RUNTIME_IMAGE=nvcr.io/nvidia/ai-dynamo/vllm-runtime:1.5.0
```

</Tab>
<Tab title="SGLang" language="sglang">

```bash
export RUNTIME_IMAGE=nvcr.io/nvidia/ai-dynamo/sglang-runtime:1.4.0
export RUNTIME_IMAGE=nvcr.io/nvidia/ai-dynamo/sglang-runtime:1.5.0
```

</Tab>
<Tab title="TensorRT-LLM" language="trtllm">

```bash
export RUNTIME_IMAGE=nvcr.io/nvidia/ai-dynamo/tensorrtllm-runtime:1.4.0
export RUNTIME_IMAGE=nvcr.io/nvidia/ai-dynamo/tensorrtllm-runtime:1.5.0
```

</Tab>
Expand Down
Loading
Loading