Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
42 commits
Select commit Hold shift + click to select a range
27791a4
feat(dynamo)!: serve any vLLM generation config through Dynamo's back…
jthomson04 Oct 24, 2026
08c50d2
fix(dynamo): address the #4905 review
jthomson04 Oct 25, 2026
2ba2cae
style(dynamo): sort the test's chat_routed_experts import
jthomson04 Oct 25, 2026
b065eb6
Merge upstream/main into jthomson04/dynamo-native-engine
jthomson04 Oct 26, 2026
94c0e96
build(gym): pin Gym#4817 rebased onto Gym main
jthomson04 Oct 26, 2026
e182811
fix(dynamo): apply the Gym Dynamo dialect per actor, after shard comp…
jthomson04 Oct 26, 2026
b498510
Merge upstream/main into jthomson04/dynamo-native-engine
jthomson04 Oct 27, 2026
db52655
Merge upstream/main into jthomson04/dynamo-native-engine
jthomson04 Oct 27, 2026
9223ffb
Merge upstream/main into jthomson04/dynamo-native-engine
jthomson04 Oct 27, 2026
b3cd664
Merge upstream/main into jthomson04/dynamo-native-engine
jthomson04 Oct 28, 2026
e0d3789
refactor(vllm): share the per-sample rollout pipeline and chat token …
jthomson04 Oct 29, 2026
89469ca
Merge jthomson04/vllm-request-pipeline into jthomson04/dynamo-native-…
jthomson04 Oct 29, 2026
d217dd8
refactor(dynamo): split NeMo-Gym support out to its own PR
jthomson04 Oct 29, 2026
3928525
refactor(vllm): serve every rollout through one worker method
jthomson04 Oct 29, 2026
1a2a581
Merge jthomson04/vllm-request-pipeline into jthomson04/dynamo-native-…
jthomson04 Oct 29, 2026
ed3ee35
refactor(dynamo): serve only NeMo RL rollouts; general OpenAI path mo…
jthomson04 Oct 29, 2026
83cab88
fix(dynamo): address round-2 review (C26, C28-C31, C33, C34, C36, C37…
jthomson04 Oct 29, 2026
1e73994
refactor(dynamo): debloat the Dynamo serving path
jthomson04 Oct 29, 2026
b5e19b1
Merge upstream/main into jthomson04/dynamo-native-engine
jthomson04 Oct 30, 2026
23f13fa
Merge upstream/main into jthomson04/dynamo-native-engine
jthomson04 Oct 30, 2026
8ae17af
Merge upstream/main into jthomson04/dynamo-native-engine
jthomson04 Nov 1, 2026
c2e4ef9
Merge upstream/main into jthomson04/vllm-request-pipeline
jthomson04 Nov 2, 2026
6a02b99
Merge jthomson04/vllm-request-pipeline (#4978) into jthomson04/dynamo…
jthomson04 Nov 2, 2026
0521cfc
Merge upstream/main into jthomson04/vllm-request-pipeline
jthomson04 Nov 3, 2026
e798c86
refactor(vllm): address review nits
jthomson04 Nov 3, 2026
7945239
Merge jthomson04/vllm-request-pipeline (#4978) into jthomson04/dynamo…
jthomson04 Nov 3, 2026
d67311f
style(vllm): sort check_sampling_param import
jthomson04 Nov 3, 2026
ea0ecb7
Merge jthomson04/vllm-request-pipeline (#4978) into jthomson04/dynamo…
jthomson04 Nov 3, 2026
d15b6f4
test(vllm): expect ValueError from the chat handler's sampling check
jthomson04 Nov 4, 2026
b0b6b32
Merge jthomson04/vllm-request-pipeline (#4978) into jthomson04/dynamo…
jthomson04 Nov 4, 2026
1fe249f
Merge upstream/main into jthomson04/vllm-request-pipeline
jthomson04 Nov 4, 2026
44671d3
Merge jthomson04/vllm-request-pipeline (#4978) into jthomson04/dynamo…
jthomson04 Nov 4, 2026
7688abe
Merge upstream/main into jthomson04/dynamo-native-engine
jthomson04 Nov 5, 2026
4cfab93
Merge upstream/main into jthomson04/dynamo-native-engine
jthomson04 Nov 6, 2026
ee813c3
fix(dynamo): address round-3 review
jthomson04 Nov 6, 2026
614fb0a
test(dynamo): cover the round-3 gaps; trim stale docstrings
jthomson04 Nov 6, 2026
f774780
refactor(dynamo): split the engine adapter into a shared Dynamo side …
jthomson04 Nov 7, 2026
096e872
docs(dynamo): scope the Dynamo guide to the SingleController
jthomson04 Nov 7, 2026
6f6de14
Merge upstream/main into jthomson04/dynamo-native-engine
jthomson04 Nov 7, 2026
c6a4c66
fix(dynamo): keep the generation worker's logging under Dynamo's Worker
jthomson04 Nov 9, 2026
4cbebb8
Merge upstream/main into jthomson04/dynamo-native-engine
jthomson04 Nov 10, 2026
9919396
Merge upstream/main into jthomson04/dynamo-native-engine
jthomson04 Nov 11, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/workflows/cicd-main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -238,8 +238,8 @@ jobs:
h100_functional_test_scripts=$(jq -c \
'[.functional[] | select((ascii_downcase | contains("gb200")) | not)]' \
<<< "$test_plan")
# The managed Dynamo runtime is currently amd64-only and is not
# installed in the arm64 GB200 image.
# The GB200 image is built without BUILD_DYNAMO, so it has no etcd or
# nats-server for the managed Dynamo runtime.
gb200_functional_test_scripts=$(jq -c \
'[.functional[] | select(. != "L1_Functional_Tests_Dynamo")]' \
<<< "$test_plan")
Expand Down
5 changes: 0 additions & 5 deletions .github/workflows/lockfile-check.yml
Original file line number Diff line number Diff line change
Expand Up @@ -26,8 +26,6 @@ on:
- "pyproject.toml"
- "uv.lock"
- "3rdparty/**"
- "docker/dynamo/pyproject.toml"
- "docker/dynamo/uv.lock"
- ".github/workflows/lockfile-check.yml"

jobs:
Expand All @@ -50,6 +48,3 @@ jobs:
# uv.lock is slow (~5 min); cached runs are fast.
- name: Check lockfile is up to date
run: uv lock --check

- name: Check Dynamo lockfile is up to date
run: uv lock --check --directory docker/dynamo
20 changes: 8 additions & 12 deletions docker/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,8 @@
# --build-arg SKIP_VLLM_BUILD=1 # Skip vLLM dependencies
# --build-arg SKIP_SGLANG_BUILD=1 # Skip SGLang dependencies
# --build-arg SKIP_TRTLLM_BUILD=1 # Skip TRT-LLM dependencies
# Optional isolated managed Dynamo runtime (Dynamo + vLLM only):
# Optional etcd and nats-server binaries for Dynamo serving
# (policy.generation.dynamo=true; ai-dynamo itself ships in the vLLM venvs):
# --build-arg BUILD_DYNAMO=1
# Optional: drop the uv cache from the image (smaller, but no offline `uv run --extra`):
# --build-arg STRIP_UV_CACHE=1
Expand Down Expand Up @@ -654,7 +655,6 @@ ARG BASE_IMAGE
ARG UV_VERSION
ARG HYBRID_EP_MULTINODE
ARG BUILD_DYNAMO
ARG DYNAMO_PYTHON_VERSION=3.12.11
ARG ETCD_VERSION=v3.5.21
ARG NATS_VERSION=v2.11.6
# Space-separated config paths whose NeMo Gym venvs are prefetched into the image
Expand All @@ -674,7 +674,7 @@ ENV NVIDIA_BUILD_REF=${NVIDIA_BUILD_REF:-<unknown>}
LABEL com.nvidia.build.id="${NVIDIA_BUILD_ID}"
LABEL com.nvidia.build.ref="${NVIDIA_BUILD_REF}"

ENV NEMO_RL_DYNAMO_VENV_DIR=/opt/dynamo_venv
ENV NEMO_RL_DYNAMO_BIN_DIR=/opt/dynamo/bin

# AWS EFA OFI plugin discovery (p4d / p5 / p5en multi-node NCCL via SRD).
# The host's aws-efa-installer places libnccl-net-ofi.so under
Expand All @@ -691,19 +691,15 @@ ENV LD_LIBRARY_PATH="/opt/amazon/ofi-nccl/lib:/opt/amazon/efa/lib:${LD_LIBRARY_P
# Exclude pyproject.toml and uv.lock since those may be altered by build-custom-vllm.sh
COPY --from=nemo-rl --exclude=pyproject.toml --exclude=uv.lock . /opt/nemo-rl

# Keep Dynamo's Python 3.12 dependency graph isolated from NeMo-RL's normal
# Ray/vLLM environments. Default image builds do not create this environment.
# The control-plane binaries Dynamo serving starts. Default image builds skip them.
# Not a uv-writing RUN (install.sh only downloads etcd and nats-server), so it
# needs no uv cache mount.
RUN <<"EOF" bash -exu -o pipefail
if [[ "${BUILD_DYNAMO:-0}" != "1" ]]; then
echo "BUILD_DYNAMO is not 1; skipping the managed Dynamo runtime"
echo "BUILD_DYNAMO is not 1; skipping the Dynamo service binaries"
exit 0
fi
# Dynamo syncs its own lock (Python 3.12) into a --link-mode copy venv that nothing re-syncs at
# runtime, so its wheels are never needed from the shared cache. A throwaway cache keeps them
# out of this layer instead of shipping them twice, and out of the durable cache mount, so this
# RUN needs neither the mount nor the uv_cache_mount.sh restore/persist.
UV_CACHE_DIR=/tmp/dynamo-uv-cache bash docker/dynamo/install.sh
rm -rf /tmp/dynamo-uv-cache
bash docker/dynamo/install.sh
EOF
# Unshallow the repo to get the full history (in the case it was from the scratch layer).
# Potentially not necessary if the repo is passed in as a complete repository (w/ full git history),
Expand Down
51 changes: 10 additions & 41 deletions docker/dynamo/install.sh
Original file line number Diff line number Diff line change
Expand Up @@ -13,13 +13,13 @@
# See the License for the specific language governing permissions and
# limitations under the License.

# Install the etcd and nats-server binaries the managed Dynamo services run.
# ai-dynamo itself ships in NeMo RL's vLLM worker venvs (the vllm extra), so
# Dynamo serves the same vLLM release NeMo RL pins; nothing here touches Python.

set -euo pipefail

script_dir=$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)
project_dir=${NEMO_RL_DYNAMO_PROJECT_DIR:-${script_dir}}
repo_root=$(realpath "${script_dir}/../..")
dynamo_venv_dir=${NEMO_RL_DYNAMO_VENV_DIR:-${repo_root}/venvs/dynamo}
dynamo_python_version=${DYNAMO_PYTHON_VERSION:-3.12.11}
bin_dir=${NEMO_RL_DYNAMO_BIN_DIR:-/opt/dynamo/bin}
etcd_version=${ETCD_VERSION:-v3.5.21}
nats_version=${NATS_VERSION:-v2.11.6}

Expand All @@ -42,36 +42,7 @@ case "${target_arch}" in
;;
esac

uv python install "${dynamo_python_version}"
uv venv --python "${dynamo_python_version}" "${dynamo_venv_dir}"
UV_PROJECT_ENVIRONMENT="${dynamo_venv_dir}" uv sync \
--directory "${project_dir}" \
--locked \
--no-dev \
--no-install-project \
--link-mode copy

dynamo_python=${dynamo_venv_dir}/bin/python
vllm_version=$("${dynamo_python}" -c \
'from importlib.metadata import version; print(version("vllm"))')
if [[ "${vllm_version}" != "0.28.0" ]]; then
echo "Expected vllm==0.28.0 from ai-dynamo[vllm]==1.5.1; got ${vllm_version}" >&2
exit 1
fi

# Layerwise weight reload must count only the elements each composed loader
# actually writes (vLLM PR #44814). Without it, trailing NemotronH/Mamba2
# parameters such as mixer.D stay unloaded after a refit and corrupt logits.
# vLLM 0.28.0 ships the fix; fail the build if a future pin drops it.
"${dynamo_python}" - <<'PY'
import inspect

import vllm.model_executor.model_loader.reload.meta as meta

if "min(numel," not in inspect.getsource(meta):
raise SystemExit("vLLM is missing the PR #44814 layerwise-reload fix")
PY

mkdir -p "${bin_dir}"
download_dir=$(mktemp -d "${TMPDIR:-/tmp}/nemorl-dynamo-install.XXXXXX")
trap 'rm -rf "${download_dir}"' EXIT

Expand All @@ -81,17 +52,15 @@ curl --fail --location --retry 3 \
tar -xzf "${download_dir}/etcd.tgz" -C "${download_dir}"
install -m 0755 \
"${download_dir}/etcd-${etcd_version}-linux-${target_arch}/etcd" \
"${dynamo_venv_dir}/bin/etcd"
"${bin_dir}/etcd"

curl --fail --location --retry 3 \
"https://github.com/nats-io/nats-server/releases/download/${nats_version}/nats-server-${nats_version}-linux-${target_arch}.tar.gz" \
--output "${download_dir}/nats.tgz"
tar -xzf "${download_dir}/nats.tgz" -C "${download_dir}"
install -m 0755 \
"${download_dir}/nats-server-${nats_version}-linux-${target_arch}/nats-server" \
"${dynamo_venv_dir}/bin/nats-server"
"${bin_dir}/nats-server"

"${dynamo_python}" -c \
'import importlib.metadata as m; assert m.version("ai-dynamo") == "1.5.1"; assert m.version("vllm") == "0.28.0"; assert m.version("nvidia-nccl-cu13") == "2.30.7"'
"${dynamo_venv_dir}/bin/etcd" --version
"${dynamo_venv_dir}/bin/nats-server" --version
"${bin_dir}/etcd" --version
"${bin_dir}/nats-server" --version
9 changes: 0 additions & 9 deletions docker/dynamo/pyproject.toml

This file was deleted.

Loading
Loading