Skip to content
Merged
Show file tree
Hide file tree
Changes from 26 commits
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
8862943
docs: expand reinforcement learning guides
athreesh Aug 27, 2026
41649c4
docs: clarify verl variants and fix CI
athreesh Aug 27, 2026
51c5e9b
docs: simplify RL guide scope
athreesh Aug 27, 2026
429837f
docs: tighten RL guide boundaries
athreesh Aug 27, 2026
a1711e3
docs: refresh RL guides for latest main
athreesh Aug 27, 2026
ff77d50
docs: add NeMo RL integration guide
athreesh Aug 27, 2026
d6b9102
docs: refresh RL integration evidence
athreesh Aug 27, 2026
45cd677
docs: qualify RL backend failure boundaries
athreesh Aug 27, 2026
bdc23dd
docs: close verl recipe pin mismatch
athreesh Aug 27, 2026
098895c
docs: harden NeMo RL launch commands
athreesh Aug 27, 2026
6c6759b
docs: make NeMo Slurm variables executable
athreesh Aug 27, 2026
ac484d9
docs: close RL guide migration gaps
athreesh Aug 27, 2026
0d26372
docs: qualify RL router configuration precedence
athreesh Aug 27, 2026
bca9b0a
docs: refresh vllm sidecar evidence pin
athreesh Aug 27, 2026
e4a90fb
docs: track NeMo RL contract follow-ons
athreesh Aug 27, 2026
f8a914f
docs: track proposed verl NIXL path
athreesh Aug 27, 2026
4311180
docs: refresh Prime-RL integration evidence
athreesh Aug 27, 2026
6a1f50c
docs: qualify responses token estimate
athreesh Aug 27, 2026
c855703
docs: simplify RL guides
athreesh Aug 27, 2026
3515132
docs: add RL production proof points
athreesh Aug 28, 2026
e5a95c2
Merge remote-tracking branch 'origin/main' into docs/rl-documentation…
athreesh Aug 28, 2026
36c973b
docs: refine RL landing experience
athreesh Aug 28, 2026
1923abc
docs: add RL ownership architecture diagram
athreesh Aug 28, 2026
f850109
docs: tighten RL overview introduction
athreesh Aug 28, 2026
625e8d4
Merge remote-tracking branch 'origin/main' into docs/rl-documentation…
athreesh Aug 28, 2026
7fab4b8
docs: clarify external RL trainer contract
athreesh Aug 28, 2026
c0155e3
Merge remote-tracking branch 'origin/main' into docs/rl-documentation…
athreesh Aug 28, 2026
034cf6e
docs: address RL guide review feedback
athreesh Aug 28, 2026
536496d
docs: add RL Perfetto examples
athreesh Aug 28, 2026
deef949
docs: improve RL architecture diagram layout
athreesh Aug 31, 2026
350c116
docs: prevent RL diagram text overflow
athreesh Aug 31, 2026
10142b3
docs: standardize RL diagram cards
athreesh Aug 31, 2026
4f2144b
docs: refresh RL diagram asset URL
athreesh Aug 31, 2026
766aa88
docs: address remaining RL review feedback
athreesh Aug 31, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 13 additions & 1 deletion docs/fern/index.yml
Original file line number Diff line number Diff line change
Expand Up @@ -716,8 +716,20 @@ navigation:
contents:
- page: Overview
Comment thread
athreesh marked this conversation as resolved.
Resolved
path: pages/use-cases/reinforcement-learning/overview.md
- page: RL Implementation Guide
- page: verl Integration
Comment thread
athreesh marked this conversation as resolved.
Resolved
path: pages/use-cases/reinforcement-learning/verl.md
- page: NeMo RL Integration
path: pages/use-cases/reinforcement-learning/nemo-rl.md
- page: External Trainer Integration
path: pages/use-cases/reinforcement-learning/implementation-guide.md
- page: RL Integration Reference
path: pages/use-cases/reinforcement-learning/integration-reference.md
- page: KV-Aware Load Balancing
path: pages/use-cases/reinforcement-learning/routing.md
- page: Rollout Weight Updates
path: pages/use-cases/reinforcement-learning/weight-updates.md
- page: RL Profiling and Simulation
path: pages/use-cases/reinforcement-learning/operations-and-simulation.md
- section: Multimodal Serving
collapsed: true
icon: photo-film
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ title: ModelExpress
subtitle: Speed up model weight distribution across Kubernetes workers
---

ModelExpress is a model weight distribution service for faster worker startup in larger Dynamo clusters. Instead of every worker downloading the full model from storage, one worker can publish model weight availability and later workers can pull compatible tensors from that source over NIXL/RDMA. ModelExpress can also pair with ModelStreamer to stream safetensors directly from object storage into GPU memory.
ModelExpress is a model weight distribution service for faster worker startup in larger Dynamo clusters. Instead of every worker downloading the full model from storage, one worker can publish model weight availability and later workers can pull compatible tensors from that source through peer-to-peer (P2P) NIXL/RDMA. ModelExpress can also pair with ModelStreamer to stream safetensors directly from object storage into GPU memory, and its experimental RL refit client can stage live policy updates from trainer ranks, object storage, or updated inference peers.

Use ModelExpress when model rollout time, autoscale cold start, or fleet-wide model updates matter more than the simplicity of a shared PVC. For smaller clusters, start with [Model Caching](../../../../kubernetes/model-deployment/model-loading/model-caching.mdx).

Expand All @@ -23,7 +23,7 @@ Use ModelExpress when model rollout time, autoscale cold start, or fleet-wide mo
## How It Works

1. A ModelExpress server runs in the cluster and stores metadata for available model sources.
2. vLLM workers use the ModelExpress loader (`--load-format mx` on newer images, or `mx-source` / `mx-target` on older split-loader images).
2. vLLM workers use the ModelExpress loader (`--load-format modelexpress` with vLLM 0.23 and later).
3. If a compatible source worker is already serving the model, a new worker pulls model tensors from that source over NIXL/RDMA.
4. If no source is available, the worker falls back to storage. With ModelStreamer, the first worker can stream safetensors from `s3://`, `gs://`, `az://`, or a local path.
5. The Kubernetes operator can inject `MODEL_EXPRESS_URL` into all Dynamo pods from the platform `modelExpressURL` setting.
Expand Down Expand Up @@ -55,14 +55,14 @@ services:
- --model
- meta-llama/Llama-3.1-70B-Instruct
- --load-format
- mx
- modelexpress
env:
- name: VLLM_PLUGINS
value: modelexpress
```

> [!NOTE]
> Use the load format supported by your runtime image. ModelExpress v0.3 and newer document the unified `mx` loader. Some older Dynamo images expose `mx-source` and `mx-target` loader names instead.
> Use `--load-format modelexpress` for the current vLLM integration. The ModelExpress package retains `mx` as a backward-compatible alias; older runtime images can have different plugin requirements.
Comment thread
athreesh marked this conversation as resolved.
Resolved

## Stream Without Shared Storage

Expand Down Expand Up @@ -97,7 +97,7 @@ services:
- --model
- meta-llama/Llama-3.1-70B-Instruct
- --load-format
- mx
- modelexpress
env:
- name: VLLM_PLUGINS
value: modelexpress
Expand All @@ -116,8 +116,21 @@ services:

Credentials are consumed by the storage SDKs in the worker pod. They do not flow through the ModelExpress server.

## Use ModelExpress for RL Refit

The experimental `modelexpress_rl` client supports three distinct live-update source paths:

- Trainer ranks can publish their existing GPU shards for inference workers to pull and reshard over NIXL.
- An inference worker that has applied a version can publish it so a rank-compatible inference worker can stage it over ModelExpress P2P.
- An inference worker can prepare a canonical checkpoint from S3, including the current exact-base XOR delta format, before applying it through its engine adapter.

These paths move and stage weights; they do not decide when a policy is ready for rollouts. The RL framework or orchestrator must gate generation, select the target version and worker set, apply at a safe point, handle cache state, verify readiness, and recover from partial failure. The current [Dynamo vLLM refit example](https://github.com/ai-dynamo/modelexpress/tree/main/examples/rl/dynamo_vllm_refit) validates the full-weight trainer-to-inference path only; it does not validate the S3 delta path or every backend and topology.

See [Distribute and Update Rollout Weights](../../../../use-cases/reinforcement-learning/weight-updates.md) for the RL lifecycle and shared contract boundaries.

## See Also

- [Model Caching](../../../../kubernetes/model-deployment/model-loading/model-caching.mdx) - simple PVC-based model caching and the longer ModelExpress background.
- [ModelExpress deployment guide](https://github.com/ai-dynamo/modelexpress/blob/main/docs/DEPLOYMENT.md) - server, P2P, and ModelStreamer configuration.
- [ModelExpress RL refit package](https://github.com/ai-dynamo/modelexpress/tree/main/modelexpress_client/python/modelexpress_rl) - experimental trainer, inference-peer, and object-storage refit paths.
- [Installation Guide](../../../../kubernetes/installation/install-dynamo.md) - Dynamo platform install options, including `modelExpressURL`.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Loading