Skip to content

Commit b268a95

Browse files
committed
fix(multimodal): gate vLLM RDMA emit on exporter presence, not request mode
resolve_mm_rdma_enabled required the per-request transport mode to be `rdma`, but the exporter is built whenever the lane is enabled — including via the legacy `SMG_MM_PIXEL_RDMA` flag, which leaves the mode at the default `inline`. That combination built the exporter and let the worker advertise supports_rdma_pull=true, yet the gateway never emitted `remote` payloads. Gate purely on the exporter being present plus the worker's capability label, matching TokenSpeed (which emits whenever the exporter is up) and honoring every way the lane is enabled. The capability label is the per-worker opt-out. Drop the bogus `_mode` label from the test (mode is no longer read; the exporter gate is now exercised directly). (Codex, CodeRabbit) Signed-off-by: Simo Lin <25425177+slin1237@users.noreply.github.com>
1 parent fca7eb8 commit b268a95

1 file changed

Lines changed: 11 additions & 12 deletions

File tree

model_gateway/src/routers/grpc/multimodal/transport.rs

Lines changed: 11 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -155,17 +155,17 @@ pub(super) fn resolve_mm_shm_min_bytes(workers: Option<&WorkerSelection>) -> usi
155155
}
156156

157157
/// Resolve whether vLLM `pixel_values` may use the RDMA lane for this request: the
158-
/// resolved transport mode is `rdma`, the gateway exporter is up, and the worker
159-
/// advertises it can pull. The capability gate keeps SMG from emitting a `remote`
160-
/// payload to a worker that would reject it. The exporter is a process-wide
161-
/// resource (NIXL agent + arena), so a per-worker mode override can gate RDMA off
162-
/// but cannot turn it on without the router-level transport mode / env.
158+
/// gateway exporter is up and the worker advertises it can pull. The capability
159+
/// gate keeps SMG from emitting a `remote` payload to a worker that would reject
160+
/// it, and is the per-worker opt-out (a worker that shouldn't RDMA reports
161+
/// `supports_rdma_pull=false`).
162+
///
163+
/// The exporter is a process-wide resource built once the RDMA lane is enabled
164+
/// (router transport mode `rdma`, `SMG_MM_TENSOR_TRANSPORT=rdma`, or the legacy
165+
/// `SMG_MM_PIXEL_RDMA`). Gating on its presence — rather than the per-request
166+
/// transport mode — matches TokenSpeed and honors every way the lane is enabled.
163167
pub(super) fn resolve_mm_rdma_enabled(workers: Option<&WorkerSelection>) -> bool {
164-
let mode =
165-
worker_transport_mode_override(workers).unwrap_or_else(|| mm_transport_defaults().mode);
166-
mode == TransportMode::Rdma
167-
&& mm_rdma_exporter().is_some()
168-
&& worker_supports_rdma_pull(workers)
168+
mm_rdma_exporter().is_some() && worker_supports_rdma_pull(workers)
169169
}
170170

171171
/// Whether the request's worker advertises RDMA-pull support via the
@@ -666,8 +666,7 @@ mod tests {
666666
fn resolve_mm_rdma_enabled_requires_exporter() {
667667
// Even a capable worker cannot RDMA without the gateway exporter, which is
668668
// never built in the default (stub) test build -> the gate stays off.
669-
let workers =
670-
single_worker_with_labels(&[("supports_rdma_pull", "true"), ("_mode", "unused")]);
669+
let workers = single_worker_with_labels(&[("supports_rdma_pull", "true")]);
671670
assert!(!resolve_mm_rdma_enabled(Some(&workers)));
672671
}
673672

0 commit comments

Comments
 (0)