Skip to content

fix(local_vllm_model): restore vLLM runtime deps the shared policy never-installs - #6183

Closed
qiaochuz-nv wants to merge 1 commit into
mainfrom
qiaochuz/nvb-7087161-local-vllm-pycountry
Closed

qiaochuz-nv wants to merge 1 commit into
mainfrom
qiaochuz/nvb-7087161-local-vllm-pycountry

Conversation

@qiaochuz-nv

Copy link
Copy Markdown
Contributor

What does this PR do?

Restores the six vLLM runtime dependencies that the managed locks of local_vllm_model and genrm_model never install. Without them, vllm serve fails at startup for every model, text-only ones included.

#4837 copied the root never-install list (sys_platform == 'never') into these two servers' override-dependencies. Since then, the venv Gym builds for them lacks pycountry, torchvision, torchaudio, torchcodec, opencv-python-headless and pynvvideocodec. vLLM 0.29.0 lists the five codec packages in its own Requires-Dist, and pycountry comes through mistral_common → pydantic-extra-types[pycountry]. Two imports break startup:

  • transformers.MistralCommonBackend needs pycountry, so the API server dies before it loads a model;
  • with pycountry added by hand, the engine core then dies on from torchvision.transforms import InterpolationMode (vllm/transformers_utils/processors/minimax_m3.py). It imports this even for Qwen/Qwen3-0.6B.

The fix follows the policy described in the root pyproject.toml: "Servers that genuinely need these declare them in their own … override-dependencies". It records the exception in scripts/component-lock-policy.json for both components, the way toolsandbox (pycountry), vlm_eval_kit and osworld (torchvision, opencv) and agentic_mme_agent (opencv) already do:

  • The five codec packages use vLLM 0.29.0's own requirement strings, under the same sys_platform != 'darwin' marker as the vllm==0.29.0 pin. Darwin (vllm 0.11) resolves exactly as before.
  • pycountry is unpinned, as in toolsandbox.
  • scripts/component_locks.py synced the two pyproject.toml files and relocked both. In each uv.lock, only these six entries change, from the never-marker to the pins, plus their wheels. No other package version moves.
  • A new tests/test_dependencies.py in each server checks that every vLLM requirement is installed and imports the two modules that failed.

Licensing: this reverses the codec and copyleft exclusion for these two server venvs only. The image still prefetches no server venv by default (docker/Dockerfile, NEMO_GYM_PREFETCH_CONFIGS), and docker/install_excluded_deps.sh stays as is. Please confirm this exception is acceptable for the two vLLM servers; vLLM cannot serve without it.

Validation

All on an H100 (Linux x86_64), uv 0.13.8. Each run builds a throwaway worktree with uv sync --frozen in each component, then vllm serve from the local_vllm_model venv with an OpenAI-compatible request.

Before: main efd41fa161.

== local_vllm_model: vLLM requirements missing from the venv
vllm 0.29.0 missing: ['opencv-python-headless>=4.13.0', 'torchaudio==2.11.0', 'torchvision==0.28.0', 'torchcodec>=0.14', 'PyNvVideoCodec==2.0.4']
== genrm_model: vLLM requirements missing from the venv
vllm 0.29.0 missing: ['opencv-python-headless>=4.13.0', 'torchaudio==2.11.0', 'torchvision==0.28.0', 'torchcodec>=0.14', 'PyNvVideoCodec==2.0.4']
== serve text (Qwen/Qwen3-0.6B): FAILED
ModuleNotFoundError: No module named 'pycountry'
ModuleNotFoundError: Could not import module 'MistralCommonBackend'. Are this object's requirements defined correctly? ...
== serve vl (Qwen/Qwen3-VL-8B-Instruct): FAILED
ModuleNotFoundError: No module named 'pycountry'

The new test run against this venv: 2 failed (assert ['opencv-pyth...Codec==2.0.4'] == [], ModuleNotFoundError: No module named 'torchvision').

After: this branch.

== local_vllm_model: vLLM requirements missing from the venv
vllm 0.29.0 missing: []
== local_vllm_model: pytest responses_api_models/local_vllm_model/tests/test_dependencies.py
2 passed, 1 warning in 7.36s
== genrm_model: vLLM requirements missing from the venv
vllm 0.29.0 missing: []
== genrm_model: pytest responses_api_models/genrm_model/tests/test_dependencies.py
2 passed, 1 warning in 7.34s
== serve text (Qwen/Qwen3-0.6B, hermes tool parser, deepseek_r1 reasoning parser): STARTED
{"id":"chatcmpl-b3657c32d09e084f","object":"chat.completion",...,"model":"Qwen/Qwen3-0.6B",...}
== serve vl (Qwen/Qwen3-VL-8B-Instruct, one PNG of a red circle): STARTED
{..."model":"Qwen/Qwen3-VL-8B-Instruct","choices":[{"index":0,"message":{"role":"assistant","content":"Based on the image provided, here is a description of the shape and color:\n\n-   **Shape**: The image contains"...

Also:

  • python scripts/component_locks.py --jobs 8: 301 projects checked, 0 failed.
  • pre-commit run on the changed files: clean.
  • Found by a QA regression run: on efd41fa161 all 57 single-GPU QA cases that start local_vllm_model exited within about 20 s with this error.

Rollouts

N/A: dependency-only change, with no server, agent or environment code changes. The failure happens at vLLM startup, before any rollout. The two vllm serve requests above cover the server start.

Compatibility and benchmark impact

The local_vllm_model and genrm_model venvs are larger: they now install torchvision, torchaudio, torchcodec, opencv-python-headless, PyNvVideoCodec and pycountry. Darwin is unchanged. No benchmark behaviour changes, except that local-vLLM benchmarks start again.

Checklist

  • I have read the contributing guidelines.
  • The change is focused.
  • Tests added and pass locally (they fail on main).
  • N/A: no documentation change needed; the policy comment in the root pyproject.toml already describes this exception path.
  • Pre-commit checks pass locally on the changed files.
  • New source files include the required Apache-2.0 SPDX header.
  • All commits have DCO sign-off.

…ver-installs

#4837 copied the root never-install list into the vLLM servers' locks, so the
venv Gym builds for local_vllm_model and genrm_model lacks pycountry,
torchvision, torchaudio, torchcodec, opencv-python-headless and
pynvvideocodec. vLLM 0.29 requires all six, and `vllm serve` fails at startup
for every model: MistralCommonBackend needs pycountry, and the engine core
imports torchvision.

Re-enable them for these two components through
scripts/component-lock-policy.json, the way toolsandbox (pycountry) and
vlm_eval_kit/osworld (torchvision, opencv) do, using vLLM 0.29.0's own
requirement strings under its non-Darwin marker. Add a test that every vLLM
requirement is installed and the failing import chain loads.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: qiaochuz <qiaochuz@nvidia.com>
@qiaochuz-nv qiaochuz-nv added the QA_fix Fix implemented by a QA team member label Nov 14, 2026
@qiaochuz-nv
qiaochuz-nv requested a review from yaoyu-33 November 14, 2026 01:28
@copy-pr-bot

copy-pr-bot Bot commented Nov 14, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@qiaochuz-nv

Copy link
Copy Markdown
Contributor Author

/ok to test 1cb323a

@qiaochuz-nv
qiaochuz-nv requested a review from kajalj22 November 14, 2026 01:29
@nemo-automation-bot
nemo-automation-bot Bot requested review from a team November 14, 2026 01:29
@qiaochuz-nv
qiaochuz-nv requested a review from ko3n1g November 14, 2026 01:29

@svcnvidia-nemo-ci svcnvidia-nemo-ci left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SHIP

LGTM — no concerns found.

I reviewed all seven changed files and traced the managed-policy merge, component installation, model startup consumers, and test collection. The policy exceptions match both component declarations and locks without changing the root exclusions. The new tests check installed dependencies and the two reported failing imports in each component environment.

This was a static review. I didn't run dependency installation, tests, or GPU startup.

Note: formal approval is not available for this request, so this review is posted as a comment only.

Review generated by codex. This was a full review. Request a full review with /review reset=true.

@yaoyu-33
yaoyu-33 requested a review from zyzhou5 November 14, 2026 02:05
@github-actions github-actions Bot added the sla:review-overdue Review response is over the one-business-day SLA label Nov 16, 2026
@svcnvidia-nemo-ci svcnvidia-nemo-ci added bug Something isn't working area:build Packaging, dependencies, containers, and releases needs-review PR is ready for code review and waiting on a reviewer labels Nov 16, 2026
@qiaochuz-nv

Copy link
Copy Markdown
Contributor Author

Closing without merging. Leaving pycountry and the codec packages (opencv-python-headless, pynvvideocodec, torchcodec, torchvision, torchaudio) out of the local_vllm_model and genrm_model venvs is intended, per the root pyproject.toml policy (never installed; opt-in via docker/install_excluded_deps.sh). The QA runner now installs them into the vLLM venv itself before it starts vLLM. The tracking bug is closed as not a bug.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:build Packaging, dependencies, containers, and releases bug Something isn't working needs-review PR is ready for code review and waiting on a reviewer QA_fix Fix implemented by a QA team member sla:review-overdue Review response is over the one-business-day SLA

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants