Repository navigation
feat(core): run Gym without Ray installed - #4300
Merged
Merged
Conversation
|
Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually. Contributors can view more details about this message here. |
ananthsub
force-pushed
the
ananthsub/tolerate-missing-ray
branch
from
October 9, 2026 17:06
684d7ec to
86aac2a
Compare
ananthsub
marked this pull request as ready for review
October 9, 2026 17:08
Contributor
|
🌿 Preview your docs: https://nvidia-preview-pr-4300-8bba52b82506.docs.buildwithfern.com/nemo/gym |
Prepare for making Ray an optional dependency (#3669) while it is still a core dependency: - Importing Gym no longer requires Ray: the parent Ray version lookup returns None when Ray is missing, and `gym --version` reports it as not installed. - The parent's exact Ray version is passed to server venv installs as a uv constraint instead of a requirement, so it pins Ray where a server installs it without forcing Ray into every venv. - Servers that leave ray_enabled undeclared join Ray only when Ray is importable and a cluster is configured to join; otherwise they run without Ray and log a warning. This stops them from starting a private cluster per process and worker. - A Gym-launched server that needs Ray but finds no cluster raises a clear error instead of starting one. - `gym env start` stops before launching servers when a server declares ray_enabled = True and Ray isn't installed where `gym` runs. - Add a `ray` extra (also in `all`) so installs and server manifests can request Ray explicitly before it leaves the core dependencies. - Update the FAQ and system design pages. Signed-off-by: Ananth Subramaniam <ansubramania@nvidia.com>
ananthsub
force-pushed
the
ananthsub/tolerate-missing-ray
branch
from
October 9, 2026 17:17
86aac2a to
8bba52b
Compare
hemildesai
approved these changes
Oct 9, 2026
This was referenced Oct 9, 2026
kajalj22
approved these changes
Oct 9, 2026
chtruong814
approved these changes
Oct 9, 2026
Contributor
|
🔄 Merge queue validation started! You can track the progress here: https://github.com/NVIDIA-NeMo/Gym/actions/runs/37974242170 |
7 tasks done
ananthsub
added a commit
that referenced
this pull request
Oct 9, 2026
Resolve the uv.lock conflict with #4300 by taking main's lockfile and relocking with the nemo-lens v0.3.0 pin. Signed-off-by: Ananth Subramaniam <ansubramania@nvidia.com>
ehosseiniasl
pushed a commit
to ehosseiniasl/Gym
that referenced
this pull request
Oct 9, 2026
…A-NeMo#4301) ## What changed Every server implementation that declares `ray_enabled = True` now requests nemo-gym's `ray` extra (added in NVIDIA-NeMo#4300) in its own manifest: | Manifest | Change | Servers | |---|---|---| | `requirements.txt` | `-e nemo-gym[dev,ray] @ ../../` | `code_gen`, `evalplus`, `legal_agent_bench`, `longmt_eval`, `spider2_lite`, `swerl_gen`, `wmt_translation`, `mini_swe_agent`, `osworld_agent`, `stirrup_agent`, `swe_agents` | | `requirements.txt` | `-e nemo-gym[dev,ray,sandbox] @ ../../` | `anyterminal_agent`, `harbor_agent`, `mini_swe_agent_2` | | `pyproject.toml` | `"nemo-gym[dev,ray]"` | `code_fim`, `harbor_agent_general` | | `setup.py` | `"nemo-gym[dev,ray]"` | `local_vllm_model`, `genrm_model` | A new audit in `tests/unit_tests/test_ray_opt_out.py` fails when a server directory containing a class that declares `ray_enabled = True` doesn't request the extra in any of its manifests. The FAQ's "Installing Ray" section now describes the convention. ## Why This is the third step of making Ray optional (NVIDIA-NeMo#3669). After NVIDIA-NeMo#4300, a server venv gets Ray only when the server's own dependencies ask for it; the parent's version is applied as a constraint, not installed. Once Ray leaves nemo-gym's core dependencies, these servers need to request it themselves, and NVIDIA-NeMo#4295 makes sure the extra survives PyPI installs. ## Validation - `pytest tests/unit_tests/test_ray_opt_out.py`. With the manifests reverted, the new audit lists every Ray server. - **Resolution:** `uv pip install --dry-run` from each server's directory resolves with Ray included, for one server per manifest format: `code_gen` (`requirements.txt`), `anyterminal_agent` (`requirements.txt` with `sandbox`), `code_fim` (`pyproject.toml`), and `local_vllm_model` (`setup.py`). - `pytest responses_api_agents/osworld_agent/tests/test_dependency_policy.py`, which reads that server's dependency files. - `fern check` passes. - `pre-commit run` scoped to the changed files. ## Rollout evidence N/A. Ray is still a core dependency of nemo-gym, so these servers install the same packages as before. This doesn't change prompts, model requests, trajectories, verification or rewards. ## Compatibility No change for any install today. External servers that use Ray should request `nemo-gym[ray]` the same way before Ray becomes optional. Signed-off-by: Ananth Subramaniam <ansubramania@nvidia.com>
khazic
pushed a commit
to khazic/Gym-lao
that referenced
this pull request
Oct 13, 2026
## What does this PR do? Follow-up to NVIDIA-NeMo#3632, split out as requested in NVIDIA-NeMo#3632 (comment). In `_request_with_retries()`, the generic `except Exception` branch handles every error except `ServerDisconnectedError` and `ClientOSError`, which have their own branches. This includes aiohttp timeouts. When the caller does not pass `_max_num_tries`, this branch counted tries only for external callers. So an internal caller (`_internal=True`) that set `_max_connection_retries` to 2 or more never reached the cap and retried forever. NVIDIA-NeMo#3415 already counts tries for callers that pass `_max_num_tries`. This PR fixes the remaining path, where `_max_num_tries` is not passed. - Count tries for internal callers too, so `_max_connection_retries` stops them. - Keep everything else the same: - internal callers with no cap still retry with no limit and print no retry message, as NVIDIA-NeMo#90 intended; - external callers still stop at `MAX_NUM_TRIES`; - callers that pass `_max_num_tries` are unchanged. - Update the comment that said internal calls are not counted. It keeps NVIDIA-NeMo#90's reason for not limiting internal calls. - Add three tests: - the cap stops internal callers, with tracing off and on; - internal callers with no cap retry silently past `MAX_NUM_TRIES`; - external callers stop at `MAX_NUM_TRIES` and print one retry message per try. - The cap test, `test_request_retry_cap_bounds_internal_callers`, replaces `test_retry_cap_bounds_internal_callers`, which was removed from NVIDIA-NeMo#3632. - Make the retry tests fail instead of hang when the loop never stops: - The test helper `_mock_global_client` (added in NVIDIA-NeMo#2383) now takes the error to raise. - Its fake `request()` calls `pytest.fail()` once it is called more often than any test needs. `pytest.fail()` raises a `BaseException`, which the loop's `except Exception` does not catch. So the test fails at once, even if the loop never sleeps. - Two NVIDIA-NeMo#3415 tests now use this helper instead of their own mocks: `test_explicit_transport_attempt_cap_disables_request_retry` and `test_request_caps_generic_errors_at_the_lower_attempt_limit`. Before, if the generic branch ignored `_max_num_tries`, the first test hung, sleeping a real 0.5 s per retry. No caller in this repo sets both `_internal=True` and `_max_connection_retries` today: - The internal callers pass no cap: `ServerClient`, the simple agent's direct tool calls to Resources servers (`tool_access`), the Terminus 2 sandboxed agent's model client (`NeMoGymAsyncOpenAI(internal=True)`), the OpenHands capture route, and the LiteLLM version check. - The callers with a cap are not internal: the remote agent and sandbox transport use 1, Tavily search uses 0, and vLLM with `endpoint_file` uses a default of 8. - The OpenAI model server and the Gymnasium agent pass `_max_num_tries` only when `upstream_max_num_tries` or `model_server_transport_max_attempts` is set. Neither passes `_max_connection_retries`, so neither changes. Related: - NVIDIA-NeMo#3553 added the cap check to this branch without counting internal tries. - NVIDIA-NeMo#3415 added `_max_num_tries` and counts tries on that path. - NVIDIA-NeMo#3159 tracks bounding every retry by default, including internal callers with no cap. This PR only makes an explicit cap work for them. - NVIDIA-NeMo#2216 tracks the `ServerDisconnectedError` and `ClientOSError` branches, which this PR does not change. - Open PR NVIDIA-NeMo#2935 adds a timeout branch that copies the old counting, so internal callers there still do not count tries. Whichever PR merges later must count tries for every caller. ## Validation All runs used an activated venv from `uv sync --extra dev --extra sandbox --extra openshell --extra telemetry --extra mlflow --extra wandb` (the extras in `full-test-suite.yml`). Every installed package matches `uv.lock` (192 packages, including aiohttp 3.14.4, nemo-lens 0.2.0+b0f977d, and pytest 9.0.3). - `pytest tests/unit_tests/test_server_utils.py responses_api_agents/remote_agent/tests/test_app.py tests/unit_tests/telemetry/`: 465 passed. - With the old code restored, `test_request_retry_cap_bounds_internal_callers` fails in under a second with `Failed: retry loop did not stop` instead of hanging. It fails the same way when the retry sleep is also removed. - Each change below makes at least one test fail. Each run of `test_server_utils.py` took about 3 s, and none hung: - the cap is ignored; - internal callers are limited by `MAX_NUM_TRIES`; - internal callers print; - external tries are double-counted; - external callers stop printing the retry message; - the `MAX_NUM_TRIES` raise is removed; - the retry sleep after a generic error is skipped; - the cap check ignores `_max_num_tries` attempts; - the generic branch ignores `_max_num_tries`; - the traced path drops `_internal` or `_max_connection_retries`. - `pytest -n 16 tests/unit_tests responses_api_agents/remote_agent/tests`: 8089 passed, 5 skipped. The per-test results match `main` at a3cf403 (8085 passed, 5 skipped). The 4 extra results are the new tests, with the cap test run twice (tracing off and on). - The same focused tests pass on a local merge with the latest `main` (NVIDIA-NeMo#4300), which changes no package versions. - `pre-commit run --all-files`: passed. ## Rollouts N/A. This changes a retry counter in the shared HTTP helper. No caller in this repo changes behavior, and no model, agent, environment, or verifier logic is involved. ## Compatibility and benchmark impact None for callers in this repo. A caller that sets `_internal=True` and `_max_connection_retries` to 2 or more, without `_max_num_tries`, now stops after that many tries on errors that reach the generic `except Exception` branch, including timeouts, instead of retrying forever. This includes code outside this repo that uses `NeMoGymAsyncOpenAI(internal=True, max_connection_retries=N)`. `ServerDisconnectedError` and `ClientOSError` keep their own count, as before. ## Checklist - [x] I have read the [contributing guidelines](https://github.com/NVIDIA-NeMo/Gym/blob/main/CONTRIBUTING.md). - [x] The change is focused; unrelated "drive-by" edits are tracked as separate issues/PRs. - [x] Tests added or updated and pass locally, or N/A for docs-only / non-code changes (so CI unit/server checks pass when applicable). - [x] Documentation added or updated, or N/A with justification. N/A: internal helper behavior with no user-facing docs. - [x] Pre-commit checks pass locally (`pre-commit run --all-files`) (so CI lint/format checks pass). - [x] New source files include the required Apache-2.0 SPDX header. N/A: no new files. - [x] All commits have DCO sign-off (`git commit -s`) (so the DCO check passes). --------- Signed-off-by: Abhishree <athittenaman@nvidia.com>
github-merge-queue Bot
pushed a commit
that referenced
this pull request
Oct 14, 2026
Closes #4316. ## What changed `ray[default]` and its `jinja2` pin move out of nemo-gym's core dependencies and live only in the `ray` extra (also part of `all`). A plain `pip install nemo-gym` no longer installs Ray. - **Packaging:** `pyproject.toml` and `uv.lock`. The lockfile change is metadata only; Ray and `jinja2` stay locked through the extra, and no package versions move. - **CI:** - `scripts/ci/core_unit_tests.sh`, plus the unit-test and full-test-suite syncs, add `--extra ray`, so Ray code paths stay tested and stay in the coverage gate. - The wheel smoke test still installs the plain wheel, which now also covers the path without Ray. - Server-test setup stays without the extra, because each Ray server's own venv requests it. - **Docker:** the image builds add `--extra ray`. The `ray_dist.jar` removal for the CVE is unchanged. - **Tests:** two tests skip when Ray isn't installed: - `tests/unit_tests/test_ray_serve_gateway.py`, because the gateway imports Ray Serve at module level; - the Super 3.5 launcher's prebuilt-environment check, because its generated dependency check imports Ray, which the evaluation image provides. - **Docs:** the installation and development-setup pages explain when to add the `ray` extra, and the FAQ no longer says Ray is a core dependency. ## Why This is the last step of #3669. Earlier changes made Gym run without Ray (#4300), had Ray servers request the extra (#4301), and made servers run without Ray unless they declare `ray_enabled = True` (#4307). The only remaining reason every install got Ray was nemo-gym's own core dependency. ## Validation - **Package metadata:** `uv build --wheel` gives `Requires-Dist: ray[default]>=2.58.0; extra == "ray"` and `Requires-Dist: jinja2>=3.1; extra == "ray"`, with no unconditional Ray requirement. - **Without the extra:** `uv sync --locked --extra dev` gives an environment where neither `ray` nor `jinja2` is importable, and `gym --version` reports `"ray": "not installed"`. - **Full `tests/unit_tests/` run, `-m "not sandbox"` as in the core CI job.** I built the core CI job's environment (`dev`, `telemetry`, `mlflow`, `wandb`) twice: - With `--extra ray`, everything passes. - Without it, only the Ray-specific tests above skip. One timing-sensitive OpenSandbox cleanup test failed once under parallel load, and passed three times out of three on its own. - `fern check` passes. - `pre-commit run` scoped to the changed files. ## Rollout evidence N/A. This changes which packages a default install includes. It doesn't change prompts, model requests, trajectories, verification or rewards. #4300 smoke-tested `gym env start` with Ray uninstalled. ## Compatibility This is a breaking packaging change and needs a release note (#4317): - **Ray-backed servers after a plain install:** anyone running a server that declares `ray_enabled = True` (such as `local_vllm_model`, `genrm_model` or the SWE agents) needs `pip install "nemo-gym[ray]"`, or `uv sync --extra ray`, where `gym` runs. Without it, `gym env start` stops before launching anything and says so. - **Built-in Ray servers' own venvs:** they keep getting Ray through their manifests, pinned to the version where `gym` runs. - **NeMo RL:** brings its own Ray, so nothing changes there. - **External servers that use Ray:** should request `nemo-gym[ray]` in their manifests. Signed-off-by: Ananth Subramaniam <ansubramania@nvidia.com>
7 of 26 tasks
This branch was previously deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed
Gym now works when Ray isn't installed. Ray stays a core dependency in this PR; a follow-up moves it into the new
rayextra.global_config.pyno longer raises at import when Ray is missing. The parent Ray version isNone, andgym --versionreports Ray asnot installed.-c <(printf ...)), stored under a newhead_server_constraintsconfig key, instead of as aray[default]==<version>requirement inhead_server_deps.ray_enabledundeclared:ray.init()and started a private cluster, in every process and Uvicorn worker.gymruns.ray_enabled = Truealways need Ray.gymruns.Trueand Ray isn't installed there,gym env startstops before launching anything, with a one-line error naming the servers and the install command.entrypoint_may_use_ray()becomesentrypoint_ray_enabled(), which returnsTrue,False, orNonefor "can't tell".rayextra:nemo-gym[ray](also included inall) duplicates the core Ray andjinja2requirements for now. Error messages can point to it, and server manifests can request it before Ray leaves the core dependencies. Theuv.lockchange is metadata only; no package versions moved.rayextra. The FAQ also still described the pre-feat(cli): start Ray only when a configured server may use it #4245 behavior.Why
This is the second step of making Ray optional (#3669). Before the core dependency can move into an extra, every path that assumed Ray was installed has to cope without it:
gym --version;Validation
pytest tests/unit_tests/test_global_config.py tests/unit_tests/test_cli_setup_command.py tests/unit_tests/test_ray_opt_out.py tests/unit_tests/test_cli.py tests/unit_tests/test_server_utils.py tests/unit_tests/test_cli_main.py tests/unit_tests/test_cli_endpoint_readiness.py tests/unit_tests/h2_ping_sidecartests/unit_tests/run: the same set of tests fails on this branch and on its base commit. All of them come from optional extras missing in the local venv (sandbox,wandb,mlflow).pyproject.tomlpath;gym --versionwith Ray missing;Nonefor undeclared, inherited, imported, unreadable and mixed entrypoints.uvcheck, offline against hand-built wheels: a constraint passed through process substitution didn't install the constrained package when nothing requested it. A conflicting constraint made the install fail as unsatisfiable.fern checkpasses.pre-commit runscoped to the changed files.Rollout evidence
Smoke run without Ray. I made a venv with this branch installed and then ran
uv pip uninstall ray.import nemo_gym.global_config, nemo_gym.server_utils, nemo_gym.cli.envsucceeded, withray_version = None.gym --version --jsonreported"ray": "not installed".gym env startwithexample_single_tool_callandopenai_modelreachedAll 4 / 4 servers ready: the resources server, the simple agent, the legacy agent environment server and the model server. No Ray processes were started.The same venv with the
code_genconfig, which declaresray_enabled = True, exited before building any venv:No model requests were sent. This changes dependency handling and whether Ray starts, not prompts, trajectories, verification or rewards.
Compatibility
ray_enabledbehave exactly as before. Server venvs still get the parent's exact Ray version.ray_head_node_address: they no longer start a private Ray cluster at startup. If such a server calls Ray anyway, Ray initializes lazily on first use.gymruns, because Gym starts Ray for them and they join it.head_server_depsno longer contains Ray. The Ray pin lives in the new reservedhead_server_constraintskey.