Skip to content

feat(core): run Gym without Ray installed - #4300

Merged
ananthsub merged 1 commit into
mainfrom
ananthsub/tolerate-missing-ray
Oct 9, 2026
Merged

ananthsub merged 1 commit into
mainfrom
ananthsub/tolerate-missing-ray

Conversation

@ananthsub

@ananthsub ananthsub commented Oct 9, 2026 •

Copy link
Copy Markdown
Contributor

What changed

Gym now works when Ray isn't installed. Ray stays a core dependency in this PR; a follow-up moves it into the new ray extra.

  • Importing Gym: global_config.py no longer raises at import when Ray is missing. The parent Ray version is None, and gym --version reports Ray as not installed.
  • Pinning Ray in server venvs: the parent's exact Ray version is now passed to every server venv install as a uv constraint (-c <(printf ...)), stored under a new head_server_constraints config key, instead of as a ray[default]==<version> requirement in head_server_deps.
    • Where a server installs Ray, it still gets exactly the parent's version.
    • The pin no longer forces Ray into venvs whose servers don't depend on it, which matters once Ray leaves the core dependencies.
    • When the parent has no Ray, there's no constraint.
  • Servers that leave ray_enabled undeclared:
    • They join Ray only when Ray is importable in their venv and the run has a cluster address to join.
    • Otherwise they run without Ray and log a warning saying so.
    • Previously, such a server with no address called ray.init() and started a private cluster, in every process and Uvicorn worker.
  • A Gym-launched server that needs Ray but finds no cluster address now raises an error naming the class, instead of starting a private cluster. This can only happen when the supervisor decided no server needs Ray, for example because Ray isn't installed where gym runs.
  • Supervisor decision:
    • Servers that declare ray_enabled = True always need Ray.
    • Servers whose declaration can't be read from source (undeclared, inherited, imported) count only when Ray is installed where gym runs.
    • If a server declares True and Ray isn't installed there, gym env start stops before launching anything, with a one-line error naming the servers and the install command.
    • entrypoint_may_use_ray() becomes entrypoint_ray_enabled(), which returns True, False, or None for "can't tell".
  • New ray extra: nemo-gym[ray] (also included in all) duplicates the core Ray and jinja2 requirements for now. Error messages can point to it, and server manifests can request it before Ray leaves the core dependencies. The uv.lock change is metadata only; no package versions moved.
  • Docs: the FAQ's Ray section and the system design page describe when Ray starts, how undeclared servers behave, and the ray extra. The FAQ also still described the pre-feat(cli): start Ray only when a configured server may use it #4245 behavior.

Why

This is the second step of making Ray optional (#3669). Before the core dependency can move into an extra, every path that assumed Ray was installed has to cope without it:

  • the import-time version lookup;
  • the per-venv pin that installed Ray everywhere;
  • gym --version;
  • undeclared servers starting private clusters.

Validation

  • pytest tests/unit_tests/test_global_config.py tests/unit_tests/test_cli_setup_command.py tests/unit_tests/test_ray_opt_out.py tests/unit_tests/test_cli.py tests/unit_tests/test_server_utils.py tests/unit_tests/test_cli_main.py tests/unit_tests/test_cli_endpoint_readiness.py tests/unit_tests/h2_ping_sidecar
  • Full tests/unit_tests/ run: the same set of tests fails on this branch and on its base commit. All of them come from optional extras missing in the local venv (sandbox, wandb, mlflow).
  • New and updated tests:
    • the config parses without Ray, and the Ray pin is a constraint rather than a requirement;
    • install commands carry the constraint on every path, including both installs on the PyPI pyproject.toml path;
    • undeclared servers join only when Ray is importable and a cluster is configured, and otherwise run without Ray and warn;
    • a Gym-launched Ray server without a cluster fails, while a standalone one may still start its own;
    • the supervisor decision across declarations and Ray availability, including the early failure;
    • gym --version with Ray missing;
    • the source declaration reader returns None for undeclared, inherited, imported, unreadable and mixed entrypoints.
  • Real uv check, offline against hand-built wheels: a constraint passed through process substitution didn't install the constrained package when nothing requested it. A conflicting constraint made the install fail as unsatisfiable.
  • fern check passes.
  • pre-commit run scoped to the changed files.

Rollout evidence

Smoke run without Ray. I made a venv with this branch installed and then ran uv pip uninstall ray.

  • import nemo_gym.global_config, nemo_gym.server_utils, nemo_gym.cli.env succeeded, with ray_version = None.

  • gym --version --json reported "ray": "not installed".

  • gym env start with example_single_tool_call and openai_model reached All 4 / 4 servers ready: the resources server, the simple agent, the legacy agent environment server and the model server. No Ray processes were started.

  • The same venv with the code_gen config, which declares ray_enabled = True, exited before building any venv:

    Error: These servers use Ray, but Ray isn't installed where `gym` runs: code_gen. Install with:
      pip install nemo-gym[ray]
    

No model requests were sent. This changes dependency handling and whether Ray starts, not prompts, trajectories, verification or rewards.

Compatibility

  • Installs with Ray (everyone today): servers that declare ray_enabled behave exactly as before. Server venvs still get the parent's exact Ray version.
  • Undeclared servers started outside Gym with no ray_head_node_address: they no longer start a private Ray cluster at startup. If such a server calls Ray anyway, Ray initializes lazily on first use.
  • Undeclared servers launched by Gym: unchanged when Ray is installed where gym runs, because Gym starts Ray for them and they join it.
  • head_server_deps no longer contains Ray. The Ray pin lives in the new reserved head_server_constraints key.

@copy-pr-bot

copy-pr-bot Bot commented Oct 9, 2026

Copy link
Copy Markdown

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

Base automatically changed from ananthsub/pypi-keep-server-extras to main October 9, 2026 17:01
@ananthsub
ananthsub force-pushed the ananthsub/tolerate-missing-ray branch from 684d7ec to 86aac2a Compare October 9, 2026 17:06
@ananthsub
ananthsub marked this pull request as ready for review October 9, 2026 17:08
@ananthsub
ananthsub requested a review from a team as a code owner October 9, 2026 17:08
@nemo-automation-bot
nemo-automation-bot Bot requested review from a team October 9, 2026 17:08
@github-actions

github-actions Bot commented Oct 9, 2026 •

Copy link
Copy Markdown
Contributor

Prepare for making Ray an optional dependency (#3669) while it is still
a core dependency:

- Importing Gym no longer requires Ray: the parent Ray version lookup
  returns None when Ray is missing, and `gym --version` reports it as
  not installed.
- The parent's exact Ray version is passed to server venv installs as a
  uv constraint instead of a requirement, so it pins Ray where a server
  installs it without forcing Ray into every venv.
- Servers that leave ray_enabled undeclared join Ray only when Ray is
  importable and a cluster is configured to join; otherwise they run
  without Ray and log a warning. This stops them from starting a
  private cluster per process and worker.
- A Gym-launched server that needs Ray but finds no cluster raises a
  clear error instead of starting one.
- `gym env start` stops before launching servers when a server declares
  ray_enabled = True and Ray isn't installed where `gym` runs.
- Add a `ray` extra (also in `all`) so installs and server manifests
  can request Ray explicitly before it leaves the core dependencies.
- Update the FAQ and system design pages.

Signed-off-by: Ananth Subramaniam <ansubramania@nvidia.com>
@ananthsub
ananthsub added this pull request to the merge queue Oct 9, 2026
@github-actions

github-actions Bot commented Oct 9, 2026

Copy link
Copy Markdown
Contributor

🔄 Merge queue validation started! You can track the progress here: https://github.com/NVIDIA-NeMo/Gym/actions/runs/37974242170

Merged via the queue into main with commit 7204f6b Oct 9, 2026
49 of 51 checks passed
@ananthsub
ananthsub deleted the ananthsub/tolerate-missing-ray branch October 9, 2026 18:51
ananthsub added a commit that referenced this pull request Oct 9, 2026
Resolve the uv.lock conflict with #4300 by taking main's lockfile and
relocking with the nemo-lens v0.3.0 pin.

Signed-off-by: Ananth Subramaniam <ansubramania@nvidia.com>
ehosseiniasl pushed a commit to ehosseiniasl/Gym that referenced this pull request Oct 9, 2026
…A-NeMo#4301)

## What changed

Every server implementation that declares `ray_enabled = True` now
requests nemo-gym's `ray` extra (added in NVIDIA-NeMo#4300) in its own manifest:

| Manifest | Change | Servers |
|---|---|---|
| `requirements.txt` | `-e nemo-gym[dev,ray] @ ../../` | `code_gen`,
`evalplus`, `legal_agent_bench`, `longmt_eval`, `spider2_lite`,
`swerl_gen`, `wmt_translation`, `mini_swe_agent`, `osworld_agent`,
`stirrup_agent`, `swe_agents` |
| `requirements.txt` | `-e nemo-gym[dev,ray,sandbox] @ ../../` |
`anyterminal_agent`, `harbor_agent`, `mini_swe_agent_2` |
| `pyproject.toml` | `"nemo-gym[dev,ray]"` | `code_fim`,
`harbor_agent_general` |
| `setup.py` | `"nemo-gym[dev,ray]"` | `local_vllm_model`, `genrm_model`
|

A new audit in `tests/unit_tests/test_ray_opt_out.py` fails when a
server directory containing a class that declares `ray_enabled = True`
doesn't request the extra in any of its manifests. The FAQ's "Installing
Ray" section now describes the convention.

## Why

This is the third step of making Ray optional (NVIDIA-NeMo#3669). After NVIDIA-NeMo#4300, a
server venv gets Ray only when the server's own dependencies ask for it;
the parent's version is applied as a constraint, not installed. Once Ray
leaves nemo-gym's core dependencies, these servers need to request it
themselves, and NVIDIA-NeMo#4295 makes sure the extra survives PyPI installs.

## Validation

- `pytest tests/unit_tests/test_ray_opt_out.py`. With the manifests
reverted, the new audit lists every Ray server.
- **Resolution:** `uv pip install --dry-run` from each server's
directory resolves with Ray included, for one server per manifest
format: `code_gen` (`requirements.txt`), `anyterminal_agent`
(`requirements.txt` with `sandbox`), `code_fim` (`pyproject.toml`), and
`local_vllm_model` (`setup.py`).
- `pytest
responses_api_agents/osworld_agent/tests/test_dependency_policy.py`,
which reads that server's dependency files.
- `fern check` passes.
- `pre-commit run` scoped to the changed files.

## Rollout evidence

N/A. Ray is still a core dependency of nemo-gym, so these servers
install the same packages as before. This doesn't change prompts, model
requests, trajectories, verification or rewards.

## Compatibility

No change for any install today. External servers that use Ray should
request `nemo-gym[ray]` the same way before Ray becomes optional.

Signed-off-by: Ananth Subramaniam <ansubramania@nvidia.com>
khazic pushed a commit to khazic/Gym-lao that referenced this pull request Oct 13, 2026
## What does this PR do?

Follow-up to NVIDIA-NeMo#3632, split out as requested in
NVIDIA-NeMo#3632 (comment).

In `_request_with_retries()`, the generic `except Exception` branch
handles every error except `ServerDisconnectedError` and
`ClientOSError`, which have their own branches. This includes aiohttp
timeouts. When the caller does not pass `_max_num_tries`, this branch
counted tries only for external callers. So an internal caller
(`_internal=True`) that set `_max_connection_retries` to 2 or more never
reached the cap and retried forever.

NVIDIA-NeMo#3415 already counts tries for callers that pass `_max_num_tries`. This
PR fixes the remaining path, where `_max_num_tries` is not passed.

- Count tries for internal callers too, so `_max_connection_retries`
stops them.
- Keep everything else the same:
- internal callers with no cap still retry with no limit and print no
retry message, as NVIDIA-NeMo#90 intended;
  - external callers still stop at `MAX_NUM_TRIES`;
  - callers that pass `_max_num_tries` are unchanged.
- Update the comment that said internal calls are not counted. It keeps
NVIDIA-NeMo#90's reason for not limiting internal calls.
- Add three tests:
  - the cap stops internal callers, with tracing off and on;
  - internal callers with no cap retry silently past `MAX_NUM_TRIES`;
- external callers stop at `MAX_NUM_TRIES` and print one retry message
per try.
- The cap test, `test_request_retry_cap_bounds_internal_callers`,
replaces `test_retry_cap_bounds_internal_callers`, which was removed
from NVIDIA-NeMo#3632.
- Make the retry tests fail instead of hang when the loop never stops:
- The test helper `_mock_global_client` (added in NVIDIA-NeMo#2383) now takes the
error to raise.
- Its fake `request()` calls `pytest.fail()` once it is called more
often than any test needs. `pytest.fail()` raises a `BaseException`,
which the loop's `except Exception` does not catch. So the test fails at
once, even if the loop never sleeps.
- Two NVIDIA-NeMo#3415 tests now use this helper instead of their own mocks:
`test_explicit_transport_attempt_cap_disables_request_retry` and
`test_request_caps_generic_errors_at_the_lower_attempt_limit`. Before,
if the generic branch ignored `_max_num_tries`, the first test hung,
sleeping a real 0.5 s per retry.

No caller in this repo sets both `_internal=True` and
`_max_connection_retries` today:
- The internal callers pass no cap: `ServerClient`, the simple agent's
direct tool calls to Resources servers (`tool_access`), the Terminus 2
sandboxed agent's model client (`NeMoGymAsyncOpenAI(internal=True)`),
the OpenHands capture route, and the LiteLLM version check.
- The callers with a cap are not internal: the remote agent and sandbox
transport use 1, Tavily search uses 0, and vLLM with `endpoint_file`
uses a default of 8.
- The OpenAI model server and the Gymnasium agent pass `_max_num_tries`
only when `upstream_max_num_tries` or
`model_server_transport_max_attempts` is set. Neither passes
`_max_connection_retries`, so neither changes.

Related:
- NVIDIA-NeMo#3553 added the cap check to this branch without counting internal
tries.
- NVIDIA-NeMo#3415 added `_max_num_tries` and counts tries on that path.
- NVIDIA-NeMo#3159 tracks bounding every retry by default, including internal
callers with no cap. This PR only makes an explicit cap work for them.
- NVIDIA-NeMo#2216 tracks the `ServerDisconnectedError` and `ClientOSError`
branches, which this PR does not change.
- Open PR NVIDIA-NeMo#2935 adds a timeout branch that copies the old counting, so
internal callers there still do not count tries. Whichever PR merges
later must count tries for every caller.

## Validation

All runs used an activated venv from `uv sync --extra dev --extra
sandbox --extra openshell --extra telemetry --extra mlflow --extra
wandb` (the extras in `full-test-suite.yml`). Every installed package
matches `uv.lock` (192 packages, including aiohttp 3.14.4, nemo-lens
0.2.0+b0f977d, and pytest 9.0.3).

- `pytest tests/unit_tests/test_server_utils.py
responses_api_agents/remote_agent/tests/test_app.py
tests/unit_tests/telemetry/`: 465 passed.
- With the old code restored,
`test_request_retry_cap_bounds_internal_callers` fails in under a second
with `Failed: retry loop did not stop` instead of hanging. It fails the
same way when the retry sleep is also removed.
- Each change below makes at least one test fail. Each run of
`test_server_utils.py` took about 3 s, and none hung:
  - the cap is ignored;
  - internal callers are limited by `MAX_NUM_TRIES`;
  - internal callers print;
  - external tries are double-counted;
  - external callers stop printing the retry message;
  - the `MAX_NUM_TRIES` raise is removed;
  - the retry sleep after a generic error is skipped;
  - the cap check ignores `_max_num_tries` attempts;
  - the generic branch ignores `_max_num_tries`;
  - the traced path drops `_internal` or `_max_connection_retries`.
- `pytest -n 16 tests/unit_tests
responses_api_agents/remote_agent/tests`: 8089 passed, 5 skipped. The
per-test results match `main` at a3cf403 (8085 passed, 5 skipped). The
4 extra results are the new tests, with the cap test run twice (tracing
off and on).
- The same focused tests pass on a local merge with the latest `main`
(NVIDIA-NeMo#4300), which changes no package versions.
- `pre-commit run --all-files`: passed.

## Rollouts

N/A. This changes a retry counter in the shared HTTP helper. No caller
in this repo changes behavior, and no model, agent, environment, or
verifier logic is involved.

## Compatibility and benchmark impact

None for callers in this repo. A caller that sets `_internal=True` and
`_max_connection_retries` to 2 or more, without `_max_num_tries`, now
stops after that many tries on errors that reach the generic `except
Exception` branch, including timeouts, instead of retrying forever. This
includes code outside this repo that uses
`NeMoGymAsyncOpenAI(internal=True, max_connection_retries=N)`.
`ServerDisconnectedError` and `ClientOSError` keep their own count, as
before.

## Checklist

- [x] I have read the [contributing
guidelines](https://github.com/NVIDIA-NeMo/Gym/blob/main/CONTRIBUTING.md).
- [x] The change is focused; unrelated "drive-by" edits are tracked as
separate issues/PRs.
- [x] Tests added or updated and pass locally, or N/A for docs-only /
non-code changes (so CI unit/server checks pass when applicable).
- [x] Documentation added or updated, or N/A with justification. N/A:
internal helper behavior with no user-facing docs.
- [x] Pre-commit checks pass locally (`pre-commit run --all-files`) (so
CI lint/format checks pass).
- [x] New source files include the required Apache-2.0 SPDX header. N/A:
no new files.
- [x] All commits have DCO sign-off (`git commit -s`) (so the DCO check
passes).

---------

Signed-off-by: Abhishree <athittenaman@nvidia.com>
github-merge-queue Bot pushed a commit that referenced this pull request Oct 14, 2026
Closes #4316.

## What changed

`ray[default]` and its `jinja2` pin move out of nemo-gym's core
dependencies and live only in the `ray` extra (also part of `all`). A
plain `pip install nemo-gym` no longer installs Ray.

- **Packaging:** `pyproject.toml` and `uv.lock`. The lockfile change is
metadata only; Ray and `jinja2` stay locked through the extra, and no
package versions move.
- **CI:**
- `scripts/ci/core_unit_tests.sh`, plus the unit-test and
full-test-suite syncs, add `--extra ray`, so Ray code paths stay tested
and stay in the coverage gate.
- The wheel smoke test still installs the plain wheel, which now also
covers the path without Ray.
- Server-test setup stays without the extra, because each Ray server's
own venv requests it.
- **Docker:** the image builds add `--extra ray`. The `ray_dist.jar`
removal for the CVE is unchanged.
- **Tests:** two tests skip when Ray isn't installed:
- `tests/unit_tests/test_ray_serve_gateway.py`, because the gateway
imports Ray Serve at module level;
- the Super 3.5 launcher's prebuilt-environment check, because its
generated dependency check imports Ray, which the evaluation image
provides.
- **Docs:** the installation and development-setup pages explain when to
add the `ray` extra, and the FAQ no longer says Ray is a core
dependency.

## Why

This is the last step of #3669. Earlier changes made Gym run without Ray
(#4300), had Ray servers request the extra (#4301), and made servers run
without Ray unless they declare `ray_enabled = True` (#4307). The only
remaining reason every install got Ray was nemo-gym's own core
dependency.

## Validation

- **Package metadata:** `uv build --wheel` gives `Requires-Dist:
ray[default]>=2.58.0; extra == "ray"` and `Requires-Dist: jinja2>=3.1;
extra == "ray"`, with no unconditional Ray requirement.
- **Without the extra:** `uv sync --locked --extra dev` gives an
environment where neither `ray` nor `jinja2` is importable, and `gym
--version` reports `"ray": "not installed"`.
- **Full `tests/unit_tests/` run, `-m "not sandbox"` as in the core CI
job.** I built the core CI job's environment (`dev`, `telemetry`,
`mlflow`, `wandb`) twice:
  - With `--extra ray`, everything passes.
- Without it, only the Ray-specific tests above skip. One
timing-sensitive OpenSandbox cleanup test failed once under parallel
load, and passed three times out of three on its own.
- `fern check` passes.
- `pre-commit run` scoped to the changed files.

## Rollout evidence

N/A. This changes which packages a default install includes. It doesn't
change prompts, model requests, trajectories, verification or rewards.
#4300 smoke-tested `gym env start` with Ray uninstalled.

## Compatibility

This is a breaking packaging change and needs a release note (#4317):
- **Ray-backed servers after a plain install:** anyone running a server
that declares `ray_enabled = True` (such as `local_vllm_model`,
`genrm_model` or the SWE agents) needs `pip install "nemo-gym[ray]"`, or
`uv sync --extra ray`, where `gym` runs. Without it, `gym env start`
stops before launching anything and says so.
- **Built-in Ray servers' own venvs:** they keep getting Ray through
their manifests, pinned to the version where `gym` runs.
- **NeMo RL:** brings its own Ray, so nothing changes there.
- **External servers that use Ray:** should request `nemo-gym[ray]` in
their manifests.

Signed-off-by: Ananth Subramaniam <ansubramania@nvidia.com>
@linear-code linear-code Bot mentioned this pull request Nov 12, 2026
7 of 26 tasks

This branch was previously deployed

1 inactive deployment
public — 8bba52b8 Deployed Oct 9, 2026 by copy-pr-bot[bot] via release / finalize / notify #4879
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants