Skip to content

feat(environment): route BaseEnvironmentServerWithTools tool calls to the worker that runs the episode - #5350

Merged
ffrujeri merged 3 commits into
ffrujeri/self-contained-mcpfrom
ffrujeri/self-contained-session-routing
Nov 16, 2026
Merged

ffrujeri merged 3 commits into
ffrujeri/self-contained-mcpfrom
ffrujeri/self-contained-session-routing

Conversation

@ffrujeri

@ffrujeri ffrujeri commented Oct 30, 2026 •

Copy link
Copy Markdown
Contributor

Stacked on #5349 (MCP). The stack: #5346 (episode tokens) → #5347 (the base class) → #5348 (shutdown sweep) → #5349 (MCP) → this PR. Review those first; this PR's diff against #5349 is the routing.

What changed

With several uvicorn workers, a tool call for an episode can reach a worker that does not hold the episode. BaseEnvironmentServerWithTools now routes each call to the worker that runs the episode, with the session routing that Resources and Agent Servers already use, and the interim num_workers > 1 refusal from #5347 is gone.

agent ──POST /add, episode token──▶ worker 2 ──forward (claim names worker 1)──▶ worker 1: runs the episode, checks the nonce, calls the tool
  • Claim. serve_tools() claims the episode, by its capture key, on the worker that runs it. The episode's cleanup releases the claim when it stops serving the tools. A retried attempt (-a<N>) has its own key, so its own claim. Serving the same attempt on another worker is refused.
  • Routing reads the token. The router reads which episode a call names from the signed episode token in its header (X-NeMo-Gym-Episode-Token). It does not need the nonce; only the episode's worker checks it, so a forged token is forwarded and then refused there with 403.
  • MCP is routed the same way. feat(environment): serve BaseEnvironmentServerWithTools episode tools over MCP #5349's MCP grant carries the same header on its connection, so every MCP request (initialize, tools/list, tools/call) is routed by it. /mcp is stateless Streamable HTTP, so no MCP session lives on a worker; each request only has to reach the episode's worker, which the e2e test checks.
  • nemo_gym/server_runtime/request_handling/session_routing.py, generic and opt-in:
    • session_header and session_from_header: a header that names the session, with the server's reader of its value, checked before the cookie and MCP token.
    • SessionHolder: a server that claims and releases each session itself, as an Agent Server releases its own.
    • SessionKind's cookie key, body field, and seed and close routes may be None, for a kind named only in a header.
    • Resources and Agent Servers pass none of these, so their routing is unchanged.
    • Main's rollout-stop path claim (SessionKind.path_claim, feat(agent): let a caller stop a running rollout and still grade it #5145) is unchanged. The episode kind sets none, so a tool call never takes that path.
  • CI. The session-routing e2e job also runs when nemo_gym/server_runtime/base_environment_server.py changes.
Tool call Answer
Reaches another worker Forwarded to the episode's worker
The episode's worker exited 410 from the router ("no longer running")
The episode ended (claim released) 410 from the worker it reached ("not running on this server")
Token that names no episode 401 where it lands

Why

Linear GYMINT-645. Single-worker is not an option at Gym's scale, so an Environment Server that serves tools needs the same per-worker routing as a Resources Server. This removes the limit #5347 had to add.

Validation

Command Result
pytest -n 12 tests/unit_tests/server_runtime/ tests/unit_tests/test_public_api.py tests/unit_tests/test_import_linter_coverage.py tests/unit_tests/test_moved_paths.py (after the rebase onto main at dda609f86) 1337 passed
NEMO_GYM_MULTIWORKER_E2E=1 pytest tests/e2e/session_routing (after the rebase onto main at dda609f86) 8 passed. One earlier run, right after the full unit suite, failed test_every_mcp_request_runs_on_the_worker_that_runs_its_episode and main's test_session_of_an_exited_worker_gets_an_explicit_error; both passed when rerun
NEMO_GYM_MULTIWORKER_STRESS=1 NEMO_GYM_STRESS_EPISODES=400 NEMO_GYM_STRESS_SESSIONS=100 pytest tests/e2e/session_routing/test_session_routing_stress.py (after the rebase onto main at dda609f86) 8 passed
pytest tests/unit_tests/server_runtime/test_episode_tools_session_routing.py 8 passed. Two in-process workers share a real socket directory: a call that reaches the other worker runs on the episode's worker; a forged nonce is forwarded and refused there (403); a token that names no episode gets 401 where it lands; each attempt has its own claim; one attempt cannot be served on two workers; the claim ends with the episode (then 410); an exited worker's episode gets 410; one worker serves calls without claims.
Same, with the claim in serve_tools disabled 5 of 8 fail
NEMO_GYM_MULTIWORKER_E2E=1 pytest tests/e2e/session_routing/test_episode_tools_routing_e2e.py 3 passed. A test-only BaseEnvironmentServerWithTools with 4 real workers and tool_transports: [direct_http, mcp]: 200 episodes each make 8 direct HTTP tool calls through the public port on new connections; all 1,600 return 200 and ran on their episode's worker, and the episodes ran on several workers. 50 episodes each act as an MCP client (the MCP SDK's, without keep-alive, so each request lands on any worker): initialize, tools/list and 4 tools/call; every episode listed exactly its tool and all 200 calls ran on its worker. Killing the worker that runs an episode makes its next tool call 410.
Same, with the claim disabled (before the rebase, direct HTTP tests) Both fail: 1,150 of 1,600 calls got 410, and the dead worker's call got the handler's 410 instead of the router's
Same, with the claim disabled, MCP test Fails: tools/list requests that reached other workers listed no tools
NEMO_GYM_MULTIWORKER_E2E=1 pytest tests/e2e/session_routing 8 passed (stress tests skipped)
NEMO_GYM_MULTIWORKER_STRESS=1 NEMO_GYM_STRESS_EPISODES=400 NEMO_GYM_STRESS_SESSIONS=100 pytest tests/e2e/session_routing/test_session_routing_stress.py (before the rebase) 8 passed (reduced from the default 4,000 episodes and 1,000 sessions)
pytest tests/unit_tests/server_runtime/test_base_environment_server_with_tools.py tests/unit_tests/server_runtime/test_episode_tools_session_routing.py tests/unit_tests/server_runtime/test_session_routing.py tests/unit_tests/server_runtime/test_episode_tools.py tests/unit_tests/server_runtime/test_episode_tokens.py tests/unit_tests/server_runtime/test_mcp_auto_exposure.py tests/unit_tests/server_runtime/test_checkpoint_resources_mcp.py tests/unit_tests/server_runtime/test_environment_server.py tests/unit_tests/server_runtime/test_base_server.py (after the rebase) 511 passed
pytest tests/unit_tests/server_runtime/test_base_resources_server.py tests/unit_tests/server_runtime/test_checkpoint_participant_workers.py tests/unit_tests/server_runtime/test_base_server.py tests/unit_tests/server_runtime/test_mcp_auto_exposure.py (before the rebase) 445 passed
pre-commit run on the changed files passed

Rollout evidence

N/A: infrastructure for a base class that no environment uses yet. The e2e test runs real multi-worker server processes; BIRD-Interact's move onto the base class (Linear GYMINT-646) will run rollouts.

Compatibility

  • Resources, Agent and other Environment Servers route as before; the new install_session_routing arguments are optional and they pass none.
  • SessionKind fields are now Optional. The two existing kinds set all of them.
  • A BaseEnvironmentServerWithTools now accepts num_workers > 1.

@copy-pr-bot

copy-pr-bot Bot commented Oct 30, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@ffrujeri

Copy link
Copy Markdown
Contributor Author

/ok to test efac265

@ffrujeri
ffrujeri force-pushed the ffrujeri/self-contained-session-routing branch from efac265 to d565468 Compare October 30, 2026 06:19
@ffrujeri
ffrujeri changed the base branch from ffrujeri/self-contained-environment-server to ffrujeri/self-contained-mcp October 30, 2026 06:19
@ffrujeri
ffrujeri added this pull request to stack #5352 October 30, 2026 06:21
@ffrujeri

Copy link
Copy Markdown
Contributor Author

/ok to test d565468

@svcnvidia-nemo-ci svcnvidia-nemo-ci added area:env-infra Shared environment framework, lifecycle, registry, scaffolding, and validation feature New capabilities, enhancements, or enablement work labels Oct 30, 2026
@ffrujeri
ffrujeri force-pushed the ffrujeri/self-contained-session-routing branch from d565468 to dcd7c47 Compare October 30, 2026 16:21
@ffrujeri ffrujeri changed the title feat(environment): route a self-contained Environment Server's tool calls to the worker that runs the episode feat(environment): route a turn-based Environment Server's tool calls to the worker that runs the episode Oct 30, 2026
@ffrujeri

Copy link
Copy Markdown
Contributor Author

/ok to test dcd7c47

@ffrujeri
ffrujeri force-pushed the ffrujeri/self-contained-session-routing branch from dcd7c47 to 34deb08 Compare October 31, 2026 13:16
@ffrujeri ffrujeri changed the title feat(environment): route a turn-based Environment Server's tool calls to the worker that runs the episode feat(environment): route BaseEnvironmentServerWithTools tool calls to the worker that runs the episode Oct 31, 2026
@ffrujeri

Copy link
Copy Markdown
Contributor Author

/ok to test 34deb08

ffrujeri added a commit that referenced this pull request Nov 10, 2026
…er, and survive a raising header reader

Review of #5350:

- A worker that dies leaves its owner file, so a /run resent after the
  crash found the episode owned and failed for good. serve_tools now
  raises a non-terminal handled episode failure, so the attempt budget
  retries the episode under a new attempt key.
- The router reads the session header outside the exception
  middleware, so a reader that raised became a bare 500. A reader that
  raises now names no session, and the worker the call reached handles
  it.
- The workers of a multi-worker server read each other's episode
  tokens with the secret they inherit from the main process
  (episode_token_secret, #5346).
- The e2e test bounds its wait for the episode's grant and fails at
  once, with the reply, if /run ends first.
- The routing e2e job also runs when episode_tokens.py,
  episode_tools.py, tool_access.py or core/episode_types.py change.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Felipe Vieira Frujeri <ffrujeri@nvidia.com>
@ffrujeri
ffrujeri force-pushed the ffrujeri/self-contained-session-routing branch from f51983e to dbe13b9 Compare November 10, 2026 05:23
@ffrujeri

Copy link
Copy Markdown
Contributor Author

/ok to test dbe13b9

@ffrujeri

Copy link
Copy Markdown
Contributor Author

/ok to test deec85a

@ffrujeri

Copy link
Copy Markdown
Contributor Author

/ok to test 90013e5

@ffrujeri

Copy link
Copy Markdown
Contributor Author

/ok to test 398a8cf

@adil-a

adil-a commented Nov 11, 2026

Copy link
Copy Markdown
Contributor

/review reset=True

@svcnvidia-nemo-ci svcnvidia-nemo-ci left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SHIP

LGTM — no concerns found.

I reviewed all seven changed files and traced episode ownership and cleanup, shared token signing, direct HTTP/MCP forwarding, existing agent/resources routing, and CI dependencies.

The prior bot finding is fixed: the grant wait now has a deadline, detects an early response, and cancels and awaits the request when waiting fails. It retains the router-specific 410 assertion. I also verified the nonterminal ownership-conflict response, raising-reader fallback, shared signing secret, and expanded CI filter against the current source.

One delegated inspection was cancelled; I covered its episode-lifecycle topic directly. Validation was static; I couldn't execute tests in this read-only environment.

Review generated by codex. This was a full review. Request a full review with /review reset=true.

@ffrujeri

Copy link
Copy Markdown
Contributor Author

/ok to test 794e86a

@svcnvidia-nemo-ci svcnvidia-nemo-ci left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SHIP

LGTM — no concerns found within this follow-up scope.

I rechecked the fixes recorded in the latest clean bot review:

  • The grant wait retains its deadline, early-response detection, cancellation and awaiting on failure, and router-specific 410 assertion.
  • The ownership-conflict branch remains nonterminal. I traced that value through the response and retry consumers; retries can receive a fresh attempt key within the budget.
  • The header-reader fallback, shared worker signing secret, and expanded routing-test filter remain in place. I also checked their regression assertions and directly affected wiring.

The previously reviewed tree at 398a8cf62508230fd7513f410cc37ece22419cca wasn't supplied, so I couldn't compare the inter-review changes. The supplied evidence establishes no subsequent regression; this wasn't a fresh whole-PR review. Validation was static only—I couldn't execute tests or verify current CI results in this read-only environment.

Note: formal approval is not available for this request, so this review is posted as a comment only.

Review generated by codex. This was a findings-only review. Request a full review with /review reset=true.

@ffrujeri

Copy link
Copy Markdown
Contributor Author

/ok to test 13683d1

@ffrujeri

Copy link
Copy Markdown
Contributor Author

/ok to test 0559e70

@ffrujeri

Copy link
Copy Markdown
Contributor Author

/ok to test d9df054

@ffrujeri

Copy link
Copy Markdown
Contributor Author

/ok to test 2fc1910

@github-actions

github-actions Bot commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

@ffrujeri

Copy link
Copy Markdown
Contributor Author

/ok to test f13fd8b

@ffrujeri

Copy link
Copy Markdown
Contributor Author

/ok to test 339cd1c

ffrujeri and others added 3 commits November 16, 2026 17:57
… the worker that runs the episode

With several uvicorn workers, a tool call for an episode can reach a
worker that does not hold the episode, so BaseEnvironmentServerWithTools
refused num_workers > 1. It now routes sessions to their owner, as
Resources and Agent Servers do:

- serve_tools() claims the episode, by its capture key, on the worker
  that runs it; the episode's cleanup releases the claim. A second
  serve of the same attempt on another worker is refused.
- The router reads a tool call's episode from its episode token header,
  without the nonce, which only the episode's worker checks. The MCP
  grant carries the same header and /mcp is stateless, so every MCP
  request (initialize, tools/list, tools/call) is routed the same way.
- session_routing gains a generic header source for the session ID
  (session_header with the server's reader) and SessionHolder, for a
  server that claims and releases each session itself. SessionKind's
  cookie key, body field and seed and close routes become optional.
  Resources and Agent Servers are unchanged.
- An end-to-end test runs a test-only BaseEnvironmentServerWithTools
  with four workers, with episodes that call their tools over direct
  HTTP and over MCP; the routing e2e job also runs when the base class
  changes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Felipe Vieira Frujeri <ffrujeri@nvidia.com>
…er, and survive a raising header reader

Review of #5350:

- A worker that dies leaves its owner file, so a /run resent after the
  crash found the episode owned and failed for good. serve_tools now
  raises a non-terminal handled episode failure, so the attempt budget
  retries the episode under a new attempt key.
- The router reads the session header outside the exception
  middleware, so a reader that raised became a bare 500. A reader that
  raises now names no session, and the worker the call reached handles
  it.
- The workers of a multi-worker server read each other's episode
  tokens with the secret they inherit from the main process
  (episode_token_secret, #5346).
- The e2e test bounds its wait for the episode's grant and fails at
  once, with the reply, if /run ends first.
- The routing e2e job also runs when episode_tokens.py,
  episode_tools.py, tool_access.py or core/episode_types.py change.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Felipe Vieira Frujeri <ffrujeri@nvidia.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Felipe Vieira Frujeri <ffrujeri@nvidia.com>
@ffrujeri

Copy link
Copy Markdown
Contributor Author

/ok to test 81dd153

@github-actions

Copy link
Copy Markdown
Contributor

🔄 Merge queue validation started! You can track the progress here: https://github.com/NVIDIA-NeMo/Gym/actions/runs/43943713556

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:env-infra Shared environment framework, lifecycle, registry, scaffolding, and validation feature New capabilities, enhancements, or enablement work needs-review PR is ready for code review and waiting on a reviewer

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants