Repository navigation
feat(environment): route BaseEnvironmentServerWithTools tool calls to the worker that runs the episode - #5350
Conversation
|
/ok to test efac265 |
efac265 to
d565468
Compare
|
/ok to test d565468 |
d565468 to
dcd7c47
Compare
|
/ok to test dcd7c47 |
dcd7c47 to
34deb08
Compare
|
/ok to test 34deb08 |
…er, and survive a raising header reader Review of #5350: - A worker that dies leaves its owner file, so a /run resent after the crash found the episode owned and failed for good. serve_tools now raises a non-terminal handled episode failure, so the attempt budget retries the episode under a new attempt key. - The router reads the session header outside the exception middleware, so a reader that raised became a bare 500. A reader that raises now names no session, and the worker the call reached handles it. - The workers of a multi-worker server read each other's episode tokens with the secret they inherit from the main process (episode_token_secret, #5346). - The e2e test bounds its wait for the episode's grant and fails at once, with the reply, if /run ends first. - The routing e2e job also runs when episode_tokens.py, episode_tools.py, tool_access.py or core/episode_types.py change. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Felipe Vieira Frujeri <ffrujeri@nvidia.com>
f51983e to
dbe13b9
Compare
|
/ok to test dbe13b9 |
|
/ok to test deec85a |
|
/ok to test 90013e5 |
|
/ok to test 398a8cf |
|
/review reset=True |
svcnvidia-nemo-ci
left a comment
There was a problem hiding this comment.
SHIP
LGTM — no concerns found.
I reviewed all seven changed files and traced episode ownership and cleanup, shared token signing, direct HTTP/MCP forwarding, existing agent/resources routing, and CI dependencies.
The prior bot finding is fixed: the grant wait now has a deadline, detects an early response, and cancels and awaits the request when waiting fails. It retains the router-specific 410 assertion. I also verified the nonterminal ownership-conflict response, raising-reader fallback, shared signing secret, and expanded CI filter against the current source.
One delegated inspection was cancelled; I covered its episode-lifecycle topic directly. Validation was static; I couldn't execute tests in this read-only environment.
Review generated by codex. This was a full review. Request a full review with /review reset=true.
|
/ok to test 794e86a |
svcnvidia-nemo-ci
left a comment
There was a problem hiding this comment.
SHIP
LGTM — no concerns found within this follow-up scope.
I rechecked the fixes recorded in the latest clean bot review:
- The grant wait retains its deadline, early-response detection, cancellation and awaiting on failure, and router-specific
410assertion. - The ownership-conflict branch remains nonterminal. I traced that value through the response and retry consumers; retries can receive a fresh attempt key within the budget.
- The header-reader fallback, shared worker signing secret, and expanded routing-test filter remain in place. I also checked their regression assertions and directly affected wiring.
The previously reviewed tree at 398a8cf62508230fd7513f410cc37ece22419cca wasn't supplied, so I couldn't compare the inter-review changes. The supplied evidence establishes no subsequent regression; this wasn't a fresh whole-PR review. Validation was static only—I couldn't execute tests or verify current CI results in this read-only environment.
Note: formal approval is not available for this request, so this review is posted as a comment only.
Review generated by codex. This was a findings-only review. Request a full review with /review reset=true.
|
/ok to test 13683d1 |
|
/ok to test 0559e70 |
|
/ok to test d9df054 |
|
/ok to test 2fc1910 |
|
🌿 Preview your docs: https://nvidia-preview-pr-5350-81dd1530c13c.docs.buildwithfern.com/nemo/gym |
|
/ok to test f13fd8b |
|
/ok to test 339cd1c |
… the worker that runs the episode With several uvicorn workers, a tool call for an episode can reach a worker that does not hold the episode, so BaseEnvironmentServerWithTools refused num_workers > 1. It now routes sessions to their owner, as Resources and Agent Servers do: - serve_tools() claims the episode, by its capture key, on the worker that runs it; the episode's cleanup releases the claim. A second serve of the same attempt on another worker is refused. - The router reads a tool call's episode from its episode token header, without the nonce, which only the episode's worker checks. The MCP grant carries the same header and /mcp is stateless, so every MCP request (initialize, tools/list, tools/call) is routed the same way. - session_routing gains a generic header source for the session ID (session_header with the server's reader) and SessionHolder, for a server that claims and releases each session itself. SessionKind's cookie key, body field and seed and close routes become optional. Resources and Agent Servers are unchanged. - An end-to-end test runs a test-only BaseEnvironmentServerWithTools with four workers, with episodes that call their tools over direct HTTP and over MCP; the routing e2e job also runs when the base class changes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Felipe Vieira Frujeri <ffrujeri@nvidia.com>
…er, and survive a raising header reader Review of #5350: - A worker that dies leaves its owner file, so a /run resent after the crash found the episode owned and failed for good. serve_tools now raises a non-terminal handled episode failure, so the attempt budget retries the episode under a new attempt key. - The router reads the session header outside the exception middleware, so a reader that raised became a bare 500. A reader that raises now names no session, and the worker the call reached handles it. - The workers of a multi-worker server read each other's episode tokens with the secret they inherit from the main process (episode_token_secret, #5346). - The e2e test bounds its wait for the episode's grant and fails at once, with the reply, if /run ends first. - The routing e2e job also runs when episode_tokens.py, episode_tools.py, tool_access.py or core/episode_types.py change. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Felipe Vieira Frujeri <ffrujeri@nvidia.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Felipe Vieira Frujeri <ffrujeri@nvidia.com>
|
/ok to test 81dd153 |
|
🔄 Merge queue validation started! You can track the progress here: https://github.com/NVIDIA-NeMo/Gym/actions/runs/43943713556 |
Stacked on #5349 (MCP). The stack: #5346 (episode tokens) → #5347 (the base class) → #5348 (shutdown sweep) → #5349 (MCP) → this PR. Review those first; this PR's diff against #5349 is the routing.
What changed
With several uvicorn workers, a tool call for an episode can reach a worker that does not hold the episode.
BaseEnvironmentServerWithToolsnow routes each call to the worker that runs the episode, with the session routing that Resources and Agent Servers already use, and the interimnum_workers > 1refusal from #5347 is gone.serve_tools()claims the episode, by its capture key, on the worker that runs it. The episode's cleanup releases the claim when it stops serving the tools. A retried attempt (-a<N>) has its own key, so its own claim. Serving the same attempt on another worker is refused.X-NeMo-Gym-Episode-Token). It does not need the nonce; only the episode's worker checks it, so a forged token is forwarded and then refused there with 403.tools/list,tools/call) is routed by it./mcpis stateless Streamable HTTP, so no MCP session lives on a worker; each request only has to reach the episode's worker, which the e2e test checks.nemo_gym/server_runtime/request_handling/session_routing.py, generic and opt-in:session_headerandsession_from_header: a header that names the session, with the server's reader of its value, checked before the cookie and MCP token.SessionHolder: a server that claims and releases each session itself, as an Agent Server releases its own.SessionKind's cookie key, body field, and seed and close routes may beNone, for a kind named only in a header.SessionKind.path_claim, feat(agent): let a caller stop a running rollout and still grade it #5145) is unchanged. The episode kind sets none, so a tool call never takes that path.nemo_gym/server_runtime/base_environment_server.pychanges.Why
Linear GYMINT-645. Single-worker is not an option at Gym's scale, so an Environment Server that serves tools needs the same per-worker routing as a Resources Server. This removes the limit #5347 had to add.
Validation
pytest -n 12 tests/unit_tests/server_runtime/ tests/unit_tests/test_public_api.py tests/unit_tests/test_import_linter_coverage.py tests/unit_tests/test_moved_paths.py(after the rebase onto main atdda609f86)NEMO_GYM_MULTIWORKER_E2E=1 pytest tests/e2e/session_routing(after the rebase onto main atdda609f86)test_every_mcp_request_runs_on_the_worker_that_runs_its_episodeand main'stest_session_of_an_exited_worker_gets_an_explicit_error; both passed when rerunNEMO_GYM_MULTIWORKER_STRESS=1 NEMO_GYM_STRESS_EPISODES=400 NEMO_GYM_STRESS_SESSIONS=100 pytest tests/e2e/session_routing/test_session_routing_stress.py(after the rebase onto main atdda609f86)pytest tests/unit_tests/server_runtime/test_episode_tools_session_routing.pyserve_toolsdisabledNEMO_GYM_MULTIWORKER_E2E=1 pytest tests/e2e/session_routing/test_episode_tools_routing_e2e.pyBaseEnvironmentServerWithToolswith 4 real workers andtool_transports: [direct_http, mcp]: 200 episodes each make 8 direct HTTP tool calls through the public port on new connections; all 1,600 return 200 and ran on their episode's worker, and the episodes ran on several workers. 50 episodes each act as an MCP client (the MCP SDK's, without keep-alive, so each request lands on any worker): initialize,tools/listand 4tools/call; every episode listed exactly its tool and all 200 calls ran on its worker. Killing the worker that runs an episode makes its next tool call 410.tools/listrequests that reached other workers listed no toolsNEMO_GYM_MULTIWORKER_E2E=1 pytest tests/e2e/session_routingNEMO_GYM_MULTIWORKER_STRESS=1 NEMO_GYM_STRESS_EPISODES=400 NEMO_GYM_STRESS_SESSIONS=100 pytest tests/e2e/session_routing/test_session_routing_stress.py(before the rebase)pytest tests/unit_tests/server_runtime/test_base_environment_server_with_tools.py tests/unit_tests/server_runtime/test_episode_tools_session_routing.py tests/unit_tests/server_runtime/test_session_routing.py tests/unit_tests/server_runtime/test_episode_tools.py tests/unit_tests/server_runtime/test_episode_tokens.py tests/unit_tests/server_runtime/test_mcp_auto_exposure.py tests/unit_tests/server_runtime/test_checkpoint_resources_mcp.py tests/unit_tests/server_runtime/test_environment_server.py tests/unit_tests/server_runtime/test_base_server.py(after the rebase)pytest tests/unit_tests/server_runtime/test_base_resources_server.py tests/unit_tests/server_runtime/test_checkpoint_participant_workers.py tests/unit_tests/server_runtime/test_base_server.py tests/unit_tests/server_runtime/test_mcp_auto_exposure.py(before the rebase)pre-commit runon the changed filesRollout evidence
N/A: infrastructure for a base class that no environment uses yet. The e2e test runs real multi-worker server processes; BIRD-Interact's move onto the base class (Linear GYMINT-646) will run rollouts.
Compatibility
install_session_routingarguments are optional and they pass none.SessionKindfields are nowOptional. The two existing kinds set all of them.BaseEnvironmentServerWithToolsnow acceptsnum_workers > 1.