Repository navigation
Conversation
|
/ok to test baba7e3 |
baba7e3 to
81a94e0
Compare
|
/ok to test 81a94e0 |
81a94e0 to
ced94d4
Compare
|
/ok to test ced94d4 |
ced94d4 to
09d7ce5
Compare
|
/ok to test 09d7ce5 |
…refuse a token-less MCP listing Review of #5349: - With tool_transports [mcp, direct_http], both grants were required, and every agent refuses a required grant it cannot use, so no agent could take the episode. With both transports, each grant is now optional, so each agent takes the one it supports. - tools/list without a valid episode token returned an empty list, so an MCP-only harness that checks only that its server connected ran the episode with no tools. It is now a JSON-RPC error. - Tests cover a broken tool, invalid arguments and another episode's tool over MCP: each is an isError result and is recorded as an error. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Felipe Vieira Frujeri <ffrujeri@nvidia.com>
9b3eb26 to
659f7eb
Compare
|
/ok to test 659f7eb |
|
/ok to test cbede0d |
|
/ok to test d442eb9 |
|
🌿 Preview your docs: https://nvidia-preview-pr-5349-e278016b097b.docs.buildwithfern.com/nemo/gym |
|
/ok to test a2fc487 |
|
/review reset=True |
svcnvidia-nemo-ci
left a comment
There was a problem hiding this comment.
SHIP — one documentation NOTE; no confirmed functional regressions found.
I reviewed all five changed files and traced episode authorization, shared call serialization and recording, grant consumers, MCP mounting and cleanup, Resources Server compatibility, and renamed imports. The new guide section addresses the earlier configuration-documentation gap, apart from the current worker limit noted inline.
This was a source-only review. I didn't execute tests or independently verify the validation results reported in the PR.
Review generated by codex. This was a full review. Request a full review with /review reset=true.
|
/ok to test 6790cc4 |
|
/ok to test 5e96ace |
|
/ok to test 63d55fb |
|
/ok to test 8eec981 |
|
/ok to test c128e50 |
|
/ok to test 0206e88 |
|
/ok to test 957f853 |
… over MCP Codex, Pi and Hermes accept only MCP tool grants, so a BaseEnvironmentServerWithTools serves an episode's toolset over MCP as well. - BaseEnvironmentServerWithToolsConfig.tool_transports picks the grants serve_tools returns: direct_http (the default) and/or mcp. - With mcp, the server mounts /mcp. tools/list shows only the tools of the episode that the X-NeMo-Gym-Episode-Token header names; a missing or invalid token lists none. - An MCP tools/call takes the same path as a direct HTTP call: the same token checks, one call at a time per episode, and the same ledger. - mcp_auto_exposure gains mount_mcp_endpoint, extracted from install_auto_exposure, and to_call_tool_result (was _to_result) with an is_error flag. Resources Server auto-exposure is unchanged. - An MCP grant refuses tool names MCP clients reject, such as a dot. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Felipe Vieira Frujeri <ffrujeri@nvidia.com>
…refuse a token-less MCP listing Review of #5349: - With tool_transports [mcp, direct_http], both grants were required, and every agent refuses a required grant it cannot use, so no agent could take the episode. With both transports, each grant is now optional, so each agent takes the one it supports. - tools/list without a valid episode token returned an empty list, so an MCP-only harness that checks only that its server connected ran the episode with no tools. It is now a JSON-RPC error. - Tests cover a broken tool, invalid arguments and another episode's tool over MCP: each is an isError result and is recorded as an error. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Felipe Vieira Frujeri <ffrujeri@nvidia.com>
… tools over MCP The MCP guide covered only a Resources Server's expose_tools_over_mcp. Add a section on BaseEnvironmentServerWithToolsConfig.tool_transports: the YAML, which grants each value gives and which agents take them, passing ServedTools.tool_accesses unchanged to the agent's seed, and how a tool's error result reaches the agent over each transport. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Felipe Vieira Frujeri <ffrujeri@nvidia.com>
… worker Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Felipe Vieira Frujeri <ffrujeri@nvidia.com>
|
/ok to test e278016 |
|
🔄 Merge queue validation started! You can track the progress here: https://github.com/NVIDIA-NeMo/Gym/actions/runs/43943710542 |
Stacked on #5348 (shutdown sweep), which is on #5347 (
BaseEnvironmentServerWithTools) and #5346 (episode tokens). Review those first; this PR's diff against #5348 is MCP only. The stack: #5346 → #5347 → #5348 → #5349 → #5350 (multi-worker routing).What changed
A
BaseEnvironmentServerWithToolscan serve an episode's tools over MCP as well as direct HTTP.BaseEnvironmentServerWithToolsConfig.tool_transports(on the config class from feat(environment): add BaseEnvironmentServerWithTools, which serves an episode's tools #5347) lists the grantsserve_toolsreturns, in order:direct_http(the default, so existing behavior is unchanged) and/ormcp. The opt-in is not namedexpose_tools_over_mcp:run_webserverruns Resources Server auto-exposure for any config with that field, which would harvest routes and wrap a/seed_sessionthis server does not have./mcpis mounted only whenmcpis configured, so a server without it never imports the MCP SDK. The grant is anMCPToolAccessnamed after the server, whose connection points at<server>/mcpwith the episode token inX-NeMo-Gym-Episode-Token, the same token as the direct HTTP grant.tools/listshows only the tools of the episode the token names, built from the episode'sEpisodeToolsetspecs, not routes harvested at startup. A missing or invalid token lists no tools.tools/calltakes the same path as a direct HTTP call (_episode_for_token, then_run_tool): the same token checks, one call at a time per episode whatever the transport, the sameToolCallRecordledger, and the same error handling. A refusal (no token, another grant's nonce, an episode that has ended) and a tool error come back as an MCP result withisError: true.mcpconfigured,serve_toolsrefuses a tool name MCP clients reject, such as one with a dot./mcpruns its session manager in the app's lifespan, and feat(environment): finish every Environment Server's running cleanups at shutdown #5348's shutdown sweep wraps that lifespan, so at shutdown the MCP manager stops and then the sweep finishes the running episodes' cleanups.nemo_gym/server_runtime/mcp_auto_exposure.py:mount_mcp_endpointis extracted frominstall_auto_exposure(the stateless Streamable HTTP manager, the/mcproute and mount, and the lifespan), and_to_resultbecomes the publicto_call_tool_resultwith anis_errorflag. The old name still resolves, with a deprecation warning, through aMOVED_NAMESentry innemo_gym/_moved_paths.py, as main's import-surface test requires of every name a module had. Resources Server auto-exposure behaves as before.environment-tutorials/mcp-resources-server.mdx) gets an "Environment Servers that serve their own tools" section: thetool_transportsYAML, which grants each value gives and which agents take them, passingServedTools.tool_accessesunchanged to the agent's seed, and how a tool's error result reaches the agent over each transport. It says thatexpose_tools_over_mcpis a different, Resources Server field.Why
Codex, Pi and Hermes accept only MCP tool grants, so an environment that serves its tools from its Environment Server, instead of a Resources Server, needs MCP to run them (Linear GYMINT-644). BIRD-Interact's a-Interact runs on Codex over MCP today through its Resources Server; this keeps that possible after it moves.
Validation
pytest tests/unit_tests/test_moved_paths.py tests/unit_tests/test_public_api.py tests/unit_tests/test_import_linter_coverage.py tests/unit_tests/server_runtime/test_mcp_auto_exposure.py tests/unit_tests/server_runtime/test_checkpoint_resources_mcp.py tests/unit_tests/server_runtime/test_base_environment_server_with_tools.py tests/unit_tests/server_runtime/test_episode_tools.py tests/unit_tests/server_runtime/test_episode_tokens.py tests/unit_tests/server_runtime/test_environment_server.py(after the rebase onto main atdda609f86)test_every_recorded_name_still_resolves_at_its_old_path), which fails without the_to_resultentry. Importingbase_environment_serverdoes not load the MCP SDKpytest tests/unit_tests/server_runtime/test_base_environment_server_with_tools.py(after the rebase onto #5348)/mcpmounted (the MCP manager's lifespan and the sweep both run), plus 11 MCP tests with the MCP SDK's own client (ClientSessionoverstreamablehttp_client) against the app in-process. They cover: the grants for eachtool_transportssetting share one token; no/mcpwithoutmcp;tools/listshows exactly the episode's tools; an MCP call and a direct HTTP call change the same state and land in one ledger, in order; a missing or garbage token lists no tools and its call is refused; an MCP call after the episode ends is refused; two concurrent episodes each list only their own tools; a dotted tool name is refused; a repeated transport is refused.call_toolcalls the toolset directly instead of the shared pathtest_mcp_lists_the_episodes_tools_and_shares_its_calls_with_direct_httpfails (the MCP call is missing from the ledger)list_toolslists every open episode's toolspytest tests/unit_tests/server_runtime/test_mcp_auto_exposure.py tests/unit_tests/server_runtime/test_checkpoint_resources_mcp.py tests/unit_tests/server_runtime/test_environment_server.py tests/unit_tests/server_runtime/test_episode_tools.py tests/unit_tests/server_runtime/test_episode_tokens.py(with the test above, at this PR's head after the rebase onto #5348)pytest tests/unit_tests/server_runtime/test_session_routing.py tests/unit_tests/server_runtime/test_checkpoint_participant_workers.pycd fern && npm run check(docs section)pre-commit run --fileson the changed filesRollout evidence
N/A: no environment uses the base class yet. A Codex smoke run over MCP comes with BIRD-Interact's move onto it (Linear GYMINT-646).
Compatibility
BaseEnvironmentServerWithToolsConfiggainstool_transports, which defaults to[direct_http]. The class is new in feat(environment): add BaseEnvironmentServerWithTools, which serves an episode's tools #5347, so nothing else uses it.mcp_auto_exposure._to_resultis renamedto_call_tool_result; it had no callers outside the module. Resources Server auto-exposure is unchanged.BaseEnvironmentServerWithToolsstill refusesnum_workers > 1in this PR; feat(environment): route BaseEnvironmentServerWithTools tool calls to the worker that runs the episode #5350 adds multi-worker routing, which covers MCP requests through the same episode token header.