Repository navigation
Open
Description
Follow-ups from a re-review of the Environment Server tool stack. #5346–#5350 and #5917 merged on 16 Nov, so those items need follow-up PRs. #5465 is still open, so its items can go in that PR. No item is major. Most are on error, cancellation and malformed-input paths, which happen often at RL scale. Line links point to main at 8a0b467c92.
Runtime bugs that can happen today
- A
call_idwith CR, LF or NUL fails the rollout after 600 s (feat(environment): add BaseEnvironmentServerWithTools, which serves an episode's tools #5347). simple_agent now sends the model'scall_idinX-NeMo-Gym-Tool-Call-Idon every direct HTTP tool call, including calls to Resources Servers. Some vLLM tool parsers, such askimi_k2, take the ID from generated text. With such an ID, aiohttp raises before sending, andhttp_requestretries for 600 s before failing. Ask: skip or percent-encode the header when the ID isn't a safe header value. simple_agent/app.py:753 - A lone surrogate character passes the new JSON check, then the reply returns a 500 (feat(environment): add BaseEnvironmentServerWithTools, which serves an episode's tools #5347).
_is_jsonusesjson.dumpswith its defaultensure_ascii=True. A lone surrogate in tool output or arguments is therefore recorded as a success, and thenJSONResponseraisesUnicodeEncodeError. The rawargumentsinToolCallRecordalso break BIRD's/runreply. Ask: checkargumentsandoutputthe way the reply encodes them (ensure_ascii=False, then UTF-8), and record a failing call as an error. base_environment_server.py:1158 - Ending an episode cancels the running call only at its LIFO cleanup slot (feat(environment): add BaseEnvironmentServerWithTools, which serves an episode's tools #5347, feat(environment): require a sandbox provider for BaseEnvironmentServerWithTools and add start_episode_sandbox #5917). A sandbox registered after
serve_tools, such as BIRD's_replace_sandbox, is stopped while the call is still running. A sandbox the call starts duringacloseis skipped and lives untilttl_s. Ask: runstop_servingbefore every other cleanup, whatever the registration order, then re-scan for entries added in the meantime. base_environment_server.py:991 - A cancelled Docker start leaks its container, and the cleanup reports success (feat(environment): require a sandbox provider for BaseEnvironmentServerWithTools and add start_episode_sandbox #5917).
AsyncSandboxstores the handle only afterprovider.createreturns, and Docker'screatedoesn't remove the container onCancelledError. So when the episode deadline or a shutdown cancels the start, the registered stop does nothing, and the container runs until its TTL. The test passes only because_FakeSandbox.stoprecords a stop unconditionally. Ask: clean up on cancellation in Docker'screate, as OpenSandbox does, and test it with a realAsyncSandbox. base_environment_server.py:936 - A failed upload of
spec.filesskips the setup fix (feat(environment): require a sandbox provider for BaseEnvironmentServerWithTools and add start_episode_sandbox #5917).sandbox.startuploads the files and stops the sandbox itself on failure, outside the cleanup timeout and the entry lock. The error can then wait for Docker's 180 srm -ftimeout. Ask: upload the files inside the helper's existingtryblock. - Harbor's and SWE-Together's own
shutdown()waits behind the whole shutdown sweep (feat(environment): finish every Environment Server's running cleanups at shutdown #5348). One context kept from an earlier failed close can hold the sweep for the fullcleanup_timeout_seconds(180 s or 300 s). The server is then SIGKILLed before Harbor removes its snapshot temp directory, or before SWE-Together stops its runtime sandbox (ttl_s=86400). Ask: give retained contexts a short budget in the sweep, or sweep them after these hooks. base_environment_server.py:538 - On mcp 1.29, a dropped
/mcpcall keeps running underepisode.lock(feat(environment): serve BaseEnvironmentServerWithTools episode tools over MCP #5349). mcp 1.29 is the locked version, and on it a client disconnect doesn't cancel the call, contrary to the class docstring. Ask: raise the floor to mcp>=1.30, or limit the docstring's claim to direct HTTP and mcp>=1.30 and point totool_call_timeout_secondsotherwise. base_environment_server.py:805 - With
tool_transports: [mcp, direct_http], neither grant is required (feat(environment): serve BaseEnvironmentServerWithTools episode tools over MCP #5349). The agents' required-grant checks then stop firing, so a failed/mcpconnection runs the rollout without tools and silently scores 0. Ask: document that the combined config gives up this check, or keep one grant required. base_environment_server.py:1006 - A
CancelledErrorraised inside a tool gets 410 "ended while the call ran" while the episode is still open (feat(environment): add BaseEnvironmentServerWithTools, which serves an episode's tools #5347). Ask: return 410 only when the episode is gone, and 500 otherwise. base_environment_server.py:1095 - A tool call named after a route (
run,docs, or any name containing/) gets 403 and is never recorded inserved.calls(feat(environment): add BaseEnvironmentServerWithTools, which serves an episode's tools #5347). The model is told the token doesn't grant the call, which reads like an auth failure. Ask: record these calls as errors, or document that they aren't recorded. base_environment_server.py:761
#5465 (still open)
- A checkpoint retire doesn't cancel an in-flight continuation activation.
SimpleAgent.retire_agent_sessiononly pops the record. Ask: close the ledger first, as the base class does. - A failed or cancelled activation drops the turns, model calls and tool calls it had already finished, including tool calls that changed Environment Server state. Ask: have
_create_episodereturn its partial trajectory when it raises. - The cancellation (
incomplete) path is untested. BIRD's turn-deadline close takes this path. Ask: add a test that closes the session while activation 2 is blocked. - Docs. simple_agent's continuation rules aren't documented, and
multi-step-tasks.mdx:223still says onlyopencode_agentresumes sessions. The PR description still describes the removedCHECKPOINT_RESTART_HEADERexport and a 409 restore path that can't be reached.
Security and tracking
- Resources Server MCP session tokens are still signed with the name-derived key (
session_token_serializerinmcp_auto_exposure.py). This predates the stack. A policy can re-sign its own token withtools: nulland skip per-session tool narrowing. - The per-run episode-token secret lives in
os.environ. Apptainer, which doesn't pass--cleanenvby default, and local sandboxes therefore pass it to the policy. The nonce check limits the impact. Ask: keep the secret out of child environments, or recommend--cleanenv/--containallfor Apptainer.
Latent (nothing in the repo triggers these yet)
- The object-schema check rejects recursive argument models, so every episode of such a tool would fail. A root
$refshould be inlined. - Only simple_agent reads
X-NeMo-Gym-Tool-Error. nooa_agent would record error results as completed and fail the rollout on a 504. - Continuation activations call
_create_episodedirectly and skip feat(agents): deliver per-task agent context to agent sessions #5703's_create_session_episodehook (feat(simple_agent): continue an agent session across activations #5465). - Running each call in its own task lets a client disconnect break anyio shields inside a tool.
Full context, code excerpts and verification notes for each item: https://claude.ai/artifact/1VfJbSqakv4sHSCaMTW6Ju
Metadata
Metadata
Assignees
Labels
No labels
Activity