Skip to content

Environment Server stack follow-ups from the post-merge re-review (#5346–#5350, #5917, #5465) #6273

Open

Description

@linear-code

Follow-ups from a re-review of the Environment Server tool stack. #5346–#5350 and #5917 merged on 16 Nov, so those items need follow-up PRs. #5465 is still open, so its items can go in that PR. No item is major. Most are on error, cancellation and malformed-input paths, which happen often at RL scale. Line links point to main at 8a0b467c92.

Runtime bugs that can happen today

  1. A call_id with CR, LF or NUL fails the rollout after 600 s (feat(environment): add BaseEnvironmentServerWithTools, which serves an episode's tools #5347). simple_agent now sends the model's call_id in X-NeMo-Gym-Tool-Call-Id on every direct HTTP tool call, including calls to Resources Servers. Some vLLM tool parsers, such as kimi_k2, take the ID from generated text. With such an ID, aiohttp raises before sending, and http_request retries for 600 s before failing. Ask: skip or percent-encode the header when the ID isn't a safe header value. simple_agent/app.py:753
  2. A lone surrogate character passes the new JSON check, then the reply returns a 500 (feat(environment): add BaseEnvironmentServerWithTools, which serves an episode's tools #5347). _is_json uses json.dumps with its default ensure_ascii=True. A lone surrogate in tool output or arguments is therefore recorded as a success, and then JSONResponse raises UnicodeEncodeError. The raw arguments in ToolCallRecord also break BIRD's /run reply. Ask: check arguments and output the way the reply encodes them (ensure_ascii=False, then UTF-8), and record a failing call as an error. base_environment_server.py:1158
  3. Ending an episode cancels the running call only at its LIFO cleanup slot (feat(environment): add BaseEnvironmentServerWithTools, which serves an episode's tools #5347, feat(environment): require a sandbox provider for BaseEnvironmentServerWithTools and add start_episode_sandbox #5917). A sandbox registered after serve_tools, such as BIRD's _replace_sandbox, is stopped while the call is still running. A sandbox the call starts during aclose is skipped and lives until ttl_s. Ask: run stop_serving before every other cleanup, whatever the registration order, then re-scan for entries added in the meantime. base_environment_server.py:991
  4. A cancelled Docker start leaks its container, and the cleanup reports success (feat(environment): require a sandbox provider for BaseEnvironmentServerWithTools and add start_episode_sandbox #5917). AsyncSandbox stores the handle only after provider.create returns, and Docker's create doesn't remove the container on CancelledError. So when the episode deadline or a shutdown cancels the start, the registered stop does nothing, and the container runs until its TTL. The test passes only because _FakeSandbox.stop records a stop unconditionally. Ask: clean up on cancellation in Docker's create, as OpenSandbox does, and test it with a real AsyncSandbox. base_environment_server.py:936
  5. A failed upload of spec.files skips the setup fix (feat(environment): require a sandbox provider for BaseEnvironmentServerWithTools and add start_episode_sandbox #5917). sandbox.start uploads the files and stops the sandbox itself on failure, outside the cleanup timeout and the entry lock. The error can then wait for Docker's 180 s rm -f timeout. Ask: upload the files inside the helper's existing try block.
  6. Harbor's and SWE-Together's own shutdown() waits behind the whole shutdown sweep (feat(environment): finish every Environment Server's running cleanups at shutdown #5348). One context kept from an earlier failed close can hold the sweep for the full cleanup_timeout_seconds (180 s or 300 s). The server is then SIGKILLed before Harbor removes its snapshot temp directory, or before SWE-Together stops its runtime sandbox (ttl_s=86400). Ask: give retained contexts a short budget in the sweep, or sweep them after these hooks. base_environment_server.py:538
  7. On mcp 1.29, a dropped /mcp call keeps running under episode.lock (feat(environment): serve BaseEnvironmentServerWithTools episode tools over MCP #5349). mcp 1.29 is the locked version, and on it a client disconnect doesn't cancel the call, contrary to the class docstring. Ask: raise the floor to mcp>=1.30, or limit the docstring's claim to direct HTTP and mcp>=1.30 and point to tool_call_timeout_seconds otherwise. base_environment_server.py:805
  8. With tool_transports: [mcp, direct_http], neither grant is required (feat(environment): serve BaseEnvironmentServerWithTools episode tools over MCP #5349). The agents' required-grant checks then stop firing, so a failed /mcp connection runs the rollout without tools and silently scores 0. Ask: document that the combined config gives up this check, or keep one grant required. base_environment_server.py:1006
  9. A CancelledError raised inside a tool gets 410 "ended while the call ran" while the episode is still open (feat(environment): add BaseEnvironmentServerWithTools, which serves an episode's tools #5347). Ask: return 410 only when the episode is gone, and 500 otherwise. base_environment_server.py:1095
  10. A tool call named after a route (run, docs, or any name containing /) gets 403 and is never recorded in served.calls (feat(environment): add BaseEnvironmentServerWithTools, which serves an episode's tools #5347). The model is told the token doesn't grant the call, which reads like an auth failure. Ask: record these calls as errors, or document that they aren't recorded. base_environment_server.py:761

#5465 (still open)

  1. A checkpoint retire doesn't cancel an in-flight continuation activation. SimpleAgent.retire_agent_session only pops the record. Ask: close the ledger first, as the base class does.
  2. A failed or cancelled activation drops the turns, model calls and tool calls it had already finished, including tool calls that changed Environment Server state. Ask: have _create_episode return its partial trajectory when it raises.
  3. The cancellation (incomplete) path is untested. BIRD's turn-deadline close takes this path. Ask: add a test that closes the session while activation 2 is blocked.
  4. Docs. simple_agent's continuation rules aren't documented, and multi-step-tasks.mdx:223 still says only opencode_agent resumes sessions. The PR description still describes the removed CHECKPOINT_RESTART_HEADER export and a 409 restore path that can't be reached.

Security and tracking

  1. Resources Server MCP session tokens are still signed with the name-derived key (session_token_serializer in mcp_auto_exposure.py). This predates the stack. A policy can re-sign its own token with tools: null and skip per-session tool narrowing.
  2. The per-run episode-token secret lives in os.environ. Apptainer, which doesn't pass --cleanenv by default, and local sandboxes therefore pass it to the policy. The nonce check limits the impact. Ask: keep the secret out of child environments, or recommend --cleanenv/--containall for Apptainer.

Latent (nothing in the repo triggers these yet)

  1. The object-schema check rejects recursive argument models, so every episode of such a tool would fail. A root $ref should be inlined.
  2. Only simple_agent reads X-NeMo-Gym-Tool-Error. nooa_agent would record error results as completed and fail the rollout on a 504.
  3. Continuation activations call _create_episode directly and skip feat(agents): deliver per-task agent context to agent sessions #5703's _create_session_episode hook (feat(simple_agent): continue an agent session across activations #5465).
  4. Running each call in its own task lets a client disconnect break anyio shields inside a tool.

Full context, code excerpts and verification notes for each item: https://claude.ai/artifact/1VfJbSqakv4sHSCaMTW6Ju

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions