Repository navigation
Conversation
SandboxPythonRuntime raised a plain RuntimeError when the task sandbox's platform has no runtime (not x86_64/aarch64 Linux, or glibc older than 2.17). Hermes, Stirrup and BrowseComp install the runtime in their seed hook, so the agent route answered 500 and the Environment Server reran the whole episode on a new sandbox from the same image, which fails the same way every time. The helper now raises UnsupportedSandboxPlatformError (a RuntimeError subclass, exported from nemo_gym.agents) for those cases. Each seed hook turns only that error into AgentSeedRejectedError (final 422) after its failed setup is cleaned up. A failed probe, host build, download or timeout stays retryable. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com>
Contributor
Author
|
/ok to test c8151f1 |
Contributor
|
🌿 Preview your docs: https://nvidia-preview-pr-5949-5abdfbf22c7b.docs.buildwithfern.com/nemo/gym |
yaoyu-33
marked this pull request as ready for review
November 10, 2026 17:44
Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com>
Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com>
Contributor
Author
|
/ok to test 5abdfbf |
This branch had an error being deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed and why
SandboxPythonRuntime.install()raised a plainRuntimeErrorwhen the task sandbox runs on a platform no runtime is built for: not x86_64/aarch64 Linux, or glibc older than 2.17. Hermes, Stirrup and BrowseComp callinstall()in their seed hook (_seed_agent_session_state), so the agent route turned that into HTTP 500 (agent_route_failed). The Environment Server treats a 5xx seed as retryable and reruns the whole episode on a new sandbox, but the new sandbox comes from the same image and fails the same way on every attempt.The agent review checklist (item 2) says a seed the agent cannot serve must raise
AgentSeedRejectedError(final 422). This PR does that:nemo_gym/agents/utils/sandbox_python.py: newUnsupportedSandboxPlatformError(RuntimeError), raised by the platform probe (OS or machine), the glibc floor inSandboxPlatform.wheel_platform, and the "no pinned Python build" check. It subclassesRuntimeError, so existing callers that catchRuntimeErrorbehave as before. It is exported fromnemo_gym.agents.AgentSeedRejectedError. BrowseComp already did this forSandboxIsolationError; the other two now use the same pattern. If cleanup fails, the seed keeps today's behavior: the session is retained for close and the original error is raised.add-or-update-a-harness.mdx): one sentence on the refusal, next to the existing list of supported sandbox platforms.No issue: this came up while reviewing a harness PR against the agent review checklist. The fix is in shared code, so it gets its own PR. The DeepAgents sessions PR (#5003) also calls
SandboxPythonRuntime.install()in its seed hook. It needs the same mapping once it rebases onto this.Validation
Current head
5abdfbf22c7b82bd4fd898498377978412db0ad9merges main64348970d991bb36ef28ec465ee87cafbe28d760. Both import conflicts were resolved by preserving the new platform error alongside main's platform probe and timeout helpers.Validated on a Linux workstation with Python 3.13.14:
python -m pytest tests/unit_tests/agents/test_sandbox_python.py responses_api_agents/hermes_agent/tests/ responses_api_agents/browsecomp_agent/tests/ tests/unit_tests/test_public_api_facades.py tests/unit_tests/test_public_api.py -q -o addopts= -o log_cli=false: 1,559 passed, with five configuration checks initially failing because reused dependencies exposed old installed Gym metadata (openai==2.44.0) ahead of the merged source'sopenai==2.45.0. After installing the current metadata and isolating Hermes's dependency paths,python -m pytest responses_api_agents/hermes_agent/tests/test_benchmark_profile.py responses_api_agents/hermes_agent/tests/test_recipe.py --lf -q -o addopts= -o log_cli=falsepassed all 5 failed checks. No source change was needed.python -m pytest responses_api_agents/stirrup_agent/tests/ -q -o addopts= -o log_cli=false: 186 passed.PYTHONPATHandNEMO_GYM_EXTRA_ROOTS, with source imports verified.python -m pre_commit run --all-files: all applicable hooks passed, with no source changes.ac12343863e5175baf1ba1b68dc95d407c280183;git diff --checkpassed and all three PR commits carry DCO sign-offs.The review found that glibc rejection originally happened after downloading Python. A download failure could therefore hide the permanent incompatibility and still return retryable HTTP 500. The follow-up checks the glibc floor immediately after probing, before any download.
Both strengthened regressions were run against the original installer and failed for glibc 2.12 (with the Darwin control passing), then passed with the fix:
The existing tests also cover unsupported OS/architecture, no pinned build, transient probe failures remaining retryable, Hermes cleanup before refusal, and BrowseComp isolation/platform refusal.
Full repository CI is required because this PR changes shared core code. CI authorization has been requested for the current full head SHA; earlier CI results do not validate this merged head.
Rollout evidence
N/A. The change affects only seeds on sandbox images the runtime cannot serve. Such episodes already failed; they now fail once with a 422 instead of being retried. There is no score impact beyond fewer reruns. Supported platforms take the same code path as before, so no model rollout is needed.
Compatibility
UnsupportedSandboxPlatformErroris a new public name innemo_gym.agents. It subclassesRuntimeError, so no caller has to change.🤖 Generated with Claude Code