Repository navigation
fix(agent): stop waiting for a stalled harness after a rollout stop - #6452
Open
gwarmstrong wants to merge 2 commits into
Open
gwarmstrong wants to merge 2 commits into
gwarmstrong wants to merge 2 commits into
Conversation
A rollout stop (NVIDIA-NeMo#5145) only writes the supervisor's expire.request and returns; the activation then waits on the supervised command until the provider's execution deadline, the agent timeout plus the cleanup reserve. When memory pressure stalls the sandbox, the supervisor does not act on the marker, and Gym keeps polling the command for hours. In a TB-Science run, a Codex rollout stopped at its 8 h budget came back 292 minutes later, with its model replicas held all along. Once the expire marker is in the sandbox, the activation now waits for the command only for the supervisor's own cleanup bound: its three cleanup phases plus 30 s (cleanup_bound, the same reserve the provider deadline keeps after the harness timeout; session_close_timeout_seconds plus 30 s, 90 s by default and 330 s for TB-Science). Agents can set rollout_stop_grace_seconds instead. Past it, the session logs one warning naming the rollout and reads the supervisor's receipt within the close timeout, without a stop request, which would turn the expiry into a cancellation if the supervisor resumed: - a receipt that confirms termination (the command's status hung after the supervisor finished) lets the activation capture and be graded as a stop, as before; - otherwise termination is unconfirmed, as after a negative receipt: the session records harness_termination_unconfirmed, stops polling the command, and raises it, so the rollout fails cleanly instead of hanging. Close releases a borrowed sandbox for the Resources Server's teardown, or stops an owned one, unchanged. stop_and_confirm_cleanup now reads the receipt through the new read_cleanup_receipt; its behavior is unchanged. Signed-off-by: gwarmstrong <gwarmstrong@users.noreply.github.com>
SimpleResponsesAPIAgent.stop_rollout now passes grace to SandboxSession.expire; the session-routing tests' stand-in session must accept it. Signed-off-by: gwarmstrong <gwarmstrong@users.noreply.github.com>
Contributor
Author
|
/ok to test fe71308 |
Contributor
|
🌿 Preview your docs: https://nvidia-preview-pr-6452-fe713081f9c3.docs.buildwithfern.com/nemo/gym |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
This follows up #5145 (rollout stop). A rollout stop only writes the supervisor's
expire.requestmarker and returns. The activation then waits on the supervised command until the provider's execution deadline, which is the agent timeout plus the cleanup reserve. When memory pressure stalls the sandbox, the supervisor never acts on the marker, and Gym keeps polling the command for hours. In a TB-Science run, a Codex rollout was stopped at its 8 h budget and came back 292 minutes later. Its model replicas were held the whole time.With this change, once the expiry marker is in the sandbox, the activation waits for the command only as long as the supervisor's own cleanup bound. That bound is the supervisor's three cleanup phases plus 30 s:
session_close_timeout_secondsplus 30 s, which is 90 s by default and 330 s for TB-Science. It is the same reserve the provider deadline keeps after the harness timeout. An agent can setrollout_stop_grace_secondsto use a different value. Once the bound passes, the session logs one warning that names the rollout. It then reads the supervisor's receipt within the close timeout. It does not send a stop request, because a stop request would turn the expiry into a cancellation if the supervisor resumed.harness_termination_unconfirmed, stops polling the command and raises the failure, so the rollout fails cleanly instead of hanging. Close works as before: it releases a borrowed sandbox for the Resources Server's teardown, or stops an owned one.Changes:
supervisor_client:cleanup_bound()is split out ofexec_timeout(). The newread_cleanup_receipt()reads the receipt without asking anything of the supervisor.stop_and_confirm_cleanupnow uses it, and its behavior is unchanged.SandboxSession:expire()takes an optionalgrace. New_await_exec()waits for the command with that bound once an expiry is delivered, and new_stop_overdue_harness()settles a harness that outlives it.BaseResponsesAPIAgentConfig.rollout_stop_grace_secondsis optional. Unset means the cleanup bound. It is passed toexpire()both from the stop route and for a stop that arrives during the seed.HARNESS_TERMINATION_UNCONFIRMEDcomment now covers the new cause, and the harness guide (add-or-update-a-harness.mdx) documents the bound.graceargument.No issue was filed. The failure is the TB-Science run described above.
Validation
All runs are on upstream
mainat 155b27d plus this branch:pytest tests/unit_tests/agents tests/unit_tests/server_runtime/test_session_routing.py: 735 passed, 2 skipped.pytest responses_api_agents/codex_agent/tests tests/unit_tests/core: 1257 passed, 6 skipped and 1 failed. The failure istest_opensandbox_sdk_is_available_in_agent_environment, which checks the Codex agent's own venv for the OpenSandbox SDK. It fails because I ran from the root venv. It is unrelated to this change.pre-commit run --files <the 9 changed files>: all hooks pass.supervisor_client.py: 15/15.sandbox_session.py: 53/55 (97.4% for the whole file). The 2 missed lines are the warning logged when releasing the overdue command itself fails.test_sandbox_session.py: a stopped harness that outlives its supervisor fails within the grace (borrowed and owned sandboxes, stop before and after launch, receipt missing or hanging). A hung command status is graded from a confirming receipt. A negative receipt found after the grace is the failure. A supervisor that ends the harness promptly is waited for without a warning. The default grace is the cleanup bound.test_sandbox_sessions.py: a stalled Codex rollout fails cleanly after the grace, and one whose command status hangs is still graded.Rollouts
This was a TB-Science smoke with the Codex agent on OpenSandbox, run locally through the EFB runner against a multiplexer. It used Gym a631623d4a, which is this fix plus unrelated branch work, before the rebase onto
main. It ran 2 tasks withsession_close_timeout_seconds: 300.One rollout's sandbox processes were frozen with SIGSTOP (the supervisor, runner, node and codex), and the rollout was then stopped:
the Codex harness did not end within 330 s of its stop; no longer waiting for its command.harness_termination_unconfirmed(it did not end within 330 s of its stop, and no receipt confirms that it ended).Without the fix, the same stall kept the rollout until the provider deadline, 292 minutes after the stop in the original run.
Compatibility and benchmark impact
rollout_stop_grace_seconds. Leaving it unset gives the cleanup bound. Existing configs need no change.harness_termination_unconfirmedand no longer waits until the provider deadline. A supervisor that works as designed ends its harness within the bound, so those results do not change. Runs that do not use the stop route are not affected.SandboxSession.expire()takes a new optional keyword,grace. Test doubles that replaceexpiremust accept it, as the routing tests' fake now does.Checklist
pre-commit run --all-files) (so CI lint/format checks pass). Run on the changed files only; see Validation.git commit -s) (so the DCO check passes).