Repository navigation
feat(agent): let a caller stop a running rollout and still grade it - #5145
Conversation
POST /rollouts/{rollout_id}/stop ends a rollout's harness exactly as at
its time limit, so the episode is still verified and graded as a
timeout. It is for callers that enforce their own budget (agent time,
tokens, cost) and for operators. Cancelling the /run request instead
discards the rollout.
The supervisor gains an expire request file that moves its deadline to
now; one present before launch keeps the harness from starting. No
stop.request marker is written, so harness runners see their deadline
path. SandboxSession.expire writes the file, or marks the next launch
when the harness has not started. A stop during seeding applies when
the harness would launch.
Agents opt in by returning the rollout's SandboxSession from
_rollout_sandbox_session and passing the response through
report_rollout_stop, which keeps the timeout's incomplete status and
adds the agent_limit cause. Other agents answer 501. Codex opts in.
Signed-off-by: gwarmstrong <gwarmstrong@users.noreply.github.com>
Signed-off-by: gwarmstrong <gwarmstrong@users.noreply.github.com>
… failure kind Signed-off-by: gwarmstrong <gwarmstrong@users.noreply.github.com>
|
/ok to test 21744ea |
|
🌿 Preview your docs: https://nvidia-preview-pr-5145-caacdbdc2e51.docs.buildwithfern.com/nemo/gym |
svcnvidia-nemo-ci
left a comment
There was a problem hiding this comment.
SHIP WITH CARE — three RISK findings: rollout-only stops can miss the owning worker, expiry delivery can queue behind the harness, and an immediate post-launch stop can become an ungraded artifact failure.
Completed review of all 12 changed files and their directly affected routing, provider, artifact, and grading paths. The findings below include correction invariants and proposed regression checks. Static inspection only; no tests were executed.
Review generated by codex. This was a full review. Request a full review with /review reset=true.
Signed-off-by: gwarmstrong <gwarmstrong@users.noreply.github.com>
Signed-off-by: gwarmstrong <gwarmstrong@users.noreply.github.com>
Signed-off-by: gwarmstrong <gwarmstrong@users.noreply.github.com>
Signed-off-by: gwarmstrong <gwarmstrong@users.noreply.github.com>
|
/ok to test ed95769 |
yuhezhang-ai
left a comment
There was a problem hiding this comment.
Thanks for the follow-up fixes. I rechecked ed9576949 before posting: the cookieless cross-worker stop and the stop-before-events startup reproductions now pass, so I am dropping those original findings. The new Linux Local-provider tests also cover the actual marker/supervisor path that was missing from the earlier tests; I could not execute those Linux-only cases on this macOS host.
One issue remains in the startup fix: a genuine transcript download failure after a stop is now converted into an empty graded response. I would address that before merging. The other three inline comments are nonblocking clarifications or follow-ups.
I do not see a reason to wait solely for #3302 or #5072. The intended separation still makes sense: an intentional stop can be a normal graded result, while genuine agent, artifact-collection, and verifier failures retain their normal failure handling. The artifact-download comment matters to that boundary. I have not independently rerun the combined #3302 branch.
Validation on the current revision: 23 focused checks passed, including the routing/startup reproductions and fault-injection probes described inline; two Linux-only tests skipped. This was a focused check of the updates after the earlier full review, not a repeat of the full suite or a live model/container run.
…-stop Signed-off-by: gwarmstrong <gwarmstrong@users.noreply.github.com> # Conflicts: # nemo_gym/base_responses_api_agent.py # tests/unit_tests/test_agent_sessions.py
|
/ok to test 417f7f1 |
…-stop Signed-off-by: gwarmstrong <gwarmstrong@users.noreply.github.com>
|
/ok to test caacdbd |
|
@yuhezhang-ai one more, sorry. #5214 broke |
yuhezhang-ai
left a comment
There was a problem hiding this comment.
Rechecked caacdbdc2 and the intervening main merges. The conflict resolutions preserve the reviewed implementation; session routing moved intact into server_runtime, with only its import path changed. The final merge bringing in #5388 reproduces cleanly. No new code-review findings.
Independent validation on this exact head: 75 focused tests passed, covering stop delivery, saturation, supervisor behavior, cross-worker routing, and grading; lint and formatting passed for all 17 changed Python files. No fresh model runs were needed for these import/move resolutions. GitHub's routing end-to-end and Codex conformance checks have passed; the full CI run is still in progress. Reapproving the code review; merge remains gated on the remaining required checks.
|
🔄 Merge queue validation started! You can track the progress here: https://github.com/NVIDIA-NeMo/Gym/actions/runs/41142997696 |
What does this PR do?
Lets a caller end a running rollout early and still get it graded. Today an agent's only time budget is its own
agent_timeout_sec, fixed when the rollout starts, and cancelling a rollout in flight (#4029) discards it. Some budgets can only be judged from outside the agent while it runs: time that leaves out waiting on the model endpoint, token or cost caps, a training-step deadline for stragglers, an operator stopping a runaway rollout. Design and discussion: #5091.POST /rollouts/{rollout_id}/stopon the agent server, with an optional{"reason": ...}.rollout_idis the id in/ng-rollout/<id>. It answers{"rollout_id", "stopped"};stoppedis false when the run already ended or a stop was already requested, so repeating it is safe. An unknown or closed rollout answers 404, and an agent that has not opted in answers 501. With several server workers the request needs no session cookie: the worker running the rollout claims its/rollouts/<rollout_id>path, and the others forward the stop to it. A stop applies to the rollout's current agent session: an environment that opens a new session for each turn, such as Harbor's multi-step tasks, still schedules the later turns, so a caller enforcing a whole-episode budget must also keep the environment from starting further turns.stopped: truemeans the stop was accepted; the run can still finish on its own before it takes effect.--expire-file, whichprocess_supervisor.shpolls alongside its stop file, written as a file within the stop's timeout, so it does not wait for exec capacity the running harness holds). The harness then ends through the existing timeout path: TERM, grace, cleanup. If the marker write fails after reaching the sandbox, the route answers 502, and the stop is still kept once the receipt saystimed_outand the marker is confirmed in place. Finalization waits for a marker write still in flight before that check and bounds the whole check by its timeout; without a confirmation the stop stays unattributed. A stop that arrives while the session is still seeding takes effect when the harness would launch, and the harness never starts. Runners see a timeout, not a cancellation: nostop.requestmarker is written. Stopping a running harness needs a sandbox provider whose uploads never wait for capacity that running commands hold, declared by the new optional capabilitySupportsSandboxIndependentUploads: Local, and OpenSandbox withoperations.background_exec: true. On other providers (Docker, Apptainer and Enroot share CLI slots, OpenShell worker threads, OpenSandbox in foreground mode its HTTP connection pool; Daytona, E2B and ECS Fargate are not shown to be independent),SandboxSession.expire()raisesExpiryUnsupportedErrorfor a running harness, and the route answers 501 with the reason.status: incomplete, whichsingle_agent_turnverifies. The response metadata gainsng_rollout_stop, a JSON object with the caller's reason, which tells a stop apart from a timeout or a cancellation. A stop is not a failure, so the response carries nong_agent_outcomefailure kind and feat(agent): report native causes for shared grading policy #4834's files are unchanged; a stopped and graded rollout is an ordinary result./stopreturns right away. The caller keeps awaiting its original/runrequest, which returns the graded result once collection and verification finish. Callingaclose()on arun_exampleshandle first would cancel those requests.nemo_gym(route, multi-worker routing, seed-time stop,SandboxSession.expire(),report_rollout_stop()); an agent opts in by returning itsSandboxSessionfrom_rollout_sandbox_session()and passing its response throughreport_rollout_stop(). Codex is wired here. Each otherSandboxSessionagent (Claude Code, Hermes, Kilo, OpenClaw, OpenCode, Pi, Stirrup) needs about five lines, as follow-ups.Size: +257/−5 non-test lines in
nemo_gym, +23/−1 incodex_agent, plus a paragraph inadd-or-update-a-harness.mdx.Closes #5091.
Validation
codex_agent468 passed (7 skipped),single_agent_turn123, the supervisor, sandbox-session and agent-session tests 144, and session routing 89. New cases cover an unknown rollout, a finished run, a repeated stop, a stop during seeding, a stop after close, an agent that has not opted in, a failed stop write that can be retried, Codex stopped mid-run (incomplete, partial output kept, marker set) and before launch (never started), andsingle_agent_turnverifying a stopped run whose result keeps the marker. Added for the review: a stop through another server worker, during that worker's seed, and after close; a stop for an unknown rollout leaving no routing claim; expiry while the harness holds every Local exec slot (one slot, and all eight default slots); an expiry write that never returns failing within the timeout and succeeding on retry; Codex stopped after its worker launched but before it wrote events; a transcript that exists but fails to download after a stop, and a failed transcript check, keeping the error path; a marker write that lands but loses its acknowledgment keeping the stop's reason, and one that never lands leaving a natural timeout unattributed; the bash supervisor's expire file before launch and while running, and a stop file winning over it; a marker write that fails during finalization; the marker check while another workload holds every Local exec slot; a stop request that disconnects before its write starts, which still delivers the marker or stays retryable; a write still pending when finalization stops waiting, which leaves the stop unattributed; a running harness on a provider without independent uploads, refused with 501 and no marker written, while a stop before launch works on any provider; and which providers declare the capability.background_exec: true, a stop of a running harness returned in 0.07 s and the harness ended 0.8 s later through the timeout path (receipttimed_out, exit 143, cleanup confirmed). In foreground mode the stop was refused at once and the harness kept running. Earlier, on Gym's Docker provider (before running-harness stops were limited to providers with independent uploads): a stop mid-run ended the harness in 0.4 to 0.5 s through the timeout path, and a stop before launch kept it from starting; no containers were left.agent_limitinstead of the marker) ran under arollout_collection_driverthat enforced a 1-hour budget of counted agent time, with a 3-houragent_timeout_secas the safety cap. Three Codex rollouts on Terminal-Bench-Science were each stopped once, 0.6 to 0.85 s after the budget, and then verified normally.Rollouts
N/A: agent-server control route.
Compatibility and benchmark impact
New optional route; nothing changes for callers that don't use it. Timeouts and cancellation behave as before. Agents that don't opt in answer 501. Stopping a running harness answers 501 on providers without
SupportsSandboxIndependentUploads; a stop before launch works on every provider. Reserving stop capacity in Docker, Apptainer, Enroot, OpenShell and OpenSandbox's foreground mode, so they can stop a running harness too, is a follow-up.