Repository navigation
Description
Problem
An agent's only time budget today is its own wall-clock timeout (agent_timeout_sec, which a sample can set), fixed when the rollout starts. When the harness hits it, the supervisor ends the harness and the episode still goes on to verification. A caller can also cancel rollouts that are in flight (#4029), but a cancelled rollout is discarded, not graded.
Some budgets can only be judged from outside the agent while the rollout runs. For those there is no way to end the rollout and keep a graded result.
Use cases
- Budgets that leave out infrastructure time. A long-horizon task with an 8-hour budget that loses an hour waiting on its model endpoint (queueing for capacity, rate-limit backoff, a cold start, a restart) got 7 hours of work, and its score reflects the infrastructure. feat(browsecomp): per-rollout wall-clock limit and run timing summary #4174 makes this visible for BrowseComp: its timing summary flags rollouts that queued on the model servers. A stop hook lets the caller hold the budget fixed in the time that counts instead.
- Token or cost caps across a whole rollout, enforced by the caller that sees the usage.
- Stragglers in RL rollout collection. At a step deadline, end the slowest rollouts and keep them as graded partial episodes instead of discarding them, for the deadline consumers feat(rollout): let run_examples callers cancel rollouts still in flight #4029 was built for.
- An operator stopping a runaway rollout without losing it.
Proposal
One endpoint on agent servers whose harness runs under SandboxSession: POST /rollouts/{rollout_id}/stop, with an optional body {"reason": "..."}. rollout_id is the id in /ng-rollout/<id>.
- It ends the harness through the same supervisor path as the agent's own timeout. It writes the session's existing stop-request marker first, so harnesses that read it (Hermes, Stirrup) can tell a requested stop from a wall-time one.
- The agent then returns as it does at a timeout, and the episode is verified and graded. The response carries a typed cause, so environments and
agent_outcome_policycan tell a requested stop from a timeout. - It returns 200 with whether this call stopped the harness. It is idempotent, does nothing once the rollout has finished, and returns 404 for an unknown rollout.
- Agents opt in through a small shared helper: Codex first, the others as their owners want.
Nothing changes for callers that don't use it. The budget logic stays with the caller (for example a rollout_collection_driver), so Gym does not have to define budgets.
Why this shape
- It builds on what
SandboxSessionalready has: the stop-request marker andstop_harness(). - It does not depend on how the model is served: vLLM directly, a hosted API, or a router in between.
- It can stop a harness at any point, including during a long tool call.
Alternatives considered
- An API that moves the agent's deadline. Gym keeps the clock and the caller adds time. That puts more state in the agent, and the caller still has to know the budget.
Prototyping this against current main and can send a PR once the shape is agreed.
Activity
yuhezhang-ai commentedon Oct 27, 2026
Thanks George—this looks useful, and keeping budget accounting with the caller makes sense.
A few related changes to keep in mind:
I don’t think you need to wait for those open PRs to prototype this on main. If the implementation needs changes to shared outcome or failure fields, let’s coordinate those before merging. A useful integration check with #3302 would cover a stopped-and-graded result surviving save/resume without rerunning, plus a grading failure after stopping following the existing recovery rules.
One question about the infrastructure-time example: how would this interact with the existing
agent_timeout_sec? An eight-hour hard timeout would still fire before the caller’s eight hours of counted work if there was infrastructure waiting. Would callers configure a larger safety timeout while enforcing their actual budget through/stop?One other API detail worth making explicit: after
/stopreturns, does the caller keep awaiting the original rollout request to receive the graded result? For the training-deadline use case, stopping the agent still leaves cleanup and verification to finish. It would help to show the intended caller sequence, including how it avoids cancelling the result through the existing close/drain API.gwarmstrong commentedon Oct 27, 2026
Thanks, this is very helpful. Proposed adjustments first, then the two questions.
Outcome. A requested stop won't be a failure kind. The stopped run returns
status: incomplete, which #4834 already verifies, with a Gym-owned markerng_rollout_stop: {"reason": ...}in the response metadata. That keeps it distinct fromagent_timeoutandcancelledwithout touching the failure-kind registry. (incomplete_details.reasonwould be the natural place, but the OpenAI SDK types limit it tomax_output_tokensandcontent_filter.) Does that marker name and placement work for you, or would you prefer it elsewhere in the shared outcome fields?#3302. Since the stop isn't a failure, a stopped and graded rollout should save as a normal result. I'll run the check you suggested against #3302: a stopped, graded result survives save/resume without rerunning or using the failure budget, and a grading failure after a stop follows the existing recovery rules. I'll post the results here.
#5072. Agreed: failures while stopping, collecting or verifying keep their causes, and the stop itself isn't one. Those will go through the shared reporting once #5072 lands.
agent_timeout_sec. Yes. The caller setsagent_timeout_secas a wall-clock safety cap above its budget plus whatever waiting it is willing to tolerate, and enforces the counted budget with/stop. In a test with a 1-hour counted budget and a 3-hour cap, all three rollouts were stopped within a second of the budget and then verified normally.Caller sequence.
/stopends the agent's run, not the rollout request. Grading happens inside the same/runrequest that started the rollout: the environment server calls the agent, then verifies. After/stop, the agent returns its response as it would at a timeout, the environment server collects and verifies as usual, and the original/runreturns the graded result. So the caller keeps awaiting that request instead of cancelling it:For a training deadline: post
/stopfor each straggler, keep consuming results until they arrive, and only then callaclose_and_drain(). Callingaclose()first would cancel the requests that are about to return those graded results.gwarmstrong commentedon Oct 27, 2026
Following up with the #3302 check. I applied the stop change to #3302's head (f60544d) and added an integration test that goes through the real pieces: the agent's
report_rollout_stopmarks the response,single_agent_turnverifies it, and #3302's collector saves and resumes the result.single_agent_turnreturns it as a result (status: incomplete, the verifier's reward,ng_rollout_stopin the response metadata), and the collector saves it as a successful, measured result with the marker intact.stage: verificationand their own reason; the stopped response stays attached as the partial-response diagnostic, marker included.incompleteor the marker specially. feat(eval): persist failures and resume unfinished evaluations #3302's recovery and records suites,single_agent_turn's and the session tests pass with the change (473 tests).The branch now matches what I described above: a stopped run carries no failure kind, #4834's files are untouched, and the caller's reason is recorded as
ng_rollout_stop: {"reason": ...}, a JSON string likeng_agent_outcome. I'll open the PR once the marker works for you. The integration test depends on #3302, so it can land with whichever of the two merges second.gwarmstrong commentedon Oct 27, 2026
@yuhezhang-ai the PR is up: #5145. It uses the
ng_rollout_stopmarker as described above; happy to move it if you'd rather it live elsewhere.ananthsub commentedon Oct 28, 2026
@gwarmstrong @yuhezhang-ai is there infrastructure from the partial rollout checkpointing that can be re-used for this? #3024 and stack here: #3882
gwarmstrong commentedon Oct 28, 2026
Thanks @ananthsub. Yes, and #5145 already builds on two parts of it:
/ng-rollout/<id>(r, orr-aNafter a restore), the identity feat(checkpoint): add the participant control plane, episode steps, and coordination #3882 introduced, so a stop names exactly one attempt and never reaches a restored one./rollouts/<capture_key>, and the other workers forward the stop to it.Two other pieces looked close but don't fit the "end it now and grade it" case:
retirealso stops an attempt's work, but it discards the attempt and refuses its later requests untilforget. A stop has to keep the run and send it to verification.One more piece we could reuse if you'd like: the control plane's auth. Today
POST /rollouts/{rollout_id}/stopsits next to the agent's other routes with no token. It could move under/ng-control/v1/and require a bearer token like the checkpoint routes, so only the controller can stop rollouts. I'd keep that token separate fromcheckpoint.enabled, so a caller can stop rollouts without turning on checkpointing; arollout_collection_driverreads it from the same global config the servers use. Happy to make that change in #5145 if that's the direction you prefer.