Skip to content

mini_swe_agent_2: end the rollout when its sandbox is gone or an exec does not run - #4472

Merged
zyzhou5 merged 2 commits into
NVIDIA-NeMo:mainfrom
dims:abort-dead-sandbox
Oct 13, 2026
Merged

zyzhou5 merged 2 commits into
NVIDIA-NeMo:mainfrom
dims:abort-dead-sandbox

Conversation

@dims

@dims dims commented Oct 12, 2026 •

Copy link
Copy Markdown
Contributor

What does this PR do?

Two ways a rollout kept going after its sandbox could no longer run commands:

  • The sandbox had crashed or been deleted: the agent stepped against it until the step limit, each step waiting out the request timeout.
  • An exec timed out or failed on the provider side: it came back with the runtime return code 125, error_type set and no output, and the agent treated it like any other observation. A timed-out write step left an empty patch, the rollout scored 0 and eval_error_rate stayed at 0.

execute now raises whenever the exec result carries an error_type. The sandbox contract sets it only for a runtime sentinel (timeout, transport error, gone sandbox) in place of a process exit code, so a non-zero exit code from the command itself still returns as an observation. The sandbox status goes into the error message.

Validation

gym env test +entrypoint=responses_api_agents/mini_swe_agent_2 (29 passed). The new tests fail without the source change: a gone sandbox raises, a timed-out exec raises, command exit codes 1 and 125 without an error_type return as observations. ruff check, ruff format, pre-commit on the changed files.

Rollouts

Rollouts whose sandbox is gone or whose exec did not run end with an error instead of stepping to the limit or scoring a silent 0.

Compatibility and benchmark impact

eval_error_rate rises where a reward of 0 hid the failure before. No change for rollouts whose commands run.

Checklist

  • I have read the contributing guidelines.
  • The change is focused; unrelated "drive-by" edits are tracked as separate issues/PRs.
  • Tests added or updated and pass locally, or N/A for docs-only / non-code changes (so CI unit/server checks pass when applicable).
  • Documentation added or updated, or N/A with justification. N/A, no user-facing change.
  • Pre-commit checks pass locally (pre-commit run --all-files) (so CI lint/format checks pass). On the changed files.
  • New source files include the required Apache-2.0 SPDX header. N/A, no new files.
  • All commits have DCO sign-off (git commit -s) (so the DCO check passes).

A rollout whose sandbox crashed or was deleted kept stepping against it
until the step limit, each step waiting out the request timeout. When the
provider reports a sandbox error and the sandbox is ERROR or STOPPED,
raise instead.

Signed-off-by: Davanum Srinivas <dsrinivas@nvidia.com>
@copy-pr-bot

copy-pr-bot Bot commented Oct 12, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@nemo-automation-bot nemo-automation-bot Bot added the community-request Issue reported or requested by someone from the community label Oct 12, 2026
@dims
dims marked this pull request as ready for review October 12, 2026 15:14
@yaoyu-33 yaoyu-33 added area:agent Agent harnesses and Responses API agent behavior bug Something isn't working complexity:low Localized change in one scope with a small, straightforward review surface needs-review PR is ready for code review and waiting on a reviewer and removed community-request Issue reported or requested by someone from the community labels Oct 12, 2026
A sandbox exec that times out on the provider side comes back with the
runtime return code 125, error_type set and no output. The agent treated
it like any other observation. A timed-out write step left an empty
patch, the rollout scored 0 and eval_error_rate stayed at 0, so the
failure was invisible.

Raise whenever the exec result carries an error_type. The sandbox
contract sets it only for a runtime sentinel (timeout, transport error,
gone sandbox) in place of a process exit code. This replaces the status
gate that raised only for a sandbox in ERROR or STOPPED; the status now
goes into the error message. A non-zero exit code from the command
itself still returns as an observation.

Signed-off-by: Davanum Srinivas <dsrinivas@nvidia.com>
@dims dims changed the title mini_swe_agent_2: end the rollout when its sandbox is gone mini_swe_agent_2: end the rollout when its sandbox is gone or an exec does not run Oct 13, 2026
@github-actions github-actions Bot added the sla:triage-overdue Review assignment is over the one-business-day SLA label Oct 13, 2026
@zyzhou5
zyzhou5 requested a review from hemildesai October 13, 2026 20:10
@zyzhou5

zyzhou5 commented Oct 13, 2026

Copy link
Copy Markdown
Contributor

/ok to test e7efc25

@nemo-automation-bot nemo-automation-bot Bot added the community-request Issue reported or requested by someone from the community label Oct 13, 2026
@nemo-automation-bot
nemo-automation-bot Bot requested a review from a team October 13, 2026 20:12

@zyzhou5 zyzhou5 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@zyzhou5 zyzhou5 added ready-to-merge PR is approved, current, and waiting for final CI or merge and removed needs-review PR is ready for code review and waiting on a reviewer labels Oct 13, 2026
@zyzhou5
zyzhou5 enabled auto-merge October 13, 2026 20:21
@svcnvidia-nemo-ci svcnvidia-nemo-ci added the waiting-on-customer Waiting on the original author to respond label Oct 13, 2026
@zyzhou5
zyzhou5 added this pull request to the merge queue Oct 13, 2026
@github-actions

Copy link
Copy Markdown
Contributor

🔄 Merge queue validation started! You can track the progress here: https://github.com/NVIDIA-NeMo/Gym/actions/runs/38464522274

Merged via the queue into NVIDIA-NeMo:main with commit 3833e16 Oct 13, 2026
53 checks passed
@svcnvidia-nemo-ci svcnvidia-nemo-ci removed the waiting-on-customer Waiting on the original author to respond label Oct 14, 2026
khazic pushed a commit to khazic/Gym-lao that referenced this pull request Oct 15, 2026
## Summary
- Add the SA-V tracking verifier and its server configuration.
- Declare the SA-V environment servers used by video RL.
- Add portable five-row example data, schema, metrics, and checked
rollout artifacts.

## Why
This is the Gym-side verifier foundation for mixed SA-V tracking and
CapRL video training in NeMo-RL. No separate issue is needed; this is
part of the Super 3.5 video RL upstreaming stack.

## Validation
- Runtime evidence: the [healthy 16-node async `r18` W&B
run](https://wandb.ai/adlr/Nemotron-omni-RL-debug/runs/oci-hsg-cs-001_n3post_main_1d76d243b_vllm029_super35_video_step120_async16n_trainmb45056_expandable_uncapped_r18_20261003)
shows repeated GRPO cycles, stable TMPE, and increasing reward on the
equivalent video implementation.
- `gym dataset collate ... +mode=example_validation`: 5 examples;
generated `data/example_metrics.json`.
- Focused validation: 25 verifier tests and all 5 example rows against
`task_data.py` passed.
- Ruff format and lint passed for the changed Python files.
- Both Gym stack ranges pass `git diff --check`.

## Rollouts
`data/example_rollouts.jsonl` contains one checked rollout for each of
the five example rows. Representative task checks:

- `pt`:
`<answer>{"tracks":[{"point":1,"frame":1,"xy":[500,500],"visible":true}]}</answer>`
— `format_ok=true`, reward `1.0`, visible-frame mask membership `true`.
- `box`:
`<answer>{"tracks":[{"box":1,"frame":1,"bbox":[250,250,750,750],"visible":true}]}</answer>`
— `format_ok=true`, reward `1.0`, frame IoU `1.0`.
- `ref`:
`<answer>{"tracks":[{"id":7,"frame":1,"xy":[500,500]}]}</answer>` —
`format_ok=true`, reward `1.0`, predicted id `7` assigned to the GT
object and mask membership `true`.

The artifact also contains visible/invisible `pt` coverage and an
all-absent `ref` negative case, preserving the verifier's current
unmatched-ID behavior.

## Compatibility
Adds a new verifier/configuration and does not change unrelated
environments. Example media is embedded as a small inline JPEG, so no
employee or cluster-local paths are required.

## Stack
Gym PR 1 of 2. Consolidates and supersedes the implementation scope of
NVIDIA-NeMo#3504 after review history is preserved.

## Consolidated stack links

### Gym
1. [Gym NVIDIA-NeMo#3999 — SA-V tracking
verifier](NVIDIA-NeMo#3999)
2. [Gym NVIDIA-NeMo#4000 — R3 routing replay and rollout
identity](NVIDIA-NeMo#4000)

### NeMo-RL
1. [NeMo-RL NVIDIA-NeMo#4472 — Super video data and model
loading](NVIDIA-NeMo/RL#4472)
2. [NeMo-RL NVIDIA-NeMo#4473 — RL/MOPD video frame
contract](NVIDIA-NeMo/RL#4473)
3. [NeMo-RL NVIDIA-NeMo#4474 — sync/async Gym routing
runtime](NVIDIA-NeMo/RL#4474)
4. [NeMo-RL NVIDIA-NeMo#4475 — SA-V pin and CI
fixtures](NVIDIA-NeMo/RL#4475)
5. [NeMo-RL NVIDIA-NeMo#4476 — final dependency
pins](NVIDIA-NeMo/RL#4476)

### Child dependency refs
- [MCore integration
ref](https://github.com/ehosseiniasl/Megatron-LM/tree/ehsan/video-r3-mtp-gradient-integration-20261004),
adapting [MCore #7278](NVIDIA/Megatron-LM#7278)
and [MCore #7279](NVIDIA/Megatron-LM#7279) to
the pinned base.
- [MBridge integration
ref](https://github.com/NVIDIA-NeMo/Megatron-Bridge/tree/ehsan/video-r3-mcore-pin-integration-20261004).

---------

Signed-off-by: ehosseiniasl <ehsan.hosseiniasl@gmail.com>
Co-authored-by: Pulkit Kumar <pulkitk@nvidia.com>

This branch was previously deployed

1 inactive deployment
public — e7efc251 Deployed Oct 13, 2026 by copy-pr-bot[bot] via release / finalize / notify #5210
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:agent Agent harnesses and Responses API agent behavior bug Something isn't working community-request Issue reported or requested by someone from the community complexity:low Localized change in one scope with a small, straightforward review surface ready-to-merge PR is approved, current, and waiting for final CI or merge sla:triage-overdue Review assignment is over the one-business-day SLA

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants