Repository navigation
mini_swe_agent_2: end the rollout when its sandbox is gone or an exec does not run - #4472
Merged
Merged
Conversation
A rollout whose sandbox crashed or was deleted kept stepping against it until the step limit, each step waiting out the request timeout. When the provider reports a sandbox error and the sandbox is ERROR or STOPPED, raise instead. Signed-off-by: Davanum Srinivas <dsrinivas@nvidia.com>
dims
marked this pull request as ready for review
October 12, 2026 15:14
A sandbox exec that times out on the provider side comes back with the runtime return code 125, error_type set and no output. The agent treated it like any other observation. A timed-out write step left an empty patch, the rollout scored 0 and eval_error_rate stayed at 0, so the failure was invisible. Raise whenever the exec result carries an error_type. The sandbox contract sets it only for a runtime sentinel (timeout, transport error, gone sandbox) in place of a process exit code. This replaces the status gate that raised only for a sandbox in ERROR or STOPPED; the status now goes into the error message. A non-zero exit code from the command itself still returns as an observation. Signed-off-by: Davanum Srinivas <dsrinivas@nvidia.com>
Contributor
|
/ok to test e7efc25 |
zyzhou5
enabled auto-merge
October 13, 2026 20:21
Contributor
|
🔄 Merge queue validation started! You can track the progress here: https://github.com/NVIDIA-NeMo/Gym/actions/runs/38464522274 |
khazic
pushed a commit
to khazic/Gym-lao
that referenced
this pull request
Oct 15, 2026
## Summary - Add the SA-V tracking verifier and its server configuration. - Declare the SA-V environment servers used by video RL. - Add portable five-row example data, schema, metrics, and checked rollout artifacts. ## Why This is the Gym-side verifier foundation for mixed SA-V tracking and CapRL video training in NeMo-RL. No separate issue is needed; this is part of the Super 3.5 video RL upstreaming stack. ## Validation - Runtime evidence: the [healthy 16-node async `r18` W&B run](https://wandb.ai/adlr/Nemotron-omni-RL-debug/runs/oci-hsg-cs-001_n3post_main_1d76d243b_vllm029_super35_video_step120_async16n_trainmb45056_expandable_uncapped_r18_20261003) shows repeated GRPO cycles, stable TMPE, and increasing reward on the equivalent video implementation. - `gym dataset collate ... +mode=example_validation`: 5 examples; generated `data/example_metrics.json`. - Focused validation: 25 verifier tests and all 5 example rows against `task_data.py` passed. - Ruff format and lint passed for the changed Python files. - Both Gym stack ranges pass `git diff --check`. ## Rollouts `data/example_rollouts.jsonl` contains one checked rollout for each of the five example rows. Representative task checks: - `pt`: `<answer>{"tracks":[{"point":1,"frame":1,"xy":[500,500],"visible":true}]}</answer>` — `format_ok=true`, reward `1.0`, visible-frame mask membership `true`. - `box`: `<answer>{"tracks":[{"box":1,"frame":1,"bbox":[250,250,750,750],"visible":true}]}</answer>` — `format_ok=true`, reward `1.0`, frame IoU `1.0`. - `ref`: `<answer>{"tracks":[{"id":7,"frame":1,"xy":[500,500]}]}</answer>` — `format_ok=true`, reward `1.0`, predicted id `7` assigned to the GT object and mask membership `true`. The artifact also contains visible/invisible `pt` coverage and an all-absent `ref` negative case, preserving the verifier's current unmatched-ID behavior. ## Compatibility Adds a new verifier/configuration and does not change unrelated environments. Example media is embedded as a small inline JPEG, so no employee or cluster-local paths are required. ## Stack Gym PR 1 of 2. Consolidates and supersedes the implementation scope of NVIDIA-NeMo#3504 after review history is preserved. ## Consolidated stack links ### Gym 1. [Gym NVIDIA-NeMo#3999 — SA-V tracking verifier](NVIDIA-NeMo#3999) 2. [Gym NVIDIA-NeMo#4000 — R3 routing replay and rollout identity](NVIDIA-NeMo#4000) ### NeMo-RL 1. [NeMo-RL NVIDIA-NeMo#4472 — Super video data and model loading](NVIDIA-NeMo/RL#4472) 2. [NeMo-RL NVIDIA-NeMo#4473 — RL/MOPD video frame contract](NVIDIA-NeMo/RL#4473) 3. [NeMo-RL NVIDIA-NeMo#4474 — sync/async Gym routing runtime](NVIDIA-NeMo/RL#4474) 4. [NeMo-RL NVIDIA-NeMo#4475 — SA-V pin and CI fixtures](NVIDIA-NeMo/RL#4475) 5. [NeMo-RL NVIDIA-NeMo#4476 — final dependency pins](NVIDIA-NeMo/RL#4476) ### Child dependency refs - [MCore integration ref](https://github.com/ehosseiniasl/Megatron-LM/tree/ehsan/video-r3-mtp-gradient-integration-20261004), adapting [MCore #7278](NVIDIA/Megatron-LM#7278) and [MCore #7279](NVIDIA/Megatron-LM#7279) to the pinned base. - [MBridge integration ref](https://github.com/NVIDIA-NeMo/Megatron-Bridge/tree/ehsan/video-r3-mcore-pin-integration-20261004). --------- Signed-off-by: ehosseiniasl <ehsan.hosseiniasl@gmail.com> Co-authored-by: Pulkit Kumar <pulkitk@nvidia.com>
This branch was previously deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Two ways a rollout kept going after its sandbox could no longer run commands:
error_typeset and no output, and the agent treated it like any other observation. A timed-out write step left an empty patch, the rollout scored 0 andeval_error_ratestayed at 0.executenow raises whenever the exec result carries anerror_type. The sandbox contract sets it only for a runtime sentinel (timeout, transport error, gone sandbox) in place of a process exit code, so a non-zero exit code from the command itself still returns as an observation. The sandbox status goes into the error message.Validation
gym env test +entrypoint=responses_api_agents/mini_swe_agent_2(29 passed). The new tests fail without the source change: a gone sandbox raises, a timed-out exec raises, command exit codes 1 and 125 without anerror_typereturn as observations.ruff check,ruff format, pre-commit on the changed files.Rollouts
Rollouts whose sandbox is gone or whose exec did not run end with an error instead of stepping to the limit or scoring a silent 0.
Compatibility and benchmark impact
eval_error_raterises where a reward of 0 hid the failure before. No change for rollouts whose commands run.Checklist
pre-commit run --all-files) (so CI lint/format checks pass). On the changed files.git commit -s) (so the DCO check passes).