Skip to content

feat(anyswe_agent): add opt-in sanitize_git_history - #4017

Open
linj-glitch wants to merge 5 commits into
NVIDIA-NeMo:mainfrom
linj-glitch:linj/anyswe-sanitize-git-history
Open

linj-glitch wants to merge 5 commits into
NVIDIA-NeMo:mainfrom
linj-glitch:linj/anyswe-sanitize-git-history

Conversation

@linj-glitch

@linj-glitch linj-glitch commented Oct 4, 2026 •

Copy link
Copy Markdown
Collaborator

What and why

Some SWE images contain future branches, tags, remote refs, or reflogs that expose the reference fix. In observed GRPO training, policies learned to mine that history: training reward rose while performance on SWE-bench Verified regressed.

This adds opt-in sanitize_git_history to anyswe_agent, disabled by default. Before the agent starts, it keeps the checked-out history, deletes other refs/remotes/reflogs, and prunes unreachable objects. A detached HEAD receives a branch. If garbage collection fails or exceeds sanitize_git_history_timeout_s (default 300 seconds, enforced when timeout is available), it falls back to one commit of the current working tree. Grading still uses a fresh sandbox. Output is saved as git_sanitize.txt.

The sanitizer compares retained refs literally, rejects linked/shared Git metadata and alternate object stores before mutation, propagates Git and fallback failures, and checks that unreachable objects are actually gone. Unsupported shared layouts or incomplete cleanup fail setup instead of reporting success or allowing the agent to read retained future commits. The fallback also preserves previously tracked files that match ignore rules, using NUL-delimited literal paths, so subsequent fixes remain visible to patch capture. A missing repository remains a no-op. No separate issue is needed for these reproduced in-PR defects.

Validation

Validated head: 79a310f0ab0c569913796d6cd04f93a69ad7066a, including an additive merge of main f36e126547866601bee9c0671c9b59484ef83723. Python 3.13.14; before running pytest, the runner asserted that nemo_gym, responses_api_agents.anyswe_agent.app, and agent_runner were imported from the reviewed checkout.

  • Component suite with pytest -q -o addopts= --disable-warnings responses_api_agents/anyswe_agent/tests: 80 passed.
  • pre-commit run --all-files: passed; final PR diff whitespace checks passed and the worktree was clean.
  • All new commits carry DCO sign-off; updates are additive, without rewriting history.

Nine real-Git safety regressions failed before the correction and now pass: linked and shared repositories, alternate object stores, separate/symlinked Git directories, kept packs retaining future objects, branch names containing regex metacharacters, ownership errors, and failed fallback initialization. An additional independently identified regression failed before the follow-up fix and now passes: forced garbage-collection failure followed by the real runner snapshot and patch extraction, including ignored tracked filenames containing a newline and brackets. The existing real-repository tests still verify that future commits become unreadable and working files are preserved; sandbox wiring tests cover the opt-in flag.

Rollouts and compatibility

Historical author-reported use of the flag spans several thousand Claude Code GRPO rollouts on R2E-Gym. No fresh model or sandbox rollout was run for this revision; the new failure paths were validated using real temporary Git repositories.

The default remains unchanged. When enabled, setup may reject unsupported Git layouts or cleanup failures; the fallback intentionally discards historical commits while preserving working-tree contents. Current-head CI and maintainer review remain merge gates.

@copy-pr-bot

copy-pr-bot Bot commented Oct 4, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@yaoyu-33 yaoyu-33 added area:agent Agent harnesses and Responses API agent behavior complexity:low Localized change in one scope with a small, straightforward review surface feature New capabilities, enhancements, or enablement work needs-review PR is ready for code review and waiting on a reviewer labels Oct 5, 2026
@linj-glitch
linj-glitch marked this pull request as draft October 5, 2026 06:11
@linj-glitch
linj-glitch force-pushed the linj/anyswe-sanitize-git-history branch from 5c41010 to b05637e Compare October 8, 2026 00:15
linj-glitch added a commit to linj-glitch/Gym that referenced this pull request Oct 8, 2026
Some task images ship the upstream clone with history past the base commit:
the fix sits on origin/* refs, on a release branch it was backported to, on
a later tag, or in the reflog of a clone that was reset to the base commit.
`git log --all --grep=<issue words>` plus `git show <sha>` then recovers the
reference solution, and an RL policy learns to do that instead of solving
the task. The OpenHands harness's own reset only drops refs that descend from
the base commit, so a backport onto an older branch stays reachable.

With `sanitize_git_history: true`, both harness paths (OpenHands, opencode)
run a short script in the image's repository before the agent starts:
remove remotes, delete every ref that is not an ancestor of HEAD (or of the
instance's base_commit, which the OpenHands harness resets to), delete refs
to non-commit objects, expire reflogs and `git gc --prune=now`. Ancestor tags
stay, so `git describe` / setuptools-scm versions inside the agent container
are unchanged; HEAD and the working tree are untouched, so patch capture and
grading are unaffected. Output goes to git_sanitize.log in the rollout's
trajectory directory; a failure never blocks the rollout. Off by default.

Same flag name as the anyswe_agent option (NVIDIA-NeMo#4017).

Signed-off-by: Lin Jia <linj@nvidia.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Some task images (R2E-Gym, SWE-Gym and SWE-rebench among them) ship the
full upstream clone checked out at the base commit, so the fix commit is
still reachable through other refs: `git log --all --grep=<issue words>`
plus `git show` recovers it. An RL policy learns to mine the history
instead of solving the task, and the skill does not transfer to images
without future refs.

With `sanitize_git_history: true`, the agent's sandbox keeps only the
history reachable from HEAD before the agent starts. Every other ref,
reflog and remote is deleted, and unreachable objects are pruned with
`git gc --prune=now`, bounded by `sanitize_git_history_timeout_s`. If gc
exceeds the budget, the history is replaced by a single commit of the
current tree. The working tree is untouched. Grading runs in a fresh
sandbox and is unaffected. The option defaults to off.

Signed-off-by: Lin Jia <linj@nvidia.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@linj-glitch
linj-glitch force-pushed the linj/anyswe-sanitize-git-history branch from b05637e to 095312f Compare October 8, 2026 00:20
linj-glitch added a commit to linj-glitch/Gym that referenced this pull request Oct 8, 2026
Some task images ship the upstream clone with history past the base commit:
the fix sits on origin/* refs, on a release branch it was backported to, on
a later tag, or in the reflog of a clone that was reset to the base commit.
`git log --all --grep=<issue words>` plus `git show <sha>` then recovers the
reference solution, and an RL policy learns to do that instead of solving
the task. The OpenHands harness's own reset only drops refs that descend from
the base commit, so a backport onto an older branch stays reachable.

With `sanitize_git_history: true`, both harness paths (OpenHands, opencode)
run a short script in the image's repository before the agent starts:
remove remotes, delete every ref that is not an ancestor of HEAD (or of the
instance's base_commit, which the OpenHands harness resets to), delete refs
to non-commit objects, expire reflogs and `git gc --prune=now`. Ancestor tags
stay, so `git describe` / setuptools-scm versions inside the agent container
are unchanged; HEAD and the working tree are untouched, so patch capture and
grading are unaffected. Output goes to git_sanitize.log in the rollout's
trajectory directory; a failure never blocks the rollout. Off by default.

Same flag name as the anyswe_agent option (NVIDIA-NeMo#4017).

Signed-off-by: Lin Jia <linj@nvidia.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Lin Jia <linj@nvidia.com>
@linj-glitch
linj-glitch marked this pull request as ready for review October 13, 2026 23:08
@github-actions github-actions Bot added the sla:triage-overdue Review assignment is over the one-business-day SLA label Oct 14, 2026
Signed-off-by: Lin Jia <linj@nvidia.com>
Compare retained refs literally and reject linked/shared Git metadata or
alternate object stores before mutation. Check repository access and HEAD,
propagate setup/fallback failures, and verify unreachable objects are gone.
Document strict setup behavior and cover real Git layouts and failure paths.

Signed-off-by: Lin Jia <linj@nvidia.com>
(cherry picked from commit 5e41f05218731ef9e669fd9b2134501f3918620e)
Signed-off-by: Lin Jia <linj@nvidia.com>
@linj-glitch

Copy link
Copy Markdown
Collaborator Author

/ok to test 5d0ccb3

…back

Signed-off-by: Lin Jia <linj@nvidia.com>
@linj-glitch

Copy link
Copy Markdown
Collaborator Author

/ok to test 79a310f

This branch was previously deployed

1 inactive deployment
public — 79a310f0 Deployed Oct 24, 2026 by copy-pr-bot[bot] via release / finalize / notify #7061
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:agent Agent harnesses and Responses API agent behavior complexity:low Localized change in one scope with a small, straightforward review surface feature New capabilities, enhancements, or enablement work needs-review PR is ready for code review and waiting on a reviewer sla:triage-overdue Review assignment is over the one-business-day SLA

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants