Skip to content

feat(deepagents): run agent sessions in process or inside a task sandbox - #5003

Merged
yaoyu-33 merged 55 commits into
mainfrom
yuya/gymint-141-deepagents-sessions
Nov 13, 2026
Merged

yaoyu-33 merged 55 commits into
mainfrom
yuya/gymint-141-deepagents-sessions

Conversation

@yaoyu-33

@yaoyu-33 yaoyu-33 commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor

DeepAgents now honors Gym's agent-session lifecycle through one app.py entrypoint. tool_profile: builtin runs its file, shell and subagent tools inside a task sandbox; tavily_search retains the in-process search graph. No separate issue is needed for this focused session integration and harness-layout correction.

  • Both profiles share admission, one activation per session, identical-request retries, rollout identity checks and restart-only checkpointing. Required external tool grants are refused; optional grants are skipped.
  • The builtin profile installs a host-built private Python runtime at seed and runs the graph under Gym's process supervisor. It routes inference through the rollout-prefixed Model Server, honors task deadlines/stops, and disconnects a borrowed sandbox or stops an owned one at close.
  • The Responses bridge preserves routing, sampling and model errors. Usage becomes and remains unknown when any model call lacks usage or is interrupted. Confirmed deadlines retain provider-wait diagnostics for close without replacing the graded partial response.
  • Configuration and setup failures preserve their distinction and original cause through cleanup. Close reports sandbox identity/outcome where available and explicit gaps for missing graph evidence.

Compatibility and limits

Existing config filenames, agent names and launch commands remain valid. reasoning_search_agent.py remains a compatibility subclass; shipped configs use app.py. Search retains legacy /run and custom build_agent(model) overrides; builtin execution requires an Environment Server.

Both profiles and legacy search /run now reject non-null request max_output_tokens with HTTP 422 before graph work or resource seeding. Remove the field and configure the Model Server's per-call cap. Other supported sampling controls remain available, and corrected requests can reuse a seeded session.

This remains Evaluation-level support: invocation/tool/turn ownership gaps are explicit, and checkpoints restart unfinished rollouts. In-process search does not enforce the task timeout or support rollout stop. Terminal-Bench 2.1 remains an unsupported pairing. Historical full-evaluation results in the existing results comment are not scores for this implementation.

Validation

  • Final source: 16355b546; all 4 changed-file hashes match the workstation sources, and the commit has DCO sign-off.
  • Linux component suite: python -m pytest responses_api_agents/langchain_deepagents_agent/tests -q -o addopts= — 103 passed.
  • pre-commit run --files <changed paths> — passed on the Linux workstation; no full-repository pre-commit claim.
  • Real-model rollout: builtin sandbox profile: one unchanged Reasoning Gym example, NIM nvidia/nemotron-3.5-super-vl-120b-a12b, agent-owned Docker debian:bookworm-slim, temperature 0, top_p 1, Model Server max_tokens 4096, thinking disabled, 180-second activation budget. Response completed, reward 0.0, unmasked, no failure outcome. Verifier output was inspected; all task process groups and containers were removed. This one-row smoke checks runtime and verification behavior, not benchmark qualification. Existing representative evidence covers search and builtin profiles; no live Tavily API qualification is claimed.
  • Latest-head GitHub checks: six reported checks passed (DCO, lint, both lock checks, copyright, secrets); request is skipped. Full behavioral CI is not established by that status. Checks.

Targeted regression controls: 3 tests fail against the pre-fix production sources and pass in the full fixed-source suite above.

Review follow-up (2026-11-09)

Fixed retirement cleanup and retry, partial-start ownership, pre-launch rollout stop, correctable tool admission, graph-agent composition, builtin model networking defaults, provider metadata and unknown usage after an interrupted successful response body. No separate issue is needed for these review corrections.

Validated on the Linux workstation at b0ae73be2: component tests plus tests/unit_tests/config/test_global_config.py — 350 passed; pre-commit run --all-files passed. One unchanged Reasoning Gym example with NIM Nemotron 3.5 Super, agent-owned Docker Debian bookworm-slim, temperature 0/top_p 1/per-call cap 4096: completed, reward 0.0, unmasked, no failure kind. Inspected agent and verifier output; task container removed. This is runtime evidence, not benchmark qualification.

Review follow-up (2026-11-10)

Merged main (6c21277b4; one docs-table conflict in integrate-existing-agents.mdx, where the DeepAgents row now says "Not tested" under Harbor task folders) and fixed the findings of a review against the agent review checklist. No separate issue is needed for these review corrections.

  • app.py declares calls_model_from_sandbox: its default builtin profile calls the Model Server from the sandbox, and Harbor reads that literal from source.
  • Search profile, agent sessions. A Model Server refusal that another episode attempt would not change (a non-retryable 4xx) now gets a typed reply instead of an HTTP 500 that reruns the episode. context_length_exceeded ends incomplete and is verified; any other refusal ends failed with model_request_error and its status. Retryable statuses, transport errors and legacy /run still raise.
  • Search profile. Request instructions, temperature and top_p now reach every model call; before, they were silently dropped. Score impact: search-profile runs whose rows or overrides set these fields can score differently. The bundled example rows set none of them.
  • Builtin profile, deadlines.
    • A wall_time stop counts only when the supervisor's receipt shows the deadline. A SIGTERM from inside the sandbox, such as a task's pkill python3, ends the run failed with agent_run_error.
    • A deadline before any model reply ends failed with model_transport_error, which is retried, instead of grading an empty answer. A rollout stop stays a graded stop.
  • Builtin profile, runner. The runner writes its report before tearing down its event loop and exits without joining deepagents' file-tool threads. A stop therefore no longer outlasts the supervisor's grace period and leaves no report, and a SIGTERM during the runner's imports still leaves a report. The close record says cancelled only for a run that never launched. Runner error text is capped.
  • Tests. The sandbox-episode tests have bounded waits, and the setup-failure test checks that the lent sandbox is disconnected.

Validated at b537de56a:

  • Component tests. python -m pytest responses_api_agents/langchain_deepagents_agent/tests -q -o addopts= on the Linux workstation: 133 passed.
    • The new tests fail against the previous sources: 3 for the in-process path, 8 for the lifecycle and runner changes.
    • The strengthened setup-failure test fails when the setup cleanup is removed.
  • Other affected tests. tests/unit_tests/config/test_global_config.py, tests/unit_tests/tooling/test_add_legacy_agent_environment_servers.py and tests/unit_tests/test_session_contract_repo.py pass on the Linux workstation. tests/unit_tests/test_public_api_facades.py passes on a git checkout (863 passed).
  • Pre-commit. pre-commit run --files <every changed path> passed.
  • Real-model smokes. NIM nvidia/nemotron-3.5-super-vl-120b-a12b, thinking off, Model Server per-call cap 4096:
    • Search profile. Two unchanged Reasoning Gym rows behind single_agent_turn_legacy, with request instructions, temperature: 0.6 and top_p: 0.95 added. Both completed with reward 1.0, and all 4 captured upstream calls carried those sampling values and the instructions.
    • Builtin profile, row 0. Agent-owned Docker debian:bookworm-slim, temperature 0, top_p 1. It completed with reward 0.0, unmasked: the answer is correct but has no <answer> tags.
    • Builtin profile, row 1. Same setup with agent_timeout_sec: 8. The deadline hit before the first model reply, and the run ended failed with model_transport_error from the runner's report, so it is retryable and not scored.
    • No containers or server processes were left. This is runtime evidence, not benchmark qualification.

CI repair (2026-11-11)

Final head 201a5454809803ed6561ddeb60b9ad1e537d08d6 scopes release-rehearsal changelogs on PR mirror pushes to changes against the default branch. The previous release validation failed before Bash could start because repository-wide notes exceeded the environment-variable size limit. Release branches and explicit manual changelog overrides retain their existing behavior. The shared template's general large-note fix is tracked in NVIDIA-NeMo/FW-CI-templates#619.

The release repair passed release validation at f38e4f56e, including both previously failing jobs. Then merge main 122724b0b965b8029b37db2d53779e2a19f6e82d to clear the lock-freshness advisory and bring the shared OpenAI pin and component lock to 2.45.0. This merge is clean and preserves main's compatibility-entrypoint initialization. No separate issue is needed for these focused CI and dependency-integration repairs.

Validation on the Linux workstation against the final tree:

  • python -m pytest responses_api_agents/langchain_deepagents_agent/tests tests/unit_tests/config/test_global_config.py tests/unit_tests/tooling/test_agent_environment_default_configs.py tests/unit_tests/test_session_contract_repo.py --import-mode=importlib -q -o addopts= -o log_cli=false — 395 passed.
  • python scripts/component_locks.py --jobs 4, using uv 0.13.4 — 283 projects, zero failures.
  • Full pre-commit run --all-files passed using Python 3.13 hook environments, with no file changes.
  • cd fern && npm run check passed.
  • The workflow expression passed seven cases using GitHub's @actions/expressions evaluator (PR mirror, main, release branch, deployment branch, and manual dispatch/override cases).
  • Fresh model rollout: N/A for this CI/dependency synchronization; earlier adapter smoke evidence remains pinned to its recorded revisions.

Both the source and PR mirror point to the final SHA. CI restarted for it: release validation and behavioral CI are pending.

CI repair — November 12, 2026

Source e180806f1f103ef53b535a1a1a039951d36bccc0 cherry-picks the three-line baseline syntax repair from #6083. The inherited HealthBench entry was missing its closing object and array, so the secrets detector stopped before scanning. No detector settings, baseline entries, or adapter runtime code were removed or changed.

Validated on Linux workstation ws against the exact committed tree: detect-secrets-hook --baseline .github/workflows/config/.secrets.baseline <all PR-changed files> passed with detect-secrets 1.5.0; pre-commit run --all-files passed using Python 3.13 hook environments; hooks left tracked files unchanged. Model rollout: N/A for this baseline-only repair. No separate issue is needed; #6083 is the upstream fix. Fresh CI requested for this exact source.

@copy-pr-bot

copy-pr-bot Bot commented Oct 23, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@yaoyu-33 yaoyu-33 added bug Something isn't working area:agent Agent harnesses and Responses API agent behavior labels Oct 23, 2026
@yaoyu-33

Copy link
Copy Markdown
Contributor Author

/ok to test 674c828

@yaoyu-33

Copy link
Copy Markdown
Contributor Author

CI note: the non-blocking Harness conformance failure is the KiloCode probe (2/7 scenarios, first failure model_error: the Environment Server does not receive the terminal model error). The same failure is on main: scheduled run 39984595370 at e0811535f fails the KiloCode probes identically, along with Publish check. This PR changes only langchain_deepagents_agent and its two configs, so no change is needed here.

@svcnvidia-nemo-ci svcnvidia-nemo-ci added the needs-review PR is ready for code review and waiting on a reviewer label Oct 23, 2026
@yaoyu-33

Copy link
Copy Markdown
Contributor Author

/ok to test a1e2d88

@yaoyu-33 yaoyu-33 left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SHIP WITH CARE. Thanks for closing these session gaps. The new seed, close and restart-only checkpoint paths check out against SimpleAgent and single_agent_turn:

  • /verify keeps the Resources seed cookies.
  • Restart-only tracking covers the overridden responses().
  • The seed's 422 is final.

a1e2d88 fixes the test_global_config.py assertion that failed Unit tests / Test at 674c828.

The inline comments hold 1 RISK and 2 NOTEs:

  • RISK: the documented --agent-type path breaks once #4953 lands.
  • NOTE: skip_verification no longer applies on the new relay.
  • NOTE: a test gap for optional grants.

PR hygiene

  • Could the Validation section list the core pytest tests/unit_tests/ run at a1e2d88? That suite is where the stale legacy_agent assertion showed up.
  • The KiloCode conformance failure matches scheduled main run 39984595370, so I agree it needs no change here.

Follow-up outside this PR

  • In the Tavily pairing, the agent's own TavilySearch tool (reasoning_search_agent.py:50) never applies the resources server's exclude_domains_file_path policy.
  • Yet tavily_search/README.md says that policy applies to every Tavily configuration.
  • This dates from #2929 and is worth its own issue.

Not covered by this review: a live Tavily rollout and GPU tests. Reviewed by a nine-reviewer team.

@yaoyu-33

Copy link
Copy Markdown
Contributor Author

/ok to test e73a26b

yaoyu-33 added a commit that referenced this pull request Oct 26, 2026
- Set the TB 2.1 full evaluation to 8 repeats per task, as the
  benchmark's OpenCode, Terminus 2, and Inkling configs declare; the
  README's run config declares none.
- Note that exporting NEMO_GYM_ALLOW_UNSUPPORTED_PAIRING=1 also works
  with gym eval run --no-serve.
- Link the deepagents session caveat to #5003, which fixes it.
- Say in Step 4 that the probes are optional at the Evaluation level.
- Replace "very few unhealthy rollouts" with a stated cause for each.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com>
yaoyu-33 and others added 3 commits October 27, 2026 14:33
…t_turn names none

A legacy_agent relay names no resources server: the agent's /run uses its
own. A single_agent_turn_legacy server that replaces the relay had to pin
one, and a pinned name breaks a benchmark that reaches the server through
an _inherit_from rename of the agent. The rename also renames the
resources server, and Gym follows renames only for agent references, so
the pinned name dangles (for example indic/math_500 through
math_with_judge's server).

resources_server is now optional. Unset, the server binds the resources
server the bound agent names, as the relay did, so it follows the agent
rename. A server that sets none in front of an agent that names none
fails at startup instead of on the first episode.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com>
About 307 configs reach simple_agent through a legacy_agent relay, so a
crash repeats the whole episode. Under single_agent_turn_legacy, it
repeats only the current step, and simple_agent resumes inside its turn
(GYMINT-549).

add_legacy_agent_environment_servers.py gains --switch-simple-agent. It
replaces each relay in front of a simple_agent with a
single_agent_turn_legacy server of the same name, so rows keep their
routing. The server names no resources server, so it binds the agent's,
as the relay did, and follows an agent rename. It grants direct HTTP
access to the resources server's tools, the routes the agent's /run
called.

It resolves each config with its own config_paths, as Gym does, since
instance names are unique only within a run. A relay stays, listed with
its reason, when its agent binds no resources server, declares a grant
simple_agent sessions reject (a required MCP access or a second direct
HTTP access), binds a resources server whose seed_session takes only the
legacy body (19 relays today, read from the server's source so a server
passes once it migrates), or its run cannot load an include. A relay that inherits
another keeps inheriting it, and the script refuses to switch only one
of the two. The default mode is unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com>
The pairing guard only compared `_inherit_from` children that carry their own
`legacy_agent` key, so a run that merged a switched relay with a same-name
`legacy_agent` override (the hle and apex_shortlist opencode/pi recipes) or a
bare `_inherit_from` child exited 0 with two environment server types.

- Resolve every run after the planned switches. A relay that a run merges with
  a legacy_agent server stays on the relay; when the relay it inherits switches,
  the script refuses (exit 2) and names the config. Top-level nulls now remove
  instances, as in Gym.
- Keep relays whose resources server reads the row's rollout coordinates
  (genrm_compare), whose implementation is not under resources_servers/, or
  that set keys the switched stanza would drop.
- Build every rewrite before writing any file; a stanza that cannot be found
  exits 2 instead of raising StopIteration.
- Dedupe scanned paths after resolving them.
- Give the hle and apex_shortlist opencode/pi recipes a legacy_agent server of
  their own, so their benchmarks can switch.
- Fix the docstring and docs examples, document the new stay reasons, note how
  to inherit the single_agent_turn templates, and test the legacy adapter with
  resources_server unset.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com>
@copy-pr-bot

copy-pr-bot Bot commented Nov 10, 2026

Copy link
Copy Markdown

/ok to test bd0404d

@yaoyu-33, there was an error processing your request: E2

See the following link for more information: https://docs.gha-runners.nvidia.com/cpr/e/2/

Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com>
@yaoyu-33

Copy link
Copy Markdown
Contributor Author

/ok to test 7579a80

1 similar comment
@yaoyu-33

Copy link
Copy Markdown
Contributor Author

/ok to test 7579a80

main split tests/unit_tests/tooling/test_add_legacy_agent_environment_servers.py
into per-topic files (#5946). Take main's version of that file and move
this branch's test_deepagents_sandbox_template_composes_with_one_environment
into test_agent_environment_default_configs.py, beside the OpenCode and
OpenClaw session-template tests it sat between.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com>
@yaoyu-33

Copy link
Copy Markdown
Contributor Author

/ok to test eb161da

Resolve fern/versions/latest/pages/agent-server/integrate-existing-agents.mdx:
main and this branch each added a row to the sandboxed-harness table.
Keep main's Copilot row and this branch's DeepAgents row, each as written.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com>
@yaoyu-33

Copy link
Copy Markdown
Contributor Author

/ok to test cc4625d

Main added mini-SWE beside DeepAgents in AGENT_SESSION_AGENT_TYPES and in the
two harness tables; keep both rows.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: yaoyu-33 <yuya@nvidia.com>
@yaoyu-33

Copy link
Copy Markdown
Contributor Author

/ok to test 2a4a00f

Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com>
@yaoyu-33

Copy link
Copy Markdown
Contributor Author

/ok to test f38e4f5

Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com>
@yaoyu-33

Copy link
Copy Markdown
Contributor Author

/ok to test 201a545

Take main's release.yaml: #6066 starts the changelog at the previous
release tag for main and pull-request pushes, which replaces this
branch's PR-only workaround. Keep Droid beside DeepAgents in
AGENT_SESSION_AGENT_TYPES and the harness tables.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: yaoyu-33 <yuya@nvidia.com>
@yaoyu-33

Copy link
Copy Markdown
Contributor Author

/ok to test e637ab1

Main added OpenHands beside DeepAgents in the harness tools table;
keep both rows.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: yaoyu-33 <yuya@nvidia.com>
@yaoyu-33

Copy link
Copy Markdown
Contributor Author

/ok to test a7b6eb4

#5775 left the benchmarks/healthbench_professional/prepare.py entry unterminated, so detect-secrets cannot read the baseline and the secrets-detector check fails on every PR. Restore the missing is_secret field and closing brackets.

Signed-off-by: Chris Wing <cwing@nvidia.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit b2a2791)
Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com>
@yaoyu-33

Copy link
Copy Markdown
Contributor Author

/ok to test e180806

weijiac0619
weijiac0619 previously approved these changes Nov 12, 2026
Main added Cline beside DeepAgents in the harness tools table; keep
both rows.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: yaoyu-33 <yuya@nvidia.com>
@yaoyu-33

Copy link
Copy Markdown
Contributor Author

/ok to test 7e9a588

Main added kimi_code_agent beside langchain_deepagents_agent in
AGENT_SESSION_AGENT_TYPES; keep both.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: yaoyu-33 <yuya@nvidia.com>
@yaoyu-33

Copy link
Copy Markdown
Contributor Author

/ok to test ba9db04

@github-actions

Copy link
Copy Markdown
Contributor

🔄 Merge queue validation started! You can track the progress here: https://github.com/NVIDIA-NeMo/Gym/actions/runs/43337740900

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:agent Agent harnesses and Responses API agent behavior bug Something isn't working needs-review PR is ready for code review and waiting on a reviewer

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants