Skip to content

fix(anyterminal_agent): carry the training-capture prefix on the model URL - #3397

Merged
yaoyu-33 merged 1 commit into
NVIDIA-NeMo:mainfrom
jiwasawa:jiwasawa/anyterminal-token-capture-prefix
Sep 15, 2026
Merged

yaoyu-33 merged 1 commit into
NVIDIA-NeMo:mainfrom
jiwasawa:jiwasawa/anyterminal-token-capture-prefix

Conversation

@jiwasawa

@jiwasawa jiwasawa commented Sep 15, 2026 •

Copy link
Copy Markdown
Contributor

What does this PR do?

Passes token_capture=self._token_id_capture_enabled() where anyterminal_agent builds the model URL for the task container, so training-token capture records the rollout. Same class of silent failure as verifiers_agent in #3321.

Training-token capture is keyed by the /ng-rollout/<id>/training-token-capture URL prefix that an agent hands its model client, see rollout_path_prefix in nemo_gym/server_utils.py and its callers in nemo_gym/base_responses_api_agent.py. anyterminal_agent/app.py:_setup_params builds the model URL for the task container without it:

server_config["model_server_url"] = apply_rollout_prefix(server_config["model_server_url"], rollout_id)

apply_rollout_prefix in nemo_gym/server_utils.py takes a keyword-only token_capture that defaults to False, so the container gets the evaluation prefix /ng-rollout/<id> and the model server records no training tokens for the rollout.

With token_id_capture.enabled: true and token_id_capture: true on the agent, the configuration validates and the run completes normally. Every rollout comes back with _ng_token_capture.n_calls == 0 and mask_sample: true. No error, no warning. A run that captured nothing looks like a healthy one until the trainer masks every sample.

The base class passes the keyword in url_path_for_run, base_url_for_run, and resolve_model_base_url, and claude_code_agent and swe_agents pass it since #2124. The anyterminal wrapper builds its URL on its own path, and that path did not get the keyword.

With capture off, _token_id_capture_enabled() is False and the URL is unchanged, so evaluation runs do not move.

Tests

Two tests in TestSetupParams. They call _setup_params with a real global config dict instead of patching the capture protocol.

test asserts
test_model_server_url_carries_the_training_capture_prefix capture on gives http://policy:8000/ng-rollout/7-2/training-token-capture
test_model_server_url_keeps_the_plain_rollout_prefix_without_capture capture off gives http://policy:8000/ng-rollout/7-2

The first fails on main with 'http://policy:8000/ng-rollout/7-2' != '.../training-token-capture'. The second is a negative control.

pytest responses_api_agents/anyterminal_agent/tests/test_app.py

81 passed. pre-commit run --all-files clean.

On real rollouts

gym eval run on three Terminal-Bench 2 tasks, Codex CLI 0.144.4 running inside the task container through anyterminal_agent, Qwen3-8B served by vLLM on one GPU, token_id_capture.enabled: true with the agent opted in. Both arms use the same overlay, the same task list, and the same vLLM process. The only difference is this keyword. Gym is main @ 5b34fcdac in both arms.

without the keyword (main) with the keyword
task containers handed a /training-token-capture prefix, per params.json 0 of 3 3 of 3
rollouts with _ng_token_capture.n_calls > 0 0 3
rollouts with mask_sample: true 3 of 3 0 of 3
captured model calls 0 147
generation tokens delivered 0 14,449
function_call items Codex produced 136 145

The last row shows Codex ran normally in both arms. The difference is only in what the model server recorded. On the unfixed arm gym eval run also prints Rollout health: 3 checked, 0 healthy, 0 unhealthy, 3 unobserved.

Both arms also carry local workarounds for #3312 and #3313, which Codex needs to complete a turn against a chat-backed model server. They touch the Responses converter and the streaming event, not the URL path.

Checklist

  • I have read the contributing guidelines.
  • The change is focused; unrelated "drive-by" edits are tracked as separate issues/PRs.
  • Tests added or updated and pass locally, or N/A for docs-only / non-code changes (so CI unit/server checks pass when applicable).
  • Pre-commit checks pass locally (pre-commit run --all-files) (so CI lint/format/copyright pass).
  • All commits have DCO sign-off (git commit -s) (so the DCO check passes).

…l URL

Signed-off-by: jiwasawa <jiwasawa@nvidia.com>
@copy-pr-bot

copy-pr-bot Bot commented Sep 15, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@jiwasawa
jiwasawa marked this pull request as ready for review September 15, 2026 07:09
@yaoyu-33 yaoyu-33 added area:agent Agent harnesses and Responses API agent behavior bug Something isn't working complexity:low Localized change in one scope with a small, straightforward review surface needs-review PR is ready for code review and waiting on a reviewer labels Sep 15, 2026
@yaoyu-33 yaoyu-33 added ready-to-merge PR is approved, current, and waiting for final CI or merge and removed needs-review PR is ready for code review and waiting on a reviewer labels Sep 15, 2026
@yaoyu-33

Copy link
Copy Markdown
Contributor

/ok to test 4fb25ce

@yaoyu-33
yaoyu-33 enabled auto-merge September 15, 2026 19:09
@yaoyu-33
yaoyu-33 added this pull request to the merge queue Sep 15, 2026
Merged via the queue into NVIDIA-NeMo:main with commit c06cbf2 Sep 15, 2026
47 checks passed
pthombre added a commit that referenced this pull request Sep 16, 2026
Under training token capture the model server only returns token metadata
for calls made under /ng-rollout/<id>/training-token-capture/...; the
SciCode agent posted to the bare /v1/responses (self-call and model call),
so every sub-step failed NeMoGymChatCompletion validation with 'Token
metadata must include all required fields' (HTTP 500 on /run). Mirror the
simple_agent / anyterminal (#3397) pattern with url_path_for_run and
url_path_for_request.

Signed-off-by: Pranav Thombre <pthombre@nvidia.com>
linj-glitch added a commit to linj-glitch/Gym that referenced this pull request Oct 8, 2026
…the inner harness in capture mode

`_setup_params` built the in-sandbox model URL with
`apply_rollout_prefix(url, rollout_id)` and no `token_capture=` keyword, so
the harness only ever got the eval prefix `/ng-rollout/<id>`. With
`token_id_capture.enabled: true` the run looks healthy, but every rollout
reports `n_calls == 0` and is masked. NVIDIA-NeMo#3397 fixed the same bug in
anyterminal_agent. Pass `self._token_id_capture_enabled()`.

Two more gaps kept captured rollouts from being attributed:

- The in-sandbox runner built the inner agent's global config without a
  `token_id_capture` block, so the inner agent ran in eval mode. When the
  model URL carries the training-token-capture segment, the runner now
  enables capture for the inner agent.
- The outer response replaced the inner agent's id with
  `anyswe-<instance_id>`. Under capture, keep the inner id (the served id
  the inner agent reports) so terminal attribution can match it. Eval mode
  keeps the synthetic id.

Signed-off-by: Lin Jia <linj@nvidia.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
linj-glitch added a commit to linj-glitch/Gym that referenced this pull request Oct 8, 2026
…the inner harness in capture mode

`_setup_params` built the in-sandbox model URL with
`apply_rollout_prefix(url, rollout_id)` and no `token_capture=` keyword, so
the harness only ever got the eval prefix `/ng-rollout/<id>`. With
`token_id_capture.enabled: true` the run looks healthy, but every rollout
reports `n_calls == 0` and is masked. NVIDIA-NeMo#3397 fixed the same bug in
anyterminal_agent. Pass `self._token_id_capture_enabled()`.

Two more gaps kept captured rollouts from being attributed:

- The in-sandbox runner built the inner agent's global config without a
  `token_id_capture` block, so the inner agent ran in eval mode. When the
  model URL carries the training-token-capture segment, the runner now
  enables capture for the inner agent.
- The outer response replaced the inner agent's id with
  `anyswe-<instance_id>`. Under capture, keep the inner id (the served id
  the inner agent reports) so terminal attribution can match it. Eval mode
  keeps the synthetic id.

Signed-off-by: Lin Jia <linj@nvidia.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@jiwasawa
jiwasawa deleted the jiwasawa/anyterminal-token-capture-prefix branch October 28, 2026 03:03

This branch was previously deployed

1 inactive deployment
public — 4fb25ce1 Deployed Sep 15, 2026 by copy-pr-bot[bot] via release / finalize / notify #2995
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:agent Agent harnesses and Responses API agent behavior bug Something isn't working complexity:low Localized change in one scope with a small, straightforward review surface ready-to-merge PR is approved, current, and waiting for final CI or merge

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants