Skip to content

fix(openclaw): honor max_output_tokens in sandbox sessions - #4476

Merged
athitten merged 2 commits into
NVIDIA-NeMo:mainfrom
AdamRajfer:openclaw-sandbox-max-output-tokens
Oct 12, 2026
Merged

athitten merged 2 commits into
NVIDIA-NeMo:mainfrom
AdamRajfer:openclaw-sandbox-max-output-tokens

Conversation

@AdamRajfer

Copy link
Copy Markdown
Contributor

Sandbox (native) OpenClaw sessions rejected the configured max_output_tokens, so the generated model entry had no maxTokens and OpenClaw used its own per-call output default. On a long-form agentic benchmark, a quarter of the judged deliverables ended truncated at that default while a comparable harness ran at 32768.

Changes:

  • Accept max_output_tokens in sandbox mode and write it as maxTokens of the generated model entry, as the local CLI path already does.
  • Validate it (positive, not above context_window) at activation.
  • README: describe the behaviour; keep the note that metadata does not prove an effective inference limit and that an episode-wide budget is not implemented.

Default stays unset, so existing configs behave exactly as before. Measured on a full 220-task run with the recipe setting 32768: truncated deliverables fell from 55 to 3 and the comparison ELO rose by about 95 points on the same tasks and judges.

Mirrors yaoyu-33#13 onto main now that #3637 is merged.

🤖 Generated with Claude Code

Sandbox sessions rejected the configured max_output_tokens, so the
generated model entry never carried maxTokens and OpenClaw fell back
to its own per-call output default. Accept the value, validate it
against context_window, and write it as maxTokens like the local CLI
path already does. Default stays unset, so existing configs behave
the same.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Signed-off-by: Adam Rajfer <arajfer@nvidia.com>
@nemo-automation-bot
nemo-automation-bot Bot requested a review from a team October 12, 2026 15:16
@yaoyu-33 yaoyu-33 added area:agent Agent harnesses and Responses API agent behavior bug Something isn't working complexity:low Localized change in one scope with a small, straightforward review surface needs-review PR is ready for code review and waiting on a reviewer labels Oct 12, 2026
@AdamRajfer
AdamRajfer requested a review from yaoyu-33 October 12, 2026 16:00
@AdamRajfer AdamRajfer self-assigned this Oct 12, 2026
@AdamRajfer

Copy link
Copy Markdown
Contributor Author

/ok to test aea3464

Two regression tests: a configured max_output_tokens reaches the
generated model entry as maxTokens (with and without context_window),
and invalid values (non-positive, or above context_window) are
rejected at activation before any sandbox command runs.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Signed-off-by: Adam Rajfer <arajfer@nvidia.com>

@athitten athitten left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @AdamRajfer ! LGTM

@athitten
athitten added this pull request to the merge queue Oct 12, 2026
@github-actions

Copy link
Copy Markdown
Contributor

🔄 Merge queue validation started! You can track the progress here: https://github.com/NVIDIA-NeMo/Gym/actions/runs/38301367328

Merged via the queue into NVIDIA-NeMo:main with commit 26496cb Oct 12, 2026
49 checks passed
khazic pushed a commit to khazic/Gym-lao that referenced this pull request Oct 13, 2026
## What does this PR do?

README only. Tells OpenClaw users to set `context_window` to the model
server's maximum context length, and explains how OpenClaw uses it and
`max_output_tokens`. Follow-up to NVIDIA-NeMo#4476, which lets sandbox sessions use
`max_output_tokens`; there is no issue.

Why: when `context_window` is unset, OpenClaw assumes 200,000 tokens. On
a server with a smaller limit, it keeps requesting its full output limit
and does not compact in time, so long episodes can send requests the
server rejects. Gym's `vllm_model` returns an empty reply, not an error,
by default, and the run ends early.

Example, worked out from the code and not measured in a run: released
vLLM (up to 0.31) rejects a call when prompt + requested output >
`max_model_len`. At 131,072 with no `context_window`, calls fail once
the prompt passes 122,880 tokens (131,072 − 8,192 default), or 98,304
with `max_output_tokens: 32768` (131,072 − 32,768). With
`context_window: 131072`, OpenClaw lowers each call's output to fit the
room left by its estimate of the input; the estimate can undercount code
or JSON. Unreleased vLLM main (vllm-project/vllm#57035) lowers the
output instead of rejecting, but still rejects a prompt longer than
`max_model_len`.

The sandbox section now also says that the request field is rejected but
the agent config field applies, and that a `max_output_tokens` above
`context_window` is rejected (the check from NVIDIA-NeMo#4476).

## Validation

- Read the OpenClaw npm package at 2026.6.11 (pinned in
`configs/openclaw_agent.yaml`) and at 2026.6.35 (the pin in NVIDIA-NeMo#4082). Both
match the README:
- `contextWindow` defaults to 200,000 and `maxTokens` to min(8,192,
`contextWindow`) (`dist/io-*.js`);
- each call requests at most min(`maxTokens`, `contextWindow` −
estimated input − 1), where the estimate is characters / 4 × 1.25. This
applies to any base URL other than OpenAI's or Azure OpenAI's, which
covers Gym's model server (`dist/openai-transport-stream-*.js`);
- two in-run limits are computed from `contextWindow`. One is the
context guard: once a run passes its limit, OpenClaw compacts and
retries (`installToolResultContextGuard` in `dist/selection-*.js`). The
other is the cap on each tool result's size
(`resolveLiveToolResultMaxChars` in `dist/tool-result-truncation-*.js`).
- Gym code on main: `_build_openclaw_config` writes `contextWindow` and
`maxTokens` only when they are set, and `vllm_model` returns an empty
`length` completion on a context-length error unless
`propagate_context_overflow_errors` is true.
- Open PRs touching OpenClaw: NVIDIA-NeMo#3902 and NVIDIA-NeMo#2733 edit this README, and
NVIDIA-NeMo#4082 moves the pin to 2026.6.35. `git merge-tree` shows this PR adds no
conflicts with any of them.
- `pre-commit run --all-files`: passed.

## Rollouts

N/A: README-only change, with no code or config changes.

## Compatibility and benchmark impact

None from this PR. Following the advice can change rollouts with long
episodes or long tool results, because `context_window` also sets when
OpenClaw compacts and how much of a tool result it keeps.

## Checklist

- [x] I have read the [contributing
guidelines](https://github.com/NVIDIA-NeMo/Gym/blob/main/CONTRIBUTING.md).
- [x] The change is focused; unrelated "drive-by" edits are tracked as
separate issues/PRs.
- [x] Tests added or updated and pass locally, or N/A for docs-only /
non-code changes (so CI unit/server checks pass when applicable). N/A:
docs-only.
- [x] Documentation added or updated, or N/A with justification.
- [x] Pre-commit checks pass locally (`pre-commit run --all-files`) (so
CI lint/format checks pass).
- [x] New source files include the required Apache-2.0 SPDX header. N/A:
no new files.
- [x] All commits have DCO sign-off (`git commit -s`) (so the DCO check
passes).

Signed-off-by: Abhishree <athittenaman@nvidia.com>
khazic pushed a commit to khazic/Gym-lao that referenced this pull request Oct 15, 2026
## Summary
- Add the SA-V tracking verifier and its server configuration.
- Declare the SA-V environment servers used by video RL.
- Add portable five-row example data, schema, metrics, and checked
rollout artifacts.

## Why
This is the Gym-side verifier foundation for mixed SA-V tracking and
CapRL video training in NeMo-RL. No separate issue is needed; this is
part of the Super 3.5 video RL upstreaming stack.

## Validation
- Runtime evidence: the [healthy 16-node async `r18` W&B
run](https://wandb.ai/adlr/Nemotron-omni-RL-debug/runs/oci-hsg-cs-001_n3post_main_1d76d243b_vllm029_super35_video_step120_async16n_trainmb45056_expandable_uncapped_r18_20261003)
shows repeated GRPO cycles, stable TMPE, and increasing reward on the
equivalent video implementation.
- `gym dataset collate ... +mode=example_validation`: 5 examples;
generated `data/example_metrics.json`.
- Focused validation: 25 verifier tests and all 5 example rows against
`task_data.py` passed.
- Ruff format and lint passed for the changed Python files.
- Both Gym stack ranges pass `git diff --check`.

## Rollouts
`data/example_rollouts.jsonl` contains one checked rollout for each of
the five example rows. Representative task checks:

- `pt`:
`<answer>{"tracks":[{"point":1,"frame":1,"xy":[500,500],"visible":true}]}</answer>`
— `format_ok=true`, reward `1.0`, visible-frame mask membership `true`.
- `box`:
`<answer>{"tracks":[{"box":1,"frame":1,"bbox":[250,250,750,750],"visible":true}]}</answer>`
— `format_ok=true`, reward `1.0`, frame IoU `1.0`.
- `ref`:
`<answer>{"tracks":[{"id":7,"frame":1,"xy":[500,500]}]}</answer>` —
`format_ok=true`, reward `1.0`, predicted id `7` assigned to the GT
object and mask membership `true`.

The artifact also contains visible/invisible `pt` coverage and an
all-absent `ref` negative case, preserving the verifier's current
unmatched-ID behavior.

## Compatibility
Adds a new verifier/configuration and does not change unrelated
environments. Example media is embedded as a small inline JPEG, so no
employee or cluster-local paths are required.

## Stack
Gym PR 1 of 2. Consolidates and supersedes the implementation scope of
NVIDIA-NeMo#3504 after review history is preserved.

## Consolidated stack links

### Gym
1. [Gym NVIDIA-NeMo#3999 — SA-V tracking
verifier](NVIDIA-NeMo#3999)
2. [Gym NVIDIA-NeMo#4000 — R3 routing replay and rollout
identity](NVIDIA-NeMo#4000)

### NeMo-RL
1. [NeMo-RL NVIDIA-NeMo#4472 — Super video data and model
loading](NVIDIA-NeMo/RL#4472)
2. [NeMo-RL NVIDIA-NeMo#4473 — RL/MOPD video frame
contract](NVIDIA-NeMo/RL#4473)
3. [NeMo-RL NVIDIA-NeMo#4474 — sync/async Gym routing
runtime](NVIDIA-NeMo/RL#4474)
4. [NeMo-RL NVIDIA-NeMo#4475 — SA-V pin and CI
fixtures](NVIDIA-NeMo/RL#4475)
5. [NeMo-RL NVIDIA-NeMo#4476 — final dependency
pins](NVIDIA-NeMo/RL#4476)

### Child dependency refs
- [MCore integration
ref](https://github.com/ehosseiniasl/Megatron-LM/tree/ehsan/video-r3-mtp-gradient-integration-20261004),
adapting [MCore #7278](NVIDIA/Megatron-LM#7278)
and [MCore #7279](NVIDIA/Megatron-LM#7279) to
the pinned base.
- [MBridge integration
ref](https://github.com/NVIDIA-NeMo/Megatron-Bridge/tree/ehsan/video-r3-mcore-pin-integration-20261004).

---------

Signed-off-by: ehosseiniasl <ehsan.hosseiniasl@gmail.com>
Co-authored-by: Pulkit Kumar <pulkitk@nvidia.com>

This branch was previously deployed

1 inactive deployment
public — 03ed4c7f Deployed Oct 12, 2026 by copy-pr-bot[bot] via release / finalize / notify #5079
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:agent Agent harnesses and Responses API agent behavior bug Something isn't working complexity:low Localized change in one scope with a small, straightforward review surface needs-review PR is ready for code review and waiting on a reviewer

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants