Repository navigation
fix(openclaw): honor max_output_tokens in sandbox sessions - #4476
Merged
athitten merged 2 commits intoOct 12, 2026
Merged
Conversation
Sandbox sessions rejected the configured max_output_tokens, so the generated model entry never carried maxTokens and OpenClaw fell back to its own per-call output default. Accept the value, validate it against context_window, and write it as maxTokens like the local CLI path already does. Default stays unset, so existing configs behave the same. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Signed-off-by: Adam Rajfer <arajfer@nvidia.com>
Contributor
Author
|
/ok to test aea3464 |
athitten
reviewed
Oct 12, 2026
Two regression tests: a configured max_output_tokens reaches the generated model entry as maxTokens (with and without context_window), and invalid values (non-positive, or above context_window) are rejected at activation before any sandbox command runs. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Signed-off-by: Adam Rajfer <arajfer@nvidia.com>
athitten
approved these changes
Oct 12, 2026
athitten
left a comment
Contributor
There was a problem hiding this comment.
Thank you @AdamRajfer ! LGTM
Contributor
|
🔄 Merge queue validation started! You can track the progress here: https://github.com/NVIDIA-NeMo/Gym/actions/runs/38301367328 |
This was referenced Oct 12, 2026
7 tasks done
khazic
pushed a commit
to khazic/Gym-lao
that referenced
this pull request
Oct 13, 2026
## What does this PR do? README only. Tells OpenClaw users to set `context_window` to the model server's maximum context length, and explains how OpenClaw uses it and `max_output_tokens`. Follow-up to NVIDIA-NeMo#4476, which lets sandbox sessions use `max_output_tokens`; there is no issue. Why: when `context_window` is unset, OpenClaw assumes 200,000 tokens. On a server with a smaller limit, it keeps requesting its full output limit and does not compact in time, so long episodes can send requests the server rejects. Gym's `vllm_model` returns an empty reply, not an error, by default, and the run ends early. Example, worked out from the code and not measured in a run: released vLLM (up to 0.31) rejects a call when prompt + requested output > `max_model_len`. At 131,072 with no `context_window`, calls fail once the prompt passes 122,880 tokens (131,072 − 8,192 default), or 98,304 with `max_output_tokens: 32768` (131,072 − 32,768). With `context_window: 131072`, OpenClaw lowers each call's output to fit the room left by its estimate of the input; the estimate can undercount code or JSON. Unreleased vLLM main (vllm-project/vllm#57035) lowers the output instead of rejecting, but still rejects a prompt longer than `max_model_len`. The sandbox section now also says that the request field is rejected but the agent config field applies, and that a `max_output_tokens` above `context_window` is rejected (the check from NVIDIA-NeMo#4476). ## Validation - Read the OpenClaw npm package at 2026.6.11 (pinned in `configs/openclaw_agent.yaml`) and at 2026.6.35 (the pin in NVIDIA-NeMo#4082). Both match the README: - `contextWindow` defaults to 200,000 and `maxTokens` to min(8,192, `contextWindow`) (`dist/io-*.js`); - each call requests at most min(`maxTokens`, `contextWindow` − estimated input − 1), where the estimate is characters / 4 × 1.25. This applies to any base URL other than OpenAI's or Azure OpenAI's, which covers Gym's model server (`dist/openai-transport-stream-*.js`); - two in-run limits are computed from `contextWindow`. One is the context guard: once a run passes its limit, OpenClaw compacts and retries (`installToolResultContextGuard` in `dist/selection-*.js`). The other is the cap on each tool result's size (`resolveLiveToolResultMaxChars` in `dist/tool-result-truncation-*.js`). - Gym code on main: `_build_openclaw_config` writes `contextWindow` and `maxTokens` only when they are set, and `vllm_model` returns an empty `length` completion on a context-length error unless `propagate_context_overflow_errors` is true. - Open PRs touching OpenClaw: NVIDIA-NeMo#3902 and NVIDIA-NeMo#2733 edit this README, and NVIDIA-NeMo#4082 moves the pin to 2026.6.35. `git merge-tree` shows this PR adds no conflicts with any of them. - `pre-commit run --all-files`: passed. ## Rollouts N/A: README-only change, with no code or config changes. ## Compatibility and benchmark impact None from this PR. Following the advice can change rollouts with long episodes or long tool results, because `context_window` also sets when OpenClaw compacts and how much of a tool result it keeps. ## Checklist - [x] I have read the [contributing guidelines](https://github.com/NVIDIA-NeMo/Gym/blob/main/CONTRIBUTING.md). - [x] The change is focused; unrelated "drive-by" edits are tracked as separate issues/PRs. - [x] Tests added or updated and pass locally, or N/A for docs-only / non-code changes (so CI unit/server checks pass when applicable). N/A: docs-only. - [x] Documentation added or updated, or N/A with justification. - [x] Pre-commit checks pass locally (`pre-commit run --all-files`) (so CI lint/format checks pass). - [x] New source files include the required Apache-2.0 SPDX header. N/A: no new files. - [x] All commits have DCO sign-off (`git commit -s`) (so the DCO check passes). Signed-off-by: Abhishree <athittenaman@nvidia.com>
khazic
pushed a commit
to khazic/Gym-lao
that referenced
this pull request
Oct 15, 2026
## Summary - Add the SA-V tracking verifier and its server configuration. - Declare the SA-V environment servers used by video RL. - Add portable five-row example data, schema, metrics, and checked rollout artifacts. ## Why This is the Gym-side verifier foundation for mixed SA-V tracking and CapRL video training in NeMo-RL. No separate issue is needed; this is part of the Super 3.5 video RL upstreaming stack. ## Validation - Runtime evidence: the [healthy 16-node async `r18` W&B run](https://wandb.ai/adlr/Nemotron-omni-RL-debug/runs/oci-hsg-cs-001_n3post_main_1d76d243b_vllm029_super35_video_step120_async16n_trainmb45056_expandable_uncapped_r18_20261003) shows repeated GRPO cycles, stable TMPE, and increasing reward on the equivalent video implementation. - `gym dataset collate ... +mode=example_validation`: 5 examples; generated `data/example_metrics.json`. - Focused validation: 25 verifier tests and all 5 example rows against `task_data.py` passed. - Ruff format and lint passed for the changed Python files. - Both Gym stack ranges pass `git diff --check`. ## Rollouts `data/example_rollouts.jsonl` contains one checked rollout for each of the five example rows. Representative task checks: - `pt`: `<answer>{"tracks":[{"point":1,"frame":1,"xy":[500,500],"visible":true}]}</answer>` — `format_ok=true`, reward `1.0`, visible-frame mask membership `true`. - `box`: `<answer>{"tracks":[{"box":1,"frame":1,"bbox":[250,250,750,750],"visible":true}]}</answer>` — `format_ok=true`, reward `1.0`, frame IoU `1.0`. - `ref`: `<answer>{"tracks":[{"id":7,"frame":1,"xy":[500,500]}]}</answer>` — `format_ok=true`, reward `1.0`, predicted id `7` assigned to the GT object and mask membership `true`. The artifact also contains visible/invisible `pt` coverage and an all-absent `ref` negative case, preserving the verifier's current unmatched-ID behavior. ## Compatibility Adds a new verifier/configuration and does not change unrelated environments. Example media is embedded as a small inline JPEG, so no employee or cluster-local paths are required. ## Stack Gym PR 1 of 2. Consolidates and supersedes the implementation scope of NVIDIA-NeMo#3504 after review history is preserved. ## Consolidated stack links ### Gym 1. [Gym NVIDIA-NeMo#3999 — SA-V tracking verifier](NVIDIA-NeMo#3999) 2. [Gym NVIDIA-NeMo#4000 — R3 routing replay and rollout identity](NVIDIA-NeMo#4000) ### NeMo-RL 1. [NeMo-RL NVIDIA-NeMo#4472 — Super video data and model loading](NVIDIA-NeMo/RL#4472) 2. [NeMo-RL NVIDIA-NeMo#4473 — RL/MOPD video frame contract](NVIDIA-NeMo/RL#4473) 3. [NeMo-RL NVIDIA-NeMo#4474 — sync/async Gym routing runtime](NVIDIA-NeMo/RL#4474) 4. [NeMo-RL NVIDIA-NeMo#4475 — SA-V pin and CI fixtures](NVIDIA-NeMo/RL#4475) 5. [NeMo-RL NVIDIA-NeMo#4476 — final dependency pins](NVIDIA-NeMo/RL#4476) ### Child dependency refs - [MCore integration ref](https://github.com/ehosseiniasl/Megatron-LM/tree/ehsan/video-r3-mtp-gradient-integration-20261004), adapting [MCore #7278](NVIDIA/Megatron-LM#7278) and [MCore #7279](NVIDIA/Megatron-LM#7279) to the pinned base. - [MBridge integration ref](https://github.com/NVIDIA-NeMo/Megatron-Bridge/tree/ehsan/video-r3-mcore-pin-integration-20261004). --------- Signed-off-by: ehosseiniasl <ehsan.hosseiniasl@gmail.com> Co-authored-by: Pulkit Kumar <pulkitk@nvidia.com>
This branch was previously deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Sandbox (native) OpenClaw sessions rejected the configured
max_output_tokens, so the generated model entry had nomaxTokensand OpenClaw used its own per-call output default. On a long-form agentic benchmark, a quarter of the judged deliverables ended truncated at that default while a comparable harness ran at 32768.Changes:
max_output_tokensin sandbox mode and write it asmaxTokensof the generated model entry, as the local CLI path already does.context_window) at activation.Default stays unset, so existing configs behave exactly as before. Measured on a full 220-task run with the recipe setting 32768: truncated deliverables fell from 55 to 3 and the comparison ELO rose by about 95 points on the same tasks and judges.
Mirrors yaoyu-33#13 onto main now that #3637 is merged.
🤖 Generated with Claude Code