Repository navigation
fix(health): do not flag a context overflow after an answer, and flag a reasoning-only runaway #4921
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
AdamRajfer
merged 19 commits into
NVIDIA-NeMo:main
from
AdamRajfer:eval/health-context-overflow
Oct 26, 2026
Merged
fix(health): do not flag a context overflow after an answer, and flag a reasoning-only runaway #4921
Changes from 3 commits
Commits
Show all changes
19 commits
Select commit
Hold shift + click to select a range
721f30a
fix(health): do not flag a rollout that ended on a context overflow
AdamRajfer 2e71c8f
fix(health): an empty turn from the model's own context overflow is n…
AdamRajfer 00f1ee5
fix(health): match an overflow call by the reference its turn uses
AdamRajfer 247f0f7
fix(health): excuse an empty turn only by its own last call's overflow
AdamRajfer c111485
test(health): an ambiguous response reference excuses no empty turn
AdamRajfer 71c72c1
Merge main into the PR branch
AdamRajfer a281b6c
fix(vllm_model): label an invalid output budget apart from a context …
AdamRajfer 6fea91d
fix(health): excuse an overflow only once the model has answered in t…
AdamRajfer 914ab32
fix(health): excuse an overflow only after an answer in the same conv…
AdamRajfer 2df9023
docs(health): say what a missing reference resolves to in the call bi…
AdamRajfer e0fc1ca
fix(health): two captured sessions that open alike are not one conver…
AdamRajfer 49a906a
fix(health): flag a length-limited call that gave the agent nothing t…
AdamRajfer 3d4bf6c
fix(health): keep an empty overflow turn hollow when no other check a…
AdamRajfer b9cc04f
fix(vllm_model): name an output budget larger than the whole window a…
AdamRajfer a0af975
fix(vllm_model): read the context window from vLLM 0.27's refusal wor…
AdamRajfer 3dd9063
fix(health): skip the token checks for any upstream length refusal
AdamRajfer e549a58
fix(health): exempt any length refusal from the runaway observation gate
AdamRajfer 74fee0c
Merge branch 'main' into eval/health-context-overflow
oyilmaz-nvidia 14ae4dd
Merge main into eval/health-context-overflow
AdamRajfer File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Some comments aren't visible on the classic Files Changed page.
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.