You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
fix: masked results that still need a failure_kind #5693
Checked every row against main (1f89875) by reading the code. Every site in "Needs a fix" already gives its masked result a failure_kind, so I'm closing this issue. I did not run rollouts; the kinds are covered by unit tests, not observed in runs.
Component
Kind now set
Where
anyswe_agent, sandbox exception
the event's kind, else provider_unavailable; reason from repr(exc)
anyswe_agent/app.py
anyswe_agent, agent_timed_out / eval_timeout / resolved with a limit
agent_timeout / anyswe_agent:eval_timeout / agent_limit or model_context_limit
_masked_failure_kind
anyterminal_agent
error_type carried through first, then provider_unavailable, agent_timeout or anyterminal_agent:eval_timeout
_masked_failure
osworld_agent, configuration refusals
osworld_agent:configuration_error (no failure event, so not retried)
Activity
fix(anyswe_agent,anyterminal_agent): label masked results with a fail…
fix(osworld_agent,interactive_browser,gymnasium): add failure_kind to…
fix(terminal_bench_4): label a masked agent infrastructure failure wi… (
oyilmaz-nvidia commentedon Nov 11, 2026
Checked every row against
main(1f89875) by reading the code. Every site in "Needs a fix" already gives its masked result afailure_kind, so I'm closing this issue. I did not run rollouts; the kinds are covered by unit tests, not observed in runs.provider_unavailable; reason fromrepr(exc)anyswe_agent/app.pyagent_timed_out/eval_timeout/ resolved with a limitagent_timeout/anyswe_agent:eval_timeout/agent_limitormodel_context_limit_masked_failure_kinderror_typecarried through first, thenprovider_unavailable,agent_timeoutoranyterminal_agent:eval_timeout_masked_failureosworld_agent:configuration_error(no failure event, so not retried)verifier_errorandsession_lostsession_lostandenvironment_server_failedinfrastructure_error, so a masked result always has a kindDecision items:
episode_failedand a warning that names the environment (fix(osworld_agent,interactive_browser,gymnasium): add failure_kind to masked and failed results #5773).verifier_error(fix(osworld_agent,interactive_browser,gymnasium): add failure_kind to masked and failed results #5773), but the row is still not masked. Whether it should be masked is a behavior change and stays with refactor(core): replace reward-zero failure sentinels with typed outcomes #3180 / tracking: audit reward-zero and fake-success failure producers #4883.The "Blocked by" PRs listed in the table did not need to merge first.