Skip to content

fix(gdpval): count a model that delivers nothing as a lost battle, not a masked row - #6077

Merged
AdamRajfer merged 3 commits into
NVIDIA-NeMo:mainfrom
AdamRajfer:arajfer/gdpval-no-deliverable-loss
Nov 16, 2026
Merged

AdamRajfer merged 3 commits into
NVIDIA-NeMo:mainfrom
AdamRajfer:arajfer/gdpval-no-deliverable-loss

fix(health): register GDPVal's agent_timeout as the failure_kind it n…

83226be
Select commit
Loading
Failed to load commit list.
Sign in for the full log view
GitHub Actions / Harness conformance (non-blocking) succeeded Nov 12, 2026 in 0s

10 passed, 0 failed, 0 unavailable

Harness P0 result Failed scenario checks First failure Evidence
simple passed 1 / 9 context_compaction: expected one completed automatic compaction, or none in the control Download report and logs
opencode passed 0 / 9 None Download report and logs
pi passed 1 / 9 context_compaction: expected one completed automatic compaction, or none in the control Download report and logs
codex passed 0 / 9 None Download report and logs
hermes passed 0 / 9 None Download report and logs
kilocode passed 0 / 9 None Download report and logs
stirrup passed 0 / 9 None Download report and logs
copilot passed 1 / 9 context_compaction: expected one completed automatic compaction, or none in the control Download report and logs
openclaw passed 1 / 9 context_compaction: expected one completed automatic compaction, or none in the control Download report and logs
nooa passed 1 / 9 context_compaction: expected one completed automatic compaction, or none in the control Download report and logs

Counts include execution errors and count each scenario once. Unavailable means setup, tests, execution, or evidence collection failed.

Each artifact contains probes/conformance_report.md, setup.log, tests.log, and per-scenario probes/<harness>/<scenario>/scenario_result.json and episode.log.

Tested commit: c0ff2f590d9f4d52460fc1ecea7d2cc8709f368c.

Probe summaries and downloadable evidence. Local setup instructions.