Repository navigation
feat(scripts): switch simple_agent relays to single_agent_turn_legacy - #4952
Merged
Merged
GitHub Actions / Harness conformance (non-blocking)
failed
Oct 28, 2026 in 0s
5 passed, 0 failed, 1 unavailable
| Harness | P0 result | Failed scenario checks | First failure | Evidence |
|---|---|---|---|---|
| opencode | passed | 0 / 7 | None | Download report and logs |
| pi | passed | 0 / 7 | None | Download report and logs |
| codex | passed | 0 / 7 | None | Download report and logs |
| hermes | passed | 0 / 7 | None | Download report and logs |
| kilocode | unavailable | 1 / 7 | model_error: the EnvironmentServer must receive a failed response retaining the model error and ng_agent_outcome with model_request_error and the witnessed HTTP status | Download report and logs |
| stirrup | passed | 0 / 7 | None | Download report and logs |
Counts include execution errors and count each scenario once. Unavailable means setup, tests, execution, or evidence collection failed.
Each artifact contains probes/conformance_report.md, setup.log, tests.log, and per-scenario probes/<harness>/<scenario>/scenario_result.json and episode.log.
Rerun locally
From the tested Gym revision, activate your Gym environment and install the selected adapters' requirements and pinned runtimes. No model service or model credentials are needed.
python scripts/run_harness_conformance.py --harness kilocode --output "$(mktemp -d)/probes"Each command runs the full suite for that harness and creates a fresh output directory. Open conformance_report.md in that directory for the full report.
Tested commit: 84e0a93ef8755f18a607dd653f7820455d4be2dd.
Probe summaries and downloadable evidence. Local setup instructions.
Loading