Skip to content

harbor: reward.json without a reward key is scored as the sum of its fields #5641

Open

Description

@qiaochuz-nv

Found while running SkillEvaluator's Tier 3 task folders on Gym. Gym cc8991e4 (main, 2026-11-04), Docker sandbox.

Summary. If a Harbor verifier writes a multi-metric reward.json with no reward key, the trial's reward is the sum of every field. SkillEvaluator's verifier writes six 0–1 metrics plus overall, so its trials score up to about 7.

Repro (no model):

gym dataset init local/repro --root .
cat > repro/hello/tests/test.sh <<'EOF'
#!/bin/bash
mkdir -p /logs/verifier
echo '{"accuracy": 1.0, "overall": 0.5}' > /logs/verifier/reward.json
EOF
gym dataset validate repro --sandbox docker

Observed. results/harbor/validation/repro/rollouts_run_summary.json: "reward": 1.5, verifier.rewards: {"accuracy": 1.0, "overall": 0.5}; mean/reward 1.5; the command exits 0.

Expected. A way to name the field that is the score (here overall), so per-metric scores are kept as metrics and not added up.

Proposed fix. Opt-in reward_key in dataset.toml ([gym.defaults], overridable under [gym.tasks."<id>".verifier]), read by select_reward in environment_servers/harbor/verifier.py; a missing key is harbor:invalid_reward. The default (sum) is unchanged. Related: #2137.

Activity

  1. added a commit that references this issue on Nov 8, 2026
    000034d
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions