Repository navigation
chore(hermes_agent): use pinned upstream Hermes release - #2742
cwing-nvidia wants to merge 7 commits into
Conversation
Run Hermes in its official isolated environment so Gym can track a named upstream release without mixing incompatible dependencies. Signed-off-by: Chris Wing <cwing@nvidia.com>
|
Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually. Contributors can view more details about this message here. |
Mark the upstream commit and installer checksum as intentional non-secret integrity values so secret scanning can distinguish them from credentials. Signed-off-by: Chris Wing <cwing@nvidia.com>
Handle text-list and refusal outputs as declared by the Chat Completions schema so Hermes conversion does not lose valid assistant content. Signed-off-by: Chris Wing <cwing@nvidia.com>
Guard per-rollout Hermes homes and child cleanup under concurrent execution. Signed-off-by: Chris Wing <cwing@nvidia.com>
|
/claude review |
|
Review incomplete: the PR revision changed during review. This review was performed against head |
Translate upstream deferred-tool dispatcher messages into the executed actions expected by Responses trajectories while leaving raw observability intact. Signed-off-by: Chris Wing <cwing@nvidia.com>
|
could you validate training with tokenid capture for a couple steps? it seems calls are being marked incomplete maybe due to some harness generated calls that trigger mark_incomplete |
|
/claude review |
This fix might need to go into tokenid capture @ananthsub I am not certain it is the cause but it seems so: The main chain is retained correctly, but then a separate “capture incomplete” flag masks the whole rollout afterward.
|
|
SHIP WITH CARE Reviewed the full diff. This refactors the Hermes agent from an in-process One conditional correctness finding (inline, Other notes (author call, not blocking):
Verifier/scoring logic ( |
Preserve every new output item when resumed input history contains Hermes-internal dispatch messages that projection removes. Signed-off-by: Chris Wing <cwing@nvidia.com>
|
/claude review |
|
SHIP WITH CARE — no blockers found. Two changes, both well-tested; the risk is concentrated in blast radius, not defects. 1.
2. hermes_agent → managed subprocess Worth an explicit call-out for the training team, since it's a deliberate contract change, not a bug:
Tests are substantive (assert real projection/isolation/conversion behavior, not pass-throughs). No async-client or |
@cmunley1 it is, also filed at #2678 - auxiliary calls to endpoints not supported by the gym model server are treated as failures that mask the whole rollout. i will send out a fix |
|
We’re doing some cleanup of the open PR list. This PR has had no activity for over a month, so I’ve moved it back to draft. If this change is still needed, please resolve any merge conflicts and mark the PR ready for review again when it’s ready to merge. Thanks! |
Summary
Move Gym's Hermes agent from an old fork commit to a reproducible upstream release while preserving the rollout, verification, and observability contracts needed by Gym.
Details
v2026.8.19/0.20.5by release, commit, and installer checksum so the runtime is named, auditable, and reproducible.uv.lock, avoiding unsupported wheel/sdist installation.HERMES_HOME, disabled persistence/streaming, bounded concurrency, and child-process signal handling.ResponsesConverterfor request history and response translation instead of maintaining Hermes-specific trajectory conversion.tool_calldispatcher into the semantic action it executed, while keepingtool_search,tool_describe, failed pre-dispatch attempts, and the complete raw event stream out of the verifier-facing action list and available to observability.function_callitems for those steps, soturns_usedcontinues to mean assistant-message count while model-call count comes from observability.AgentObservationBundlerecords in the parent.ResponsesConverter, preserving ordered text and refusal parts instead of failing on text lists or silently dropping refusals.Closes #2672.
Testing
Automated checks
pytest responses_api_agents\/hermes_agent\/tests tests\/unit_tests\/test_responses_converter.py -q— 149 passed, including concurrent subprocess isolation, deferred-tool trajectory projection, text-list, list-refusal, and top-level-refusal conversion coverage.pytest responses_api_agents/opencode_sandboxed_agent/tests/test_app.py responses_api_models/litellm_model/tests/test_app.py tests/unit_tests/test_openai_utils.py -q— 149 passed, covering every non-Hermes direct consumer ofpostprocess_assistant_message_dictplus shared OpenAI schema compatibility.ruff checkandruff format --checkon all changed Python files — passed.pre-commit run --all-files— passed all repository hooks without modifying tracked files.Managed-runtime validation
0.20.5runtime on macOS, including frozen lock reconciliation.0.20.5at the pinned commit, validated the frozen-lock marker andAIAgentimport, and reused the same runtime on a secondensure_hermes()call.0.20.5child runtimes against a local OpenAI-compatible model stub. All three overlapped, used distinctHERMES_HOMEand working directories, returned valid results and observations, exited successfully, removed their temporary directories, leftactive_processesempty, and left no orphaned runner process. A deterministic regression test now enforces this contract without external credentials.Real rollouts and comparison
nvidia/nvidia/nemotron-3.5-lightning; it completed model calls, invoked the local terminal tool, converted the trajectory, verified the response, and returned Gym observation records.math_with_judgeexamples, model, and concurrency against originalmainand this branch. Both scored 3/5 and produced the same extracted answer and reward for every example. Original Hermes made 32 model calls and 27 tool calls (three failed); updated Hermes made 10 model calls and five successful tool calls. This is one small behavioral sample, not a performance benchmark.turns_usedaveraged 6.4 on original Hermes and 1.0 on the updated adapter because this repository metric counts assistant message items, not internal model or tool calls. The old custom conversion emitted synthetic empty assistant messages for tool-only steps; shared Responses conversion emits canonicalfunction_callitems without empty messages. Updated observability separately recorded all 10 model calls.tool_search,tool_describe, andtool_call, while the stored Responses trajectory retained only the successfully executed semantic action. That rollout and its stored-artifact reverification both scored1.0.Pending validation