Repository navigation
Description
Summary
Evaluate moving responses_api_agents/hermes_agent from its current commit-pinned fork to an exact named upstream Hermes Agent release. Keep the dependency reproducible; this is not a proposal to install latest without validation.
Current state
Gym currently installs:
hermes-agent @ git+https://github.com/cmunley1/hermes-agent@26bb847a
That fork commit is dated May 7, 2026. The Hermes README says the fork carries patches for token-ID tracking, chat-template behavior, and sampling parameters needed for training. The in-process Gym adapter also depends directly on the fork's run_agent.AIAgent API.
Upstream Hermes has continued evolving, including changing native MCP tool names from the fork's mcp_<server>_<tool> form to the mcp__<server>__<tool> convention Gym already normalizes.
Why reconsider the fork
A named release would give the integration a clearer compatibility and upgrade contract than an aging commit on a personal fork. More importantly, Gym's newer model-server token-capture work may allow external agent harnesses to participate in training without maintaining every training-specific patch inside each harness fork.
Relevant work from @ananthsub includes:
- feat(vllm-model): pin sampling params for on-policy training #2190 — merged model-server sampling overrides for on-policy training
- feat(token-id-capture): deliver rebuilt trajectories safely #2126 — merged durable token capture and rebuilt trajectory delivery
- feat(token-id-capture): resolve each call's parent at request time #2180 — open request-time parent resolution for multi-call trajectories
- docs: training on rollouts from an external agent harness #2341 — open guide for training on rollouts from an external agent harness
- docs: update external harness token capture guide #2349 — open token-capture data-plane and prefix-supply documentation/tests
These changes do not automatically prove the Hermes fork is unnecessary. They make it worth re-evaluating which fork patches remain required and which responsibilities can now be provided generically by Gym's model server.
Investigation
- Inventory every behavior Gym relies on from
cmunley1/hermes-agent@26bb847a, including token IDs/logprobs, sampling forwarding, chat templates, reasoning/history preservation, and theAIAgentAPI. - Identify the first suitable named upstream Hermes release and compare it with the fork.
- Determine which training requirements are satisfied by Gym's token-capture and sampling infrastructure versus which still require harness changes.
- Check whether upgrading removes the Hermes-specific MCP alias incompatibility described in bug: MCP tool identity not preserved for verification #2668.
- Assess migration work required in
hermes_agent/app.py, tests, container dependency installation, and documentation.
Acceptance criteria
- The fork delta and every Gym dependency on it are documented.
- A specific named Hermes release is selected and pinned exactly, or concrete blockers to using a named release are documented.
- If migration is viable, the personal-fork commit dependency is removed.
- Existing Hermes unit tests pass, including token metadata and sampling behavior.
- Training compatibility is validated against the external-harness token-capture contract rather than assumed from unit tests.
- A real multi-turn Hermes rollout is inspected for trajectory fidelity, MCP naming, token capture, and reward behavior.
- The README accurately describes the final dependency and training contract.
Activity
cwing-nvidia commentedon Aug 25, 2026
Latest Hermes agent
v0.20.5explicitly refuses wheel/sdist installation and is supported only through its managed CLI installer, Docker, or Nix. Gym’s current Hermes adapter imports AIAgent in-process, so v0.20.5 is not a simple dependency bump