Skip to content

design(environment): prototype environment authoring runtime - #3557

Closed
cwing-nvidia wants to merge 14 commits into
cwing/episode-orchestration-snapshot-f20a6f1from
feat/gym-hello-world-runtime
Closed

cwing-nvidia wants to merge 14 commits into
cwing/episode-orchestration-snapshot-f20a6f1from
feat/gym-hello-world-runtime

Conversation

@cwing-nvidia

@cwing-nvidia cwing-nvidia commented Sep 19, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Implements the runtime prototype for the environment-authoring design explored in #3469. Its purpose is to exercise the proposed contracts end to end and gather early feedback from environment authors and Gym maintainers before the public design and implementation are finalized.

The prototype makes hello-world, hello-taskset, and hello-verifier-reuse directly executable from a source checkout with gym eval run --environment .... Gym loads environment.yaml, materializes its tasks, prepares the declared local Docker runtime, composes the existing server architecture, runs the selected agent, and dispatches the selected verifier.

Scope and dependencies

This draft is based on the immutable cwing/episode-orchestration-snapshot-f20a6f1 branch, which pins validated commit f20a6f1 from ananthsub/episode-orchestration-prototype. It depends on that unmerged Environment Server stack but is isolated from subsequent force-pushes to Ananth's working branch. This is an integration and design-feedback PR, not a merge-ready public feature.

  • #3469 owns the proposed author-facing format, examples, and authoring feedback.
  • #3556 owns the Docker pipe-session capability required by borrowed-sandbox agents. This integration branch carries the same provider changes for end-to-end testing.
  • The branch follows the snapshot's simplified TaskId contract. Later upstream changes will be adopted explicitly rather than changing this PR's diff automatically.

No issue is linked because this is a focused integration prototype supporting the design proposal in #3469.

Motivation

The broader authoring motivation is described in #3469: environment authors should be able to define the work, its runtime requirements, and what counts as success without first learning Gym's internal inference and server wiring.

This companion PR focuses on the runtime consequence of that proposal. A simpler authoring format improves the user experience only if authors can run it directly. Requiring them to also prepare Responses API JSONL, write a custom Resources Server, or maintain internal server-composition configuration would move the complexity rather than remove it.

Design under review

User-facing execution path

gym eval run --environment ...:

  1. Finds an environment by local path or declared name.
  2. Loads and validates environment.yaml.
  3. Materializes singleton, JSONL-backed, or directory-based tasks internally.
  4. Builds or reuses the declared local Docker image.
  5. Resolves the default Environment Server, or an explicit environment_server override.
  6. Binds the caller-selected agent to the Environment Server's required role.
  7. Runs each episode in a fresh sandbox and invokes the selected verifier.

Authors using this path are not required to maintain prepared Responses API JSONL, an environment-local config.yaml, or a prepare.py script. Users who need exact control over already-materialized Responses API inputs can continue using Gym's existing dataset and split workflow. The caller selects the agent and model; this prototype currently fixes execution to a local Docker sandbox.

Tasksets use explicit name and path fields. File paths select JSONL-backed collections; directory paths discover
child directories containing task-local task.yaml files. A taskset named default is selected when --taskset is
omitted.

Internal runtime boundaries

The following modules and generated configuration are implementation details for Gym runtime maintainers. A typical environment author should only need to define environment.yaml and its referenced task, runtime, and verifier files, then use the CLI; they should not need to import, configure, or understand these components.

  • authoring.py loads definitions and materializes typed tasks.
  • environment_servers.py resolves the Environment Server and its required agent roles.
  • runtime_composition.py combines the selected Environment Server and sandbox runtimes into temporary rollout input and server configuration.
  • local_docker_image.py builds or reuses an OCI image; sandbox-provider selection remains independent.
  • The reusable environment_adapter Resources Server owns the standard sandbox, verification, and cleanup lifecycle.

Gym generates agent_bindings to connect the caller-selected agent to the roles required by the Environment Server. This field is not part of the environment-authoring format.

Advanced integrations can still provide a custom Resources Server for genuinely custom services, state, or lifecycle requirements. The design question is whether the reusable adapter hides the right runtime details for the standard authoring experience.

Prototype coverage

  • hello-world exercises one instruction and one task-local verifier.
  • hello-taskset exercises JSONL materialization, instruction rendering, Pydantic task-data validation, and a shared verifier.
  • hello-verifier-reuse exercises discovered directory tasks, task-local manifests, and per-task verifier selection.
  • hello-mcp-tool remains authoring-only until managed MCP startup, routing, isolation, and cleanup are implemented.

Prototype-only integration choices

The end-to-end path intentionally makes several choices that must be revisited before the runtime contract is stabilized:

  • Generated typed rollout_input bypasses the existing dataset and split preparation path. It proves that materialized environment tasks can reach rollout collection, but it does not decide whether both mechanisms should remain separate public contracts.
  • The CLI always builds with the local Docker daemon and selects the Docker sandbox provider with fixed TTL and readiness settings. Image preparation and provider selection are separated internally, but additional builders, providers, and user-facing selection are not implemented.
  • Only one --environment value and the default single_agent Environment Server are supported. Authors omit environment_server; explicit alternatives such as iterative proof refinement or framework adapters are not yet registered. The generated role-binding design has not been exercised with multi-agent Environment Servers.
  • The adapter sets trusted_environment_code=true, imports verifier code into the Resources Server process, and keeps session state in one worker's memory. The trust, isolation, persistence, and multi-worker contracts are unresolved.
  • The CLI and adapter independently load and materialize the same environment because no compiled environment artifact is handed between processes. A final design should avoid redundant work and guarantee that execution and verification consume one immutable definition.
  • Generated task input and server composition are run-scoped temporary files. There is no dedicated inspect, replay, or provenance contract for that generated composition.
  • Generated component resolution currently depends on the source-checkout layout. Distribution of built-in environments and discovery from installed packages are not implemented.
  • Agent and model compatibility is selected manually through profiles such as hermes_agent/borrowed_sandbox_openai_compatible; Gym does not yet negotiate capabilities.
  • Declared MCP services are rejected before launch rather than managed by the runtime.
  • The generic adapter has no fixed dataset, but its location under resources_servers/ causes Gym's legacy server validator to require five example inputs, rollouts, and metrics. Its tests pass; the validation command still exits nonzero after testing because this infrastructure adapter does not provide those benchmark artifacts. The prototype does not add fake artifacts solely to satisfy that mismatched contract.

Feedback requested

For environment authors

  • What internal Gym concepts, if any, did you still need to understand before you could run an environment.yaml environment?
  • Does gym eval run --environment ... make the environment, taskset, agent, model, runtime-image preparation, and current local-Docker requirement clear?
  • Does the standard lifecycle—one fresh sandbox per episode followed by environment-selected verification and cleanup—match what your environment needs?
  • Could this format and reusable adapter represent an environment you maintain today? Which requirement, if any, would force you to write a custom Resources Server?
  • If your environment requests an unsupported capability, such as managed MCP, does the error explain what is unsupported and what to do next?
  • After a run, can you tell which tasks ran, how each task was scored, and where to inspect failures and results?

For Gym runtime maintainers

  • Is the boundary between the reusable environment_adapter and a custom Resources Server correct?
  • Are task materialization, Environment Server resolution, runtime composition, and image building separated at the right boundaries?
  • Is internal rollout_input a reasonable prototype bridge for reaching rollout collection, and what should be reconciled with the existing dataset and split path before the runtime contract is stabilized?
  • Do Environment Server-declared agent roles and generated agent_bindings generalize beyond the default implementation?
  • Is separating OCI image preparation from sandbox-provider selection the right abstraction for additional providers?
  • What trust or isolation requirements must be resolved before environment-local verifier code becomes a public contract?

Validation

Runtime smoke evidence

  • Hello World: A one-command Hermes rollout discovered environment.yaml, materialized the task, generated server composition, prepared the local Docker image, bound Hermes to the Environment Server's agent role, and invoked the task-local verifier. The task returned reward 1.0 with no episode failure.
  • Hello Taskset: Both JSONL tasks ran with Hermes in fresh Docker workspaces and returned reward 1.0 with no episode failures.
  • Hello Verifier Reuse: Both discovered directory tasks ran with Hermes in fresh Docker workspaces after omitting
    --taskset. The complete reusable verifier and task-local uppercase verifier each returned reward 1.0 with no
    episode failures.
  • Docker process session: A real Docker smoke passed stdin, output, process-group signaling, exit, and cleanup.

Local checks

  • The focused authoring, adapter, CLI, rollout-input, global-config, local-image, Docker-provider, and example suite passes: 522 tests.
  • The isolated Hermes suite passes: 54 tests.
  • The isolated environment-adapter suite passes: 6 tests. gym env test +entrypoint=resources_servers/environment_adapter +should_validate_data=true then exits nonzero in legacy data validation because the generic adapter has no fixed five-row dataset artifacts, as described above.
  • Regression coverage includes invalid task data, unknown references, path escapes, untrusted code, Environment Server dispatch, task identity, agent-role binding, image caching and build failures, session lifecycle, and concurrency.
  • Focused Ruff lint and format checks and git diff --check pass.
  • pre-commit run --all-files has not been rerun after rebasing onto the rewritten prototype base.

Compatibility

This is an additive design prototype on an unmerged base branch. It does not migrate existing datasets or Resources Servers, and its contracts should not yet be treated as stable public APIs.

The prototype-only choices above, public contract documentation, production error handling, integration coverage, and full CI must be resolved before this path is presented as shipped functionality.

@cwing-nvidia cwing-nvidia added feature New capabilities, enhancements, or enablement work area:env-infra Shared environment framework, lifecycle, registry, scaffolding, and validation complexity:high Cross-component or shared-contract change with a large review surface labels Sep 19, 2026
@copy-pr-bot

copy-pr-bot Bot commented Sep 19, 2026

Copy link
Copy Markdown

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

@cwing-nvidia cwing-nvidia changed the title feat(environment): prototype environment authoring runtime design(environment): prototype environment authoring runtime Sep 19, 2026
@ananthsub
ananthsub force-pushed the ananthsub/episode-orchestration-prototype branch 12 times, most recently from 4fadeae to f20a6f1 Compare September 22, 2026 16:28
@cwing-nvidia
cwing-nvidia force-pushed the feat/gym-hello-world-runtime branch from d0d51d5 to 1571cd8 Compare September 22, 2026 16:29
Signed-off-by: Chris Wing <cwing@nvidia.com>
Signed-off-by: Chris Wing <cwing@nvidia.com>
Signed-off-by: Chris Wing <cwing@nvidia.com>
Signed-off-by: Chris Wing <cwing@nvidia.com>
Signed-off-by: Chris Wing <cwing@nvidia.com>
Signed-off-by: Chris Wing <cwing@nvidia.com>
Signed-off-by: Chris Wing <cwing@nvidia.com>
Signed-off-by: Chris Wing <cwing@nvidia.com>
Signed-off-by: Chris Wing <cwing@nvidia.com>
Signed-off-by: Chris Wing <cwing@nvidia.com>
Signed-off-by: Chris Wing <cwing@nvidia.com>
Signed-off-by: Chris Wing <cwing@nvidia.com>
Signed-off-by: Chris Wing <cwing@nvidia.com>
@cwing-nvidia
cwing-nvidia force-pushed the feat/gym-hello-world-runtime branch from 1571cd8 to 7376f17 Compare September 22, 2026 16:30
@ananthsub
ananthsub force-pushed the ananthsub/episode-orchestration-prototype branch 2 times, most recently from 4a79da8 to ff5875d Compare September 22, 2026 17:37
@cwing-nvidia
cwing-nvidia changed the base branch from ananthsub/episode-orchestration-prototype to cwing/episode-orchestration-snapshot-f20a6f1 September 22, 2026 18:16
Signed-off-by: Chris Wing <cwing@nvidia.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:env-infra Shared environment framework, lifecycle, registry, scaffolding, and validation complexity:high Cross-component or shared-contract change with a large review surface feature New capabilities, enhancements, or enablement work

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant