Skip to content

feat(sandbox): support Docker pipe process sessions - #3556

Closed
cwing-nvidia wants to merge 3 commits into
mainfrom
feat/docker-pipe-sessions
Closed

cwing-nvidia wants to merge 3 commits into
mainfrom
feat/docker-pipe-sessions

Conversation

@cwing-nvidia

@cwing-nvidia cwing-nvidia commented Sep 19, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • Add pipe-mode process sessions to the local Docker sandbox provider through docker exec -i.
  • Stream stdin, stdout, and stderr through the existing process-session interface.
  • Preserve the capability boundary: pipe sessions do not claim terminal emulation, resize behavior, or interactive TTY semantics.

Why

Borrowed-sandbox agents need a long-lived bidirectional process inside a container, while the Docker provider previously exposed only one-shot commands. This adds the provider-neutral process-stream capability those agents require without conflating it with PTY support.

No issue is linked because this is a focused provider capability extracted from the environment-runtime design prototyping work.

Behavior and lifecycle

The implementation uses a startup handshake, process-group signaling, control-file cleanup, cancellation-safe close behavior, and a dedicated concurrency limit for long-lived sessions.

Detached sessions and pty=true remain unsupported.

Compatibility

This is additive. Existing Docker create, one-shot exec, copy, connect, and close behavior is unchanged. Process sessions remain local to the Docker client and container, consistent with the provider's existing local-daemon scope.

Feedback requested

For borrowed-sandbox agent authors

  • Does a bidirectional pipe session provide the process behavior your agent needs without PTY emulation?
  • When a process fails to start, exits unexpectedly, or is closed, are the reported errors actionable?
  • Are the unsupported PTY and detached-session capabilities explicit enough for your integration to fail or fall back safely?

For sandbox-provider maintainers

  • Does the startup handshake remove races without coupling the provider to a particular agent?
  • Are process-group signaling and cleanup semantics consistent with the provider-neutral process-session contract?
  • Is a dedicated concurrency limit appropriate for long-lived sessions, separate from one-shot Docker commands?
  • Are cancellation, failed startup, normal exit, and forced close covered at the correct lifecycle boundary?
  • Which timeout and concurrency settings should remain provider defaults, and which need user configuration?

Validation

  • Focused Docker provider tests cover session startup, streaming, signaling, cleanup, cancellation, failures, and concurrency.
  • A real Docker smoke streams input and output, signals the process group, observes exit, and confirms cleanup.
  • The environment integration in #3557 uses this capability for fresh Hermes rollouts across all five executable Hello tasks; each returned reward 1.0 with no episode failure.

Signed-off-by: Chris Wing <cwing@nvidia.com>
@cwing-nvidia cwing-nvidia added feature New capabilities, enhancements, or enablement work area:sandbox Sandbox providers, isolation, and sandbox runtime contracts complexity:medium Single-domain change with interacting parts or a moderate review surface labels Sep 19, 2026
@copy-pr-bot

copy-pr-bot Bot commented Sep 19, 2026

Copy link
Copy Markdown

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

Signed-off-by: Chris Wing <cwing@nvidia.com>
@cwing-nvidia

Copy link
Copy Markdown
Contributor Author

/claude review

@claude

claude Bot commented Sep 19, 2026

Copy link
Copy Markdown
Contributor

SHIP — clean, well-tested addition of pipe-mode process sessions to the Docker provider. No correctness or async-hang concerns.

Reviewed against the areas that page an on-call engineer:

  • Session-semaphore lifecycle is correct. _session_semaphore is acquired once in create_pty and released exactly once: by the except BaseException handler when _create_pipe_session raises before constructing the session, or by _DockerProcessSession._cleanup_after_exit's finally once the process exits. No path both constructs the session and releases in create_pty, so no double-release (which would silently over-admit sessions past session_concurrency) and no leak.
  • No self._semaphore deadlock. The exec-concurrency semaphore wraps only create_subprocess_exec; the long-lived session process does not hold it, and the polling/cleanup _run calls acquire it independently. Safe even at concurrency=1.
  • PGID signaling is race-free. setsid --wait makes the inner shell a process-group leader (\$\$ == PGID); the script writes the pid file before the ready file and the probe checks test -e ready last, so a visible pgid is always populated. int(out) > 1 guards against pid 1. Signals target -\$pgid (the group) via shlex.quote.
  • Cleanup covers the failure modes. Startup timeout, early process exit, and close()-before-exit all kill the process and run rm -rf on the per-session control dir; close() drives the process to exit then reaps through the shared cleanup task.
  • Tests assert real behavior — argv construction, in-container signal delivery, startup-failure propagation, the dedicated concurrency gate, tty rejection, and the two new config validations. Not pass-through.

NOTE (author's call, not blocking): _DockerProcessSession.read()/read_stderr() return b\"\" on pipe EOF, but the SandboxPtySession protocol docstring says read() should raise SandboxPtyError "if the session died without exiting." In pipe mode an abnormal death is indistinguishable from a clean EOF at the stream level, so consumers must rely on wait_exit()'s return code to tell the difference. Reasonable given the transport; worth a one-line note in the class docstring so the contract gap is explicit.

Signed-off-by: Chris Wing <cwing@nvidia.com>
@cwing-nvidia
cwing-nvidia marked this pull request as ready for review September 19, 2026 20:24
@cwing-nvidia
cwing-nvidia requested a review from a team as a code owner September 19, 2026 20:24
@nemo-automation-bot
nemo-automation-bot Bot requested a review from a team September 19, 2026 20:24
@yaoyu-33 yaoyu-33 added the needs-review PR is ready for code review and waiting on a reviewer label Sep 19, 2026
@github-actions github-actions Bot added the sla:triage-overdue Review assignment is over the one-business-day SLA label Sep 21, 2026
@yaoyu-33
yaoyu-33 marked this pull request as draft November 18, 2026 01:54
@yaoyu-33

Copy link
Copy Markdown
Contributor

We’re doing some cleanup of the open PR list. This PR has had no activity for over a month, so I’ve moved it back to draft.

If this change is still needed, please resolve any merge conflicts and mark the PR ready for review again when it’s ready to merge. Thanks!

@cwing-nvidia

Copy link
Copy Markdown
Contributor Author

Closing as superseded by #4862, which added create_pty to the Docker provider. It covers pipe sessions (pty=False) as well as real PTY sessions, so this change is no longer needed.

@cwing-nvidia
cwing-nvidia deleted the feat/docker-pipe-sessions branch November 18, 2026 02:03

This branch was previously deployed

1 inactive deployment
public — 24e8d7c2 Deployed Sep 19, 2026 by copy-pr-bot[bot] via release / finalize / notify #3274
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:sandbox Sandbox providers, isolation, and sandbox runtime contracts complexity:medium Single-domain change with interacting parts or a moderate review surface feature New capabilities, enhancements, or enablement work needs-review PR is ready for code review and waiting on a reviewer sla:triage-overdue Review assignment is over the one-business-day SLA

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants