Skip to content

feat(checkpoint): add the resources server participant with asynchronous session hooks - #3886

Merged
ananthsub merged 1 commit into
mainfrom
ananthsub/partial-ckpt-resources
Oct 15, 2026
Merged

ananthsub merged 1 commit into
mainfrom
ananthsub/partial-ckpt-resources

Conversation

@ananthsub

@ananthsub ananthsub commented Oct 1, 2026 •

Copy link
Copy Markdown
Contributor

What changed and why

Every resources server takes part in checkpoints once checkpointing is on, in one of three modes, declared with the checkpoint_mode class attribute:

  • restart_only (the default): nothing of a session is saved, so its rollout is a restart. It never blocks prepare, keeps running through the checkpoint, and is reported in prepare's restarts; the seed reply carries x-ng-checkpoint-restart so the environment server knows. After a crash the controller retires the rollout and starts it over from its input. Only a new session start waits while a checkpoint is open, so the restarts a prepare reported cannot change before its commit. A seed already in flight when admission closes is reported as a restart too, without waiting for it, because its reply will say restart; otherwise it would register during the checkpoint and the commit would fail with restart_in_scope. A waiting seed checks again each time it wakes, so a new checkpoint that closes admission before it runs keeps it waiting.
  • stateless: tool results depend only on the request, so nothing is exported. The weather example server uses it.
  • exported: the server implements the session hooks. The counter example server (example_session_state_mgmt) uses it. It exports only the sessions it still holds, so a dropped session is left out and the participant stops tracking it, instead of being exported as a new counter at zero.

The session hooks are coroutines, and export takes every session at once: export_session_states(session_ids), restore_session_states(states), and retire_session_state(session_id). A server whose session state lives outside the process, such as a sandbox per session, can snapshot all of it concurrently within the commit. A session missing from the export result is treated as already dropped (for example, after a failed verification) and is no longer tracked, instead of failing the commit. With checkpoint_mode = "exported", a server that leaves any of these hooks as the base stub fails at startup; hooks from an intermediate base class or mixin count. retire_session_state is called in every mode, and its base version does nothing, so a restart_only or stateless server that keeps per-session state can free it on retire by overriding it.

checkpoint_mode and checkpoint_verify are checked against their allowed values at startup, whether or not checkpointing is on. A typo such as "exportd" raises a ValueError naming the attribute, the value, and the allowed values, instead of acting as a fourth mode that exports nothing and reports no restarts.

The session lifecycle comes from the server's session-start routes, /close_session, and /verify, keyed by the rollout attempt, so no request body is inspected.

  • Declared session starts. A server lists its session-start routes in checkpoint_session_starts, which defaults to ("/seed_session",); a Gymnasium-style server declares /reset. The checkpoint middleware treats every declared route as it treats /seed_session, before the handler runs: it waits out an open checkpoint in every mode, restart_only included, counts as a seed from admission, carries the seed reply headers, and binds the session when it succeeds. There is no other way to start a tracked session, so no start can slip in after a commit has fixed its scope. Sessions end through /close_session, or checkpoint_session_ended for a server that ends them elsewhere.
  • A session start needs a rollout ID. With checkpointing on, a session start without the /ng-rollout/<id> prefix is refused with 400 rollout_id_required before its handler runs, so no session can exist that the participant does not track.
  • A resources server that bypasses the shared setup fails closed. SimpleServer.setup_checkpoint_participant(app) runs for every server after setup_webserver. With checkpointing on, a resources server whose app did not install the checkpoint participant refuses to start, naming its class, instead of being silently left out of every checkpoint. Gymnasium servers on main build their own app, so they refuse to start with checkpointing on until fix(gymnasium): build Gymnasium servers on the shared resources server setup #3895 moves them onto the shared setup; with checkpointing off nothing changes.
  • A session belongs to its attempt. Any request on an existing session whose rollout prefix names an episode other than the session's is refused with stale_attempt, seeds excepted, so after a restore a replaced attempt's /close_session or /verify cannot end the replacement's session. A close from the owning attempt, including a retired attempt freeing its own session, still goes through. With checkpointing on, /close_session and /verify need the /ng-rollout/<id> prefix, as session starts do.
  • A restored session needs its attempt named. A request on a session a restore installed that names no attempt, with no rollout prefix and no MCP token, is refused with 400 rollout_id_required. The cookie alone cannot tell the replacement from the attempt the restore replaced. The refusal comes before the participant takes the session out of the restored sessions no request has used yet, so a refused call never claims one. Sessions seeded in the running process are not affected, and neither are restart-only servers, which never restore a session.
  • Marking an attempt retired fences its sessions at once. A retire or a restore that marks an attempt refuses the attempt's existing sessions before anything awaits, so a retire cut short by its deadline, or one that arrives after it and only marks, still leaves them refused, holding up readiness, and out of every export until a retried retire frees them. forget is refused while any tracked session of the rollout is at or below the retired mark.
  • MCP tokens name their attempt. With checkpointing on, the session token minted at seed carries an episode claim with the seeding attempt's capture key; with checkpointing off the token is unchanged. A call whose token names a fenced attempt, an attempt other than the session's, or no attempt gets 409 stale_attempt before its tool runs, and never claims a restored session. A restore therefore no longer lets the replaced attempt's harness keep using its token: the replacement gets a token by seeding under its own attempt with the session's cookie. ResourcesParticipant.is_open(session_id) reports whether a session is tracked and not fenced, for revocation at session close.
  • Calls on closed or unknown sessions are out of scope. A tool call that creates or changes state on a session that is closed or was never seeded is a session-lifecycle bug with checkpointing off as well, so it will be fixed on main for MCP and direct HTTP alike.
  • Restarts reported by earlier prepare stages carry forward. Coordination prepares stages in order (environment, model, agent, resources) and sends each later stage the restarts every earlier stage reported, as capture keys in a new optional restarts field of the prepare request. The commit leaves those episodes out, so a participant keeps them in carried_restarts until the checkpoint resumes, aborts or expires, and leaves them out of its blockers and its export. The resources participant no longer waits for requests in flight on their sessions, so an episode the environment server reported as a restart, for example from an MCP grant, keeps running through the checkpoint. Coordination sends the field only when an earlier stage reported a restart, and a prepare without it behaves as before.
  • Commit cleanup after a restore costs as much as each rollout's own sessions, through the rollout index, instead of scanning every restored session per episode.

The checkpoint_verify class attribute declares whether /verify may run again after a crash (replay) or must finish before a checkpoint (wait, the default). The server reports it in the x-ng-checkpoint-verify header on every /seed_session reply and in its checkpoint status. A replayable /verify is admitted even after admission closes, because the checkpoint does not wait for the episode step that sends it. A /seed_session that arrives while a checkpoint is open waits for resume instead, and a seed already in flight holds up prepare and is exported. If the seed handler replaces the session ID, the session is tracked under the new one. The participant applies a seed, close, or verify to its session tracking as soon as the response starts, before the rest of the response is sent. A client that disconnects after the handler ran cancels the request, as an environment server's retire does to its calls in flight, and that cancellation can no longer leave a created session untracked or an ended one tracked. The environment server seeds inside a replay step that can finish while the checkpoint is open, so the session must match whichever boundary the environment server records.

Tools called over MCP (expose_tools_over_mcp: true). A CLI agent harness calls tools over MCP with a signed session token and no session cookie. Before this change, such a server failed at startup with checkpointing on: MCP auto-exposure refuses middleware that direct MCP dispatch would skip, and the checkpoint middleware was one of them.

  • The checkpoint middleware works on the whole MCP request, so MCP auto-exposure accepts it. Admission and in-flight counting apply to the one POST that carries each tool call.
  • The middleware reads the session from the MCP session token, so a retire drains and refuses MCP calls like any other request on the session. The token key is deterministic, and the token's episode claim decides whether its attempt may still use it after a restore.
  • A request made while a checkpoint is open, over HTTP or MCP, waits for resume instead of getting a 409. It comes from a restart, which keeps running through a checkpoint, often a third-party harness that would pass the refusal to the model as a tool error. A waiting request is not in flight, so it does not hold up prepare. A replayable /verify is still admitted at once.
  • Checkpoint control routes are no longer harvested as MCP tools.

Ending sessions.

  • /close_session is never refused. Closing is how a server frees a session's state. A close that arrives while a checkpoint is open, from an episode that ended during it, waits for resume as an MCP call does; a refusal would leave the session unfreed, since the environment server closes once.
  • A retire refuses the attempt's sessions, waits until no request is in flight on them (over HTTP or MCP), then frees them through retire_session_state, in every mode. A session stays tracked until its state is freed: a retire cut short by its deadline leaves it for a retry, and meanwhile it blocks the next checkpoint and is not exported. retire_session_state can therefore run again for the same session, and must tolerate state it already freed. The refusal lasts until the controller forgets the rollout:
    • a request that carries the rollout prefix is checked against the retired attempts, so a late /seed_session cannot create state nothing frees;
    • a tool call over MCP names only its session, so the retired session IDs are kept by rollout until forget, and counted in status as retired_sessions.
  • Every mode, stateless and restart_only included, records which attempt each session belongs to, so a retire drains and refuses their sessions too. Export and readiness still depend on the mode.
  • A retire waits for any /seed_session of the same rollout, up to the retired attempt, that is already admitted, then frees the session that seed created. A seed is counted at its admission, with no await in between, so a seed still waiting for resume cannot hold up the retire, and the retire's mark refuses it once it is admitted. Retire visits only the rollout's own sessions, through an index from rollout to sessions.
  • A seed can finish after its attempt was marked retired, for example when a retire's deadline passed while it waited for that seed. The session the seed created is then registered as retired at once instead of live: requests on it are refused, including MCP calls that carry only the session token, it is never exported, and it blocks checkpoints until a retried retire frees it.
  • forget is refused with retire_incomplete, changing nothing, while a retired session it targets is still tracked, or while a seed of a retired attempt of that rollout is still in flight; the controller retires again first. Forgetting during such a seed would clear the mark before the seed finished, and its session would then be bound as live.
  • A replayable /verify whose caller closes the session afterwards does not end it. The caller says so with the x-ng-checkpoint-close-follows: 1 header, and the session stays tracked and exported until the /close_session that follows, a declared end, or a retire. A checkpoint can keep the environment server's boundary from before the verification even when the verification finishes before the resources server prepares, so the session must still be exported, and a restore runs the verification again against the restored state. single_agent_turn, the UserSim environment server, and the sandboxed Hermes agent send the header, since each closes the session in its final cleanup; Simple Agent's legacy /run sends it in feat(checkpoint): add the agent session participant and Simple Agent continuation #3888. A /verify without the header, and a wait-mode /verify, end the session at once, so a caller that never closes cannot leave it tracked.
  • Unclaimed restored sessions, ones no request has used yet, are reported as pending, so a commit whose scope no longer continues their episode does not export them. It deletes them through retire_session_state without refusing anything later, since the controller may still start the rollout over as that attempt. A restored session that the server drops before any request uses it does not break the commit.
  • A commit exports only the live sessions whose episode is inside its scope, as it already did for restored sessions. The controller leaves every participant's restarts out of the scope, so a session another server reports as a restart is not exported here either.
  • The participant answers the readiness wait's ready() without building its blocker list, and admission refusals are the shared AdmissionClosedError. The MCP session token helpers live in base_resources_server.py and are shared with MCP auto-exposure.

How it works

Where this PR sits in the overall flow

The highlighted part is what this PR adds.

flowchart LR
  C["Controller<br/>NeMo RL, or rollout collection"]
  CO["Coordination<br/>prepare, commit, restore, resume,<br/>retire, forget"]
  K["Control plane on every server<br/>phases, lease, storage,<br/>retire: refuse, stop, free;<br/>forget"]
  subgraph G["One participant per Gym server"]
    E["Environment server<br/>episode steps"]
    M["Policy model<br/>held responses, generation cuts"]
    A["Agent<br/>sessions parked at boundaries"]
    R["Resources server<br/>session state"]
  end
  W["Inference worker<br/>stages cut prefixes"]
  D[("Checkpoint directory<br/>records, then manifest")]
  L[("Capture ledger<br/>retire and delete from #3938, #3939")]
  C --> CO --> K
  K --> E & M & A & R
  M --> W
  M --> L
  G --> D
  classDef this fill:#fde68a,stroke:#b45309,stroke-width:2px,color:#1f2937
  class R this
Loading

One checkpoint, a crash, and the restore, end to end:

sequenceDiagram
  participant C as Controller
  participant G as Gym participants
  participant D as Checkpoint directory
  C->>G: prepare, in order environment, model, agent, resources
  Note over G: admission closes, in-flight work parks at a boundary,<br/>undelivered model responses are held
  G-->>C: prepared, or blockers at the deadline
  C->>G: commit with the episodes the controller continues
  G->>D: each participant writes its records, then its manifest
  Note over G: restored state the commit's scope leaves out is deleted
  C->>C: publish the checkpoint with the controller's own state
  C->>G: resume, in order resources, agent, model, environment
  Note over C,G: crash - every Gym process dies
  C->>G: restore the checkpoint in fresh processes, all or nothing
  D-->>G: records installed under attempt N+1, attempt N's capture ledger retired
  C->>G: resume
  C->>G: /run as attempt N+1 continues each episode from its boundary
  Note over C,G: retiring an episode, only while no checkpoint is open
  C->>G: retire - environment, then agent, then model and resources
  Note over G: each server marks the attempt, stops its work, waits, frees its state, then replies
  C->>G: forget, once the rollout is done for good - same order
  Note over G: the mark refusing the retired attempt goes away
Loading

This PR

How a resources server takes part, chosen by a class attribute.

flowchart TD
  M{"checkpoint_mode"} -->|stateless| S["Nothing to export.<br/>Never blocks a checkpoint."]
  M -->|exported| X["export_session_states at commit<br/>restore_session_states after a crash<br/>retire_session_state on retire"]
  M -->|"restart_only, the default"| O["Its rollout is a restart: reported in prepare,<br/>never blocks, starts over after a crash"]
Loading

A session's lifecycle, as the participant learns it from the routes that already define it.

sequenceDiagram
  participant E as Environment server or agent
  participant R as Resources server
  E->>R: /seed_session for rollout r
  R-->>E: session cookie, plus x-ng-checkpoint-verify - wait or replay
  E->>R: tool calls - wait while a checkpoint is open
  E->>R: /verify, with x-ng-checkpoint-close-follows if replayable
  E->>R: /close_session from final cleanup, which waits for resume if a checkpoint is open
Loading

Retire drains a session before freeing it.

sequenceDiagram
  participant C as Controller
  participant R as Resources server
  C->>R: retire(r, N)
  Note over R: refuse the attempt's sessions, except /close_session
  Note over R: wait until no request is in flight on them, over HTTP or MCP
  Note over R: retire_session_state for exported servers, stop tracking
  R-->>C: stopped and freed
Loading

Where this sits in the stack

This is one PR in a stack of draft PRs that re-cut partial-rollout checkpointing onto environment servers. Each PR is one commit, and its base is the branch of the PR before it, so each diff shows only that PR's change.

The stack builds on the token-capture cleanup PRs #3938 (capture ledger retire and delete) and #3939 (complete-record retire and delete), which have merged. #3882 to #3885 have merged too, so #3886's base is main. The checkpoint stack uses those operations to free the ledgers of retired attempts and to clear a dead execution's capture files before a restore. The capture-ledger cache bound (#4268) and the striped lock files (#3937) are independent of the stack.

  1. feat(checkpoint): add the participant control plane, episode steps, and coordination #3882 (ananthsub/partial-ckpt-core): feat(checkpoint): add the participant control plane, episode steps, and coordination (merged)
  2. feat(checkpoint): make policy model servers checkpoint participants with generation cuts #3883 (ananthsub/partial-ckpt-policy-model): feat(checkpoint): make policy model servers checkpoint participants with generation cuts (merged)
  3. feat(token-capture): add worker staging helpers for generation cuts #3884 (ananthsub/partial-ckpt-model-worker-cuts): feat(token-capture): add worker staging helpers for generation cuts (merged)
  4. feat(checkpoint): continue environment server episodes from their boundaries #3885 (ananthsub/partial-ckpt-environment): feat(checkpoint): continue environment server episodes from their boundaries (merged)
  5. feat(checkpoint): add the resources server participant with asynchronous session hooks #3886 (ananthsub/partial-ckpt-resources): feat(checkpoint): add the resources server participant with asynchronous session hooks (this PR)
  6. feat(checkpoint): declare twenty training verifiers stateless with replayable verification #3887 (ananthsub/partial-ckpt-verifier-declarations): feat(checkpoint): declare twenty training verifiers stateless with replayable verification
  7. feat(checkpoint): add the agent session participant and Simple Agent continuation #3888 (ananthsub/partial-ckpt-agent): feat(checkpoint): add the agent session participant and Simple Agent continuation
  8. test(checkpoint): add a process-level end-to-end suite driven by coordination #3889 (ananthsub/partial-ckpt-e2e): test(checkpoint): add a process-level end-to-end suite driven by coordination
  9. feat(checkpoint): checkpoint evaluation runs from rollout collection #3893 (ananthsub/partial-ckpt-rollout-collection): feat(checkpoint): checkpoint evaluation runs from rollout collection
  10. feat(checkpoint): spans and metrics for partial-rollout checkpoints #3903 (ananthsub/partial-ckpt-telemetry): feat(checkpoint): spans and metrics for partial-rollout checkpoints
  11. feat(checkpoint): resources, agent, and environment servers with several workers #3909 (ananthsub/partial-ckpt-multi-worker): feat(checkpoint): resources, agent, and environment servers with several workers
  12. test(checkpoint): check checkpoint invariants over random sequences of operations #4197 (ananthsub/partial-ckpt-fault-tests): test(checkpoint): check checkpoint invariants over random sequences of operations

Partial-rollout checkpointing lets a training controller, such as NeMo RL, checkpoint Gym while rollouts are in flight and, after a crash, continue those rollouts from their last safe point instead of starting them over. Checkpointing is off by default; with the checkpoint: block unset, no server installs any checkpoint routes or behavior.

Relationship to the old stack (#2939 to #2946)

Issue

No tracking issue exists for this re-cut. The design and the mapping from the old stack are described in this PR series, and the old stack's PRs (#2939 to #2946) carry the original discussion.

Validation

Run on this branch, on the current main:

  • test_each_prepare_stage_carries_the_restarts_every_earlier_stage_reported and test_a_request_in_flight_for_a_restart_an_earlier_stage_reported_does_not_hold_up_prepare, each failing without its change: later stages get the earlier stages' restarts, the restarts are cleared at resume, and a resources request in flight for such an episode no longer holds up prepare.
  • This round's tests, each failing without its change: test_a_retire_cut_short_while_a_seed_is_in_flight_still_fences_the_attempt_s_existing_session, test_a_retire_that_arrives_after_its_deadline_still_fences_the_attempt_s_sessions, test_after_a_restore_a_close_from_the_replaced_attempt_is_refused_and_keeps_the_session, test_a_request_on_a_session_from_another_rollout_is_refused, test_a_session_end_without_a_rollout_prefix_is_refused_before_its_handler_runs, test_deleting_one_rollout_s_restored_sessions_does_not_scan_every_restored_session, test_a_restore_refuses_the_replaced_attempt_s_token_and_a_seed_of_the_replacement_gets_one_that_works, test_a_stale_harness_cannot_create_state_for_a_session_the_fresh_process_never_saw, test_a_token_without_an_attempt_is_refused_when_checkpointing_is_on, test_a_second_restore_refuses_the_first_replacement_s_token, test_a_refused_stale_call_does_not_claim_a_restored_session, test_a_token_minted_with_checkpointing_names_the_attempt_that_seeded_it, and the is_open tests; test_a_token_minted_without_checkpointing_names_only_its_session_and_tools guards the unchanged token with checkpointing off.
  • test_a_declared_session_start_during_a_commit_s_export_waits_for_resume, in all three modes: a declared /reset arriving while the commit is in export waits for resume, and the commit has no record for it (without the fix, the restart_only reset finished during the checkpoint); test_a_restart_only_declared_session_start_in_flight_is_reported_as_a_restart (without the fix, prepare reported no restart).
  • test_a_session_start_without_a_rollout_prefix_is_refused_before_its_handler_runs, for /seed_session and /reset (without the fix, a 200 that created untracked state).
  • test_a_resources_server_whose_app_left_out_the_checkpoint_participant_refuses_to_start and test_a_resources_server_whose_app_installs_the_participant_passes_the_startup_check, including a server with its own app starting with checkpointing off.
  • RAY_TMPDIR=/tmp .venv/bin/python -m pytest -q -p no:cacheprovider tests/unit_tests/test_checkpoint_*.py: passed, including, each failing without its change:
    • a /close_session during a checkpoint waits for resume, then ends the session;
    • a retire waits for the session's request in flight, and refuses a new one meanwhile;
    • a commit whose scope leaves out a restored session retires it through the server's hook;
    • a session whose retire was cut short in the server's hook stays until a retry frees it, and blocks prepare meanwhile;
    • a seed arriving during an open checkpoint waits for resume, and a seed in flight when admission closes holds up prepare and is exported;
    • a retire waits for an admitted seed of its attempt and frees the session it creates, and a seed still waiting out a restore does not hold up the retire;
    • forget is refused while a retire has not freed its sessions;
    • a replayable /verify whose caller closes afterwards keeps the session tracked and exported until the close, including one that finishes between environment and resources prepare, after which a restore verifies again and closes it;
    • a replayable /verify without the close-follows header, and a wait-mode /verify with or without it, end the session at once;
    • after a restore, a call under the replaced attempt's prefix is refused with stale_attempt and a cookie-only call with rollout_id_required, across two restores, and neither changes the restored state or claims the session;
    • a stateless server drains and refuses a retired session;
    • deleting an unclaimed restored session refuses nothing later, and a restored session the server drops does not break the commit;
    • a request cancelled while notifying is no longer counted;
    • a session whose ID the seed replaces is tracked under its new ID;
    • an exported server without its hooks fails at startup;
    • retiring one rollout does not scan every session, and ready() does not list blockers;
    • in restart_only mode, a seed in flight when admission closes is reported as a restart by prepare, its reply says restart, and the commit of the remaining scope succeeds;
    • in restart_only mode, a seed waiting for resume waits again when a new checkpoint closes admission before it runs;
    • a seed, close, or verify whose reply has started still updates the session tracking when the request is cancelled before the reply finishes: the seeded session is tracked, and the closed or verified one is not;
    • an unknown checkpoint_mode or checkpoint_verify fails at startup;
    • a retire frees the sessions of restart_only and stateless servers through retire_session_state;
    • a commit does not export live sessions outside its scope;
    • a seed that finishes after its retire was cut short while waiting for it leaves a retired session: a request with only the session cookie is refused with stale_attempt, a checkpoint does not export it, forget is refused with retire_incomplete, and a retried retire frees it, after which forget succeeds;
    • forget is refused with retire_incomplete while a seed of a retired attempt is in flight, and the session that seed creates is refused afterwards.
  • RAY_TMPDIR=/tmp .venv/bin/python -m pytest -q -p no:cacheprovider tests/unit_tests/test_mcp_auto_exposure.py tests/unit_tests/test_checkpoint_resources_mcp.py: passed. The MCP checkpoint tests go through the real control routes and MCP JSON-RPC:
    • a tool call during a checkpoint waits and runs after resume, and the checkpoint holds the state from before it;
    • a tool call in flight holds up prepare;
    • a retire waits for a tool call in flight over MCP;
    • a restored server refuses the replaced attempt's token, and a seed under the replacement gets one that continues the session;
    • control routes are not tools.
  • RAY_TMPDIR=/tmp .venv/bin/python -m pytest -q -p no:cacheprovider resources_servers/example_session_state_mgmt/tests: passed, including a new test that the counter example leaves a session it no longer holds out of its export.
  • RAY_TMPDIR=/tmp .venv/bin/python -m pytest -q -p no:cacheprovider resources_servers/example_single_tool_call/tests: passed.
  • environment_servers/single_agent_turn/tests, environment_servers/nemo_user_sim/tests (in its own environment, with the UserSim package) and responses_api_agents/hermes_sandboxed_agent/tests: passed, including that /verify says the session is closed afterwards.
  • The Codex, OpenClaw, OpenCode, and Pi agent tests, each in its own environment: passed, except one Pi test that needs a newer Node.js than this machine has and fails the same way on main.
  • The full core unit suite (tests/unit_tests, eight processes, without test_opensandbox_compose.py, which needs the optional opensandbox package): passed except a sandbox retry test, which fails the same way on main in this development environment.
  • pre-commit run --files <files changed by this PR>: all hooks passed, and no hook modified a file.
  • Every commit carries a Signed-off-by line.

Rollout evidence

  • Process-level evidence is in the top PR: restored counter state is required for the right reward, with a negative control that skips the resources restore and gets reward 0.0.
  • Real model: see the rollout evidence in test(checkpoint): add a process-level end-to-end suite driven by coordination #3889, whose opt-in tests drove this PR's code with Qwen/Qwen3-4B-Instruct-2507 on vLLM: saved rollouts continued after a kill and restore, restarts started over, and every rollout finished with reward 1.0. Its restart test uses a restart_only resources server.

Compatibility

  • New resources server hooks and class attributes: checkpoint_mode, checkpoint_verify, export_session_states, restore_session_states, retire_session_state, checkpoint_session_starts, checkpoint_session_ended, and the setup_checkpoint_participant startup check. All are new; existing servers need no change and default to restart_only with wait verification.
  • An unknown checkpoint_mode or checkpoint_verify value raises a ValueError at startup, even with checkpointing off.
  • With checkpointing on, a retire calls retire_session_state in every mode, not only exported, and the base hook does nothing. A restart_only or stateless server that already overrides it must expect the call.
  • The prepare control request has a new optional restarts field. A participant from an older Gym build would refuse a prepare that carries it, which coordination sends only when an earlier stage reported a restart; within one deployment every server runs the same build.
  • A commit exports only the live sessions inside its episode_ids scope. A commit request without episode_ids still exports every live session.
  • /seed_session replies carry a new x-ng-checkpoint-verify header when checkpointing is on.
  • With checkpointing on, a caller that runs a replayable /verify and then closes the session should send x-ng-checkpoint-close-follows: 1 on /verify; without it the session ends at the verification, as before.
  • With checkpointing on, a request on a restored session must carry its attempt's rollout prefix or MCP token; a cookie-only request is refused with rollout_id_required.
  • Nothing changes unless the global checkpoint: block sets enabled: true.

@ananthsub ananthsub added feature New capabilities, enhancements, or enablement work area:env-infra Shared environment framework, lifecycle, registry, scaffolding, and validation labels Oct 1, 2026
@copy-pr-bot

copy-pr-bot Bot commented Oct 1, 2026

Copy link
Copy Markdown

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

@ananthsub
ananthsub requested a review from zyzhou5 October 1, 2026 19:04
@ananthsub
ananthsub force-pushed the ananthsub/partial-ckpt-resources branch from aa41669 to defe5d3 Compare October 1, 2026 19:38
@ananthsub
ananthsub force-pushed the ananthsub/partial-ckpt-resources branch from defe5d3 to f481624 Compare October 1, 2026 21:53
@ananthsub
ananthsub force-pushed the ananthsub/partial-ckpt-resources branch from f481624 to 4e9f81d Compare October 1, 2026 22:05
@ananthsub
ananthsub force-pushed the ananthsub/partial-ckpt-resources branch from 4e9f81d to b72fe50 Compare October 2, 2026 13:22
@ananthsub
ananthsub force-pushed the ananthsub/partial-ckpt-resources branch from b72fe50 to 7f1cfb6 Compare October 2, 2026 19:43
@ananthsub
ananthsub force-pushed the ananthsub/partial-ckpt-resources branch from 7f1cfb6 to 412e854 Compare October 2, 2026 21:17
Comment thread nemo_gym/_checkpoint/resources.py Outdated

def readiness(self) -> PrepareReport:
restarts = (
sorted({episode_id.capture_key for _, episode_id in self._checkpointed_sessions()})

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

restart_only: a seed in flight at prepare changes restarts mid-checkpoint, so the commit fails with restart_in_scope. restarts only lists registered sessions. A /seed_session admitted before close_admission isn't in it at prepare, so the controller keeps that episode in scope. The seed then finishes and registers, the commit's re-computed readiness() lists the episode as a restart, and the commit fails with RestartInScopeError on every retry of that checkpoint.

Repro on 98bdc0c (slow seed_session on a RestartOnlyServer, prepare while it's in flight, release it, commit with episode_ids=[{"rollout_id": "r"}]): prepare → prepared, restarts=[]; seed → 200 with x-ng-checkpoint-restart: 1; commit → 409 restart_in_scope: ['r'].

Suggest reporting in-flight seeds as restarts too, without waiting for them, so the server still never holds up prepare. Its seed reply always says restart, so the outcome is already known:

restarts = (
    sorted(
        {episode_id.capture_key for _, episode_id in self._checkpointed_sessions()}
        | {episode_id.capture_key for episode_id in self._seeding}
    )
    if self.mode == "restart_only"
    else []
)

With this, the same repro gives prepare restarts=['r'] and the commit succeeds; the existing resources tests still pass. A restart_only variant of test_a_seed_in_flight_when_admission_closes_holds_up_prepare_and_is_exported would cover it.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed as you suggested. In restart_only mode, the restarts now include the episodes of seeds in flight, without waiting for them, so the server still never holds up prepare. Covered by test_a_restart_only_seed_in_flight_when_admission_closes_is_reported_as_a_restart, your reproduction: prepare reports the seed's episode as a restart, the seed reply says restart, and the commit of the remaining scope succeeds.

await self.participant.wait_open()
if path == _SESSION_START:
# A restart_only server stays open during a checkpoint, but a new session would add a restart to it.
await self.participant.wait_open()

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit, same invariant: this is a single wait_open(), not a loop. In restart_only, if a new checkpoint clears _open between open_admission() setting it and this task resuming, the seed is admitted during that checkpoint and adds a restart after prepare reported. Suggest looping until open, like the accepting loop above.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed. wait_open() now checks again each time it wakes, like the accepting loop, so a seed whose wait ended just before a new checkpoint closed admission keeps waiting. Covered by test_a_restart_only_seed_waits_again_when_a_new_checkpoint_closes_before_it_resumes, which reopens and closes admission before the waiting seed runs again.

counted = path not in self.participant.replayable_paths
self.participant.request_started(session_id, counted=counted)
try:
await self.app(scope, receive, capture_status)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A disconnect can lose the session change. seeded() / session_ended_by() run only after await self.app(...) returns. Every resources server is wrapped in ClientDisconnectCancellationMiddleware, which cancels the request when the client disconnects (server_utils.py:1261). If that happens after the handler has changed state but before the response finishes sending, the CancelledError skips this block: a created seed is left untracked, or a session the server already closed/verified stays tracked. That's exactly what an environment retire does when it cancels its in-flight calls.

Suggest applying the change in capture_status on the first successful http.response.start, before awaiting the outer send, guarded to run once and re-reading the final session ID from scope["session"] there. A regression where the request is cancelled (or send raises) right after response start would cover it.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed. capture_status applies seeded() or session_ended_by() on the first successful http.response.start, before it awaits the outer send, once per request, and reads the final session ID from scope["session"] there. The session is still registered while the request counts as in flight. Covered by test_a_session_change_survives_a_disconnect_once_the_reply_has_started, where send is cancelled right after the response starts: the seed is tracked, and the closed or verified session is not.

return
if (self.config.num_workers or 1) != 1:
raise ValueError("resources checkpointing requires num_workers=1: sessions live in one process")
if self.checkpoint_mode == "exported":

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

checkpoint_mode and checkpoint_verify are ClassVar[Literal[...]], so nothing checks them at runtime. A typo like "exportd" starts fine and acts as a fourth mode: nothing is exported and no restarts are reported, so session state is silently dropped. Suggest validating both against their allowed values here at startup.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed. Startup now checks checkpoint_mode and checkpoint_verify against their allowed values and raises a ValueError that names the attribute, the value, and the allowed values. The check runs whether or not checkpointing is enabled, so a typo fails before anyone turns it on. Covered by test_an_unknown_checkpoint_declaration_fails_at_startup.

session_id_to_counter: Dict[str, int] = Field(default_factory=dict)

async def export_session_states(self, session_ids: list[str]) -> dict[str, JsonValue]:
return {session_id: {"count": self.session_id_to_counter.get(session_id, 0)} for session_id in session_ids}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

.get(session_id, 0) makes up state for sessions the server no longer holds. The hook contract says to leave those out, so the participant can stop tracking them; with this, lost state is exported as count=0 and a restore installs a wrong counter. Since this is the example people will copy, suggest only returning IDs present in session_id_to_counter.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed. The example now returns only the sessions present in session_id_to_counter, and says why in a comment, since people copy it. Covered by test_export_leaves_out_a_session_the_server_no_longer_holds.

Comment thread nemo_gym/_checkpoint/resources.py Outdated
# Inside an episode every resources call carries the rollout prefix, native seeds included.
episode_id = current_episode_id()
if episode_id is not None:
self.participant.seeded(session_id, episode_id)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Follow-up: a successful retire now handles an in-flight seed, but the path where the retire is cut short is still unsafe. retire() waits for _seeding (line 241) before it adds any session to _retired_sessions / _retired_session_rollouts. If the deadline cancels it during that wait, the seed finishes afterwards and seeded() binds the session as live. Then:

  • MCP calls that carry only the session token (no rollout prefix) are admitted;
  • the session is in _checkpointed_sessions(), so a checkpoint taken before the retry exports the retired attempt's state;
  • check_forget() lets forget through because _retired_sessions[rollout] is empty, and the session stays live after the rollout mark is cleared.

Slow seeds (sandbox/container start) make this realistic. Suggest having seeded() check whether self.retired already covers the EpisodeId and, if so, add the new session to the retired maps immediately (or install an equivalent rollout cutoff before waiting). A gated-seed / short-deadline regression would cover it.

Comment thread nemo_gym/_checkpoint/resources.py Resolved
Comment thread nemo_gym/base_resources_server.py Resolved
Comment thread nemo_gym/_checkpoint/resources.py Resolved
@github-actions

github-actions Bot commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

"""Refuse the attempts' sessions, wait for their requests in flight, then release them."""
# A seed admitted before the retire marked its attempt registers its session when it finishes; wait for it,
# so that session is freed here and not left behind.
while any(

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocker: a retire cut short leaves the attempt's existing sessions unfenced. retire() waits for _seeding before it adds the existing sessions to _retired_sessions / _retired_session_rollouts. If the deadline cancels that wait, sessions the attempt already had are never fenced. The rollout mark doesn't cover them for requests without a prefix, such as MCP calls.

Repro on fb5c857: one existing session for r/0, a second seed in flight, retire with a short deadline (409), then the late seed fails:

  • an unprefixed request on the existing session → 200;
  • ready() → True;
  • forget(["r"]) → 200, and the session stays live with the mark gone.

(If the late seed succeeds, its own session gets fenced and happens to block prepare and forget; the existing session is still admitted.) The commit only exports the retired attempt when its scope is None; with a normal scope, export filters it out.

Suggest fencing the sessions already known before the first await, then collecting and fencing again once the seeds finish. As a safety net, check_forget could use the rollout index (any tracked session with attempt ≤ the retired mark) rather than only the fence maps, so a missed fence can't make a session live again.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed, a retire that never got past the seed wait left the attempt's existing sessions admitted.
The participant now refuses those sessions as the attempt is marked, before anything awaits.
So a retire that is cut short, or that arrives after its deadline and only marks the attempt, still leaves them refused, holding up readiness, and out of every export until a retried retire frees them.
The retire itself collects the rollout's sessions again once the in-flight seeds finish, so it frees the sessions those seeds created as well.
With several workers, each worker fences its own sessions when the coordinator marks the attempt.
As the safety net you suggest, check_forget now looks up the rollout's tracked sessions through the rollout index and refuses while any of them belongs to an attempt at or below the retired mark, whether or not a fence reached it.
Covered by test_a_retire_cut_short_while_a_seed_is_in_flight_still_fences_the_attempt_s_existing_session, your reproduction: an existing session, a second seed in flight, a retire cut short, and the late seed failing.
The unprefixed request is now refused with stale_attempt, ready() is false, and forget is refused with retire_incomplete until a retried retire frees the session.
On the previous code the unprefixed request returned 200.
test_a_retire_that_arrives_after_its_deadline_still_fences_the_attempt_s_sessions covers a retire that only marks; on the previous code its late request also returned 200.


def admit(self, session_id: Optional[str], path: str) -> None:
# A close is never refused: it is how the server frees a session's state.
if path != _SESSION_CLOSE:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocker: a stale /close_session can delete a restored replacement. Closes skip every stale-attempt check. After a restore, the session is bound to attempt N+1, but a close from attempt N (with N marked retired by the restore) is still admitted. restore() marks the replaced attempts precisely because "a harness of a crashed attempt that outlived the crash may still call this server" (control.py:786).

Repro on fb5c857 (commit r/0, restore with episode_ids=[r], resume):

  • stale POST /ng-rollout/r/increment → 409 (correctly refused);
  • stale POST /ng-rollout/r/close_session → 200;
  • tracked sessions → 0, and the next commit writes 0 records.

Suggest checking ownership instead of exempting closes: in admit(), refuse any non-seed request whose EpisodeId (from the prefix) differs from the session's bound episode. That covers /verify too, and the close body already carries episode_id. A close from the owning attempt, including a retired one freeing its own session, still goes through. Requiring the rollout prefix on session-ending routes, as seeds already do, would make this complete.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed, closes skipped every check, so a replaced attempt could end its replacement's session.
admit() now checks ownership: a request whose rollout prefix names an episode other than the one the session is bound to is refused with stale_attempt.
Seeds are the exception, since a seed binds the session.
This covers /close_session and /verify alike, and a request from another rollout on the same cookie.
A close from the owning attempt still goes through, including a retired attempt freeing its own session.
With checkpointing on, /close_session and /verify now need the rollout prefix, as session starts do, and are refused with rollout_id_required without it.
Every caller already sends it: the environment server runs each episode inside its rollout context, Simple Agent's /run does the same, and ServerClient adds the prefix to every resources call made in that context; rollout collection and the end-to-end tests reach resources servers only through those.
Covered by test_after_a_restore_a_close_from_the_replaced_attempt_is_refused_and_keeps_the_session, your reproduction: commit, restore, resume, a stale close, then the next commit.
The stale close is refused, the next commit writes the session's record, and the replacement's own close then ends it.
On the previous code the stale close returned 200.
test_a_request_on_a_session_from_another_rollout_is_refused and test_a_session_end_without_a_rollout_prefix_is_refused_before_its_handler_runs cover the other two rules.

Comment thread nemo_gym/_checkpoint/resources.py Outdated
session_id = (scope.get("session") or {}).get(SESSION_ID_KEY)
is_mcp = self.mcp_session_id is not None and (path == self.mcp_path or path.startswith(self.mcp_path + "/"))
if is_mcp:
session_id = self.mcp_session_id(scope) or session_id

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocker / design decision: old MCP tokens stay valid after a restore. The token holds only sid and tools. A restored session keeps its sid and is bound to attempt N+1, and an MCP call carries no rollout prefix and isn't in the fence maps. So a harness of attempt N that outlived the crash (the case restore() guards against at control.py:786) can still change the restored session: the original token moves it from count 1 to 2.

test_the_harness_keeps_its_token_across_a_restore requires exactly this: the replacement harness keeps the same bearer credential. Since the old and new harness hold the same token, the server can't tell them apart, so "retired attempts stay fenced" and "the replacement keeps its token" can't both hold.

This needs an explicit decision. Options: put the attempt (or a session generation that restore bumps) in the token and issue the replacement a new one during recovery; or, if that's out of scope here, document it as a known limitation with a follow-up issue and narrow the test's claim.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Decided: the token now names the attempt whose seed minted it, and the resources server refuses a token whose attempt a retire or a restore has fenced.
With checkpointing on, _mint_session_metadata adds an episode claim holding the seed's capture key, such as r or r-a1.
With checkpointing off, the token is byte for byte what it was.
The checkpoint middleware reads the claim before the call reaches /mcp, and ResourcesParticipant.admit checks it the way it checks a rollout prefix: the session must belong to that attempt, and the attempt must not be retired.
A verified token without an episode claim is refused too, because a server with checkpointing on only mints tokens that have one, so such a token comes from a process without checkpointing, such as the one before a restart.
A refused call gets 409 stale_attempt before its tool runs, and it does not claim a restored session, so a commit that no longer continues the episode still deletes that session.
The replacement attempt never inherits the old token.
It gets its own by seeding under its own attempt, and a seed that carries the session's cookie claims the restored session, so the new token continues it.
No agent that holds an MCP token continues across a restore today.
Simple Agent, the only agent that continues from a checkpoint, refuses a required MCP grant and ignores an optional one.
Hermes, Codex, OpenCode, Pi and Haystack are restart-only, so their episodes start over from their input after a crash and seed a new session, which mints a new token.
To make that a rule instead of a coincidence, the environment server's _seed_agent now marks an episode as a restart when it grants MCP access.
So no checkpoint records such an episode past its seed while its agent holds a token for an earlier attempt.
The same fence closes a related gap: a harness that outlived a crash could call the fresh process with a token for a session that process never saw, and create state there that nothing would free.
Covered by:

  • test_a_restore_refuses_the_replaced_attempt_s_token_and_a_seed_of_the_replacement_gets_one_that_works, which replaces test_the_harness_keeps_its_token_across_a_restore;
  • test_a_stale_harness_cannot_create_state_for_a_session_the_fresh_process_never_saw;
  • test_a_token_without_an_attempt_is_refused_when_checkpointing_is_on;
  • test_a_second_restore_refuses_the_first_replacement_s_token;
  • test_a_refused_stale_call_does_not_claim_a_restored_session;
  • test_a_token_minted_without_checkpointing_names_only_its_session_and_tools and test_a_token_minted_with_checkpointing_names_the_attempt_that_seeded_it;
  • test_after_a_restore_onto_several_workers_every_worker_refuses_the_replaced_attempt_s_mcp_token, including a worker that joins after the restore;
  • test_an_episode_that_grants_mcp_access_is_a_restart in the single_agent_turn tests;
  • test_mcp_tool_calls_wait_out_a_checkpoint_and_a_restore_hands_their_sessions_to_new_tokens in the end-to-end suite, across a restart onto three workers.
    Ending a token's lifetime when its session closes is separate revocation work.
    It will use the new ResourcesParticipant.is_open while checkpointing is on.

Comment thread nemo_gym/_checkpoint/resources.py Outdated
async def delete_restored(self, episode_id: EpisodeId) -> None:
"""Free the restored sessions of ``episode_id`` that no request has used, without refusing anything later:
the controller may still start the rollout over as that attempt."""
for session_id in [key for key in self._restored_pending if self._sessions.get(key) == episode_id]:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perf: post-restore commit cleanup is quadratic. _delete_unscoped() calls delete_restored() once per unscoped restored episode, and each call scans all of _restored_pending, comparing pydantic EpisodeIds. Timing on fb5c857 (restore N sessions, then a commit whose scope drops them all):

restored sessions dropped time
1,000 0.37 s
5,000 9.7 s
10,000 41 s

This runs inside the commit's deadline. Each scan is synchronous, so it also stalls the event loop in ~4 ms chunks per episode. That can happen whenever the controller drops many restored rollouts after a restore.

Suggest using the rollout index, as retire() does:

for session_id in [
    key
    for key in self._rollout_sessions.get(episode_id.rollout_id, ())
    if key in self._restored_pending and self._sessions[key] == episode_id
]:

With this, the same probe takes 20 ms at 10,000 and 118 ms at 50,000, and the resources tests still pass.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Applied as you suggest: delete_restored now walks the rollout's sessions through the rollout index, so each call costs as much as its own rollout.
Measured with your probe, restoring N sessions and then deleting them all as a commit whose scope drops them does:

  • 1,000 sessions: 0.29 seconds before, 0.005 seconds after;
  • 10,000 sessions: 31 seconds before, 0.05 seconds after;
  • 50,000 sessions: 945 seconds before, 0.44 seconds after.
    Covered by test_deleting_one_rollout_s_restored_sessions_does_not_scan_every_restored_session, which makes iterating the restored sessions fail; on the previous code it did.

episode_ids.append(mcp_token.episode_id)
bound = self._sessions.get(session_id) if session_id is not None else None
for episode_id in episode_ids:
if not starts_session and bound is not None and bound != episode_id:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Require a tracked, open session for every non-start session-scoped call.

This ownership check runs only when bound is not None. A valid MCP token is therefore still admitted after /close_session, or against a fresh process before restore/retire installs its session. A prefixed direct-HTTP request can likewise target an untracked session ID. The tool can then create or mutate state while the participant reports no session, so later checkpoints neither save nor retire it.

Please fail closed for every MCP or direct-HTTP non-start session call unless the session is currently tracked and open. /seed_session must remain able to create the binding; an unknown /close_session can be idempotent, but it should not invoke a destructive hook when ownership cannot be established. Genuinely sessionless routes such as /aggregate_metrics should be explicit exceptions.

Please cover MCP and prefixed direct-HTTP calls both after close and before restore/retire has installed the session.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A tool call that creates or changes state on a closed or unknown session is a session-lifecycle bug, and it exists with checkpointing off as well.
It will be fixed on main for MCP and direct HTTP alike, together with revoking a session's MCP token when the session closes and binding the default seed's cookie session to resources_session_id, so the rule holds whether checkpointing is on or off.
That is why this PR does not add it.
What #3886 covers today is the attempt fence:

  • an MCP token names the attempt whose seed minted it, and a token of a retired or replaced attempt, or one that names no attempt, is refused before its tool runs;
  • a request whose rollout prefix names an attempt other than the one a session is bound to is refused, and so is a request on a session a restore installed that names no attempt (see the reply on the unprefixed direct HTTP call);
  • ResourcesParticipant.is_open reports whether a session is tracked and not retired, for the lifecycle work to use with checkpointing on.

# Refusing it would fail that episode; its episode re-runs it after a crash anyway.
if not self.accepting and path not in self.replayable_paths:
raise AdmissionClosedError("resources admission is closed for a checkpoint")
self._restored_pending.discard(session_id)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

An unprefixed direct-HTTP tool call can mutate and claim a restored session.

_seed_agent() gives direct-HTTP clients the bare Resources URL plus the session cookie. Such a request supplies session_id but no attempt in episode_ids, so it skips ownership validation and still removes the session from _restored_pending.

Reproduced after restoring r as r-a1: /ng-rollout/r/increment is correctly rejected, but /increment with the same cookie returns 200, mutates the restored state, and claims the session. A later commit that drops r-a1 writes no record but also fails to delete that external state.

Please propagate the current activation's capture key into direct-HTTP tool URLs and reject session-bearing requests with no attempt identity. These need to land together: requiring identity alone breaks current direct HTTP, while adding prefixes alone leaves cookie-only callers able to bypass ownership checks.

Please test that stale r is rejected, replacement r-a1 succeeds, and a refused stale call does not remove the session from _restored_pending.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed, with both halves together.
With checkpointing on, the environment server's direct HTTP grant now carries the seeding attempt's /ng-rollout/<capture key> prefix in its base URL.
Simple Agent sends each direct HTTP tool call under the rollout prefix of the activation it is serving, in place of the prefix in the grant, so after a restore the calls of the continued session name the replacement attempt.
Its legacy /v1/responses tool calls carry the same prefix.
No other agent with checkpoint support uses direct HTTP grants: every other agent rejects a required one, and restart-only agents seed again after a crash and get a fresh grant.
On the resources server, a request on a session a restore installed that names no attempt (no rollout prefix and no MCP token) is refused with 400 rollout_id_required, since the cookie alone cannot tell the replacement from the replaced attempt.
The refusal comes before the participant takes the session out of _restored_pending, so a refused call never claims a restored session.
Sessions seeded in the running process are not affected: a stale attempt can reach one only through a cookie, and a retire refuses the sessions of the attempts it retires.
Keeping it to restored sessions also leaves restart-only servers, which never restore a session, and agents whose tool calls carry only the cookie unaffected.
Covered by:

  • test_after_a_restore_only_the_replacement_s_prefixed_calls_reach_the_session: after restoring r as r-a1, /ng-rollout/r/increment is refused with stale_attempt, /increment with the cookie alone is refused with rollout_id_required, neither changes the restored count or claims the session, and /ng-rollout/r-a1/increment continues it;
  • test_stale_and_cookie_only_calls_stay_refused_across_a_second_restore;
  • test_with_checkpointing_on_a_direct_http_grant_names_the_seeding_attempt in the single_agent_turn tests;
  • test_a_restored_session_s_direct_http_tool_calls_reach_the_restored_resources_session in the Simple Agent tests, a continuation through Simple Agent and a real resources server, whose tool call reaches the restored session under the replacement's prefix;
  • test_resources_state_is_restored_with_the_episode in the end-to-end suite, where the replacement's legacy tool call increments the restored counter.

Comment thread nemo_gym/_checkpoint/resources.py Outdated

def session_ended_by(self, session_id: str, path: str) -> None:
"""A request on ``path`` ended ``session_id``. A replayable /verify during a checkpoint ends it at resume."""
if path in self.replayable_paths and not self.accepting:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocker: replayable /verify can end its session between environment and resources prepare.

session_ended_by() defers the end only after this resources participant has closed admission. Coordination prepares environment before resources. If the environment is inside a replayable /verify, environment prepare freezes a pre-verify boundary; if the request finishes before resources prepare reaches this server, accepting is still true and this branch calls ended(). Resources then commits no session record, but recovery replays /verify from the environment's pre-verify boundary with no restored state.

I reproduced this ordering: /verify returned 200, the resources commit wrote zero records, and the external counter remained present but untracked. The existing replayable-verify test starts /verify only after resources is already prepared, so it misses this cross-participant window.

For checkpoint_verify="replay", please do not make /verify the final resources-lifecycle end; keep the session tracked until an explicit /close_session or retire. Alternatively, introduce a coordinator-wide checkpoint epoch/fence visible to resources before environment prepare. Checking only not self.accepting cannot cover this window.

Please add a coordination-order regression: start replayable /verify → prepare environment while it is in flight → finish /verify → prepare and commit resources; assert that the session is exported and restore can replay verification.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed: a replayable /verify no longer ends the session when its caller closes the session afterwards.
The caller says so with a new x-ng-checkpoint-close-follows: 1 header on /verify.
With checkpoint_verify = "replay", the participant then keeps the session tracked, open and exported until the /close_session that follows, a declared end, or a retire.
So the order you reproduced no longer loses the session: whether the verification finishes before or after the environment and resources prepares, the session is still tracked when the resources server exports, and a restore runs the verification again against the restored state.
single_agent_turn (and so the legacy flat-row wrapper, which runs the same episode), the UserSim environment server and the sandboxed Hermes agent send the header, because each of them closes the session in its final cleanup.
Simple Agent's legacy /run now sends it after a replayable verification and then closes the resources session, including when a replacement is restored at its return step.
A /verify without the header still ends the session at once, as does a waited-for /verify with or without it.
A caller that never closes, such as a restart-only agent's own /run, would otherwise leave the session tracked forever, and no checkpoint continues its episode.
A checkpoint waits for a waited-for verification, so no boundary from before it can outlive it.
The deferred-end path (_deferred_ends) is gone.
Covered by:

  • test_a_replayable_verify_that_ends_between_environment_and_resources_prepare_is_exported_and_replays, your coordination order: the verification finishes before resources prepare, the commit exports the session, and after a restore the replacement verifies again and closes it;
  • test_a_replayable_verify_whose_caller_closes_the_session_afterwards_leaves_it_open_until_the_close;
  • test_a_replayable_verify_whose_caller_never_closes_ends_the_session_even_during_a_checkpoint and test_a_waited_for_verify_ends_the_session_whether_or_not_its_caller_closes_afterwards;
  • test_verify_says_that_the_resources_session_is_closed_afterwards in the single_agent_turn tests;
  • test_a_legacy_run_closes_its_resources_session_after_a_verification_a_checkpoint_does_not_wait_for in the Simple Agent tests.

macandro96
macandro96 previously approved these changes Oct 15, 2026

@macandro96 macandro96 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm!

…ous session hooks

A resources server declares a checkpoint mode: stateless, exported, or
restart-only. In exported mode its session state is exported and
restored through asynchronous hooks. A checkpoint waits for requests in
flight; a request that arrives during one waits for resume instead of
being refused; tools called over MCP are covered; and retire frees a
session once its requests have stopped.

Nothing of a restart-only server is in a checkpoint. Its admission
never closes, its requests never hold up prepare, and its live sessions
are reported as restarts. Its seed reply says so, so the episode that
seeded the session starts over from its input after a crash, and keeps
running through the checkpoint if nothing crashes.

Signed-off-by: Ananth Subramaniam <ansubramania@nvidia.com>
@github-actions

Copy link
Copy Markdown
Contributor

🔄 Merge queue validation started! You can track the progress here: https://github.com/NVIDIA-NeMo/Gym/actions/runs/38766077417

@github-actions

Copy link
Copy Markdown
Contributor

🔄 Merge queue validation started! You can track the progress here: https://github.com/NVIDIA-NeMo/Gym/actions/runs/38768162309

@github-actions

Copy link
Copy Markdown
Contributor

🔄 Merge queue validation started! You can track the progress here: https://github.com/NVIDIA-NeMo/Gym/actions/runs/38770334609

This branch was previously deployed

1 inactive deployment
public — 71c32bf1 Deployed Oct 15, 2026 by copy-pr-bot[bot] via release / finalize / notify #5514
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:env-infra Shared environment framework, lifecycle, registry, scaffolding, and validation complexity:high Cross-component or shared-contract change with a large review surface feature New capabilities, enhancements, or enablement work needs-review PR is ready for code review and waiting on a reviewer sla:review-overdue Review response is over the one-business-day SLA

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants