Skip to content

feat(simple_agent): continue an agent session across activations - #5465

Open
ffrujeri wants to merge 6 commits into
mainfrom
ffrujeri/simple-agent-session-continuation
Open

ffrujeri wants to merge 6 commits into
mainfrom
ffrujeri/simple-agent-session-continuation

Conversation

@ffrujeri

@ffrujeri ffrujeri commented Nov 2, 2026 •

Copy link
Copy Markdown
Contributor

On main (the BaseEnvironmentServerWithTools stack, #5346 → #5917, has merged). This is the first PR of the BIRD-Interact stack: this PR → #5409 (BIRD-Interact). Its diff is simple_agent, plus a small nemo_gym/server_runtime change that lets a tool reply end the agent's activation (second section below).

What changed

simple_agent accepts a seed that asks for continuation (AgentSeedSessionRequest.continuation). Each activation of the session then runs the turn loop on the session's conversation plus its new input, and keeps the first activation's tools and instructions.

  • An activation without new input, or one that changes any setting of the first activation (tools, instructions, tool_choice, temperature, ...), gets 422; only input and metadata change per activation.
  • A retried activation gets its reply without another model call.
  • The close returns the trajectory of every activation, also without observability_enabled, because the seed confirms it. Turns and invocations carry their activation_id. Each invocation keeps only its activation's new input and output; each turn's question still holds the model-visible prefix, as in one activation. A failed or cancelled activation adds a failed or incomplete invocation and an agent_activation_interrupted gap.
  • The session keeps the model server's cookies across activations, so it stays on one model-server session.
  • Only a class that sets continuation_supported in its own body supports continuation, so a subclass that replaces the turn loop, such as DeepAgentsAgent, refuses a continuation seed.
  • max_steps bounds each activation; the Environment Server bounds the activations.
  • With partial-rollout checkpoints on, a continuation seed is accepted and its reply carries CHECKPOINT_RESTART_HEADER (now exported from nemo_gym.server_runtime): a checkpoint exports no conversation, so the episode starts over after a restore. A session restored without its conversation refuses its next activation with AgentSessionConflictError (409), as main's session contract requires since refactor(sessions): raise the session conflict errors for every 409 refusal #5691.

A tool reply can end the activation

  • ToolResult.ends_activation (nemo_gym/server_runtime/episode_tools.py): a BaseEnvironmentServerWithTools tool marks its reply as the end of the agent's activation. Over direct HTTP the reply carries X-NeMo-Gym-End-Activation: true (END_ACTIVATION_HEADER, next to TOOL_ERROR_HEADER). MCP has no field for it, so an MCP agent calls the model again; the Fern MCP guide says so.
  • simple_agent runs the rest of the model response's tool calls and returns the activation without calling the model again.

Why

An Environment Server that simulates the user activates the agent once for each turn and sends only the new user message. BIRD-Interact's c-Interact (#5409) needs it: each user reply is a new activation of the same session. #5202 prototyped it. Tracking: GYMINT-646 (Linear).

The BIRD-Interact ADK ends the agent's run right after submit_sql: its before_model_callback answers the next model call itself. An Environment Server sees only tool calls, so without a signal simple_agent made one more model call per turn, which the ADK does not. Rewards were the same, but token counts and trajectories were not, and both are compared with auto-ontology-eval and used for training. #5478 uses the signal.

Validation

Command Result
pytest responses_api_agents/simple_agent/tests/test_app.py tests/unit_tests/test_agent_sessions.py 115 passed
pytest responses_api_agents/simple_agent/tests (review fixes) 169 passed, including a test for each fix and one that activation 2's model request carries activation 1's prompt_token_ids, generation_token_ids and generation_log_probs
The same tests with each fix undone (11 mutations) Each fails at least one test
agentic_turn_simulation_agent and labbench2_vlm_agent tests (SimpleAgent subclasses) 93 and 4 passed
pytest tests/unit_tests/server_runtime (after adding ends_activation) 2107 passed; new: only a result marked ends_activation carries the header, a call that never ran does not
pytest responses_api_agents/simple_agent/tests (after adding ends_activation) 183 passed; new: a reply with the header ends the activation after the response's other tool calls, with one model call. It fails without the change
pre-commit run --files on the changed files Passed

Rollout evidence

The c-Interact smoke run in #5409 uses this continuation: 10 rollouts with simple_agent, none failed. That run is from before the rebase onto main at dda609f86 and the split into two stacks, which changed no behavior here.

For ends_activation: the BIRD-Interact server's five example tasks in c-Interact (SQLite, Docker sandboxes, with #5478's use of the signal), before and after. Before, every turn made one model call after submit_sql: 8 extra calls over 8 turns. After, none: each turn made exactly one model call fewer, and all 5 episodes completed. Details in #5478.

Compatibility

Opt-in through the seed. An agent seeded without continuation behaves as before. ends_activation defaults to False, so other tools' replies do not change; an agent that ignores the header calls the model again, as before. SimpleAgent subclasses no longer accept continuation seeds unless they set continuation_supported.

Team review fixes (2026-11-19)

A team review (the repository's review-pr-team skill) found notes only. Fixed in c3167e7:

  • A continuation activation now goes through _create_session_episode, as an activation without continuation does, so a subclass's task context applies on every activation (new test).
  • The trajectory comment said the close payload grows linearly; each turn's question still holds the model-visible prefix.
  • The continuation test checks each activation's model route and its own metadata (each fails when its line is removed).
  • integrate-existing-agents.mdx describes simple_agent's continuation; multi-step-tasks.mdx says which harness can continue in a lent sandbox.

Not changed: #6162 adds the same model_server_cookies field with another type; whichever lands second reconciles them.

@ffrujeri ffrujeri added feature New capabilities, enhancements, or enablement work area:agent Agent harnesses and Responses API agent behavior labels Nov 2, 2026
@copy-pr-bot

copy-pr-bot Bot commented Nov 2, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@ffrujeri

ffrujeri commented Nov 2, 2026

Copy link
Copy Markdown
Contributor Author

/ok to test be98a71

@ffrujeri

ffrujeri commented Nov 4, 2026

Copy link
Copy Markdown
Contributor Author

/ok to test 371e2bd

@ffrujeri

Copy link
Copy Markdown
Contributor Author

/ok to test 456ad61

@ffrujeri

Copy link
Copy Markdown
Contributor Author

/ok to test 5af47df

@ffrujeri

Copy link
Copy Markdown
Contributor Author

/ok to test 9b88bd4

@ffrujeri

Copy link
Copy Markdown
Contributor Author

/ok to test 1e90e61

@ffrujeri

Copy link
Copy Markdown
Contributor Author

/ok to test b253632

@ffrujeri

Copy link
Copy Markdown
Contributor Author

/ok to test 8f00ca7

@ffrujeri

Copy link
Copy Markdown
Contributor Author

/ok to test e44aa46

@ffrujeri

Copy link
Copy Markdown
Contributor Author

/ok to test 6e8da1d

@ffrujeri

Copy link
Copy Markdown
Contributor Author

/ok to test d9fdc6b

@ffrujeri

Copy link
Copy Markdown
Contributor Author

/ok to test d9679b7

@github-actions

github-actions Bot commented Nov 16, 2026 •

Copy link
Copy Markdown
Contributor

@ffrujeri

Copy link
Copy Markdown
Contributor Author

/ok to test e90cdbe

@ffrujeri

Copy link
Copy Markdown
Contributor Author

/ok to test c725911

ffrujeri and others added 5 commits November 18, 2026 23:26
A user-simulation Environment Server activates the agent once for each
turn of the conversation and sends only the new user message. Simple
Agent now accepts a seed that asks for continuation: each activation
runs the turn loop on the session's conversation plus its new input,
and keeps the first activation's tools and instructions.

- An activation without new input, or one that replaces the session's
  tools or instructions, gets 422.
- A retried activation gets its reply without another model call.
- The close returns the trajectory of every activation.
- A continuation seed is refused when partial-rollout checkpoints are
  on, because a checkpoint exports no conversation history.

Carried over from #5202, which prototyped it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Felipe Vieira Frujeri <ffrujeri@nvidia.com>
…able and explicit

Review fixes for simple_agent's continuation sessions:

- Checkpointing: a continuation seed is accepted, and its reply carries
  CHECKPOINT_RESTART_HEADER, because a checkpoint does not export the
  conversation; the episode starts over after a restore. A session
  restored without its conversation refuses its next activation (409).
- Opt-in: only a class that sets continuation_supported in its own body
  supports continuation, so DeepAgentsAgent, which replaces responses(),
  refuses a continuation seed again.
- Settings: a later activation that changes any setting of the first
  (tool_choice, temperature, ...), not only tools or instructions, gets
  422 instead of having it silently dropped.
- Observations: a continuation session always collects the trajectory and
  timing its seed confirms, also without observability_enabled. A failed
  or cancelled activation adds a failed or incomplete invocation and an
  agent_activation_interrupted gap before it re-raises. Turns carry their
  activation_id. Each invocation keeps only its activation's new input
  and output, so the close payload grows linearly.
- Model server: the session keeps the model server's cookies across
  activations, so it stays on one model-server session.
- A test checks that a later activation resends the earlier turns' token
  IDs and log probabilities.

max_steps still bounds each activation, as it bounds one /v1/responses
call; the Environment Server bounds the activations.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Felipe Vieira Frujeri <ffrujeri@nvidia.com>
…base hook

Main checks that no agent overrides seed_agent_session. Simple Agent now
implements the base agent's _seed_restarts hook instead: under
checkpointing, a checkpoint does not export a continuation session's
conversation, so the seed reply marks the episode as a restart.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Felipe Vieira Frujeri <ffrujeri@nvidia.com>
A protocol whose turn ends at a submission, such as the BIRD-Interact ADK's
c-Interact, must end the agent's activation without another model call. The
ADK does it in a before-model callback; a Gym Environment Server sees only
the tool calls, so it signals it on the reply.

ToolResult.ends_activation sets X-NeMo-Gym-End-Activation: true on the direct
HTTP reply. simple_agent runs the rest of the model response's tool calls and
returns the activation without calling the model again. MCP has no field for
it, so an MCP agent calls the model again.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Felipe Vieira Frujeri <ffrujeri@nvidia.com>
…ntinuation lost its conversation

Main now requires every 409 session refusal in an agent to raise AgentSessionConflictError
(#5691, enforced by tests/unit_tests/test_session_contract_repo.py). The continuation session's
refusal after a restore raised a plain HTTPException(409); it keeps the same status and detail.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Felipe Vieira Frujeri <ffrujeri@nvidia.com>
@ffrujeri

Copy link
Copy Markdown
Contributor Author

/ok to test 94ff125

…ook, and document continuation

A continuation activation called _create_episode directly, so a subclass whose
task context lives in its session state (_create_session_episode) lost it from
the first activation on. It now goes through the hook, as an activation
without continuation does.

The comment on the trajectory said the close payload grows linearly; each
turn's question still holds the model-visible prefix, as in one activation.
The continuation test now checks each activation's model route and metadata.
integrate-existing-agents.mdx describes simple_agent's continuation, and
multi-step-tasks.mdx says which harness can continue in a lent sandbox.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Felipe Vieira Frujeri <ffrujeri@nvidia.com>
@ffrujeri

Copy link
Copy Markdown
Contributor Author

/ok to test c3167e7

This branch had an error being deployed

1 failed (outdated) deployment
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:agent Agent harnesses and Responses API agent behavior feature New capabilities, enhancements, or enablement work needs-review PR is ready for code review and waiting on a reviewer risk:high Autobot PR risk: high

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants