Repository navigation
feat(environment): sign episode tokens for Environment Server tool grants - #5346
Merged
Merged
Conversation
Contributor
Author
|
/ok to test e050173 |
ffrujeri
force-pushed
the
ffrujeri/episode-tokens
branch
from
November 2, 2026 22:21
e050173 to
3dc43be
Compare
Contributor
Author
|
/ok to test 3dc43be |
ffrujeri
force-pushed
the
ffrujeri/episode-tokens
branch
from
November 4, 2026 21:33
3dc43be to
fa0b016
Compare
Contributor
Author
|
/ok to test fa0b016 |
Contributor
|
🌿 Preview your docs: https://nvidia-preview-pr-5346-4bc8e95296c7.docs.buildwithfern.com/nemo/gym |
Contributor
Author
|
/ok to test bfa9415 |
ffrujeri
added a commit
that referenced
this pull request
Nov 12, 2026
…er, and survive a raising header reader Review of #5350: - A worker that dies leaves its owner file, so a /run resent after the crash found the episode owned and failed for good. serve_tools now raises a non-terminal handled episode failure, so the attempt budget retries the episode under a new attempt key. - The router reads the session header outside the exception middleware, so a reader that raised became a bare 500. A reader that raises now names no session, and the worker the call reached handles it. - The workers of a multi-worker server read each other's episode tokens with the secret they inherit from the main process (episode_token_secret, #5346). - The e2e test bounds its wait for the episode's grant and fails at once, with the reply, if /run ends first. - The routing e2e job also runs when episode_tokens.py, episode_tools.py, tool_access.py or core/episode_types.py change. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Felipe Vieira Frujeri <ffrujeri@nvidia.com>
ffrujeri
added a commit
that referenced
this pull request
Nov 12, 2026
…er, and survive a raising header reader Review of #5350: - A worker that dies leaves its owner file, so a /run resent after the crash found the episode owned and failed for good. serve_tools now raises a non-terminal handled episode failure, so the attempt budget retries the episode under a new attempt key. - The router reads the session header outside the exception middleware, so a reader that raised became a bare 500. A reader that raises now names no session, and the worker the call reached handles it. - The workers of a multi-worker server read each other's episode tokens with the secret they inherit from the main process (episode_token_secret, #5346). - The e2e test bounds its wait for the episode's grant and fails at once, with the reply, if /run ends first. - The routing e2e job also runs when episode_tokens.py, episode_tools.py, tool_access.py or core/episode_types.py change. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Felipe Vieira Frujeri <ffrujeri@nvidia.com>
ffrujeri
added a commit
that referenced
this pull request
Nov 12, 2026
…er, and survive a raising header reader Review of #5350: - A worker that dies leaves its owner file, so a /run resent after the crash found the episode owned and failed for good. serve_tools now raises a non-terminal handled episode failure, so the attempt budget retries the episode under a new attempt key. - The router reads the session header outside the exception middleware, so a reader that raised became a bare 500. A reader that raises now names no session, and the worker the call reached handles it. - The workers of a multi-worker server read each other's episode tokens with the secret they inherit from the main process (episode_token_secret, #5346). - The e2e test bounds its wait for the episode's grant and fails at once, with the reply, if /run ends first. - The routing e2e job also runs when episode_tokens.py, episode_tools.py, tool_access.py or core/episode_types.py change. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Felipe Vieira Frujeri <ffrujeri@nvidia.com>
ffrujeri
added a commit
that referenced
this pull request
Nov 12, 2026
…m with a per-run secret Review of #5346: - read() caught only BadSignature, so a validly signed payload that does not decode raised BadPayload. It now catches BadData and never raises. - A non-ASCII nonce made hmac.compare_digest raise TypeError in matches(). read() now refuses it. - The signing key was the server's derivable session key. Tokens now use a random secret for each server run (episode_token_secret). The main process sets it in NEMO_GYM_EPISODE_TOKEN_SECRET before uvicorn starts the workers, so every worker reads the tokens another worker minted. The nonce still authenticates each call. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Felipe Vieira Frujeri <ffrujeri@nvidia.com>
ffrujeri
force-pushed
the
ffrujeri/episode-tokens
branch
from
November 12, 2026 20:09
bfa9415 to
c1a9e4c
Compare
ffrujeri
added a commit
that referenced
this pull request
Nov 12, 2026
…er, and survive a raising header reader Review of #5350: - A worker that dies leaves its owner file, so a /run resent after the crash found the episode owned and failed for good. serve_tools now raises a non-terminal handled episode failure, so the attempt budget retries the episode under a new attempt key. - The router reads the session header outside the exception middleware, so a reader that raised became a bare 500. A reader that raises now names no session, and the worker the call reached handles it. - The workers of a multi-worker server read each other's episode tokens with the secret they inherit from the main process (episode_token_secret, #5346). - The e2e test bounds its wait for the episode's grant and fails at once, with the reply, if /run ends first. - The routing e2e job also runs when episode_tokens.py, episode_tools.py, tool_access.py or core/episode_types.py change. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Felipe Vieira Frujeri <ffrujeri@nvidia.com>
Contributor
Author
|
/ok to test c1a9e4c |
ffrujeri
added a commit
that referenced
this pull request
Nov 13, 2026
…m with a per-run secret Review of #5346: - read() caught only BadSignature, so a validly signed payload that does not decode raised BadPayload. It now catches BadData and never raises. - A non-ASCII nonce made hmac.compare_digest raise TypeError in matches(). read() now refuses it. - The signing key was the server's derivable session key. Tokens now use a random secret for each server run (episode_token_secret). The main process sets it in NEMO_GYM_EPISODE_TOKEN_SECRET before uvicorn starts the workers, so every worker reads the tokens another worker minted. The nonce still authenticates each call. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Felipe Vieira Frujeri <ffrujeri@nvidia.com>
ffrujeri
force-pushed
the
ffrujeri/episode-tokens
branch
from
November 13, 2026 18:11
c1a9e4c to
19feb7c
Compare
ffrujeri
added a commit
that referenced
this pull request
Nov 13, 2026
…er, and survive a raising header reader Review of #5350: - A worker that dies leaves its owner file, so a /run resent after the crash found the episode owned and failed for good. serve_tools now raises a non-terminal handled episode failure, so the attempt budget retries the episode under a new attempt key. - The router reads the session header outside the exception middleware, so a reader that raised became a bare 500. A reader that raises now names no session, and the worker the call reached handles it. - The workers of a multi-worker server read each other's episode tokens with the secret they inherit from the main process (episode_token_secret, #5346). - The e2e test bounds its wait for the episode's grant and fails at once, with the reply, if /run ends first. - The routing e2e job also runs when episode_tokens.py, episode_tools.py, tool_access.py or core/episode_types.py change. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Felipe Vieira Frujeri <ffrujeri@nvidia.com>
Contributor
Author
|
/ok to test 19feb7c |
adil-a
approved these changes
Nov 16, 2026
ffrujeri
added a commit
that referenced
this pull request
Nov 16, 2026
…m with a per-run secret Review of #5346: - read() caught only BadSignature, so a validly signed payload that does not decode raised BadPayload. It now catches BadData and never raises. - A non-ASCII nonce made hmac.compare_digest raise TypeError in matches(). read() now refuses it. - The signing key was the server's derivable session key. Tokens now use a random secret for each server run (episode_token_secret). The main process sets it in NEMO_GYM_EPISODE_TOKEN_SECRET before uvicorn starts the workers, so every worker reads the tokens another worker minted. The nonce still authenticates each call. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Felipe Vieira Frujeri <ffrujeri@nvidia.com>
ffrujeri
force-pushed
the
ffrujeri/episode-tokens
branch
from
November 16, 2026 17:08
19feb7c to
73bc632
Compare
Contributor
Author
|
/ok to test 73bc632 |
…ants An Environment Server that serves an episode's tools needs a credential that names one episode and cannot be guessed. A rollout ID is predictable, and a Gym server's signing key is derived from its class and instance names, so neither is enough. EpisodeTokens signs the episode's capture key with a random nonce; the server accepts a token only while it holds the same claims for that episode. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Felipe Vieira Frujeri <ffrujeri@nvidia.com>
…m with a per-run secret Review of #5346: - read() caught only BadSignature, so a validly signed payload that does not decode raised BadPayload. It now catches BadData and never raises. - A non-ASCII nonce made hmac.compare_digest raise TypeError in matches(). read() now refuses it. - The signing key was the server's derivable session key. Tokens now use a random secret for each server run (episode_token_secret). The main process sets it in NEMO_GYM_EPISODE_TOKEN_SECRET before uvicorn starts the workers, so every worker reads the tokens another worker minted. The nonce still authenticates each call. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Felipe Vieira Frujeri <ffrujeri@nvidia.com>
Contributor
Author
|
/ok to test 96d85a3 |
kajalj22
approved these changes
Nov 16, 2026
Contributor
|
🔄 Merge queue validation started! You can track the progress here: https://github.com/NVIDIA-NeMo/Gym/actions/runs/43936963334 |
Contributor
|
🔄 Merge queue validation started! You can track the progress here: https://github.com/NVIDIA-NeMo/Gym/actions/runs/43943698668 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed
nemo_gym/server_runtime/episode_tokens.pyaddsEpisodeTokens, which mints and reads the credential an Environment Server grants for one episode's tools:mint(episode_id)returns a signed token and theEpisodeTokenClaims(episode ID and a random nonce) that the server keeps.read(token)returns the claims of a token this server signed, orNonefor anything else, including an MCP session token signed with the same key.EpisodeTokenClaims.matchescompares the episode and the nonce in constant time.EPISODE_TOKEN_HEADERnames the header that carries the token on a direct HTTP tool call.Why
This is the first PR of a stack that lets an Environment Server own an episode's state and tools without a Resources Server (Linear GYMINT-641, under the design in the user-simulation RFC's layer 1,
BaseEnvironmentServerWithTools). The tool calls arrive as separate requests, so they need a credential for the episode:<task index>-<rollout index>unless the caller sets one, and the agent sees its own in its capture path. A bare ID would let a policy call another episode's tools, for example to read hidden knowledge without paying for it.SimpleServer.get_session_middleware_key), so a signature alone is forgeable.The nonce makes the token unguessable; the server accepts it only while it holds the same claims. The signature keeps the claims intact, and lets a worker that does not hold the episode read which episode a token names, so a later PR can route the call to the worker that does.
Nothing uses the module yet; the next PR in the stack, the Environment Server base class that serves episode tools, does.
Validation
pytest tests/unit_tests/server_runtime/test_episode_tokens.py tests/unit_tests/test_import_linter_coverage.py(after the rebase onto main atdda609f86)nemo_gym/server_runtime/episode_tokens.py, next to the other server runtime modulespytest tests/unit_tests/server_runtime/test_episode_tokens.pypre-commit run --files nemo_gym/server_runtime/episode_tokens.py tests/unit_tests/server_runtime/test_episode_tokens.pyRollout evidence
N/A: a library module that no server uses yet. The BIRD-Interact migration onto the Environment Server base class will run rollouts.
Compatibility
New module only. No existing behavior changes.