Skip to content

feat(capture): support framework-owned context reconstruction - #4128

Merged
ananthsub merged 17 commits into
NVIDIA-NeMo:mainfrom
aroshanghias-nvd:cc-capture-infrastructure-20261006
Oct 26, 2026
Merged

ananthsub merged 17 commits into
NVIDIA-NeMo:mainfrom
aroshanghias-nvd:cc-capture-infrastructure-20261006

Conversation

@aroshanghias-nvd

@aroshanghias-nvd aroshanghias-nvd commented Oct 6, 2026 •

Copy link
Copy Markdown
Contributor

Training frameworks need to distinguish an exact captured-prefix continuation from a rewritten context without requiring harnesses to define training segments. Gym now carries replay evidence, completion identity, and committed capture coordinates through its shared ledger and ordinary vLLM server. Harnesses can own compaction while the training framework owns reconstruction and training-call selection.

Framework-owned capture validates requests before recording call intents, supports overlapping independent calls, and exposes exact context measurement under the rollout route. Buffered Responses and Chat streaming commit capture before returning answer chunks, including behind the checkpoint gate. Confirmed context-length refusals before generation are settled so a harness can compact and retry; cancellation preserves uncertain-call failure evidence. Ambiguous source-prefix matches start fresh roots with root_reason="ambiguous_candidate" and no guessed parent. Training selection must still validate accepted-call membership independently.

Captured generation and rollout requests reuse the existing connection-safe retry control: connections that were never established may retry, while requests that may have been delivered are not resent. The duplicate _retry mechanism is removed. Ordinary tokenization retains its retries.

Compatibility: capture's staging wire schema changes to version 3. Gym and every external-staging worker must upgrade together, including for ordinary capture. Version 2 admissions, staged rows, receipts, and checkpoints are incompatible; no migration is provided. Finish or retire existing attempts before upgrading, and do not resume their persisted version 2 data.

Part 1 of 2: Gym #3368 provides the optional compaction-agent migration and depends on this PR. NeMo-RL #4124 uses this shared infrastructure; external harnesses that own compaction do not need #3368. The existing compaction agent/model remain in this lower layer until the migration merges.

Validation of the current update:

The smoke uses manufactured image observations and synthetic rewards with real model generation and learning. It is a bounded integration check, not a new 20-optimizer-step qualification or a concurrent-harness benchmark. The optional agent layer separately passed 254 native/parity/checkpoint CPU tests, including six real transport retry regressions. Repository CI remains required before merge.

Merged Gym main at ed830b41, preserving upstream rollout tracing and retry behavior. No dependency, lockfile, or secrets-baseline changes relative to that main revision are included.

@aroshanghias-nvd
aroshanghias-nvd requested a review from a team as a code owner October 6, 2026 19:42
@copy-pr-bot

copy-pr-bot Bot commented Oct 6, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

Let training frameworks own capture roots and continuation decisions using request replay evidence and committed coordinates. Preserve completion identity, buffered Responses capture, explicit retry controls, and exact measurement through the ordinary vLLM server.

Keep the existing compaction agent and model intact in this infrastructure layer. The native agent migration is a separate stacked change.

Validation: 618 core/model cases and 56 retained legacy agent/model cases passed on Mac CPU across the documented runs. Combined-head GPU smoke evidence is recorded separately.

Co-authored-by: alexandery <alexandery@nvidia.com>
Signed-off-by: Ali Roshan Ghias <aroshanghias@nvidia.com>
@aroshanghias-nvd
aroshanghias-nvd force-pushed the cc-capture-infrastructure-20261006 branch from 5d32ffa to 66ce5c0 Compare October 6, 2026 19:51
@yaoyu-33 yaoyu-33 added area:training Training framework integrations and training-data interfaces complexity:high Cross-component or shared-contract change with a large review surface feature New capabilities, enhancements, or enablement work needs-review PR is ready for code review and waiting on a reviewer labels Oct 6, 2026
@github-actions github-actions Bot added the sla:triage-overdue Review assignment is over the one-business-day SLA label Oct 7, 2026
Comment thread nemo_gym/token_id_capture/lineage.py Outdated
return manifest.model_dump(mode="json")


def _validate_sequential_intent(rollout_id: str, rows: list[dict], expected_latest_call_id: str | None) -> None:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nemo_gym/token_id_capture/lineage.py:222

Why does sequential capture need to be mandated here? Some harnesses enqueue concurrent model calls (parallel tool calls, speculative branches), and with this check the first overlap is refused and the rollout is lost. The ledger already records siblings (same parent_call_id) and main's fingerprint lineage commits them today.

Two questions: is call order a hard requirement to maintain? And if it is, could the intent row in the manifest serve as the order marker (intents are appended in admission order, and admitted_at is on each record) instead of refusing calls whose commit order differs?

@aroshanghias-nvd aroshanghias-nvd Oct 13, 2026 •

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed—the sequential restriction was too strong. In the published update, independent calls can overlap, and RL uses validated parent ancestry rather than manifest completion order to reconstruct rows. The receipt identifies the scored response separately from selection order. Before finalization, RL atomically closes new admissions and drains already-admitted calls, preventing a late writer from racing cleanup. Tests cover both ledgers and both completion orders; the controlled GPU smoke covered overlapping calls, closure with calls pending, late-admission rejection, and once-only selected loss. This enables concurrent capture/reconstruction; it does not choose an all-captured training policy for subagents.

Published in Gym f4565363. Paired RL changes: dae5a4ad5.

Comment thread nemo_gym/base_responses_api_model.py Outdated
# Exact prefix supply and capture share this decision.
if current_capture_context() is not None:
context = current_capture_context()
parent_header = request.headers.get(CAPTURE_PARENT_HEADER)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nemo_gym/base_responses_api_model.py:536

Is it necessary to carry x-nemo-gym-capture-parent as a field? Black-box harnesses cannot send it, so the gateway already has to be correct from text inference alone; a second, optional lineage path that only Gym-aware callers use adds a contract surface (and is currently read on Responses but not on Chat) without changing what the black-box path must handle.

@aroshanghias-nvd aroshanghias-nvd Oct 13, 2026 •

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed. Removed the header, decoder, explicit-parent override, and selection enum in the published update. Both Responses and Chat use ordinary request-history matching; equal-length ambiguous candidates still fail rather than guessing. Updated the separate native-agent client test to remove the deleted header contract.

Published in Gym f4565363. Native-agent test adjustment: 507d15c3.

Comment thread nemo_gym/base_responses_api_model.py Outdated
Comment on lines 545 to 549
if parent_header is None:
await resolve_parent(request_messages)
else:
await resolve_parent(request_messages, parent_response_id=_decode_capture_parent(parent_header))
await register_call_intent()

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nemo_gym/base_responses_api_model.py:543

ValueErrors raised by _admit_candidate for client-attributable conditions (missing/ambiguous candidate, incomplete capture) surface here — and from measure_context — as bare 500s with no detail. Gym's own agent clients retry 500 (RETRY_ERROR_CODES includes 500), yet once an intent is pending the retry is guaranteed to fail, so each refusal costs a backoff cycle before the harness sees it. Mapping these to 409/422 would make them non-retryable and distinguishable from a gateway fault:

Suggested change
if parent_header is None:
await resolve_parent(request_messages)
else:
await resolve_parent(request_messages, parent_response_id=_decode_capture_parent(parent_header))
await register_call_intent()
try:
if parent_header is None:
await resolve_parent(request_messages)
else:
await resolve_parent(request_messages, parent_response_id=_decode_capture_parent(parent_header))
except ValueError as error:
raise HTTPException(status_code=409, detail=str(error)) from error
await register_call_intent()

@aroshanghias-nvd aroshanghias-nvd Oct 13, 2026 •

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Implemented in the published update: expected admission conflicts use CaptureAdmissionError and return HTTP 409 on Responses, Chat, and measurement. Intent-registration conflicts are covered too; unrelated internal ValueErrors remain server faults. Rejected requests that never registered an intent no longer create a spurious failed-call record. HTTP tests cover the refusal behavior, including buffered Responses streaming before the stream is committed. Healthy pending sibling calls are now allowed rather than classified as incomplete-capture failures.

Published in Gym f4565363.

context.capture_admission = CaptureAdmission(
**{**candidate.model_dump(exclude={"request_replay", "candidate_replay"}), "mode": "token_in"}
)
handler.prepare_request({"model": "model", "messages": items})

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tests/unit_tests/test_framework_context_capture.py:45

Could a test be added where the worker continues the previous call instead of starting a new one? Every worker stub in the new tests roots: _serve acks with parent None / prev_len 0, and the HTTP harness converts each candidate to mode="text" unless decide_input is supplied (unused in Gym). So the ordinary-turn branch — external_capture.py committing a candidate as token_in with the chain extended, and _admit_candidate rebuilding a staging chain longer than one element — is never exercised. A splice=True flag on _serve that echoes the candidate's parent_call_id/prev_len into the coords covers it; this passes on both ledgers:

async def test_accepted_candidate_chains_records_and_rebuilds_the_full_staging_chain(ledger):
    answer = [{"type": "message", "role": "assistant", "content": [{"type": "output_text", "text": "answer"}]}]
    history = [{"role": "user", "content": "task"}]
    await _serve(ledger, "c1", history)
    history = history + answer + [{"role": "user", "content": "more"}]
    second, admission = await _serve(ledger, "c2", history, splice=True)
    assert admission.parent_call_id == "c1" and admission.prev_len == 3
    row = (await ledger.manifest("rollout-1"))["records"][1]
    assert (row["parent_call_id"], row["mode"], row["prev_len"], row["cum_len"]) == ("c1", "token_in", 3, 6)
    history = history + answer + [{"role": "user", "content": "again"}]
    _, third = await _serve(ledger, "c3", history, splice=True)
    assert third.staging_chain == ["rollout-1/c1", "rollout-1/c2"]

@aroshanghias-nvd aroshanghias-nvd Oct 13, 2026 •

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added test_continuation_rebuilds_full_staging_chain in the published update, parametrized across in-memory and file-backed ledgers. The worker fixture can accept the candidate and commit token_in coordinates; three calls verify the parent/length metadata and reconstruction of the two-record staging chain. The paired RL/Gym tests also cover root -> append -> rewritten root -> append and compare the reconstructed tokens.

Published in Gym f4565363. Paired RL changes: dae5a4ad5.

Comment thread nemo_gym/token_id_capture/config.py Outdated
rebuild_response: bool = True
# The framework compares source history and chooses physical capture roots.
# This is serving configuration, never a harness-provided segment decision.
framework_owned_context: bool = False

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nemo_gym/token_id_capture/config.py:155

Could this surface be documented in fern/versions/latest/pages/model-server/model-call-capture.mdx (which already covers external_staging)? Specifically: framework_owned_context and its prerequisites (external_staging, external_staging_backend: vllm_worker, rebuild_response: false), what it refuses (/v1/messages, previous_response_id/conversation, Chat stream/n != 1), that a pending or failed call ends the attempt by design, POST /context/{rollout_id}/measure (called unprefixed), and run_examples(retry_requests=False). external-agent-harnesses.mdx is where harness authors would look for the framework-owned option.

@aroshanghias-nvd aroshanghias-nvd Oct 13, 2026 •

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated both Fern pages with the framework-owned configuration prerequisites, supported/rejected API forms, the unprefixed measurement route, and run_examples(retry_requests=False). The concurrency wording reflects the revised implementation: healthy pending calls may overlap; failed captures poison the attempt, and finalization closes admissions before draining pending calls.

Published in Gym f4565363.

Comment thread nemo_gym/token_id_capture/lineage.py Outdated
def is_process_shared(self) -> bool:
return False

async def begin_call(self, rollout_id: str, model_call_id: str, expected_latest_call_id: str | None) -> None:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nemo_gym/token_id_capture/lineage.py:494

New public surface without docstrings: InMemoryLineageStore.begin_call here and FileLineageStore.begin_call at L930 (the only implementations of a new ledger operation), and retry_requests in openai_utils.py:1388 lacks the Field(description=...) its neighbours carry.

Suggested change
async def begin_call(self, rollout_id: str, model_call_id: str, expected_latest_call_id: str | None) -> None:
async def begin_call(self, rollout_id: str, model_call_id: str, expected_latest_call_id: str | None) -> None:
"""Append a durable intent row for ``model_call_id``.
Fails if any call in the rollout is pending or failed, or if the ledger head is not
``expected_latest_call_id`` (another call committed since admission).
"""

@aroshanghias-nvd aroshanghias-nvd Oct 13, 2026 •

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added begin_call/close_admissions docstrings for both ledger implementations and a Field description for retry_requests in the published update. The begin_call contract now explicitly permits independent overlapping calls; the old expected-latest-call argument has been removed.

Published in Gym f4565363.

Comment thread nemo_gym/token_id_capture/lineage.py Outdated
"output_fingerprint",
"continuation_fingerprint",
"fingerprint_version",
"output_items",

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nemo_gym/token_id_capture/lineage.py:90

A few small things, none blocking:

  • _CUSTODY_FIELDS has no readers (grep finds only the definition) and now omits replay, which _custody_columns does write — add it or drop the constant.
  • lineage.py:121: record.replay.model_dump() / item.model_dump() embed schema_version: 3 inside the on-disk replay and output_items columns, so the next bump turns v3 rows into a ValidationError from manifest() rather than the fail-closed poison used for other columns; model_dump(exclude={"schema_version"}) avoids coupling nested columns to the version.
  • base_responses_api_model.py:410: body.get("n", 1) != 1 rejects an explicit "n": null, which _validate_single_choice_token_request accepts — body.get("n") not in (None, 1).
  • vllm_model/app.py:467: _resolve_client is called inside the framework-owned context, so the idempotent /tokenize for measurement also runs with retry_requests=False; resolving the client before set_token_sink keeps retries for measurement.
  • rollout_collection.py:3675: **({"_retry": False} if not retry_requests else {}) only exists to keep the mocked call signature stable; _retry=retry_requests reads more clearly (adjust the test).

@aroshanghias-nvd aroshanghias-nvd Oct 13, 2026 •

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All five cleanups are in the published update: removed the unused custody constant; excluded nested schema_version fields from persisted replay/output evidence and tested reload; accepted Chat n:null; resolved the measurement client before installing capture context so tokenization retains retries; and passed _retry=retry_requests directly with adjusted mocks.

Published in Gym f4565363.

@github-actions github-actions Bot removed the sla:triage-overdue Review assignment is over the one-business-day SLA label Oct 12, 2026
@pthombre

Copy link
Copy Markdown
Contributor

Suggestion: one owner per check in the continuity decision

Right now the decision about whether a call continues a previous context is split. Gym proposes a parent and the worker (decide_capture_input) decides, but most of the worker's checks repeat Gym's on Gym's own data (request_replay vs candidate_replay). The result:

  • Checks run twice, and the two copies differ. Gym matches leniently and ignores render options; the worker is strict and does check them.
  • Gym proposes parents it knows won't work. When no prefix matches, it falls back to the latest call (sink.py:342), and the worker always rejects that. A full compaction rewrite takes a detour through a parent that can't be used.
  • Why a new root was started isn't recorded. A tools change or a compaction both end up as a plain root.

Simplification: Gym decides everything it can see. The worker only confirms what it alone can see.

  • Gym: no prefix match, or render options changed → text. Otherwise → candidate with the parent. Add a flag when the match relied on null/"" tool-content equivalence.
  • Worker: check the boundary on the converted messages, plus renderer equivalence if the flag is set. If either fails, start a root; otherwise splice.

The conditions for splicing are the same as today, and the worker can still only downgrade. The difference is that each check runs in one place, and what Gym admits matches what gets recorded. Optionally, a root_reason on the record would separate compaction from config drift in the metrics.

@aroshanghias-nvd

aroshanghias-nvd commented Oct 13, 2026 •

Copy link
Copy Markdown
Contributor Author

@pthombre Following your continuity-ownership suggestion, the published update gives each check one owner: Gym matches source history and compares effective render options, emitting a direct text root when either cannot continue. Candidates carry an explicit flag when matching required null/empty tool-content equivalence. RL checks the converted-message boundary and actual renderer qualification, and can only downgrade the candidate to a root. Managed roots retain prefix stripping, capture/replay bookkeeping, and media validation. I left the optional root_reason metadata out to keep this change narrow.

Validation includes paired lifecycle tests, a controlled concurrent GPU smoke, and passing 20-step manufactured identity/image-compaction and custom OpenCode compaction runs. The unchanged custom OpenCode identity repeat also passed all 20 steps with maximum TMPE 1.00178 and exact token/mask/logprob and selected-call coverage checks. An earlier identity run had a 1.1787 spike that remains unexplained; the bounded old/new comparison did not reproduce it, so I am not claiming a numerical bug was fixed. This evidence qualifies the tested fixtures and pinned fork, not arbitrary multi-agent harnesses.

The changes are now pushed to this PR and RL #4124. I have left the review threads open for your review.

Published in Gym f4565363. Paired RL changes: dae5a4ad5. Native-agent test adjustment: 507d15c3.

Signed-off-by: Ali Roshan Ghias <aroshanghias@nvidia.com>
Signed-off-by: Ali Roshan Ghias <aroshanghias@nvidia.com>
Preserve concurrent framework capture alongside upstream rollout timing, outbound telemetry, tool-call attribution, and canonical finish reasons. Keep retry suppression effective with metrics and tracing enabled.

Validation: 2,475 affected Gym CPU tests and 184 paired RL compatibility tests passed; two optional custom OpenCode exporter cases skipped. Full pre-commit passed. Fern check passed with the unauthenticated redirect check skipped.
Signed-off-by: Ali Roshan Ghias <aroshanghias@nvidia.com>
Signed-off-by: Ali Roshan Ghias <aroshanghias@nvidia.com>
Comment thread nemo_gym/token_id_capture/lineage.py Outdated
raise CaptureAdmissionError("Capture admissions are closed for this rollout")
if any(row.get("model_call_id") == model_call_id for row in rows):
raise CaptureAdmissionError(f"Capture call ID already registered: {model_call_id}")
if any(row.get("failure_reason") is not None for row in rows):

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Once any call in an attempt has a failure row, this check refuses every later begin_call in the same attempt with a 409. _admit_candidate refuses the same calls again at sink.py#L355-L356, with a different message ("incomplete capture" there, "failed capture" here). The refused calls are never generated, so they are missing from the ledger entirely.

In practice one failed or cancelled call ends the rollout. With parallel tool calls or subagents, a transport error on one branch refuses every other branch's next call, including calls that start new contexts and have nothing to do with the failed one:

sequenceDiagram
    participant Child as "Subagent call"
    participant Parent as "Parent call"
    participant Gym as "Gym admission"
    participant Ledger as "Ledger"
    Child->>Gym: "request"
    Gym->>Ledger: "intent row"
    Note over Child,Gym: "worker or transport fails"
    Gym->>Ledger: "failure row"
    Parent->>Gym: "next request, unrelated context"
    Gym-->>Parent: "409, never generated"
Loading

The context-overflow and request-check cases in the other comments become dead rollouts for the same reason.

The refusal is not needed for correctness. A later call that would have continued the failed call cannot match its prefix, because the failed call has no committed replay, so it is admitted as a new root. The failure row is already in the manifest, and NeMo RL already masks attempts that have failures or pending calls.

Could both checks be removed, keeping the failure rows as recorded outcomes? The trainer can then decide whether to mask the whole attempt or drop only the failed chain.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed both attempt-wide admission blocks. Later independent calls can proceed; failed and pending calls remain in the manifest rather than being discarded. NeMo-RL still masks an incomplete attempt, so this does not introduce selective failed-chain training. Regression tests.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confirmed at 09adc12: a failure row no longer refuses later calls. _validate_call_intent now checks only closed admissions and duplicate ids (lineage.py#L234-L238). Our original reproduction now shows the failed subagent call's row in the manifest while the parent's next call and a fresh root both return 200.

Comment thread responses_api_models/vllm_model/app.py Outdated
await self._external_capture_handler.finalize_response(_jsonable(response))
context = current_capture_context()
if context is not None and context.framework_owned_context and not context.committed:
raise RuntimeError("Framework-owned context requires a committed capture before serving a response")

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When the prompt exceeds the context window, the worker refuses the request before generating or staging anything. With the default propagate_context_overflow_errors: false, the model server then builds a local response with finish_reason="length", and this guard raises because nothing was committed, so the harness receives a 500. The middleware writes a failure row for the call, and because of the check at lineage.py#L259-L260 the harness's next call gets a 409:

sequenceDiagram
    participant Harness as "Harness"
    participant Gym as "Gym vLLM model server"
    participant Worker as "Inference worker"
    participant Ledger as "Ledger"
    Harness->>Gym: "call with long history"
    Gym->>Ledger: "intent row"
    Gym->>Worker: "generate"
    Worker-->>Gym: "context length exceeded, nothing staged"
    Gym-->>Harness: "500 from the committed-capture guard"
    Gym->>Ledger: "failure row"
    Harness->>Gym: "compacted retry"
    Gym-->>Harness: "409, attempt refused"
Loading

That next call is usually the compacted retry. Codex, OpenCode and Gym's built-in compaction agent all compact when they see a context-length error, so with framework-owned context the compaction flow this PR is meant to support ends the rollout instead. Setting propagate_context_overflow_errors: true does not help: the harness gets the 400, but the failure row still blocks the retry. A streaming request gets response.failed and then the same 409.

Could the guard let the response through when the worker refused before generating, and could that call be recorded as not generated rather than as a failure? With the check at lineage.py#L259 removed, the compacted retry would then be admitted as a new root.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed the overflow/retry path: framework mode preserves the recognizable HTTP 400 context-length refusal, regardless of propagate_context_overflow_errors. A matching worker acknowledgement settles the pre-generation intent while retaining its audit row; unacknowledged errors remain incomplete. The paired worker acknowledgement is in NVIDIA-NeMo/RL#4124; harnesses need no new header. Regression tests.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confirmed at 09adc12: with the worker acknowledgement, the overflow returns a 400 context_length_exceeded for both propagation settings and both streaming modes. The call is settled as refused_before_generation, and the compacted retry commits as a new root (app.py#L539-L556). We also checked the NeMo RL side in NVIDIA-NeMo/RL#4124 end to end.

Optional: the refused call does not appear in RolloutManifest, and the retry reads root_reason="first_call", so a trainer cannot see that a compaction followed an overflow. A refused_call_ids list would make that visible.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Keeping refused_call_ids out of this update: it would be additional trainer-visible diagnostics, and the confirmed pre-generation refusal already settles safely and permits the compacted retry. The ledger audit row is retained. root_reason="first_call" continues to mean no committed predecessor, not proof that no earlier request was refused; the root reasons are documented as observations rather than compaction attribution.

if current_capture_context() is not None:
context = current_capture_context()
if context is not None:
request_messages = _request_messages(params)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Framework-owned admission builds its replay evidence from _request_messages(params), and that helper (L219-L226) only reads input when it is a list. A Responses request with a string input, which is the most common form, therefore contributes no user item to the evidence.

This has two effects. If the first call sends input: "old question", its committed replay contains only the assistant answer, so a follow-up that sends the full history as a list no longer matches and silently becomes a new root. Ordinary capture chains the same requests correctly.

The second effect is worse. If the first call sends input: "question A" and a later call sends [<first call's output>, {"role": "user", "content": "unrelated"}], the later request matches the first call's replay and is proposed as a candidate continuing it. NeMo RL's decide_capture_input checks the message roles at the boundary, not the prefix content, so it accepts the candidate and splices the first call's prompt tokens, including "question A", into a context the harness never sent. This is the shape a compaction harness produces when it keeps only the last answer plus a short continuation prompt.

Could _request_messages turn a string input into [{"role": "user", "content": input}]? The measurement route uses the same helper, so it is fixed too.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed _request_messages to normalize string input into a user message. Measurement and generation now use the same evidence; the regression checks both a valid continuation and requests that omit or change the original question, which must start new roots. Regression tests.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confirmed at 09adc12: a string input becomes a user message (base_responses_api_model.py#L216-L218). The full-history follow-up continues as a candidate, and the request that omits the original question now starts a new root.

admission = context.capture_admission
if admission is None:
return request_payload
if context.framework_owned_context:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These checks run inside the model handler, after register_call_intent has already written the intent row. When one of them raises, the call ends with a 500, and the middleware records request_finished_without_staged_coordinates even though the worker was never called. Every later call in the attempt then gets a 409.

Two paths reach these checks today. A server configured with continue_final_assistant_message: true fails on any request that ends with an assistant message, and a request whose metadata.extra_body carries truncate_prompt_tokens fails on its first call. The direct body fields are already rejected with a 422 by the request schema, so only these indirect paths get through.

Could these checks move before intent registration and return a 422, next to the stream and history gates in the dispatch methods? A config validator that rejects continue_final_assistant_message and extra_body.truncate_prompt_tokens together with framework_owned_context would also catch the server-level case at startup.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moved request-option validation before intent registration, returning HTTP 422 for unsupported metadata.extra_body options. Incompatible server-level options now fail at startup. Tests verify no worker call or capture intent is created, and a corrected request can proceed in the same attempt. Regression tests.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confirmed at 09adc12: metadata.extra_body truncation returns a 422 with no intent and no worker call (app.py#L518-L530), and continue_final_assistant_message and truncation in server options now fail at startup. Request conversion errors still happen after the intent; the new comment on app.py#L767 covers that case.

Comment thread nemo_gym/token_id_capture/config.py Outdated
@model_validator(mode="after")
def _validate(self) -> "TokenIdCaptureConfig":
block = self.token_id_capture
if block.framework_owned_context and (

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This validator checks the staging backend, but it does not reject framework_owned_context together with a checkpoint policy on the same model server, and that combination fails every call.

With the checkpoint gate attached, finalize_response defers the ledger row until the response starts, so committed is still false when the committed-capture guard in vllm_model/app.py runs. Every non-streaming call returns a 500. The failure is not recorded either, because _fail_uncommitted_external_call skips calls with a deferred commit, so the call stays in pending_call_ids forever.

Streaming fails differently but is also wrong. The harness gets response.failed, while the gate still publishes the deferred row, so the ledger records an answer the harness never received, and a retry adds a second root.

A restored attempt has a related problem: checkpoint export keeps the {"admissions_closed": true} row (_checkpoint/model.py#L629-L636), so the restored attempt refuses its first call.

NeMo RL refuses checkpointing in this mode on its side, but nothing in Gym enforces it. Could the configuration reject the combination, with one line in the docs, until deferred commits work with framework-owned context?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed deferred publication and restore rather than rejecting this combination. The guard recognizes a deferred capture, the checkpoint gate publishes the ledger record before forwarding the first answer chunk, and restore does not inherit closed admissions. Regression tests cover streaming/non-streaming delivery, publication failure, pending state, and a restored closed attempt; these are component checks, not a claim of arbitrary distributed checkpoint compatibility.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confirmed at 09adc12: non-streaming and streaming calls behind the checkpoint gate return 200, with the ledger row published before the first answer chunk (_checkpoint/model.py#L930-L936). A failed publication sends no answer, and a restored attempt admits its next call, including from an older export that still carries the closure row. Supporting the combination is better than rejecting it, thanks.

elif kind == "function_call_output":
role = "tool"
else:
raise ValueError(f"Unsupported replay item kind: {kind}")

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A Responses request containing an item kind this function does not know, such as web_search_call or custom_tool_call, raises a plain ValueError here. The client receives a 500, which SDKs retry. Ordinary capture also fails on these items, so this is not a regression, but a 422 would tell the harness the request will never be accepted. Could it return a 422, and could the docs table list the unsupported item kinds?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unsupported replay item kinds are now mapped to HTTP 422 before intent registration instead of leaking a ValueError as HTTP 500. Regression tests verify the response, no generation/capture writes, and subsequent reuse of the attempt. The supported-input table lists the unsupported kinds.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confirmed at 09adc12: unsupported item kinds return a 422 before the intent, and the docs table lists them.

Comment thread nemo_gym/token_id_capture/sink.py Outdated
for record in manifest.records:
replay = record.replay
if replay is None:
raise ValueError("Captured call is missing source replay evidence")

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the ledger contains a committed row without replay, every later admission in the attempt raises this plain ValueError. That happens after restoring a ledger written without framework-owned context, or after turning the flag on for an existing ledger. The client gets a 500, nothing is recorded because the intent is not registered yet, and SDKs keep retrying. The cyclic and missing-ancestor errors further down behave the same way. Could records without replay evidence be skipped as candidates, or could these raise CaptureAdmissionError?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Converted missing replay, cyclic-chain, and missing-ancestor cases to CaptureAdmissionError (HTTP 409) before intent registration. I chose explicit refusal over skipping evidence and guessing a different parent; the ledger is preserved for diagnosis. Regression tests verify repeated dispatch/measurement refusals without worker calls or ledger mutation.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confirmed at 09adc12: these now return a 409 before the intent, with the ledger unchanged (sink.py#L367-L369). Refusing loudly on a mixed or corrupt ledger is fine.

then drains already-admitted calls through the manifest endpoint. Closure refuses new generation but
allows admitted calls to finish. Missing acknowledgements remain pending; failure is not permission
to delete data that a worker may still be writing. NeMo RL masks an incomplete attempt.
Closed, failed, or ambiguous admission returns HTTP 409 with a reason instead of a retryable 500.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Docs suggestion: several framework-mode refusals still return a 500 rather than a 409, namely a context-length overflow, the truncation and prefill checks in prepare_request, unsupported input item kinds, and a ledger row without replay evidence. The OpenAI, Anthropic and Vercel AI SDKs also retry a 409. Could this sentence say which cases return a 409? The section could also state that framework-owned context cannot be combined with the checkpoint gate, and that the measurement route sits outside the rollout prefix without the control bearer token.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated the documentation to distinguish HTTP 422 invalid/unsupported input, HTTP 409 closed/ambiguous/unusable history, and acknowledged HTTP 400 context-length refusals, including SDK retries of 409. It also describes the now-supported checkpoint gate and rollout-prefixed measurement route; measurement is a model route, whereas the control endpoints require the control bearer token.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confirmed at 09adc12: the section now separates the 422, 409 and acknowledged 400 cases, warns about SDK retries of 409, and describes the checkpoint gate and the measurement route (model-call-capture.mdx#L148-L180).

await next(RolloutCollectionHelper().run_examples([failing_row()]))

@pytest.mark.parametrize("retry_requests", [True, False])
async def test_run_examples_controls_transport_replay(

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

post is mocked at the ServerClient level, which is above the retry loop, so it is awaited once whether retry_requests is True or False. The True case therefore passes even when retries are disabled, and the test only shows that the keyword is forwarded. Could it use a fake that fails once at the transport layer and assert one attempt for False and two for True? The docstring also says "CC callers", which would read better as "framework callers".

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Replaced the ServerClient.post mock with a fake underlying transport that disconnects once and then succeeds, exercising the real ServerClient retry loop. The test now asserts one transport attempt with retries disabled and two when enabled, and uses “framework callers” wording. Updated test.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confirmed at 09adc12: the test goes through the real retry loop, and it fails when _retry is ignored.


@pytest.mark.parametrize("route", ["responses", "responses-stream", "chat/completions", "measure"])
@pytest.mark.parametrize("failure", ["ambiguous", "failed"])
def test_capture_refusals_are_non_retryable(monkeypatch, tmp_path, route, failure):

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The name says these refusals are non-retryable, but the OpenAI and Anthropic Python SDKs and the Vercel AI SDK that OpenCode uses all retry a 409; only Gym's NeMoGymAsyncOpenAI treats it as final. The refusals tested here happen before an intent is written, so a retry is harmless. Could the name say that instead, for example test_capture_refusals_happen_before_generation?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed the non-retryable claim from the tests. The ambiguous-admission test verifies refusal before generation, with separate closed-admission coverage. The docs warn that external SDKs may retry HTTP 409; these refusals create no intent and do not dispatch inference.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confirmed at 09adc12: the names now describe refusal before generation, and the docs cover SDK retries.

Normalize string input as a user message before capture matching and measurement. Verify unchanged history continues while omitted or changed initial questions start new roots.

Validation: 840 affected Gym CPU tests, three HTTP prefix checks using the actual RL input decision, and pre-commit checks passed.
Signed-off-by: Ali Roshan Ghias <aroshanghias@nvidia.com>
Signed-off-by: Ali Roshan Ghias <aroshanghias@nvidia.com>
Signed-off-by: Ali Roshan Ghias <aroshanghias@nvidia.com>
Signed-off-by: Ali Roshan Ghias <aroshanghias@nvidia.com>
Signed-off-by: Ali Roshan Ghias <aroshanghias@nvidia.com>

@ananthsub ananthsub left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@aroshanghias-nvd please address the merge conflicts as well

chat_completion_create_params = self._converter.responses_to_chat_completion_create_params(
body, drop_unrepresentable_request_fields=self.config.drop_unrepresentable_request_fields
)
chat_completion_create_params = self._responses_to_chat_request(body)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Framework generation converts the Responses body here, after dispatch has already registered the call intent at base_responses_api_model.py#L736. A conversion error therefore reaches the client as a 500, and the middleware records request_finished_without_staged_coordinates for a call the worker never saw.

Two requests show it. A user message with an input_file part raises NotImplementedError("Unsupported part param type: input_file"), and tool_choice: "required" without tools raises ValueError. In both cases the manifest ends with a pending call plus a failure row, so NeMo RL masks the attempt, and an SDK that retries the 500 adds another intent and failure row each time. Measurement maps the same errors to a 422 before anything is written (app.py#L621-L625).

Could framework mode run _responses_to_chat_request inside _validate_capture_request, before the intent, and map ValueError and NotImplementedError to a 422? A test that asserts the 422 and an empty manifest would pin it down.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moved Responses-to-Chat conversion into _validate_capture_request, before intent registration, and map ValueError/NotImplementedError to HTTP 422. Regression coverage checks both examples through buffered/streaming Responses and measurement, with and without existing history: the manifest and worker-call count stay unchanged, and a corrected request can reuse the attempt. Published in c566cd5.

Comment thread nemo_gym/token_id_capture/lineage.py Outdated
raise CaptureAdmissionError("Capture admissions are closed for this rollout")
if any(row.get("model_call_id") == model_call_id for row in rows):
raise CaptureAdmissionError(f"Capture call ID already registered: {model_call_id}")
if any(row.get("failure_reason") is not None for row in rows):

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confirmed at 09adc12: a failure row no longer refuses later calls. _validate_call_intent now checks only closed admissions and duplicate ids (lineage.py#L234-L238). Our original reproduction now shows the failed subagent call's row in the manifest while the parent's next call and a fresh root both return 200.

Comment thread responses_api_models/vllm_model/app.py Outdated
await self._external_capture_handler.finalize_response(_jsonable(response))
context = current_capture_context()
if context is not None and context.framework_owned_context and not context.committed:
raise RuntimeError("Framework-owned context requires a committed capture before serving a response")

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confirmed at 09adc12: with the worker acknowledgement, the overflow returns a 400 context_length_exceeded for both propagation settings and both streaming modes. The call is settled as refused_before_generation, and the compacted retry commits as a new root (app.py#L539-L556). We also checked the NeMo RL side in NVIDIA-NeMo/RL#4124 end to end.

Optional: the refused call does not appear in RolloutManifest, and the retry reads root_reason="first_call", so a trainer cannot see that a compaction followed an overflow. A refused_call_ids list would make that visible.

if current_capture_context() is not None:
context = current_capture_context()
if context is not None:
request_messages = _request_messages(params)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confirmed at 09adc12: a string input becomes a user message (base_responses_api_model.py#L216-L218). The full-history follow-up continues as a candidate, and the request that omits the original question now starts a new root.

admission = context.capture_admission
if admission is None:
return request_payload
if context.framework_owned_context:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confirmed at 09adc12: metadata.extra_body truncation returns a 422 with no intent and no worker call (app.py#L518-L530), and continue_final_assistant_message and truncation in server options now fail at startup. Request conversion errors still happen after the intent; the new comment on app.py#L767 covers that case.

elif kind == "function_call_output":
role = "tool"
else:
raise ValueError(f"Unsupported replay item kind: {kind}")

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confirmed at 09adc12: unsupported item kinds return a 422 before the intent, and the docs table lists them.

Comment thread nemo_gym/token_id_capture/sink.py Outdated
for record in manifest.records:
replay = record.replay
if replay is None:
raise ValueError("Captured call is missing source replay evidence")

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confirmed at 09adc12: these now return a 409 before the intent, with the ledger unchanged (sink.py#L367-L369). Refusing loudly on a mixed or corrupt ledger is fine.

then drains already-admitted calls through the manifest endpoint. Closure refuses new generation but
allows admitted calls to finish. Missing acknowledgements remain pending; failure is not permission
to delete data that a worker may still be writing. NeMo RL masks an incomplete attempt.
Closed, failed, or ambiguous admission returns HTTP 409 with a reason instead of a retryable 500.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confirmed at 09adc12: the section now separates the 422, 409 and acknowledged 400 cases, warns about SDK retries of 409, and describes the checkpoint gate and the measurement route (model-call-capture.mdx#L148-L180).

await next(RolloutCollectionHelper().run_examples([failing_row()]))

@pytest.mark.parametrize("retry_requests", [True, False])
async def test_run_examples_controls_transport_replay(

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confirmed at 09adc12: the test goes through the real retry loop, and it fails when _retry is ignored.


@pytest.mark.parametrize("route", ["responses", "responses-stream", "chat/completions", "measure"])
@pytest.mark.parametrize("failure", ["ambiguous", "failed"])
def test_capture_refusals_are_non_retryable(monkeypatch, tmp_path, route, failure):

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confirmed at 09adc12: the names now describe refusal before generation, and the docs cover SDK retries.

@github-actions

Copy link
Copy Markdown
Contributor

@ananthsub
ananthsub added this pull request to the merge queue Oct 26, 2026
@github-actions

Copy link
Copy Markdown
Contributor

🔄 Merge queue validation started! You can track the progress here: https://github.com/NVIDIA-NeMo/Gym/actions/runs/40419833248

Merged via the queue into NVIDIA-NeMo:main with commit 11c874b Oct 26, 2026
53 of 56 checks passed
cmunley1 added a commit that referenced this pull request Oct 27, 2026
…r than 64 bits

The fingerprint fallback for integers beyond orjson's 64-bit range landed on main in
#4128. This keeps the test that a tool call whose arguments hold such an integer still
fingerprints, matches its reserialized form, and changes when the value changes.

Signed-off-by: Christian Munley <cmunley@nvidia.com>
ananthsub added a commit that referenced this pull request Oct 27, 2026
…tests (#5121)

#4128 renamed the admission's rollout_id field to capture_key to separate
capture ownership from rollout identity, and updated the fake backend but
not the two tests that read the admission, so they failed with KeyError.

Signed-off-by: Ananth Subramaniam <ansubramania@nvidia.com>
ananthsub added a commit that referenced this pull request Oct 29, 2026
* fix(capture): accept capture key names from before the rename

#4128 renamed rollout_id to capture_key on the capture wire models and
the old CallRecord.capture_key to source_capture_key. Frameworks still
on the old names, including NeMo RL main, now fail every rollout with
AttributeError or a rejected extra field.

Accept the old names as validation aliases and expose them as read-only
properties. Serialization still writes only the new names, and a payload
that carries both names is rejected.

Signed-off-by: Ananth Subramaniam <ansubramania@nvidia.com>

* fix(capture): warn once when pre-rename capture names are used

Emit a FutureWarning the first time a process passes or reads
rollout_id or CallRecord.capture_key, naming the field that replaced
it, so frameworks still on the old names learn to migrate without
flooding their logs.

Signed-off-by: Ananth Subramaniam <ansubramania@nvidia.com>

---------

Signed-off-by: Ananth Subramaniam <ansubramania@nvidia.com>
ananthsub pushed a commit that referenced this pull request Nov 6, 2026
…r than 64 bits

The fingerprint fallback for integers beyond orjson's 64-bit range landed on main in
#4128. This keeps the test that a tool call whose arguments hold such an integer still
fingerprints, matches its reserialized form, and changes when the value changes.

Signed-off-by: Christian Munley <cmunley@nvidia.com>
khazic pushed a commit to khazic/Gym-lao that referenced this pull request Nov 8, 2026
…64 bits (NVIDIA-NeMo#5049)

* test(token_id_capture): fingerprint tool arguments with integers wider than 64 bits

The fingerprint fallback for integers beyond orjson's 64-bit range landed on main in
NVIDIA-NeMo#4128. This keeps the test that a tool call whose arguments hold such an integer still
fingerprints, matches its reserialized form, and changes when the value changes.

Signed-off-by: Christian Munley <cmunley@nvidia.com>

* fix(server_utils): send request bodies that hold integers wider than 64 bits

The outbound JSON body in request() is encoded with orjson, which rejects integers outside the 64-bit range.
A tool call whose arguments carry one therefore failed before it was sent, the same way the token-capture
fingerprint did. Fall back to the stdlib encoder, compact and non-ASCII preserved, which accepts them.

Signed-off-by: Christian Munley <cmunley@nvidia.com>

* fix(server_utils): escape non-ASCII text in the fallback request body

With ensure_ascii=False the fallback still raised UnicodeEncodeError for a lone
surrogate, which orjson also rejects and a model can write into a tool argument.
The default escaping sends it, and any other non-ASCII text, as valid JSON.

Signed-off-by: Christian Munley <cmunley@nvidia.com>

---------

Signed-off-by: Christian Munley <cmunley@nvidia.com>
@qiaochuz-nv

Copy link
Copy Markdown
Contributor

QA verified: PR 4128 coverage — feat(capture): support framework-owned context reconstruction

Existing coverage

  • token_capture_external_staging (function, h100, vllm): PR 2872 external staging with a suite framework worker
    (testdata/token_capture/external_staging_worker.py), ordinary fingerprint mode. No source-history coverage.
  • token_capture_auxiliary_requests, model_call_capture*: unrelated to source history.

Contract

# Contract item PR unit/HTTP test QA
1 source_history + external_staging + vllm_worker + rebuild_response:false starts; incompatible server options rejected at startup test_framework_context.py::test_framework_rejects_unsupported_server_options_at_startup unit
2 Exact source-history extension → candidate admission with source-match proof; worker restores exact tokens test_framework_context_capture.py::test_source_and_effective_options_choose_root_or_proven_candidate, ::test_continuation_rebuilds_full_staging_chain e2e-gap (real template + real worker)
3 Rewritten history → new root with root_reason (first_call, no_prefix_match, non_user_suffix, render_options_changed, worker_rooted, ambiguous_candidate) ::test_root_reason_distinguishes_source_boundary_and_worker_decision, test_framework_context.py::test_an_unexplained_history_rewrite_roots_a_new_segment, ::test_ambiguous_capture_roots_without_choosing_a_parent e2e-gap (real compacting agent)
4 Unsupported requests → 422 before capture intent ::test_unsupported_request_does_not_reach_generation, ::test_invalid_metadata_is_rejected_before_capture_intent, ::test_unsupported_replay_item_is_rejected_before_intent unit
5 Buffered SSE commits before first chunk (also behind checkpoint gate) ::test_streaming_chat_continuation_commits_before_answer, ::test_streaming_responses_capture_like_their_non_streaming_twin, ::test_checkpoint_gate_delivers_framework_capture unit
6 Overlapping calls; close-admission → 409 for new generation; admitted calls drain via manifest ::test_http_calls_overlap_and_drain_after_close, ::test_closed_capture_refuses_dispatch_without_poisoning unit (e2e-gap only as the framework's finalize step)
7 Context-length refusal forwarded as HTTP 400 even with propagate_context_overflow_errors: false; settled only with worker ack, excluded from manifest ::test_context_refusal_settles_capture_only_with_worker_ack, test_token_capture_ledger.py::test_refused_intent_settles_without_losing_audit_or_other_pending_calls e2e-gap (real vLLM error text; detection is a text match, app.py:140-183)
8 Measure route count = generation prompt; no capture writes ::test_measurement_matches_generation_prompt_and_options, ::test_measurement_uses_same_candidate_without_ledger_writes e2e-gap (real tokenizer vs real prompt_token_ids)
9 Completion status completed/incomplete/failed/unknown on source-history rows test_completion_evidence.py::test_completion_status_is_bounded_and_does_not_invent_success e2e-gap (real finish_reason=length)
10 Staging wire schema v3; Gym + worker upgrade together test_token_capture_golden_vectors.py unit
11 Captured generation retries only failed connection setup ::test_capture_transport_retries_only_before_connection, ::test_transport_loss_never_replays_inference unit
12 run_examples(retry_requests=False) test_rollout_collection.py::test_run_examples_controls_transport_replay unit
13 RL reconstructs training rows from returned history + captured calls RL tests (not this PR) e2e-gap (RL ↔ Gym main)

Doc check (2026-11-12, workflow wf_554f4197-ab2, Gym 5d22439)

model-call-capture.mdx L65-196 + external-agent-harnesses.mdx L405-423. 5/5 checkers returned (26/34/30/24/34
claims), 0 findings → no vote round, no docs PR, no discussion rows, no nits.

Decisions

  • 2026-11-12 user: Gym-only e2e case now (h100, 1 GPU); NeMo-RL CC functional run against Gym main later as a
    separate session. Reason: exact-token continuation needs RL's worker (nemo_rl/models/generation/prefix_recovery/,
    ~1,500 lines); the suite stand-in worker declines candidates, so Gym records worker_rooted
    (nemo_gym/models/token_capture/external_capture.py:328).

Design (Gym-only case token_capture_source_history, h100, vllm)

  • Stack: gym env start --model-type vllm_model --environment workplace_assistant + the doc's 5 run-wide settings
    • all_agents=true; vllm_model → suite worker (testdata/token_capture/external_staging_worker.py, extended) →
      real vLLM Qwen3-0.6B, --max-model-len 16384, hermes tools + deepseek_r1 reasoning.
  • Worker stand-in: declines candidate admissions (stages a text root → Gym worker_rooted), acks vLLM
    context-length 400 with x-ng-capture-refused: <model_call_id>, answers /tokenize + ng_capture with
    prompt_token_count + ng_context_measured, logs each admission to decisions.jsonl.
  • Case 1 (README gym eval run, 5 examples, _ng_rollout_id=qa-sh-<id>): per rollout close-admission + manifest
    drain; first record first_call; every later record was a candidate whose parent is the previous call and is
    worker_rooted; response_status matches vLLM finish_reason; each root verifies; ≥1 rollout has ≥2 calls.
  • Case 2 (harness): first → continuation → compacted (measured first) → overflow (real vLLM 400, acked) →
    compacted retry → 16-token continuation; root reasons first_call/worker_rooted/no_prefix_match/no_prefix_match/
    worker_rooted; refused call absent and not pending; measure == worker count == served prompt tokens;
    incomplete + length for the short call.
  • Launcher moved to lib/token_capture_worker.py; token_capture_external_staging now imports it (rerun to prove
    unchanged).
  • make test:lint red only on pre-existing function/genrm_cohort_failure_contract: tier 'e2e' does not match its section (on main too; not ours).

Runs

# Gym Cases Result Run
1 0a5110a6c both ENV: h100 uv 0.12.20 < Gym main's required 0.13.4; upgraded bootstrap venv uv → 0.13.8 20261112T184424576448Z-4a650ffc
2 0a5110a6c both BLOCKED before Gym ran: vllm serve exits in 15 s, ModuleNotFoundError: Could not import module 'MistralCommonBackend' ← pycountry missing from the local_vllm_model venv. Same for token_capture_external_staging (not caused by this change) 20261112T184514318207Z-57419a36
3 0a5110a6c both BLOCKED: engine init No module named 'torchvision' (pycountry workaround in) 20261112T190328741843Z-69ed0590
4 0a5110a6c both harness PASS; external_staging PASS; README FAIL = test bug: ledger is <key>.lineage.jsonl, case globbed *.tokens.jsonl. Real data: qa-sh-4 tool call → continuation admitted as candidate (parent = first call) → worker_rooted; 4 rollouts answered without tools. Fixed glob; 5 repeats per example 20261112T190745945428Z-ea473c29
5 efaa56a17 both harness PASS; external_staging PASS; README FAIL = test bug in the stand-in's verify: it passed all 39 staged rows (25 rollouts) for a 2-row manifest (row_count_mismatch). Fixed: filter staged rows by capture_key. Evidence: 10/25 rollouts called tools, 14 candidate admissions 20261112T191327438857Z-cb87ec63
6 efaa56a17 all 3 PASS (README 73.3 s, harness 45.1 s, external_staging 41.8 s), pycountry + torchvision workaround 20261112T191640267263Z-f8f15a82
control efaa56a17 source_history case with parent_resolution: fingerprint (1-line copy, direct run) FAIL ×2 on behavior: first call has no root_reason; first continuation is a Gym-chosen token_in staging_chain admission, which the source-history worker refuses (CaptureError … prefix_token_ids) → 500 direct 20261112T191949Z, 20261112T192231Z (kept)

Registry: status: fail, nvbugs: ["7087161"] (open), reason in the docstring. Branch commit c1990fee on
nmfw_tests main a9f72574; MR body mr-body.md; NVBug comment draft
../nvb-7087161-local-vllm-pycountry/nvbug-comment-torchvision.html.

Blocker: vLLM 0.29 cannot start from a Gym-main local_vllm_model venv (2026-11-12)

Filed as NVBug 7087161 (board bugs/nvb-7087161-local-vllm-pycountry.md). Workaround per user: uv pip install --no-config --no-deps "pycountry==26.2.16" into h100's checkout venv for 0a5110a6cb (survives Gym's --inexact
pre-warm). Runs from 3 on carry this workaround; the MR says so. Run 3 then died on torchvision (same
never-install group, imported at engine init); from run 4 the venv also has torchvision==0.28.0+cu130 (cu130 index,
--no-deps).

  • Cause (observed): Gym build(env): AUT-2574 lock managed environments and use uv sync #4837 (1203b0be21, 2026-11-04) added "pycountry; sys_platform == 'never'" to
    responses_api_models/local_vllm_model/pyproject.toml override-dependencies (legal exclusion, security: remove dependencies flagged in legal review #2492). vLLM 0.29
    backend_guidance → transformers.MistralCommonBackend → mistral_common → pydantic_extra_types.language_code
    → pycountry. Reproduced in isolation on h100: python -c "from vllm.v1.structured_output.backend_guidance import GuidanceBackend" in that venv fails the same way.
  • Gym's own GPU e2e works around it: tests/e2e/gpu_e2e_test.sh:143-146 installs pycountry==26.2.16 "even for
    non-Mistral models". docker/install_excluded_deps.sh (the documented opt-in) says pycountry is for VLMs and is
    documented for containers only; local_vllm_model/README.md does not mention it.
  • Suite: lib/image_requirements.py runs that installer only when NEMO_GYM_CONTAINER=1; checkout-lane vLLM cases
    get nothing, so every vllm case on the h100 lane is blocked on Gym main since 11-04 (board trap
    environment-traps.md:238 recorded it for the nightly image only, when main still had pycountry).

Review (fresh agent, 2026-11-12): Minor fix needed

  1. Needs run evidence on main + a control (pending; blocked above).
  2. Case 1 continued > 0 depends on Qwen3-0.6B calling a tool on ≥1 of 5 tasks → decide after the first real run.
  3. Measure assertion prompt_tokens == worker count could not fail → dropped; the served-prompt equality stays.
    Worker protocol, root reasons, manifest fields, measure path and the external-staging refactor: no issue found.

Handoff

2026-11-12: MR !539 open (branch
qiaochuz/nemo-gym-pr4128-framework-context, commit c1990fee on nmfw_tests main a9f72574). NVBug 7087161 comment
#1 posted. Next:

  • /gym-review-bots !539 once Greptile/CodeRabbit have posted.
  • When NVBug 7087161 is fixed: rerun on h100 without the workaround, then flip the registry to pass and drop
    nvbugs (a /gym-merged-style follow-up MR).
  • NeMo-RL compaction run against Gym main: proposed as its own session (card "NeMo-RL compaction run vs Gym main").
  • h100 state left behind: uv 0.13.8 in ~/.local/share/nemo-qa-bootstrap; pycountry + torchvision in the
    local_vllm_model venvs of checkouts 0a5110a6cb… and efaa56a175…; landing worktree
    ~/nemo/workdir/pr4128-framework-context/nmfw_tests (its config/config.py is modified by the runner).

This branch was previously deployed

1 inactive deployment
public — a058736f Deployed Oct 26, 2026 by copy-pr-bot[bot] via release / finalize / notify #7408
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:training Training framework integrations and training-data interfaces complexity:high Cross-component or shared-contract change with a large review surface feature New capabilities, enhancements, or enablement work needs-review PR is ready for code review and waiting on a reviewer QA_verified

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants