Repository navigation
fix(nooa): preserve complete assistant history across persistence - #4351
Conversation
|
Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually. Contributors can view more details about this message here. |
fb70cd2 to
ddca469
Compare
… failures Adapt the useful lifecycle, failure handling, and NVIDIA gateway changes from Razvan Dinu's PR #4291 (49b8daa) on top of #4351's durable ordered assistant history. Keep successful BenchAgent services alive through verification, bound finish and cleanup, preserve terminal exceptions, isolate optional summary failures, and retain unrecovered infrastructure failures as ungraded. Align all active NOOA pins, including controller bootstrap, at 19caab1. Add regressions for cleanup ordering, cancellation, fresh delegated shells, recovery, failed-call evidence, and gateway copy-on-write normalization. Signed-off-by: Nikhil Deshmudre <nikhil.deshmudre@gmail.com>
d511c3d to
4640fd8
Compare
… failures Adapt the useful lifecycle, failure handling, and NVIDIA gateway changes from Razvan Dinu's PR #4291 (49b8daa) on top of #4351's durable ordered assistant history. Keep successful BenchAgent services alive through verification, bound finish and cleanup, preserve terminal exceptions, isolate optional summary failures, and retain unrecovered infrastructure failures as ungraded. Align all active NOOA pins, including controller bootstrap, at 19caab1. Add regressions for cleanup ordering, cancellation, fresh delegated shells, recovery, failed-call evidence, and gateway copy-on-write normalization. Signed-off-by: Nikhil Deshmudre <nikhil.deshmudre@gmail.com> NOOA-only rebuild of PR #4357 from 4849245. NOOA-only scope: shared infrastructure remains identical to main. Signed-off-by: Nikhil Deshmudre <nikhil.deshmudre@gmail.com>
4640fd8 to
089e0d2
Compare
… failures Adapt the useful lifecycle, failure handling, and NVIDIA gateway changes from Razvan Dinu's PR #4291 (49b8daa) on top of #4351's durable ordered assistant history. Keep successful BenchAgent services alive through verification, bound finish and cleanup, preserve terminal exceptions, isolate optional summary failures, and retain unrecovered infrastructure failures as ungraded. Align all active NOOA pins, including controller bootstrap, at 19caab1. Add regressions for cleanup ordering, cancellation, fresh delegated shells, recovery, failed-call evidence, and gateway copy-on-write normalization. Signed-off-by: Nikhil Deshmudre <nikhil.deshmudre@gmail.com> NOOA-only rebuild of PR #4357 from 4849245. NOOA-only scope: shared infrastructure remains identical to main. Signed-off-by: Nikhil Deshmudre <nikhil.deshmudre@gmail.com>
… failures Adapt the useful lifecycle, failure handling, and NVIDIA gateway changes from Razvan Dinu's PR #4291 (49b8daa) on top of #4351's durable ordered assistant history. Keep successful BenchAgent services alive through verification, bound finish and cleanup, preserve terminal exceptions, isolate optional summary failures, and retain unrecovered infrastructure failures as ungraded. Align all active NOOA pins, including controller bootstrap, at 19caab1. Add regressions for cleanup ordering, cancellation, fresh delegated shells, recovery, failed-call evidence, and gateway copy-on-write normalization. Signed-off-by: Nikhil Deshmudre <nikhil.deshmudre@gmail.com> NOOA-only rebuild of PR #4357 from 4849245. NOOA-only scope: shared infrastructure remains identical to main. Signed-off-by: Nikhil Deshmudre <nikhil.deshmudre@gmail.com>
089e0d2 to
72072eb
Compare
… failures Adapt the useful lifecycle, failure handling, and NVIDIA gateway changes from Razvan Dinu's PR #4291 (49b8daa) on top of #4351's durable ordered assistant history. Keep successful BenchAgent services alive through verification, bound finish and cleanup, preserve terminal exceptions, isolate optional summary failures, and retain unrecovered infrastructure failures as ungraded. Align all active NOOA pins, including controller bootstrap, at 19caab1. Add regressions for cleanup ordering, cancellation, fresh delegated shells, recovery, failed-call evidence, and gateway copy-on-write normalization. Signed-off-by: Nikhil Deshmudre <nikhil.deshmudre@gmail.com> NOOA-only rebuild of PR #4357 from 4849245. NOOA-only scope: shared infrastructure remains identical to main. Signed-off-by: Nikhil Deshmudre <nikhil.deshmudre@gmail.com>
72072eb to
199d650
Compare
0d2ae72 to
71a6ff4
Compare
init-nikhil
left a comment
There was a problem hiding this comment.
CODEX SELF REVIEW
No actionable findings were returned by the official Codex review engine. This is not a guarantee of correctness.
Reviewed head: 71a6ff48bffdc0c4bb653af3481aa7c9ebb77022
Actual PR base: 4c56c44e964c1c5348d64fa66f3f7623f20c3c8d
Engine: official codex-cli 0.154.0, fresh codex exec --sandbox read-only --ephemeral --json review --base process using the unchanged account configuration. Exit code: 0.
Validation performed by the engine:
- All 44 adapter tests passed against NOOA
051472343211914222e24ce36d8752f4e86bbe43. - 15 selected runner tests passed; 1 runner test was deselected.
- The source checkout and Codex configuration fingerprints were unchanged after review.
Live-model smoke rollouts were not run by this review engine. Initial pytest setup failures were resolved by using in-memory output capture and binding imports to the reviewed checkout; the passing test invocations and original errors are preserved.
Run evidence: gymcheck-4351-20261020T074700Z/reviewer contains the invocation, stdout/stderr, original final response, test output, and before/after source/configuration fingerprints. Final response SHA-256: 08373c017889d941d00d96625de79dbf3e38269a9aba0bf03f9fb4cd0d1edef8.
|
Verification checks PASS — evidence date: 2026-10-20. Head: Three component scenarios (14 local HTTP calls) and one real-model rollout (two inference calls) passed. Native metadata survives NOOA’s existing serialization/storage interfaces; the PR adds no disk-writing feature.
The live run is fresh. The three component PASS results are reused from the same manual run: their primary harness, source/runtime bindings and executed support paths are unchanged; the subsequent fixture correction affects only the live runtime selector. Earlier blocked fixture attempts remain preserved locally. Production source and cleanup assertions were unchanged. Tested with local Linux transport, pinned NOOA |
|
🌿 Preview your docs: https://nvidia-preview-pr-4351-83be41989f40.docs.buildwithfern.com/nemo/gym |
|
/ok to test 6dc2cc0 |
|
/ok to test da6a5d9 |
|
/ok to test d537a13 |
furgalep
left a comment
There was a problem hiding this comment.
A few findings on the new replay/persistence path, roughly most important first: the parsed/content mismatch (L437), missing gap for dict-rendered turns (L208), native replay ignoring public fields (L175), and the permissive scope guard (L172). The rest are lower risk or pre-existing.
init-nikhil
left a comment
There was a problem hiding this comment.
CODEX SELF REVIEW
Reviewed head 055e0cfcee5fac6986f2c2dc4be5f833a370bdb7 against actual base bc6a7e7976b29e08af1d9986489c2aabcdeea82c using a fresh official codex exec review process.
No actionable regressions were identified across the five changed files. The focused adapter and model-failure suites passed all 88 tests against the pinned NOOA dependency; real model rollouts were not run.
This is a source-review result, not an approving review or a substitute for the separate validation and CI gates.
|
/ok to test 5db6c04 |
|
/ok to test 60dff88 |
|
/ok to test 9b9a4f5 |
|
/ok to test 5ddd254 |
Signed-off-by: Nikhil Deshmudre <nikhil.deshmudre@gmail.com>
|
/ok to test 83be419 |
|
🔄 Merge queue validation started! You can track the progress here: https://github.com/NVIDIA-NeMo/Gym/actions/runs/41952348655 |
|
🔄 Merge queue validation started! You can track the progress here: https://github.com/NVIDIA-NeMo/Gym/actions/runs/41958832710 |
Current heads — November 4, 2026
Head
83be41989f40e708158e1c4dd47b327cd720905e; refreshed onto main653846f949ad08a29df19c7de2ee4166d1c1bab3before final CI/review. #4357 resolves main's dependency migration by keepingrequirements.txtremoved, updating all three NOOA pins in the managed environmentpyproject.toml, and regenerating itsuv.lock. The runtime loader and package layout from main are preserved. #4357 requires automation-team code-owner review for these dependency files.Validation: 186 focused history, observability, result and sandbox-runtime tests passed; all-files pre-commit passed;
uv lock --check --project responses_api_agents/nooa_agentpassed. Current-head CI requested with/ok to testand must pass before merge. Below are earlier descriptions/evidence and may reference superseded heads.Current stack revision — November 3, 2026
Head:
9b9a4f5b9e4e5bf5388193035023ea1500644d10. Stack refreshed onto main3880fa5024bf97199cba96ade2e0fa6ec9eb6fbfwith current import locations and merge conflicts resolved. Each PR remains a separate layer; landing uses native stack merge.#4088 now packages the launching Gym checkout using its unchanged root
pyproject.tomland full core dependencies. The separateresponses_api_agents/nooa_agent/runtime/pyproject.tomlwas removed. Tracked source and package assets are copied with their current contents; untracked files, Git metadata, and caches are excluded. This requires Git on the controller. Sandbox installation stays in its private Python, and source changes invalidate prepared offline wheelhouses. #4813 retains its workflow change and code-owner review requirement.Validation: 87 focused runtime, offline-bundle, and task-network-policy tests passed after the refresh; all-files pre-commit passed (private guardword catalog unavailable). A real Gym wheel built successfully from the new archive format, and its NOOA entrypoint, resource dispatcher, and core package asset were verified. No model rollout or Linux installation was run for this packaging change. Earlier evidence below is historical and does not establish fresh results for this head.
GitHub dismissed earlier reviews after this update; current-head reviews and required CI are still needed.
Stack refresh — 2026-11-02
Current head:
f2910420f5f14795021c5764e2d1d61492890949. Stack refreshed onto main2b8efc987af03951a80f941f2a278317badfe085. This PR remains a separate layer in the intended stack merge: #4088 → #4820 → #4821 → #4351 → #4357. #4813 is excluded.Reconstructed each layer from its final reviewed patch instead of replaying foundation commits already merged into main. Resolved current package-import and legacy-environment renames while retaining upstream checkpoint handling and newer harness support. #4088 also exports its networking helpers through the current public API and mechanically updates existing callers; #4351 exports its newly used response types.
Fresh validation on the combined refreshed stack:
19caab169b018476ac433d040f6ae3f06aeff101.pre-commit run --all-files: passed, including package layering, guardwords, lint and format.No fresh real-model benchmark was run for this conflict/import refresh. Prior live-rollout evidence below is historical. Fresh reviews and required GitHub CI remain outstanding; this is not a merge-readiness claim.
Earlier implementation and validation (before this refresh)
Tests and validation
Head
055e0cfcee5fac6986f2c2dc4be5f833a370bdb7; basebc6a7e7976b29e08af1d9986489c2aabcdeea82c. The existing stack tracks this work; no separate issue is needed.46 focused history/structured-output/preparation checks passed. Independent native JSON-on-disk and reopened SQLite cases preserved ordered training metadata; edited history and changed model routes produced portable fallback. The same-candidate large-array serialization/replay probe was retained and audited.
Fresh verification made 14 real HTTP history calls across three controlled cases and completed a separate two-call NVIDIA TaskAgent rollout. The synthetic marker task earned reward 1; exact history replay, raw capture, close and cleanup were inspected.
Focused test selection:
python -m pytest -q -p no:cacheprovider --import-mode=importlib \ responses_api_agents/nooa_agent/tests/test_gym_llm.py \ responses_api_agents/nooa_agent/tests/test_model_failures.py \ -k 'history or replay or structured or unsupported or nontext or metadata or preparation or mixed or public or foreign or different_model'Training-array durability is established by component cases; the live model did not return training token IDs. Public-dictionary provenance and non-model terminal-output gaps remain explicit, so strict ATIF export is not certified. The synthetic rollout used local Linux provisioning and an existing source-bound runtime; it does not qualify cloud infrastructure or cold installation.
pre-commit run --all-filespassed. Fresh official Codex review returned no actionable findings; the independent bug hunt passed for this head and base.CI snapshot (2026-10-24 05:44 UTC) on this head: 6 checks remain pending. Current checks. The all-green readiness gate remains unmet; no final verification-success post or new review request is implied.