Repository navigation
fix(sc): checkpoint monotonic NeMo Gym task IDs - #4245
Conversation
|
/ok to test 0d915e9 |
mehraakash
left a comment
There was a problem hiding this comment.
Self-review performed with the upstream /review-pr-team skill using Codex agents: RL/design, bugs, tests/existing comments, and an independent devil's-advocate pass. One P2 correctness finding and one optional P3 test improvement survived the adversarial review. Both will be addressed before human review. The change remains limited to per-attempt NeMo Gym task identity and checkpoint continuity; no telemetry, masking, or recipe changes. Validation so far: 1,177 CPU tests passed; CI lint passed. OCI-HSG GPU validation is being prepared and is not yet claimed. No reward-corruption or convergence claim is made.
|
/ok to test d7028b2 |
|
/ok to test fc77827 |
terrykong
left a comment
There was a problem hiding this comment.
This PR needs a rebase on main. It conflicts in one file, tests/unit/single_controller/test_checkpointing.py, in two spots:
_FakeRolloutManager: keep bothmain's newreserve_prompt_groupand this PR'sget/set_next_nemo_gym_task_index.test_periodic_snapshot_restores_dispatch_cursor_and_task_counter: keep this PR's block, but writemc.logger.log_dir = ...instead ofmc.logger["log_dir"] = ....mainturned the logger config into a BaseModel (#4278), so the dict-style line fails after the merge.
The change itself looks right. Giving each attempt its own ID stops Gym keys from repeating when a dataset ID repeats (every epoch), and gives Gym an ID when the dataset row has none. The counter is saved together with the dispatch cursor, in both trainer checkpoints and rollout-only snapshots. It is restored the same way the sampler restores its dispatch index, and old checkpoints and snapshots still load. The tests cover save, restore, bootstrap, old manifests, and the retry-within-an-attempt case. We found no correctness bugs.
One note, no action in this PR: as the description says, the counter is per checkpoint. IDs handed out after the last save are handed out again after a resume, and a fresh run starts at 0. Any Gym agent that caches by (task_index, rollout_index) in a folder that outlives the run (Harbor's harbor_jobs_dir today) could return an old result. No NeMo-RL recipe uses Harbor. If that changes, the fix belongs in Gym: build the job name from _ng_rollout_id or a per-run ID.
Visual explainer: https://terrykong.github.io/gh-pages-poc/terryk/pr-4245-gym-attempt-ids.html
Generated by Claude Code
Signed-off-by: Akash Mehra <akamehra@nvidia.com>
Signed-off-by: Akash Mehra <akamehra@nvidia.com>
Signed-off-by: Akash Mehra <akamehra@nvidia.com>
Signed-off-by: Akash Mehra <mehra.akash@gmail.com>
fc77827 to
fcaaffb
Compare
|
/ok to test fcaaffb |
What does this PR do?
Give Single Controller's NeMo Gym rollout attempts a fresh, monotonically increasing
_ng_task_index, and persist the next value in trainer checkpoints and rollout-only snapshots.run_rolloutattempt; all sibling rows share it.next_nemo_gym_task_indexinGRPOSaveState; older checkpoints default to zero._ng_group_id,_ng_group_attempt,_ng_rollout_index, and token-capture rollout IDs.This is the narrow task-ID/counter change from e919a519, adapted to current main. No telemetry, masking, advantage-estimation, recipe, or submodule changes.
Validation
Current head: review follow-up (28 October 2026)
Rebased onto main
1849f2256725b783b34c9f91ae9444f93bb26587; current PR head isfcaaffbefadf6f11d23894d015839e5d4a656a60.reserve_prompt_groupand this PR's task-counter get/set methods in the checkpointing fake. Updated the periodic snapshot test tomc.logger.log_dir.git diff --checkpassed. Full local type/config hooks remain unverified.Earlier revisions (historical validation)
git diff --checkpassed on the review fixes. Local type/config hooks remain unverified because the available test environment is below the project's Python minimum; CI lint passed for the initial head, and CI is retriggered for the fix head./review-pr-teamskill using Codex agents, including an adversarial pass. Addressed the snapshot persistence finding and optional internal-retry identity test; bounded independent re-review found no further issues.fc778271db0667ae3a73458942c5f1dff5821c6b: initialize the counter in the runtime-agent-ref rollout test, and expose the native counter interface in the Megatron train-pump stub. The expanded local Single Controller + experience suites passed (1,475 passed, 1 skipped). No checks, assertions, or coverage thresholds were relaxed. Full L0 CI has been retriggered for this head; its result remains pending.Usage and compatibility
No configuration change. Native rollouts are unchanged. The counter is checkpoint-local, not a globally unique ID across independent runs or rollback to an older checkpoint. The snapshot manifest gains a backward-compatible optional counter field; legacy artifacts cannot reconstruct historical attempt counts that were never saved.
Checklist