Skip to content

refactor(code-gen): replace legacy Ray execution with sandboxes - #4958

Merged
ananthsub merged 6 commits into
NVIDIA-NeMo:mainfrom
Kipok:codex/lcb-sandbox-no-parent-ray
Nov 5, 2026
Merged

ananthsub merged 6 commits into
NVIDIA-NeMo:mainfrom
Kipok:codex/lcb-sandbox-no-parent-ray

Conversation

@Kipok

@Kipok Kipok commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor

code_gen now verifies every candidate in a provider sandbox. Remove the legacy Ray dispatch, its batch-metric helpers, and the Ray dependency/lock entries; mark the default app.py entrypoint as non-Ray. sandbox_app.py remains a compatibility alias to the same server, with no separate execution branch. This addresses the request to finish the migration from #4607; no separate issue is needed.

Migration: existing code_gen configurations must supply a sandbox provider and sandbox_config.image (the shipped config reads CODE_GEN_SANDBOX_IMAGE). Missing provider/image values fail validation. The README documents provider setup and the migration. Scoring, candidate harness and execution deadlines are unchanged; switching execution environments can change time-limit-sensitive grades.

Declare verification stateless for gym eval reverify: saved responses and test inputs fully specify grading. Both entrypoints import with Ray blocked, and launcher tests verify that neither requires or initializes parent Ray.

Validation after merging main 2ac7908c4:

  • code_gen suite: 81 passed, 1 skipped (optional Docker integration).
  • Ray inventory/launcher and public API suites: 958 passed.
  • pre-commit run --all-files and diff whitespace checks passed; commits have DCO sign-offs.
  • Fresh model smoke on the final server implementation: openai/openai/gpt-5.6-luna generated a stdin-sum program, which passed all three test inputs through /verify and a real OpenSandbox sandbox (reward 1). An incorrect control scored 0. Both requests returned HTTP 200 without masks or failures; Ray was never imported. Inspected the generated code and verifier results. This is an integration smoke, not a benchmark score comparison.

Earlier evidence for the unchanged sandbox verifier: a 9/9 model-generated canary; a 1362/1362 cached-answer regrade with exact input/response preservation, no masked rows, and one timeout changing to a pass; and a fresh 1362/1362 benchmark with no missing, duplicate, failed, masked, unscored or truncated rows. Five generations were inspected in each model run. These are earlier-revision results, not fresh full-benchmark results for this head.

Signed-off-by: Igor Gitman <igitman@nvidia.com>
@copy-pr-bot

copy-pr-bot Bot commented Oct 23, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@svcnvidia-nemo-ci svcnvidia-nemo-ci added area:environment Individual environments, benchmarks, verifiers, and environment-specific resources servers bug Something isn't working labels Oct 23, 2026
Signed-off-by: Igor Gitman <igitman@nvidia.com>
@Kipok
Kipok marked this pull request as ready for review October 23, 2026 05:37
@svcnvidia-nemo-ci svcnvidia-nemo-ci added the needs-review PR is ready for code review and waiting on a reviewer label Oct 23, 2026
@cmunley1

Copy link
Copy Markdown
Contributor

/review

@cmunley1

Copy link
Copy Markdown
Contributor

/ok to test 92ff1e0

@nemo-automation-bot
nemo-automation-bot Bot requested a review from a team October 24, 2026 17:11
@github-actions github-actions Bot added the sla:triage-overdue Review assignment is over the one-business-day SLA label Oct 26, 2026
Signed-off-by: Igor Gitman <igitman@nvidia.com>
@Kipok

Kipok commented Oct 27, 2026

Copy link
Copy Markdown
Contributor Author

Merged upstream main 778a37bf3. pre-commit run --all-files and the PR diff whitespace check pass; the merge commit has a DCO sign-off. Also fixed the two reproduced core CI failures: the static Ray inventory now recognizes CompCodingSandboxResourcesServer as the explicit non-Ray entrypoint within the otherwise Ray-backed code_gen component. This preserves its runtime opt-out rather than enabling Ray to satisfy the inventory. Code-gen component tests: 77 passed, 1 optional integration test skipped. Ray inventory and resubmit core tests: 68 passed. The old unrelated terminal-shell shard failure is not claimed fixed by this update; fresh CI remains outstanding. No new live model rollout was run for the merge/inventory-only update.

Signed-off-by: Igor Gitman <igitman@nvidia.com>
@Kipok

Kipok commented Nov 3, 2026

Copy link
Copy Markdown
Contributor Author

Merged upstream main 483674910 in 5a22e53aaaf6ec6b1b4ff305422fd0cd62af95dc and reconciled the package-layout changes.

77 code-generation component tests pass (1 optional Docker test skipped), plus 28 Ray inventory/opt-out tests. The sandbox entrypoint remains independent of parent Ray, and patch targets follow main’s new module layout. The old terminal-shell failure was unrelated to this patch.

pre-commit run --all-files, diff whitespace checks and 724 public-API/import-boundary tests pass. This update merges main and adapts imports; no new model rollout or full local coverage run is claimed. Trusted CI needs vetter authorization for the new head.

@cmunley1

cmunley1 commented Nov 3, 2026

Copy link
Copy Markdown
Contributor

/ok to test 5a22e53

@cmunley1

cmunley1 commented Nov 3, 2026

Copy link
Copy Markdown
Contributor

/review

yaoyu-33
yaoyu-33 previously approved these changes Nov 4, 2026

@yaoyu-33 yaoyu-33 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks. A separate sandbox_app.py is the simplest way to get past the launcher's static Ray inventory. It also refuses to start without sandbox_provider, so a config slip can't quietly fall back to Ray. Deferring the Ray import to the Ray branch and declaring ReverifyMode.STATELESS both look right, and the 1362/1362 OpenSandbox regrade is good evidence.

One nit inline. CI is green apart from the Autobot check.

@yaoyu-33 yaoyu-33 added ready-to-merge PR is approved, current, and waiting for final CI or merge and removed needs-review PR is ready for code review and waiting on a reviewer labels Nov 4, 2026

Copy link
Copy Markdown
Contributor

💬 The non-blocking harness check failed: Kilocode’s model_error scenario didn’t produce the expected failure response, so conformance was marked unavailable. I haven’t established the cause or requested a rerun. I’m checking the probe evidence against current main; the core and server test jobs were still running in the last snapshot.

Copy link
Copy Markdown
Contributor

💬 The core/server tests and Nemo_CICD_Test passed at a1bc989d. The remaining non-blocking failure turns Kilocode’s model HTTP 400 into an agent HTTP 500. Merged #5377 adds the error-preservation fix; current main contains it, but this PR’s head doesn’t.

Exact-main reproduction remains unverified: main’s CI skipped the probes, and the managed Gym test profile doesn’t expose this conformance suite. That blocks a validated integration candidate. The next step is to incorporate the existing fix and validate conformance on the resulting head; rerunning the unchanged head wouldn’t address the missing code.

@yfw

yfw commented Nov 5, 2026

Copy link
Copy Markdown
Contributor

it's already at parity I believe, so we can probably remove the other path completely. @ananthsub @yfw any concerns with that? If not I can push this to this MR I guess

no concerns with me. it's simpler to maintain if we support only the sandbox path. @yfw what do you think?

Sounds good to me, no concerns from my end

Signed-off-by: Igor Gitman <igitman@nvidia.com>
@Kipok
Kipok requested a review from a team as a code owner November 5, 2026 20:10
@Kipok Kipok changed the title fix(code-gen): run sandbox verification without parent Ray refactor(code-gen): replace legacy Ray execution with sandboxes Nov 5, 2026
@Kipok

Kipok commented Nov 5, 2026

Copy link
Copy Markdown
Contributor Author

Updated to main 2ac7908c41438e025f763e8cbf00134ce2f80906; new head b799bc8148c521745b40bcc91956fe5837248f46.

Removed the old Ray execution branch completely, including its wrapper, unused batch-metric helpers and dependency. app.py is sandbox-only; sandbox_app.py is just a compatibility alias. The existing public-import review fix is retained. Validation: 81 code_gen tests passed (one optional Docker test skipped), 958 core/API tests passed, and full pre-commit passed. A fresh model-generated program passed three inputs in real OpenSandbox (reward 1); the wrong-answer control scored 0, neither was masked, and Ray was never imported. Updated the description and migration docs.

New-head CI is pending and may require qualified-maintainer trust authorization; local checks are not a claim that CI has passed.

Copy link
Copy Markdown
Contributor

💬 I verified that b799bc81 contains the merged Kilocode error-preservation fix. Lint, both component-lock checks, DCO, copyright, and secrets checks passed. No core/server or harness workflow is reported for this head yet, so the earlier conformance failure isn’t verified resolved.

@adil-a

adil-a commented Nov 5, 2026

Copy link
Copy Markdown
Contributor

/ok to test b799bc8

Copy link
Copy Markdown
Contributor

✅ CI is green on nvidia-nemo/gym PR #4958 at b799bc8148c5 — still watching.

Copy link
Copy Markdown
Contributor

💬 Kilocode’s conformance check passed at b799bc81 with no failed scenario checks (0/9), so the earlier conformance CI failure is now resolved. The PR remains open.

@ananthsub
ananthsub added this pull request to the merge queue Nov 5, 2026
@github-actions

github-actions Bot commented Nov 5, 2026

Copy link
Copy Markdown
Contributor

🔄 Merge queue validation started! You can track the progress here: https://github.com/NVIDIA-NeMo/Gym/actions/runs/42113477993

Merged via the queue into NVIDIA-NeMo:main with commit fd1fa27 Nov 5, 2026
69 checks passed

Copy link
Copy Markdown
Contributor

The PR is no longer an open ready-to-merge PR targeting main.

This branch was previously deployed

1 inactive deployment
public — b799bc81 Deployed Nov 5, 2026 by copy-pr-bot[bot] via release / finalize / notify #10478
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:environment Individual environments, benchmarks, verifiers, and environment-specific resources servers bug Something isn't working ready-to-merge PR is approved, current, and waiting for final CI or merge

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants