Repository navigation
feat(sandbox): verify code_gen submissions in sandboxes - #4607
Merged
Merged
Conversation
yfw
force-pushed
the
yifuw/sandbox-verify-code-gen
branch
from
October 14, 2026 19:24
4bc13be to
dd37714
Compare
yfw
marked this pull request as ready for review
October 16, 2026 10:14
Contributor
Author
|
/ok to test 6999591 |
hemildesai
reviewed
Oct 16, 2026
ananthsub
reviewed
Oct 16, 2026
Contributor
Author
|
/ok to test fc91e33 |
hemildesai
approved these changes
Oct 21, 2026
ananthsub
approved these changes
Oct 21, 2026
Add an opt-in sandbox path to code_gen: with sandbox_provider set, each verification runs the unchanged LCB harness in a fresh provider sandbox instead of a Ray worker, with the same per-test limits. Sandbox or runner failures return mask_sample with failure_kind provider_unavailable or verifier_error instead of a zero reward. The default network policy denies all egress. The runner sets a one-byte RLIMIT_CORE so hosts that pipe core dumps to systemd-coredump (as the AWS cells do) do not stall a crashing candidate for seconds and turn it into a timeout. The sandbox lifecycle lives in nemo_gym.sandbox.verification (run_verification_sandbox, SandboxVerificationConfig, SandboxVerificationError) so other verifiers can reuse it; the competitive_coding_challenges sandbox path will build on it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Yi-Fu Wu <yifu.wu@gmail.com>
Shows how to enable the sandbox path with a config file (provider config via config_paths, sandbox_provider, image), alongside the existing command-line overrides, and points the server config's comment at it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Yi-Fu Wu <yifu.wu@gmail.com>
- Delete request.json before grading: it holds every expected output, and candidates run in its directory with a real open(). - Give each verification its own work subdirectory, so providers that share /tmp between sandboxes (local, Apptainer) cannot mix up concurrent results. - Let ImportError from a missing provider SDK propagate instead of masking every sample as provider_unavailable. - Add scipy and pandas to the verifier image at the versions the code_gen venv resolves, and pin them in requirements.txt, so candidates that import them grade the same as on Ray. - README: add --model-type to the launch examples, note the stack hard limit, and clarify num_processes, CPU limits, network_policy support and masking. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Yi-Fu Wu <yifu.wu@gmail.com>
The runner cannot raise a zero hard limit to one byte, so the test failed instead of skipping on hosts that set it that way. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Yi-Fu Wu <yifu.wu@gmail.com>
…tils nemo_gym.sandbox holds only the sandbox lifecycle APIs and providers. The single-use verification helper is a verifier utility built on them, so it moves to nemo_gym/verifier_utils/sandbox_verification.py, next to agent_utils for CLI harnesses. The code is unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Yi-Fu Wu <yifu.wu@gmail.com>
The runner and the image's Dockerfile sat at the server root next to the server modules, so it was hard to tell what runs in the sandbox. Both now live in sandbox/: sandbox_runner.py runs only in the sandbox, and the Dockerfile builds its image. sandbox_execution.py names the upload set explicitly: the runner plus the three lcb_integration grading files the Ray path also runs. Inside the sandbox the files keep their paths, and no grading code changes. The README gains a table of what runs where. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Yi-Fu Wu <yifu.wu@gmail.com>
ananthsub
force-pushed
the
yifuw/sandbox-verify-code-gen
branch
from
October 21, 2026 21:51
fc91e33 to
7b52b4b
Compare
Contributor
|
/ok to test 7b52b4b |
Contributor
|
🔄 Merge queue validation started! You can track the progress here: https://github.com/NVIDIA-NeMo/Gym/actions/runs/39690556063 |
This branch was previously deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What and why
Part of #3216 (migrate in-scope resources servers to the Sandbox API). Running model-written code in remote sandboxes keeps it off the training cluster.
This adds an opt-in sandbox path to
code_gen. Withsandbox_providerset, each verification runs the unchanged LCB harness in a fresh provider sandbox instead of a Ray worker, with the same per-test time and memory limits. Without it, behavior is unchanged.nemo_gym/verifier_utils/sandbox_verification.pyprovidesrun_verification_sandbox,SandboxVerificationConfigandSandboxVerificationError. It starts a sandbox, uploads the harness and request, runs it and parses the result, and always stops the sandbox. The CCC sandbox path (stacked PR) reuses it./tmpbetween sandboxes (local, and Apptainer by default) can't mix up concurrent verifications.nemo-gym[sandbox]) raises instead of masking every sample.mask_samplewithfailure_kindprovider_unavailableorverifier_error, not a zero reward. Wrong answers, timeouts and crashes are still normal results.request.json, which holds the expected outputs, before running the candidate, so the sandbox exposes no more than a Ray worker does.RLIMIT_COREbefore running candidates. The AWS cells pipe core dumps tosystemd-coredumpwith an unlimited core limit, so every crash took 4–7 s to write a dump. Near a time limit, that turned crashes into timeouts. A zero limit doesn't help because the kernel ignores it for piped dumps; at exactly 1 byte it skips the dump.resources_servers/code_gen/sandbox/:sandbox_runner.py, uploaded with every verification, and the image'sDockerfile.sandbox_execution.pyruns in the server and names the upload set: the runner plus threelcb_integrationgrading files (__init__.py,correctness.py,testing_util.py), which the Ray path also runs. Nothing else is uploaded. The README has a table of what runs where.sandbox/Dockerfile, with the server venv's CPython build and numpy, scipy and pandas versions (scipy and pandas are pinned inrequirements.txttoo), so candidates that import them grade the same as on Ray. The config reads the image fromCODE_GEN_SANDBOX_IMAGE.Validation
resources_servers/code_genvenv) withCODE_GEN_SANDBOX_IMAGEset to an image built fromsandbox/: 73 passed. Without the image, the Docker test is skipped.tests/unit_tests/test_sandbox_verification.py: 11 passed. Regression tests that fail without their fixes:test_candidates_cannot_dump_core(RLIMIT_CORE),test_candidate_cannot_read_expected_outputs_from_the_request,test_each_call_gets_its_own_work_dirandtest_missing_provider_sdk_fails_loudly.pre-commit runon the changed files: clean.sandbox/:sandbox_execution.pyuploaded the runner from there and graded a correct answer, a correct answer that imports scipy, and a wrong answer as expected.localprovider, half right and half wrong: 0 wrong rewards.gym env resolve.Rollout evidence
The RL scale test above. Earlier, a 5-step GRPO smoke run on HSG, host mode vs sandbox mode, otherwise identical:
code_genreward 0.095 vs 0.096.Compatibility
Opt-in only. Existing configs, and runs without
sandbox_provider, are unchanged.🤖 Generated with Claude Code