Repository navigation
feat(officeenv): add the OfficeEnv resources server - #5009
Conversation
664808c to
4f25da6
Compare
b6a008c to
aaaca21
Compare
svcnvidia-nemo-ci
left a comment
There was a problem hiding this comment.
HOLD — multi-worker startup is broken, and the private dependency prevents credential-free installation.
Reviewed all 12 changed paths and their directly affected startup, agent, session, MCP, installation, and data-contract paths. This was static inspection; no tests were executed. OfficeEnv's external implementation was unavailable, and long rollout JSONL lines were limited to bounded previews, so recorded rewards were not independently verified.
NOTE — Surface OfficeEnv in the published documentation. The integration adds substantial user-facing functionality, but the published environment catalog contains no OfficeEnv entry, and the changes add no Fern instructions. The pinned rubric requires published documentation for notable user-facing features. Please add a discoverable OfficeEnv entry and concise usage instructions, linking the detailed component README. Preserve the distinction between OfficeEnv's simulated mcp exposure and Gym's MCP transport, the absolute artifact-store paths, the opt-in length penalty, and the examples' smoke-test—not benchmark—status. Proposed acceptance: the published catalog reaches the instructions; launch/evaluate examples match the shipped configuration; links and make docs-check pass. These checks were not run.
Note: formal approval is not available for this request, so this review is posted as a comment only.
Review generated by codex. This was a full review. Request a full review with /review reset=true.
|
|
||
|
|
||
| if __name__ == "__main__": | ||
| OfficeEnvResourcesServer.run_webserver() |
There was a problem hiding this comment.
BLOCKER — Initialize the module-level application in spawned workers. Setting officeenv_resources_server.resources_servers.officeenv.num_workers=2 takes the inherited multi-worker startup path, which tells Uvicorn to import resources_servers.officeenv.app:app. This module only handles __name__ == "__main__" and never defines that module-level app, so the workers cannot start. The in-process tests call setup_webserver() directly and cannot catch this deployment failure.
Could you add the existing worker-entrypoint hook? Add from nemo_gym.utils import is_nemo_gym_fastapi_entrypoint with the imports, then replace the final block with:
if __name__ == "__main__":
OfficeEnvResourcesServer.run_webserver()
elif is_nemo_gym_fastapi_entrypoint(__file__):
app = OfficeEnvResourcesServer.run_webserver()This matches the existing resources-server entrypoint contract. In a worker, run_webserver() returns the configured application; an ordinary test import must still avoid starting a server, and single-worker script startup must remain unchanged.
Proposed acceptance, not executed: with the dependency installed and valid model configuration, launch OfficeEnv with two workers and confirm readiness, then seed, call a tool, and verify using the returned cookie. Repeat with one worker. Importing this module outside a Gym worker must not invoke run_webserver(). Please verify the suggested code before adopting it.
There was a problem hiding this comment.
Confirmed and fixed in 71edeaeda6 with the hook you suggested, as example_single_tool_call has it. With num_workers above one, Uvicorn imports resources_servers.officeenv.app:app in each worker. The module defined no app, so no worker could start.
Each worker holds the episodes it seeded, and Gym's session routing (on for resources servers) forwards each episode's requests to that worker, by session cookie or MCP token.
I ran your acceptance as written, starting the server as Gym does (python app.py, with the config in NEMO_GYM_CONFIG_DICT):
- Before the fix, two workers never served. Uvicorn logged
Error loading ASGI app. Attribute "app" not found in module "resources_servers.officeenv.app", and the server exited. - With the fix, I ran 16 concurrent episodes with 1, 2 and 4 workers. Each episode was seeded, made a Slack post that meets the task's authored fault, made a second post that succeeds, and was verified, all on its own cookie. All 16 reached reward 1.0, unmasked, at every worker count.
- Every request opened its own connection, so most of an episode's requests reached a worker that had not seeded it. The sessions were spread 6/10 over two workers and 2/4/5/5 over four.
- Importing the module outside a worker starts no server: the test suite imports it throughout.
I haven't committed a multi-worker test. Like main's other servers with this hook, it relies on core's session-routing e2e for the routing itself. I'm happy to add one if you'd like it here.
| @@ -0,0 +1,2 @@ | |||
| -e nemo-gym[dev] @ ../../ | |||
| officeenv @ git+ssh://git@gitlab-master.nvidia.com:12051/dmolinari/officeenv.git@v1.9.0 | |||
There was a problem hiding this comment.
RISK — Replace the private dependency before merging. A fresh installation requires SSH access to an unpublished internal repository, as the README confirms. The venv installer installs this requirement unconditionally. Users without that access cannot launch OfficeEnv; the changed-server job and scheduled server suite also include it. The draft status appropriately acknowledges this but does not resolve the merge-time failure.
Please replace the pin with an approved public, reproducibly pinned release providing the required 1.9.0 APIs, and remove the internal URL/access instructions from the README. Protected public-repository guidance also prohibits internal URLs. Public replacement coordinates are unavailable here, so an exact dependency patch would be speculative.
Preserve: artifact resolution, row/tool compatibility, and real server-test coverage; do not merely remove the dependency or skip CI. Proposed acceptance, not executed: from a clean checkout without internal credentials or cached OfficeEnv packages, gym env test --resources-server officeenv +should_validate_data=true must install and pass. Retain empty tool_drift for all five version-matched rows, unchanged workbook/text retrieval through both seed shapes, and rejection of tampered or missing artifacts.
| class TaskData(BaseModel): | ||
| model_config = ConfigDict(extra="allow") | ||
|
|
||
| officeenv: dict[str, Any] = Field( |
There was a problem hiding this comment.
NOTE — Suggestion: enforce the duplicated TaskSpec field contract. officeenv is declared independently here and in OfficeEnvSeedSessionRequest. The repository drift gate and example seed test exercise valid rows, but do not compare these declarations. Making the field optional in only one declaration would leave those checks satisfied. This is a missing drift guard, not a demonstrated current seed failure; the pinned rubric calls for enforcement when declarations are parallel.
A focused addition to the server tests would protect the current invariant:
def test_task_spec_wire_contract_matches_schema() -> None:
from typing import Any
from resources_servers.officeenv.app import OfficeEnvSeedSessionRequest
from resources_servers.officeenv.task_data import TaskData
for model in (TaskData, OfficeEnvSeedSessionRequest):
field = model.model_fields["officeenv"]
assert field.annotation == dict[str, Any]
assert field.is_required()
assert model.model_config["extra"] == "allow"Preserve: optional provenance fields, allowance of extra row fields, and delegation of TaskSpec-content validation to OfficeEnv rather than copying its external schema. Proposed acceptance: this test passes now and fails when either declaration alone changes the field's type, requiredness, or unknown-field policy. Existing committed-row validation and unstamped-row seeding must remain valid. The suggested test was not executed and should be verified before adoption.
There was a problem hiding this comment.
Added in caa6268b0b as test_the_row_schema_and_the_seed_request_declare_the_taskspec_alike in tests/test_app.py, following your sketch.
- It passes now.
- It fails when either declaration alone makes the field optional, changes its type, or stops allowing extra fields. I checked each case by editing one declaration at a time.
- The provenance fields, the allowance of extra row fields, and OfficeEnv's own TaskSpec validation are unchanged.
#5009 now pins OfficeEnv 1.10.0, whose sandbox examples hold four more tasks: sandbox_budget_variance, sandbox_statement_aging, sandbox_stock_reorder and sandbox_timesheet_billing, on files in Box, SharePoint, OneDrive and a Gmail attachment. sandbox/prepare_rows.py writes data/sandbox_pilot.jsonl again with it: nine rows, the five it held changed only in their version stamp. The test of those rows asked every task for Drive's download tool; it now asks each for user_complete_task and for a tool that reads a stored file in any app (CONTENT_DOWNLOADS and ATTACHMENT_DOWNLOADS), as the new tasks read theirs from the other four. The README no longer counts the tasks. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Daniele Molinari <dmolinari@nvidia.com>
8a118d2 to
caa6268
Compare
#5009 now pins OfficeEnv 1.10.0, whose sandbox examples hold four more tasks: sandbox_budget_variance, sandbox_statement_aging, sandbox_stock_reorder and sandbox_timesheet_billing, on files in Box, SharePoint, OneDrive and a Gmail attachment. sandbox/prepare_rows.py writes data/sandbox_pilot.jsonl again with it: nine rows, the five it held changed only in their version stamp. The test of those rows asked every task for Drive's download tool; it now asks each for user_complete_task and for a tool that reads a stored file in any app (CONTENT_DOWNLOADS and ATTACHMENT_DOWNLOADS), as the new tasks read theirs from the other four. The README no longer counts the tasks. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Daniele Molinari <dmolinari@nvidia.com>
#5009 now pins OfficeEnv 1.10.0, whose sandbox examples hold four more tasks: sandbox_budget_variance, sandbox_statement_aging, sandbox_stock_reorder and sandbox_timesheet_billing, on files in Box, SharePoint, OneDrive and a Gmail attachment. sandbox/prepare_rows.py writes data/sandbox_pilot.jsonl again with it: nine rows, the five it held changed only in their version stamp. The test of those rows asked every task for Drive's download tool; it now asks each for user_complete_task and for a tool that reads a stored file in any app (CONTENT_DOWNLOADS and ATTACHMENT_DOWNLOADS), as the new tasks read theirs from the other four. The README no longer counts the tasks. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Daniele Molinari <dmolinari@nvidia.com>
#5009 now pins OfficeEnv 1.10.0, whose sandbox examples hold four more tasks: sandbox_budget_variance, sandbox_statement_aging, sandbox_stock_reorder and sandbox_timesheet_billing, on files in Box, SharePoint, OneDrive and a Gmail attachment. sandbox/prepare_rows.py writes data/sandbox_pilot.jsonl again with it: nine rows, the five it held changed only in their version stamp. The test of those rows asked every task for Drive's download tool; it now asks each for user_complete_task and for a tool that reads a stored file in any app (CONTENT_DOWNLOADS and ATTACHMENT_DOWNLOADS), as the new tasks read theirs from the other four. The README no longer counts the tasks. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Daniele Molinari <dmolinari@nvidia.com>
|
/ok to test f239b09 |
|
/review |
|
🌿 Preview your docs: https://nvidia-preview-pr-5009-aa1206fd82d2.docs.buildwithfern.com/nemo/gym |
svcnvidia-nemo-ci
left a comment
There was a problem hiding this comment.
SHIP WITH CARE — the worker-startup and TaskSpec drift-guard findings are fixed; the private dependency and published-documentation findings remain unresolved.
I verified the worker entrypoint against the current runner: workers now receive the module-level application, while ordinary imports and single-worker startup retain their existing behavior. The new TaskSpec contract test checks both declarations for the required dictionary type and allowance of extra fields.
This follow-up checked the prior findings, identifiable fixes, and directly affected contracts. The previously reviewed tree wasn't provided, so I couldn't compare the complete inter-review delta. OfficeEnv's external implementation wasn't available. This was static inspection; I didn't execute installation, tests, or model rollouts. No qualifying new regression was established.
NOTE — Suggestion: surface OfficeEnv in the published documentation. This is the unresolved body finding from review 5721205156. The current Agentic / Tool Use catalog still has no OfficeEnv entry, and the changeset adds no Fern instructions. Readers browsing the published catalog therefore cannot discover the integration or its launch instructions. The pinned rubric requires published documentation for notable user-facing features.
Could you add a catalog row and a short published usage section, linking the component README for advanced configuration? A small suggested addition at the end of that catalog table is:
| [officeenv](#officeenv) | Deterministic connected-app tasks | Rule-based (final-state checks) | |
### OfficeEnv
With your OpenAI model configuration set, start the environment and collect the example rollouts:
```bash
gym env start --resources-server officeenv --model-type openai_model
gym eval run --no-serve --agent officeenv_simple_agent \
--input resources_servers/officeenv/data/example.jsonl \
--output results/officeenv_rollouts.jsonl
```
These five examples are integration smoke tests, not a benchmark. OfficeEnv's simulated `mcp` exposure is distinct from Gym's opt-in MCP transport. Artifact-store paths must be absolute. The length penalty is off by default and intended for training only.This preserves the existing exposure, artifact-path, reward, and smoke-test distinctions. Proposed acceptance, not executed: the published catalog reaches the usage section and detailed README; the commands match the shipped configuration and example dataset; the links and make docs-check pass. Please verify the suggested addition before adopting it.
Review generated by codex. This was a findings-only review. Request a full review with /review reset=true.
| -e nemo-gym[dev] @ ../../ | ||
| # OfficeEnv is not public yet. It will be released at a later date, and this pin will then point to that release; | ||
| # until then it installs from NVIDIA's internal GitLab. Point of contact: Daniele Molinari <dmolinari@nvidia.com> | ||
| officeenv @ git+ssh://git@gitlab-master.nvidia.com:12051/post-training/officeenv.git@v1.10.0 |
There was a problem hiding this comment.
RISK — The credential-free installation failure remains unresolved. This is the same failure as prior comment 4430467912 in review 5721205156. From a clean checkout without internal repository access, gym env test --resources-server officeenv +should_validate_data=true still requires this private SSH dependency. The README confirms that prerequisite. The installer installs the requirement before pytest runs, so users without access cannot install or test the integration. Changing the repository coordinates and adding an explanatory comment doesn't resolve that failure.
The changed-server job and scheduled suite include this server and require the same access. I haven't established whether those runners have credentials. Protected public-content guidance also prohibits internal URLs.
Could you replace this requirement with an approved public, reproducibly pinned release providing the current EpisodeService and ArtifactStore APIs, then remove the internal URL/access instructions from this manifest and the README? The invariant is that a fresh public installation can install and test the actual integration without internal credentials—not by removing the dependency or skipping CI. Public replacement coordinates aren't supplied, so an exact dependency patch would be speculative.
Preserve and check: all five version-matched rows must seed successfully with empty tool_drift, as asserted by the compatibility test. The artifact tests must retain exact workbook/text retrieval through both seed shapes, ordered-store lookup, artifact_mismatch for tampered content, and missing_artifact with no stored episode for unavailable content; rows without references must still seed. The fault-then-success episode must still finish at reward 1.0, unmasked.
Proposed acceptance, not executed: with no internal credentials, cached OfficeEnv package, or prebuilt server venv, the command above installs and passes, including those assertions.
#5009 now pins OfficeEnv 1.10.0, whose sandbox examples hold four more tasks: sandbox_budget_variance, sandbox_statement_aging, sandbox_stock_reorder and sandbox_timesheet_billing, on files in Box, SharePoint, OneDrive and a Gmail attachment. sandbox/prepare_rows.py writes data/sandbox_pilot.jsonl again with it: nine rows, the five it held changed only in their version stamp. The test of those rows asked every task for Drive's download tool; it now asks each for user_complete_task and for a tool that reads a stored file in any app (CONTENT_DOWNLOADS and ATTACHMENT_DOWNLOADS), as the new tasks read theirs from the other four. The README no longer counts the tasks. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Daniele Molinari <dmolinari@nvidia.com>
aa1206f to
f42e4da
Compare
|
Thanks for the update. One request on the lock exemption: rather than adding an exception to
That keeps the lock rule the same for every component, avoids the core change, and nothing internal ends up in the lock. Once OfficeEnv is published it moves into the lock like any other dependency. |
…OfficeEnv apart until it is published Main now requires every server to declare its dependencies in a committed pyproject.toml and uv.lock (#4837). OfficeEnv is not published yet, and a lock that held it would name NVIDIA's internal repository, which CI cannot reach and the guardwords check refuses. The server's pyproject.toml and uv.lock now list only its public dependencies (nemo-gym[dev], declared and locked as the other servers with that dependency do), and requirements.txt is gone. OfficeEnv installs apart, into the venv Gym builds, with the step the README gives, from the repository OFFICEENV_GIT_URL names; Gym's later builds only add to that venv. Without OfficeEnv, app.py stops with an ImportError that names the step, which a test checks with OfficeEnv hidden; the tests that need OfficeEnv skip, so a venv without it still runs one test rather than none. Once OfficeEnv is published it moves into pyproject.toml and the lock like any other dependency. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Daniele Molinari <dmolinari@nvidia.com>
#5009 now pins OfficeEnv 1.10.0, whose sandbox examples hold four more tasks: sandbox_budget_variance, sandbox_statement_aging, sandbox_stock_reorder and sandbox_timesheet_billing, on files in Box, SharePoint, OneDrive and a Gmail attachment. sandbox/prepare_rows.py writes data/sandbox_pilot.jsonl again with it: nine rows, the five it held changed only in their version stamp. The test of those rows asked every task for Drive's download tool; it now asks each for user_complete_task and for a tool that reads a stored file in any app (CONTENT_DOWNLOADS and ATTACHMENT_DOWNLOADS), as the new tasks read theirs from the other four. The README no longer counts the tasks. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Daniele Molinari <dmolinari@nvidia.com>
f42e4da to
461815c
Compare
|
/ok to test 461815c |
|
🔄 Merge queue validation started! You can track the progress here: https://github.com/NVIDIA-NeMo/Gym/actions/runs/41920399486 |
|
🔄 Merge queue validation started! You can track the progress here: https://github.com/NVIDIA-NeMo/Gym/actions/runs/41920845141 |
|
🔄 Merge queue validation started! You can track the progress here: https://github.com/NVIDIA-NeMo/Gym/actions/runs/41937686420 |
|
🔄 Merge queue validation started! You can track the progress here: https://github.com/NVIDIA-NeMo/Gym/actions/runs/41937735853 |
#5009 now pins OfficeEnv 1.10.0, whose sandbox examples hold four more tasks: sandbox_budget_variance, sandbox_statement_aging, sandbox_stock_reorder and sandbox_timesheet_billing, on files in Box, SharePoint, OneDrive and a Gmail attachment. sandbox/prepare_rows.py writes data/sandbox_pilot.jsonl again with it: nine rows, the five it held changed only in their version stamp. The test of those rows asked every task for Drive's download tool; it now asks each for user_complete_task and for a tool that reads a stored file in any app (CONTENT_DOWNLOADS and ATTACHMENT_DOWNLOADS), as the new tasks read theirs from the other four. The README no longer counts the tasks. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Daniele Molinari <dmolinari@nvidia.com>
#5009 now pins OfficeEnv 1.10.0, whose sandbox examples hold four more tasks: sandbox_budget_variance, sandbox_statement_aging, sandbox_stock_reorder and sandbox_timesheet_billing, on files in Box, SharePoint, OneDrive and a Gmail attachment. sandbox/prepare_rows.py writes data/sandbox_pilot.jsonl again with it: nine rows, the five it held changed only in their version stamp. The test of those rows asked every task for Drive's download tool; it now asks each for user_complete_task and for a tool that reads a stored file in any app (CONTENT_DOWNLOADS and ATTACHMENT_DOWNLOADS), as the new tasks read theirs from the other four. The README no longer counts the tasks. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Daniele Molinari <dmolinari@nvidia.com>
…urces_servers/officeenv_sandbox The sandbox mode the OfficeEnv server ran when sandbox_provider was set is now a server of its own: resources_servers/officeenv_sandbox, whose OfficeEnvSandboxResourcesServer subclasses OfficeEnvResourcesServer. What a sandboxed episode does is unchanged. The OfficeEnv server keeps #5009's routes and options, plus GET /task_state and close_tools_on_task_end for its Environment Server, and goes back to nemo-gym[dev]: the sandbox providers' libraries install with the new server only. - resources_servers/officeenv/app.py goes from 1,860 to 473 lines (371 on main). Its seed, episode call and verification keep hook points the subclass extends: _new_episode, _unreachable with a workspace, _refuse_after_task_end and _publish_task_state around _episode_call, and _verification. - resources_servers/officeenv_sandbox holds the sandbox lending and shutdown, the cli and api_search surfaces, the workspace-mode host and file API, the guarded routes and per-process secret, and the seed turns. Its configs, images, rows, README sections and tests moved from the OfficeEnv server's folder. The configs name the server officeenv_sandbox (++oe.resources_servers.officeenv_sandbox.<option>), where sandbox_provider is required. An Agent's seed there gets no sandbox, as before. - The new server has its own lock, task-data schema (sandbox_needs), an install test, and example data: the first five pilot rows, Pi's rollouts of them with gpt-5.5 on Docker sandboxes (all at reward 1.0), and their metrics. Gym's README lists it. The OfficeEnv server's tests are 41 and the sandboxed server's 134 (the 133 that moved, and its install test). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Daniele Molinari <dmolinari@nvidia.com>
What
Adds
resources_servers/officeenv, the resources server for OfficeEnv: a deterministic simulator of connected SaaS apps (107 apps at v1.10.0: Slack, Gmail, Google Calendar/Sheets/Drive, HubSpot, Jira, Salesforce, Zendesk, ...). A row'sofficeenvtask spec names the connected apps and the data they hold, the acting user, how the apps reach the policy (tools,api_search,cliormcpexposure), the faults and data injections that happen during the episode, and the final-state checks that produce the reward.The environment is the
officeenvpackage (pydantic and the standard library), atv1.10.0, installed as the Dependency section says. The server is a thin wrapper that maps Gym's routes onto OfficeEnv's framework-neutralEpisodeService(seed,call,verify,catalog) and adds Gym's own response fields:/run, behind thelegacy_agentEnvironment Server inconfigs/officeenv.yaml): the row goes to/seed_session, each function call toPOST /<tool_name>with its arguments as the body, the row and the response to/verify. App-level errors are tool outputs the model sees (200). A body that is not a JSON object (400) and a call without a session (404,ResourcesSessionLostErrorwithfailure_kind: session_lost) are integration failures.ResourcesSeedSessionRequestseeds the OfficeEnv session under itsresources_session_id, which the session cookie also carries. A repeated seed for the same episode returns the same session; a seed for another episode is refused with 409 (ResourcesSessionConflictError)./close_sessionfrees a world that verification has not, and a seed of a closed ID is refused while Gym remembers the close.retire_session_statefrees the world too. The tests runcheck_resources_session_contract, including its state-release and masked-verify-after-close probes.expose_tools_over_mcp: true): every tool of OfficeEnv'stoolsexposure is served through the catch-all tool route, and each episode's MCP token admits only that episode's tools. MCP clients read tool definitions from the endpoint, not from the row, so a seed whose tools differ from what the endpoint serves (a row in another exposure mode or with another tool-name separator) fails with 400unsupported_exposure./verify: OfficeEnv's reward, per-check verdicts, metrics with flat numeric copies for aggregation, scenario and integration logs, and the grading version. A rollout without a session to grade gets reward 0 withmask_sample: trueandfailure_kind: session_lost(lost_session_verification).artifacts_dirs): a row may name its TaskSpec's large strings by SHA-256 instead of carrying them (OfficeEnv's artifact format,officeenv.artifacts), and a corpus is one directory,<corpus>/<rows>.jsonlbeside<corpus>/artifacts/. The server hands the stores (absolute paths, searched in order) to OfficeEnv's episode service, which reads the references back at seed, checks each against its hash and size, and refuses a row whose artifacts cannot be read with 400 (missing_artifact,artifact_mismatchorinvalid_artifact) before anything is seeded, for an Agent's seed and an Environment Server's alike. The README describes the corpus layout.length_penalty.enabled, off by default): AutomationBench's, reported as its own term inreward_components(aBaseMultiRewardVerifyResponse), so a trainer can schedule it separately (NeMo RLgrpo.phasic_reward_components).num_workers): above one, Uvicorn serves the app from each worker (the module-levelapp), and Gym's session routing sends each episode's requests to the worker that seeded it.The example rows are five of OfficeEnv's built-in examples, one per exposure mode plus a task with a scripted user.
task_data.pydeclares the row's task fields for Gym's data tooling and the task-data drift gate: the TaskSpec (the only field the wire requires; OfficeEnv validates it at seed), the emitting version, and thetask_id.Dependency
OfficeEnv will be released at a later date. Until then, the server's
pyproject.tomlanduv.lockhold only its public dependencies, as main's lock rule asks (#4837), and OfficeEnv 1.10.0 installs apart, from NVIDIA's internal GitLab, with a README step that takes the repository's git URL fromOFFICEENV_GIT_URL. Without OfficeEnv the server stops at start with an error naming that step, and its tests skip, so CI builds the venv and runs the suite without it. Once the release is out, OfficeEnv moves intopyproject.tomland the lock. Point of contact: Daniele Molinari (dmolinari@nvidia.com).Validation
Linux aarch64, Python 3.13, OfficeEnv 1.10.0 (the
v1.10.0tag's commit06e65854), on main83b5aaf08c(2026-11-04); earlier heads also on macOS arm64. The rebase onto83b5aaf08cleaves the 12 commits approved atf239b09109unchanged (git range-diff). One commit follows: it moves the server to locked public dependencies (pyproject.tomlanduv.lock) and installs OfficeEnv apart, which also keeps the internal host, which the guardwords check flagged, out of the repository.gym env test --resources-server officeenv +should_validate_data=true: 31 passed with OfficeEnv installed, and the data is validated. Without OfficeEnv the guard's test passes, the other tests skip, and the data still validates.python app.py, the config inNEMO_GYM_CONFIG_DICT) with 1, 2 and 4 workers, and 16 concurrent episodes for each: seed, a Slack post that meets the task's authored fault, a second that succeeds, and verify, each episode on its own session cookie. All 16 reach reward 1.0, unmasked, with each worker count. Every request opens its own connection, so most of an episode's requests reach a worker that did not seed it; the sessions were spread 6/10 over two workers and 2/4/5/5 over four. Before the worker fix, two workers never served: Uvicorn found noappinresources_servers.officeenv.app, and the server exited.pre-commitover every file this PR changes: all hooks pass, import-linter included (the guardwords hook skips itself without the private catalog).gym dataset collate "+config_paths=[resources_servers/officeenv/configs/officeenv.yaml]" +mode=example_validation:data/example_metrics.jsoncollated again for the rows 1.10.0 emits; it moves only in the tools' statistics (schedule_kickoff's Calendar tools takesend_updatesin 1.10.0), and a second collate into a fresh folder finds it up to date.detect-secrets-hook1.5.0 with.github/workflows/config/.secrets.baselineover every file this PR changes: no findings.nemo_gym/. Gym's core unit suite (pytest tests -m "not sandbox", thedevextra only) fails here only as on main83b5aaf08citself, in the same venv: the same 28 tests (eval resubmission, Harbor's compose overlay and run summary, reward profiles, outcome policies, the dataset orchestrator, the OpenSandbox scripts, the CI environment, the benchmark listing, failure reports) and 5 test modules that need packages the venv lacks. That includestests/unit_tests/data/test_task_data.py(the officeenv schema checks and the drift gate over committed rows),tests/unit_tests/test_public_api_facades.py(the server's imports come from the packages' public APIs) andtests/unit_tests/tooling/test_component_dependencies.py(main's lock rule).Test coverage: Agent seeds through tool calls and verify; every exposure mode reaching the apps through the tool route; integration-failure statuses; lost sessions (404 tool calls, masked verify); artifact stores (a moved corpus seeds from the store beside its rows through both seed kinds, stores are searched in order, a row whose artifacts no store holds is refused and seeds nothing, relative stores are refused); the example rows matching the pinned release (no tool drift); the row schema and the seed request declaring the TaskSpec alike; the Environment Server session contract (
check_resources_session_contract), episode binding, freeing on close and retire; MCP grants (tools/listnarrowed per episode,tools/callthrough the catch-all route, verify afterwards), the agent seed's MCP token, and refusal of unserved tools; the catalog matching what rows show the model; the length penalty and turn counting.Rollout evidence
Real model smoke runs on this branch against OfficeEnv 1.9.0 (the example rows have since been emitted again by 1.10.0, which changes only their version stamp and
schedule_kickoff's Calendar tools), before its rebase onto main (the rollouts replay unchanged on it, above),azure/openai/gpt-5.5through the NVIDIA inference gateway, the five example rows (emitted by 1.9.0), concurrency 2:officeenv_simple_agent)single_agent_turn_legacy,direct_http)data/example_rollouts.jsonl; every rollout was graded by 1.9.0.schedule_kickoffturns on where the model books the meeting: the Agent-seed run booked 14:00, after a meeting injected during the episode, and passes every check; the Environment Server run booked 13:30, over that meeting, and failsgcalendar.no_conflicts, as the 1.7.0 and 1.8.0 runs did.open_dm, an unknown HubSpot op). No rollout met 1.9.0's new refusal of a Gmail attachment sent by file name alone (T50). There were no internal errors, timer failures, tool drift, failed rollouts or masked samples.single_agent_turn_legacywithresources_tool_transports: [direct_http], so every episode went through typed seed, cookie-scoped tool calls, verify and close.Compatibility
A new server: nothing else changes except the generated row in the root README's environment table.
configs/officeenv.yamlkeeps the internal branch's instance names (officeenv_resources_server,officeenv_simple_agent,officeenv_environment_server), so existing overrides such as++env.nemo_gym.officeenv_resources_server.resources_servers.officeenv.length_penalty.enabled=truekeep working.🤖 Generated with Claude Code