Skip to content

Add Auto Ontology's ReAct agent as a Gym Agent Server, one sandbox per episode - #74

Draft
ffrujeri wants to merge 3 commits into
NVIDIA:mainfrom
ffrujeri:feat/auto-ontology-agent-server
Draft

ffrujeri wants to merge 3 commits into
NVIDIA:mainfrom
ffrujeri:feat/auto-ontology-agent-server

Conversation

@ffrujeri

@ffrujeri ffrujeri commented Nov 17, 2026 •

Copy link
Copy Markdown

Adds a third arm, react: Auto Ontology's ReAct agent as a NeMo Gym Agent Server, one sandbox per episode, on the policy model. The schema-only arm's exec_match grades it, so it compares directly with the control.

Why

The auto_ontology arm runs Auto Ontology inside the Resources Server's verify and ignores the policy's answer. The react arm runs the agent as Gym runs any agent: every model call goes through the policy's Model Server on the rollout's route, so Gym records it and a training run could use it.

What changed

  • Agent Server (responses_api_agents/auto_ontology_agent/). The seed starts a sandbox from the agent image and starts its catalog. The activation runs ontology_sql_eval/gym/react_runner.py in the sandbox, which calls Auto Ontology's run_episode as its ReactSQLAgent does for a SQL answer. The reply is the submitted query in a ```sql block. The close stops the sandbox.
  • Tasks. scripts/build_gym_tasks.py --arm react writes rows whose agent context holds the question, evidence and database. single_agent_turn runs them.
  • Image (docker/auto_ontology_agent/, scripts/build_agent_image.sh). Auto Ontology at the pinned commit, the BIRD dev databases, and their catalog, built once at image build time: schema, BIRD's column descriptions, embeddings and the semantic layer. The build fails when a model call failed or a database has no vectors; Auto Ontology logs both and goes on.
  • Network. Local Docker sandboxes run on the host network, because a host firewall often closes Gym's address to Docker's bridge. Each episode's catalog takes its own loopback port.
  • Pins.

Compatibility

  • auto_ontology arm. At a630ebd, stream_agent_response runs the ReAct agent plus a format step, not the LangGraph agent. The lock drops LangGraph, MLflow and their dependencies with it. Results of this arm before and after this PR are not comparable.
  • Gym's per-server venvs. With nemo-gym pinned to a Git commit, gym env test and the server venvs cannot resolve nemo-gym. The Agent Server README says how to install an editable Gym checkout at the pinned commit instead.

Results

The first 200 BIRD dev tasks, aws/anthropic/eccn-claude-opus-5-5, one run per arm, both graded by exec_match:

Arm Accuracy Simple / moderate / challenging W&B run
schema_only (control) 86.5% 113/119, 53/68, 7/13 dgr022t9
react 82.0% 107/119, 49/68, 8/13 c5gur860

The same arms on two more models:

Model Arm Accuracy Simple / moderate / challenging W&B run
openai/openai/eccn-gpt-6-astra schema_only (control) 66.5% 94/119, 35/68, 4/13 u24ste3y
openai/openai/eccn-gpt-6-astra react 66.5% 88/119, 42/68, 3/13 2oxmktun
nvidia/nvidia/eccn-nemotron-3.5-lightning schema_only (control) 47.5% 73/119, 21/68, 1/13 oezab6zy
nvidia/nvidia/eccn-nemotron-3.5-lightning react 43.0% 64/119, 21/68, 1/13 3r5hpwex

On both models the arms differ within noise (Astra 17 and 17 tasks, p = 1.0; Lightning 34 and 25, p ≈ 0.30).

NeMo Gym runs the same agent in NVIDIA-NeMo/Gym#6289 with its own catalog image and grader (bird_sql): 79.5% with Opus. The two runs differ on 13 tasks (4 and 9), within noise (p ≈ 0.27).

Two findings from these runs, not fixed here:

  • exec_match puts SELECT 1 in place of a missing prediction. It credits a missing answer whenever the gold result is a single 1 (tasks 0, 16 and 62 in this run).
  • scripts/seed/seed_bird.py parses BIRD's description CSVs from raw.splitlines(), which drops the line breaks inside a quoted cell: a multi-line value description has its lines joined with nothing between them.

Validation

  • uv run pytest: 102 passed.
  • ruff check . and ruff format --check .: clean.
  • uv lock --check: the lock is current.
  • The real runs above: 200 episodes per arm and model, no infrastructure failures, every sandbox stopped.

ffrujeri and others added 3 commits November 17, 2026 05:14
The Auto Ontology Agent Server needs Gym's agent sessions and sandbox
library, which no Gym release has yet (0.7.0 predates them), so pin Gym's
main at 5d6429f5 in pyproject.toml and in both servers' requirements.
Gym's main makes Ray, wandb and mlflow optional extras, so the lock drops
them; no server here enables Ray.

Pin Auto Ontology to its main at a630ebd, its ReAct agent, plus the
commit of NVIDIA/auto-ontology#543, which accepts the openai version
Gym pins (2.45.0). Move to NVIDIA/auto-ontology once #543 merges.

test_server_requirements_pin_the_parents_nemo_gym now accepts a full Git
commit as an exact pin.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Felipe Vieira Frujeri <ffrujeri@nvidia.com>
…isode

Run Auto Ontology's ReAct agent as a NeMo Gym Agent Server, on the policy
model, so its rollouts compare with the schema-only arm's and can train.
The treatment arm instead runs Auto Ontology in a Resources Server's
verify() and discards the policy's output.

- responses_api_agents/auto_ontology_agent: at the seed, start a sandbox
  from the agent image and its catalog Postgres; at the activation, run
  one ReAct episode there and answer with the submitted query in a sql
  block; at the close, stop the sandbox. The process imports only Gym.
- ontology_sql_eval/gym/react_runner.py: the in-sandbox runner. It calls
  run_episode as Auto Ontology's ReactSQLAgent does (SQL answers only,
  evidence apart), with its reasoning client on the policy's Gym Model
  Server URL, on the rollout's route.
- docker/auto_ontology_agent and scripts/build_agent_image.sh: the image,
  with BIRD dev and a catalog built by main.py --skip-eval. The keys reach
  that step as a BuildKit secret only. The build fails when a database has
  no vectors, since main.py logs a missing key and goes on.
- The config pairs the agent with schema_only_sql through
  single_agent_turn, so both arms are graded by the same exec_match.
- build_gym_tasks.py --arm react writes rows with the task in their
  agent_context and no schema dump.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Felipe Vieira Frujeri <ffrujeri@nvidia.com>
…n port

A sandbox on Docker's bridge could not reach the policy's Model Server on a
host whose firewall closes it to the bridge: every episode's first model call
timed out, and the run scored only what the grader's SELECT 1 filler matched.
The sandboxes now run on the host network. They share loopback, so the agent
starts each sandbox's catalog on a random port, tries another when one is
taken, and gives the episode its port.

Also:
- build-catalog fails when a model call fails after Auto Ontology's
  retries: a 429 burst during the compile otherwise left columns and joins
  undescribed without failing the build.
- The image copies the scripts an episode runs after the catalog, so
  editing them no longer rebuilds it.

Smoke: 5 of the first BIRD dev questions on Opus 5.5, 5 correct, each in
2 or 3 tool calls, every model call captured on its rollout's route.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Felipe Vieira Frujeri <ffrujeri@nvidia.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant