Repository navigation
Conversation
Add a Mermaid diagram of today's paired run (seed, ingest, compile, two arms, deterministic verifier, report) and of the planned ReAct-on-Gym setup, marked as not built. State the paired-difference method, how to choose the answer architecture through the pinned Auto-Ontology revision, link the WWI seed script Auto-Ontology's Quickstart uses, and leave a TODO for tokens-per-question reporting. Signed-off-by: Cameron Afzal <cafzal@nvidia.com>
Merged
1 task done
Signed-off-by: Cameron Afzal <cafzal@nvidia.com>
The README keeps the Now diagram and links to the planned ReAct-on-Gym flow, which moves to docs/architecture.md beside it. Signed-off-by: Cameron Afzal <cafzal@nvidia.com>
Signed-off-by: Cameron Afzal <cafzal@nvidia.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Docs only. Adds a Now Mermaid diagram to the README (how a run works today) and a new
docs/architecture.mdwith Now plus Next (Auto-Ontology's ReAct tools on NeMo Gym), linked from the README, matching the Now/Next labels in NVIDIA/auto-ontology#489.main.pystages andresources_servers/README.md.ontology_sql_eval/gym/exec_match.py). The benchmark list matches the README table: BIRD, BEAVER, FDABench-Lite, WWI.auto-ontology-serverrevision inpyproject.tomlhas.mainpinsdd78d06, the fixed LangGraph chain. The branchfeat/react-ontology-agent_cleanpins the ReAct library branch. The README says how to repin,uv sync, and rebuild withGYM_REBUILD_VENV=1.retrieve,run_sql,submit) directly, and the verifier's score becomes the reward. Nothing in either repo implements this yet; the ReAct agent's limits are written to be usable as an RL environment (react/config.py,react/loop.pyon auto-ontology #488). Today the treatment arm's Gym policy call is made and discarded.agent_*_tokensper rollout (ontology_sql_eval/gym/model_routing.pycaptured_usage, merged in Count Auto Ontology's tokens per question through NeMo Gym #44), and Gym averages them intomean/....compare.pyprints neither.scripts/seed/seed_wwi.pyand notes that Auto-Ontology's Quickstart uses it.Companion PR: NVIDIA/auto-ontology#489.
Validation
npx -y @mermaid-js/mermaid-cliand rendered in the default and dark themes.The README links Auto-Ontology's
docs/architecture.md, which exists once NVIDIA/auto-ontology#489 merges.