Skip to content

docs(training): document task_source routing instead of agent_ref - #4681

Open
yaoyu-33 wants to merge 4 commits into
NVIDIA-NeMo:mainfrom
yaoyu-33:docs/training-task-source-routing
Open

yaoyu-33 wants to merge 4 commits into
NVIDIA-NeMo:mainfrom
yaoyu-33:docs/training-task-source-routing

Conversation

@yaoyu-33

@yaoyu-33 yaoyu-33 commented Oct 16, 2026 •

Copy link
Copy Markdown
Contributor

What does this PR do?

Updates the data and training docs to match how rows are routed today.

Since v0.6.0 (#2717), gym dataset collate works like this:

  • It strips any agent_ref from the source data.
  • It stamps each row with task_source, which is the config instance that declared the row's dataset (nemo_gym/train_data_utils.py).
  • NeMo Gym resolves task_source to an agent at dispatch, in RolloutCollectionHelper.resolve_task_sources, called from run_examples, preprocess_examples and run_from_config.
  • Rows that still carry a baked-in agent_ref and no task_source log a DeprecationWarning.

The docs still said collate adds agent_ref, and the multi-environment page hand-stamped agent_ref with jq … >>. That uses the deprecated path and duplicates rows on rerun.

Changes:

  • data/index.mdx:
    • The required-fields table and the routing section now describe task_source, with +agent_map for rerouting.
    • Added a note that datasets collated before v0.6.0 still run but are deprecated.
  • data/prepare-validate.mdx, data/download-huggingface.mdx: say collate adds task_source, including the collate step in the validation-process list.
  • training-tutorials/nemo-rl-grpo/setup.mdx: explains task_source. NeMo RL main resolves it through Gym's run_examples, and RL main pinned Gym da9f67149 when this PR was opened and now pins ad4f4804f; both include feat(data): collate stamps task_source; datasets declarable on resources servers #2717.
  • training-tutorials/multi-environment-training.mdx: builds the mixed dataset with gym dataset collate --resources-server A --resources-server B instead of jq.
  • training-tutorials/verl.mdx:
    • The veRL recipe still needs agent_ref. It reads it from each row and rejects rows without it (verl-recipe/nemo_gym/agent_loop.py: raise ValueError(f"dataset row {i} is missing agent_ref")). dataset.py already forwards task_source through extra_env_info, but the agent_ref check rejects the row first (verl-recipe 81d78276, agent_loop.py:291-292).
    • The page now adds agent_ref from task_source after collating, and shows the same flow for multi-environment collation.

Issue

Closes #4669 (Linear GYM-1082, part of the GYM-1080 docs audit).

Validation

  • Multi-environment collate: ran gym dataset collate --model-type openai_model --resources-server example_single_tool_call --resources-server example_multi_step --mode example_validation. It writes one example.jsonl with 10 rows and no agent_ref:
    • task_source = example_single_tool_call_simple_agent (5 rows)
    • task_source = example_multi_step_simple_agent (5 rows)
  • Routing of the collated rows: fed them to RolloutCollectionHelper.resolve_task_sources with the merged env start config for both servers. Each resolved to its own agent (5 and 5), with no DeprecationWarning.
  • The old jq flow: rows with only agent_ref log 5 rows routed via their baked-in agent_ref (no task_source). This legacy path is deprecated….
  • The veRL jq stamp: ran it on the collated file. Every row gains agent_ref.name == task_source.
  • Dataset declarations: workplace_assistant and math_with_judge both declare their datasets on their agent instance, so task_source is the agent name, as the veRL page says.
  • NeMo RL main (nemo_rl/environments/nemo_gym.py) calls run_examples, which resolves task_source to agent_ref, and falls back to task_source for route names.
  • Follow-up commit (f66fd17): fixed the collate step in prepare-validate.mdx ("Combine samples with agent references"). A grep of fern/versions/latest/pages finds no other page saying collate adds agent_ref. The remaining agent_ref mentions describe rollout output, where it is still correct, except two in reference/cli-commands.mdx that the review-fix commit below corrects. pre-commit run --files: pass.
  • Review fixes (a251904f5):
    • data/index.mdx no longer promises a DeprecationWarning for agent_ref-only rows: on gym eval run and run_examples, Python's default filters hide it.
    • multi-environment-training.mdx: --mode train_preparation --download fails for both example environments (one declares no training data; the other downloads from an internal GitLab), so the page says they have no public training data.
    • verl.mdx gives the rule Gym uses to pick the agent for a dataset declared on a resources server.
    • reference/cli-commands.mdx: collate's --resources-server is marked repeatable, and --num-repeats keys are described as task_source.
    • pytest tests/unit_tests/test_fern_docs_links.py: 24 passed. pre-commit run --files: pass.
  • pre-commit run --all-files: pass.
  • cd fern && npm run check: 0 errors. The only warning is the redirects check, skipped without fern login.

Rollout evidence

N/A. This is a docs-only change and no runtime behavior changes. Routing was checked with Gym's own resolver on real collated output, as listed above.

Compatibility / follow-ups

  • veRL users with Gym ≥ 0.6.0 must add agent_ref after collating, as the page now shows. The recipe's own README still says rows need agent_ref.
  • A follow-up for the veRL recipe only needs to drop the agent_ref check in agent_loop.py. task_source already reaches run_examples, which resolves it to agent_ref the way it does for NeMo RL. That would remove the extra step.

🤖 Generated with Claude Code

Since v0.6.0, `gym dataset collate` strips `agent_ref` and stamps each row
with `task_source`, the config instance that declared its dataset. NeMo Gym
resolves it to an agent when rollouts are dispatched. The data and training
pages still said collate adds `agent_ref`, and the multi-environment page
built its mixed dataset by hand-stamping `agent_ref`, which now routes
through the deprecated path and duplicates rows on rerun.

Describe `task_source` routing and `+agent_map` in the data docs and the
NeMo RL tutorial, and build the multi-environment dataset with
`gym dataset collate --resources-server A --resources-server B`. The veRL
recipe still reads `agent_ref` from each row and rejects rows without it,
so the veRL page now adds `agent_ref` from `task_source` after collating.

Refs NVIDIA-NeMo#4669

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com>
@copy-pr-bot

copy-pr-bot Bot commented Oct 16, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@yaoyu-33 yaoyu-33 added area:training Training framework integrations and training-data interfaces docs Documentation-only updates or documentation debt labels Oct 16, 2026
prepare-validate.mdx step 5 still said collate combines samples with agent references.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com>
@yaoyu-33
yaoyu-33 marked this pull request as ready for review October 16, 2026 21:34
Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com>
@yaoyu-33 yaoyu-33 added complexity:medium Single-domain change with interacting parts or a moderate review surface needs-review PR is ready for code review and waiting on a reviewer labels Oct 16, 2026
- Drop the DeprecationWarning promise for agent_ref-only rows: on real
  dispatch paths Python's default filters hide it.
- Say that the two example environments have no public training data, so
  train_preparation does not apply to them.
- In the veRL page, give the rule Gym uses to pick the agent for a dataset
  declared on a resources server.
- In the CLI reference, mark collate's --resources-server as repeatable and
  key --num-repeats by task_source, not agent_ref.name.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com>
@github-actions github-actions Bot added the sla:triage-overdue Review assignment is over the one-business-day SLA label Oct 19, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:training Training framework integrations and training-data interfaces complexity:medium Single-domain change with interacting parts or a moderate review surface docs Documentation-only updates or documentation debt needs-review PR is ready for code review and waiting on a reviewer sla:triage-overdue Review assignment is over the one-business-day SLA

Projects

None yet

Development

Successfully merging this pull request may close these issues.

docs(training): replace stale agent_ref routing with task_source

1 participant