Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -302,6 +302,7 @@ The Dataset column links to publicly available datasets (e.g., on HuggingFace).
| Ifbench | instruction_following | IFBench instruction following evaluation using AllenAI's IFBench library (57 instruction types) | Improve IFBench instruction following | - | - | - | <a href='resources_servers/ifbench/configs/ifbench.yaml'>ifbench.yaml</a> | - |
| Iheval | instruction_following | IHEval instruction-hierarchy benchmark (8 single-turn tasks — task execution, safety, tool use, rule following) scored with the upstream rule-based checkers. | Improve instruction-hierarchy following across aligned/conflict/reference settings, including native tool-use trajectories. | - | ✓ | - | <a href='resources_servers/iheval/configs/iheval.yaml'>iheval.yaml</a> | - |
| Image Tools | agent | PivotRL verifier for VLM image-tool trajectories; scores one tool call against the demonstrated SFT action. | Improve per-turn image-tool decisions (which tool, which image, which region). | ✓ | ✓ | NVIDIA Internal Use Only, Do Not Distribute | <a href='resources_servers/image_tools/configs/image_tools_pivot.yaml'>image_tools_pivot.yaml</a> | - |
| Image Tools Agent | | - | - | ✓ | ✓ | NVIDIA Internal Use Only, Do Not Distribute | <a href='responses_api_agents/image_tools_agent/configs/image_tools_agent.yaml'>image_tools_agent.yaml</a> | - |
| Imo Gradingbench | math | Four-class grading of math proofs — the policy model reads a problem plus a candidate proof and emits one of correct / almost / partial / incorrect as the last word. | Improve the IMO-GradingBench benchmark and proof-grading skill. | - | - | - | <a href='resources_servers/imo_gradingbench/configs/imo_gradingbench.yaml'>imo_gradingbench.yaml</a> | - |
| Imo Proofbench Judge | math | IMO ProofBench grader using a strong LLM judge with the IMO 0-7 rubric | Score IMO-style proof submissions with a problem-specific grading rubric | - | - | - | <a href='resources_servers/imo_proofbench_judge/configs/imo_proofbench_judge.yaml'>imo_proofbench_judge.yaml</a> | - |
| Indian Banking | agent | Indian retail-banking customer-support environment: a multi-turn dialog with an LLM-simulated customer over 33 deterministic banking tools (accounts, deposits, loans, cards, mandates, service requests), scored with partial credit against gold tool calls, final DB state, required disclosures and an NL-assertion judge. | Improve stateful multi-turn tool use, policy compliance and customer communication in a regulated domain. | ✓ | ✓ | Apache 2.0 | <a href='resources_servers/indian_banking/configs/indian_banking.yaml'>indian_banking.yaml</a> | <a href='https://huggingface.co/datasets/NPCI/nemo-gym-indian-banking'>nemo-gym-indian-banking</a> |
Expand Down
5 changes: 4 additions & 1 deletion responses_api_agents/image_tools_agent/app.py
Original file line number Diff line number Diff line change
Expand Up @@ -372,7 +372,10 @@ async def run(self, request: Request, body: ImageToolsAgentRunRequest) -> ImageT
aux_reward = float(rollout_info["image_tools_aux_reward"])
verify_response_json["base_reward"] = base_reward
verify_response_json["image_tools_aux_reward"] = aux_reward
verify_response_json["reward"] = base_reward + aux_reward
# Add auxiliary rewards to the base score, capped above at 1.0.
# Positive bonuses can raise any score below the cap; negative penalties
# still apply at every base score, including a correct answer's 1.0.
verify_response_json["reward"] = min(1.0, base_reward + aux_reward)
verify_response_json["image_tools_base_agent_ref"] = body.image_tools_base_agent_ref
verify_response_json.update(rollout_info)
verify_response_json["response"] = model_response.model_dump()
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,20 @@ image_tools_simple_agent:
model_server:
type: responses_api_models
name: policy_model
# `license` is REQUIRED on train/validation datasets: DatasetConfig asserts
# it, and without it pydantic falls through to BenchmarkDatasetConfig and
# the server is silently demoted to an "almost-server" that never starts.
datasets:
- name: train
type: train
jsonl_fpath: resources_servers/image_tools/data/grpo_train.jsonl
num_repeats: 1
license: NVIDIA Internal Use Only, Do Not Distribute
- name: validation
type: validation
jsonl_fpath: resources_servers/image_tools/data/grpo_validation.jsonl
num_repeats: 1
license: NVIDIA Internal Use Only, Do Not Distribute
resource_servers_by_agent:
string_match_simple_agent:
type: resources_servers
Expand All @@ -29,9 +43,9 @@ image_tools_simple_agent:
crop_max_pixels: 1048576
max_tool_calls: 20
max_tool_calls_per_turn: 1
tool_success_reward: 0.02
tool_success_reward_cap: 0.20
invalid_tool_call_penalty: -0.05
tool_success_reward: 0.01
tool_success_reward_cap: 0.05
invalid_tool_call_penalty: -0.01
duplicate_tool_call_penalty: -0.02
force_final_after_duplicate: true
terminate_on_invalid_tool_call: true
Expand Down
12 changes: 9 additions & 3 deletions responses_api_agents/image_tools_agent/tests/test_app.py
Original file line number Diff line number Diff line change
Expand Up @@ -114,11 +114,17 @@ def test_image_tool_markup_validation_rejects_nested_or_extra_tags() -> None:
("training", "/ng-rollout/test-rollout/training-token-capture/v1/responses"),
],
)
@pytest.mark.parametrize(
"base_reward,expected_reward",
[(0.0, 0.02), (0.5, 0.52), (0.99, 1.0), (1.0, 1.0)],
)
async def test_image_tools_agent_runs_tool_loop_and_delegates_reward(
tmp_path: Path,
entrypoint: str,
capture_mode: str,
model_url_path: str,
base_reward: float,
expected_reward: float,
) -> None:
image_path = tmp_path / "source.png"
Image.new("RGB", (256, 192), color=(120, 80, 40)).save(image_path)
Expand Down Expand Up @@ -191,7 +197,7 @@ async def _post_side_effect(*, server_name: str, url_path: str, **kwargs: Any):
verify_payload = kwargs["json"]
assert len(verify_payload["response"]["output"]) == 1
assert verify_payload["response"]["output"][0]["content"][0]["text"] == final_text
return _FakeClientResponse(_base_verify_response(kwargs["json"], reward=1.0))
return _FakeClientResponse(_base_verify_response(kwargs["json"], reward=base_reward))
raise AssertionError(f"Unexpected call: {server_name} {url_path}")

server_client_post.side_effect = _post_side_effect
Expand Down Expand Up @@ -244,9 +250,9 @@ async def _post_side_effect(*, server_name: str, url_path: str, **kwargs: Any):

result = await agent.run(SimpleNamespace(cookies={}), body)
payload = result.model_dump(mode="json")
assert payload["base_reward"] == 1.0
assert payload["base_reward"] == base_reward
assert payload["image_tools_aux_reward"] == 0.02
assert payload["reward"] == 1.02
assert payload["reward"] == pytest.approx(expected_reward)
assert payload["image_tools_call_count"] == 1
assert payload["image_tools_error_count"] == 0
assert len(payload["image_tools_output_paths"]) == 1
Expand Down
Loading