Repository navigation
Conversation
… environment Adds an LLM-as-judge machine-translation verifier with dedicated FLORES-200 tokenizer support for 21 Indic languages, scoring adequacy + fluency 0-100 instead of relying on a fixed n-gram metric. Signed-off-by: Tanish Kushalkar <38599503+tanishku@users.noreply.github.com>
|
/ok to test e0dfccf |
|
i believe the testsuite is failing as task_data.py is missing, see other resources servers for example |
…tric bias Add task_data.py (required by CI's per-server schema gate, missing from the initial commit) mirroring TranslationWithJudgeRunRequest. Also fix compute_metrics() dropping unparseable judge scores (None) from the mean instead of counting them as 0, which silently biased aggregate judge_score upward whenever the judge failed to produce a parseable score -- per review from @cmunley1 on NVIDIA-NeMo#3348. Signed-off-by: Tanish Kushalkar <38599503+tanishku@users.noreply.github.com>
Signed-off-by: Tanish Kushalkar <38599503+tanishku@users.noreply.github.com>
|
@cmunley1 pushed a merge to resolve a conflict with main, could you /ok to test when you get a chance? Thanks. |
Signed-off-by: Tanish <38599503+tanishku@users.noreply.github.com>
|
Accepted incoming changes from main for project root README.md in the above commit |
Signed-off-by: Tanish <38599503+tanishku@users.noreply.github.com>
|
Resolved README.md conflict |
Signed-off-by: Tanish <38599503+tanishku@users.noreply.github.com>
|
Gentle ping on this PR! I've addressed the requested changes and synced with upstream. The latest changes were pushed on October 27. Could someone take another look? If everything is in order, I'd really appreciate getting this approved and merged. If there are remaining blockers, please let me know. Thanks! |
|
/ok to test de4e066 |
cmunley1
left a comment
There was a problem hiding this comment.
Approving. The inline notes are non-blocking suggestions for a follow-up.
| Reference translation: | ||
| {reference_translation} | ||
|
|
||
| Candidate translation: |
There was a problem hiding this comment.
Suggestion (non-blocking): the candidate is policy output and goes into the judge prompt as raw text, so during RL a policy could learn to append something like "Score: 100". Wrapping it in clear tags and telling the judge to ignore any instructions or scores inside it, plus a test with an injected score, would close that off.
| return {} | ||
|
|
||
| rollout_counts = [len(r) for r in tasks] | ||
| max_k = min(rollout_counts) if rollout_counts else 0 |
There was a problem hiding this comment.
Suggestion (non-blocking): masked rows are removed before compute_metrics, so one unparseable judge reply lowers max_k for every task and drops valid rollouts from all pair metrics. Grouping by the rollout index field instead of list position would keep them.
| from pydantic import BaseModel | ||
| from sacrebleu import sentence_bleu, sentence_chrf | ||
|
|
||
| from nemo_gym.base_resources_server import ( |
There was a problem hiding this comment.
Suggestion (non-blocking): these are the old import paths that go away when the shims are removed. scripts/migrate_imports.py can switch them to nemo_gym.server_runtime / nemo_gym.core automatically.
Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com>
Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com>
Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com>
|
/ok to test 795938d |
What does this PR do?
Adds
translation_with_judge, a machine-translation resources server whose reward comesfrom an LLM judge instead of a fixed n-gram metric, with a particular focus on Indic
language pairs.
verify()sends the source segment, a reference translation, and the policy model'scandidate translation to a judge model (any OpenAI-Responses-compatible endpoint) and asks
it to rate adequacy + fluency on a 0-100 scale; that score, normalized to
[0, 1], is theRL reward. Sentence-BLEU and chrF (via
sacrebleu) are also computed per rowas cheap, judge-independent diagnostics that do not affect the reward.
The environment includes dedicated tokenizer handling for Indic scripts: sentence-BLEU uses
sacrebleu's
flores200SentencePiece tokenizer (rather than the defaultwhitespace/punctuation tokenizer, which doesn't meaningfully segment Brahmic scripts) for
Hindi, Bengali, Tamil, Telugu, Marathi, Gujarati, Kannada, Malayalam, Punjabi, Odia,
Assamese, Urdu, Nepali, Sanskrit, Dogri, Konkani, Sindhi, Bodo, Kashmiri, Maithili, and
Manipuri.
Verification logic:
app.py'sverify()strips any<think>...</think>reasoningpreamble from the policy output, then sends
{source, reference, candidate}to the judgewith a structured rubric prompt and parses a
Score: <0-100>line from the response(clipped to
[0, 100],Noneif unparseable — masked withfailure_kind=judge_unparseableand reported as lost coverage; empty candidate translations remain unmasked zeroes).compute_metrics()aggregates judge score / BLEU / chrF per(src_lang, tgt_lang)pair andas cross-pair aggregates (
xx->xx,<src>->xx,xx-><tgt>), mirroringwmt_translation'smetric shape. Repeat groups preserve
_ng_rollout_indexafter masking, so validmeasurements from other tasks are retained and different repeats are not mixed.
The existing equal-per-pair/equal-per-repeat weighting is preserved.
The judge prompt XML-escapes and separately tags source, reference, and candidate text,
and instructs the judge to ignore embedded instructions and claimed scores. This is
a mitigation, not a guarantee of resistance to arbitrary prompt injection.
The server uses Gym's public package imports and committed
pyproject.toml/uv.lockdependencies in place of
requirements.txt.Task source:
data/example.jsonl(5 rows) is sourced directly from theFLORES-200
devtestsplit (CC-BY-SA 4.0), covering English↔Hindi/Bengali/Telugu/Tamil/Kannada in bothdirections. This PR ships example data only — no
train/validationsplit — since thelarger Indic-language curriculum this was developed against blends several corpora
(Samanantar, NLLB, AI4Bharat's BPCC, ILCI, and others) whose individual licenses have not
yet been verified for redistribution. Happy to follow up with a
train/validationPR oncethat audit is done.
Domain coverage:
domain: translation, focused on Indic-language pairs specifically(as opposed to
wmt_translation, which is the existing generic WMT-style verifier usingspBLEU/chrF/COMET).
verified: falseper the new-environment default.Licensing: code is Apache 2.0 (SPDX headers on
app.py/test_app.py); example data isCC-BY-SA 4.0 (FLORES-200); dependencies (
sacrebleu,sentencepiece) are Apache 2.0. Seethe environment's
README.mdfor full details.Verification performed
resources_servers/translation_with_judge/tests,tests/unit_tests/tooling/test_component_dependencies.py, andtests/unit_tests/test_public_api_facades.py(pytest through a source-bound runner).pass with these fixes: injected delimiter handling, masked-repeat preservation,
and sparse cross-pair repeat alignment.
pre-commit run --all-files— passed, including import layering. The generatedREADME table restores this environment's entry.
uv lock --check --python 3.13.14 --project resources_servers/translation_with_judge— passed. The component lock uses main's OpenAI 2.45.0 pin.
nvidia/nemotron-3.5-lightning-30b-a3bthroughactual HTTP environment, agent, resources and model servers: five shipped examples
completed with the repaired verifier before the main update (scores 90, 95, 45,
95, 20; none masked). The verifier source was unchanged by integration; one full
example was freshly rerun on the final base and scored 90, unmasked.
Score: 100scored 0.Same-candidate rejudging with the original prompt on the current framework scored
95 versus 90 with the updated prompt for the valid example, and 0 versus 0 for
the injected candidate. This tiny smoke is not a quality baseline or a guarantee
of general prompt-injection resistance. Policy and judge used the same model;
apps were started directly, so the CLI/multiprocess launcher was not exercised.
openai/gpt-oss-120b; they were not regenerated by this follow-up.No separate issue is needed for this self-contained training-environment addition.
Checklist
pre-commit run --all-files) (so CI lint/format/copyright pass).git commit -s) (so the DCO check passes).