Repository navigation
Conversation
|
Pushed SGLang hard-rejects input The splice fix + this guard have now been exercised end-to-end by a 16-node R=32 SGLang async-GRPO run: 0 chat-500s, 15+ training steps with graceful trajectory truncation and no stall. Still draft pending a full convergence-vs-baseline comparison. |
|
Convergence validation complete. The splice fix + over-context guard in this PR (as carried on the fork branch) have now been validated by a full-scale end-to-end run: Qwen3-30B-A3B-Thinking SWE-bench async-GRPO on SGLang, 16 nodes (8 train + 8 gen, 32 generation replicas), 63 training steps across four clean checkpoint+resume segments.
Remaining known caveat: this port of the fix onto refactored Gym main is still not GPU-tested in this exact form (the validated code is the fork's pre-refactor equivalent) — keeping the PR draft until that's run. |
Signed-off-by: Serge Panev <spanev@nvidia.com>
SGLang /generate hard-rejects input tokens >= context_length (HTTP 500), unlike vLLM which tolerates it. In a multi-turn SWE rollout the spliced prompt can already fill the window; the 500 then corrupts the trajectory and trips the downstream contiguity assert, stalling the whole async step. Mirror the vLLM context-length handling: pre-check the prompt length against sglang_max_total_sequence_length, and defensively catch the ClientResponseError, ending the turn cleanly with finish_reason="length" (empty completion) in both cases. Validated by a 16-node R=32 SGLang GRPO run (0 chat-500s; 15+ steps, graceful truncation, no stall). Signed-off-by: Serge Panev <spanev@nvidia.com>
The SGLang engine path re-parses tool calls from raw generated text client-side. The existing parser only handled the hermes JSON format (Qwen3-thinking). Add a dependency-free parser for the qwen3_coder XML-ish format (<tool_call><function=NAME><parameter=KEY>...) used by e.g. Qwen3-Coder-style chat templates, with schema-aware argument type coercion from the request's tools, selectable via a new sglang_tool_format config field (default "hermes" = no behavior change). Unit tests mirror the template's canonical example. Signed-off-by: Serge Panev <spanev@nvidia.com>
Chat templates iterate assistant tool-call arguments as a mapping
(tool_call.arguments|items), but OpenAI-format history carries
function.arguments as a JSON string - exactly what our /generate-path
parsers emit. When a request misses the token-splice cache and falls
back to a full template render of a tool-call-bearing conversation,
jinja raises TypeError('Can only get item pairs from a mapping.') and
the request 500s.
Normalize on the full-render path only: decode string arguments that
parse to a JSON object, leave everything else (and the splice path,
which never re-renders old assistant turns) untouched. Inputs are not
mutated - the splice session cache holds the original messages.
Signed-off-by: Serge Panev <spanev@nvidia.com>
Signed-off-by: Serge Panev <spanev@nvidia.com>
Mechanical ruff check --fix (I001 import order) + ruff format on the SGLang proxy files; no logic changes. Fixes the failing Lint check on the PR. Signed-off-by: Serge Panev <spanev@nvidia.com>
Signed-off-by: Serge Panev <spanev@nvidia.com>
Signed-off-by: Serge Panev <spanev@nvidia.com>
|
Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually. Contributors can view more details about this message here. |
The adapter renders the chat template and tokenizes locally before calling the SGLang server, so its transformers version has to agree with the server's. It was pinned to 5.6.0 to match the NeMo-RL sglang extra; that extra moves to 5.8.1 alongside sglang 0.5.13.post1, so this pin follows. A mismatch here does not fail loudly at import. It surfaces as a rollout contiguity failure when the locally rendered prefix stops matching what the server tokenized, which is far harder to attribute, hence the comment on the pin. Signed-off-by: Serge Panev <spanev@nvidia.com>
Signed-off-by: Serge Panev <spanev@nvidia.com>
|
/ok to test 3122202 |
Signed-off-by: Serge Panev <spanev@nvidia.com>
|
/ok to test 9537b32 |
Signed-off-by: Serge Panev <spanev@nvidia.com>
|
/ok to test caa72c5 |
Signed-off-by: Serge Panev <spanev@nvidia.com>
|
/ok to test 32db78d |
Adds a text-only SGLang model server for NeMo Gym rollouts. It calls the native
/generateendpoint to preserve sampled token IDs and logprobs, then splices those exact IDs into later turns. Changed rendering inputs fail explicitly, and a full context window ends generation without truncating the prompt.The adapter uses Gym's current public package imports and shared HTTP client. Its component lock inherits the repository's resolver policy and pins Transformers 5.17.0. The tokenizer and chat template must match training; serving can use its own environment because requests carry integer token IDs.
Validation on the current main merge and dependency fix:
pre-commit run --all-filesandgit diff --checkpassed; commits are signed off.This remains a draft pending a representative model-backed smoke on the final head. Earlier GPU runs do not validate this refreshed dependency set. Single-controller and multimodal integration remain separate work.
Supports NVIDIA-NeMo/RL#3127. Episode truncation is tracked separately in NVIDIA-NeMo/RL#3063; no additional issue is needed for this existing integration.