Repository navigation
[Bugfix] Harden DeepSeek V4 reasoning and DSML tool-call parsing - #40833
Open
SilverNarcissus wants to merge 15 commits into
Open
SilverNarcissus wants to merge 15 commits into
SilverNarcissus wants to merge 15 commits into
Conversation
SilverNarcissus
requested review from
Fridge003,
JustinTong0323,
ispobock and
merrymercy
as code owners
September 23, 2026 03:33
Backport upstream PR sgl-project#36285 (c79efab) with its regression tests. Preserve the existing guards for forced reasoning and already-closed reasoning blocks. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 52fd8e11-8b90-4cbc-a227-032e000bae8d
Preserve append-only argument deltas for numeric and structured values. Reject an incomplete non-string parameter at EOF and retain incremental string arguments. Include V3.2/V4 chunk-boundary regressions. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 52fd8e11-8b90-4cbc-a227-032e000bae8d
Keep malformed DSML wrappers out of ordinary preambles once an invoke is parsed. Preserve quoted examples and use one parser path for streaming and nonstreaming. Add opt-in strict validation and EOF checks, with boundary, literal and public-parser regressions. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 52fd8e11-8b90-4cbc-a227-032e000bae8d
Add opt-in decoded-output delta tracing with request/choice correlation and replacement events. Exclude request bodies, input IDs and credentials. Wire streaming and nonstreaming Chat paths without changing sampling or parser results; include privacy and reconstruction tests. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 52fd8e11-8b90-4cbc-a227-032e000bae8d
Reject unquoted malformed parameter-close fragments in strict XML string arguments before dispatch. Preserve quoted, fenced, heredoc and escaped literals. Validate direct-JSON string values before releasing their arguments. Add minimal regressions for the observed boundary defect without committing benchmark captures. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 52fd8e11-8b90-4cbc-a227-032e000bae8d
Gate V4 reasoning-to-tool boundary handling behind active tool parsing and strict output mode. Preserve post-reasoning prose and quoted examples, remove only standalone redundant closers immediately before a native tool block, and never reinterpret think literals inside tool payloads. Share the existing literal masking helper with the V4 reasoning detector and cover streaming chunk boundaries, nonstreaming parity, continuation behavior, and end-to-end argument preservation. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 52fd8e11-8b90-4cbc-a227-032e000bae8d
Recognize a redundant unquoted trailing think closer at an established native tool boundary even when it is attached to prose. Keep escaped markers, quoted examples, intervening prose, and all tool payload untouched; retain the existing strict active-tool gate. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 52fd8e11-8b90-4cbc-a227-032e000bae8d
In strict V4 mode reject unexpected parameter boundaries, unclosed or duplicate parameters, invalid type flags, ambiguous JSON keys and non-JSON constants. Validate emitted-prefix consistency and completed JSON objects before marking a tool call complete. Keep V3.2 and non-strict behavior unchanged through a no-op validation hook. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 52fd8e11-8b90-4cbc-a227-032e000bae8d
Validate completed strict V4 calls against their supplied tool schemas without coercion. Reuse schema type normalization, preserve explicit unknown-tool forwarding, and disable all external schema retrieval. The policy defaults off because rejection is not generation repair and may change recovery behavior. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 52fd8e11-8b90-4cbc-a227-032e000bae8d
Reject literal think delimiters in parsed tool arguments when explicitly enabled, including quoted document strings and nested JSON values. Keep the policy off by default, preserve legitimate literals when disabled, and never rewrite commands or claim to sanitize arbitrary program outputs. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 52fd8e11-8b90-4cbc-a227-032e000bae8d
Add an explicit strict-only, default-off lexical rejection policy covering normal content, tool names and parsed arguments. Preserve native partial parameter framing. The opt-in policy deliberately rejects quoted protocol examples instead of rewriting them; separate reasoning remains untouched. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 52fd8e11-8b90-4cbc-a227-032e000bae8d
…daries Gate cleanup on strict active-tool parsing and the explicit protocol-marker policy. Preserve quoted and paired markup, incomplete EOF, interrupted reasoning, and tool payloads. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 52fd8e11-8b90-4cbc-a227-032e000bae8d
Keep raw output collection disabled even when a trace directory is configured. Preserve all parsing and validation behavior; document the independent logging gate and cover disabled/enabled configurations. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 52fd8e11-8b90-4cbc-a227-032e000bae8d
Keep executable code, protocol markers, and test data unchanged. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 52fd8e11-8b90-4cbc-a227-032e000bae8d
Remove experimental strict/schema/marker switches. Enforce structural, JSON, schema, and append-only checks by default; retain confirmed reasoning-boundary normalization. Drop blanket marker rejection and its ordinary-text holdback so valid protocol examples remain usable. Raw output logging stays explicitly opt-in. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 52fd8e11-8b90-4cbc-a227-032e000bae8d
SilverNarcissus
force-pushed
the
fix/dsv4-dsml-v0.5.18
branch
from
September 23, 2026 04:28
591c4d7 to
00a8f59
Compare
SilverNarcissus
requested review from
Jiminator,
Qiaolin-Yu,
alphabetc1,
hnyls2002,
kpham-sgl,
pyc96 and
wisclmy0611
as code owners
September 23, 2026 04:28
SilverNarcissus
requested review from
1am9trash,
AgainstEntropy,
Alisehen,
AniZpZ,
ByronHsu,
CatherineSue,
ClawSeven,
DarkSharpness,
Duyi-Wang,
Edwardf0t1,
Oasis-Git,
OrangeRedeng,
YAMY1234,
billishyahao,
celve,
hebiao064,
hubertlu-tw,
iforgetmyname,
kkHuang-amd,
rainj-me,
slin1237,
sogalin,
stmatengss,
xutizhou,
yeahdongcn,
yichiche and
yingluosanqian
as code owners
September 23, 2026 04:28
This was referenced Nov 7, 2026
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
DeepSeek V4 streaming parsing can emit unstable non-string arguments, confuse protocol boundaries with literal text,
or report completion before malformed tool arguments are detected.
This PR preserves append-only argument semantics and validates actual tool frames. It does not repair arbitrary
generation or forbid protocol-related strings in otherwise valid content.
Modifications
release/v0.5.20(94602c9c2b7cbdb8efd5c52802dac6a1c180089e).supplied tool schema by default. No coercion, missing-field insertion, or external schema retrieval.
and its ordinary-text buffering, so legitimate protocol examples remain usable.
SGLANG_ENABLE_MODEL_OUTPUT_LOGGING=1andSGLANG_MODEL_OUTPUT_TRACE_DIRare required. Output text is not redacted; request bodies and authentication headersare not copied directly into this trace.
Malformed or schema-invalid tool calls now fail explicitly instead of being forwarded as successful calls.
Ordinary text is not globally scanned for forbidden words. Middle-of-reasoning orphan markers remain outside the
confirmed-boundary cleanup scope.
Accuracy Tests
No kernel, model-forward, or weight-quantization implementation is changed by this patch.
230 tests passed, including 3,953 subtests, on the v0.5.20 code and again inside the resulting image.
The final-image run copied test files only; runtime source hashes were verified unchanged.
Coverage includes chunk boundaries, EOF, malformed structure, mandatory schema checks, valid protocol literals,
reasoning boundaries, and disabled/enabled raw logging.
Live smoke validation on a four-GPU FP8 setup completed 5/6 fixed API probes successfully. A compound literal request
was rejected for incomplete DSML framing. Two additional, separately recorded diagnostics passed 1/2; one requested
literal string was generated with a different value. A separate opt-in trace run confirmed that the malformed frame
and changed string were already present in decoded output before the reasoning/tool parsers; replay reproduced the
observed parsing behavior.
These failures are retained rather than replaced with successful reruns. This is limited integration evidence, not
a model-accuracy claim. Earlier v0.5.18 task scores are not attributed to this revision.
Speed Tests and Profiling
No isolated TTFT, TPOT, throughput benchmark, or profiler run was performed. No speedup is claimed.
Validation and enabled tracing can add CPU/I/O work; tracing is disabled by default.
Checklist
Black formatting and code-level documentation are updated. Full pre-commit, public-facing documentation, and
isolated inference-performance validation remain pending.
Review and Merge Process
CI States
Latest PR Test (Base): ❌ Missing
run-cilabel -- add it to run CI tests.Latest PR Test (Extra): ❌ Blocked --
run-ciis required first.Latest PR Test (AMD ROCm 10): ➖ No AMD PR run found for this commit.