Repository navigation
Improve tensor parallel performance - #625
Merged
Merged
Conversation
timethink
pushed a commit
to timethink/sglang
that referenced
this pull request
Mar 9, 2025
Co-authored-by: Mingyi <wisclmy0611@gmail.com>
Jiminator
added a commit
to Jiminator/sglang
that referenced
this pull request
May 28, 2026
Adds an investigation report for the nightly-test-perf-4-gpu-b200 failure of test_a_gsm8k under TestDeepseekV3FP4CuteDSLMoE. The regression is traced to commit 044649c (PR sgl-project#22669) which under-sizes the CuteDslMoEWrapper buffer for the EP=1, TP=4, no-DP-attention DeepSeek V3 FP4 configuration. The report includes: - pass/fail boundary (run sgl-project#624 pass -> sgl-project#625 fail on main) - bisection across the 50 commits in 7f154ba..c4a7d12 - root-cause diff in flashinfer_cutedsl.py:ensure_cutedsl_wrapper - direct repro at offending and parent SHAs (GPU 4 only) - a four-line recommended patch and short-term workarounds Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
efschu
added a commit
to efschu/htsglang
that referenced
this pull request
Aug 7, 2026
… route pricing, PP-vs-TP prefill measurement
efschu
pushed a commit
to efschu/htsglang
that referenced
this pull request
Aug 7, 2026
…contract, path B verdict Seven commits from the sgl-project#625 Route A line, all desk-proven, none needing cards. #631a guard 1 (57e13eb): a PD pair whose arms hold different weights is refused, on model_identity_hash only with include_parallel_vectors=False so a PP prefill and a TP+DCP decode on the same weights still pair. Surfaced and fixed a latent TypeError: revision and quantization were the only identity parts not str()-coerced, harmless while every caller ran late with a resolved ServerArgs, reachable from the registration path. sgl-project#636 (4f486ab): the four preconditions of the DCP token-sharded handover become ONE boot-time contract instead of four runtime deaths on first transfer. Placed after _reject_unsupported_draft_kv_dcp because dcp_size is not resolved until _handle_uneven_tp (server_args.py:10319, called at :5957, after the PD hook at :5916) -- a gate at the hook reads the unresolved value and passes silently on the exact production configuration it guards. The ordering is pinned by a test, not by a comment. #631b contract (214433e) and spec (ec3a440): versioned canonical full-head draft-KV layout with a geometry gate whose bound is the rejected alternative's own cost rather than a constant in MB, plus the L1-L11 lift criterion for the #631a refusal. #631b composition verdict (66da8b3): --draft-kv-layout and disaggregation_mode were never considered together. Under the default 'replicated' layout the draft pool is sized to the full global context while the decode arm addresses every registered buffer with compact owner-rule rows -- two coordinate systems, silent wrong-slot drift. That makes 'dcp' the only layout on which the existing shared-index transfer is correct at all, not merely the cheaper one. sgl-project#642 (66ca5ee, a79fc3c): guards for that composition, with a hazard test that stays green when the guard is removed so the arms distinguish a broken guard from a broken hazard, and an identity-hash invariant pinned by behaviour after the source-text pin was shown to miss the likely refactor. The #631a refusal stays in force: the wiring slice needs cards and a half-wired draft path that boots and silently decodes without speculation is the failure this line exists to remove.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.