Repository navigation
Fix stop str merging - #225
Merged
Merged
Conversation
Collaborator
Author
|
closes #190 |
timethink
pushed a commit
to timethink/sglang
that referenced
this pull request
Mar 9, 2025
Co-authored-by: Enrique Shockwave <33002121+qeternity@users.noreply.github.com>
IzacharyI
added a commit
to IzacharyI/sglang
that referenced
this pull request
Mar 30, 2026
IzacharyI
added a commit
to IzacharyI/sglang
that referenced
this pull request
Mar 31, 2026
* Move and adaptation of GDN opt from Qwen3next to Qwen3.5 (sgl-project#215) * rm 3*copy after casual conv, do q/k/v split in causal_conv1d_update_split_qkv kernel during decode time * Add fused Sigmoid and mul triton kernel * add fused gdn gating and sigmoid func for prefill phase * Add fused causal conv1d fwd split qkv kernel * Add fused cumsum_kkt, solve_tril, and merge_recompute kernels for prefill optimization * Increase parallel params for chunk_gated_delta_rule_fwd_kernel * add chunk o kernel opt * Add sigmoid and mul for shared expert module --------- Co-authored-by: yixionghuo <66975556+yixionghuo@users.noreply.github.com> * [Fix] Load weights error in compressed-tensors * Fuse 4 GDN input projections (qkv/z/b/a) into single GEMM (sgl-project#219) * Revert "Fuse 4 GDN input projections (qkv/z/b/a) into single GEMM (sgl-project#219)" (sgl-project#223) This reverts commit 17b0749. * Fuse 4 GDN input projections (qkv/z/b/a) into single GEMM (sgl-project#225) * [Fix] Core dump issue in stress test --------- Co-authored-by: zachYao <zayao@amd.com> Co-authored-by: root <root@hjbog-srdc-29.amd.com>
liuxianglong17
pushed a commit
to shun8686/sglang
that referenced
this pull request
Apr 7, 2026
multi item scoring testcase
rucnyz
added a commit
to rucnyz/sglang
that referenced
this pull request
Jun 9, 2026
…ding holders Kills the TP=4 A3 remove_hbm_not_device_leaf reject storm (1384/cycle, 187 hot units re-failing ~15x). Root cause is persistent TOCTOU, not a mis-identification: a session-tail leaf is a device-leaf *at dump time* only in the brief gap between an active program's forward passes (its lock_ref oscillates per pass; the dump samples a released instant). By apply time the state is up to ~840ms stale and the holder has re-locked its tail, so sglang rejects. Fix: gate on the program-level signal, which is STABLE across the per-pass oscillation. _holder_actively_decoding(u, state) returns True iff any holder pid has T26 per_program_usage[pid].hbm.inflight > 0 (a request in the running batch). migrate_candidates skips remove-HBM for such units, after the existing sgl-project#210 structural leaf guards. Node-level lock_ref is the wrong signal (0 at the dump instant). Tool-parked holders (inflight empty) keep their idle tail evictable -- the core demote-during-the-tool-gap relief. Absent/cold-start inflight never suppresses. Layers cleanly with sgl-project#223 (reactive post-reject cooldown) and sgl-project#210 (structural leaf guards); sgl-project#224 is the proactive complement. Audit (sgl-project#225 follow-ups): holders come from add-only session_ids, so the guard slightly over-approximates (safe direction -- forgoes a valid demote, never proposes an invalid one); bounded small under temp-0 linear agentic decode. Re-run must confirm migrate_applied>0 + HBM occ drops, not merely that rejects fell to zero. Tests: verify/joint_decide A-inflight (active blocked, parked/zero/ no-hbm/cold preserved, shared + multi-subpool blocked). DESIGN.md §7/§9 apply-feasibility pre-filter subsection. All pure suites green.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.