Skip to content

Fix stop str merging - #225

Merged
hnyls2002 merged 3 commits into
mainfrom
fix-stop-str-merging
Feb 24, 2024
Merged

hnyls2002 merged 3 commits into
mainfrom
fix-stop-str-merging

Conversation

@hnyls2002

Copy link
Copy Markdown
Collaborator

No description provided.

@hnyls2002

hnyls2002 commented Feb 24, 2024 •

Copy link
Copy Markdown
Collaborator Author

closes #190

@hnyls2002
hnyls2002 merged commit c4e9ebe into main Feb 24, 2024
@hnyls2002
hnyls2002 deleted the fix-stop-str-merging branch February 24, 2024 08:05
timethink pushed a commit to timethink/sglang that referenced this pull request Mar 9, 2025
Co-authored-by: Enrique Shockwave <33002121+qeternity@users.noreply.github.com>
IzacharyI added a commit to IzacharyI/sglang that referenced this pull request Mar 30, 2026
IzacharyI added a commit to IzacharyI/sglang that referenced this pull request Mar 31, 2026
* Move and adaptation of GDN opt from Qwen3next to Qwen3.5 (sgl-project#215)

* rm 3*copy after casual conv, do q/k/v split in causal_conv1d_update_split_qkv kernel during decode time

* Add fused Sigmoid and mul triton kernel

* add fused gdn gating and  sigmoid func for prefill phase

* Add fused causal conv1d fwd split qkv kernel

* Add fused cumsum_kkt, solve_tril, and merge_recompute kernels for prefill optimization

* Increase parallel params for chunk_gated_delta_rule_fwd_kernel

* add chunk o kernel opt

* Add sigmoid and mul for shared expert module

---------

Co-authored-by: yixionghuo <66975556+yixionghuo@users.noreply.github.com>

* [Fix] Load weights error in compressed-tensors

* Fuse 4 GDN input projections (qkv/z/b/a) into single GEMM (sgl-project#219)

* Revert "Fuse 4 GDN input projections (qkv/z/b/a) into single GEMM (sgl-project#219)" (sgl-project#223)

This reverts commit 17b0749.

* Fuse 4 GDN input projections (qkv/z/b/a) into single GEMM (sgl-project#225)

* [Fix] Core dump issue in stress test

---------

Co-authored-by: zachYao <zayao@amd.com>
Co-authored-by: root <root@hjbog-srdc-29.amd.com>
liuxianglong17 pushed a commit to shun8686/sglang that referenced this pull request Apr 7, 2026
rucnyz added a commit to rucnyz/sglang that referenced this pull request Jun 9, 2026
…ding holders

Kills the TP=4 A3 remove_hbm_not_device_leaf reject storm (1384/cycle,
187 hot units re-failing ~15x). Root cause is persistent TOCTOU, not a
mis-identification: a session-tail leaf is a device-leaf *at dump time*
only in the brief gap between an active program's forward passes (its
lock_ref oscillates per pass; the dump samples a released instant). By
apply time the state is up to ~840ms stale and the holder has re-locked
its tail, so sglang rejects.

Fix: gate on the program-level signal, which is STABLE across the
per-pass oscillation. _holder_actively_decoding(u, state) returns True
iff any holder pid has T26 per_program_usage[pid].hbm.inflight > 0 (a
request in the running batch). migrate_candidates skips remove-HBM for
such units, after the existing sgl-project#210 structural leaf guards. Node-level
lock_ref is the wrong signal (0 at the dump instant). Tool-parked
holders (inflight empty) keep their idle tail evictable -- the core
demote-during-the-tool-gap relief. Absent/cold-start inflight never
suppresses.

Layers cleanly with sgl-project#223 (reactive post-reject cooldown) and sgl-project#210
(structural leaf guards); sgl-project#224 is the proactive complement.

Audit (sgl-project#225 follow-ups): holders come from add-only session_ids, so the
guard slightly over-approximates (safe direction -- forgoes a valid
demote, never proposes an invalid one); bounded small under temp-0
linear agentic decode. Re-run must confirm migrate_applied>0 + HBM occ
drops, not merely that rejects fell to zero.

Tests: verify/joint_decide A-inflight (active blocked, parked/zero/
no-hbm/cold preserved, shared + multi-subpool blocked). DESIGN.md
§7/§9 apply-feasibility pre-filter subsection. All pure suites green.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants