Repository navigation
Conversation
merrymercy
force-pushed
the
main
branch
4 times, most recently
from
January 16, 2024 05:44
5a84411 to
97b04d4
Compare
merrymercy
requested changes
Jan 16, 2024
Ying1123
reviewed
Jan 16, 2024
Ying1123
reviewed
Jan 17, 2024
Ying1123
reviewed
Jan 17, 2024
Contributor
|
|
5 tasks done
yhyang201
pushed a commit
that referenced
this pull request
Dec 13, 2025
* test: add EPD disaggregation integration tests * fix comment for encoder-only * revert http_server warmup for vlm
fstandhartinger
pushed a commit
to fstandhartinger/sglang
that referenced
this pull request
Jan 13, 2026
Devstral tools
tpoisonooo
pushed a commit
to tpoisonooo/sglang
that referenced
this pull request
Feb 12, 2026
5 tasks done
chx96642264
pushed a commit
to chx96642264/sglang
that referenced
this pull request
Mar 11, 2026
rucnyz
added a commit
to rucnyz/sglang
that referenced
this pull request
Apr 30, 2026
…s 28 xfers v9 pool-binding-shift trace produces real differentiation: - Phase B (KV-bound 8K random): L1+L2 -37% mean TTFT vs stock - Phase C (mixed 4K random): L1+L2 -38% median E2E vs stock - Cross-pool transfers: stock 0, L1-only 0, L2-only 0, L1+L2 28 Two surprising findings documented: 1. Layer 2 alone fires zero transfers — Layer 1 retention is what makes Layer 2 cross the firing threshold. 2. Phase A regresses with L1 (-20% TPS) because K_big=8192 hurts on prefix-friendly GSP. Consistent with A2's K_big=0-wins finding. Adaptive K_big control marked as follow-up. Settings status: Setting 1 marked **DONE v6 NULL + v9 PASS**. All 4 user-requested follow-ups (sgl-project#9 Q3.A 4-arm, sgl-project#10 Sweep 1 multi-seed, sgl-project#11 Setting 4 fallback rule, sgl-project#12 Setting 1 v9 trace) now complete.
predator2k
pushed a commit
to predator2k/sglang
that referenced
this pull request
May 12, 2026
Second independent review of the TTExecutionBackend amendment caught
issues the first round missed. All addressed here.
CRITICAL:
- test_env_var_selects_backend called `.get.cache_clear()` on
envs.SGLANG_TT_EXECUTION_BACKEND.get, but EnvField.get (environ.py:54)
is a plain method that reads os.environ on every call — no lru_cache,
so the call AttributeError'd. Verified via direct read of environ.py.
Removed the spurious cache_clear() call; updated the test docstring
to record the verification.
- Same test referenced `envs.X.get()` but the test file's import block
didn't `from sglang.srt.environ import envs`. Side-effect of the
cache_clear removal: the only `envs` reference is now gone, so no
new import line needed — the dependency vanishes with the fix.
IMPORTANT:
- Spec §5.1 worker pseudocode still used `self.wrapper.X` / called
`ttnn.to_torch(logits_ttnn)` in the worker. Plan G.2 was already
updated to `self.execution_backend` returning host tensors directly.
Synchronized spec §5.1 to match: worker calls
`self.execution_backend.{extend,decode_step}`, gets host torch
tensors, never imports ttnn (invariant sgl-project#7 enforcement).
- Spec §2 prereq (line 85) still mentioned "Phase F's TTLlamaWrapper
glue" as the target of the Phase 0.2 API capture. Updated to
TTTransformersExecutionBackend with pointer to the amendment note.
- Spec §12 open question sgl-project#9 still referenced TTLlamaWrapper. Updated;
noted that the ABC + registry remain valid even if the
tt_transformers adapter has to be redone.
NIT:
- noqa: E402 comment on the registry __init__.py snippet is now
inline with a leading explanatory comment block; future maintainers
shouldn't accidentally silence E402 elsewhere.
- Plan F.1 Step 6 says `git rm`, not plain `rm` — preserves blame
traceability from tt_transformers_backend.py back to commit
152b67c (the original llama_adapter.py).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
16 of 19 tasks
yushengsu-thu
referenced
this pull request
in yushengsu-thu/sglang
May 19, 2026
BCG was the fallback path designated for cases where PCG hits a torch.compile / Inductor incompatibility (B5/B6 in the design doc). Adds LoRA-aware ForwardBatch construction and prepare_lora_batch hooks to mirror what we did for PCG, so --enable-breakable-cuda-graph also works under --enable-lora. Changes (breakable_cuda_graph_runner.py): * _build_capture_forward_batch: stamp lora_ids=[None]*bs when --enable-lora is set (was hard-coded None). Same semantics as PCG: empty ids drive the kernels to launch but early-return on rank=0. * New _maybe_prepare_lora helper that, when enable_lora and the batch has lora_ids, calls lora_manager.prepare_lora_batch(forward_batch, force_cuda_graph=True). This populates the pinned cuda_graph_batch_info that gets baked into the captured BreakableCUDAGraph. * _warmup and _capture_one now call _maybe_prepare_lora before init_forward_metadata, mirroring PCG's _capture_one path. * replay() re-runs prepare_lora_batch(force_cuda_graph=True) at the top so live batch's adapter metadata lands in the same pinned buffer (init_new's earlier non-cuda-graph alloc is overwritten). BCG bypasses torch.compile entirely (it uses raw torch.cuda.CUDAGraph plus manual @eager_on_graph break points), so it should not hit the Dynamo-vs-Triton-config issues that block chunked + PCG and MoE + PCG. End-to-end smoke for BCG path still TODO (would require --enable-breakable-cuda-graph + --enforce-piecewise-cuda-graph). The PCG path is the production default and is already validated by smoke #9 (Qwen3-8B + Qwen3-8B all-linear adapter + triton backend). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This was referenced May 19, 2026
szewang
pushed a commit
to szewang/sglang
that referenced
this pull request
May 26, 2026
2 tasks done
5 tasks done
4 tasks done
1 of 3 tasks
This was referenced Sep 5, 2026
Merged
1 task
4 of 5 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
add support for gemini backend