Skip to content

Change llama2 to llama - #595

Closed
ZackZeng999 wants to merge 4 commits into
sgl-project:mainfrom
ZackZeng999:zzw
Closed

ZackZeng999 wants to merge 4 commits into
sgl-project:mainfrom
ZackZeng999:zzw

Conversation

@ZackZeng999

Copy link
Copy Markdown
Contributor

I renamed the file llama2.py to llama.py and updated the references accordingly.

@ZackZeng999 ZackZeng999 closed this Jul 6, 2024
@ZackZeng999
ZackZeng999 deleted the zzw branch July 6, 2024 20:56
efschu pushed a commit to efschu/htsglang that referenced this pull request Aug 5, 2026
… backend-dispatched attention workspace term
efschu pushed a commit to efschu/htsglang that referenced this pull request Aug 13, 2026
…a caller

TASK 3 (missing boundaries) and TASK 6 (corridor_trace production call site).

NCCL BOUNDARY. TERM_NCCL_BUFFERS has been in the ledger taxonomy since sgl-project#595
with nowhere to be measured, so it read UNMEASURED on every boot ever
recorded. BOOT_PHASES gains nccl_init_begin/nccl_init_end and model_runner
marks them around the whole group-building block. The end mark sits AFTER the
phase-flip secondary groups, not after initialize_model_parallel: those groups
are communicators too, and a boundary drawn before them would price a launch's
NCCL buffers at whatever fraction the primary topology happens to build.

GDN SCRATCH GETS A NAMED BOUNDARY BUT NO CALL SITE, deliberately. The GDN
prefill scratch is allocated inside a forward pass, not at boot, so there is
no honest boot-phase home for it; inventing one would put a mark on the hot
path to satisfy a table. TERM_TO_POST names gdn_scratch_begin/end so the
reconciliation reports the term as UNMEASURED-with-a-named-missing-boundary --
a to-do with an address rather than a permanent blank.

CORRIDOR CALL SITE. R2 open item 1: the sampler was tested, ready and
callerless -- a module the tree carried and never ran. Scheduler gains
_corridor_trace_tick(), called beside _census_tick(). It arms ONCE (a sampler
per iteration would be one NVML thread per scheduler round), never raises, and
never retries a failed arming. Off by default in two independent ways: the
state lives in class-level attributes so an un-armed scheduler carries no
per-instance attribute at all, and corridor_trace.start() returns None unless
SGLANG_CORRIDOR_TRACE_MS is set.

Tests: 9 new (test_corridor_trace_call_site_605.py), red before the change
with AttributeError on Scheduler._corridor_trace_tick. The tick is exercised
by binding the REAL Scheduler method to a stub, not by mocking the method
under test, so a rename on the production side fails this file. Three tests
pin the off-by-default property and two pin never-raise/never-retry.
The existing phase-contract tests (every declared phase has a call site; no
call site invents a phase) both accept the new pair.
Suite: 424 passed, 1 failed (inherited test_communicator_group_contract_612).
black 26.1.0 clean.
whybeyoung added a commit to whybeyoung/sglang that referenced this pull request Sep 9, 2026
Per review: the additionalProperties fix we need (xgrammar sgl-project#595) shipped in
0.2.2, and 0.2.2 keeps apache-tvm-ffi>=0.1.9 — so it does not drag the
tvm-ffi pin (0.2.6 requires >=0.1.10, which breaks the MUSA vendored
apache_tvm_ffi 0.1.9.post2 install and a clean PyPI resolve). 0.2.2 also
predates 0.2.6's JSON carriage-return whitespace grammar change, avoiding
the reported test_json_openai unbounded-whitespace decode regression.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QKuuYNwwQvgNsYGeYidw5K
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant