Repository navigation
Use model loader from vllm - #459
Merged
Merged
Conversation
timethink
pushed a commit
to timethink/sglang
that referenced
this pull request
Mar 9, 2025
vschandramourya
pushed a commit
to vschandramourya/sglang
that referenced
this pull request
Feb 3, 2026
EdwardXuy
pushed a commit
to shun8686/sglang
that referenced
this pull request
Apr 30, 2026
modify minimax
efschu
added a commit
to efschu/htsglang
that referenced
this pull request
Aug 3, 2026
…ockstep max (two all-reduces per layer), barrier skew reported per candidate; zero-parameter skew prediction 27.6 vs 27.9 ms/1k measured; FP8 ladder byte-identical, INT8 honestly has no prefill lever on this rig; floor-explosion root = harness clock ramp (-> sgl-project#459/sgl-project#483)
efschu
added a commit
to efschu/htsglang
that referenced
this pull request
Aug 3, 2026
… correct the §1.3 model, and make the sgl-project#431 guard non-blocking Two commits: test(sgl-project#483,sgl-project#459): sgl-project#483 ("check_regressions prices prefill on stale rates") is REFUTED -- check_regressions already prices on the RESOLVED rates (landed with sgl-project#475). What was actually missing was the PIN: the only test calling the function is checkpoint-gated and skips in every hermetic run, so a revert would be invisible. test_check_regressions_pricing_483.py is the hermetic falsifier. Also makes the s12 harness honest: an absent measurement is None, never 0.0 (a spec-off tick has no accept length, and 0.0 silently survives every arithmetic it enters); and any procedural property a verdict rests on (warm-up discarded, draws back-to-back, gap between draws) is recorded as a measured number in the artifact, not taken on the harness's word -- the s12 warm-up draw ran for months and was discarded silently, so a warmed-up point and a cold one wrote the same file. perf(sgl-project#517): names the actual decode-loop BAR1 collectives and corrects the catalog's round model -- a decode round is NOT free of host-path collectives (the belief the old model rested on). Per NEXTN round there are 3 replay boundaries (the draft chain is one captured graph, not one per step) and 5 host-path BAR1 broadcasts, all sgl-project#50 speculative rank-agreement syncs, reaching barlink because a barlink boot never constructs pynccl. Measured cost: the sgl-project#431 blocking status-word read costs -9.22% decode TPS against the same-tree NCCL baseline; removing it recovers +2.68%. Fix: the abort-guard status word is now read asynchronously (non-blocking D2H + cudaEventQuery, returning what an earlier check staged), trading reporting LATENCY for zero stream synchronization per check -- detection is unaffected because ctlStatus is sticky. SGLANG_BARLINK_BAR1_ABORT_MAX_LAG (default 4) bounds the deferral with a can-fail proof; ..._ABORT_DEFER=0 restores the exact pre-sgl-project#517 blocking behavior. NOTE_517_bar1_guard_desk.md has the full model, desk-only (no GPU arm run yet -- ticketed). Two real conflicts, both resolved by combining rather than choosing a side: - FEATURE_CATALOG.md §12 (same insertion-point pattern as every merge today): HEAD's "Verified carve-outs (sgl-project#514)" paragraph kept, this branch's two sgl-project#459 paragraphs ("absent measurement is None" / "procedural property must be in the artifact") appended after. Two OTHER hunks (the sgl-project#483-REFUTED note near §2, the "Guard cost" note near §7) landed with no conflict at all -- neither region had been touched since this branch's base. - test/registered/unit/distributed/_bar1_marker_source.py: both HEAD (via sgl-project#514's earlier edits) and this branch carry DIFFERENT LINE_* pins into barlink.py/barlink_bar1.py, because each was written against its own version of those files. Neither side's numbers are correct once both sets of source edits are combined. Resolved by reading the ACTUAL post-merge source: grepped the real logger.info/logger.warning/raise RuntimeError call-start lines in the merged barlink.py and barlink_bar1.py (2236/2250/3207/810) and verified all four renderers execute correctly against them before committing -- not a textual merge of the two guesses. All 17 catalog sections and every other additive paragraph verified present. Tests (CUDA_VISIBLE_DEVICES=99, PYTHONPATH pinned to this worktree): - The five directly touched/new test files (_bar1_marker_source.py's renderers verified live against the real merged source, plus test_barlink_bar1_abort_431.py, test_barlink_bar1_abort_deferred_517.py, test_s12_harness_459.py, test_check_regressions_pricing_483.py): 67 + 7 passed, 0 failed. - test_s12_log_analyse.py specifically (s12_log_analyse.py is touched): same 4 pre-existing failures, unchanged (fixture-data-dependent, not source-code-dependent). - test/registered/unit/distributed/ (full): 24 failed, 2206 passed, 8 skipped, 723 subtests -- same 24 failing IDs as the established baseline, byte-identical. - ruff check --select=F401,F821,UP037: clean. - codespell: 7 hits, all confirmed pre-existing (a German prose sample string already in scripts/gpu_battery/s12_prefill_kurve.py and s14_decode_punkt.py at this branch's base, untouched by its diff).
efschu
added a commit
to efschu/htsglang
that referenced
this pull request
Aug 3, 2026
… medians, honest server state Three new server-side computed blocks the Monitor tab's landing poll now returns (all server-side so a curl client sees exactly what the page draws, per the runbook's "curl is a complete client" architecture rule): - server_state.py: a four-state discriminator (not_running | starting | running_no_metrics | running_with_metrics) that keys off an explicit API probe, never a metrics-scrape failure alone -- a connection-refused scrape used to be rendered as "no --enable-metrics", which is a different diagnosis than "not running at all". running_no_metrics is only ever reported with a passing API probe. - rate_medians.py: medians behind the rate tiles, windowed over PROCESSING polls only -- idle polls never enter the window, so an idle server still reports the median of what it did while working, not a zero-diluted average. - spill_tiers.py / tier_occupancy.py: one row per spill/offload tier type x place, with absent tiers PRINTED alongside their reason rather than hidden or shown as zero. Two tiers are deliberately never populated: the sgl-project#407 memtier registry (no production consumer, so reading it would print zeros that look measured) and the sgl-project#286 short-term register (no reachable byte ledger in this build) -- both documented as absent-by- construction, not omissions. webui.py and live_metrics.py wire these into the existing landing poll; metrics_reporter.py and metrics_collector.py get the small hooks the new blocks read from. Clean auto-merge on both touched docs (FEATURE_CATALOG.md and rig-runbook.md) -- neither had a conflicting edit on this line since this branch's base (3e3aca2, the sgl-project#510 merge). rig-runbook.md gains new §8.0 documenting the shell-only verification recipe (curl + three python3 one-liners for state/medians/tiers) and the restart requirement: the dashboard serves from one long-lived process, so a change is live only after that process restarts -- done as the merge step below, not mid-session. All 17 catalog sections and all 21 previously-preserved additive paragraphs verified present (two-axis §1, matrix doctrine §2, all six §12 families from sgl-project#510/sgl-project#513/sgl-project#511/sgl-project#487/qwen38-prep, sgl-project#514's five families + value-pinning, sgl-project#499's resolution-ordering, sgl-project#517's two sgl-project#459 paragraphs, MERGE-DUTY owner-sites + SITREP §12, NVENC §13). Tests (CUDA_VISIBLE_DEVICES=99, PYTHONPATH pinned to this worktree): - The six directly touched/new test files: 517 passed, 0 failed. - test/registered/unit/planner/ (full, HTSGLANG_TEST_MODEL_DIR set): 2337 passed, 1 skipped, 352 subtests -- no failures. - ruff check --select=F401,F821,UP037: 1 finding (live_metrics.py's LiveSnapshot import), confirmed pre-existing -- already unused in the pre-merge base, only mentioned in a docstring, the line just shifted from sgl-project#522's unrelated insertions above it. - codespell: 1 hit (webui.py:1786 "unparseable"), confirmed pre-existing at the identical line in the pre-merge base, in unrelated SVG chess-piece-parsing code this branch never touches.
efschu
added a commit
to efschu/htsglang
that referenced
this pull request
Aug 3, 2026
…-cost probe, the F2 instrument sgl-project#462 (the breakable MoE-offload CUDA-graph route) was already merged onto this line long ago; this pulls in two new commits added on top since, building the sgl-project#494 F2 measurement instrument that ticket named as missing. srt/utils/break_cost_clock.py prices ONE CUDA-graph crossing of a breakable capture (segment_end -> eager slot -> segment_start) with a CUDA event pair around every segment and every break slot in BreakableCUDAGraph.replay(). Per crossing: gap_in_ms / slot_ms / gap_out_ms (device wait, device compute, device wait) plus host_ms split into the four F2 terms (rendezvous, planning, publish, fetch), bracketed via break_cost_phase() in MoEExpertOffloadCache.prepare_breakable; per round: compute_ms / wait_ms / span_ms / residual_ms, per rank, as JSONL. Read via SGLANG_BREAK_COST_DEFER_ROUNDS (default 2) rounds late through query() only, so the read never synchronises the round it measures. SGLANG_BREAK_COST_PROBE=1, OFF by default and byte-neutral off (the disabled replay() loop is the original one statement-for-statement -- no event created, none recorded, nothing allocated per round, pinned by a call-count spy). Route-agnostic: any eager_on_graph break point is priced, not only the MoE one. Reader: scripts/dev/494_break_cost/summarise.py. 21 hermetic tests, three executed can-fail arms. No GPU number exists yet -- the probe has never run on a card; it is the instrument F2 was missing, not a result. FEATURE_CATALOG.md §16 conflict as expected: this branch's own base predates every merge from today (sgl-project#493/sgl-project#505/sgl-project#459/etc.), so its diff carried the OLD short §16 base prose alongside its new "Break-cost probe" paragraph. Kept HEAD's full precise §16 (all of today's corrections) and appended the Break-cost-probe paragraph after it, discarding only the superseded old base text the branch's own diff had inherited. §3's small in-place edit ("its instrument now exists (sgl-project#494, §16 break-cost probe)") landed with no conflict at all -- untouched by anything else on the line. All 17 sections and every other additive paragraph verified present. Tests (CUDA_VISIBLE_DEVICES=99, PYTHONPATH pinned to this worktree; test tree is tests/moe_offload/, not test/registered/): - test_break_cost_probe_494.py + test_breakable_route_462.py: 58 passed (21 + 37, matching the catalog's own count for each file). - tests/moe_offload/ (full): 2 failed, 455 passed, 34 skipped -- both failures in test_gguf_moe_offload.py, a file this branch's diff never touches. Bisected across today's entire merge history AND the prior session's tips back to 9fab358 (before sgl-project#500's audit branch base): the same 2 failures reproduce identically at every single point, including before this session's work began. Confirmed NOT a new regression from sgl-project#494/sgl-project#462 or anything merged today -- a long-standing pre-existing issue, out of scope for this merge. - ruff check --select=F401,F821,UP037: 18 findings in expert_offload.py and break_cost_clock.py (including two F821 undefined-name hits), ALL confirmed pre-existing at cc5a92e and outside this branch's three diff hunks (import addition + two break_cost_phase() wraps). - codespell: clean on all 6 touched/added files.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.