Repository navigation
docs: add llama 3.1 405b instruction - #739
Merged
Merged
Conversation
Co-authored-by: Ying1123 <sqy1415@gmail.com>
timethink
pushed a commit
to timethink/sglang
that referenced
this pull request
Mar 9, 2025
Co-authored-by: Ying1123 <sqy1415@gmail.com>
cen121212
pushed a commit
to cen121212/sglang
that referenced
this pull request
Nov 10, 2025
<!-- Thank you for your contribution! Please follow these guidelines to enhance your pull request. If anything is unclear, submit your PR and reach out to maintainers for assistance. Join our Slack community at https://slack.sglang.ai to discuss further. --> ## Motivation <!-- Describe the purpose and goals of this pull request. --> ## Modifications <!-- Detail the changes made in this pull request. --> ## Accuracy Tests <!-- If this pull request affects model outputs (e.g., changes to the kernel or model forward code), provide accuracy test results. --> ## Benchmarking and Profiling <!-- If this pull request impacts inference speed, provide benchmarking and profiling results. --> ## Checklist - [x] Format your code according to the [Format code with pre-commit](https://docs.sglang.ai/developer_guide/contribution_guide.html#format-code-with-pre-commit). - [x] Add unit tests according to the [Run and add unit tests](https://docs.sglang.ai/developer_guide/contribution_guide.html#run-and-add-unit-tests). - [x] Update documentation according to [Write documentations](https://docs.sglang.ai/developer_guide/contribution_guide.html#write-documentations). - [x] Provide accuracy and speed benchmark results according to [Test the accuracy](https://docs.sglang.ai/developer_guide/contribution_guide.html#test-the-accuracy) and [Benchmark the speed](https://docs.sglang.ai/developer_guide/contribution_guide.html#benchmark-the-speed).
cherryblo
added a commit
to cherryblo/sglang-project
that referenced
this pull request
Jul 2, 2026
update minimax scripts
efschu
added a commit
to efschu/htsglang
that referenced
this pull request
Aug 17, 2026
…ct#739 prefill-progress stamp
efschu
pushed a commit
to efschu/htsglang
that referenced
this pull request
Aug 17, 2026
…- unstrangle the funder My sgl-project#744 fix has a live regression and this is the refinement. Specimen 21:46:32 on 72696b0aec: tp_to_pp refused 35x, IDLE-LOCK with 407,622 tokens pending and 0 resident, the guard reporting "no KV provider is registered" because the armed-only gate refused the rung at BOTH sites and left the provider list empty. Seam staging (PP1: needs 1269 MiB, spendable 883, rung wanted SHRINK to 126506 with slack 45526 = a 1455 MiB deficit that WAS coverable) could never be funded. THE ERROR WAS MINE AND IT WAS A DESIGN ERROR, not a slip. Seam-funding eviction of recomputable prefix rows is REQUESTED BY the flip machinery -- it is what "KV capacity is the funder" means -- so refusing the rung for the duration of a flip protects against the 21:18 crash by disabling the thing the flip is waiting on. A wholesale gate cannot be right when the protected party and the requesting party are the same machinery. THERE WERE TWO STRANGLE POINTS, not one. The armed gate was the obvious half. The second was quieter: _nothing_resident() returning False on a parked extent dropped through to the unknown-refuse branch, so even with the gate removed the rung would still have declined. Both are fixed. THE REFINEMENT. The parked extent already carries exactly the information needed to be selective, which is why this is clean rather than a compromise: rows INSIDE it are what the flip is about to pack and may not be touched; every row ABOVE it is recomputable prefix and is precisely what the funding wants. So the extent PINS THE CEILING instead of closing the rung. _parked_ceiling() returns the highest parked row id, -1 when nothing is parked, and -2 for the one case that still refuses wholesale: an UNKNOWN extent while a flip is armed, where there is no boundary to name. Both call sites take req_max = max(req_max, parked), so the evictor is handed a ceiling that cannot reach into the extent. UNKNOWN, DECIDED AND DOCUMENTED as asked: it refuses ONLY while a flip is armed. Outside a flip there is nothing parked to protect, and closing the rung there is exactly the strangle this commit removes. sgl-project#746 (the exact arm-time snapshot) is what removes this last wholesale case; until then it is one narrow, named condition rather than the whole flip window. TEST RESULTS test_evict_rung_flip_park_744.py refined to the new semantics, 19 tests + 2 subtests green. The two tests that asserted the wholesale gate are REPLACED rather than deleted -- they now assert the funding path delivers -- and the crash protection gets its own explicit test that the evictor's ceiling never reaches into the parked extent. Mutation matrix, 4 mutants, ALL KILLED, covering both directions: N1 exclusion removed (parked ignored) -> 1 failed CRASH protection dies N2 exclusion covers everything (gate back) -> 1 failed FUNDING dies N3 collecting site ignores parked ceiling -> 2 failed N4 unknown-while-armed no longer refuses -> 3 failed N1 and N2 are the (c) pair: neither over- nor under-covering survives. Pinned suites green together, 142 passed + 87 subtests: sgl-project#744, sgl-project#731 + the #731x#744 interaction pin, both sgl-project#717 suites, sgl-project#656, sgl-project#684, sgl-project#662-f4, sgl-project#713 admission intake, sgl-project#739 prefill progress. Ruff and codespell clean. PP2 SECONDARY, checked and NOT claimed as fixed. The 0 MiB shrink against an unmoved driver-free column is an already-instrumented condition (kv_backing_relief.py:1915-1927): the code detects it, names retained handles (SGLANG_FLIP_SEAM_RETAIN_HANDLES) or a missing commit chunk as the cause, keeps the cap on and returns 0. The gate refinement lets the rung PRICE again; it cannot make a retained-handle arena PAY. Necessary, not sufficient -- the retain setting needs verifying at the boot, and this commit does not pretend to close it. No boot.
efschu
pushed a commit
to efschu/htsglang
that referenced
this pull request
Aug 17, 2026
…ip named Pass 2's state was unverified after the process-exit waves, so this pass began by CHECKING every worklist item against the tip rather than by merging. Five of eight were already on the train, verified with merge-base --is-ancestor rather than assumed: feat/407-registry-reconcile (ahead=0) + docs/407 + VERDICT_407 present the two DESIGN_407 corrections filed under sgl-project#732 -- both committed fix/485-gdn-family-report (ahead=0) feat/363-remainder (ahead=0) <- the worklist flagged this MISSING. STALE. fix/602-fill-side (ahead=0) the sgl-project#710 adapter item -- the named site carries the repair, from sgl-project#540 Merged (all clean, zero conflicts): origin/train/0817-desk -- sgl-project#737 ack drain, sgl-project#734 dead-peer discriminator, sgl-project#721 help text, and sgl-project#738 direct-io which F4-r4 pushed during the pass, resolving that item feat/516-miss-slot-budget feat/553-remainder fix/739-prefill-progress-signal Plus sgl-project#748 shipped ON the train before the freeze: my sgl-project#744 armed-only gate strangled the flip's own funder (35 refused tp_to_pp, IDLE-LOCK at 407,622 tokens pending), and the review boot must not carry the unrefined gate. TEST EVIDENCE, per-suite and compared by FAILED/ERROR line rather than total: baseline before any merge 57 failed, 4 errors, 5510 passed after the three main merges 57 failed, 4 errors, 5575 passed new failures ZERO, disappeared ZERO Pinned suites after sgl-project#748: 142 passed + 87 subtests. ONE FINDING RECORDED RATHER THAN RE-RUN AWAY. Three later full-scope runs showed ~50 extra failures in the distributed/ collective-floor family, and the obvious suspect was the last merge. sgl-project#739 is EXONERATED by a same-commit contradiction: commit 3c49cbc, the tree BEFORE sgl-project#739, was run twice on the identical scope -- once ZERO new, once FIFTY. Same commit, same command, opposite results. Corroborated: the suites pass in isolation, pairing each new suite with a victim stays green, and distributed/ alone gives 27 rather than 27+50. So the combined run is order-dependent at ~50 tests on this lineage: a test-infrastructure defect worth its own ticket, not a property of this train. A suite that gives fifty different answers on one commit cannot gate anything, and the tempting move -- re-running until it looked green -- would have buried that. NOT on the train: Slot-2's sgl-project#747, which arrived after the freeze. Nothing else was withheld; every worklist item is merged or verified present.
efschu
pushed a commit
to efschu/htsglang
that referenced
this pull request
Oct 23, 2026
…ueue clock (Qwen review gap of ee5fb03) A 25 s old queue with a prefill chunk 1 s ago does not alarm; the same timeline without the prefill-progress signal still alarms. Written by a Qwen agent (deskq/done/qwen-test-flipverdict-739-1023.py), taken over unchanged. pytest_gedeckelt: 2 passed.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Thank you for your contribution, we really appreciate it. The following instructions will help improve your pull request and make it easier to receive feedback. If there are any items you don't understand, don't worry. Just submit the pull request and ask the maintainers for help.
Motivation
Please explain the motivation behind this PR and the goal you aim to achieve with it.
Modification
Briefly describe the changes made in this PR.
Checklist
pre-commit run --all-filesor other linting tools are used to fix potential lint issues.