Repository navigation
Conversation
How the plugin plugs into SGLang's megakernel layer (registration, the argument-resolution checks, install after the load, the dispatch wrapper and the capture check), what a covered step gets (the binding, the tolerated calls, the contract test, the draft sampler factory), the deepseek_v41 package as a worked example, and how to add a model. Linked from the README.
YAMY1234
force-pushed
the
docs-developer-guide
branch
from
October 20, 2026 04:08
edaa616 to
315d5ad
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
Contributors have asked for an example SGLang change that shows how modeling code calls the megakernels. There isn't one, by design: no SGLang model file calls a megakernel. SGLang's megakernel layer (sgl-project/sglang#44945) wraps the loaded model's
forward, and this plugin decides per step whether to run its own step or the model'sforward.The README covers usage and the design rules, but not the plugin API that a change to an existing model, or a new model, works against. This adds that guide.
Modifications
docs/developer_guide.md(new), in four sections:register(); the argument-resolution checks before the weights load;install_megakernelafter the load and before KV cache sizing; the dispatch wrapper,plan→Step/None, and the capture check.MegakernelBinding/ForwardViewdata; the tolerated calls, writes and reads;SGLANG_READSand the contract test that pins them; thedraft_samplerfactory.deepseek_v41:spec.py→plan.py→runtime/→ the executor's layer loop; stock-equivalent step functions and how a slot swaps in a kernel;SHIPPED_SLOTSandSGLANG_NVIDIA_MEGAKERNEL_SLOTS.README.md: a link from Contributing, anddocs/in the repository tree.Accuracy Tests
Not applicable: documentation only, no code change.
Speed Tests and Profiling
Not applicable: documentation only, no code change.
Checklist
pre-commit run --all-filespasses.test/cover the change: not applicable (documentation only)..github/CODEOWNERSthat names its owner: not applicable.