Skip to content

[Doc] docs: Add a developer guide for the plugin API - #13

Open
YAMY1234 wants to merge 1 commit into
sgl-project:mainfrom
YAMY1234:docs-developer-guide
Open

YAMY1234 wants to merge 1 commit into
sgl-project:mainfrom
YAMY1234:docs-developer-guide

Conversation

@YAMY1234

Copy link
Copy Markdown
Collaborator

Motivation

Contributors have asked for an example SGLang change that shows how modeling code calls the megakernels. There isn't one, by design: no SGLang model file calls a megakernel. SGLang's megakernel layer (sgl-project/sglang#44945) wraps the loaded model's forward, and this plugin decides per step whether to run its own step or the model's forward.

The README covers usage and the design rules, but not the plugin API that a change to an existing model, or a new model, works against. This adds that guide.

Modifications

  • docs/developer_guide.md (new), in four sections:
    1. How the plugin plugs in: the entry point and register(); the argument-resolution checks before the weights load; install_megakernel after the load and before KV cache sizing; the dispatch wrapper, plan → Step / None, and the capture check.
    2. What a step gets: the MegakernelBinding / ForwardView data; the tolerated calls, writes and reads; SGLANG_READS and the contract test that pins them; the draft_sampler factory.
    3. Worked example, deepseek_v41: spec.py → plan.py → runtime/ → the executor's layer loop; stock-equivalent step functions and how a slot swaps in a kernel; SHIPPED_SLOTS and SGLANG_NVIDIA_MEGAKERNEL_SLOTS.
    4. Adding a model: the package layout, the stock path first, the test tiers, and the design rules applied to a new model.
  • README.md: a link from Contributing, and docs/ in the repository tree.

Accuracy Tests

Not applicable: documentation only, no code change.

Speed Tests and Profiling

Not applicable: documentation only, no code change.

Checklist

@YAMY1234
YAMY1234 requested a review from Oasis-Git as a code owner October 20, 2026 04:07
How the plugin plugs into SGLang's megakernel layer (registration, the
argument-resolution checks, install after the load, the dispatch wrapper and
the capture check), what a covered step gets (the binding, the tolerated
calls, the contract test, the draft sampler factory), the deepseek_v41
package as a worked example, and how to add a model. Linked from the README.
@YAMY1234
YAMY1234 force-pushed the docs-developer-guide branch from edaa616 to 315d5ad Compare October 20, 2026 04:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant