Skip to content

Fix constrained grammar stop-token handling - #32509

Closed
emilyhanyf wants to merge 1 commit into
sgl-project:mainfrom
modal-projects:constrained-decoding-fix
Closed

emilyhanyf wants to merge 1 commit into
sgl-project:mainfrom
modal-projects:constrained-decoding-fix

Conversation

@emilyhanyf

@emilyhanyf emilyhanyf commented Jul 27, 2026 •

Copy link
Copy Markdown

Union tokenizer stop tokens with model-configured EOS tokens so grammar-constrained generation can terminate consistently with runtime stop detection.

Motivation

Modifications

Accuracy Tests

Speed Tests and Profiling

Checklist

Review and Merge Process

  1. Ping Merge Oncalls to start the process. See the PR Merge Process.
  2. Get approvals from CODEOWNERS and other reviewers.
  3. Trigger CI tests with comments or contact authorized users to do so.
    • Common commands include /tag-and-rerun-ci, /tag-run-ci-label, /rerun-failed-ci
  4. After green CI and required approvals, ask Merge Oncalls or people with Write permission to merge the PR.

CI States

Latest PR Test (Base): ❌ Run #30252840626
Latest PR Test (Extra): ❌ Run #30252838665

Union tokenizer stop tokens with model-configured EOS tokens so grammar-constrained generation can terminate consistently with runtime stop detection.

Co-authored-by: Cursor <cursoragent@cursor.com>
@gemini-code-assist

Copy link
Copy Markdown
Contributor

Caution

The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased.

@github-actions

Copy link
Copy Markdown
Contributor

Thanks @emilyhanyf. Closing this because it has had no updates in 91 days.

Reopen it if the work is still relevant.

Some directories moved recently, so an older branch may need retargeting:
sgl-kernel/ -> python/sglang/kernels/aot/, python/sglang/jit_kernel/
-> python/sglang/kernels/jit/, docs/ -> docs/docs/ (.mdx),
bench_serving.py -> benchmark/serving.py, test/srt/ -> test/registered/.

@github-actions github-actions Bot closed this Oct 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants