Skip to content

server: improve speed of speculative decoding#1119

Merged
ikawrakow merged 2 commits into
mainfrom
fcp/speculative_imprv
Jan 10, 2026
Merged

server: improve speed of speculative decoding#1119
ikawrakow merged 2 commits into
mainfrom
fcp/speculative_imprv

Fix n_batch_size not set to context size for draft model

e448b5c
Select commit
Loading
Failed to load commit list.

Workflow runs completed with no jobs