server : fix false speculative warning when not enabled#217
Conversation
Assisted-by: Antigravity
|
Thanks @ken00H — landed a corrected version of this on What was right in the PRGating speculative warnings + `common_speculative_init` on an actual non-`NONE` speculative type is correct. Default server startup was spamming false positives. Bug in the PR as writtenThe PR also moved `ctx_tgt_seq_rm_type = common_context_can_seq_rm(ctx_tgt)` inside the speculative-only branch. That field is not speculative-only — completion checkpointing uses it later: // server-context.cpp ~3297
do_checkpoint = do_checkpoint && (
ctx_tgt_seq_rm_type == COMMON_CONTEXT_SEQ_RM_TYPE_FULL ||
ctx_tgt_seq_rm_type == COMMON_CONTEXT_SEQ_RM_TYPE_RS ||
n_swa > 0);If speculative is off, the member would stay at the default `COMMON_CONTEXT_SEQ_RM_TYPE_NO` and FULL/RS models would skip checkpoints unless SWA forced them. Silent correctness regression for non-spec runs. What we landed instead
Metal `llama-server` build verified locally. Closing this PR in favor of the tip commit. |
Based on #217 (ken00H): skip speculative init and its startup warnings when no non-NONE speculative type is configured. Do NOT skip common_context_can_seq_rm(ctx_tgt) — ctx_tgt_seq_rm_type also drives completion checkpointing (FULL / RS models), independent of speculative decoding. Leaving it at the default NO would disable those checkpoints for non-spec runs. TurboQuant: no kernel impact.
Overview of the Change
Whenever you started llama-server , it checked if the context could remove tokens (sequence removal) and tried to set up speculative decoding. Because speculative decoding is disabled
by default, it would print these warnings during startup:
• "speculative decoding will use checkpoints"
• "no implementations specified for speculative decoding"
We added a check is_speculative_enabled in tools/server/server-context.cpp . It checks if the speculative types configured actually contain a speculative method (i.e. anything other
than NONE ).
If speculative decoding is not enabled, the server skips the check and initialization, keeping the startup logs clean of these false-positive warnings.