Feature Request: Self speculative decoding

Prerequisites

• I am running the latest code. Mention the version if possible as well.
• I carefully followed the README.md.
• I searched using keywords relevant to my issue to make sure that I am creating a new issue that is not already open (or closed).
• I reviewed the Discussions, and have a new and useful enhancement to share.

Feature Description

I suggest porting self speculative decoding from mainline if possible:

ggml-org/llama.cpp#18471

"The algorithm is simple: search for a pattern of length draft-min in the token history and use the subsequent draft-max tokens for speculation"

From what I gather, whenever the model produces a sequence of tokens of length "draft min" that has been already been written in the past, speculate that the next "draft-max" number of tokens will be the same as its continuation.

Motivation

Basically high t/s speeds improvement for inference with lots of rewriting of the same stuff, such as code refactoring or debugging.

Possible Implementation

ggml-org/llama.cpp#18471

[ikawrakow]

The PR in mainline looks interesting, but it is a large change, so the ideal solution would be to motivate the author to submit a PR also here.