Add alpaca chat template (repush of #7383)

[jukofyork]

This PR adds the commonly used 'alpaca' chat template (repush of #7383 due to conflicts).

I can't actually find any models on huggingface that have the 'alpaca' template in their tokenizer_config.json files so had to manually add it to the Templates-supported-by-llama_chat_apply_template script (edit: see lower for corrected wiki python script addition).

I used the Jinga template given by text-generation-webui:

https://github.com/oobabooga/text-generation-webui/blob/main/instruction-templates/Alpaca.yaml

---

I should add that even though this template isn't actually used much now, people still do use it for creative writing as a workaround for the problematic Mistral [INST] type templates, so having the ability to manually specify 'alpaca' for these models would help.

It is also subtly different to the existing deepseek-coder template: no'<|EOT|>' and also uses double newlines (edit: and as I found out in the other PR; there is no </s> EOS to be added!).

I also chose meta-math/MetaMath-7B-V1.0 fairly arbitrarily, but this template is used in lots of other commonly used models, such as: the wizard family of models, phind-codellama, etc.

---

• I have read the contributing guidelines
• Self-reported review complexity:
  • Low
  • Medium
  • High

[jukofyork]

Somehow lost the other PR when I tried to fix the conflicts so made a new one.

I have tested this quite a lot using phind-codellama and can confirm it is working as expected for multi-turn alpaca (and appears to work equivalently to Ollama's Go text/template version of the same).

[jukofyork]

Copied these important posts from the other PR:

I'm also not sure if the Python script above should do something to deal with default system messages that end with a newline, eg:

----- deepseek-ai/deepseek-coder-33b-instruct -----
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
<｜begin▁of▁sentence｜>You are a helpful assistant### Instruction:
Hello
### Response:
Hi there
<|EOT|>
### Instruction:
Who are you
### Response:
   I am an assistant   
<|EOT|>
### Instruction:
Another question
### Response:

Doesn't look correct to me for deepseek-coder as the default system message ends with a newline like so:

----- deepseek-ai/deepseek-coder-33b-instruct -----
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
<｜begin▁of▁sentence｜>You are a helpful assistant
### Instruction:
Hello
### Response:
Hi there
<|EOT|>
### Instruction:
Who are you
### Response:
   I am an assistant   
<|EOT|>
### Instruction:
Another question
### Response:

Perhaps the Python script should detect this and move the newline from the default system message into the template itself?

---

I've figured it out now! The oobabooga/text-generation-webui/ template it actually correct:

https://github.com/oobabooga/text-generation-webui/blob/main/instruction-templates/Alpaca.yaml

and there is no </s> EOS to be added!

---

This is the code that I used to generate the tests using the oobabooga/text-generation-webui template if the wiki needs to be updated:

from transformers import AutoTokenizer

VARIANTS_TO_TEST = [
    'deepseek-ai/deepseek-coder-33b-instruct',
    'meta-math/MetaMath-7B-V1.0',
]

HISTORY = [
    { 'role': 'system', 'content': 'You are a helpful assistant' },
    { 'role': 'user', 'content': 'Hello' },
    { 'role': 'assistant', 'content': 'Hi there' },
    { 'role': 'user', 'content': 'Who are you' },
    { 'role': 'assistant', 'content': '   I am an assistant   ' },
    { 'role': 'user', 'content': 'Another question' },
]

for variant in VARIANTS_TO_TEST:
    history = [m for m in HISTORY] # copy
    if 'meta-math' in variant:
        print("\n----- Alpaca -----")
        ALPACA_TMPL = "{%- set ns = namespace(found=false) -%} {%- for message in messages -%} {%- if message['role'] == 'system' -%} {%- set ns.found = true -%} {%- endif -%} {%- endfor -%} {%- if not ns.found -%} {{- '' + 'Below is an instruction that describes a task. Write a response that appropriately completes the request.' + '\n\n' -}} {%- endif %} {%- for message in messages %} {%- if message['role'] == 'system' -%} {{- '' + message['content'] + '\n\n' -}} {%- else -%} {%- if message['role'] == 'user' -%} {{-'### Instruction:\n' + message['content'] + '\n\n'-}} {%- else -%} {{-'### Response:\n' + message['content'] + '\n\n' -}} {%- endif -%} {%- endif -%} {%- endfor -%} {%- if add_generation_prompt -%} {{-'### Response:\n'-}} {%- endif -%}"
        output = AutoTokenizer.from_pretrained(VARIANTS_TO_TEST[1]).apply_chat_template(history, tokenize=False, add_generation_prompt=True, chat_template=ALPACA_TMPL)
        print(output)
        print("\n[Test String]\n// meta-math/MetaMath-7B-V1.0")
        print(output.replace("\n", "\\n"))
        print('"' + output.replace("\n", "\\n") + '",')
    else:
        print("\n----- " + variant + " -----")
        tokenizer = AutoTokenizer.from_pretrained(variant)
        output = tokenizer.apply_chat_template(history, tokenize=False, add_generation_prompt=True)
        print(output)
        print("\n[Test String]\n// " + variant)
        print('"' + output.replace("\n", "\\n") + '",')

[ngxson]

In any cases, having \n\n after system message makes more sense to me. Could we delete the template for deepseek below and move it here?

} else if (tmpl == "alpaca" || tmpl == "deepseek" || (tmpl.find("### Instruction:") != std::string::npos && tmpl.find("<|EOT|>") == std::string::npos)) {

[jukofyork]

I think it might give people the impression that they are the same prompt though when actually they are quite different: deepseek-coder requires an EOS token (and a non-default <|EOT|> vs </s> one!) and from the two models I tested that can use multi-turn alpaca; it seems adding an EOS token like </s> confuses them quite badly (AFAIK, it's the only template like this).

[jukofyork]

If you look in the previous PR that I closed accidentally, then there are some examples of phind-codellama getting very confused when I tried to use the EOS token.

I am unsure how people use alpaca for creative writing, but am going to try it over the weekend using mikupad, which seems to have forgotten to add any EOS tokens to any of their prompts (but nobody seems to have noticed or cared, and the models seem to have been writing fiction fine!?).

[ngxson]

Probably we can also patch llama_chat_apply_template_internal to take the EOS as an input (can be read from llama_model). Since this function is used internally, we can change the signature whenever we want.

Do you think that will be a better solution?

I am unsure how people use alpaca for creative writing, but am going to try it over the weekend using mikupad, which seems to have forgotten to add any EOS tokens to any of their prompts (but nobody seems to have noticed or cared, and the models seem to have been writing fiction fine!?).

That sounds to me like a bit of mess when the world moving from single-turn to multi-turn template 1 year ago. Back then, no one care about putting EOS after model's response because we can match stop sequence (### User:). In fact, for this reason, I was skeptical to add alpaca template in the first place, since some models are trained solely with single-turn dataset.

[jukofyork]

Probably we can also patch llama_chat_apply_template_internal to take the EOS as an input (can be read from llama_model). Since this function is used internally, we can change the signature whenever we want.

Do you think that will be a better solution?

It may be worth looking at the model files of the few genuine multiturn Alpaca models to see if they even have a EOS token defined. Sadly it was the now deleted "wizard" models that mainly used this but some of the early "meta" models and phind-codellama also used it.

I am unsure how people use alpaca for creative writing, but am going to try it over the weekend using mikupad, which seems to have forgotten to add any EOS tokens to any of their prompts (but nobody seems to have noticed or cared, and the models seem to have been writing fiction fine!?).

That sounds to me like a bit of mess when the world moving from single-turn to multi-turn template 1 year ago. Back then, no one care about putting EOS after model's response because we can match stop sequence (### User:). In fact, for this reason, I was skeptical to add alpaca template in the first place, since some models are trained solely with single-turn dataset.

The mikupad devs just fixed all their templates after I pointed this out.

[jukofyork]

A recent post showing Alpaca may even improve llama-3 for business use benchmark:

https://www.reddit.com/r/LocalLLaMA/comments/1dro5t0/mmlupro_fun_part_2_surprising_result_comparing/

Lots of discussion about its use for creative writing in the comments too.

I'm interested if things like SillyTavern add the </s> or not, as from my tests with the few genuine multiturn Alpaca models; it really hurt them to add this.

There's definitely something about its similarities to Markdown headers that is the cause I think. A few months ago I found most coding models could be improved by making the first character a # symbol eg # Your task is... but thought this phenomenon might be due to lots of training examples given via Python comments, etc.