Add Phi-4-mini-instruct

[jackzhxng]

Summary

Add phi-4-mini 3.8B with fractional rotary embeddings. Only works for short context, still need to implement longrope for longer sequence lengths.

Sample prompt and response (xnnpack + 8da4w quant):

> A California roll is a type of sushi roll that is unique to the state of California. It is made with the same basic ingredients used for regular sushi rolls, but often includes some unique ingredients native to the state of California. Here is the basic ingredients used for a California roll:

> Prefill time: 0.547189474105835
> Token generation (tok/s): 6.3541260502510895
> Peak memory: 2.3 GB
> .pte size: 2.3 GB

Closes #8813

Test plan

Convert weights:

python examples/models/phi-4-mini/convert_weights.py ~/.cache/huggingface/hub/models--microsoft--Phi-4-multimodal-instruct/snapshots/879783f7b23e43c12d1c682e3458f115f3a7718d/ phi_4_mini.pth

Export xnnpack with quantization:

python -m examples.models.llama.export_llama   --model phi-4-mini \
--params examples/models/phi-4-mini/config.json  --checkpoint phi_4_mini.pth \
-kv --use_sdpa_with_kv_cache  -X -d fp32 \
--metadata '{"get_bos_id":199999, "get_eos_ids":[200020,199999]}'  \
--output_name phi4_mini_x_8da_4w.pte
--verbose
-qmode 8da4w --group_size 128
--embedding-quantize 4,32
--quantize_kv_cache

Run via pybindings:

python -m examples.models.llama.runner.native --model ph-4-mini \
--pte phi4_mini_x_8da_4w.pte  \
--tokenizer ~/.cache/huggingface/hub/models--microsoft--Phi-4-multimodal-instruct/snapshots/879783f7b23e43c12d1c682e3458f115f3a7718d/tokenizer.json \
--tokenizer_config ~/.cache/huggingface/hub/models--microsoft--Phi-4-multimodal-instruct/snapshots/879783f7b23e43c12d1c682e3458f115f3a7718d/tokenizer_config.json \
--prompt "What ingredients are in a California roll?" \
--params examples/models/phi-4-mini/config.json --max_len 64 \
--temperature 0 -kv

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/8856

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 2 New Failures

As of commit a8231d8 with merge base 7aa6494 ():

NEW FAILURES - The following jobs have failed:

• pull / unittest / linux / linux-job (gh)
  backends/xnnpack/test/ops/test_conv1d.py::TestConv1d::test_qs8_conv1d_batchnorm_seq
• trunk / test-llama-runner-mac (fp32, xnnpack+custom+quantize_kv) / macos-job (gh)
  The process '/usr/bin/git' failed with exit code 128

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[jackzhxng]

@guangy10 any way I can run some on demand benchmarks for this model?

[iseeyuan]

This is awesome to enable a new model in a day!

[iseeyuan]

Any reason to remove it, probably it's mostly covered by llama tests?

[jackzhxng]

Oh it's just too large to run on every pull request, we only run the small ones on pull

[iseeyuan]

LGTM. Thanks

[iseeyuan]

It's nice that the exported .pte file can be verified via python binding. That inspired me if we could automate the process, similar to MLX. Essentially we could have hugging face model card name as input, and with a prompt we get the output.
I create #8872. Please let me know if it makes sense.