diff --git a/examples/models/llama2/README.md b/examples/models/llama2/README.md index a437f2c7a8f..b996e89cce6 100644 --- a/examples/models/llama2/README.md +++ b/examples/models/llama2/README.md @@ -260,6 +260,16 @@ Please refer to [this tutorial](https://pytorch.org/executorch/main/llm/llama-de ### Android Please refer to [this tutorial](https://pytorch.org/executorch/main/llm/llama-demo-android.html) to for full instructions on building the Android LLAMA Demo App. +## Optional: Smaller models delegated to other backends +Currently we supported lowering the stories model to other backends, including, CoreML, MPS and QNN. Please refer to the instruction +for each backend ([CoreML](https://pytorch.org/executorch/main/build-run-coreml.html), [MPS](https://pytorch.org/executorch/main/build-run-mps.html), [QNN](https://pytorch.org/executorch/main/build-run-qualcomm.html)) before trying to lower them. After the backend library is installed, the script to export a lowered model is + +- Lower to CoreML: `python -m examples.models.llama2.export_llama -kv --coreml -c stories110M.pt -p params.json` +- MPS: `python -m examples.models.llama2.export_llama -kv --mps -c stories110M.pt -p params.json` +- QNN: `python -m examples.models.llama2.export_llama -kv --qnn -c stories110M.pt -p params.json` + +The iOS LLAMA app supports the CoreML and MPS model and the Android LLAMA app supports the QNN model. On Android, it also allow to cross compiler the llama runner binary, push to the device and run. + # What is coming next? ## Quantization - Enabling FP16 model to leverage smaller groupsize for 4-bit quantization.