Skip to content

Update docs for the demo app. #2921

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Binary file added docs/source/_static/img/llama_ios_app.mp4
Binary file not shown.
Binary file added docs/source/_static/img/llama_ios_app.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
2 changes: 1 addition & 1 deletion docs/source/demo-apps-ios.md
Original file line number Diff line number Diff line change
@@ -1 +1 @@
```{include} ../../examples/demo-apps/apple_ios/README.md
```{include} ../../examples/demo-apps/apple_ios/ExecuTorchDemo/README.md
2 changes: 1 addition & 1 deletion docs/source/llm/getting-started.md
Original file line number Diff line number Diff line change
Expand Up @@ -626,7 +626,7 @@ The rest of the steps will be the same as the normal flow. Now you can run this
## How to build Mobile Apps
You can also execute an LLM using ExecuTorch on iOS and Android

**For iOS details see the [iOS Sample App](https://github.com/pytorch/executorch/tree/main/examples/demo-apps/apple_ios).**
**For iOS see the [iLLaMA App](https://pytorch.org/executorch/main/llm/llama-demo-ios.html).**


**For Android see the [Android Instructions](https://pytorch.org/executorch/main/llm/llama-demo-android.html).**
2 changes: 2 additions & 0 deletions docs/source/llm/llama-demo-ios.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
```{include} ../../../examples/demo-apps/apple_ios/LLaMA/README.md
```
37 changes: 37 additions & 0 deletions examples/demo-apps/apple_ios/LLaMA/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
# Building ExecuTorch LLaMA iOS Demo App

This app demonstrates the use of the LLaMA chat app demonstrating local inference use case with ExecuTorch.

<img src="../_static/img/llama_ios_app.png" alt="iOS LLaMA App" /><br>

## Prerequisites
* [Xcode 15](https://developer.apple.com/xcode).
* [iOS 17 SDK](https://developer.apple.com/ios).
* Set up your ExecuTorch repo and environment if you haven’t done so by following the [Setting up ExecuTorch](https://pytorch.org/executorch/stable/getting-started-setup) to set up the repo and dev environment.

## Exporting models
Please refer to the [ExecuTorch Llama2 docs](https://github.com/pytorch/executorch/blob/main/examples/models/llama2/README.md) to export the model.

## Run the App

1. Open the [project](https://github.com/pytorch/executorch/blob/main/examples/demo-apps/apple_ios/LLaMA/LLaMA.xcodeproj) in Xcode.
2. Run the app (cmd+R).
3. In app UI pick a model and tokenizer to use, type a prompt and tap the arrow buton as on the [video](../_static/img/llama_ios_app.mp4).

```{note}
ExecuTorch runtime is distributed as a Swift package providing some .xcframework as prebuilt binary targets. Xcode will dowload and cache the package on the first run, which will take some time.
```

## Copy the model to Simulator

1. Drag&drop the model and tokenizer files onto the Simulator window and save them somewhere inside the iLLaMA folder.
2. Pick the files in the app dialog, type a prompt and click the arrow-up button.

## Copy the model to Device

1. Wire-connect the device and open the contents in Finder.
2. Navigate to the Files tab and drag&drop the model and tokenizer files onto the iLLaMA folder.
3. Wait until the files are copied.

## Reporting Issues
If you encountered any bugs or issues following this tutorial please file a bug/issue here on [Github](https://github.com/pytorch/executorch/issues/new).
8 changes: 5 additions & 3 deletions examples/models/llama2/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -218,11 +218,13 @@ adb push cmake-out-android/examples/models/llama2/llama_main /data/local/tmp/lla
```
adb shell "cd /data/local/tmp/llama && ./llama_main --model_path <model.pte> --tokenizer_path <tokenizer.bin> --prompt "Once upon a time" --seq_len 120
```
## Step 6: Build iOS and/or Android apps
## Step 6: Build Mobile apps

TODO
### iOS

Please refer to [this tutorial](https://pytorch.org/executorch/main/llm/llama-demo-ios.html) to for full instructions on building the iOS LLAMA Demo App.

### Android app
### Android
Please refer to [this tutorial](https://pytorch.org/executorch/main/llm/llama-demo-android.html) to for full instructions on building the Android LLAMA Demo App.

# What is coming next?
Expand Down
Loading