Skip to content

[MPS] Update MPS documentation; add helper script to build mps_executor_runner #3329

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Apr 24, 2024
Merged

[MPS] Update MPS documentation; add helper script to build mps_executor_runner #3329

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
101 changes: 66 additions & 35 deletions backends/apple/mps/setup.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,15 +15,28 @@ The MPS backend device maps machine learning computational graphs and primitives
* [Introduction to ExecuTorch](intro-how-it-works.md)
* [Setting up ExecuTorch](getting-started-setup.md)
* [Building ExecuTorch with CMake](runtime-build-and-cross-compilation.md)
* [ExecuTorch iOS Demo App](demo-apps-ios.md)
* [ExecuTorch iOS LLaMA Demo App](llm/llama-demo-ios.md)
:::
::::


## Prerequisites (Hardware and Software)

In order to be able to successfully build and run a model using the MPS backend for ExecuTorch, you'll need the following hardware and software components.
- macOS 12 / iOS 15 or later (for MPS runtime)
- Xcode command-line tools: xcode-select --install
In order to be able to successfully build and run a model using the MPS backend for ExecuTorch, you'll need the following hardware and software components:

### Hardware:
- A [mac](https://www.apple.com/mac/) for tracing the model

### Software:

- **Ahead of time** tracing:
- [macOS](https://www.apple.com/macos/) 12

- **Runtime**:
- [macOS](https://www.apple.com/macos/) >= 12.4
- [iOS](https://www.apple.com/ios) >= 15.4
- [Xcode](https://developer.apple.com/xcode/) >= 14.1

## Setting up Developer Environment

Expand All @@ -40,47 +53,34 @@ In order to be able to successfully build and run a model using the MPS backend
### AOT (Ahead-of-time) Components

**Compiling model for MPS delegate**:
- In this step, you will generate a simple ExecuTorch program that lowers MobileNetV3 model to the MPS delegate. You'll then pass this Program(the `.pte` file) during the runtime to run it using the MPS backend.
- In this step, you will generate a simple ExecuTorch program that lowers MobileNetV3 model to the MPS delegate. You'll then pass this Program (the `.pte` file) during the runtime to run it using the MPS backend.

```bash
cd executorch
python3 -m examples.apple.mps.scripts.mps_example --model_name="mv3" --bundled
# Note: `mps_example` script uses by default the MPSPartitioner for ops that are not yet supported by the MPS delegate. To turn it off, pass `--no-use_partitioner`.
python3 -m examples.apple.mps.scripts.mps_example --model_name="mv3" --bundled --use_fp16

# To see all options, run following command:
python3 -m examples.apple.mps.scripts.mps_example --help
```

### Runtime

**Building the MPS executor runner**
- In this step, you'll be building the `mps_executor_runner` that is able to run MPS lowered modules.

**Building the MPS executor runner:**
```bash
# Build the mps_executor_runner
# In this step, you'll be building the `mps_executor_runner` that is able to run MPS lowered modules:
cd executorch
./examples/apple/mps/scripts/build_mps_executor_runner.sh
```

## Run the mv3 generated model using the mps_executor_runner

```bash
# Build and install executorch
cmake -DBUCK2="$BUCK" \
-DCMAKE_INSTALL_PREFIX=cmake-out \
-DCMAKE_BUILD_TYPE=Release \
-DEXECUTORCH_BUILD_SDK=ON \
-DEXECUTORCH_ENABLE_EVENT_TRACER=ON \
-DEXECUTORCH_BUILD_MPS=ON \
-DPYTHON_EXECUTABLE="$PYTHON_EXECUTABLE" \
-Bcmake-out .
cmake --build cmake-out -j9 --target install --config Release
CMAKE_PREFIX_PATH="${PWD}/cmake-out/lib/cmake/ExecuTorch;${PWD}/cmake-out/third-party/gflags"
# build mps_executor_runner
rm -rf cmake-out/examples/apple/mps
cmake \
-DCMAKE_PREFIX_PATH="$CMAKE_PREFIX_PATH" \
-DCMAKE_BUILD_TYPE=Release \
-DPYTHON_EXECUTABLE="$PYTHON_EXECUTABLE" \
-Bcmake-out/examples/apple/mps \
examples/apple/mps

cmake --build cmake-out/examples/apple/mps -j9 --config Release

# Run the mv2 generated model using the mps_executor_runner
./cmake-out/examples/apple/mps/mps_executor_runner --model_path mv3_mps_bundled_fp16.pte --bundled_program
```

# You should see the following results. Note that no output file will be generated in this example:
- You should see the following results. Note that no output file will be generated in this example:
```
I 00:00:00.003290 executorch:mps_executor_runner.mm:286] Model file mv3_mps_bundled_fp16.pte is loaded.
I 00:00:00.003306 executorch:mps_executor_runner.mm:292] Program methods: 1
I 00:00:00.003308 executorch:mps_executor_runner.mm:294] Running method forward
Expand All @@ -94,12 +94,43 @@ I 00:00:00.118731 executorch:mps_executor_runner.mm:438] Model executed successf
I 00:00:00.122615 executorch:mps_executor_runner.mm:501] Model verified successfully.
```

### [Optional] Run the generated model directly using pybind
1. Make sure `pybind` MPS support was installed:
```bash
./install_requirements.sh --pybind mps
```
2. Run the `mps_example` script to trace the model and run it directly from python:
```bash
cd executorch
# Check correctness between PyTorch eager forward pass and ExecuTorch MPS delegate forward pass
python3 -m examples.apple.mps.scripts.mps_example --model_name="mv3" --no-use_fp16 --check_correctness
# You should see following output: `Results between ExecuTorch forward pass with MPS backend and PyTorch forward pass for mv3_mps are matching!`

# Check performance between PyTorch MPS forward pass and ExecuTorch MPS forward pass
python3 -m examples.apple.mps.scripts.mps_example --model_name="mv3" --no-use_fp16 --bench_pytorch
```

### Profiling:
1. [Optional] Generate an [ETRecord](./sdk-etrecord.rst) while you're exporting your model.
```bash
cd executorch
python3 -m examples.apple.mps.scripts.mps_example --model_name="mv3" --generate_etrecord -b
```
2. Run your Program on the ExecuTorch runtime and generate an [ETDump](./sdk-etdump.md).
```
./cmake-out/examples/apple/mps/mps_executor_runner --model_path mv3_mps_bundled_fp16.pte --bundled_program --dump-outputs
```
3. Create an instance of the Inspector API by passing in the ETDump you have sourced from the runtime along with the optionally generated ETRecord from step 1.
```bash
python3 -m sdk.inspector.inspector_cli --etdump_path etdump.etdp --etrecord_path etrecord.bin
```

## Deploying and Running on Device

***Step 1***. Create the ExecuTorch core and MPS delegate frameworks to link on iOS
```bash
cd executorch
./build/build_apple_frameworks.sh --Release --mps
./build/build_apple_frameworks.sh --mps
```

`mps_delegate.xcframework` will be in `cmake-out` folder, along with `executorch.xcframework` and `portable_delegate.xcframework`:
Expand All @@ -123,4 +154,4 @@ In this tutorial, you have learned how to lower a model to the MPS delegate, bui

## Frequently encountered errors and resolution.

If you encountered any bugs or issues following this tutorial please file a bug/issue on the ExecuTorch repository, with hashtag **#mps**.
If you encountered any bugs or issues following this tutorial please file a bug/issue on the [ExecuTorch repository](https://github.com/pytorch/executorch/issues), with hashtag **#mps**.
64 changes: 64 additions & 0 deletions examples/apple/mps/scripts/build_mps_executor_runner.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
#!/usr/bin/env bash
# Copyright (c) 2024 Apple Inc. All rights reserved.
# Provided subject to the LICENSE file in the top level directory.

set -e

MODE="Release"
OUTPUT="cmake-out"

usage() {
echo "Usage: $0 [OPTIONS]"
echo "Build frameworks for Apple platforms."
echo "SOURCE_ROOT_DIR defaults to the current directory if not provided."
echo
echo "Options:"
echo " --output=DIR Output directory. Default: 'cmake-out'"
echo " --Debug Use Debug build mode. Default: 'Release'"
echo "Example:"
echo " $0 --output=cmake-out --Debug"
exit 0
}

for arg in "$@"; do
case $arg in
-h|--help) usage ;;
--output=*) OUTPUT="${arg#*=}" ;;
--Debug) MODE="Debug" ;;
*)
if [[ -z "$SOURCE_ROOT_DIR" ]]; then
SOURCE_ROOT_DIR="$arg"
else
echo "Invalid argument: $arg"
exit 1
fi
;;
esac
done

rm -rf "$OUTPUT"

cmake -DBUCK2="$BUCK" \
-DCMAKE_INSTALL_PREFIX=cmake-out \
-DCMAKE_BUILD_TYPE="$MODE" \
-DEXECUTORCH_BUILD_SDK=ON \
-DEXECUTORCH_ENABLE_EVENT_TRACER=ON \
-DEXECUTORCH_BUILD_MPS=ON \
-DPYTHON_EXECUTABLE="$PYTHON_EXECUTABLE" \
-Bcmake-out .
cmake --build cmake-out -j9 --target install --config "$MODE"
CMAKE_PREFIX_PATH="${PWD}/cmake-out/lib/cmake/ExecuTorch;${PWD}/cmake-out/third-party/gflags"
# build mps_executor_runner
rm -rf cmake-out/examples/apple/mps
cmake \
-DCMAKE_PREFIX_PATH="$CMAKE_PREFIX_PATH" \
-DCMAKE_BUILD_TYPE="$MODE" \
-DPYTHON_EXECUTABLE="$PYTHON_EXECUTABLE" \
-Bcmake-out/examples/apple/mps \
examples/apple/mps

cmake --build cmake-out/examples/apple/mps -j9 --config "$MODE"

echo "Build succeeded!"

./cmake-out/examples/apple/mps/mps_executor_runner --model_path mps_logical_not.pte --bundled_program
Loading