Skip to content

Commit cfefe29

Browse files
DenisVieriu97DenisVieriu97
committed
[MPS] Update MPS documentation; add helper script to build mps_executor_runner
1 parent c78c45b commit cfefe29

File tree

2 files changed

+130
-35
lines changed

2 files changed

+130
-35
lines changed

‎backends/apple/mps/setup.md

+66-35
Original file line numberDiff line numberDiff line change
@@ -15,15 +15,28 @@ The MPS backend device maps machine learning computational graphs and primitives
1515
* [Introduction to ExecuTorch](intro-how-it-works.md)
1616
* [Setting up ExecuTorch](getting-started-setup.md)
1717
* [Building ExecuTorch with CMake](runtime-build-and-cross-compilation.md)
18+
* [ExecuTorch iOS Demo App](demo-apps-ios.md)
19+
* [ExecuTorch iOS LLaMA Demo App](llm/llama-demo-ios.md)
1820
:::
1921
::::
2022

2123

2224
## Prerequisites (Hardware and Software)
2325

24-
In order to be able to successfully build and run a model using the MPS backend for ExecuTorch, you'll need the following hardware and software components.
25-
- macOS 12 / iOS 15 or later (for MPS runtime)
26-
- Xcode command-line tools: xcode-select --install
26+
In order to be able to successfully build and run a model using the MPS backend for ExecuTorch, you'll need the following hardware and software components:
27+
28+
### Hardware:
29+
- A [mac](https://www.apple.com/mac/) for tracing the model
30+
31+
### Software:
32+
33+
- **Ahead of time** tracing:
34+
- [macOS](https://www.apple.com/macos/) 12
35+
36+
- **Runtime**:
37+
- [macOS](https://www.apple.com/macos/) >= 12.4
38+
- [iOS](https://www.apple.com/ios) >= 15.4
39+
- [Xcode](https://developer.apple.com/xcode/) >= 14.1
2740

2841
## Setting up Developer Environment
2942

@@ -40,47 +53,34 @@ In order to be able to successfully build and run a model using the MPS backend
4053
### AOT (Ahead-of-time) Components
4154

4255
**Compiling model for MPS delegate**:
43-
- In this step, you will generate a simple ExecuTorch program that lowers MobileNetV3 model to the MPS delegate. You'll then pass this Program(the `.pte` file) during the runtime to run it using the MPS backend.
56+
- In this step, you will generate a simple ExecuTorch program that lowers MobileNetV3 model to the MPS delegate. You'll then pass this Program (the `.pte` file) during the runtime to run it using the MPS backend.
4457

4558
```bash
4659
cd executorch
47-
python3 -m examples.apple.mps.scripts.mps_example --model_name="mv3" --bundled
60+
# Note: `mps_example` script uses by default the MPSPartitioner for ops that are not yet supported by the MPS delegate. To turn it off, pass `--no-use_partitioner`.
61+
python3 -m examples.apple.mps.scripts.mps_example --model_name="mv3" --bundled --use_fp16
62+
63+
# To see all options, run following command:
64+
python3 -m examples.apple.mps.scripts.mps_example --help
4865
```
4966

5067
### Runtime
5168

52-
**Building the MPS executor runner**
53-
- In this step, you'll be building the `mps_executor_runner` that is able to run MPS lowered modules.
54-
69+
**Building the MPS executor runner:**
5570
```bash
56-
# Build the mps_executor_runner
71+
# In this step, you'll be building the `mps_executor_runner` that is able to run MPS lowered modules:
72+
cd executorch
73+
./examples/apple/mps/scripts/build_mps_executor_runner.sh
74+
```
75+
76+
## Run the mv3 generated model using the mps_executor_runner
77+
5778
```bash
58-
# Build and install executorch
59-
cmake -DBUCK2="$BUCK" \
60-
-DCMAKE_INSTALL_PREFIX=cmake-out \
61-
-DCMAKE_BUILD_TYPE=Release \
62-
-DEXECUTORCH_BUILD_SDK=ON \
63-
-DEXECUTORCH_ENABLE_EVENT_TRACER=ON \
64-
-DEXECUTORCH_BUILD_MPS=ON \
65-
-DPYTHON_EXECUTABLE="$PYTHON_EXECUTABLE" \
66-
-Bcmake-out .
67-
cmake --build cmake-out -j9 --target install --config Release
68-
CMAKE_PREFIX_PATH="${PWD}/cmake-out/lib/cmake/ExecuTorch;${PWD}/cmake-out/third-party/gflags"
69-
# build mps_executor_runner
70-
rm -rf cmake-out/examples/apple/mps
71-
cmake \
72-
-DCMAKE_PREFIX_PATH="$CMAKE_PREFIX_PATH" \
73-
-DCMAKE_BUILD_TYPE=Release \
74-
-DPYTHON_EXECUTABLE="$PYTHON_EXECUTABLE" \
75-
-Bcmake-out/examples/apple/mps \
76-
examples/apple/mps
77-
78-
cmake --build cmake-out/examples/apple/mps -j9 --config Release
79-
80-
# Run the mv2 generated model using the mps_executor_runner
8179
./cmake-out/examples/apple/mps/mps_executor_runner --model_path mv3_mps_bundled_fp16.pte --bundled_program
80+
```
8281

83-
# You should see the following results. Note that no output file will be generated in this example:
82+
- You should see the following results. Note that no output file will be generated in this example:
83+
```
8484
I 00:00:00.003290 executorch:mps_executor_runner.mm:286] Model file mv3_mps_bundled_fp16.pte is loaded.
8585
I 00:00:00.003306 executorch:mps_executor_runner.mm:292] Program methods: 1
8686
I 00:00:00.003308 executorch:mps_executor_runner.mm:294] Running method forward
@@ -94,12 +94,43 @@ I 00:00:00.118731 executorch:mps_executor_runner.mm:438] Model executed successf
9494
I 00:00:00.122615 executorch:mps_executor_runner.mm:501] Model verified successfully.
9595
```
9696

97+
### [Optional] Run the generated model directly using pybind
98+
1. Make sure `pybind` MPS support was installed:
99+
```bash
100+
./install_requirements.sh --pybind mps
101+
```
102+
2. Run the `mps_example` script to trace the model and run it directly from python:
103+
```bash
104+
cd executorch
105+
# Check correctness between PyTorch eager forward pass and ExecuTorch MPS delegate forward pass
106+
python3 -m examples.apple.mps.scripts.mps_example --model_name="mv3" --no-use_fp16 --check_correctness
107+
# You should see following output: `Results between ExecuTorch forward pass with MPS backend and PyTorch forward pass for mv3_mps are matching!`
108+
109+
# Check performance between PyTorch MPS forward pass and ExecuTorch MPS forward pass
110+
python3 -m examples.apple.mps.scripts.mps_example --model_name="mv3" --no-use_fp16 --bench_pytorch
111+
```
112+
113+
### Profiling:
114+
1. [Optional] Generate an [ETRecord](./sdk-etrecord.rst) while you're exporting your model.
115+
```bash
116+
cd executorch
117+
python3 -m examples.apple.mps.scripts.mps_example --model_name="mv3" --generate_etrecord -b
118+
```
119+
2. Run your Program on the ExecuTorch runtime and generate an [ETDump](./sdk-etdump.md).
120+
```
121+
./cmake-out/examples/apple/mps/mps_executor_runner --model_path mv3_mps_bundled_fp16.pte --bundled_program --dump-outputs
122+
```
123+
3. Create an instance of the Inspector API by passing in the ETDump you have sourced from the runtime along with the optionally generated ETRecord from step 1.
124+
```bash
125+
python3 -m sdk.inspector.inspector_cli --etdump_path etdump.etdp --etrecord_path etrecord.bin
126+
```
127+
97128
## Deploying and Running on Device
98129

99130
***Step 1***. Create the ExecuTorch core and MPS delegate frameworks to link on iOS
100131
```bash
101132
cd executorch
102-
./build/build_apple_frameworks.sh --Release --mps
133+
./build/build_apple_frameworks.sh --mps
103134
```
104135

105136
`mps_delegate.xcframework` will be in `cmake-out` folder, along with `executorch.xcframework` and `portable_delegate.xcframework`:
@@ -123,4 +154,4 @@ In this tutorial, you have learned how to lower a model to the MPS delegate, bui
123154

124155
## Frequently encountered errors and resolution.
125156

126-
If you encountered any bugs or issues following this tutorial please file a bug/issue on the ExecuTorch repository, with hashtag **#mps**.
157+
If you encountered any bugs or issues following this tutorial please file a bug/issue on the [ExecuTorch repository](https://github.com/pytorch/executorch/issues), with hashtag **#mps**.

‎examples/apple/mps/scripts/build_mps_executor_runner.sh

+64
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,64 @@
1+
#!/usr/bin/env bash
2+
# Copyright (c) 2024 Apple Inc. All rights reserved.
3+
# Provided subject to the LICENSE file in the top level directory.
4+
5+
set -e
6+
7+
MODE="Release"
8+
OUTPUT="cmake-out"
9+
10+
usage() {
11+
echo "Usage: $0 [OPTIONS]"
12+
echo "Build frameworks for Apple platforms."
13+
echo "SOURCE_ROOT_DIR defaults to the current directory if not provided."
14+
echo
15+
echo "Options:"
16+
echo " --output=DIR Output directory. Default: 'cmake-out'"
17+
echo " --Debug Use Debug build mode. Default: 'Release'"
18+
echo "Example:"
19+
echo " $0 --output=cmake-out --Debug"
20+
exit 0
21+
}
22+
23+
for arg in "$@"; do
24+
case $arg in
25+
-h|--help) usage ;;
26+
--output=*) OUTPUT="${arg#*=}" ;;
27+
--Debug) MODE="Debug" ;;
28+
*)
29+
if [[ -z "$SOURCE_ROOT_DIR" ]]; then
30+
SOURCE_ROOT_DIR="$arg"
31+
else
32+
echo "Invalid argument: $arg"
33+
exit 1
34+
fi
35+
;;
36+
esac
37+
done
38+
39+
rm -rf "$OUTPUT"
40+
41+
cmake -DBUCK2="$BUCK" \
42+
-DCMAKE_INSTALL_PREFIX=cmake-out \
43+
-DCMAKE_BUILD_TYPE="$MODE" \
44+
-DEXECUTORCH_BUILD_SDK=ON \
45+
-DEXECUTORCH_ENABLE_EVENT_TRACER=ON \
46+
-DEXECUTORCH_BUILD_MPS=ON \
47+
-DPYTHON_EXECUTABLE="$PYTHON_EXECUTABLE" \
48+
-Bcmake-out .
49+
cmake --build cmake-out -j9 --target install --config "$MODE"
50+
CMAKE_PREFIX_PATH="${PWD}/cmake-out/lib/cmake/ExecuTorch;${PWD}/cmake-out/third-party/gflags"
51+
# build mps_executor_runner
52+
rm -rf cmake-out/examples/apple/mps
53+
cmake \
54+
-DCMAKE_PREFIX_PATH="$CMAKE_PREFIX_PATH" \
55+
-DCMAKE_BUILD_TYPE="$MODE" \
56+
-DPYTHON_EXECUTABLE="$PYTHON_EXECUTABLE" \
57+
-Bcmake-out/examples/apple/mps \
58+
examples/apple/mps
59+
60+
cmake --build cmake-out/examples/apple/mps -j9 --config "$MODE"
61+
62+
echo "Build succeeded!"
63+
64+
./cmake-out/examples/apple/mps/mps_executor_runner --model_path mps_logical_not.pte --bundled_program

0 commit comments

Comments
 (0)