Skip to content

Commit f060416

Browse files
AllentDangrimoiregrimoirehanrui1sensetimetehkillerbee
authored andcommitted
Torchscript support (open-mmlab#159)
* support torchscript * add nms * add torchscript configs and update deploy process and dump-info * typescript -> torchscript * add torchscript custom extension support * add ts custom ops again * support mmseg unet * [WIP] add optimizer for torchscript (open-mmlab#119) * add passes * add python api * Torchscript optimizer python api (open-mmlab#121) * add passes * add python api * use python api instead of executable * Merge Master, update optimizer (open-mmlab#151) * [Feature] add yolox ncnn (open-mmlab#29) * add yolox ncnn * add ncnn android performance of yolox * add ut * fix lint * fix None bugs for ncnn * test codecov * test codecov * add device * fix yapf * remove if-else for img shape * use channelshuffle optimize * change benchmark after channelshuffle * fix yapf * fix yapf * fuse continuous reshape * fix static shape deploy * fix code * drop pad * only static shape * fix static * fix docstring * Added mask overlay to output image, changed fprintf info messages to … (open-mmlab#55) * Added mask overlay to output image, changed fprintf info messages to stdout * Improved box filtering (filter area/score), make sure roi coordinates stay within bounds * clang-format * Support UNet in mmseg (open-mmlab#77) * Repeatdataset in train has no CLASSES & PALETTE * update result for unet * update docstring for mmdet * remove ppl for unet in docs * fix ort wrap about input type (open-mmlab#81) * Fix memleak (open-mmlab#86) * delete [] * fix build error when enble MMDEPLOY_ACTIVE_LEVEL * fix lint * [Doc] Nano benchmark and tutorial (open-mmlab#71) * add cls benchmark * add nano zh-cn benchmark and en tutorial * add device row * add doc path to index.rst * fix typo * [Fix] fix missing deploy_core (open-mmlab#80) * fix missing deploy_core * mv flag to demo * target link * [Docs] Fix links in Chinese doc (open-mmlab#84) * Fix docs in Chinese link * Fix links * Delete symbolic link and add links to html * delete files * Fix link * [Feature] Add docker files (open-mmlab#67) * add gpu and cpu dockerfile * fix lint * fix cpu docker and remove redundant * use pip instead * add build arg and readme * fix grammar * update readme * add chinese doc for dockerfile and add docker build to build.md * grammar * refine dockerfiles * add FAQs * update Dpplcv_DIR for SDK building * remove mmcls * add sdk demos * fix typo and lint * update FAQs * [Fix]fix check_env (open-mmlab#101) * fix check_env * update * Replace convert_syncbatchnorm in mmseg (open-mmlab#93) * replace convert_syncbatchnorm with revert_sync_batchnorm from mmcv * change logger * [Doc] Update FAQ for TensorRT (open-mmlab#96) * update FAQ * comment * [Docs]: Update doc for openvino installation (open-mmlab#102) * fix docs * fix docs * fix docs * fix mmcv version * fix docs * rm blank line * simplify non batch nms (open-mmlab#99) * [Enhacement] Allow test.py to save evaluation results (open-mmlab#108) * Add log file * Delete debug code * Rename logger * resolve comments * [Enhancement] Support mmocr v0.4+ (open-mmlab#115) * support mmocr v0.4+ * 0.4.0 -> 0.4.1 * fix onnxruntime wrapper for gpu inference (open-mmlab#123) * fix ncnn wrapper for ort-gpu * resolve comment * fix lint * Fix typo (open-mmlab#132) * lock mmcls version (open-mmlab#131) * [Enhancement] upgrade isort in pre-commit config (open-mmlab#141) * [Enhancement] upgrade isort in pre-commit config by refering to mmflow pr open-mmlab#87 * fix lint * remove .isort.cfg and put its known_third_party to setup.cfg * Fix ci for mmocr (open-mmlab#144) * fix mmocr unittests * remove useless * lock mmdet maximum version to 2.20 * pip install -U numpy * Fix capture_output (open-mmlab#125) Co-authored-by: hanrui1sensetime <[email protected]> Co-authored-by: Johannes L <[email protected]> Co-authored-by: RunningLeon <[email protected]> Co-authored-by: VVsssssk <[email protected]> Co-authored-by: lvhan028 <[email protected]> Co-authored-by: AllentDan <[email protected]> Co-authored-by: Yifan Zhou <[email protected]> Co-authored-by: 杨培文 (Yang Peiwen) <[email protected]> Co-authored-by: Semyon Bevzyuk <[email protected]> * configs for all tasks * use torchvision roi align * remote unnecessary code * fix ut * fix ut * export * det dynamic * det dynamic * add ut * fix ut * add ut and docs * fix ut * skip torchscript ut if no ops available * add torchscript option to build.md * update benchmark and resolve comments * resolve conflicts * rename configs * fix mrcnn cuda test * remove useless * add version requirements to docs and comments to codes * enable empty image exporting for torchscript and accelerate ORT inference for MRCNN * rebase * update example for torchscript.md * update FAQs for torchscript.md * resolve comments * only use torchvision roi_align for torchscript * fix ut * use torchvision roi align when pool model is avg * resolve comments Co-authored-by: grimoire <[email protected]> Co-authored-by: grimoire <[email protected]> Co-authored-by: hanrui1sensetime <[email protected]> Co-authored-by: Johannes L <[email protected]> Co-authored-by: RunningLeon <[email protected]> Co-authored-by: VVsssssk <[email protected]> Co-authored-by: lvhan028 <[email protected]> Co-authored-by: Yifan Zhou <[email protected]> Co-authored-by: 杨培文 (Yang Peiwen) <[email protected]> Co-authored-by: Semyon Bevzyuk <[email protected]>
1 parent 1d8a67b commit f060416

File tree

6 files changed

+231
-77
lines changed

6 files changed

+231
-77
lines changed

docs/en/build.md

Lines changed: 148 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -32,9 +32,151 @@ Note:
3232
git submodule update --init --recursive
3333
```
3434

35-
## Build MMDeploy
36-
Please visit the following links to find out how to build MMDeploy according to the target platform.
37-
- [Linux-x86_64](build/linux.md)
38-
- [Windows](build/windows.md)
39-
- [Android-aarch64](build/android.md)
40-
- [NVIDIA Jetson](tutorials/how_to_install_mmdeploy_on_jetsons.md)
35+
36+
37+
### Build backend support
38+
39+
Build the inference engine extension libraries you need.
40+
41+
- [ONNX Runtime](backends/onnxruntime.md)
42+
- [TensorRT](backends/tensorrt.md)
43+
- [ncnn](backends/ncnn.md)
44+
- [pplnn](backends/pplnn.md)
45+
- [OpenVINO](backends/openvino.md)
46+
- [TorchScript](backends/torchscript.md)
47+
48+
### Install mmdeploy
49+
50+
```bash
51+
cd ${MMDEPLOY_DIR} # To mmdeploy root directory
52+
pip install -e .
53+
```
54+
55+
**Note**
56+
57+
- Some dependencies are optional. Simply running `pip install -e .` will only install the minimum runtime requirements.
58+
To use optional dependencies, install them manually with `pip install -r requirements/optional.txt` or specify desired extras when calling `pip` (e.g. `pip install -e . [optional]`).
59+
Valid keys for the extras field are: `all`, `tests`, `build`, `optional`.
60+
61+
### Build SDK
62+
63+
Readers can skip this chapter if you are only interested in model converter.
64+
65+
#### Dependencies
66+
67+
Currently, SDK is tested on Linux x86-64, more platforms will be added in the future. The following packages are required to build MMDeploy SDK.
68+
69+
Each package's installation command is given based on Ubuntu 18.04.
70+
71+
- OpenCV 3+
72+
73+
```bash
74+
sudo apt-get install libopencv-dev
75+
```
76+
77+
- spdlog 0.16+
78+
79+
``` bash
80+
sudo apt-get install libspdlog-dev
81+
```
82+
83+
On Ubuntu 16.04, please use the following command
84+
```bash
85+
wget http://archive.ubuntu.com/ubuntu/pool/universe/s/spdlog/libspdlog-dev_0.16.3-1_amd64.deb
86+
sudo dpkg -i libspdlog-dev_0.16.3-1_amd64.deb
87+
```
88+
89+
You can also build spdlog from its source to enjoy its latest features. But be sure to add **`-fPIC`** compilation flags at first.
90+
91+
- pplcv
92+
93+
A high-performance image processing library of openPPL supporting x86 and cuda platforms.</br>
94+
It is **OPTIONAL** which only be needed if `cuda` platform is required.
95+
96+
Using v0.6.1, since latest updates have broughtup breaking changes
97+
```Bash
98+
wget https://github.com/openppl-public/ppl.cv/archive/refs/tags/v0.6.1.zip
99+
unzip v0.6.1.zip && mv ppl.cv-0.6.1 ppl.cv
100+
cd ppl.cv
101+
./build.sh cuda
102+
```
103+
104+
- backend engines
105+
106+
SDK uses the same backends as model converter does. Please follow [build backend](#build-backend-support) guide to install your interested backend.
107+
108+
#### Set Build Option
109+
110+
- Turn on SDK build switch
111+
112+
`-DMMDEPLOY_BUILD_SDK=ON`
113+
114+
115+
- Enabling Devices
116+
117+
By default, only CPU device is included in the target devices. You can enable device support for other devices by
118+
passing a semicolon separated list of device names to `MMDEPLOY_TARGET_DEVICES` variable, e.g. `-DMMDEPLOY_TARGET_DEVICES="cpu;cuda"`. </br>
119+
Currently, the following devices are supported.
120+
121+
| device | name | path setter |
122+
|--------|-------|-------------|
123+
| Host | cpu | N/A |
124+
| CUDA | cuda | CUDA_TOOLKIT_ROOT_DIR & pplcv_DIR |
125+
126+
If you have multiple CUDA versions installed on your system, you will need to pass `CUDA_TOOLKIT_ROOT_DIR` to cmake to specify the version. </br>
127+
Meanwhile, `pplcv_DIR` has to be provided in order to build image processing operators on cuda platform.
128+
129+
130+
- Enabling inference engines
131+
132+
**By default, no target inference engines are set**, since it's highly dependent on the use case.
133+
`MMDEPLOY_TARGET_BACKENDS` must be set to a semicolon separated list of inference engine names,
134+
e.g. `-DMMDEPLOY_TARGET_BACKENDS="trt;ort;pplnn;ncnn;openvino"`
135+
A path to the inference engine library is also needed. The following backends are currently supported
136+
137+
| library | name | path setter |
138+
|-------------|----------|-----------------|
139+
| PPL.nn | pplnn | pplnn_DIR |
140+
| ncnn | ncnn | ncnn_DIR |
141+
| ONNXRuntime | ort | ONNXRUNTIME_DIR |
142+
| TensorRT | trt | TENSORRT_DIR & CUDNN_DIR |
143+
| OpenVINO | openvino | InferenceEngine_DIR |
144+
145+
- Enabling codebase's postprocess components
146+
147+
`MMDEPLOY_CODEBASES` MUST be specified by a semicolon separated list of codebase names.
148+
The currently supported codebases are 'mmcls', 'mmdet', 'mmedit', 'mmseg', 'mmocr'.
149+
Instead of listing them one by one in `MMDEPLOY_CODEBASES`, user can also pass `all` to enable all of them, i.e.,
150+
`-DMMDEPLOY_CODEBASES=all`
151+
152+
153+
- Put it all together
154+
155+
The following is a recipe for building MMDeploy SDK with cpu device and ONNXRuntime support
156+
```Bash
157+
mkdir build && cd build
158+
cmake .. \
159+
-DMMDEPLOY_BUILD_SDK=ON \
160+
-DCMAKE_CXX_COMPILER=g++-7 \
161+
-DONNXRUNTIME_DIR=/path/to/onnxruntime \
162+
-DMMDEPLOY_TARGET_DEVICES=cpu \
163+
-DMMDEPLOY_TARGET_BACKENDS=ort \
164+
-DMMDEPLOY_CODEBASES=all
165+
cmake --build . -- -j$(nproc) && cmake --install .
166+
```
167+
168+
Here is another example to build MMDeploy SDK with cuda device and TensorRT backend
169+
170+
```Bash
171+
mkdir build && cd build
172+
cmake .. \
173+
-DMMDEPLOY_BUILD_SDK=ON \
174+
-DCMAKE_CXX_COMPILER=g++-7 \
175+
-Dpplcv_DIR=/path/to/ppl.cv/cuda-build/install/lib/cmake/ppl \
176+
-DTENSORRT_DIR=/path/to/tensorrt \
177+
-DCUDNN_DIR=/path/to/cudnn \
178+
-DMMDEPLOY_TARGET_DEVICES="cuda;cpu" \
179+
-DMMDEPLOY_TARGET_BACKENDS=trt \
180+
-DMMDEPLOY_CODEBASES=all
181+
cmake --build . -- -j$(nproc) && cmake --install .
182+
```

0 commit comments

Comments
 (0)