Fixed ncnn model conversion errors in Dockerfile（no module name 'ncnn. ncnn',）

[KerwinKai]

Motivation

When I use the cpu version of dockerfile to build a docker for ncnn model conversion, the command line prompts no module name 'ncnn. ncnn', and the model conversion is not successful, although the docker did not report an error during construction.

Modification

The ncnn build content in the cpu version dockerfile has been modified. For this modification, refer to @ yangc1991 and @ yangc1991 answers in Tencent/ncnn#2759

Use cases (Optional)

After the modification, I test some examples in mmclassification and mmdetection

mmclassfication（mobilenetv2）

Conversion command

python ./tools/deploy.py
configs/mmcls/classification_ncnn_static.py
$PATH_TO_MMCLS/configs/mobilenet_v2/mobilenet-v2_8xb32_in1k.py
$PATH_TO_MMCLS/checkpoints/mobilenet_v2_batch256_imagenet_20200708-3b2dc3af.pth
$PATH_TO_MMCLS/demo/orange224-224.jpg
--work-dir work_dir
--show
--device CPU

Output Log

root@8597e101b398:~/workspace/mmdeploy# python ./tools/deploy.py \

configs/mmcls/classification_ncnn_static.py \
$PATH_TO_MMCLS/configs/mobilenet_v2/mobilenet-v2_8xb32_in1k.py \
$PATH_TO_MMCLS/checkpoints/mobilenet_v2_batch256_imagenet_20200708-3b2dc3af.pth \
$PATH_TO_MMCLS/demo/orange224-224.jpg \
--work-dir work_dir \
--show \
--device cpu

2023-02-01 03:36:45,668 - mmdeploy - INFO - Start pipeline mmdeploy.apis.pytorch2onnx.torch2onnx in subprocess
load checkpoint from local path: /root/workspace/mmclassification/checkpoints/mobilenet_v2_batch256_imagenet_20200708-3b2dc3af.pth
/root/workspace/mmclassification/mmcls/apis/inference.py:45: UserWarning: Class names are not saved in the checkpoint's meta data, use imagenet by default.
warnings.warn('Class names are not saved in the checkpoint's '
2023-02-01 03:36:46,297 - mmdeploy - WARNING - DeprecationWarning: get_onnx_config will be deprecated in the future.
2023-02-01 03:36:46,297 - mmdeploy - INFO - Export PyTorch model to ONNX: work_dir/end2end.onnx.
2023-02-01 03:36:46,310 - mmdeploy - WARNING - Can not find torch._C._jit_pass_onnx_autograd_function_process, function rewrite will not be applied
/root/workspace/mmdeploy/mmdeploy/pytorch/functions/size.py:22: TracerWarning: Converting a tensor to a Python integer might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
ret = [int(r) for r in ret]
2023-02-01 03:36:47,789 - mmdeploy - INFO - Finish pipeline mmdeploy.apis.pytorch2onnx.torch2onnx
2023-02-01 03:36:47,975 - mmdeploy - INFO - Start pipeline mmdeploy.apis.utils.utils.to_backend in main process
onnx2ncnn finish
2023-02-01 03:36:48,031 - mmdeploy - INFO - Finish pipeline mmdeploy.apis.utils.utils.to_backend
2023-02-01 03:36:48,031 - mmdeploy - INFO - visualize ncnn model start.
2023-02-01 03:36:49,644 - mmdeploy - WARNING - render and display result skipped for headless device, exception no display name and no $DISPLAY environment variable
2023-02-01:03:36:49 - mmdeploy - WARNING - render and display result skipped for headless device, exception no display name and no $DISPLAY environment variable
2023-02-01 03:36:49,874 - mmdeploy - INFO - visualize ncnn model success.
2023-02-01 03:36:49,874 - mmdeploy - INFO - visualize pytorch model start.
load checkpoint from local path: /root/workspace/mmclassification/checkpoints/mobilenet_v2_batch256_imagenet_20200708-3b2dc3af.pth
/root/workspace/mmclassification/mmcls/apis/inference.py:45: UserWarning: Class names are not saved in the checkpoint's meta data, use imagenet by default.
warnings.warn('Class names are not saved in the checkpoint's '
/root/workspace/mmdeploy/mmdeploy/apis/visualize.py:82: DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead
f'render and display result skipped for headless device, exception {e}' # noqa: E501
2023-02-01 03:36:51,531 - mmdeploy - WARNING - render and display result skipped for headless device, exception no display name and no $DISPLAY environment variable
2023-02-01:03:36:51 - mmdeploy - WARNING - render and display result skipped for headless device, exception no display name and no $DISPLAY environment variable
2023-02-01 03:36:51,765 - mmdeploy - INFO - visualize pytorch model success.
2023-02-01 03:36:51,766 - mmdeploy - INFO - All process success.
root@8597e101b398:/workspace/mmdeploy# ls
CITATION.cff CMakeLists.txt LICENSE MANIFEST.in README.md README_zh-CN.md build cmake configs csrc demo docker docs mmdeploy mmdeploy.egg-info requirements requirements.txt resources service setup.cfg setup.py tests third_party tools work_dir
root@8597e101b398:/workspace/mmdeploy# cd work_dir/
root@8597e101b398:~/workspace/mmdeploy/work_dir# ls
end2end.bin end2end.onnx end2end.param output_ncnn.jpg output_pytorch.jpg

mmdetection（yolov3）

Conversion command

python ./tools/deploy.py
configs/mmdet/detection/single-stage_ncnn_dynamic.py
$PATH_TO_MMDET/configs/yolo/yolov3_d53_320_273e_coco.py
$PATH_TO_MMDET/checkpoints/yolov3_d53_320_273e_coco-421362b6.pth
$PATH_TO_MMDET/demo/demo.jpg
--work-dir work_dir
--show
--device cpu

Output Log

root@3a09afcb1563:~/workspace/mmdeploy# python ./tools/deploy.py \

configs/mmdet/detection/single-stage_ncnn_dynamic.py \
$PATH_TO_MMDET/configs/yolo/yolov3_d53_320_273e_coco.py \
$PATH_TO_MMDET/checkpoints/yolov3_d53_320_273e_coco-421362b6.pth \
$PATH_TO_MMDET/demo/demo.jpg \
--work-dir work_dir \
--show \
--device cpu

2023-02-01 06:44:03,126 - mmdeploy - INFO - Start pipeline mmdeploy.apis.pytorch2onnx.torch2onnx in subprocess
load checkpoint from local path: /root/workspace/mmdetection/checkpoints/yolov3_d53_320_273e_coco-421362b6.pth
/root/workspace/mmdetection/mmdet/datasets/utils.py:70: UserWarning: "ImageToTensor" pipeline is replaced by "DefaultFormatBundle" for batch inference. It is recommended to manually replace it in the test data pipeline in your config file.
'data pipeline in your config file.', UserWarning)
2023-02-01 06:44:04,528 - mmdeploy - WARNING - DeprecationWarning: get_onnx_config will be deprecated in the future.
2023-02-01 06:44:04,528 - mmdeploy - INFO - Export PyTorch model to ONNX: work_dir/end2end.onnx.
2023-02-01 06:44:04,546 - mmdeploy - WARNING - Can not find torch._C._jit_pass_onnx_autograd_function_process, function rewrite will not be applied
/opt/conda/lib/python3.7/site-packages/torch/onnx/utils.py:1294: UserWarning: Provided key dets for dynamic axes is not a valid input/output name
warnings.warn("Provided key {} for dynamic axes is not a valid input/output name".format(key))
/opt/conda/lib/python3.7/site-packages/torch/onnx/utils.py:1294: UserWarning: Provided key labels for dynamic axes is not a valid input/output name
warnings.warn("Provided key {} for dynamic axes is not a valid input/output name".format(key))
/root/workspace/mmdeploy/mmdeploy/pytorch/functions/getattribute.py:18: TracerWarning: Converting a tensor to a Python integer might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
ret = torch.Size([int(s) for s in ret])
/root/workspace/mmdetection/mmdet/models/dense_heads/yolo_head.py:127: UserWarning: DeprecationWarning: anchor_generator is deprecated, please use "prior_generator" instead
warnings.warn('DeprecationWarning: anchor_generator is deprecated, '
WARNING: The shape inference of mmdeploy::Yolov3DetectionOutput type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::Yolov3DetectionOutput type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::Yolov3DetectionOutput type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
2023-02-01 06:44:07,711 - mmdeploy - INFO - Finish pipeline mmdeploy.apis.pytorch2onnx.torch2onnx
2023-02-01 06:44:07,933 - mmdeploy - INFO - Start pipeline mmdeploy.apis.utils.utils.to_backend in main process
onnx2ncnn finish
2023-02-01 06:44:08,195 - mmdeploy - INFO - Finish pipeline mmdeploy.apis.utils.utils.to_backend
2023-02-01 06:44:08,195 - mmdeploy - INFO - visualize ncnn model start.
/root/workspace/mmdetection/mmdet/datasets/utils.py:70: UserWarning: "ImageToTensor" pipeline is replaced by "DefaultFormatBundle" for batch inference. It is recommended to manually replace it in the test data pipeline in your config file.
'data pipeline in your config file.', UserWarning)
2023-02-01 06:44:10,555 - mmdeploy - WARNING - render and display result skipped for headless device, exception no display name and no $DISPLAY environment variable
2023-02-01:06:44:10 - mmdeploy - WARNING - render and display result skipped for headless device, exception no display name and no $DISPLAY environment variable
2023-02-01 06:44:10,870 - mmdeploy - INFO - visualize ncnn model success.
2023-02-01 06:44:10,870 - mmdeploy - INFO - visualize pytorch model start.
load checkpoint from local path: /root/workspace/mmdetection/checkpoints/yolov3_d53_320_273e_coco-421362b6.pth
/root/workspace/mmdetection/mmdet/datasets/utils.py:70: UserWarning: "ImageToTensor" pipeline is replaced by "DefaultFormatBundle" for batch inference. It is recommended to manually replace it in the test data pipeline in your config file.
'data pipeline in your config file.', UserWarning)
2023-02-01 06:44:13,407 - mmdeploy - WARNING - render and display result skipped for headless device, exception no display name and no $DISPLAY environment variable
2023-02-01:06:44:13 - mmdeploy - WARNING - render and display result skipped for headless device, exception no display name and no $DISPLAY environment variable
2023-02-01 06:44:13,689 - mmdeploy - INFO - visualize pytorch model success.
2023-02-01 06:44:13,689 - mmdeploy - INFO - All process success.

Generate files in work_dir

• work_dir
  -- end2end.bin
  -- end2end.onnx
  -- end2end.param
  -- output_ncnn.jpg
  -- output_pytorch.jpg
  The variation of the confidence level of identification under documents output_ncnn.jpg and output_pytorch.jpg shall not exceed 0.01

[CLAassistant]

All committers have signed the CLA.

[tpoisonooo]

please keep make -j $(nproc) .

[KerwinKai]

please keep make -j $(nproc) .

oki

[hanrui1sensetime]

LGTM.

[KerwinKai]

please keep make -j $(nproc) .

Hi, I have change 'make -j8' to 'make -j $(nproc)', and delete it, sorry for misunderstanding before. Besides, I did the above test again with the latest dockerfile, and can still perform model transformation and get the same reasoning results.

[KerwinKai]

Supplement the log information of the error report prompt mentioned in the title
root@ebce389d36ef:~/workspace/mmdeploy# python ./tools/deploy.py \

configs/mmcls/classification_ncnn_static.py \
$PATH_TO_MMCLS/configs/mobilenet_v2/mobilenet-v2_8xb32_in1k.py \
$PATH_TO_MMCLS/checkpoints/mobilenet_v2_batch256_imagenet_20200708-3b2dc3af.pth \
$PATH_TO_MMCLS/demo/orange224-224.jpg \
--work-dir work_dir \
--show \
--device cpu

2023-02-02 02:38:44,587 - mmdeploy - INFO - Start pipeline mmdeploy.apis.pytorch2onnx.torch2onnx in subprocess
load checkpoint from local path: /root/workspace/mmclassification/checkpoints/mobilenet_v2_batch256_imagenet_20200708-3b2dc3af.pth
/root/workspace/mmclassification/mmcls/apis/inference.py:45: UserWarning: Class names are not saved in the checkpoint's meta data, use imagenet by default.
warnings.warn('Class names are not saved in the checkpoint's '
2023-02-02 02:38:45,451 - mmdeploy - WARNING - DeprecationWarning: get_onnx_config will be deprecated in the future.
2023-02-02 02:38:45,451 - mmdeploy - INFO - Export PyTorch model to ONNX: work_dir/end2end.onnx.
2023-02-02 02:38:45,462 - mmdeploy - WARNING - Can not find torch._C._jit_pass_onnx_autograd_function_process, function rewrite will not be applied
/root/workspace/mmdeploy/mmdeploy/pytorch/functions/size.py:22: TracerWarning: Converting a tensor to a Python integer might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
ret = [int(r) for r in ret]
2023-02-02 02:38:46,996 - mmdeploy - INFO - Finish pipeline mmdeploy.apis.pytorch2onnx.torch2onnx
2023-02-02 02:38:47,191 - mmdeploy - INFO - Start pipeline mmdeploy.apis.utils.utils.to_backend in main process
onnx2ncnn finish
2023-02-02 02:38:47,228 - mmdeploy - INFO - Finish pipeline mmdeploy.apis.utils.utils.to_backend
2023-02-02 02:38:47,229 - mmdeploy - INFO - visualize ncnn model start.
2023-02-02:02:38:48 - root - ERROR - No module named 'ncnn.ncnn'
Traceback (most recent call last):
File "/root/workspace/mmdeploy/mmdeploy/utils/utils.py", line 41, in target_wrapper
result = target(*args, **kwargs)
File "/root/workspace/mmdeploy/mmdeploy/apis/visualize.py", line 67, in visualize_model
model = task_processor.init_backend_model(model, **kwargs)
File "/root/workspace/mmdeploy/mmdeploy/codebase/mmcls/deploy/classification.py", line 83, in init_backend_model
**kwargs)
File "/root/workspace/mmdeploy/mmdeploy/codebase/mmcls/deploy/classification_model.py", line 219, in build_classification_model
**kwargs)
File "/opt/conda/lib/python3.7/site-packages/mmcv/utils/registry.py", line 237, in build
return self.build_func(*args, **kwargs, registry=self)
File "/root/workspace/mmdeploy/mmdeploy/codebase/mmcls/deploy/classification_model.py", line 16, in __build_backend_model
return registry.module_dict[cls_name](*args, **kwargs)
File "/root/workspace/mmdeploy/mmdeploy/codebase/mmcls/deploy/classification_model.py", line 53, in init
**kwargs)
File "/root/workspace/mmdeploy/mmdeploy/codebase/mmcls/deploy/classification_model.py", line 65, in _init_wrapper
**kwargs)
File "/root/workspace/mmdeploy/mmdeploy/codebase/base/backend_model.py", line 64, in _build_wrapper
output_names, deploy_cfg, **kwargs)
File "/root/workspace/mmdeploy/mmdeploy/backend/ncnn/backend_manager.py", line 34, in build_wrapper
from .wrapper import NCNNWrapper
File "/root/workspace/mmdeploy/mmdeploy/backend/ncnn/wrapper.py", line 5, in
import ncnn
File "/root/workspace/ncnn/python/ncnn/init.py", line 15, in
from .ncnn import *
ModuleNotFoundError: No module named 'ncnn.ncnn'
2023-02-02 02:38:49,078 - mmdeploy - ERROR - visualize ncnn model failed.

[tpoisonooo]

LGTM.