Model Convertion Error #assertion/root/workspace/mmdeploy/csrc/backend_ops/tensorrt/batched_nms/trt_batched_nms.cpp,98

[TheSeriousProgrammer]

Checklist

1. I have searched related issues but cannot get the expected help.
2. I have read the FAQ documentation but cannot get the expected help.
3. The bug has not been fixed in the latest version.

Describe the bug

I tried to use a convert a mmdetection yolov3 mobilenet model trained with a wildfire dataset and getting teh following error

#assertion/root/workspace/mmdeploy/csrc/backend_ops/tensorrt/batched_nms/trt_batched_nms.cpp,98
2022-02-07 06:19:20,556 - mmdeploy - ERROR - visualize tensorrt model failed.

But it gave out end2end.engine and end2end.onnx

Reproduction

1. What command or script did you run?

 python3 ~/workspace/mmdeploy/tools/deploy.py ~/workspace/mmdeploy/configs/mmdet/detection/detection_tensorrt_dynamic-320x320-1344x1344.py artifacts_2/yolov3_config.py artifacts_2/latest.pth artifacts_2/test.jpg --work-dir out --device cuda:0

2. Did you make any modifications on the code or config? Did you understand what you have modified?

Environment

Used docker/GPU with the following changes

### build sdk
RUN wget https://github.com/openppl-public/ppl.cv/archive/refs/tags/v0.6.1.zip && unzip v0.6.1.zip
RUN mv ppl.cv-0.6.1 ppl.cv && \
    cd ppl.cv &&\
    ./build.sh cuda

#RUN git clone https://github.com/openppl-public/ppl.cv.git &&\
#    cd ppl.cv &&\
#    ./build.sh cuda
RUN pip3 install mmdet

Error traceback

If applicable, paste the error trackback here.

2022-02-07 06:13:43,927 - mmdeploy - INFO - torch2onnx start.
load checkpoint from local path: artifacts_2/latest.pth
/root/workspace/mmdeploy/mmdeploy/core/optimizers/function_marker.py:158: TracerWarning: Converting a tensor to a Python integer might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  ys_shape = tuple(int(s) for s in ys.shape)
/opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:3451: UserWarning: Default upsampling behavior when mode=bilinear is changed to align_corners=False since 0.4.0. Please specify align_corners=True if the old behavior is desired. See the documentation of nn.Upsample for details.
  warnings.warn(
/opt/conda/lib/python3.8/site-packages/mmdet/models/dense_heads/yolo_head.py:126: UserWarning: DeprecationWarning: `anchor_generator` is deprecated, please use "prior_generator" instead
  warnings.warn('DeprecationWarning: `anchor_generator` is deprecated, '
/opt/conda/lib/python3.8/site-packages/mmdet/core/anchor/anchor_generator.py:333: UserWarning: ``grid_anchors`` would be deprecated soon. Please use ``grid_priors`` 
  warnings.warn('``grid_anchors`` would be deprecated soon. '
/opt/conda/lib/python3.8/site-packages/mmdet/core/anchor/anchor_generator.py:369: UserWarning: ``single_level_grid_anchors`` would be deprecated soon. Please use ``single_level_grid_priors`` 
  warnings.warn(
/opt/conda/lib/python3.8/site-packages/mmdet/core/bbox/coder/yolo_bbox_coder.py:73: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  assert pred_bboxes.size(-1) == bboxes.size(-1) == 4
/root/workspace/mmdeploy/mmdeploy/pytorch/functions/topk.py:54: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  if k > size:
/root/workspace/mmdeploy/mmdeploy/codebase/mmdet/core/post_processing/bbox_nms.py:260: TracerWarning: Converting a tensor to a Python integer might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  dets, labels = TRTBatchedNMSop.apply(boxes, scores, int(scores.shape[-1]),
/root/workspace/mmdeploy/mmdeploy/mmcv/ops/nms.py:177: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  out_boxes = min(num_boxes, after_topk)
2022-02-07 06:14:19,567 - mmdeploy - INFO - torch2onnx success.
[2022-02-07 06:14:19.891] [mmdeploy] [info] [model.cpp:97] Register 'DirectoryModel'
2022-02-07 06:14:19,921 - mmdeploy - INFO - onnx2tensorrt of out/end2end.onnx start.
[2022-02-07 06:14:22.631] [mmdeploy] [info] [model.cpp:97] Register 'DirectoryModel'
2022-02-07 06:14:22,685 - mmdeploy - INFO - Successfully loaded tensorrt plugins from /root/workspace/mmdeploy/build/lib/libmmdeploy_tensorrt_ops.so
[TensorRT] WARNING: /workspace/TensorRT/t/oss-cicd/oss/parsers/onnx/onnx2trt_utils.cpp:227: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[TensorRT] WARNING: /workspace/TensorRT/t/oss-cicd/oss/parsers/onnx/builtin_op_importers.cpp:3125: TensorRT currently uses half_pixel calculation for the pytorch_half_pixel transformation mode. These are equivalent except for interpolations down to 1D.
[TensorRT] WARNING: /workspace/TensorRT/t/oss-cicd/oss/parsers/onnx/builtin_op_importers.cpp:3125: TensorRT currently uses half_pixel calculation for the pytorch_half_pixel transformation mode. These are equivalent except for interpolations down to 1D.
[TensorRT] WARNING: /workspace/TensorRT/t/oss-cicd/oss/parsers/onnx/onnx2trt_utils.cpp:255: One or more weights outside the range of INT32 was clamped
[TensorRT] WARNING: /workspace/TensorRT/t/oss-cicd/oss/parsers/onnx/onnx2trt_utils.cpp:255: One or more weights outside the range of INT32 was clamped
[TensorRT] WARNING: /workspace/TensorRT/t/oss-cicd/oss/parsers/onnx/onnx2trt_utils.cpp:255: One or more weights outside the range of INT32 was clamped
[TensorRT] WARNING: /workspace/TensorRT/t/oss-cicd/oss/parsers/onnx/onnx2trt_utils.cpp:255: One or more weights outside the range of INT32 was clamped
[TensorRT] WARNING: /workspace/TensorRT/t/oss-cicd/oss/parsers/onnx/onnx2trt_utils.cpp:255: One or more weights outside the range of INT32 was clamped
[TensorRT] WARNING: /workspace/TensorRT/t/oss-cicd/oss/parsers/onnx/onnx2trt_utils.cpp:255: One or more weights outside the range of INT32 was clamped
[TensorRT] WARNING: /workspace/TensorRT/t/oss-cicd/oss/parsers/onnx/onnx2trt_utils.cpp:255: One or more weights outside the range of INT32 was clamped
[TensorRT] WARNING: /workspace/TensorRT/t/oss-cicd/oss/parsers/onnx/onnx2trt_utils.cpp:255: One or more weights outside the range of INT32 was clamped
[TensorRT] WARNING: /workspace/TensorRT/t/oss-cicd/oss/parsers/onnx/onnx2trt_utils.cpp:255: One or more weights outside the range of INT32 was clamped
[TensorRT] WARNING: /workspace/TensorRT/t/oss-cicd/oss/parsers/onnx/onnx2trt_utils.cpp:255: One or more weights outside the range of INT32 was clamped
[TensorRT] WARNING: /workspace/TensorRT/t/oss-cicd/oss/parsers/onnx/onnx2trt_utils.cpp:255: One or more weights outside the range of INT32 was clamped
[TensorRT] WARNING: /workspace/TensorRT/t/oss-cicd/oss/parsers/onnx/onnx2trt_utils.cpp:255: One or more weights outside the range of INT32 was clamped
[TensorRT] WARNING: /workspace/TensorRT/t/oss-cicd/oss/parsers/onnx/onnx2trt_utils.cpp:255: One or more weights outside the range of INT32 was clamped
[TensorRT] WARNING: /workspace/TensorRT/t/oss-cicd/oss/parsers/onnx/onnx2trt_utils.cpp:255: One or more weights outside the range of INT32 was clamped
[TensorRT] WARNING: /workspace/TensorRT/t/oss-cicd/oss/parsers/onnx/onnx2trt_utils.cpp:255: One or more weights outside the range of INT32 was clamped
[TensorRT] INFO: No importer registered for op: TRTBatchedNMS. Attempting to import as plugin.
[TensorRT] INFO: Searching for plugin: TRTBatchedNMS, plugin_version: 1, plugin_namespace: 
[TensorRT] INFO: Successfully created plugin: TRTBatchedNMS
[TensorRT] INFO: Some tactics do not have sufficient workspace memory to run. Increasing workspace size may increase performance, please check verbose output.
[TensorRT] INFO: Detected 1 inputs and 2 output network tensors.
2022-02-07 06:18:59,744 - mmdeploy - INFO - onnx2tensorrt of out/end2end.onnx success.
2022-02-07 06:18:59,745 - mmdeploy - INFO - visualize tensorrt model start.
[2022-02-07 06:19:15.386] [mmdeploy] [info] [model.cpp:97] Register 'DirectoryModel'
2022-02-07 06:19:15,475 - mmdeploy - INFO - Successfully loaded tensorrt plugins from /root/workspace/mmdeploy/build/lib/libmmdeploy_tensorrt_ops.so
2022-02-07 06:19:15,475 - mmdeploy - INFO - Successfully loaded tensorrt plugins from /root/workspace/mmdeploy/build/lib/libmmdeploy_tensorrt_ops.so
#assertion/root/workspace/mmdeploy/csrc/backend_ops/tensorrt/batched_nms/trt_batched_nms.cpp,98
2022-02-07 06:19:20,556 - mmdeploy - ERROR - visualize tensorrt model failed.

Bug fix

I dont know the reason of the bug , but I saw that end2end.engine and end2end.onnx was created successfully, may I know the reason

[TheSeriousProgrammer]

FIles to test with https://drive.google.com/file/d/19U1g2w67b5bMpHjXPy5jiNrt-bVcBmET/view?usp=sharing

[grimoire]

Hi, Could you please recheck the link to the files? I do not have download permission.

[TheSeriousProgrammer]

@grimoire could be an ip block , the link works fine for me in incognito too, can you try this link https://github.com/chidha-edgeneural/artifacts

[grimoire]

Hi, I can convert the model with this config : "${MMDEPLOY_PATH}/configs/mmdet/detection/detection_tensorrt_dynamic-160x160-608x608.py"

Note that the preprocess of yolov3 is slightly different from other models in mmdetection. Please use the config above and try again.

[TheSeriousProgrammer]

I tried building again with the mentioned changes

Command which I ran :

python3 ~/workspace/mmdeploy/tools/deploy.py ~/workspace/mmdeploy/configs/mmdet/detection/detection_tensorrt_dynamic-160x160-608x608.py model_configs/yolov3_config.py artifacts_2/latest.pth artifacts_2/test.jpg --work-dir out --device cuda:0 --show

Output:

2022-02-08 07:10:15,513 - mmdeploy - INFO - torch2onnx start.
2022-02-08:07:10:28,matplotlib.font_manager INFO     [font_manager.py:1443] generated new fontManager
load checkpoint from local path: artifacts_2/latest.pth
The model and loaded state dict do not match exactly

size mismatch for bbox_head.convs_pred.0.weight: copying a param with shape torch.Size([18, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([255, 96, 1, 1]).
size mismatch for bbox_head.convs_pred.0.bias: copying a param with shape torch.Size([18]) from checkpoint, the shape in current model is torch.Size([255]).
size mismatch for bbox_head.convs_pred.1.weight: copying a param with shape torch.Size([18, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([255, 96, 1, 1]).
size mismatch for bbox_head.convs_pred.1.bias: copying a param with shape torch.Size([18]) from checkpoint, the shape in current model is torch.Size([255]).
size mismatch for bbox_head.convs_pred.2.weight: copying a param with shape torch.Size([18, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([255, 96, 1, 1]).
size mismatch for bbox_head.convs_pred.2.bias: copying a param with shape torch.Size([18]) from checkpoint, the shape in current model is torch.Size([255]).
/root/workspace/mmdeploy/mmdeploy/core/optimizers/function_marker.py:158: TracerWarning: Converting a tensor to a Python integer might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  ys_shape = tuple(int(s) for s in ys.shape)
/opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:3451: UserWarning: Default upsampling behavior when mode=bilinear is changed to align_corners=False since 0.4.0. Please specify align_corners=True if the old behavior is desired. See the documentation of nn.Upsample for details.
  warnings.warn(
/opt/conda/lib/python3.8/site-packages/mmdet/models/dense_heads/yolo_head.py:126: UserWarning: DeprecationWarning: `anchor_generator` is deprecated, please use "prior_generator" instead
  warnings.warn('DeprecationWarning: `anchor_generator` is deprecated, '
/opt/conda/lib/python3.8/site-packages/mmdet/core/anchor/anchor_generator.py:333: UserWarning: ``grid_anchors`` would be deprecated soon. Please use ``grid_priors`` 
  warnings.warn('``grid_anchors`` would be deprecated soon. '
/opt/conda/lib/python3.8/site-packages/mmdet/core/anchor/anchor_generator.py:369: UserWarning: ``single_level_grid_anchors`` would be deprecated soon. Please use ``single_level_grid_priors`` 
  warnings.warn(
/opt/conda/lib/python3.8/site-packages/mmdet/core/bbox/coder/yolo_bbox_coder.py:73: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  assert pred_bboxes.size(-1) == bboxes.size(-1) == 4
/root/workspace/mmdeploy/mmdeploy/pytorch/functions/topk.py:54: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  if k > size:
/root/workspace/mmdeploy/mmdeploy/codebase/mmdet/core/post_processing/bbox_nms.py:260: TracerWarning: Converting a tensor to a Python integer might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  dets, labels = TRTBatchedNMSop.apply(boxes, scores, int(scores.shape[-1]),
/root/workspace/mmdeploy/mmdeploy/mmcv/ops/nms.py:177: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  out_boxes = min(num_boxes, after_topk)
2022-02-08 07:10:50,164 - mmdeploy - INFO - torch2onnx success.
[2022-02-08 07:10:51.312] [mmdeploy] [info] [model.cpp:97] Register 'DirectoryModel'
2022-02-08 07:10:51,461 - mmdeploy - INFO - onnx2tensorrt of out/end2end.onnx start.
[2022-02-08 07:10:54.631] [mmdeploy] [info] [model.cpp:97] Register 'DirectoryModel'
2022-02-08 07:10:54,673 - mmdeploy - INFO - Successfully loaded tensorrt plugins from /root/workspace/mmdeploy/build/lib/libmmdeploy_tensorrt_ops.so
[TensorRT] WARNING: /workspace/TensorRT/t/oss-cicd/oss/parsers/onnx/onnx2trt_utils.cpp:227: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[TensorRT] WARNING: /workspace/TensorRT/t/oss-cicd/oss/parsers/onnx/builtin_op_importers.cpp:3125: TensorRT currently uses half_pixel calculation for the pytorch_half_pixel transformation mode. These are equivalent except for interpolations down to 1D.
[TensorRT] WARNING: /workspace/TensorRT/t/oss-cicd/oss/parsers/onnx/builtin_op_importers.cpp:3125: TensorRT currently uses half_pixel calculation for the pytorch_half_pixel transformation mode. These are equivalent except for interpolations down to 1D.
[TensorRT] WARNING: /workspace/TensorRT/t/oss-cicd/oss/parsers/onnx/onnx2trt_utils.cpp:255: One or more weights outside the range of INT32 was clamped
[TensorRT] WARNING: /workspace/TensorRT/t/oss-cicd/oss/parsers/onnx/onnx2trt_utils.cpp:255: One or more weights outside the range of INT32 was clamped
[TensorRT] WARNING: /workspace/TensorRT/t/oss-cicd/oss/parsers/onnx/onnx2trt_utils.cpp:255: One or more weights outside the range of INT32 was clamped
[TensorRT] WARNING: /workspace/TensorRT/t/oss-cicd/oss/parsers/onnx/onnx2trt_utils.cpp:255: One or more weights outside the range of INT32 was clamped
[TensorRT] WARNING: /workspace/TensorRT/t/oss-cicd/oss/parsers/onnx/onnx2trt_utils.cpp:255: One or more weights outside the range of INT32 was clamped
[TensorRT] WARNING: /workspace/TensorRT/t/oss-cicd/oss/parsers/onnx/onnx2trt_utils.cpp:255: One or more weights outside the range of INT32 was clamped
[TensorRT] WARNING: /workspace/TensorRT/t/oss-cicd/oss/parsers/onnx/onnx2trt_utils.cpp:255: One or more weights outside the range of INT32 was clamped
[TensorRT] WARNING: /workspace/TensorRT/t/oss-cicd/oss/parsers/onnx/onnx2trt_utils.cpp:255: One or more weights outside the range of INT32 was clamped
[TensorRT] WARNING: /workspace/TensorRT/t/oss-cicd/oss/parsers/onnx/onnx2trt_utils.cpp:255: One or more weights outside the range of INT32 was clamped
[TensorRT] WARNING: /workspace/TensorRT/t/oss-cicd/oss/parsers/onnx/onnx2trt_utils.cpp:255: One or more weights outside the range of INT32 was clamped
[TensorRT] WARNING: /workspace/TensorRT/t/oss-cicd/oss/parsers/onnx/onnx2trt_utils.cpp:255: One or more weights outside the range of INT32 was clamped
[TensorRT] WARNING: /workspace/TensorRT/t/oss-cicd/oss/parsers/onnx/onnx2trt_utils.cpp:255: One or more weights outside the range of INT32 was clamped
[TensorRT] WARNING: /workspace/TensorRT/t/oss-cicd/oss/parsers/onnx/onnx2trt_utils.cpp:255: One or more weights outside the range of INT32 was clamped
[TensorRT] WARNING: /workspace/TensorRT/t/oss-cicd/oss/parsers/onnx/onnx2trt_utils.cpp:255: One or more weights outside the range of INT32 was clamped
[TensorRT] WARNING: /workspace/TensorRT/t/oss-cicd/oss/parsers/onnx/onnx2trt_utils.cpp:255: One or more weights outside the range of INT32 was clamped
[TensorRT] INFO: No importer registered for op: TRTBatchedNMS. Attempting to import as plugin.
[TensorRT] INFO: Searching for plugin: TRTBatchedNMS, plugin_version: 1, plugin_namespace: 
[TensorRT] INFO: Successfully created plugin: TRTBatchedNMS
[TensorRT] INFO: Some tactics do not have sufficient workspace memory to run. Increasing workspace size may increase performance, please check verbose output.
[TensorRT] INFO: Detected 1 inputs and 2 output network tensors.
2022-02-08 07:14:48,863 - mmdeploy - INFO - onnx2tensorrt of out/end2end.onnx success.
2022-02-08 07:14:48,863 - mmdeploy - INFO - visualize tensorrt model start.
[2022-02-08 07:15:04.413] [mmdeploy] [info] [model.cpp:97] Register 'DirectoryModel'
2022-02-08 07:15:04,487 - mmdeploy - INFO - Successfully loaded tensorrt plugins from /root/workspace/mmdeploy/build/lib/libmmdeploy_tensorrt_ops.so
2022-02-08 07:15:04,487 - mmdeploy - INFO - Successfully loaded tensorrt plugins from /root/workspace/mmdeploy/build/lib/libmmdeploy_tensorrt_ops.so
#assertion/root/workspace/mmdeploy/csrc/backend_ops/tensorrt/batched_nms/trt_batched_nms.cpp,98
2022-02-08 07:15:09,334 - mmdeploy - ERROR - visualize tensorrt model failed.

I think its because of my docker's having differenvt versions of github repositories than yours, can you build a docker from scratch and try

This is ,my docker build script slightly modified from docker/CUDA to use pplcv 0.6.1

FROM nvcr.io/nvidia/tensorrt:21.04-py3

ARG CUDA=10.2
ARG PYTHON_VERSION=3.8
ARG TORCH_VERSION=1.8.0
ARG TORCHVISION_VERSION=0.9.0
ARG ONNXRUNTIME_VERSION=1.8.1
ARG MMCV_VERSION=1.4.0
ARG CMAKE_VERSION=3.20.0
ENV FORCE_CUDA="1"
ENV nproc=5

ENV DEBIAN_FRONTEND=noninteractive

### update apt and install libs
RUN apt-get update &&\
    apt-get install -y vim libsm6 libxext6 libxrender-dev libgl1-mesa-glx git wget libssl-dev libopencv-dev libspdlog-dev --no-install-recommends &&\
    rm -rf /var/lib/apt/lists/*

RUN curl -fsSL -v -o ~/miniconda.sh -O  https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh  && \
    chmod +x ~/miniconda.sh && \
    ~/miniconda.sh -b -p /opt/conda && \
    rm ~/miniconda.sh && \
    /opt/conda/bin/conda install -y python=${PYTHON_VERSION} conda-build pyyaml numpy ipython cython typing typing_extensions mkl mkl-include ninja && \
    /opt/conda/bin/conda clean -ya

### pytorch
RUN /opt/conda/bin/conda install pytorch==${TORCH_VERSION} torchvision==${TORCHVISION_VERSION} cudatoolkit=${CUDA} -c pytorch
ENV PATH /opt/conda/bin:$PATH

### install mmcv-full
RUN /opt/conda/bin/pip install mmcv-full==${MMCV_VERSION} -f https://download.openmmlab.com/mmcv/dist/cu${CUDA//./}/torch${TORCH_VERSION}/index.html

WORKDIR /root/workspace
### get onnxruntime
RUN wget https://github.com/microsoft/onnxruntime/releases/download/v${ONNXRUNTIME_VERSION}/onnxruntime-linux-x64-${ONNXRUNTIME_VERSION}.tgz \
    && tar -zxvf onnxruntime-linux-x64-${ONNXRUNTIME_VERSION}.tgz &&\
    pip install onnxruntime-gpu==${ONNXRUNTIME_VERSION}

### cp trt from pip to conda
RUN cp -r /usr/local/lib/python${PYTHON_VERSION}/dist-packages/tensorrt* /opt/conda/lib/python${PYTHON_VERSION}/site-packages/

### update cmake
RUN  wget https://github.com/Kitware/CMake/releases/download/v${CMAKE_VERSION}/cmake-${CMAKE_VERSION}.tar.gz &&\
    tar -zxvf cmake-${CMAKE_VERSION}.tar.gz &&\
    cd cmake-${CMAKE_VERSION} &&\
    ./bootstrap &&\
    make &&\
    make install

### install mmdeploy
ENV ONNXRUNTIME_DIR=/root/workspace/onnxruntime-linux-x64-${ONNXRUNTIME_VERSION}
ENV TENSORRT_DIR=/workspace/tensorrt
ARG VERSION
RUN git clone https://github.com/open-mmlab/mmdeploy &&\
    cd mmdeploy &&\
    if [ -z ${VERSION} ] ; then echo "No MMDeploy version passed in, building on master" ; else git checkout tags/v${VERSION} -b tag_v${VERSION} ; fi &&\
    git submodule update --init --recursive &&\
    rm -rf build &&\
    mkdir build &&\
    cd build &&\
    cmake -DMMDEPLOY_TARGET_BACKENDS=ort .. &&\
    make -j$(nproc) &&\
    cmake -DMMDEPLOY_TARGET_BACKENDS=trt .. &&\
    make -j$(nproc) &&\
    cd .. &&\
    pip install -e .

### build sdk
#https://github.com/openppl-public/ppl.cv/archive/refs/tags/v0.1.1.zip
RUN wget https://github.com/openppl-public/ppl.cv/archive/refs/tags/v0.6.1.zip && unzip v0.6.1.zip
RUN mv ppl.cv-0.6.1 ppl.cv && \
    cd ppl.cv &&\
    ./build.sh cuda

#RUN git clone https://github.com/openppl-public/ppl.cv.git &&\
#    cd ppl.cv &&\
#    ./build.sh cuda


RUN cd /root/workspace/mmdeploy &&\
    rm -rf build/CM* &&\
    mkdir -p build && cd build &&\
    cmake .. \
        -DMMDEPLOY_BUILD_SDK=ON \
        -DCMAKE_CXX_COMPILER=g++ \
        -Dpplcv_DIR=/root/workspace/ppl.cv/cuda-build/install/lib/cmake/ppl \
        -DTENSORRT_DIR=${TENSORRT_DIR} \
        -DMMDEPLOY_BUILD_SDK_PYTHON_API=ON \
        -DMMDEPLOY_TARGET_DEVICES="cuda;cpu" \
        -DMMDEPLOY_TARGET_BACKENDS="trt" \
        -DMMDEPLOY_CODEBASES=all &&\
    cmake --build . -- -j$(nproc) && cmake --install . &&\
    cd install/example  && mkdir -p build && cd build &&\
    cmake -DMMDeploy_DIR=/root/workspace/mmdeploy/build/install/lib/cmake/MMDeploy .. &&\
    cmake --build . && export SPDLOG_LEVEL=warn &&\
    if [ -z ${VERSION} ] ; then echo "Built MMDeploy master for GPU devices successfully!" ; else echo "Built MMDeploy version v${VERSION} for GPU devices successfully!" ; fi

RUN pip3 install mmdet

ENV LD_LIBRARY_PATH=${PWD}:${LD_LIBRARY_PATH}
ENV LD_LIBRARY_PATH="/root/workspace/mmdeploy/build/lib:${LD_LIBRARY_PATH}"

[TheSeriousProgrammer]

Would this have something to do with my GPU?
NVIDIA Corporation GM108M [GeForce 940MX] / NVIDIA GeForce 940MX/PCIe/SSE2

$ nvidia-smi
Tue Feb  8 18:07:35 2022       
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 495.29.05    Driver Version: 495.29.05    CUDA Version: 11.5     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  NVIDIA GeForce ...  On   | 00000000:01:00.0 Off |                  N/A |
| N/A   47C    P0    N/A /  N/A |    477MiB /  4046MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
                                                                               
+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|    0   N/A  N/A      2181      G   /usr/lib/xorg/Xorg                 37MiB |
|    0   N/A  N/A      4088      G   /usr/lib/xorg/Xorg                198MiB |
|    0   N/A  N/A      4267      G   /usr/bin/gnome-shell               86MiB |
|    0   N/A  N/A     25401      G   /usr/lib/firefox/firefox          140MiB |
|    0   N/A  N/A     32102      G   /usr/lib/firefox/firefox            0MiB |
|    0   N/A  N/A   1086494      G   /usr/bin/nvidia-settings            0MiB |
+-----------------------------------------------------------------------------+

[grimoire]

I have tested with the dockerfile you provided on my device(1660ti). The model can be converted and visualized.

The compute capability is 5.0 for 940mx, which is listed on support matrix. So theoretically it is ok to deploy your model with TensorRT on that device. But I am not sure if the custom ops (The one in the error log) can be worked on your device.

If you have experience with Cuda, you can try to debug in source code. And I will try to find a device with cc 5.0.

[TheSeriousProgrammer]

@grimoire Thanks for the follow up, I usuall build docker images with cuda requirements on this laptop(940mx cuda), and then deploy to EC2, will build this image wit an ec2 instance, perform conversion there and let you know

[TheSeriousProgrammer]

I have checked it, it ran without an issue when I ran the conversion in an EC2 instance with GPU. Looks like my laptop gpu didnt support a tensorrt operation, thanks for helping me in figuring it out. A small headsup about the min GPU requirements requried to run the same would be helpful!

[Richard-mei]

@TheSeriousProgrammer @grimoire Hello, I have the same problem on Jetson TX2, is there any solution?

[TheSeriousProgrammer]

It might help if you could share the logs

[Richard-mei]

@TheSeriousProgrammer thank you for your reply.

JetsonTX2

NVIDIA Jetson TX2
L4T 32.6.1 [ JetPack 4.6 ]
Ubuntu 18.04.6 LTS
Kernel Version: 4.9.253-tegra
CUDA 10.2.300
CUDA Architecture: 6.2
OpenCV version: NOT_INSTALLED
OpenCV Cuda: NO
CUDNN: 8.2.1.32
TensorRT: 8.0.1.6
Vision Works: 1.6.0.501
VPI: ii libnvvpi1 1.1.15 arm64 NVIDIA Vision Programming Interface library
Vulcan: 1.2.70

command:

python3 ./tools/deploy.py configs/mmdet/detection/detection_tensorrt_dynamic-160x160-608x608.py ../MMDetection/configs/yolo/yolov3_d53_mstrain-608_273e_coco.py ../MMDetection/work_dir/yolov3_d53_mstrain-608_273e_coco_20210518_115020-a2c3acb8.pth ../MMDetection/demo/demo.jpg --work-dir test_work/test_yolo/ --show --device cuda:0

logs:

2022-03-03 15:01:40,525 - mmdeploy - INFO - torch2onnx start.
load checkpoint from local path: ../MMDetection/work_dir/yolov3_d53_mstrain-608_273e_coco_20210518_115020-a2c3acb8.pth
/home/richard/Projects/MMDetection/mmdet/datasets/utils.py:69: UserWarning: "ImageToTensor" pipeline is replaced by "DefaultFormatBundle" for batch inference. It is recommended to manually replace it in the test data pipeline in your config file.
'data pipeline in your config file.', UserWarning)
/home/richard/Projects/mmdeploy/mmdeploy/core/optimizers/function_marker.py:158: TracerWarning: Converting a tensor to a Python integer might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
ys_shape = tuple(int(s) for s in ys.shape)
/home/richard/.local/lib/python3.6/site-packages/torch/nn/functional.py:3455: UserWarning: Default upsampling behavior when mode=bilinear is changed to align_corners=False since 0.4.0. Please specify align_corners=True if the old behavior is desired. See the documentation of nn.Upsample for details.
"See the documentation of nn.Upsample for details.".format(mode)
/home/richard/Projects/MMDetection/mmdet/models/dense_heads/yolo_head.py:126: UserWarning: DeprecationWarning: anchor_generator is deprecated, please use "prior_generator" instead
warnings.warn('DeprecationWarning: anchor_generator is deprecated, '
/home/richard/Projects/MMDetection/mmdet/core/anchor/anchor_generator.py:333: UserWarning: grid_anchors would be deprecated soon. Please use grid_priors
warnings.warn('grid_anchors would be deprecated soon. '
/home/richard/Projects/MMDetection/mmdet/core/anchor/anchor_generator.py:370: UserWarning: single_level_grid_anchors would be deprecated soon. Please use single_level_grid_priors
'single_level_grid_anchors would be deprecated soon. '
/home/richard/Projects/MMDetection/mmdet/core/bbox/coder/yolo_bbox_coder.py:73: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
assert pred_bboxes.size(-1) == bboxes.size(-1) == 4
/home/richard/Projects/mmdeploy/mmdeploy/pytorch/functions/topk.py:54: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
if k > size:
/home/richard/Projects/mmdeploy/mmdeploy/codebase/mmdet/core/post_processing/bbox_nms.py:260: TracerWarning: Converting a tensor to a Python integer might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
dets, labels = TRTBatchedNMSop.apply(boxes, scores, int(scores.shape[-1]),
/home/richard/Projects/mmdeploy/mmdeploy/mmcv/ops/nms.py:177: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
out_boxes = min(num_boxes, after_topk)
/home/richard/Projects/mmdeploy/mmdeploy/mmcv/ops/nms.py:181: TracerWarning: Converting a tensor to a Python integer might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
(batch_size, out_boxes)).to(scores.device)
2022-03-03 15:03:10,601 - mmdeploy - INFO - torch2onnx success.
2022-03-03 15:03:11,531 - mmdeploy - INFO - onnx2tensorrt of test_work/test_yolo/end2end.onnx start.
2022-03-03 15:03:19,317 - mmdeploy - INFO - Successfully loaded tensorrt plugins from /home/richard/Projects/mmdeploy/build/lib/libmmdeploy_tensorrt_ops.so
[TensorRT] INFO: [MemUsageChange] Init CUDA: CPU +239, GPU +0, now: CPU 309, GPU 5456 (MiB)
[TensorRT] WARNING: onnx2trt_utils.cpp:364: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[TensorRT] WARNING: onnx2trt_utils.cpp:390: One or more weights outside the range of INT32 was clamped
[TensorRT] WARNING: onnx2trt_utils.cpp:390: One or more weights outside the range of INT32 was clamped
[TensorRT] WARNING: onnx2trt_utils.cpp:390: One or more weights outside the range of INT32 was clamped
[TensorRT] WARNING: onnx2trt_utils.cpp:390: One or more weights outside the range of INT32 was clamped
[TensorRT] WARNING: onnx2trt_utils.cpp:390: One or more weights outside the range of INT32 was clamped
[TensorRT] WARNING: onnx2trt_utils.cpp:390: One or more weights outside the range of INT32 was clamped
[TensorRT] WARNING: onnx2trt_utils.cpp:390: One or more weights outside the range of INT32 was clamped
[TensorRT] WARNING: onnx2trt_utils.cpp:390: One or more weights outside the range of INT32 was clamped
[TensorRT] WARNING: onnx2trt_utils.cpp:390: One or more weights outside the range of INT32 was clamped
[TensorRT] WARNING: onnx2trt_utils.cpp:390: One or more weights outside the range of INT32 was clamped
[TensorRT] WARNING: onnx2trt_utils.cpp:390: One or more weights outside the range of INT32 was clamped
[TensorRT] WARNING: onnx2trt_utils.cpp:390: One or more weights outside the range of INT32 was clamped
[TensorRT] WARNING: onnx2trt_utils.cpp:390: One or more weights outside the range of INT32 was clamped
[TensorRT] WARNING: onnx2trt_utils.cpp:390: One or more weights outside the range of INT32 was clamped
[TensorRT] WARNING: onnx2trt_utils.cpp:390: One or more weights outside the range of INT32 was clamped
[TensorRT] INFO: No importer registered for op: TRTBatchedNMS. Attempting to import as plugin.
[TensorRT] INFO: Searching for plugin: TRTBatchedNMS, plugin_version: 1, plugin_namespace:
[TensorRT] INFO: Successfully created plugin: TRTBatchedNMS
[TensorRT] WARNING: DLA requests all profiles have same min, max, and opt value. All dla layers are falling back to GPU
[TensorRT] INFO: [MemUsageSnapshot] Builder begin: CPU 787 MiB, GPU 5936 MiB
[TensorRT] INFO: ---------- Layers Running on DLA ----------
[TensorRT] INFO: ---------- Layers Running on GPU ----------
[TensorRT] INFO: [GpuLayer] Range_292
[TensorRT] INFO: [GpuLayer] Range_296
[TensorRT] INFO: [GpuLayer] Range_245
[TensorRT] INFO: [GpuLayer] Range_249
[TensorRT] INFO: [GpuLayer] Range_198
[TensorRT] INFO: [GpuLayer] Range_202
[TensorRT] INFO: [GpuLayer] Range_840
[TensorRT] INFO: [GpuLayer] Range_662
[TensorRT] INFO: [GpuLayer] Range_484
[TensorRT] INFO: [GpuLayer] Conv_0
[TensorRT] INFO: [GpuLayer] [HostToDeviceCopy]
[TensorRT] INFO: [GpuLayer] (Unnamed Layer* 450) [Shuffle]_output[Constant]
[TensorRT] INFO: [GpuLayer] Mul_294
[TensorRT] INFO: [GpuLayer] Unsqueeze_302
[TensorRT] INFO: [GpuLayer] Expand_309
[TensorRT] INFO: [GpuLayer] Mul_298
[TensorRT] INFO: [GpuLayer] Reshape_313
[TensorRT] INFO: [GpuLayer] Expand_323
[TensorRT] INFO: [GpuLayer] (Unnamed Layer* 327) [Shuffle]_output[Constant]
[TensorRT] INFO: [GpuLayer] Mul_247
[TensorRT] INFO: [GpuLayer] Unsqueeze_255
[TensorRT] INFO: [GpuLayer] Expand_262
[TensorRT] INFO: [GpuLayer] Mul_251
[TensorRT] INFO: [GpuLayer] Reshape_266
[TensorRT] INFO: [GpuLayer] Expand_276
[TensorRT] INFO: [GpuLayer] (Unnamed Layer* 204) [Shuffle]_output[Constant]
[TensorRT] INFO: [GpuLayer] Mul_200
[TensorRT] INFO: [GpuLayer] Unsqueeze_208
[TensorRT] INFO: [GpuLayer] Expand_215
[TensorRT] INFO: [GpuLayer] Mul_204
[TensorRT] INFO: [GpuLayer] Reshape_219
[TensorRT] INFO: [GpuLayer] Expand_229
[TensorRT] INFO: [GpuLayer] LeakyRelu_1
[TensorRT] INFO: [GpuLayer] Conv_2
[TensorRT] INFO: [GpuLayer] Tile_230
[TensorRT] INFO: [GpuLayer] Reshape_232
[TensorRT] INFO: [GpuLayer] Tile_216
[TensorRT] INFO: [GpuLayer] Tile_277
[TensorRT] INFO: [GpuLayer] Reshape_279
[TensorRT] INFO: [GpuLayer] Tile_263
[TensorRT] INFO: [GpuLayer] Tile_324
[TensorRT] INFO: [GpuLayer] Reshape_326
[TensorRT] INFO: [GpuLayer] Tile_310
[TensorRT] INFO: [GpuLayer] LeakyRelu_3
[TensorRT] INFO: [GpuLayer] Conv_4
[TensorRT] INFO: [GpuLayer] LeakyRelu_5
[TensorRT] INFO: [GpuLayer] Conv_6
[TensorRT] INFO: [GpuLayer] PWN(LeakyRelu_7, Add_8)
[TensorRT] INFO: [GpuLayer] Conv_9
[TensorRT] INFO: [GpuLayer] LeakyRelu_10
[TensorRT] INFO: [GpuLayer] Conv_11
[TensorRT] INFO: [GpuLayer] LeakyRelu_12
[TensorRT] INFO: [GpuLayer] Conv_13
[TensorRT] INFO: [GpuLayer] PWN(LeakyRelu_14, Add_15)
[TensorRT] INFO: [GpuLayer] Conv_16
[TensorRT] INFO: [GpuLayer] LeakyRelu_17
[TensorRT] INFO: [GpuLayer] Conv_18
[TensorRT] INFO: [GpuLayer] PWN(LeakyRelu_19, Add_20)
[TensorRT] INFO: [GpuLayer] Conv_21
[TensorRT] INFO: [GpuLayer] LeakyRelu_22
[TensorRT] INFO: [GpuLayer] Conv_23
[TensorRT] INFO: [GpuLayer] LeakyRelu_24
[TensorRT] INFO: [GpuLayer] Conv_25
[TensorRT] INFO: [GpuLayer] PWN(LeakyRelu_26, Add_27)
[TensorRT] INFO: [GpuLayer] Conv_28
[TensorRT] INFO: [GpuLayer] LeakyRelu_29
[TensorRT] INFO: [GpuLayer] Conv_30
[TensorRT] INFO: [GpuLayer] PWN(LeakyRelu_31, Add_32)
[TensorRT] INFO: [GpuLayer] Conv_33
[TensorRT] INFO: [GpuLayer] LeakyRelu_34
[TensorRT] INFO: [GpuLayer] Conv_35
[TensorRT] INFO: [GpuLayer] PWN(LeakyRelu_36, Add_37)
[TensorRT] INFO: [GpuLayer] Conv_38
[TensorRT] INFO: [GpuLayer] LeakyRelu_39
[TensorRT] INFO: [GpuLayer] Conv_40
[TensorRT] INFO: [GpuLayer] PWN(LeakyRelu_41, Add_42)
[TensorRT] INFO: [GpuLayer] Conv_43
[TensorRT] INFO: [GpuLayer] LeakyRelu_44
[TensorRT] INFO: [GpuLayer] Conv_45
[TensorRT] INFO: [GpuLayer] PWN(LeakyRelu_46, Add_47)
[TensorRT] INFO: [GpuLayer] Conv_48
[TensorRT] INFO: [GpuLayer] LeakyRelu_49
[TensorRT] INFO: [GpuLayer] Conv_50
[TensorRT] INFO: [GpuLayer] PWN(LeakyRelu_51, Add_52)
[TensorRT] INFO: [GpuLayer] Conv_53
[TensorRT] INFO: [GpuLayer] LeakyRelu_54
[TensorRT] INFO: [GpuLayer] Conv_55
[TensorRT] INFO: [GpuLayer] PWN(LeakyRelu_56, Add_57)
[TensorRT] INFO: [GpuLayer] Conv_58
[TensorRT] INFO: [GpuLayer] LeakyRelu_59
[TensorRT] INFO: [GpuLayer] Conv_60
[TensorRT] INFO: [GpuLayer] PWN(LeakyRelu_61, Add_62)
[TensorRT] INFO: [GpuLayer] Conv_63
[TensorRT] INFO: [GpuLayer] LeakyRelu_64
[TensorRT] INFO: [GpuLayer] Conv_65
[TensorRT] INFO: [GpuLayer] LeakyRelu_66
[TensorRT] INFO: [GpuLayer] Conv_67
[TensorRT] INFO: [GpuLayer] PWN(LeakyRelu_68, Add_69)
[TensorRT] INFO: [GpuLayer] Conv_70
[TensorRT] INFO: [GpuLayer] LeakyRelu_71
[TensorRT] INFO: [GpuLayer] Conv_72
[TensorRT] INFO: [GpuLayer] PWN(LeakyRelu_73, Add_74)
[TensorRT] INFO: [GpuLayer] Conv_75
[TensorRT] INFO: [GpuLayer] LeakyRelu_76
[TensorRT] INFO: [GpuLayer] Conv_77
[TensorRT] INFO: [GpuLayer] PWN(LeakyRelu_78, Add_79)
[TensorRT] INFO: [GpuLayer] Conv_80
[TensorRT] INFO: [GpuLayer] LeakyRelu_81
[TensorRT] INFO: [GpuLayer] Conv_82
[TensorRT] INFO: [GpuLayer] PWN(LeakyRelu_83, Add_84)
[TensorRT] INFO: [GpuLayer] Conv_85
[TensorRT] INFO: [GpuLayer] LeakyRelu_86
[TensorRT] INFO: [GpuLayer] Conv_87
[TensorRT] INFO: [GpuLayer] PWN(LeakyRelu_88, Add_89)
[TensorRT] INFO: [GpuLayer] Conv_90
[TensorRT] INFO: [GpuLayer] LeakyRelu_91
[TensorRT] INFO: [GpuLayer] Conv_92
[TensorRT] INFO: [GpuLayer] PWN(LeakyRelu_93, Add_94)
[TensorRT] INFO: [GpuLayer] Conv_95
[TensorRT] INFO: [GpuLayer] LeakyRelu_96
[TensorRT] INFO: [GpuLayer] Conv_97
[TensorRT] INFO: [GpuLayer] PWN(LeakyRelu_98, Add_99)
[TensorRT] INFO: [GpuLayer] Conv_100
[TensorRT] INFO: [GpuLayer] LeakyRelu_101
[TensorRT] INFO: [GpuLayer] Conv_102
[TensorRT] INFO: [GpuLayer] PWN(LeakyRelu_103, Add_104)
[TensorRT] INFO: [GpuLayer] Conv_105
[TensorRT] INFO: [GpuLayer] LeakyRelu_106
[TensorRT] INFO: [GpuLayer] Conv_107
[TensorRT] INFO: [GpuLayer] LeakyRelu_108
[TensorRT] INFO: [GpuLayer] Conv_109
[TensorRT] INFO: [GpuLayer] PWN(LeakyRelu_110, Add_111)
[TensorRT] INFO: [GpuLayer] Conv_112
[TensorRT] INFO: [GpuLayer] LeakyRelu_113
[TensorRT] INFO: [GpuLayer] Conv_114
[TensorRT] INFO: [GpuLayer] PWN(LeakyRelu_115, Add_116)
[TensorRT] INFO: [GpuLayer] Conv_117
[TensorRT] INFO: [GpuLayer] LeakyRelu_118
[TensorRT] INFO: [GpuLayer] Conv_119
[TensorRT] INFO: [GpuLayer] PWN(LeakyRelu_120, Add_121)
[TensorRT] INFO: [GpuLayer] Conv_122
[TensorRT] INFO: [GpuLayer] LeakyRelu_123
[TensorRT] INFO: [GpuLayer] Conv_124
[TensorRT] INFO: [GpuLayer] PWN(LeakyRelu_125, Add_126)
[TensorRT] INFO: [GpuLayer] Conv_127
[TensorRT] INFO: [GpuLayer] LeakyRelu_128
[TensorRT] INFO: [GpuLayer] Conv_129
[TensorRT] INFO: [GpuLayer] LeakyRelu_130
[TensorRT] INFO: [GpuLayer] Conv_131
[TensorRT] INFO: [GpuLayer] LeakyRelu_132
[TensorRT] INFO: [GpuLayer] Conv_133
[TensorRT] INFO: [GpuLayer] LeakyRelu_134
[TensorRT] INFO: [GpuLayer] Conv_135
[TensorRT] INFO: [GpuLayer] LeakyRelu_136
[TensorRT] INFO: [GpuLayer] Conv_167
[TensorRT] INFO: [GpuLayer] Conv_137
[TensorRT] INFO: [GpuLayer] LeakyRelu_168
[TensorRT] INFO: [GpuLayer] LeakyRelu_138
[TensorRT] INFO: [GpuLayer] Resize_140
[TensorRT] INFO: [GpuLayer] Conv_169
[TensorRT] INFO: [GpuLayer] Transpose_338 + Reshape_341
[TensorRT] INFO: [GpuLayer] 983
[TensorRT] INFO: [GpuLayer] Slice_352
[TensorRT] INFO: [GpuLayer] Slice_346
[TensorRT] INFO: [GpuLayer] PWN(Sigmoid_347)
[TensorRT] INFO: [GpuLayer] 910 copy
[TensorRT] INFO: [GpuLayer] 915 copy
[TensorRT] INFO: [GpuLayer] Slice_358
[TensorRT] INFO: [GpuLayer] Slice_379
[TensorRT] INFO: [GpuLayer] 771
[TensorRT] INFO: [GpuLayer] Unsqueeze_234
[TensorRT] INFO: [GpuLayer] Squeeze_217
[TensorRT] INFO: [GpuLayer] Unsqueeze_233
[TensorRT] INFO: [GpuLayer] 764 copy
[TensorRT] INFO: [GpuLayer] 765 copy
[TensorRT] INFO: [GpuLayer] 764 copy
[TensorRT] INFO: [GpuLayer] 765 copy
[TensorRT] INFO: [GpuLayer] Cast_238
[TensorRT] INFO: [GpuLayer] Unsqueeze_239
[TensorRT] INFO: [GpuLayer] Add_241
[TensorRT] INFO: [GpuLayer] Reshape_243 + Unsqueeze_359
[TensorRT] INFO: [GpuLayer] Expand_361
[TensorRT] INFO: [GpuLayer] Slice_366
[TensorRT] INFO: [GpuLayer] Slice_371
[TensorRT] INFO: [GpuLayer] PWN(PWN(Add_372, PWN(936 + (Unnamed Layer* 706) [Shuffle], Mul_374)), PWN(PWN(PWN(943 + (Unnamed Layer* 732) [Shuffle], Sub_381), PWN(945 + (Unnamed Layer* 735) [Shuffle], Mul_383)), Add_384))
[TensorRT] INFO: [GpuLayer] 978
[TensorRT] INFO: [GpuLayer] 973
[TensorRT] INFO: [GpuLayer] 968
[TensorRT] INFO: [GpuLayer] 985
[TensorRT] INFO: [GpuLayer] Slice_402
[TensorRT] INFO: [GpuLayer] Slice_389
[TensorRT] INFO: [GpuLayer] Slice_394
[TensorRT] INFO: [GpuLayer] PWN(PWN(Sub_395, PWN(959 + (Unnamed Layer* 786) [Shuffle], Mul_397)), PWN(Exp_403, Mul_404))
[TensorRT] INFO: [GpuLayer] 980
[TensorRT] INFO: [GpuLayer] 975
[TensorRT] INFO: [GpuLayer] 970
[TensorRT] INFO: [GpuLayer] 993
[TensorRT] INFO: [GpuLayer] Slice_437
[TensorRT] INFO: [GpuLayer] PWN(Sigmoid_438)
[TensorRT] INFO: [GpuLayer] Slice_470
[TensorRT] INFO: [GpuLayer] PWN(PWN(1040 + (Unnamed Layer* 1006) [Shuffle], Mul_472), PWN(1042 + (Unnamed Layer* 1009) [Shuffle], Add_474))
[TensorRT] INFO: [GpuLayer] Expand_477
[TensorRT] INFO: [GpuLayer] Tile_478
[TensorRT] INFO: [GpuLayer] 640 copy
[TensorRT] INFO: [GpuLayer] Conv_142
[TensorRT] INFO: [GpuLayer] 1001 copy
[TensorRT] INFO: [GpuLayer] 1048 copy
[TensorRT] INFO: [GpuLayer] Gather_431
[TensorRT] INFO: [GpuLayer] Gather_408
[TensorRT] INFO: [GpuLayer] Gather_413
[TensorRT] INFO: [GpuLayer] Gather_418
[TensorRT] INFO: [GpuLayer] Gather_423
[TensorRT] INFO: [GpuLayer] Gather_406
[TensorRT] INFO: [GpuLayer] Gather_411
[TensorRT] INFO: [GpuLayer] Gather_416
[TensorRT] INFO: [GpuLayer] Gather_421
[TensorRT] INFO: [GpuLayer] PWN(Sigmoid_432)
[TensorRT] INFO: [GpuLayer] Slice_456
[TensorRT] INFO: [GpuLayer] PWN(PWN(1025 + (Unnamed Layer* 957) [Shuffle], Mul_458), PWN(1027 + (Unnamed Layer* 960) [Shuffle], Add_460))
[TensorRT] INFO: [GpuLayer] Expand_463
[TensorRT] INFO: [GpuLayer] Tile_464
[TensorRT] INFO: [GpuLayer] LeakyRelu_143
[TensorRT] INFO: [GpuLayer] Sub_409
[TensorRT] INFO: [GpuLayer] Unsqueeze_425
[TensorRT] INFO: [GpuLayer] Sub_414
[TensorRT] INFO: [GpuLayer] Unsqueeze_426
[TensorRT] INFO: [GpuLayer] Add_419
[TensorRT] INFO: [GpuLayer] Unsqueeze_427
[TensorRT] INFO: [GpuLayer] Add_424
[TensorRT] INFO: [GpuLayer] Unsqueeze_428
[TensorRT] INFO: [GpuLayer] 988 copy
[TensorRT] INFO: [GpuLayer] 989 copy
[TensorRT] INFO: [GpuLayer] 990 copy
[TensorRT] INFO: [GpuLayer] 991 copy
[TensorRT] INFO: [GpuLayer] Slice_446
[TensorRT] INFO: [GpuLayer] Expand_449
[TensorRT] INFO: [GpuLayer] Tile_450
[TensorRT] INFO: [GpuLayer] 992 copy
[TensorRT] INFO: [GpuLayer] 1018 copy
[TensorRT] INFO: [GpuLayer] Conv_144
[TensorRT] INFO: [GpuLayer] 995 copy
[TensorRT] INFO: [GpuLayer] 1033 copy
[TensorRT] INFO: [GpuLayer] TopK_480
[TensorRT] INFO: [GpuLayer] Reshape_510
[TensorRT] INFO: [GpuLayer] Reshape_486
[TensorRT] INFO: [GpuLayer] Expand_488
[TensorRT] INFO: [GpuLayer] Cast_489
[TensorRT] INFO: [GpuLayer] (Unnamed Layer* 1056) [Shuffle]
[TensorRT] INFO: [GpuLayer] Mul_493
[TensorRT] INFO: [GpuLayer] Add_494
[TensorRT] INFO: [GpuLayer] Cast_497
[TensorRT] INFO: [GpuLayer] Cast_504
[TensorRT] INFO: [GpuLayer] Cast_511
[TensorRT] INFO: [GpuLayer] Reshape_503
[TensorRT] INFO: [GpuLayer] Reshape_496
[TensorRT] INFO: [GpuLayer] LeakyRelu_145
[TensorRT] INFO: [GpuLayer] Conv_146
[TensorRT] INFO: [GpuLayer] Gather_498
[TensorRT] INFO: [GpuLayer] Gather_505
[TensorRT] INFO: [GpuLayer] Gather_512
[TensorRT] INFO: [GpuLayer] LeakyRelu_147
[TensorRT] INFO: [GpuLayer] Conv_148
[TensorRT] INFO: [GpuLayer] LeakyRelu_149
[TensorRT] INFO: [GpuLayer] Conv_150
[TensorRT] INFO: [GpuLayer] LeakyRelu_151
[TensorRT] INFO: [GpuLayer] Conv_170
[TensorRT] INFO: [GpuLayer] Conv_152
[TensorRT] INFO: [GpuLayer] LeakyRelu_171
[TensorRT] INFO: [GpuLayer] LeakyRelu_153
[TensorRT] INFO: [GpuLayer] Resize_155
[TensorRT] INFO: [GpuLayer] Conv_172
[TensorRT] INFO: [GpuLayer] Transpose_516 + Reshape_519
[TensorRT] INFO: [GpuLayer] 1186
[TensorRT] INFO: [GpuLayer] Slice_530
[TensorRT] INFO: [GpuLayer] Slice_524
[TensorRT] INFO: [GpuLayer] PWN(Sigmoid_525)
[TensorRT] INFO: [GpuLayer] 1113 copy
[TensorRT] INFO: [GpuLayer] 1118 copy
[TensorRT] INFO: [GpuLayer] Slice_536
[TensorRT] INFO: [GpuLayer] Slice_557
[TensorRT] INFO: [GpuLayer] 832
[TensorRT] INFO: [GpuLayer] Unsqueeze_281
[TensorRT] INFO: [GpuLayer] Squeeze_264
[TensorRT] INFO: [GpuLayer] Unsqueeze_280
[TensorRT] INFO: [GpuLayer] 825 copy
[TensorRT] INFO: [GpuLayer] 826 copy
[TensorRT] INFO: [GpuLayer] 825 copy
[TensorRT] INFO: [GpuLayer] 826 copy
[TensorRT] INFO: [GpuLayer] Cast_285
[TensorRT] INFO: [GpuLayer] Unsqueeze_286
[TensorRT] INFO: [GpuLayer] Add_288
[TensorRT] INFO: [GpuLayer] Reshape_290 + Unsqueeze_537
[TensorRT] INFO: [GpuLayer] Expand_539
[TensorRT] INFO: [GpuLayer] Slice_544
[TensorRT] INFO: [GpuLayer] Slice_549
[TensorRT] INFO: [GpuLayer] PWN(PWN(Add_550, PWN(1139 + (Unnamed Layer* 1224) [Shuffle], Mul_552)), PWN(PWN(PWN(1146 + (Unnamed Layer* 1250) [Shuffle], Sub_559), PWN(1148 + (Unnamed Layer* 1253) [Shuffle], Mul_561)), Add_562))
[TensorRT] INFO: [GpuLayer] 1181
[TensorRT] INFO: [GpuLayer] 1176
[TensorRT] INFO: [GpuLayer] 1171
[TensorRT] INFO: [GpuLayer] 1188
[TensorRT] INFO: [GpuLayer] Slice_580
[TensorRT] INFO: [GpuLayer] Slice_567
[TensorRT] INFO: [GpuLayer] Slice_572
[TensorRT] INFO: [GpuLayer] PWN(PWN(Sub_573, PWN(1162 + (Unnamed Layer* 1304) [Shuffle], Mul_575)), PWN(Exp_581, Mul_582))
[TensorRT] INFO: [GpuLayer] 1183
[TensorRT] INFO: [GpuLayer] 1178
[TensorRT] INFO: [GpuLayer] 1173
[TensorRT] INFO: [GpuLayer] 1196
[TensorRT] INFO: [GpuLayer] Slice_615
[TensorRT] INFO: [GpuLayer] PWN(Sigmoid_616)
[TensorRT] INFO: [GpuLayer] Slice_648
[TensorRT] INFO: [GpuLayer] PWN(PWN(1243 + (Unnamed Layer* 1524) [Shuffle], Mul_650), PWN(1245 + (Unnamed Layer* 1527) [Shuffle], Add_652))
[TensorRT] INFO: [GpuLayer] Expand_655
[TensorRT] INFO: [GpuLayer] Tile_656
[TensorRT] INFO: [GpuLayer] 664 copy
[TensorRT] INFO: [GpuLayer] Conv_157
[TensorRT] INFO: [GpuLayer] 1204 copy
[TensorRT] INFO: [GpuLayer] 1251 copy
[TensorRT] INFO: [GpuLayer] Gather_609
[TensorRT] INFO: [GpuLayer] Gather_586
[TensorRT] INFO: [GpuLayer] Gather_591
[TensorRT] INFO: [GpuLayer] Gather_596
[TensorRT] INFO: [GpuLayer] Gather_601
[TensorRT] INFO: [GpuLayer] Gather_584
[TensorRT] INFO: [GpuLayer] Gather_589
[TensorRT] INFO: [GpuLayer] Gather_594
[TensorRT] INFO: [GpuLayer] Gather_599
[TensorRT] INFO: [GpuLayer] PWN(Sigmoid_610)
[TensorRT] INFO: [GpuLayer] Slice_634
[TensorRT] INFO: [GpuLayer] PWN(PWN(1228 + (Unnamed Layer* 1475) [Shuffle], Mul_636), PWN(1230 + (Unnamed Layer* 1478) [Shuffle], Add_638))
[TensorRT] INFO: [GpuLayer] Expand_641
[TensorRT] INFO: [GpuLayer] Tile_642
[TensorRT] INFO: [GpuLayer] LeakyRelu_158
[TensorRT] INFO: [GpuLayer] Sub_587
[TensorRT] INFO: [GpuLayer] Unsqueeze_603
[TensorRT] INFO: [GpuLayer] Sub_592
[TensorRT] INFO: [GpuLayer] Unsqueeze_604
[TensorRT] INFO: [GpuLayer] Add_597
[TensorRT] INFO: [GpuLayer] Unsqueeze_605
[TensorRT] INFO: [GpuLayer] Add_602
[TensorRT] INFO: [GpuLayer] Unsqueeze_606
[TensorRT] INFO: [GpuLayer] 1191 copy
[TensorRT] INFO: [GpuLayer] 1192 copy
[TensorRT] INFO: [GpuLayer] 1193 copy
[TensorRT] INFO: [GpuLayer] 1194 copy
[TensorRT] INFO: [GpuLayer] Slice_624
[TensorRT] INFO: [GpuLayer] Expand_627
[TensorRT] INFO: [GpuLayer] Tile_628
[TensorRT] INFO: [GpuLayer] 1195 copy
[TensorRT] INFO: [GpuLayer] 1221 copy
[TensorRT] INFO: [GpuLayer] Conv_159
[TensorRT] INFO: [GpuLayer] 1198 copy
[TensorRT] INFO: [GpuLayer] 1236 copy
[TensorRT] INFO: [GpuLayer] TopK_658
[TensorRT] INFO: [GpuLayer] Reshape_688
[TensorRT] INFO: [GpuLayer] Reshape_664
[TensorRT] INFO: [GpuLayer] Expand_666
[TensorRT] INFO: [GpuLayer] Cast_667
[TensorRT] INFO: [GpuLayer] (Unnamed Layer* 1574) [Shuffle]
[TensorRT] INFO: [GpuLayer] Mul_671
[TensorRT] INFO: [GpuLayer] Add_672
[TensorRT] INFO: [GpuLayer] Cast_675
[TensorRT] INFO: [GpuLayer] Cast_682
[TensorRT] INFO: [GpuLayer] Cast_689
[TensorRT] INFO: [GpuLayer] Reshape_681
[TensorRT] INFO: [GpuLayer] Reshape_674
[TensorRT] INFO: [GpuLayer] LeakyRelu_160
[TensorRT] INFO: [GpuLayer] Conv_161
[TensorRT] INFO: [GpuLayer] Gather_676
[TensorRT] INFO: [GpuLayer] Gather_683
[TensorRT] INFO: [GpuLayer] Gather_690
[TensorRT] INFO: [GpuLayer] LeakyRelu_162
[TensorRT] INFO: [GpuLayer] Conv_163
[TensorRT] INFO: [GpuLayer] LeakyRelu_164
[TensorRT] INFO: [GpuLayer] Conv_165
[TensorRT] INFO: [GpuLayer] LeakyRelu_166
[TensorRT] INFO: [GpuLayer] Conv_173
[TensorRT] INFO: [GpuLayer] LeakyRelu_174
[TensorRT] INFO: [GpuLayer] Conv_175
[TensorRT] INFO: [GpuLayer] Transpose_694 + Reshape_697
[TensorRT] INFO: [GpuLayer] 1389
[TensorRT] INFO: [GpuLayer] Slice_708
[TensorRT] INFO: [GpuLayer] Slice_702
[TensorRT] INFO: [GpuLayer] PWN(Sigmoid_703)
[TensorRT] INFO: [GpuLayer] 1316 copy
[TensorRT] INFO: [GpuLayer] 1321 copy
[TensorRT] INFO: [GpuLayer] Slice_714
[TensorRT] INFO: [GpuLayer] Slice_735
[TensorRT] INFO: [GpuLayer] 893
[TensorRT] INFO: [GpuLayer] Unsqueeze_328
[TensorRT] INFO: [GpuLayer] Squeeze_311
[TensorRT] INFO: [GpuLayer] Unsqueeze_327
[TensorRT] INFO: [GpuLayer] 886 copy
[TensorRT] INFO: [GpuLayer] 887 copy
[TensorRT] INFO: [GpuLayer] 886 copy
[TensorRT] INFO: [GpuLayer] 887 copy
[TensorRT] INFO: [GpuLayer] Cast_332
[TensorRT] INFO: [GpuLayer] Unsqueeze_333
[TensorRT] INFO: [GpuLayer] Add_335
[TensorRT] INFO: [GpuLayer] Reshape_337 + Unsqueeze_715
[TensorRT] INFO: [GpuLayer] Expand_717
[TensorRT] INFO: [GpuLayer] Slice_722
[TensorRT] INFO: [GpuLayer] Slice_727
[TensorRT] INFO: [GpuLayer] PWN(PWN(Add_728, PWN(1342 + (Unnamed Layer* 1742) [Shuffle], Mul_730)), PWN(PWN(PWN(1349 + (Unnamed Layer* 1768) [Shuffle], Sub_737), PWN(1351 + (Unnamed Layer* 1771) [Shuffle], Mul_739)), Add_740))
[TensorRT] INFO: [GpuLayer] 1384
[TensorRT] INFO: [GpuLayer] 1379
[TensorRT] INFO: [GpuLayer] 1374
[TensorRT] INFO: [GpuLayer] 1391
[TensorRT] INFO: [GpuLayer] Slice_758
[TensorRT] INFO: [GpuLayer] Slice_745
[TensorRT] INFO: [GpuLayer] Slice_750
[TensorRT] INFO: [GpuLayer] PWN(PWN(Sub_751, PWN(1365 + (Unnamed Layer* 1822) [Shuffle], Mul_753)), PWN(Exp_759, Mul_760))
[TensorRT] INFO: [GpuLayer] 1386
[TensorRT] INFO: [GpuLayer] 1381
[TensorRT] INFO: [GpuLayer] 1376
[TensorRT] INFO: [GpuLayer] 1399
[TensorRT] INFO: [GpuLayer] Slice_793
[TensorRT] INFO: [GpuLayer] PWN(Sigmoid_794)
[TensorRT] INFO: [GpuLayer] Slice_826
[TensorRT] INFO: [GpuLayer] PWN(PWN(1446 + (Unnamed Layer* 2042) [Shuffle], Mul_828), PWN(1448 + (Unnamed Layer* 2045) [Shuffle], Add_830))
[TensorRT] INFO: [GpuLayer] Expand_833
[TensorRT] INFO: [GpuLayer] Tile_834
[TensorRT] INFO: [GpuLayer] 1407 copy
[TensorRT] INFO: [GpuLayer] 1454 copy
[TensorRT] INFO: [GpuLayer] Gather_787
[TensorRT] INFO: [GpuLayer] Gather_764
[TensorRT] INFO: [GpuLayer] Gather_769
[TensorRT] INFO: [GpuLayer] Gather_774
[TensorRT] INFO: [GpuLayer] Gather_779
[TensorRT] INFO: [GpuLayer] Gather_762
[TensorRT] INFO: [GpuLayer] Gather_767
[TensorRT] INFO: [GpuLayer] Gather_772
[TensorRT] INFO: [GpuLayer] Gather_777
[TensorRT] INFO: [GpuLayer] PWN(Sigmoid_788)
[TensorRT] INFO: [GpuLayer] Slice_812
[TensorRT] INFO: [GpuLayer] PWN(PWN(1431 + (Unnamed Layer* 1993) [Shuffle], Mul_814), PWN(1433 + (Unnamed Layer* 1996) [Shuffle], Add_816))
[TensorRT] INFO: [GpuLayer] Expand_819
[TensorRT] INFO: [GpuLayer] Tile_820
[TensorRT] INFO: [GpuLayer] Sub_765
[TensorRT] INFO: [GpuLayer] Unsqueeze_781
[TensorRT] INFO: [GpuLayer] Sub_770
[TensorRT] INFO: [GpuLayer] Unsqueeze_782
[TensorRT] INFO: [GpuLayer] Add_775
[TensorRT] INFO: [GpuLayer] Unsqueeze_783
[TensorRT] INFO: [GpuLayer] Add_780
[TensorRT] INFO: [GpuLayer] Unsqueeze_784
[TensorRT] INFO: [GpuLayer] 1394 copy
[TensorRT] INFO: [GpuLayer] 1395 copy
[TensorRT] INFO: [GpuLayer] 1396 copy
[TensorRT] INFO: [GpuLayer] 1397 copy
[TensorRT] INFO: [GpuLayer] Slice_802
[TensorRT] INFO: [GpuLayer] Expand_805
[TensorRT] INFO: [GpuLayer] Tile_806
[TensorRT] INFO: [GpuLayer] 1398 copy
[TensorRT] INFO: [GpuLayer] 1424 copy
[TensorRT] INFO: [GpuLayer] 1401 copy
[TensorRT] INFO: [GpuLayer] 1439 copy
[TensorRT] INFO: [GpuLayer] TopK_836
[TensorRT] INFO: [GpuLayer] Reshape_866
[TensorRT] INFO: [GpuLayer] Reshape_842
[TensorRT] INFO: [GpuLayer] Expand_844
[TensorRT] INFO: [GpuLayer] Cast_845
[TensorRT] INFO: [GpuLayer] (Unnamed Layer* 2092) [Shuffle]
[TensorRT] INFO: [GpuLayer] Mul_849
[TensorRT] INFO: [GpuLayer] Add_850
[TensorRT] INFO: [GpuLayer] Cast_853
[TensorRT] INFO: [GpuLayer] Cast_860
[TensorRT] INFO: [GpuLayer] Cast_867
[TensorRT] INFO: [GpuLayer] Reshape_859
[TensorRT] INFO: [GpuLayer] Reshape_852
[TensorRT] INFO: [GpuLayer] Gather_854
[TensorRT] INFO: [GpuLayer] Gather_861
[TensorRT] INFO: [GpuLayer] Gather_868
[TensorRT] INFO: [GpuLayer] {ForeignNode[1509 + (Unnamed Layer* 2122) [Shuffle]...Mul_887]}
[TensorRT] INFO: [GpuLayer] 1072 copy
[TensorRT] INFO: [GpuLayer] 1275 copy
[TensorRT] INFO: [GpuLayer] 1478 copy
[TensorRT] INFO: [GpuLayer] Unsqueeze_888
[TensorRT] INFO: [GpuLayer] TRTBatchedNMS_889
[TensorRT] INFO: [MemUsageChange] Init cuBLAS/cuBLASLt: CPU +167, GPU +252, now: CPU 955, GPU 6189 (MiB)
[TensorRT] INFO: [MemUsageChange] Init cuDNN: CPU +250, GPU +356, now: CPU 1205, GPU 6545 (MiB)
[TensorRT] WARNING: Detected invalid timing cache, setup a local cache instead
[TensorRT] INFO: Some tactics do not have sufficient workspace memory to run. Increasing workspace size may increase performance, please check verbose output.
[TensorRT] INFO: Detected 1 inputs and 2 output network tensors.
[TensorRT] INFO: Total Host Persistent Memory: 178736
[TensorRT] INFO: Total Device Persistent Memory: 368577536
[TensorRT] INFO: Total Scratch Memory: 5808640
[TensorRT] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 18 MiB, GPU 768 MiB
[TensorRT] INFO: [MemUsageChange] Init cuBLAS/cuBLASLt: CPU +0, GPU +0, now: CPU 1462, GPU 7065 (MiB)
[TensorRT] INFO: [MemUsageChange] Init cuDNN: CPU +0, GPU +0, now: CPU 1462, GPU 7065 (MiB)
[TensorRT] INFO: [MemUsageChange] Init cuBLAS/cuBLASLt: CPU +0, GPU +0, now: CPU 1462, GPU 7065 (MiB)
[TensorRT] INFO: [MemUsageChange] Init cuBLAS/cuBLASLt: CPU +0, GPU +0, now: CPU 1461, GPU 7065 (MiB)
[TensorRT] INFO: [MemUsageSnapshot] Builder end: CPU 1460 MiB, GPU 7065 MiB
2022-03-03 15:06:15,405 - mmdeploy - INFO - onnx2tensorrt of test_work/test_yolo/end2end.onnx success.
2022-03-03 15:06:15,406 - mmdeploy - INFO - visualize tensorrt model start.
2022-03-03 15:06:54,397 - mmdeploy - INFO - Successfully loaded tensorrt plugins from /home/richard/Projects/mmdeploy/build/lib/libmmdeploy_tensorrt_ops.so
2022-03-03 15:06:54,398 - mmdeploy - INFO - Successfully loaded tensorrt plugins from /home/richard/Projects/mmdeploy/build/lib/libmmdeploy_tensorrt_ops.so
/home/richard/Projects/MMDetection/mmdet/datasets/utils.py:69: UserWarning: "ImageToTensor" pipeline is replaced by "DefaultFormatBundle" for batch inference. It is recommended to manually replace it in the test data pipeline in your config file.
'data pipeline in your config file.', UserWarning)
#assertion/home/richard/Projects/mmdeploy/csrc/backend_ops/tensorrt/batched_nms/trt_batched_nms.cpp,98
2022-03-03 15:07:06,696 - mmdeploy - ERROR - visualize tensorrt model failed.