RuntimeError: cuDNN error: CUDNN_STATUS_NOT_INITIALIZED

[mortal-Zero]

Hi, I am using mmdeploy to convert maskrcnn pth model to tensorrt, and i got this error:
RuntimeError: cuDNN error: CUDNN_STATUS_NOT_INITIALIZED

my command:
python tools/deploy.py \ configs/mmdet/instance-seg/instance-seg_tensorrt_dynamic-320x320-1344x1344.py \ /home/mmlab/mmdetection-2.20.0/project/mrcnn_r18_fpn_160e/mrcnn_r18_fpn_160e.py \ /home/mmlab/mmdetection-2.20.0/project/mrcnn_r18_fpn_160e/epoch_150.pth \ /home/mmlab/mmdetection-2.20.0/demo/demo.jpg \ --work-dir work_dir \ --device cuda:0

my env:
gcc g++ 7.5.0
cuda 10.2
cudnn 7.6.5
tensorrt 7.2.3
torch 1.8.0

[AllentDan]

CUDNN_STATUS_NOT_INITIALIZE could be the error of the env. Maybe torch or cuda && cudnn not get properly installed like here. I used the exact same env like you with no errors.

[mortal-Zero]

CUDNN_STATUS_NOT_INITIALIZE could be the error of the env. Maybe torch or cuda && cudnn not get properly installed like here. I used the exact same env like you with no errors.

May I know what command you used to install torch?
Maybe it's a problem with the pip source, I used the pip tsinghua source, not the conda source.
I will try again.

[AllentDan]

I used conda to manage the env.

conda install pytorch==1.8.1 torchvision==0.9.1 cudatoolkit=10.2 -c pytorch

Or you can refer to my new PR which provided a dockerfile for gpu usage

[mortal-Zero]

I used conda to manage the env.

conda install pytorch==1.8.1 torchvision==0.9.1 cudatoolkit=10.2 -c pytorch

Or you can refer to my new PR which provided a dockerfile for gpu usage

thanks for this.
i will try again.

[AllentDan]

CUDNN_STATUS_NOT_INITIALIZE could be the error of the env. Maybe torch or cuda && cudnn not get properly installed like here. I used the exact same env like you with no errors.

May I know what command you used to install torch? Maybe it's a problem with the pip source, I used the pip tsinghua source, not the conda source. I will try again.

Could you run a pytorch model to forward a tensor with cuda?

[mortal-Zero]

I didn't test because it was a new computer. After setting up the environment, I just tested "torch.cuda.is_available()" and is True.

[mortal-Zero]

CUDNN_STATUS_NOT_INITIALIZE could be the error of the env. Maybe torch or cuda && cudnn not get properly installed like here. I used the exact same env like you with no errors.

May I know what command you used to install torch? Maybe it's a problem with the pip source, I used the pip tsinghua source, not the conda source. I will try again.

Could you run a pytorch model to forward a tensor with cuda?

I didn't test because it was a new computer. After setting up the environment, I just tested "torch.cuda.is_available()" and is True.

[mortal-Zero]

CUDNN_STATUS_NOT_INITIALIZE could be the error of the env. Maybe torch or cuda && cudnn not get properly installed like here. I used the exact same env like you with no errors.

May I know what command you used to install torch? Maybe it's a problem with the pip source, I used the pip tsinghua source, not the conda source. I will try again.

Could you run a pytorch model to forward a tensor with cuda?

Hi, I rebuilt the environment, but when I run it again, I get this error:

python tools/deploy.py configs/mmdet/instance-seg/instance-seg_tensorrt_dynamic-320x320-1344x1344.py /home/Data/mmlab/mmdetection-2.20.0/ETOP_dirs/mrcnn_r18_fpn_160e/etop_mrcnn_r18_fpn_160e.py /home/Data/mmlab/mmdetection-2.20.0/ETOP_dirs/mrcnn_r18_fpn_160e/epoch_150.pth /home/Data/mmlab/mmdetection-2.20.0/demo/demo.jpg --work-dir work_dir --device cuda:0

2022-01-19 18:48:24,249 - mmdeploy - INFO - torch2onnx start.
2022-01-19:18:48:27,matplotlib.font_manager INFO     [font_manager.py:1073] Failed to extract font properties from /usr/share/fonts/truetype/noto/NotoColorEmoji.ttf: In FT2Font: Can not load face.  Unknown file format.
2022-01-19:18:48:28,matplotlib.font_manager INFO     [font_manager.py:1443] generated new fontManager
load checkpoint from local path: /home/Data/mmlab/mmdetection-2.20.0/ETOP_dirs/mrcnn_r18_fpn_160e/epoch_150.pth
/home/Data/mmlab/mmdetection-2.20.0/mmdet/datasets/utils.py:69: UserWarning: "ImageToTensor" pipeline is replaced by "DefaultFormatBundle" for batch inference. It is recommended to manually replace it in the test data pipeline in your config file.
  'data pipeline in your config file.', UserWarning)
/home/Data/mmlab/mmdeploy-0.1.0/mmdeploy/core/optimizers/function_marker.py:158: TracerWarning: Converting a tensor to a Python integer might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  ys_shape = tuple(int(s) for s in ys.shape)
/home/Data/mmlab/mmdetection-2.20.0/mmdet/models/dense_heads/anchor_head.py:123: UserWarning: DeprecationWarning: anchor_generator is deprecated, please use "prior_generator" instead
  warnings.warn('DeprecationWarning: anchor_generator is deprecated, '
/home/Data/mmlab/mmdetection-2.20.0/mmdet/core/anchor/anchor_generator.py:333: UserWarning: ``grid_anchors`` would be deprecated soon. Please use ``grid_priors`` 
  warnings.warn('``grid_anchors`` would be deprecated soon. '
/home/Data/mmlab/mmdetection-2.20.0/mmdet/core/anchor/anchor_generator.py:370: UserWarning: ``single_level_grid_anchors`` would be deprecated soon. Please use ``single_level_grid_priors`` 
  '``single_level_grid_anchors`` would be deprecated soon. '
/home/Data/mmlab/mmdeploy-0.1.0/mmdeploy/codebase/mmdet/models/dense_heads/rpn_head.py:77: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  assert cls_score.size()[-2:] == bbox_pred.size()[-2:]
/home/Data/mmlab/mmdeploy-0.1.0/mmdeploy/pytorch/functions/topk.py:54: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  if k > size:
/home/Data/mmlab/mmdeploy-0.1.0/mmdeploy/codebase/mmdet/core/bbox/delta_xywh_bbox_coder.py:39: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  assert pred_bboxes.size(0) == bboxes.size(0)
/home/Data/mmlab/mmdeploy-0.1.0/mmdeploy/codebase/mmdet/core/bbox/delta_xywh_bbox_coder.py:41: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  assert pred_bboxes.size(1) == bboxes.size(1)
/home/Data/mmlab/mmdeploy-0.1.0/mmdeploy/codebase/mmdet/core/post_processing/bbox_nms.py:167: TracerWarning: Converting a tensor to a Python integer might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  dets, labels = TRTBatchedNMSop.apply(boxes, scores, int(scores.shape[-1]),
/home/Data/mmlab/mmdeploy-0.1.0/mmdeploy/mmcv/ops/nms.py:177: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  out_boxes = min(num_boxes, after_topk)
/home/Data/mmlab/mmdeploy-0.1.0/mmdeploy/mmcv/ops/nms.py:181: TracerWarning: Converting a tensor to a Python integer might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  (batch_size, out_boxes)).to(scores.device)
/home/etop/miniconda3/envs/mmdeploy/lib/python3.7/site-packages/torch/onnx/symbolic_opset9.py:2624: UserWarning: Exporting aten::index operator of advanced indexing in opset 11 is achieved by combination of multiple ONNX operators, including Reshape, Transpose, Concat, and Gather. If indices include negative values, the exported graph will produce incorrect results.
  "If indices include negative values, the exported graph will produce incorrect results.")
2022-01-19 18:48:35,544 - mmdeploy - INFO - torch2onnx success.
2022-01-19 18:48:35,607 - mmdeploy - INFO - onnx2tensorrt of work_dir/end2end.onnx start.
2022-01-19 18:48:36,377 - mmdeploy - WARNING - Could not load the library of tensorrt plugins.             Because the file does not exist: 
[TensorRT] WARNING: onnx2trt_utils.cpp:220: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[TensorRT] WARNING: onnx2trt_utils.cpp:246: One or more weights outside the range of INT32 was clamped
[TensorRT] WARNING: onnx2trt_utils.cpp:246: One or more weights outside the range of INT32 was clamped
[TensorRT] WARNING: onnx2trt_utils.cpp:246: One or more weights outside the range of INT32 was clamped
[TensorRT] WARNING: onnx2trt_utils.cpp:246: One or more weights outside the range of INT32 was clamped
[TensorRT] WARNING: onnx2trt_utils.cpp:246: One or more weights outside the range of INT32 was clamped
[TensorRT] WARNING: onnx2trt_utils.cpp:246: One or more weights outside the range of INT32 was clamped
[TensorRT] WARNING: onnx2trt_utils.cpp:246: One or more weights outside the range of INT32 was clamped
[TensorRT] INFO: ModelImporter.cpp:135: No importer registered for op: TRTBatchedNMS. Attempting to import as plugin.
[TensorRT] INFO: builtin_op_importers.cpp:3770: Searching for plugin: TRTBatchedNMS, plugin_version: 1, plugin_namespace: 
[TensorRT] ERROR: INVALID_ARGUMENT: getPluginCreator could not find plugin TRTBatchedNMS version 1
2022-01-19:18:48:37,root ERROR    [utils.py:41] Failed to parse onnx, In node -1 (importFallbackPluginImporter): UNSUPPORTED_NODE: Assertion failed: creator && "Plugin not found, are the plugin name, version, and namespace correct?"

Traceback (most recent call last):
  File "/home/Data/mmlab/mmdeploy-0.1.0/mmdeploy/utils/utils.py", line 36, in target_wrapper
    result = target(*args, **kwargs)
  File "/home/Data/mmlab/mmdeploy-0.1.0/mmdeploy/backend/tensorrt/onnx2tensorrt.py", line 72, in onnx2tensorrt
    device_id=device_id)
  File "/home/Data/mmlab/mmdeploy-0.1.0/mmdeploy/backend/tensorrt/utils.py", line 76, in create_trt_engine
    raise RuntimeError(f'Failed to parse onnx, {error_msgs}')
RuntimeError: Failed to parse onnx, In node -1 (importFallbackPluginImporter): UNSUPPORTED_NODE: Assertion failed: creator && "Plugin not found, are the plugin name, version, and namespace correct?"

2022-01-19 18:48:37,375 - mmdeploy - ERROR - onnx2tensorrt of work_dir/end2end.onnx failed.

this is my mmdet config , I modified the backbone of maskrcnn to resnet18, Will this cause this problem?

model = dict(
    type='MaskRCNN',
    backbone=dict(
        type='ResNet',
        depth=18,
        num_stages=4,
        out_indices=(0, 1, 2, 3),
        frozen_stages=1,
        norm_cfg=dict(type='BN', requires_grad=True),
        norm_eval=True,
        style='pytorch',
        init_cfg=dict(type='Pretrained', checkpoint='torchvision://resnet18')),
    neck=dict(
        type='FPN',
        in_channels=[64, 128, 256, 512],
        out_channels=256,
        num_outs=5),
    rpn_head=dict(
        type='RPNHead',
        in_channels=256,
        feat_channels=256,
        anchor_generator=dict(
            type='AnchorGenerator',
            scales=[2],
            ratios=[0.17, 0.44, 1.13, 2.9, 7.46],
            strides=[4, 8, 16, 32, 64]),
        bbox_coder=dict(
            type='DeltaXYWHBBoxCoder',
            target_means=[0.0, 0.0, 0.0, 0.0],
            target_stds=[1.0, 1.0, 1.0, 1.0]),
        loss_cls=dict(
            type='CrossEntropyLoss', use_sigmoid=True, loss_weight=1.0),
        loss_bbox=dict(type='L1Loss', loss_weight=1.0)),
    roi_head=dict(
        type='StandardRoIHead',
        bbox_roi_extractor=dict(
            type='SingleRoIExtractor',
            roi_layer=dict(type='RoIAlign', output_size=7, sampling_ratio=0),
            out_channels=256,
            featmap_strides=[4, 8, 16, 32]),
        bbox_head=dict(
            type='Shared2FCBBoxHead',
            in_channels=256,
            fc_out_channels=1024,
            roi_feat_size=7,
            num_classes=1,
            bbox_coder=dict(
                type='DeltaXYWHBBoxCoder',
                target_means=[0.0, 0.0, 0.0, 0.0],
                target_stds=[0.1, 0.1, 0.2, 0.2]),
            reg_class_agnostic=False,
            loss_cls=dict(
                type='CrossEntropyLoss', use_sigmoid=False, loss_weight=1.0),
            loss_bbox=dict(type='L1Loss', loss_weight=1.0)),
        mask_roi_extractor=dict(
            type='SingleRoIExtractor',
            roi_layer=dict(type='RoIAlign', output_size=14, sampling_ratio=0),
            out_channels=256,
            featmap_strides=[4, 8, 16, 32]),
        mask_head=dict(
            type='FCNMaskHead',
            num_convs=4,
            in_channels=256,
            conv_out_channels=256,
            num_classes=1,
            loss_mask=dict(
                type='CrossEntropyLoss', use_mask=True, loss_weight=1.0))),
    train_cfg=dict(
        rpn=dict(
            assigner=dict(
                type='MaxIoUAssigner',
                pos_iou_thr=0.7,
                neg_iou_thr=0.3,
                min_pos_iou=0.3,
                match_low_quality=True,
                ignore_iof_thr=-1,
                gpu_assign_thr=50),
            sampler=dict(
                type='RandomSampler',
                num=256,
                pos_fraction=0.5,
                neg_pos_ub=-1,
                add_gt_as_proposals=False),
            allowed_border=-1,
            pos_weight=-1,
            debug=False),
        rpn_proposal=dict(
            nms_across_levels=False,
            nms_pre=2000,
            nms_post=1000,
            max_per_img=1000,
            nms=dict(type='nms', iou_threshold=0.7),
            min_bbox_size=0),
        rcnn=dict(
            assigner=dict(
                type='MaxIoUAssigner',
                pos_iou_thr=0.5,
                neg_iou_thr=0.5,
                min_pos_iou=0.5,
                match_low_quality=True,
                ignore_iof_thr=-1),
            sampler=dict(
                type='RandomSampler',
                num=512,
                pos_fraction=0.25,
                neg_pos_ub=-1,
                add_gt_as_proposals=True),
            mask_size=28,
            pos_weight=-1,
            debug=False)),
    test_cfg=dict(
        rpn=dict(
            nms_across_levels=False,
            nms_pre=1000,
            nms_post=1000,
            max_per_img=1000,
            nms=dict(type='nms', iou_threshold=0.7),
            min_bbox_size=0),
        rcnn=dict(
            score_thr=0.05,
            nms=dict(type='nms', iou_threshold=0.5),
            max_per_img=100,
            mask_thr_binary=0.5)))

dataset_type = 'CocoDataset'
data_root = '/home/allen/2T/TextDet_data/coco_20220112/'

img_norm_cfg = dict(
    mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_rgb=True)

train_pipeline = [
    dict(type='LoadImageFromFile', color_type='color_ignore_orientation'),
    dict(type='LoadAnnotations', with_bbox=True, with_mask=True),
    dict(type='Resize', img_scale=(1024, 1024), keep_ratio=True),
    dict(type='RandomFlip', flip_ratio=0.5),
    dict(
        type='Normalize',
        mean=[123.675, 116.28, 103.53],
        std=[58.395, 57.12, 57.375],
        to_rgb=True),
    dict(type='Pad', size_divisor=32),
    dict(type='DefaultFormatBundle'),
    dict(type='Collect', keys=['img', 'gt_bboxes', 'gt_labels', 'gt_masks'])
]

test_pipeline = [
    dict(type='LoadImageFromFile'),
    dict(
        type='MultiScaleFlipAug',
        img_scale=(1024, 1024),
        flip=False,
        transforms=[
            dict(type='Resize', keep_ratio=True),
            dict(type='RandomFlip'),
            dict(
                type='Normalize',
                mean=[123.675, 116.28, 103.53],
                std=[58.395, 57.12, 57.375],
                to_rgb=True),
            dict(type='ImageToTensor', keys=['img']),
            dict(type='Collect', keys=['img'])
        ])
]

data = dict(
    samples_per_gpu=4,
    workers_per_gpu=4,
    val_dataloader=dict(samples_per_gpu=1),
    test_dataloader=dict(samples_per_gpu=1),
    train=dict(
        type='CocoDataset',
        ann_file=data_root + 'annotations/instances_train2017.json',
        img_prefix=data_root + 'train2017',
        pipeline=[
            dict(
                type='LoadImageFromFile',
                color_type='color_ignore_orientation'),
            dict(type='LoadAnnotations', with_bbox=True, with_mask=True),
            dict(type='Resize', img_scale=(1024, 1024), keep_ratio=True),
            dict(type='RandomFlip', flip_ratio=0.5),
            dict(
                type='Normalize',
                mean=[123.675, 116.28, 103.53],
                std=[58.395, 57.12, 57.375],
                to_rgb=True),
            dict(type='Pad', size_divisor=32),
            dict(type='DefaultFormatBundle'),
            dict(
                type='Collect',
                keys=['img', 'gt_bboxes', 'gt_labels', 'gt_masks'])
        ]),
    val=dict(
        type='CocoDataset',
        ann_file=data_root + 'annotations/instances_val2017.json',
        img_prefix=data_root + 'val2017',
        pipeline=[
            dict(type='LoadImageFromFile'),
            dict(
                type='MultiScaleFlipAug',
                img_scale=(1024, 1024),
                flip=False,
                transforms=[
                    dict(type='Resize', keep_ratio=True),
                    dict(type='RandomFlip'),
                    dict(
                        type='Normalize',
                        mean=[123.675, 116.28, 103.53],
                        std=[58.395, 57.12, 57.375],
                        to_rgb=True),
                    dict(type='ImageToTensor', keys=['img']),
                    dict(type='Collect', keys=['img'])
                ])
        ]),
    test=dict(
        type='CocoDataset',
        ann_file=data_root + 'annotations/instances_val2017.json',
        img_prefix=data_root + 'val2017',
        pipeline=[
            dict(type='LoadImageFromFile'),
            dict(
                type='MultiScaleFlipAug',
                img_scale=(1024, 1024),
                flip=False,
                transforms=[
                    dict(type='Resize', keep_ratio=True),
                    dict(type='RandomFlip'),
                    dict(
                        type='Normalize',
                        mean=[123.675, 116.28, 103.53],
                        std=[58.395, 57.12, 57.375],
                        to_rgb=True),
                    dict(type='ImageToTensor', keys=['img']),
                    dict(type='Collect', keys=['img'])
                ])
        ]))
evaluation = dict(interval=2, metric=['bbox', 'segm'])
optimizer = dict(type='SGD', lr=0.01, momentum=0.9, weight_decay=0.0001)
optimizer_config = dict(grad_clip=None)
lr_config = dict(
    policy='step',
    warmup='linear',
    warmup_iters=500,
    warmup_ratio=0.001,
    step=[80, 128])
runner = dict(type='EpochBasedRunner', max_epochs=160)
checkpoint_config = dict(interval=2)
log_config = dict(interval=5, hooks=[dict(type='TextLoggerHook')])
dist_params = dict(backend='nccl')
log_level = 'INFO'
load_from = None
resume_from = None
workflow = [('train', 1)]
work_dir = 'work_dirs/mrcnn_r18_fpn_160e_0114'

[AllentDan]

hi, please build the tensorrt plugins first following our document.

[mortal-Zero]

hi, please build the tensorrt plugins first following our document.

How can I verify that my tensorrt environment is compliant with mmdeploy?
I can run tensorrt sampleminist.

[AllentDan]

Make sure build/lib/libmmdeploy_tensorrt_ops.so can be found after building MMDeploy following instruction. Basically, MMDeploy is compatible with all tensorrt 7+

[mortal-Zero]

Make sure build/lib/libmmdeploy_tensorrt_ops.so can be found after building MMDeploy following instruction. Basically, MMDeploy is compatible with all tensorrt 7+

Sorry to bother you, I am rebuilding the mmdeploy environment, but I am getting the error when building mmdeploy.

commond

mkdir -p build
cd build
cmake -DMMDEPLOY_TARGET_BACKENDS=trt ..
make -j$(nproc)

when run mask -j$(nproc), I got this error:

[  8%] Building CUDA object csrc/backend_ops/tensorrt/CMakeFiles/mmdeploy_tensorrt_ops_obj.dir/batched_nms/sortScoresPerClass.cu.o
[  8%] Building CUDA object csrc/backend_ops/tensorrt/CMakeFiles/mmdeploy_tensorrt_ops_obj.dir/batched_nms/sortScoresPerImage.cu.o
[ 13%] Building CUDA object csrc/backend_ops/tensorrt/CMakeFiles/mmdeploy_tensorrt_ops_obj.dir/batched_nms/kernel.cu.o
/home/Data/mmlab/mmdeploy-0.1.0/csrc/backend_ops/tensorrt/batched_nms/sortScoresPerClass.cu:6:10: fatal error: cub/cub.cuh: 没有那个文件或目录
 #include "cub/cub.cuh"
          ^~~~~~~~~~~~~
compilation terminated.
csrc/backend_ops/tensorrt/CMakeFiles/mmdeploy_tensorrt_ops_obj.dir/build.make:133: recipe for target 'csrc/backend_ops/tensorrt/CMakeFiles/mmdeploy_tensorrt_ops_obj.dir/batched_nms/sortScoresPerClass.cu.o' failed
make[2]: *** [csrc/backend_ops/tensorrt/CMakeFiles/mmdeploy_tensorrt_ops_obj.dir/batched_nms/sortScoresPerClass.cu.o] Error 1
make[2]: *** 正在等待未完成的任务....
/home/Data/mmlab/mmdeploy-0.1.0/csrc/backend_ops/tensorrt/batched_nms/kernel.cu:6:10: fatal error: cub/cub.cuh: 没有那个文件或目录
 #include <cub/cub.cuh>
          ^~~~~~~~~~~~~
compilation terminated.
csrc/backend_ops/tensorrt/CMakeFiles/mmdeploy_tensorrt_ops_obj.dir/build.make:107: recipe for target 'csrc/backend_ops/tensorrt/CMakeFiles/mmdeploy_tensorrt_ops_obj.dir/batched_nms/kernel.cu.o' failed
make[2]: *** [csrc/backend_ops/tensorrt/CMakeFiles/mmdeploy_tensorrt_ops_obj.dir/batched_nms/kernel.cu.o] Error 1
/home/Data/mmlab/mmdeploy-0.1.0/csrc/backend_ops/tensorrt/batched_nms/sortScoresPerImage.cu:6:10: fatal error: cub/cub.cuh: 没有那个文件或目录
 #include "cub/cub.cuh"
          ^~~~~~~~~~~~~
compilation terminated.
csrc/backend_ops/tensorrt/CMakeFiles/mmdeploy_tensorrt_ops_obj.dir/build.make:146: recipe for target 'csrc/backend_ops/tensorrt/CMakeFiles/mmdeploy_tensorrt_ops_obj.dir/batched_nms/sortScoresPerImage.cu.o' failed
make[2]: *** [csrc/backend_ops/tensorrt/CMakeFiles/mmdeploy_tensorrt_ops_obj.dir/batched_nms/sortScoresPerImage.cu.o] Error 1
CMakeFiles/Makefile2:150: recipe for target 'csrc/backend_ops/tensorrt/CMakeFiles/mmdeploy_tensorrt_ops_obj.dir/all' failed
make[1]: *** [csrc/backend_ops/tensorrt/CMakeFiles/mmdeploy_tensorrt_ops_obj.dir/all] Error 2
Makefile:148: recipe for target 'all' failed
make: *** [all] Error 2

[AllentDan]

Need to clone the third-party library. Follow build. Well, reading the readme of this repository first is recommended.

[mortal-Zero]

Need to clone the third-party library. Follow build. Well, reading the readme of this repository first is recommended.

Thank you for your reply, and trouble you again.
Also during the compilation process, I have placed the cub package and the pybind11 package, but the following error is reported during the make process

when run mask -j$(nproc),

[  8%] Building CUDA object csrc/backend_ops/tensorrt/CMakeFiles/mmdeploy_tensorrt_ops_obj.dir/batched_nms/sortScoresPerImage.cu.o
[ 13%] Building CXX object csrc/backend_ops/tensorrt/CMakeFiles/mmdeploy_tensorrt_ops_obj.dir/roi_align/trt_roi_align.cpp.o
[ 17%] Building CUDA object csrc/backend_ops/tensorrt/CMakeFiles/mmdeploy_tensorrt_ops_obj.dir/batched_nms/sortScoresPerClass.cu.o
[ 21%] Building CUDA object csrc/backend_ops/tensorrt/CMakeFiles/mmdeploy_tensorrt_ops_obj.dir/roi_align/trt_roi_align_kernel.cu.o
[  8%] Building CUDA object csrc/backend_ops/tensorrt/CMakeFiles/mmdeploy_tensorrt_ops_obj.dir/batched_nms/kernel.cu.o
[ 26%] Building CUDA object csrc/backend_ops/tensorrt/CMakeFiles/mmdeploy_tensorrt_ops_obj.dir/scatternd/trt_scatternd_kernel.cu.o
[ 30%] Building CUDA object csrc/backend_ops/tensorrt/CMakeFiles/mmdeploy_tensorrt_ops_obj.dir/multi_level_roi_align/trt_multi_level_roi_align_kernel.cu.o
[ 34%] Building CXX object csrc/backend_ops/tensorrt/CMakeFiles/mmdeploy_tensorrt_ops_obj.dir/scatternd/trt_scatternd.cpp.o
[ 39%] Building CXX object csrc/backend_ops/tensorrt/CMakeFiles/mmdeploy_tensorrt_ops_obj.dir/multi_level_roi_align/trt_multi_level_roi_align.cpp.o
/home/Data/mmlab/MMDeploy/third_party/cub/cub/agent/agent_merge_sort.cuh(80): error: a class or namespace qualified name is required

/home/Data/mmlab/MMDeploy/third_party/cub/cub/agent/agent_merge_sort.cuh(80): error: expected a ";"

/home/Data/mmlab/MMDeploy/third_party/cub/cub/agent/agent_merge_sort.cuh(81): error: a class or namespace qualified name is required

/home/Data/mmlab/MMDeploy/third_party/cub/cub/agent/agent_merge_sort.cuh(81): error: expected a ";"

/home/Data/mmlab/MMDeploy/third_party/cub/cub/block/specializations/../../block/../util_type.cuh(78): error: class "std::iterator_traits<<error-type>>" has no member "value_type"
          detected during:
            instantiation of type "cub::detail::value_t<<error-type>>" 
/home/Data/mmlab/MMDeploy/third_party/cub/cub/block/block_load.cuh(1295): here
            processing of template argument list for "cub::BlockLoadType" based on template arguments <Policy, <error-type>> 
/home/Data/mmlab/MMDeploy/third_party/cub/cub/agent/agent_merge_sort.cuh(83): here

/home/Data/mmlab/MMDeploy/third_party/cub/cub/agent/agent_merge_sort.cuh(387): error: a class or namespace qualified name is required

/home/Data/mmlab/MMDeploy/third_party/cub/cub/agent/agent_merge_sort.cuh(387): error: expected a ";"

/home/Data/mmlab/MMDeploy/third_party/cub/cub/agent/agent_merge_sort.cuh(388): error: a class or namespace qualified name is required

/home/Data/mmlab/MMDeploy/third_party/cub/cub/agent/agent_merge_sort.cuh(388): error: expected a ";"

/home/Data/mmlab/MMDeploy/third_party/cub/cub/agent/agent_merge_sort.cuh(389): error: a class or namespace qualified name is required

/home/Data/mmlab/MMDeploy/third_party/cub/cub/agent/agent_merge_sort.cuh(389): error: expected a ";"

/home/Data/mmlab/MMDeploy/third_party/cub/cub/agent/agent_merge_sort.cuh(390): error: a class or namespace qualified name is required

/home/Data/mmlab/MMDeploy/third_party/cub/cub/agent/agent_merge_sort.cuh(390): error: expected a ";"

/home/Data/mmlab/MMDeploy/third_party/cub/cub/device/../iterator/arg_index_input_iterator.cuh(122): error: a class or namespace qualified name is required

/home/Data/mmlab/MMDeploy/third_party/cub/cub/device/../iterator/arg_index_input_iterator.cuh(122): error: global-scope qualifier (leading "::") is not allowed

/home/Data/mmlab/MMDeploy/third_party/cub/cub/device/../iterator/arg_index_input_iterator.cuh(122): error: expected a ";"

/home/Data/mmlab/MMDeploy/third_party/cub/cub/device/../iterator/arg_index_input_iterator.cuh(127): warning: parsing restarts here after previous syntax error

/home/Data/mmlab/MMDeploy/third_party/cub/cub/device/dispatch/../../agent/../iterator/constant_input_iterator.cuh(105): error: a class or namespace qualified name is required

/home/Data/mmlab/MMDeploy/third_party/cub/cub/device/dispatch/../../agent/../iterator/constant_input_iterator.cuh(105): error: global-scope qualifier (leading "::") is not allowed

/home/Data/mmlab/MMDeploy/third_party/cub/cub/device/dispatch/../../agent/../iterator/constant_input_iterator.cuh(105): error: expected a ";"

/home/Data/mmlab/MMDeploy/third_party/cub/cub/device/dispatch/../../agent/../iterator/constant_input_iterator.cuh(110): warning: parsing restarts here after previous syntax error

/home/Data/mmlab/MMDeploy/third_party/cub/cub/agent/agent_sub_warp_merge_sort.cuh(149): error: a class or namespace qualified name is required

/home/Data/mmlab/MMDeploy/third_party/cub/cub/agent/agent_sub_warp_merge_sort.cuh(149): error: expected a ";"

/home/Data/mmlab/MMDeploy/third_party/cub/cub/agent/agent_sub_warp_merge_sort.cuh(151): error: a class or namespace qualified name is required

/home/Data/mmlab/MMDeploy/third_party/cub/cub/agent/agent_sub_warp_merge_sort.cuh(151): error: expected a ";"

/home/Data/mmlab/MMDeploy/third_party/cub/cub/agent/agent_merge_sort.cuh(80): error: a class or namespace qualified name is required

/home/Data/mmlab/MMDeploy/third_party/cub/cub/agent/agent_merge_sort.cuh(80): error: expected a ";"

/home/Data/mmlab/MMDeploy/third_party/cub/cub/agent/agent_merge_sort.cuh(81): error: a class or namespace qualified name is required

/home/Data/mmlab/MMDeploy/third_party/cub/cub/agent/agent_merge_sort.cuh(81): error: expected a ";"

/home/Data/mmlab/MMDeploy/third_party/cub/cub/block/specializations/../../block/../util_type.cuh(78): error: class "std::iterator_traits<<error-type>>" has no member "value_type"
          detected during:
            instantiation of type "cub::detail::value_t<<error-type>>" 
/home/Data/mmlab/MMDeploy/third_party/cub/cub/block/block_load.cuh(1295): here
            processing of template argument list for "cub::BlockLoadType" based on template arguments <Policy, <error-type>> 
/home/Data/mmlab/MMDeploy/third_party/cub/cub/agent/agent_merge_sort.cuh(83): here

/home/Data/mmlab/MMDeploy/third_party/cub/cub/agent/agent_merge_sort.cuh(387): error: a class or namespace qualified name is required

/home/Data/mmlab/MMDeploy/third_party/cub/cub/agent/agent_merge_sort.cuh(387): error: expected a ";"

/home/Data/mmlab/MMDeploy/third_party/cub/cub/agent/agent_merge_sort.cuh(388): error: a class or namespace qualified name is required

/home/Data/mmlab/MMDeploy/third_party/cub/cub/agent/agent_merge_sort.cuh(388): error: expected a ";"

/home/Data/mmlab/MMDeploy/third_party/cub/cub/agent/agent_merge_sort.cuh(389): error: a class or namespace qualified name is required

/home/Data/mmlab/MMDeploy/third_party/cub/cub/agent/agent_merge_sort.cuh(389): error: expected a ";"

/home/Data/mmlab/MMDeploy/third_party/cub/cub/agent/agent_merge_sort.cuh(390): error: a class or namespace qualified name is required

/home/Data/mmlab/MMDeploy/third_party/cub/cub/agent/agent_merge_sort.cuh(390): error: expected a ";"

/home/Data/mmlab/MMDeploy/third_party/cub/cub/device/dispatch/../../agent/../iterator/counting_input_iterator.cuh(103): error: a class or namespace qualified name is required

/home/Data/mmlab/MMDeploy/third_party/cub/cub/device/dispatch/../../agent/../iterator/counting_input_iterator.cuh(103): error: global-scope qualifier (leading "::") is not allowed

/home/Data/mmlab/MMDeploy/third_party/cub/cub/device/dispatch/../../agent/../iterator/counting_input_iterator.cuh(103): error: expected a ";"

/home/Data/mmlab/MMDeploy/third_party/cub/cub/device/dispatch/../../agent/../iterator/counting_input_iterator.cuh(108): warning: parsing restarts here after previous syntax error

/home/Data/mmlab/MMDeploy/third_party/cub/cub/device/dispatch/../../agent/agent_adjacent_difference.cuh(71): error: a class or namespace qualified name is required

/home/Data/mmlab/MMDeploy/third_party/cub/cub/device/dispatch/../../agent/agent_adjacent_difference.cuh(71): error: expected a ";"

/home/Data/mmlab/MMDeploy/third_party/cub/cub/iterator/cache_modified_output_iterator.cuh(136): error: a class or namespace qualified name is required

/home/Data/mmlab/MMDeploy/third_party/cub/cub/iterator/cache_modified_output_iterator.cuh(136): error: global-scope qualifier (leading "::") is not allowed

/home/Data/mmlab/MMDeploy/third_party/cub/cub/iterator/cache_modified_output_iterator.cuh(136): error: expected a ";"

/home/Data/mmlab/MMDeploy/third_party/cub/cub/iterator/cache_modified_output_iterator.cuh(141): warning: parsing restarts here after previous syntax error

/home/Data/mmlab/MMDeploy/third_party/cub/cub/iterator/discard_output_iterator.cuh(74): error: a class or namespace qualified name is required

/home/Data/mmlab/MMDeploy/third_party/cub/cub/iterator/discard_output_iterator.cuh(74): error: global-scope qualifier (leading "::") is not allowed

/home/Data/mmlab/MMDeploy/third_party/cub/cub/iterator/discard_output_iterator.cuh(74): error: expected a ";"

/home/Data/mmlab/MMDeploy/third_party/cub/cub/iterator/discard_output_iterator.cuh(79): warning: parsing restarts here after previous syntax error

/home/Data/mmlab/MMDeploy/third_party/cub/cub/iterator/tex_obj_input_iterator.cuh(120): error: a class or namespace qualified name is required

/home/Data/mmlab/MMDeploy/third_party/cub/cub/iterator/tex_obj_input_iterator.cuh(120): error: global-scope qualifier (leading "::") is not allowed

/home/Data/mmlab/MMDeploy/third_party/cub/cub/iterator/tex_obj_input_iterator.cuh(120): error: expected a ";"

/home/Data/mmlab/MMDeploy/third_party/cub/cub/iterator/tex_obj_input_iterator.cuh(125): warning: parsing restarts here after previous syntax error

/home/Data/mmlab/MMDeploy/third_party/cub/cub/iterator/tex_ref_input_iterator.cuh(261): error: a class or namespace qualified name is required

/home/Data/mmlab/MMDeploy/third_party/cub/cub/iterator/tex_ref_input_iterator.cuh(261): error: global-scope qualifier (leading "::") is not allowed

/home/Data/mmlab/MMDeploy/third_party/cub/cub/iterator/tex_ref_input_iterator.cuh(261): error: expected a ";"

/home/Data/mmlab/MMDeploy/third_party/cub/cub/iterator/tex_ref_input_iterator.cuh(266): warning: parsing restarts here after previous syntax error

/home/Data/mmlab/MMDeploy/third_party/cub/cub/iterator/transform_input_iterator.cuh(126): error: a class or namespace qualified name is required

/home/Data/mmlab/MMDeploy/third_party/cub/cub/iterator/transform_input_iterator.cuh(126): error: global-scope qualifier (leading "::") is not allowed

/home/Data/mmlab/MMDeploy/third_party/cub/cub/iterator/transform_input_iterator.cuh(126): error: expected a ";"

/home/Data/mmlab/MMDeploy/third_party/cub/cub/iterator/transform_input_iterator.cuh(131): warning: parsing restarts here after previous syntax error

/home/Data/mmlab/MMDeploy/third_party/cub/cub/device/../iterator/arg_index_input_iterator.cuh(122): error: a class or namespace qualified name is required

/home/Data/mmlab/MMDeploy/third_party/cub/cub/device/../iterator/arg_index_input_iterator.cuh(122): error: global-scope qualifier (leading "::") is not allowed

/home/Data/mmlab/MMDeploy/third_party/cub/cub/device/../iterator/arg_index_input_iterator.cuh(122): error: expected a ";"

/home/Data/mmlab/MMDeploy/third_party/cub/cub/device/../iterator/arg_index_input_iterator.cuh(127): warning: parsing restarts here after previous syntax error

/home/Data/mmlab/MMDeploy/third_party/cub/cub/device/dispatch/../../agent/../iterator/constant_input_iterator.cuh(105): error: a class or namespace qualified name is required

/home/Data/mmlab/MMDeploy/third_party/cub/cub/device/dispatch/../../agent/../iterator/constant_input_iterator.cuh(105): error: global-scope qualifier (leading "::") is not allowed

/home/Data/mmlab/MMDeploy/third_party/cub/cub/device/dispatch/../../agent/../iterator/constant_input_iterator.cuh(105): error: expected a ";"

/home/Data/mmlab/MMDeploy/third_party/cub/cub/device/dispatch/../../agent/../iterator/constant_input_iterator.cuh(110): warning: parsing restarts here after previous syntax error

/home/Data/mmlab/MMDeploy/third_party/cub/cub/agent/agent_sub_warp_merge_sort.cuh(149): error: a class or namespace qualified name is required

/home/Data/mmlab/MMDeploy/third_party/cub/cub/agent/agent_sub_warp_merge_sort.cuh(149): error: expected a ";"

/home/Data/mmlab/MMDeploy/third_party/cub/cub/agent/agent_sub_warp_merge_sort.cuh(151): error: a class or namespace qualified name is required

/home/Data/mmlab/MMDeploy/third_party/cub/cub/agent/agent_sub_warp_merge_sort.cuh(151): error: expected a ";"

/home/Data/mmlab/MMDeploy/third_party/cub/cub/device/dispatch/../../agent/../iterator/counting_input_iterator.cuh(103): error: a class or namespace qualified name is required

/home/Data/mmlab/MMDeploy/third_party/cub/cub/device/dispatch/../../agent/../iterator/counting_input_iterator.cuh(103): error: global-scope qualifier (leading "::") is not allowed

/home/Data/mmlab/MMDeploy/third_party/cub/cub/device/dispatch/../../agent/../iterator/counting_input_iterator.cuh(103): error: expected a ";"

/home/Data/mmlab/MMDeploy/third_party/cub/cub/device/dispatch/../../agent/../iterator/counting_input_iterator.cuh(108): warning: parsing restarts here after previous syntax error

/home/Data/mmlab/MMDeploy/third_party/cub/cub/device/dispatch/../../agent/agent_adjacent_difference.cuh(71): error: a class or namespace qualified name is required

/home/Data/mmlab/MMDeploy/third_party/cub/cub/device/dispatch/../../agent/agent_adjacent_difference.cuh(71): error: expected a ";"

/home/Data/mmlab/MMDeploy/third_party/cub/cub/iterator/cache_modified_output_iterator.cuh(136): error: a class or namespace qualified name is required

/home/Data/mmlab/MMDeploy/third_party/cub/cub/iterator/cache_modified_output_iterator.cuh(136): error: global-scope qualifier (leading "::") is not allowed

/home/Data/mmlab/MMDeploy/third_party/cub/cub/iterator/cache_modified_output_iterator.cuh(136): error: expected a ";"

/home/Data/mmlab/MMDeploy/third_party/cub/cub/iterator/cache_modified_output_iterator.cuh(141): warning: parsing restarts here after previous syntax error

/home/Data/mmlab/MMDeploy/third_party/cub/cub/iterator/discard_output_iterator.cuh(74): error: a class or namespace qualified name is required

/home/Data/mmlab/MMDeploy/third_party/cub/cub/iterator/discard_output_iterator.cuh(74): error: global-scope qualifier (leading "::") is not allowed

/home/Data/mmlab/MMDeploy/third_party/cub/cub/iterator/discard_output_iterator.cuh(74): error: expected a ";"

/home/Data/mmlab/MMDeploy/third_party/cub/cub/iterator/discard_output_iterator.cuh(79): warning: parsing restarts here after previous syntax error

/home/Data/mmlab/MMDeploy/third_party/cub/cub/iterator/tex_obj_input_iterator.cuh(120): error: a class or namespace qualified name is required

/home/Data/mmlab/MMDeploy/third_party/cub/cub/iterator/tex_obj_input_iterator.cuh(120): error: global-scope qualifier (leading "::") is not allowed

/home/Data/mmlab/MMDeploy/third_party/cub/cub/iterator/tex_obj_input_iterator.cuh(120): error: expected a ";"

/home/Data/mmlab/MMDeploy/third_party/cub/cub/iterator/tex_obj_input_iterator.cuh(125): warning: parsing restarts here after previous syntax error

/home/Data/mmlab/MMDeploy/third_party/cub/cub/iterator/tex_ref_input_iterator.cuh(261): error: a class or namespace qualified name is required

/home/Data/mmlab/MMDeploy/third_party/cub/cub/iterator/tex_ref_input_iterator.cuh(261): error: global-scope qualifier (leading "::") is not allowed

/home/Data/mmlab/MMDeploy/third_party/cub/cub/iterator/tex_ref_input_iterator.cuh(261): error: expected a ";"

/home/Data/mmlab/MMDeploy/third_party/cub/cub/iterator/tex_ref_input_iterator.cuh(266): warning: parsing restarts here after previous syntax error

/home/Data/mmlab/MMDeploy/third_party/cub/cub/iterator/transform_input_iterator.cuh(126): error: a class or namespace qualified name is required

/home/Data/mmlab/MMDeploy/third_party/cub/cub/iterator/transform_input_iterator.cuh(126): error: global-scope qualifier (leading "::") is not allowed

/home/Data/mmlab/MMDeploy/third_party/cub/cub/iterator/transform_input_iterator.cuh(126): error: expected a ";"

/home/Data/mmlab/MMDeploy/third_party/cub/cub/iterator/transform_input_iterator.cuh(131): warning: parsing restarts here after previous syntax error

/home/Data/mmlab/MMDeploy/third_party/cub/cub/agent/agent_merge_sort.cuh(80): error: a class or namespace qualified name is required

/home/Data/mmlab/MMDeploy/third_party/cub/cub/agent/agent_merge_sort.cuh(80): error: expected a ";"

/home/Data/mmlab/MMDeploy/third_party/cub/cub/agent/agent_merge_sort.cuh(81): error: a class or namespace qualified name is required

/home/Data/mmlab/MMDeploy/third_party/cub/cub/agent/agent_merge_sort.cuh(81): error: expected a ";"

/home/Data/mmlab/MMDeploy/third_party/cub/cub/block/specializations/../../block/../util_type.cuh(78): error: class "std::iterator_traits<<error-type>>" has no member "value_type"
          detected during:
            instantiation of type "cub::detail::value_t<<error-type>>" 
/home/Data/mmlab/MMDeploy/third_party/cub/cub/block/block_load.cuh(1295): here
            processing of template argument list for "cub::BlockLoadType" based on template arguments <Policy, <error-type>> 
/home/Data/mmlab/MMDeploy/third_party/cub/cub/agent/agent_merge_sort.cuh(83): here

/home/Data/mmlab/MMDeploy/third_party/cub/cub/agent/agent_merge_sort.cuh(387): error: a class or namespace qualified name is required

/home/Data/mmlab/MMDeploy/third_party/cub/cub/agent/agent_merge_sort.cuh(387): error: expected a ";"

/home/Data/mmlab/MMDeploy/third_party/cub/cub/agent/agent_merge_sort.cuh(388): error: a class or namespace qualified name is required

/home/Data/mmlab/MMDeploy/third_party/cub/cub/agent/agent_merge_sort.cuh(388): error: expected a ";"

/home/Data/mmlab/MMDeploy/third_party/cub/cub/agent/agent_merge_sort.cuh(389): error: a class or namespace qualified name is required

/home/Data/mmlab/MMDeploy/third_party/cub/cub/agent/agent_merge_sort.cuh(389): error: expected a ";"

/home/Data/mmlab/MMDeploy/third_party/cub/cub/agent/agent_merge_sort.cuh(390): error: a class or namespace qualified name is required

/home/Data/mmlab/MMDeploy/third_party/cub/cub/agent/agent_merge_sort.cuh(390): error: expected a ";"

/home/Data/mmlab/MMDeploy/third_party/cub/cub/device/../iterator/arg_index_input_iterator.cuh(122): error: a class or namespace qualified name is required

/home/Data/mmlab/MMDeploy/third_party/cub/cub/device/../iterator/arg_index_input_iterator.cuh(122): error: global-scope qualifier (leading "::") is not allowed

/home/Data/mmlab/MMDeploy/third_party/cub/cub/device/../iterator/arg_index_input_iterator.cuh(122): error: expected a ";"

/home/Data/mmlab/MMDeploy/third_party/cub/cub/device/../iterator/arg_index_input_iterator.cuh(127): warning: parsing restarts here after previous syntax error

/home/Data/mmlab/MMDeploy/third_party/cub/cub/device/dispatch/../../agent/../iterator/constant_input_iterator.cuh(105): error: a class or namespace qualified name is required

/home/Data/mmlab/MMDeploy/third_party/cub/cub/device/dispatch/../../agent/../iterator/constant_input_iterator.cuh(105): error: global-scope qualifier (leading "::") is not allowed

/home/Data/mmlab/MMDeploy/third_party/cub/cub/device/dispatch/../../agent/../iterator/constant_input_iterator.cuh(105): error: expected a ";"

/home/Data/mmlab/MMDeploy/third_party/cub/cub/device/dispatch/../../agent/../iterator/constant_input_iterator.cuh(110): warning: parsing restarts here after previous syntax error

/home/Data/mmlab/MMDeploy/third_party/cub/cub/agent/agent_sub_warp_merge_sort.cuh(149): error: a class or namespace qualified name is required

/home/Data/mmlab/MMDeploy/third_party/cub/cub/agent/agent_sub_warp_merge_sort.cuh(149): error: expected a ";"

/home/Data/mmlab/MMDeploy/third_party/cub/cub/agent/agent_sub_warp_merge_sort.cuh(151): error: a class or namespace qualified name is required

/home/Data/mmlab/MMDeploy/third_party/cub/cub/agent/agent_sub_warp_merge_sort.cuh(151): error: expected a ";"

43 errors detected in the compilation of "/tmp/tmpxft_00001e38_00000000-11_kernel.compute_75.cpp1.ii".
csrc/backend_ops/tensorrt/CMakeFiles/mmdeploy_tensorrt_ops_obj.dir/build.make:107: recipe for target 'csrc/backend_ops/tensorrt/CMakeFiles/mmdeploy_tensorrt_ops_obj.dir/batched_nms/kernel.cu.o' failed
make[2]: *** [csrc/backend_ops/tensorrt/CMakeFiles/mmdeploy_tensorrt_ops_obj.dir/batched_nms/kernel.cu.o] Error 1
make[2]: *** 正在等待未完成的任务....
/home/Data/mmlab/MMDeploy/third_party/cub/cub/device/dispatch/../../agent/../iterator/counting_input_iterator.cuh(103): error: a class or namespace qualified name is required

/home/Data/mmlab/MMDeploy/third_party/cub/cub/device/dispatch/../../agent/../iterator/counting_input_iterator.cuh(103): error: global-scope qualifier (leading "::") is not allowed

/home/Data/mmlab/MMDeploy/third_party/cub/cub/device/dispatch/../../agent/../iterator/counting_input_iterator.cuh(103): error: expected a ";"

/home/Data/mmlab/MMDeploy/third_party/cub/cub/device/dispatch/../../agent/../iterator/counting_input_iterator.cuh(108): warning: parsing restarts here after previous syntax error

/home/Data/mmlab/MMDeploy/third_party/cub/cub/device/dispatch/../../agent/agent_adjacent_difference.cuh(71): error: a class or namespace qualified name is required

/home/Data/mmlab/MMDeploy/third_party/cub/cub/device/dispatch/../../agent/agent_adjacent_difference.cuh(71): error: expected a ";"

/home/Data/mmlab/MMDeploy/third_party/cub/cub/iterator/cache_modified_output_iterator.cuh(136): error: a class or namespace qualified name is required

/home/Data/mmlab/MMDeploy/third_party/cub/cub/iterator/cache_modified_output_iterator.cuh(136): error: global-scope qualifier (leading "::") is not allowed

/home/Data/mmlab/MMDeploy/third_party/cub/cub/iterator/cache_modified_output_iterator.cuh(136): error: expected a ";"

/home/Data/mmlab/MMDeploy/third_party/cub/cub/iterator/cache_modified_output_iterator.cuh(141): warning: parsing restarts here after previous syntax error

/home/Data/mmlab/MMDeploy/third_party/cub/cub/iterator/discard_output_iterator.cuh(74): error: a class or namespace qualified name is required

/home/Data/mmlab/MMDeploy/third_party/cub/cub/iterator/discard_output_iterator.cuh(74): error: global-scope qualifier (leading "::") is not allowed

/home/Data/mmlab/MMDeploy/third_party/cub/cub/iterator/discard_output_iterator.cuh(74): error: expected a ";"

/home/Data/mmlab/MMDeploy/third_party/cub/cub/iterator/discard_output_iterator.cuh(79): warning: parsing restarts here after previous syntax error

/home/Data/mmlab/MMDeploy/third_party/cub/cub/iterator/tex_obj_input_iterator.cuh(120): error: a class or namespace qualified name is required

/home/Data/mmlab/MMDeploy/third_party/cub/cub/iterator/tex_obj_input_iterator.cuh(120): error: global-scope qualifier (leading "::") is not allowed

/home/Data/mmlab/MMDeploy/third_party/cub/cub/iterator/tex_obj_input_iterator.cuh(120): error: expected a ";"

/home/Data/mmlab/MMDeploy/third_party/cub/cub/iterator/tex_obj_input_iterator.cuh(125): warning: parsing restarts here after previous syntax error

/home/Data/mmlab/MMDeploy/third_party/cub/cub/iterator/tex_ref_input_iterator.cuh(261): error: a class or namespace qualified name is required

/home/Data/mmlab/MMDeploy/third_party/cub/cub/iterator/tex_ref_input_iterator.cuh(261): error: global-scope qualifier (leading "::") is not allowed

/home/Data/mmlab/MMDeploy/third_party/cub/cub/iterator/tex_ref_input_iterator.cuh(261): error: expected a ";"

/home/Data/mmlab/MMDeploy/third_party/cub/cub/iterator/tex_ref_input_iterator.cuh(266): warning: parsing restarts here after previous syntax error

/home/Data/mmlab/MMDeploy/third_party/cub/cub/iterator/transform_input_iterator.cuh(126): error: a class or namespace qualified name is required

/home/Data/mmlab/MMDeploy/third_party/cub/cub/iterator/transform_input_iterator.cuh(126): error: global-scope qualifier (leading "::") is not allowed

/home/Data/mmlab/MMDeploy/third_party/cub/cub/iterator/transform_input_iterator.cuh(126): error: expected a ";"

/home/Data/mmlab/MMDeploy/third_party/cub/cub/iterator/transform_input_iterator.cuh(131): warning: parsing restarts here after previous syntax error

43 errors detected in the compilation of "/tmp/tmpxft_00001e34_00000000-11_sortScoresPerClass.compute_75.cpp1.ii".
csrc/backend_ops/tensorrt/CMakeFiles/mmdeploy_tensorrt_ops_obj.dir/build.make:133: recipe for target 'csrc/backend_ops/tensorrt/CMakeFiles/mmdeploy_tensorrt_ops_obj.dir/batched_nms/sortScoresPerClass.cu.o' failed
make[2]: *** [csrc/backend_ops/tensorrt/CMakeFiles/mmdeploy_tensorrt_ops_obj.dir/batched_nms/sortScoresPerClass.cu.o] Error 1
43 errors detected in the compilation of "/tmp/tmpxft_00001e32_00000000-11_sortScoresPerImage.compute_75.cpp1.ii".
csrc/backend_ops/tensorrt/CMakeFiles/mmdeploy_tensorrt_ops_obj.dir/build.make:146: recipe for target 'csrc/backend_ops/tensorrt/CMakeFiles/mmdeploy_tensorrt_ops_obj.dir/batched_nms/sortScoresPerImage.cu.o' failed
make[2]: *** [csrc/backend_ops/tensorrt/CMakeFiles/mmdeploy_tensorrt_ops_obj.dir/batched_nms/sortScoresPerImage.cu.o] Error 1
CMakeFiles/Makefile2:150: recipe for target 'csrc/backend_ops/tensorrt/CMakeFiles/mmdeploy_tensorrt_ops_obj.dir/all' failed
make[1]: *** [csrc/backend_ops/tensorrt/CMakeFiles/mmdeploy_tensorrt_ops_obj.dir/all] Error 2
Makefile:148: recipe for target 'all' failed
make: *** [all] Error 2

Is it the cub package or the pybind11 package that does not match the version of cuda10.2? If yes, which version of cub package should I download?