[Fix] concatdatasets for voc train (open-mmlab#228)

Nioolek · hanrui1sensetime · triple-Mu · hhaAndroid · commit 9c250571ca03 · 2022-11-03T19:03:06.000+08:00
* add yolov5 voc training * format code * [Feature] Support VOC Dataset in YOLOv5 (open-mmlab#134) * add yolov5 voc training * fix mosaic bug * fix mosaic bug and temp config * fix mosaic bug * update config * support training on voc dataset * format code * format code * Optimize Code. Change `RandomTransform` to `OneOf` * Change `OneOf` to `mmcv.RandomChoice` * fix yolov5coco dataset * fix yolov5coco dataset * fix bug, format code * format config * format code * add yolov5 voc training * rebase * fix mosaic bug * update config * support training on voc dataset * format code * format code * Optimize Code. Change `RandomTransform` to `OneOf` * Change `OneOf` to `mmcv.RandomChoice` * fix yolov5coco dataset * fix yolov5coco dataset * fix bug, format code * format code * add yolov5 voc training * fix mosaic bug and temp config * fix mosaic bug * update config * support training on voc dataset * format code * format code * Optimize Code. Change `RandomTransform` to `OneOf` * Change `OneOf` to `mmcv.RandomChoice` * fix yolov5coco dataset * fix yolov5coco dataset * fix bug, format code * format code * add yolov5 voc training * rebase * fix mosaic bug * update config * support training on voc dataset * format code * format code * Optimize Code. Change `RandomTransform` to `OneOf` * Change `OneOf` to `mmcv.RandomChoice` * fix yolov5coco dataset * fix yolov5coco dataset * fix bug, format code * format code * format code * fix lint * add unittest * add auto loss_weight * add doc; add model log url * add doc; add model log url * add doc; add model log url * [Feature] support mmyolo deployment (open-mmlab#79) * support mmyolo deployment * mv deploy place * remove unused configs * add deploy code * fix new register * fix comments * fix dependent codebase register * remove unused initialize * refact deploy config * credit return to triplemu * Add yolov5 head rewrite * refactor deploy * refactor deploy * Add yolov5 head rewrite * fix configs * refact config * fix comment * sync name after mmdeploy 1088 * fix mmyolo * fix yapf * fix deploy config * try to fix flake8 importlib-metadata * add mmyolo models ut * add deploy uts * add deploy uts * fix trt dynamic error * fix multi-batch for dynamic batch value * fix mode * fix lint * sync model.py * add ci for deploy test * fix ci * fix ci * fix ci * extract script to command for fixing CI * fix cmake for CI * sudo ln * move ort position * remove unused sdk compile * cd mmdeploy * simplify build * add missing make * change order * add -v * add setuptools * get locate * get locate * upgrade torch * change torchvision version * fix config * fix ci * fix ci * fix lint Co-authored-by: tripleMu <gpu@163.com> Co-authored-by: RunningLeon <mnsheng@yeah.net> * [Feature] Support YOLOv5 YOLOv6 YOLOX Deploy in mmdeploy (open-mmlab#199) * Support YOLOv5 YOLOv6 YOLOX Deploy in mmdeploy * Fix lint * Rename _class to detector_type * Add some common * fix lint Co-authored-by: huanghaian <huanghaian@sensetime.com> * fix vocdatasets * fix vocdatasets Co-authored-by: hanrui1sensetime <83800577+hanrui1sensetime@users.noreply.github.com> Co-authored-by: tripleMu <gpu@163.com> Co-authored-by: RunningLeon <mnsheng@yeah.net> Co-authored-by: huanghaian <huanghaian@sensetime.com>
diff --git a/.dev_scripts/gather_models.py b/.dev_scripts/gather_models.py
@@ -114,6 +114,7 @@ def get_dataset_name(config):
         LVISV05Dataset='LVIS v0.5',
         LVISV1Dataset='LVIS v1',
         VOCDataset='Pascal VOC',
+        YOLOv5VOCDataset='Pascal VOC',
         WIDERFaceDataset='WIDER Face',
         OpenImagesDataset='OpenImagesDataset',
         OpenImagesChallengeDataset='OpenImagesChallengeDataset')
diff --git a/configs/yolov5/voc/yolov5_s-v61_fast_1xb64-50e_voc.py b/configs/yolov5/voc/yolov5_s-v61_fast_1xb64-50e_voc.py
@@ -157,7 +157,10 @@
                 data_prefix=dict(sub_data_root='VOC2012/'),
                 filter_cfg=dict(filter_empty_gt=False, min_size=32),
                 pipeline=train_pipeline)
-        ]),
+        ],
+        # Use ignore_keys to avoid judging metainfo is
+        # not equal in `ConcatDataset`.
+        ignore_keys='DATASET_TYPE'),
     collate_fn=dict(type='yolov5_collate'))
 
 test_pipeline = [
diff --git a/mmyolo/datasets/yolov5_voc.py b/mmyolo/datasets/yolov5_voc.py
@@ -7,14 +7,9 @@
 
 @DATASETS.register_module()
 class YOLOv5VOCDataset(BatchShapePolicyDataset, VOCDataset):
+    """Dataset for YOLOv5 VOC Dataset.
 
-    def __init__(self, *args, **kwargs):
-        super().__init__(*args, **kwargs)
-        # TODO: Because of the bug in mmengine `ConcatDataset`
-        # add this to avoid ValueError
-        if 'VOC2007' in self.sub_data_root:
-            self._metainfo['DATASET_TYPE'] = 'VOC'
-        elif 'VOC2012' in self.sub_data_root:
-            self._metainfo['DATASET_TYPE'] = 'VOC'
-        else:
-            self._metainfo['DATASET_TYPE'] = None
+    We only add `BatchShapePolicy` function compared with VOCDataset. See
+    `mmyolo/datasets/utils.py#BatchShapePolicy` for details
+    """
+    pass
diff --git a/tests/test_datasets/test_yolov5_voc.py b/tests/test_datasets/test_yolov5_voc.py
@@ -63,21 +63,23 @@ def test_prepare_data(self):
             assert 'dataset' not in data
 
     def test_concat_dataset(self):
-        dataset = ConcatDataset(datasets=[
-            dict(
-                type='YOLOv5VOCDataset',
-                data_root='tests/data/VOCdevkit/',
-                ann_file='VOC2007/ImageSets/Main/trainval.txt',
-                data_prefix=dict(sub_data_root='VOC2007/'),
-                filter_cfg=dict(filter_empty_gt=False, min_size=32),
-                pipeline=[]),
-            dict(
-                type='YOLOv5VOCDataset',
-                data_root='tests/data/VOCdevkit/',
-                ann_file='VOC2012/ImageSets/Main/trainval.txt',
-                data_prefix=dict(sub_data_root='VOC2012/'),
-                filter_cfg=dict(filter_empty_gt=False, min_size=32),
-                pipeline=[])
-        ])
+        dataset = ConcatDataset(
+            datasets=[
+                dict(
+                    type='YOLOv5VOCDataset',
+                    data_root='tests/data/VOCdevkit/',
+                    ann_file='VOC2007/ImageSets/Main/trainval.txt',
+                    data_prefix=dict(sub_data_root='VOC2007/'),
+                    filter_cfg=dict(filter_empty_gt=False, min_size=32),
+                    pipeline=[]),
+                dict(
+                    type='YOLOv5VOCDataset',
+                    data_root='tests/data/VOCdevkit/',
+                    ann_file='VOC2012/ImageSets/Main/trainval.txt',
+                    data_prefix=dict(sub_data_root='VOC2012/'),
+                    filter_cfg=dict(filter_empty_gt=False, min_size=32),
+                    pipeline=[])
+            ],
+            ignore_keys='DATASET_TYPE')
         dataset.full_init()
         self.assertEqual(len(dataset), 2)
