Remove the FeatureVector annotation type, various fixes, and cleanup (#1839)

This pull request removes support for the deprecated `FeatureVector`
annotation type and related functionality, add missing plugin
definitions to spec.json. It also updates documentation and test
configurations to reflect these changes.

### Removal of FeatureVector annotation type and related functionality

* Removed the `FeatureVector` annotation class, its matcher and merger
classes, and the corresponding `feature_vector` entry from the
`AnnotationType` enum in `src/datumaro/components/annotation.py`,
`src/datumaro/components/annotations/matcher.py`, and
`src/datumaro/components/annotations/merger.py`.
[[1]](diffhunk://#diff-7f2b49c4e168c9b76b6096e142d40788cdb5641cb210cc6351d7072d88313a58L53)
[[2]](diffhunk://#diff-7f2b49c4e168c9b76b6096e142d40788cdb5641cb210cc6351d7072d88313a58L260-L271)
[[3]](diffhunk://#diff-946419970a02797027e73d686303272fae7144813b7169985218935c52089f9fL36)
[[4]](diffhunk://#diff-946419970a02797027e73d686303272fae7144813b7169985218935c52089f9fL354-L360)
[[5]](diffhunk://#diff-e7d4644319a6bb820e963fff1e4a74d2a36ef659534b7f3199e456877ce4cd29L17)
[[6]](diffhunk://#diff-e7d4644319a6bb820e963fff1e4a74d2a36ef659534b7f3199e456877ce4cd29L42)
[[7]](diffhunk://#diff-e7d4644319a6bb820e963fff1e4a74d2a36ef659534b7f3199e456877ce4cd29L194-L198)
[[8]](diffhunk://#diff-45c0fe95cb538e7a0d37c2c8a6ef2f9204b02f1fd598c3eb7bf2fe259623f8d7L25)
[[9]](diffhunk://#diff-45c0fe95cb538e7a0d37c2c8a6ef2f9204b02f1fd598c3eb7bf2fe259623f8d7L450-L451)
* Removed references to `FeatureVectorMerger` from the merge logic in
`src/datumaro/components/merge/intersect_merge.py`.
[[1]](diffhunk://#diff-45c0fe95cb538e7a0d37c2c8a6ef2f9204b02f1fd598c3eb7bf2fe259623f8d7L25)
[[2]](diffhunk://#diff-45c0fe95cb538e7a0d37c2c8a6ef2f9204b02f1fd598c3eb7bf2fe259623f8d7L450-L451)
* Updated tests to remove checks for `feature_vector` statistics in
`tests/unit/operations/test_statistics.py`.
[[1]](diffhunk://#diff-c9014cbc94767631e72b9105fb3339a75fd98d9aa1aea809c4fc28dccdc24ea1L305)
[[2]](diffhunk://#diff-c9014cbc94767631e72b9105fb3339a75fd98d9aa1aea809c4fc28dccdc24ea1L430)

### Plugin additions for new formats

* Added plugin definitions for Roboflow TFRecord and TensorFlow
Detection API formats (DatasetBase, Importer, and Exporter) to
`src/datumaro/plugins/specs.json`, including TensorFlow as an extra
dependency.
[[1]](diffhunk://#diff-49a19686affac07069c2b4dfe733cf196aae57dbd46c683ca74afcdb4acba950R1175-R1195)
[[2]](diffhunk://#diff-49a19686affac07069c2b4dfe733cf196aae57dbd46c683ca74afcdb4acba950R1361-R1389)

### Documentation and compatibility updates

* Removed deprecated usage examples for project-based workflows from
`docs/source/docs/data-formats/media_formats.md` and deleted the
deprecated `src/datumaro/project.py` module.
[[1]](diffhunk://#diff-a35988ee55f8d9dfc6bd3222fba44d35635ceb3b224810db393bd7d97f59c3f0L20-L31)
[[2]](diffhunk://#diff-2816b82cd5bcd290f30697a92d9eab8fc69661f926a2e32366b145874ec0bcd3L1-L9)
* Cleaned up deprecated variable declarations from CLI and annotation
modules.
[[1]](diffhunk://#diff-6f1f61fbe373037fdf31139d07acfd57161d3557eaf7369807fea9f6ec65293fL64-L66)
[[2]](diffhunk://#diff-77d278999778b6ae6f7aeb8a0ea8d5abbaf3e79a4ff53526648d343874eaf098L25-L26)
[[3]](diffhunk://#diff-7f2b49c4e168c9b76b6096e142d40788cdb5641cb210cc6351d7072d88313a58L36)
[[4]](diffhunk://#diff-fe06b77e577603a2f17b21c10f504a4da2dd597eb82acb465864c7a6de9ca89bL28)

### Test configuration updates

* Updated `tox.ini` to include TensorFlow and TFDS for Python 3.12
environments.<!-- Contributing guide:
https://github.com/open-edge-platform/datumaro/blob/develop/CONTRIBUTING.md
-->

<!--
Resolves #111 and #222.
Depends on #1000 (for series of dependent commits).

This PR introduces this capability to make the project better in this
and that.

- Added this feature
- Removed that feature
- Fixed the problem #1234
-->

### How to test
<!-- Describe the testing procedure for reviewers, if changes are
not fully covered by unit tests or manual testing can be complicated.
-->

### Checklist
<!-- Put an 'x' in all the boxes that apply -->
- [ ] I have added unit tests to cover my changes.​
- [ ] I have added integration tests to cover my changes.​
- [ ] I have added the description of my changes into
[CHANGELOG](https://github.com/open-edge-platform/datumaro/blob/develop/CHANGELOG.md).​
- [ ] I have updated the
[documentation](https://github.com/open-edge-platform/datumaro/tree/develop/docs)
accordingly

### License

- [ ] I submit _my code changes_ under the same [MIT
License](https://github.com/open-edge-platform/datumaro/blob/develop/LICENSE)
that covers the project.
  Feel free to contact the maintainers if that's a concern.
- [ ] I have updated the license header for each file (see an example
below).

```python
# Copyright (C) 2025 Intel Corporation
#
# SPDX-License-Identifier: MIT
```

Author: gdlg

docs/source/docs/data-formats/media_formats.md

@@ -17,23 +17,11 @@ datum project import -f image_dir </path/to/directory/containing/images>
 
 or, if you work with Datumaro API:
 
-- for using with a project:
-
-  ```python
-  from datumaro.project import Project
-
-  project = Project.init('/path/to/project')
-  project.import_source('source1', format='image_dir', url='/path/to/directory/containing/images')
-  dataset = project.working_tree.make_dataset()
-  ```
-
-- for using as a dataset:
-
-  ```python
-  from datumaro import Dataset
+```python
+from datumaro import Dataset
 
-  dataset = Dataset.import_from('/path/to/directory/containing/images', 'image_dir')
-  ```
+dataset = Dataset.import_from('/path/to/directory/containing/images', 'image_dir')
+```
 
 This will search for images in the directory recursively and add
 them as dataset entries with names like `<subdir1>/<subsubdir1>/<image_name1>`.

setup.py

@@ -92,10 +92,6 @@ def parse_requirements(filename=CORE_REQUIREMENTS_FILE):
         ],
     },
     cmdclass={"build_ext": build_ext},
-    package_data={
-        "datumaro.plugins.synthetic_data": ["background_colors.txt"],
-        "datumaro.plugins.openvino_plugin.samples": ["coco.class", "imagenet.class"],
-    },
     include_package_data=True,
     rust_extensions=[
         RustExtension(

src/datumaro/cli/__main__.py

@@ -61,9 +61,6 @@ def _define_loglevel_option(parser):
         return parser
 
 
-deprecated = "[DEPRECATED, will be removed in 1.12]"
-
-
 # TODO: revisit during CLI refactoring
 def _get_known_contexts():
     return [

src/datumaro/cli/commands/__init__.py

@@ -22,8 +22,6 @@
     "get_non_project_commands",
 ]
 
-deprecated = "[DEPRECATED, will be removed in 1.12]"
-
 
 def get_non_project_commands():
     return [

src/datumaro/components/annotation.py

@@ -33,7 +33,6 @@
 
 from datumaro.components.media import Image
 from datumaro.util.attrs_util import default_if_none, not_empty
-from datumaro.util.deprecation import deprecated
 from datumaro.util.points_util import normalize_points
 
 
@@ -50,7 +49,6 @@ class AnnotationType(IntEnum):
     super_resolution_annotation = 9
     depth_annotation = 10
     ellipse = 11
-    feature_vector = 12
     tabular = 13
     rotated_bbox = 14
     cuboid_2d = 15
@@ -257,18 +255,6 @@ class Label(Annotation):
     label: int = field(converter=int)
 
 
-@deprecated(deprecated_version="1.11", removed_version="1.12")
-@attrs(eq=False, order=False)
-class FeatureVector(Annotation):
-    _type = AnnotationType.feature_vector
-    vector: np.ndarray = field(validator=attr.validators.instance_of(np.ndarray))
-
-    def __eq__(self, other):
-        if not isinstance(other, __class__):
-            return False
-        return np.array_equal(self.vector, other.vector)
-
-
 RgbColor = Tuple[int, int, int]
 
 Colormap = Dict[int, RgbColor]

src/datumaro/components/annotations/matcher.py

@@ -33,7 +33,6 @@
     "CaptionsMatcher",
     "Cuboid3dMatcher",
     "ImageAnnotationMatcher",
-    "FeatureVectorMatcher",
     "Cuboid2DMatcher",
 ]
 
@@ -351,13 +350,6 @@ def match_annotations(self, sources):
         raise NotImplementedError()
 
 
-@attrs
-@attrs
-class FeatureVectorMatcher(AnnotationMatcher):
-    def match_annotations(self, sources):
-        raise NotImplementedError()
-
-
 @attrs
 class TabularMatcher(AnnotationMatcher):
     def match_annotations(self, sources):

src/datumaro/components/annotations/merger.py

@@ -14,7 +14,6 @@
     CaptionsMatcher,
     Cuboid2DMatcher,
     Cuboid3dMatcher,
-    FeatureVectorMatcher,
     ImageAnnotationMatcher,
     LabelMatcher,
     LineMatcher,
@@ -39,7 +38,6 @@
     "Cuboid3dMerger",
     "ImageAnnotationMerger",
     "EllipseMerger",
-    "FeatureVectorMerger",
 ]
 
 
@@ -191,11 +189,6 @@ class EllipseMerger(_ShapeMerger, ShapeMatcher):
     pass
 
 
-@attrs
-class FeatureVectorMerger(AnnotationMerger, FeatureVectorMatcher):
-    pass
-
-
 @attrs
 class TabularMerger(AnnotationMerger, TabularMatcher):
     pass

src/datumaro/components/hl_ops/__init__.py

@@ -25,7 +25,6 @@
 from datumaro.components.transformer import Transform
 from datumaro.components.validator import TaskType, Validator
 from datumaro.util import parse_str_enum_value
-from datumaro.util.deprecation import deprecated
 from datumaro.util.scope import on_error_do, scoped
 
 if TYPE_CHECKING:

src/datumaro/components/merge/intersect_merge.py

@@ -22,7 +22,6 @@
     Cuboid2DMerger,
     Cuboid3dMerger,
     EllipseMerger,
-    FeatureVectorMerger,
     ImageAnnotationMerger,
     LabelMerger,
     LineMerger,
@@ -447,8 +446,6 @@ def _for_type(t, **kwargs):
                 return _make(ImageAnnotationMerger, **kwargs)
             elif t is AnnotationType.ellipse:
                 return _make(EllipseMerger, **kwargs)
-            elif t is AnnotationType.feature_vector:
-                return _make(FeatureVectorMerger, **kwargs)
             elif t is AnnotationType.tabular:
                 return _make(TabularMerger, **kwargs)
             elif t is AnnotationType.rotated_bbox:

src/datumaro/plugins/specs.json

@@ -1172,6 +1172,27 @@
     "plugin_name": "roboflow_yolo_obb",
     "plugin_type": "DatasetBase"
   },
+  {
+    "import_path": "datumaro.plugins.data_formats.roboflow.base_tfrecord.RoboflowTfrecordBase",
+    "plugin_name": "roboflow_tfrecord",
+    "plugin_type": "DatasetBase",
+    "extra_deps": [
+      "tensorflow"
+    ]
+  },
+  {
+    "import_path": "datumaro.plugins.data_formats.roboflow.base_tfrecord.RoboflowTfrecordImporter",
+    "plugin_name": "roboflow_tfrecord",
+    "plugin_type": "Importer",
+    "extra_deps": [
+      "tensorflow"
+    ],
+    "metadata": {
+      "file_extensions": [
+        ".tfrecord"
+      ]
+    }
+  },
   {
     "import_path": "datumaro.plugins.data_formats.roboflow.importer.RoboflowCocoImporter",
     "plugin_name": "roboflow_coco",
@@ -1337,6 +1358,35 @@
       ]
     }
   },
+  {
+    "import_path": "datumaro.plugins.data_formats.tf_detection_api.base.TfDetectionApiBase",
+    "plugin_name": "tf_detection_api",
+    "plugin_type": "DatasetBase",
+    "extra_deps": [
+      "tensorflow"
+    ]
+  },
+  {
+    "import_path": "datumaro.plugins.data_formats.tf_detection_api.base.TfDetectionApiImporter",
+    "plugin_name": "tf_detection_api",
+    "plugin_type": "Importer",
+    "extra_deps": [
+      "tensorflow"
+    ],
+    "metadata": {
+      "file_extensions": [
+        ".tfrecord"
+      ]
+    }
+  },
+  {
+    "import_path": "datumaro.plugins.data_formats.tf_detection_api.exporter.TfDetectionApiExporter",
+    "plugin_name": "tf_detection_api",
+    "plugin_type": "Exporter",
+    "extra_deps": [
+      "tensorflow"
+    ]
+  },
   {
     "import_path": "datumaro.plugins.data_formats.vgg_face2.VggFace2Base",
     "plugin_name": "vgg_face2",