Skip to content

Commit f67b67f

Browse files
Extend field converters (#1986)
This pull request introduces new converters for handling channels-first/channels-last metadata in image and mask fields, improves error handling for array structure issues, and expands the set of annotation field and converter utilities. The changes also include updates to the public API and add a new test for converter registration. **New converters and utilities:** * Added `ChannelsFirstConverter` for image fields and `MaskChannelsFirstConverter` for mask fields, enabling metadata-only conversion between channels-first and channels-last formats without altering the actual data. [[1]](diffhunk://#diff-370ffd190b01623b335a827152000636d5445190cd9e296694f2884649e55559R399-R449) [[2]](diffhunk://#diff-be0e9424e00f51a63ddbe9dc5c7f01046dc343d7c335eee9ab198a031a4eaa7aR466-R514) * Introduced the `copy_columns_with_shape` utility function to assist converters that only need to update metadata and not the underlying data. * Registered new converters and utilities in the `__init__.py` files and updated the public API to include them. [[1]](diffhunk://#diff-000e98833d5fba9d66f82db2422b240bd02e0fde9f86af6f7ffd9a431abd5b01R8-R34) [[2]](diffhunk://#diff-000e98833d5fba9d66f82db2422b240bd02e0fde9f86af6f7ffd9a431abd5b01R45) [[3]](diffhunk://#diff-000e98833d5fba9d66f82db2422b240bd02e0fde9f86af6f7ffd9a431abd5b01R70-R106) [[4]](diffhunk://#diff-000e98833d5fba9d66f82db2422b240bd02e0fde9f86af6f7ffd9a431abd5b01R119) **Annotation fields and factory functions:** * Added `ellipse_field` factory function and included `EllipseField` and `CaptionField` in the annotation fields public API. [[1]](diffhunk://#diff-bc3a3fc9d428403e8e117f703ffc69efaae506a2db15eea759c548e175698e75R382-R402) [[2]](diffhunk://#diff-29b9ccd281e40c6b1d4f8b918075d0a3f7ba4c4e7d464d6d6f6fb7f917c80e27R14-R22) [[3]](diffhunk://#diff-29b9ccd281e40c6b1d4f8b918075d0a3f7ba4c4e7d464d6d6f6fb7f917c80e27R59) [[4]](diffhunk://#diff-29b9ccd281e40c6b1d4f8b918075d0a3f7ba4c4e7d464d6d6f6fb7f917c80e27R86) **Error handling improvements:** * Introduced a new `ArrayStructureError` exception for clearer diagnostics when numpy arrays have improper object dtype structures during conversion. [[1]](diffhunk://#diff-c614e227e84e7aa9a984dd6207cc5f44aec1fc9af2032ecefb72273c64f883e3R1-R33) [[2]](diffhunk://#diff-e324261812079d99ca2989612441e5df1dd15dabde37fb2e5e8c0c1b639dac0dR21) [[3]](diffhunk://#diff-e324261812079d99ca2989612441e5df1dd15dabde37fb2e5e8c0c1b639dac0dR158-R182) <!-- Contributing guide: https://github.com/open-edge-platform/datumaro/blob/develop/contributing.md --> <!-- Please add a summary of changes. You may use Copilot to auto-generate the PR description but please consider including any other relevant facts which Copilot may be unaware of (such as design choices and testing procedure). Add references to the relevant issues and pull requests if any like so: Resolves #111 and #222. Depends on #1000 (for series of dependent commits). --> ### Checklist <!-- Put an 'x' in all the boxes that apply --> - [x] I have added tests to cover my changes or documented any manual tests. - [ ] I have updated the [documentation](https://github.com/open-edge-platform/datumaro/tree/develop/docs) accordingly --------- Signed-off-by: Albert van Houten <[email protected]> Co-authored-by: Copilot <[email protected]>
1 parent 6e0f046 commit f67b67f

File tree

10 files changed

+1590
-20
lines changed

10 files changed

+1590
-20
lines changed

src/datumaro/experimental/converters/__init__.py

Lines changed: 34 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,15 +5,33 @@
55
from datumaro.experimental.converters.annotation_converters import (
66
BBoxCoordinateConverter,
77
BBoxDtypeConverter,
8+
BBoxFormatConverter,
9+
BBoxToPolygonConverter,
10+
EllipseCoordinateConverter,
11+
EllipseDtypeConverter,
12+
EllipseToBBoxConverter,
13+
KeypointsCoordinateConverter,
14+
KeypointsDtypeConverter,
15+
KeypointsToBBoxConverter,
816
LabelDtypeConverter,
917
LabelIndexConverter,
18+
PolygonCoordinateConverter,
1019
PolygonDtypeConverter,
1120
PolygonToBBoxConverter,
21+
RotatedBBoxCoordinateConverter,
1222
RotatedBBoxDtypeConverter,
23+
RotatedBBoxToBBoxConverter,
1324
RotatedBBoxToPolygonConverter,
1425
)
15-
from datumaro.experimental.converters.base import AttributeRemapperConverter, ConversionError, Converter, list_eval_ref
26+
from datumaro.experimental.converters.base import (
27+
AttributeRemapperConverter,
28+
ConversionError,
29+
Converter,
30+
copy_columns_with_shape,
31+
list_eval_ref,
32+
)
1633
from datumaro.experimental.converters.image_converters import (
34+
ChannelsFirstConverter,
1735
ImageBytesToImageConverter,
1836
ImageCallableToImageConverter,
1937
ImagePathToImageConverter,
@@ -24,6 +42,7 @@
2442
from datumaro.experimental.converters.mask_converters import (
2543
InstanceMaskCallableToInstanceMaskConverter,
2644
MaskCallableToMaskConverter,
45+
MaskChannelsFirstConverter,
2746
PolygonToInstanceMaskConverter,
2847
PolygonToMaskConverter,
2948
)
@@ -48,30 +67,43 @@
4867
"AttributeRemapperConverter",
4968
"BBoxCoordinateConverter",
5069
"BBoxDtypeConverter",
70+
"BBoxFormatConverter",
71+
"BBoxToPolygonConverter",
72+
"ChannelsFirstConverter",
5173
"ConversionError",
5274
"ConversionPaths",
5375
# Base
5476
"Converter",
5577
# Registry
5678
"ConverterRegistry",
5779
"ConverterTransform",
80+
"EllipseCoordinateConverter",
81+
"EllipseDtypeConverter",
82+
"EllipseToBBoxConverter",
5883
"ImageBytesToImageConverter",
5984
"ImageCallableToImageConverter",
6085
"ImagePathToImageConverter",
6186
"ImageToImageInfo",
6287
"InstanceMaskCallableToInstanceMaskConverter",
88+
"KeypointsCoordinateConverter",
89+
"KeypointsDtypeConverter",
90+
"KeypointsToBBoxConverter",
6391
"LabelDtypeConverter",
6492
# Annotation converters
6593
"LabelIndexConverter",
6694
"MaskCallableToMaskConverter",
95+
"MaskChannelsFirstConverter",
96+
"PolygonCoordinateConverter",
6797
"PolygonDtypeConverter",
6898
"PolygonToBBoxConverter",
6999
"PolygonToInstanceMaskConverter",
70100
# Mask converters
71101
"PolygonToMaskConverter",
72102
# Image converters
73103
"RedBlueColorConverter",
104+
"RotatedBBoxCoordinateConverter",
74105
"RotatedBBoxDtypeConverter",
106+
"RotatedBBoxToBBoxConverter",
75107
"RotatedBBoxToPolygonConverter",
76108
"UInt8ToFloat32Converter",
77109
"_can_lazy_converter_handle_conversion",
@@ -84,6 +116,7 @@
84116
"_is_converter_lazy",
85117
"_separate_batch_and_lazy_converters",
86118
"converter",
119+
"copy_columns_with_shape",
87120
"find_conversion_path",
88121
"list_eval_ref",
89122
]

0 commit comments

Comments
 (0)