Bounding box filtering only respects clipping #1322
Description
Bounding boxes that result from augmentations are filtered, such that bad/unsatisfactory bounding boxes can be removed. For this, Compose takes the argument bbox_params that takes an instance of BboxParams. The process is described in the documentation: https://albumentations.ai/docs/getting_started/bounding_boxes_augmentation/#min_area-and-min_visibility
Supposedly, using min_visibility, a bounding box is filtered out according to the ratio of the area before and after augmentation:
min_visibilityis a value between 0 and 1. If the ratio of the bounding box area after augmentation to the area of the bounding box before augmentation becomes smaller thanmin_visibility, Albumentations will drop that box.
The actual filtering is performed via filter_bboxes:
def filter_bboxes(
bboxes: Sequence[BoxType], rows: int, cols: int, min_area: float = 0.0, min_visibility: float = 0.0
) -> List[BoxType]:
...
resulting_boxes: List[BoxType] = []
for bbox in bboxes:
transformed_box_area = calculate_bbox_area(bbox, rows, cols)
bbox, tail = cast(BoxType, tuple(np.clip(bbox[:4], 0, 1.0))), tuple(bbox[4:])
clipped_box_area = calculate_bbox_area(bbox, rows, cols)
if (
clipped_box_area != 0 # to ensure transformed_box_area!=0 and to handle min_area=0 or min_visibility=0
and clipped_box_area >= min_area
and clipped_box_area / transformed_box_area >= min_visibility
):
resulting_boxes.append(cast(BoxType, bbox + tail))
return resulting_boxesAreas are calculated before and after clipping. Width and height of the image do not change between the two calls to calculate_bbox_area. Therefore, resizing is not taken into account. This conflicts with the documentation and possibly the intent of the authors.
Fixing this is going to require either keeping the original bounding boxes and image size around, or precomputing and storing each bounding box's area. The alternative is simply a change to the documentation:I believe that it is not strictly necessary to filter based on visibility with regards to scaling, since there is still min_area.
However, bounding boxes with very small widths or heights can cause issues. A very tall bounding box, for example, might end up with a width considerably below even one pixel, but still boast an area of well over 200. I would propose adding min_width and min_height to filter_bboxes and BboxParams. With trivial default values, this should be entirely backwards-compatible.
One last point to add: there is also filter_bboxes_by_visibility which seems to have been written with similar intent to filter_boxes, but is entirely untested and with a different signature. Can anyone recall why filter_bboxes_by_visibility exists?