Skip to content

Support yuv420p10le with ndarray conversions #1981

@Tronic

Description

@Tronic

This format is currently not supported and a ValueError is raised with message Conversion to numpy array with format 'yuv420p10le' is not yet supported. I think it could be supported using the same method that is used for "yuv420p", if only included in the set of format names it tests for, as follows, in video/frame.py:

        # special cases
        if frame.format.name in {"yuv420p", "yuvj420p", "yuv420p10le"}:
            return np.hstack(
                [
                    useful_array(frame.planes[0]),
                    useful_array(frame.planes[1]),
                    useful_array(frame.planes[2]),
                ]
            ).reshape(-1, frame.width)

I need this to perform image rotation of vertical HDR videos, while exporting a frame to AVIF where no metadata rotation is supported (apparently). (related, frame.save should support AVIF with HDR metadata, currently only does JPG and PNG).

The PyAV to_ndarray/from_ndarray format of a single 2d ndarray shape=(height + height / 4 + height / 4, width) is cumbersome to work with, and I would prefer a Python list of planes of proper shape (U and V with half resolution). This should be a consideration if the API could be modified to return a list of planes rather than a single ndarray. I have however restructured that from what is currently returned by to_ndarray, for yuv420p format (SDR), rotate each plane with np.rot90, and restore the format expected for Frame.from_ndarray().

fplanes = frame.to_ndarray()
# Convert to useful format
planes = y, u, v = [
    fplanes[: frame.height],
    fplanes[frame.height : frame.height + frame.height // 4].reshape(
        frame.height // 2, frame.width // 2
    ),
    fplanes[frame.height + frame.height // 4 :].reshape(
        frame.height // 2, frame.width // 2
    ),
]
# Rotate
planes = [np.rot90(p, frame.rotation // 90) for p in planes]
# Restore PyAV format
planes = np.hstack([p.flat for p in planes]).reshape(
    -1, planes[0].shape[1]
)
frame = av.VideoFrame.from_ndarray(planes, format=frame.format.name)

A built-in function to handle rotation would be helpful as well, e.g. with frame.reformat(). I couldn't find such functionality in PyAV, but this is a common problem with vertical video made by phone cameras, and is easily fixed in the 420 format (or RGB) by a simple np.rot90 on each plane.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions