-
Notifications
You must be signed in to change notification settings - Fork 406
Description
This format is currently not supported and a ValueError is raised with message Conversion to numpy array with format 'yuv420p10le' is not yet supported
. I think it could be supported using the same method that is used for "yuv420p", if only included in the set of format names it tests for, as follows, in video/frame.py:
# special cases
if frame.format.name in {"yuv420p", "yuvj420p", "yuv420p10le"}:
return np.hstack(
[
useful_array(frame.planes[0]),
useful_array(frame.planes[1]),
useful_array(frame.planes[2]),
]
).reshape(-1, frame.width)
I need this to perform image rotation of vertical HDR videos, while exporting a frame to AVIF where no metadata rotation is supported (apparently). (related, frame.save should support AVIF with HDR metadata, currently only does JPG and PNG).
The PyAV to_ndarray/from_ndarray format of a single 2d ndarray shape=(height + height / 4 + height / 4, width) is cumbersome to work with, and I would prefer a Python list of planes of proper shape (U and V with half resolution). This should be a consideration if the API could be modified to return a list of planes rather than a single ndarray. I have however restructured that from what is currently returned by to_ndarray, for yuv420p format (SDR), rotate each plane with np.rot90, and restore the format expected for Frame.from_ndarray().
fplanes = frame.to_ndarray()
# Convert to useful format
planes = y, u, v = [
fplanes[: frame.height],
fplanes[frame.height : frame.height + frame.height // 4].reshape(
frame.height // 2, frame.width // 2
),
fplanes[frame.height + frame.height // 4 :].reshape(
frame.height // 2, frame.width // 2
),
]
# Rotate
planes = [np.rot90(p, frame.rotation // 90) for p in planes]
# Restore PyAV format
planes = np.hstack([p.flat for p in planes]).reshape(
-1, planes[0].shape[1]
)
frame = av.VideoFrame.from_ndarray(planes, format=frame.format.name)
A built-in function to handle rotation would be helpful as well, e.g. with frame.reformat(). I couldn't find such functionality in PyAV, but this is a common problem with vertical video made by phone cameras, and is easily fixed in the 420 format (or RGB) by a simple np.rot90 on each plane.