[VideoDecoder] Frames are rotated when the video is in portrait mode / vertical-oriented

[dbouidaine]

I used the MP4Demuxer (as in the example here : webcodecs/samples/mp4-decoder, decoding videos in landscape mode works fine but when I decode a video in a portrait mode it gets rotated (from portrait -> landscape mode). So the video is not decoded as its original orientation and scale.

[sandersdan]

Raw bitstreams do not carry orientation metadata, instead this detail is part of the media container.

Issue #351 is about defining properties that would enable you to pass this metadata through WebCodecs via VideoDecoderConfig. Until that is implemented, you'll need to plumb the metadata yourself.

In any case you'll need to be able to extract the orientation metadata from the container. My understanding is the mp4box.js does not offer any help here, it just puts the raw unparsed data in mvhd.matrix and tkhd.matrix.