Support reflective video transformations during ExoPlayer playback (e.g. from QuickTime/iOS MP4 files)

[hust-twj]

Version

Media3 main branch

More version details

No response

Devices that reproduce the issue

Any device, it seems have nothing to do with device

Devices that do not reproduce the issue

No response

Reproducible in the demo app?

Yes

Reproduction steps

1. Change media.exolist.json file -> "Playlists" -> "Cats -> Dogs"
2. Replace its original uri with below：
   "https://zh-files.oss-cn-qingdao.aliyuncs.com/20170808223928mJ1P3n57.mp4"，"https://wx-love-img.afunapp.com/ec34db03f040ee1cfb9cfef5c087aaf3"
   In fact, the two urls above are all rotated -90 degree
   

Expected result

And the weird thing is, for "https://zh-files.oss-cn-qingdao.aliyuncs.com/20170808223928mJ1P3n57.mp4"，the video frame is OK
for "https://wx-love-img.afunapp.com/ec34db03f040ee1cfb9cfef5c087aaf3", the video frame is not OK, it rotates -90 degree. It should like this:

Actual result

for "https://wx-love-img.afunapp.com/ec34db03f040ee1cfb9cfef5c087aaf3" , the video frame rotate -90 degree, like this

Media

The reproducible url："https://wx-love-img.afunapp.com/ec34db03f040ee1cfb9cfef5c087aaf3"

Bug Report

• You will email the zip file produced by adb bugreport to android-media-github@google.com after filing this issue.

[claincly]

@rohitjoins I feel this could be related to extractors not parsing the correct rotation? Could you take a look?

[hust-twj]

@rohitjoins I feel this could be related to extractors not parsing the correct rotation? Could you take a look?

@claincly @rohitjoins Thanks in advance.

[rohitjoins]

Hi @hust-twj,

Thank you for your patience. I investigated the media in question and found that its transformation matrix is not purely rotational, it also involves flipping. Our current implementation assumes that the transformation matrix only represents rotation, which is why it doesn’t handle this case correctly.

Tools like ffmpeg, mediainfo, or even media players like VLC use a mathematical formula to calculate rotation based on the assumption that the transformation matrix is purely rotational. In this case, that assumption doesn’t hold, leading to incorrect rotation values and ultimately displaying the wrong output since the video should also be flipped.

I’ll mark this issue as a feature request to add support for flipping using the transformation matrix. Thanks for reporting this!

Here’s a comparison of VLC (incorrect) vs. QuickTime (correct):

[icbaker]

The commit linked above makes ExoPlayer behave like VLC - so these videos are now the right way up, but still incorrectly mirrored in the y-axis. Resolving the mirroring issue is a bit more complicated. I will re-title this issue to reflect that's what it's tracking.